AR Gesture-Based Virtual Object Resizing With Low-Latency Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enabling computing devices to perform image processing operations on digital images captured in varying conditions, such as changes in scale, noise, lighting, movement, or geometric distortion, is computationally intensive and challenging, especially in resource-constrained environments.
Innovation Solution
An augmented reality (AR) system that integrates real and virtual environments, allowing for efficient image processing and interaction with electronic information overlaid in the real world, utilizing a messaging client application to support capturing, modifying, and sharing AR content, including 3D objects and effects, with motion sensor input and external asset data loading.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If image processing operations are performed on digital images captured in varying conditions, then image processing capability is improved, but computational resource consumption increases
Solution Approach 1:
The system segments image processing tasks by identifying and processing specific regions of interest (such as virtual objects or hand gestures) rather than entire images. This selective processing reduces computational load while maintaining processing capability for critical elements.
Solution Approach 2:
The patent introduces intermediary processing layers including gesture recognition systems and object detection algorithms that act as mediators between raw image capture and final processing. These intermediaries pre-process and filter data, reducing the complexity of subsequent processing operations.
2Ease of operation
If AR content processing and rendering is enhanced, then user experience is improved, but power consumption increases
Solution Approach 1:
The system implements periodic rendering and processing updates rather than continuous full-frame processing. By updating AR content at optimized intervals and only when necessary (based on gesture detection or object changes), the system maintains user experience while significantly reducing power consumption during idle periods.
Solution Approach 2:
The patent applies local quality enhancement by processing and rendering only specific regions of the display containing virtual objects or interaction elements, rather than processing the entire display area. This selective rendering approach reduces computational requirements while maintaining visual quality where users interact.
3Adaptability or versatility
If gesture recognition and virtual object manipulation are added, then interaction capability is improved, but system complexity increases
Solution Approach 1:
The system implements multi-functional gesture recognition that can identify multiple gesture types (pinch, swipe, tap, rotate) using a unified detection algorithm. This universal approach allows diverse interaction capabilities to be achieved through a single, integrated system rather than separate specialized modules for each gesture type.
Solution Approach 2:
The patent uses simplified digital representations (colliders, bounding boxes, landmark points) that copy essential geometric properties of physical objects and gestures. These simplified models enable complex interaction recognition while reducing computational complexity by representing three-dimensional objects and gestures as two-dimensional projections with key特征 points.
Data Source
AI summary
The subject technology detects a first gesture and a second gesture, each gesture corresponding to an open trigger finger gesture. The subject technology detects a third gesture and a fourth gesture, each gesture corresponding to a closed trigger finger gesture. The subject technology, selects a first virtual object in a first scene. The subject technology detects a first location and a first position of a first representation of a first finger from the third gesture and a second location and a second position of a second representation of a second finger from the fourth gesture. The subject technology detects a first change in the first location and the first position and a second change in the second location and the second position. The subject technology modifies a set of dimensions of the first virtual object to a different set of dimensions.