Finger-Gesture AR Shooting for Low-Latency Messaging Scenes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enabling computing devices to perform image processing operations on captured images in varying conditions such as changes in scale, noise, lighting, movement, or geometric distortion is computationally intensive and challenging, particularly in augmented reality environments where real and virtual worlds are combined.
Innovation Solution
An augmented reality system that includes image processing operations to enhance real-world environments with electronic information, allowing users to interact with overlaid content, and supports features like media overlays, geolocation-based filters, and augmented reality content generation, while reducing latency and power consumption on resource-constrained devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image processing operations are performed on captured images in varying conditions (scale, noise, lighting, movement, geometric distortion), then image processing capability is improved, but computational complexity and resource consumption increase
Solution Approach 1:
The patent segments the AR system into multiple components: image capture module, image processing module, and AR content generation module. Each module handles specific tasks independently, allowing complex image processing to be broken down into manageable operations that can be optimized separately, reducing overall computational complexity while maintaining processing capability.
Solution Approach 2:
The system performs preliminary actions by pre-processing captured images to correct geometric distortions and normalize lighting conditions before further AR processing. This preliminary correction reduces the computational burden on subsequent processing stages by addressing challenging conditions early in the pipeline.
2Ease of operation
If real-time augmented reality content is generated and overlaid on captured images, then user interaction capability is improved, but latency increases
Solution Approach 1:
The system implements periodic action by processing images at optimized frame rates and using temporal buffering to smooth processing intervals. This approach maintains consistent real-time performance by regulating when processing occurs, reducing variability in latency while preserving user interaction capability through steady-state operation.
Solution Approach 2:
The patent introduces an intermediary processing layer that buffers and queues image data between capture and display. This intermediary mechanism decouples the capture rate from the display rate, allowing the system to maintain smooth real-time rendering while managing processing latency through controlled data flow.
3Manufacturing precision
If comprehensive image processing operations are performed on resource-constrained devices, then processing quality is improved, but power consumption increases
Solution Approach 1:
The system applies local quality by performing intensive image processing operations only on specific regions of interest within captured images, rather than processing entire frames uniformly. This selective processing maintains high quality where needed while reducing overall computational load and power consumption on resource-constrained devices.
Solution Approach 2:
The patent dynamically adjusts processing parameters such as resolution, frame rate, and processing intensity based on device resource availability and environmental conditions. This adaptive parameter adjustment allows the system to maintain acceptable processing quality while optimizing power consumption by scaling operations to match available device resources.
Data Source
AI summary
The subject technology receives a set of frames. The subject technology detect a first gesture correspond to an open trigger finger gesture. The subject technology receives a second set of frames. The subject technology detects from the second set of frames, a second gesture correspond to a closed trigger finger gesture. The subject technology detects a location and a position of a representation of a finger from the closed trigger finger gesture. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology renders a movement of the first virtual object along a vector away from the location and the position of the representation of the finger within a first scene. The subject technology provides for display the rendered movement of the first virtual object along the vector within the first scene.