Event-Based Camera Dynamic Patch Tracking for AR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) and mixed reality (MR) systems face challenges in acquiring information about physical objects with low latency and reduced power consumption, while maintaining high-resolution frames and low latency in processing.
Innovation Solution
The use of image sensors with dynamic vision sensing (DVS) techniques and patch tracking, which selectively output differential image information in specific regions (patches) of the image array, reducing processing requirements and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image sensors are used to capture high-resolution frames, then image quality is maintained, but processing latency and power consumption increase
Solution Approach 1:
The patent segments the image array into multiple regions of interest (ROIs) and processes only those segments rather than the entire high-resolution frame. This allows the system to maintain high measurement precision in critical areas while reducing overall processing latency by excluding non-critical regions from intensive processing.
Solution Approach 2:
The patent extracts and processes only the differential image information from specific regions of interest rather than processing the complete high-resolution frame. This extraction approach maintains image quality where needed while significantly reducing processing latency and power consumption by eliminating unnecessary processing of static or non-critical areas.
2Loss of information
If conventional image sensors process full frames, then complete image information is obtained, but power consumption increases
Solution Approach 1:
The patent applies local quality by differentiating processing intensity across different regions of the image array. Regions of interest receive full processing attention to maintain information completeness, while other regions use reduced processing. This spatial variation in processing quality maintains necessary image information while significantly reducing overall power consumption.
Solution Approach 2:
The patent implements partial action by processing only the necessary portions of the image (regions of interest) rather than the complete frame. This partial processing approach maintains sufficient image information for AR/MR applications while reducing power consumption by avoiding excessive processing of non-critical areas.
3Speed
If dynamic vision sensing is used to reduce latency, then processing speed improves, but image resolution may be reduced
Solution Approach 1:
The patent segments the high-resolution image array into regions of interest and processes only those segments using dynamic vision sensing techniques. This segmentation allows the system to achieve high processing speeds in critical areas while maintaining the underlying high-resolution sensor architecture, thus preserving image resolution where it matters most.
Solution Approach 2:
The patent applies partial action by using dynamic vision sensing (which operates at higher speeds) only for regions of interest rather than the entire frame. This selective application maintains high processing speed for critical areas while the high-resolution sensor continues to capture complete image data, preventing resolution loss.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables AR and MR systems to acquire and process image information with low latency and reduced power consumption, while maintaining high-resolution frames, thereby enhancing the realism and usability of AR and MR experiences.
Implementation Method 1
Each pixel cell may comprise a light-sensitive component
Data Source
AI summary
A high-resolution image sensor suitable for use in an augmented reality (AR) system to provide low latency image analysis with low power consumption. The AR system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object, selectively output imaging information for that region, and synchronously output high-resolution image frames. The region may be updated dynamically as the image sensor and/or the object moves. The image sensor may output the high-resolution image frames less frequently than the region being updated when the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an AR scene can be developed.