Patch-Tracking Image Sensor for Low-Latency AR Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented and mixed reality systems face challenges in acquiring information about physical objects with low latency and low power consumption, leading to unrealistic virtual object placement and high power consumption, which limits their utility and user enjoyment.
Innovation Solution
The use of image sensors with patch tracking and dynamic vision sensing techniques that selectively output information from specific regions of the imaging array based on object movement, reducing the amount of data processed and consumed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full image data is processed to maintain high angular resolution, then measurement precision is improved, but use of energy increases and device complexity increases
Solution Approach 1:
The imaging array is divided into multiple patches, and only specific patches containing objects of interest are processed. The system dynamically selects and processes only the necessary patches based on motion detection, rather than processing the entire image array, thereby reducing power consumption while maintaining angular resolution for tracked objects
Solution Approach 2:
Instead of processing the complete image data, the system applies partial action by processing only the selected patches that contain relevant information. This selective processing approach reduces computational load and power consumption while maintaining sufficient measurement precision for the tracked objects
2Measurement precision
If full image data is processed to maintain high angular resolution, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The imaging array is divided into multiple patches, and only specific patches containing objects of interest are processed. The system dynamically selects and processes only the necessary patches based on motion detection, thereby reducing processing complexity while maintaining angular resolution for tracked objects
Solution Approach 2:
Instead of processing the complete image data, the system applies partial action by processing only the selected patches that contain relevant information. This selective processing approach reduces computational complexity while maintaining sufficient measurement precision for the tracked objects
3Measurement precision
If traditional image processing is used, then measurement precision is maintained, but loss of time increases
Solution Approach 1:
The imaging array is divided into multiple patches, and only specific patches containing objects of interest are processed. The system dynamically selects and processes only the necessary patches based on motion detection, thereby reducing processing time while maintaining tracking accuracy for objects of interest
Solution Approach 2:
Instead of processing the complete image data, the system applies partial action by processing only the selected patches that contain relevant information. This selective processing approach reduces processing latency while maintaining sufficient measurement precision for tracked objects
4Productivity
If high frame rate processing is implemented, then productivity is improved, but use of energy increases
Solution Approach 1:
The imaging array is divided into multiple patches, and only specific patches containing objects of interest are processed at high frame rates. The system dynamically selects and processes only the necessary patches based on motion detection, thereby achieving high productivity for tracked objects while reducing overall power consumption compared to processing the entire array at high frame rates
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables low-latency, low-power image processing, allowing for realistic virtual object placement relative to physical objects, reducing power consumption and sensor size while maintaining high angular resolution.
Implementation Method 1
Each pixel cell of the plurality of pixel cells may comprise a light-sensitive component. The at least one comparator may comprise an output providing signals indicating a change in sensed light at at least a portion of the light-sensitive components of the plurality of pixel cells.
Data Source
AI summary
An image sensor suitable for use in an augmented reality system to provide low latency image analysis with low power consumption. The augmented reality system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object and selectively output imaging information for that region. The region may be updated dynamically as the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an augmented reality scene can be developed. The amount of data may be further reduced by configuring the image sensor to output indications of pixels for which the measured intensity of incident light changes.