AR Display Latency Reduction via GPU Transform Matrix Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented-reality systems experience noticeable latency when rendering graphics on wearable displays, such as glasses, due to the need to regenerate images in response to changes in the user's view, which can be minimized by generating display pixel data on a graphics processing unit using a method that applies a transform matrix to display pixel coordinates to project graphics onto a three-dimensional planar region defined by orientation data from sensors and LEDs.
Innovation Solution
The method involves a graphics processing unit that receives display pixel coordinates, applies a transform matrix calculated from orientation data from sensors and LEDs to identify a three-dimensional planar region, and projects graphics onto this region, allowing for real-time rendering without the need to regenerate entire images, thereby reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the system regenerates entire images in response to changes in user's view, then the augmented-reality display can maintain accuracy, but noticeable latency occurs
Solution Approach 1:
The patent segments the rendering process by identifying and processing only the specific planar region that needs to be updated rather than regenerating the entire image. The GPU receives a set of pixels corresponding to a specific planar region and processes only those pixels, dividing the large rendering task into smaller, manageable segments that can be updated independently and more quickly.
Solution Approach 2:
The system performs partial action by updating only the necessary portion of the display (the specific planar region affected by the view change) rather than performing excessive action of regenerating the complete image. This selective updating approach reduces the computational workload to the minimum required amount, thereby reducing latency while maintaining rendering accuracy for the affected region.
2Loss of time
If the system processes only specific planar regions, then latency is reduced, but the complexity of identifying and processing specific regions increases
Solution Approach 1:
The system performs preliminary action by pre-defining and storing the characteristics of multiple planar regions in memory before rendering occurs. Each planar region is characterized in advance, allowing the GPU to quickly identify which pre-defined region corresponds to the current view change without performing complex real-time analysis. This preparation work is done once and reused, reducing the complexity of the real-time processing.
Solution Approach 2:
The patent uses parameter changes to represent different planar regions, where each region is defined by specific parameters (such as orientation data from sensors and LEDs, coordinate transformations). By changing and comparing these parameters efficiently, the system can identify the affected region through simple parameter matching rather than complex geometric calculations, reducing processing complexity.
3Productivity
If the GPU receives and processes a set of pixels for a specific planar region, then rendering speed improves, but the precision of projecting pixels onto the three-dimensional region must be maintained
Solution Approach 1:
The patent introduces an intermediary coordinate transformation process that maps pixels from the two-dimensional display space to the three-dimensional planar region space. This intermediary transformation layer handles the complex projection mathematics, allowing the GPU to process pixels in a simplified manner while ensuring precise positioning on the target planar region. The intermediary transformation acts as a buffer that decouples the simple pixel processing from the complex 3D positioning requirements.
Data Source
Figure 1A~1B
Figure 2
Figure 3
AI summary
Disclosed are methods and systems for generating display pixel data so as to reduce latency when rendering a representation of a graphic on a display, such as for augmented-reality applications. The method comprises: receiving a set of display pixel coordinate-pairs at the graphics processing unit; applying a transform matrix to the set of display pixel coordinate-pairs to obtain a set of graphic pixel coordinate-pairs, the transform matrix calculated using orientation data received from an external reference; retrieving a set of graphic pixel data associated with the set of graphic pixel coordinate-pairs; and, determining a set of display pixel data based on the retrieved set of graphic pixel data.