AR Display Latency Reduction via GPU Transform Matrix
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented-reality systems suffer from noticeable latency when rendering graphics on wearable displays, such as glasses, due to the need to regenerate images in response to changes in the user's view, which can be exacerbated by the process of storing and rendering images in a frame buffer.
Innovation Solution
A method and system that utilize a graphics processing unit to generate display pixel data by applying a transform matrix to display pixel coordinate-pairs based on orientation data from an external reference, allowing for real-time adjustment and rendering of graphics on a three-dimensional plane, thereby reducing latency and improving rendering efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If images are stored and rendered in a frame buffer, then rendering can be performed, but noticeable latency occurs
Solution Approach 1:
The patent extracts the frame buffer operation from the rendering pipeline and replaces it with direct coordinate transformation. Instead of storing images in a frame buffer and then rendering, the system directly transforms display pixel coordinates to graphic pixel coordinates using a transform matrix, eliminating the intermediate frame buffer storage step that causes latency.
Solution Approach 2:
The system performs preliminary calculation of the transform matrix based on orientation data before rendering is needed. By pre-calculating the transformation parameters that map display coordinates to graphic coordinates, the system prepares the rendering pipeline in advance, allowing for real-time coordinate transformation without the delay of frame buffer operations.
2Speed
If graphics are regenerated in response to view changes, then real-time rendering is achieved, but processing time increases
Solution Approach 1:
The patent changes the parameter representation from full image data to transformation parameters (transform matrix). Instead of regenerating complete images in response to view changes, the system updates the transform matrix parameters that define the coordinate mapping. This parameter-based approach reduces processing time while maintaining real-time rendering capability.
Solution Approach 2:
The rendering process is segmented into coordinate transformation and graphic retrieval operations. By separating the coordinate mapping function (handled by the transform matrix) from the actual graphic rendering, the system can efficiently process view changes through parameter updates without the overhead of complete image regeneration.
Data Source
AI summary
Disclosed are methods and systems for generating display pixel data so as to reduce latency when rendering a representation of a graphic on a display, such as for augmented-reality applications. The method comprises: receiving a set of display pixel coordinate-pairs at the graphics processing unit; applying a transform matrix to the set of display pixel coordinate-pairs to obtain a set of graphic pixel coordinate-pairs, the transform matrix calculated using orientation data received from an external reference; retrieving a set of graphic pixel data associated with the set of graphic pixel coordinate-pairs; and, determining a set of display pixel data based on the retrieved set of graphic pixel data.