VR Headset Latency Reduction via Feature Point Pre-Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality head-mounted devices (HMDs) experience increased motion-to-photon latency due to prolonged calculation times for analyzing image frames and generating graphic objects, leading to user dizziness from mismatched motion and visual information.
Innovation Solution
An electronic device equipped with a camera and processor that displays a first image frame, determines graphic objects based on feature points, and combines them with subsequent image frames to reduce latency by immediate display and sequential processing, using feature extraction and recognition techniques to accurately position graphic objects in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the system performs complete image frame analysis and graphic object generation before display, then the accuracy of graphic object positioning is improved, but the motion-to-photon latency increases causing user dizziness
Solution Approach 1:
The system performs preliminary actions by extracting feature points from the first image frame and pre-determining graphic objects based on these feature points before the user's head movement occurs. This allows the processing to be partially completed in advance, reducing the latency when the actual display needs to occur. The feature points and graphic objects are prepared beforehand so that when the second image frame arrives, the system only needs to perform location transformation rather than complete analysis from scratch.
2Loss of time
If the system processes image frames sequentially to reduce calculation time, then the motion-to-photon latency is reduced, but the accuracy of tracking user motion decreases
Solution Approach 1:
The system creates a copy of the feature points from the first image frame and uses this copied data to determine graphic objects. Instead of processing the entire second image frame from scratch, the system copies the relevant feature information and transforms it to the second image frame's coordinate system. This copying approach maintains accuracy by preserving the original feature point relationships while significantly reducing the calculation time required for full frame analysis.
3Loss of time
If the system displays the first image frame immediately without processing, then the initial display latency is reduced, but the graphic objects cannot be properly positioned in subsequent frames
Solution Approach 1:
The system performs preliminary determination of graphic objects based on feature points from the first image frame before displaying subsequent frames. By extracting feature points and identifying graphic objects in advance from the first frame, the system prepares all necessary positioning information beforehand. This allows immediate display of the first frame with graphic objects already positioned correctly, and enables rapid transformation to subsequent frames without sacrificing positioning accuracy.
Data Source
AI summary
Disclosed is an electronic device for displaying a graphic object. The electronic device may include: a camera; a display; and a processor configured to control the display to display a first image frame acquired through the camera, to determine at least one graphic object to be displayed on the display based on a plurality of first feature points of the first image frame, to determine a location of the at least one graphic object to be displayed in a second image frame based on locations of a plurality of second feature points of the second image frame acquired through the camera and locations of the plurality of first feature points, to combine the at least one graphic object and the second image frame based on the determined locations and control the display to display the combined graphic object and second image frame.