AR HMD Image Processing Apparatus Reducing Display Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality (MR) and augmented reality (AR) techniques using video see-through head mount displays (HMDs) suffer from image delay issues due to transmission and processing delays, causing user discomfort such as dizziness and time differences between real and virtual images.
Innovation Solution
An image processing apparatus that includes an acquisition unit for capturing real images, a generation unit for generating virtual images, a synthesis unit for combining real and virtual images, and a display control unit for controlling the display of synthesized images, allowing the adjustment of delay times by selecting whether to output the real image or a converted image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the real image is transmitted to an external device for CG superimposition and the combined image is transmitted back to the HMD, then the virtual image can be superimposed on the real image, but the transmission and processing cause delay in the displayed image
Solution Approach 1:
The patent segments the image processing into two separate paths: one path processes the real image for display, and another path generates virtual images for superimposition. The real image is displayed with minimal processing delay, while virtual images are generated separately and superimposed, avoiding the delay caused by transmitting the entire combined image back and forth.
Solution Approach 2:
The patent applies preliminary action by pre-generating virtual images in advance and storing them, so that when the real image is displayed, the virtual images are already prepared and can be superimposed without causing additional delay. This allows the system to maintain low latency while still providing accurate virtual image superimposition.
2Loss of time
If the real image is displayed on the HMD with minimal processing, then the delay of the real image is decreased, but there is a time difference between the real image and the virtual image
Solution Approach 1:
The patent implements feedback mechanisms where the system continuously monitors the timing and synchronization between real image display and virtual image generation. Based on this feedback, the system adjusts the virtual image generation timing and applies delay compensation to ensure that virtual images remain synchronized with the real image even when the real image is displayed with minimal processing delay.
Solution Approach 2:
The patent applies dynamics by making the virtual image generation and synchronization adaptive rather than static. The system dynamically adjusts the timing of virtual image generation based on the actual display timing of the real image, allowing it to maintain synchronization under varying conditions while keeping the real image delay minimal.
3Loss of time
If path switching and status monitoring are implemented to select transmission methods, then the delay can be optimized, but the configuration becomes complicated
Solution Approach 1:
The patent merges the real image display path and virtual image generation path into a unified processing architecture where both operations occur simultaneously within the same system. This eliminates the need for separate path switching mechanisms and status monitoring, as the system naturally coordinates both paths without requiring complex control logic.
Data Source
AI summary
An image processing apparatus includes an acquisition unit configured to acquire a real image captured by an image capturing device and output the real image, a generation unit configured to receive the real image output from the acquisition unit, generate a virtual image, output the virtual image, and output the received real image or a converted image of the real image, a synthesis unit configured to combine the real image output from the acquisition unit, the virtual image output from the generation unit, and the real image or the converted image output from the generation unit, and a display control unit configured to display an image synthesized by the synthesis unit on a display device, wherein the generation unit changes a delay time of the real image in the image displayed on the display device depending on whether to output the real image or the converted image.