Image Processing Apparatus for HMDs Using Motion-Predicted Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-resolution image data display on head-mounted displays (HMDs) is hindered by large data volumes and increased processing time, leading to delayed or choppy image rendering when the viewer's head motion changes the desired viewing direction.
Innovation Solution
An image processing apparatus that predicts the viewer's line-of-sight direction using motion information from sensors, determines target and path areas, and dynamically adjusts the resolution of image data from stored environment maps to ensure smooth and timely display of high-resolution images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution image data is displayed on HMD, then image quality is improved, but processing time increases and display smoothness deteriorates
Solution Approach 1:
The system performs preliminary actions by predicting the viewer's future line-of-sight direction based on current motion information, and pre-processing image data for the predicted target area before it becomes the current viewing area. This allows high-resolution image data to be prepared in advance, reducing processing delays when the viewer actually looks at that direction.
Solution Approach 2:
The environment map is divided into multiple areas including current area, predicted target area, and intermediate areas. Different resolution settings are applied to different segments based on their importance and predicted viewing timing. The path area is further segmented into multiple intermediate areas that are processed sequentially, allowing gradual resolution improvement without overwhelming the processing system.
2Measurement precision
If high-resolution image data is displayed on HMD, then image quality is improved, but display smoothness deteriorates due to large data volume
Solution Approach 1:
The system dynamically adjusts the resolution of image data based on the predicted viewing sequence. The resolution setting changes over time as different areas become the current viewing area, with higher resolutions applied to currently viewed and predicted target areas, and lower resolutions to less important intermediate areas. This dynamic adaptation maintains display smoothness while maximizing image quality where needed.
Solution Approach 2:
Different resolution qualities are applied to different spatial regions of the environment map based on their predicted importance. The current area and predicted target area receive high-resolution treatment, while intermediate path areas receive progressively lower resolutions. This local differentiation ensures high image quality in critical viewing zones while reducing overall data volume for smoother display.
3Speed
If image data is processed in real-time according to viewer motion, then display responsiveness is improved, but processing load increases
Solution Approach 1:
Instead of waiting for real-time viewer motion to trigger image processing, the system performs preliminary processing by predicting future viewing directions based on current motion trends. Image data for predicted target areas is prepared in advance, reducing the processing burden during actual viewing transitions and maintaining responsiveness without excessive real-time computational load.
4Speed
If the viewer's head motion is tracked and displayed immediately, then display responsiveness is improved, but image data processing time increases
Solution Approach 1:
The system performs preliminary processing by predicting future line-of-sight directions based on current head motion information. Image data for the predicted target area is processed and prepared before the viewer actually looks at that direction, so when the viewing transition occurs, the data is already ready for immediate display, maintaining responsiveness while reducing actual processing time during viewing.
Data Source
AI summary
Provided is an image processing apparatus, for displaying image data in accordance with a line-of-sight direction of a viewer, that includes: storage means for storing a plurality of environment maps having different resolutions; obtaining means for obtaining as motion information of the viewer; retaining means for retaining a current area that is being displayed; target area determination means for determining a target area on the basis of the current area and the motion information; path area determination means for determining as a path area an area that includes a path, from the current area to the target area, along which the line-of-sight direction changes; setting means for setting resolutions that respectively correspond to the path area and the target area; and generation means for reading pieces of image data that respectively correspond to the path area and the target area from environment maps having the resolutions set by the setting means, and for generating image data for display in the path area and image data for display in the target area.