Direct Distortion Rendering for Low-Latency HMD Image Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image generation techniques for head-mounted displays require excessive memory capacity and processing time for distortion correction, leading to increased power consumption and motion sickness due to latency.
Innovation Solution
The proposed method involves direct rendering of distorted images in the frame buffer, eliminating the need for intermediate non-distorted images and allowing simultaneous execution of rendering and distortion transformation, thereby reducing memory requirements and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If distortion correction is performed on a rendered image by temporarily holding it in a frame buffer, then the video can be correctly viewed through the high-curvature lens, but memory capacity is required and processing time is excessive
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing distortion correction information in a lookup table before rendering. The rendering process then directly uses this pre-prepared correction data to generate distorted images without needing to temporarily store and post-process the rendered image, thereby eliminating the time delay associated with frame buffer holding and post-rendering distortion correction.
Solution Approach 2:
The patent extracts the distortion correction operation from the traditional post-rendering processing stage and integrates it into the rendering process itself. By separating the distortion correction information storage (in lookup table) from the rendering process, the system avoids the need to hold rendered images in frame buffers, thus reducing both memory requirements and processing time while maintaining correction accuracy.
2Manufacturing precision
If distortion correction is performed on a rendered image by temporarily holding it in a frame buffer, then the video can be correctly viewed through the high-curvature lens, but memory capacity is required
Solution Approach 1:
The patent extracts the distortion correction operation from the traditional post-rendering processing stage and integrates it into the rendering process itself. By separating the distortion correction information storage (in lookup table) from the rendering process, the system avoids the need to hold rendered images in frame buffers, thus reducing both memory requirements and processing time while maintaining correction accuracy.
Solution Approach 2:
The patent uses a lookup table that stores pre-calculated distortion correction information as a simplified copy or representation of the full distortion correction process. Instead of storing and processing complete rendered images in frame buffers, the system stores compact distortion correction data that can be applied during rendering, significantly reducing memory capacity requirements while preserving correction functionality.
3Manufacturing precision
If distortion correction is performed on a rendered image, then the video can be correctly viewed through the high-curvature lens, but rendering latency is increased causing motion sickness
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing distortion correction information in a lookup table before rendering. The rendering process then directly uses this pre-prepared correction data to generate distorted images without needing to temporarily store and post-process the rendered image, thereby eliminating the time delay associated with frame buffer holding and post-rendering distortion correction.
Solution Approach 2:
The patent merges the distortion correction operation with the rendering process by directly generating distorted images during rendering using lookup table data. This integration eliminates the separate post-processing stage that caused latency, allowing the system to produce corrected images in real-time and reduce motion sickness caused by delayed image updates.
4Area of moving object
If a high-curvature optical lens is used to display a wide field of view, then the field of view is expanded, but the video is distorted due to distortion aberration
Solution Approach 1:
The patent converts the harmful distortion aberration caused by the high-curvature lens into a beneficial feature by pre-calculating the exact distortion pattern and storing it in a lookup table. The rendering process then intentionally applies this known distortion pattern, which compensates for the lens's optical distortion, thereby eliminating the harmful effect while preserving the wide field of view capability.
Solution Approach 2:
The patent applies preliminary action by pre-calculating and storing distortion correction information in a lookup table before rendering. The rendering process then directly uses this pre-prepared correction data to generate distorted images without needing to temporarily store and post-process the rendered image, thereby eliminating the time delay associated with frame buffer holding and post-rendering distortion correction.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
An image generation apparatus 200 includes a distortion information storage section 234 that stores, as distortion information, information for deforming and distorting an image, and a distortion rendering section 232 that renders a distorted image by rasterization processing using the distortion information without generating a non-distorted image in a memory. The distortion rendering section 232 renders the distorted image by executing the rasterization processing on a distorted triangle obtained by transforming a triangle to be rasterized on the basis of the distortion information such that the processing is executed upward and downward from positions of vertices of the distorted triangle.