HMD Image Processing Apparatus for Real-Time Stereoscopic Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques fail to efficiently generate and display stereoscopic omnidirectional images in real time on head-mounted displays (HMDs) due to high calculation costs, as they do not adequately account for the distortion of eyepiece lenses.
Innovation Solution
An image processing apparatus that acquires viewer orientation information and extracts an overlapping region where relative positions from optical axes of lenses are equal, allowing for shared generation processing between right and left eye images, reducing calculation costs by deriving and sharing ray directions and pixel values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If simple projection conversion from plane to plane is used, then device complexity is reduced, but manufacturing precision deteriorates because it cannot account for eyepiece lens distortion
Solution Approach 1:
The patent segments the omnidirectional image into multiple planar regions (e.g., front, rear, left, right views) and processes each region separately with appropriate projection conversion. This allows different processing methods to be applied to different regions, accounting for lens distortion while maintaining manageable complexity.
Solution Approach 2:
The patent changes the parameter of projection conversion from simple plane-to-plane to a more complex conversion that incorporates eyepiece lens distortion characteristics. By adjusting projection parameters to match the optical properties of the HMD lenses, accurate image generation is achieved while still maintaining real-time processing capability.
2Manufacturing precision
If full projection conversion with lens distortion consideration is performed for both eyes, then manufacturing precision is improved, but productivity deteriorates due to high calculation cost
Solution Approach 1:
The patent merges the processing of left and right eye images by extracting and reusing common calculation results. Since both eyes view the same omnidirectional scene, the projection conversion calculations for corresponding regions can be shared, reducing redundant computations while maintaining stereoscopic accuracy.
Solution Approach 2:
The patent performs preliminary extraction of the overlapping region and pre-calculates common parameters before generating the final stereoscopic images. By preparing shared data structures and calculation results in advance, the processing time for generating both eye images is significantly reduced, enabling real-time display.
3Ease of operation
If omnidirectional image is displayed on planar display, then ease of operation is improved, but manufacturing precision deteriorates because it does not provide stereoscopic effect
Solution Approach 1:
The patent transitions from displaying omnidirectional images on a single planar surface to generating stereoscopic images for HMD with multiple display surfaces (left and right eye displays). This dimensional change enables the incorporation of depth information and lens distortion effects, providing authentic stereoscopic realism while maintaining ease of operation through automated processing.
Data Source
AI summary
An image processing apparatus includes a first acquisition unit configured to acquire orientation information of a viewer, and an extraction unit configured to extract an overlapping region being a region in which relative positions from points intersecting with optical axes of lenses disposed in front of respective eyes of the viewer are equal, for a right eye display image and a left eye display image to be presented to the viewer. In addition, the image processing apparatus includes a generation unit configured to generate the display image from a predetermined reference image, based on the orientation information acquired by the first acquisition unit, and the overlapping region extracted by the extraction unit. In the overlapping region, the generation unit shares information about generation processing of the display image, between the right eye display image and the left eye display image.