Head-Mounted Display Object Rendering for Accurate VR Scale
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR systems face challenges in accurately displaying real-world objects in virtual environments, particularly in maintaining their size and allowing users to interact with specific objects of choice, due to the lack of efficient 3D reconstruction algorithms and limitations of pass-through technology.
Innovation Solution
A head-mounted display device equipped with a camera, processor, and displayer that captures and processes specific images to obtain feature points, transforms them into world and view coordinates, and generates projection coordinates for displaying a virtual image, enabling accurate representation of real-world objects in VR.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If pass-through technology is used to display real-world objects in VR, then users can see real-world environment without removing HMD, but real objects appear with incorrect size and distortion artifacts due to unknown depth information
Solution Approach 1:
The patent creates a virtual copy of the real-world object by capturing its image through the camera, extracting feature points, and rendering it in the virtual environment. This virtual copy can be displayed at the correct size and position without requiring accurate depth measurement of the real object, thus resolving the size accuracy issue while maintaining the ability to see real-world environment through pass-through.
Solution Approach 2:
The patent introduces an intermediary virtual representation of the real object that acts as a mediator between the real-world object and the user's perception. Instead of directly displaying the real object through pass-through (which causes size distortion), the system creates a virtual intermediate representation that can be accurately sized and positioned in the VR environment.
2Ease of operation
If pass-through occupies the entire scene, then users can see real-world environment, but users cannot choose what they want to see and interaction is limited
Solution Approach 1:
The patent segments the visual field by separating the real-world pass-through view from the virtual object representation. Instead of occupying the entire scene with pass-through, the system divides the display into regions where users can see both the real environment and selected virtual objects, enabling users to choose what they want to see and improving interaction versatility.
Solution Approach 2:
The patent makes the viewing experience dynamic by allowing users to selectively view different virtual objects overlaid on the pass-through background. The system can dynamically adjust which virtual objects are displayed and how they are positioned, giving users control over their viewing experience rather than being forced to see the entire real-world scene.
3Measurement precision
If 3D reconstruction is performed using stereo images, then depth information can be obtained, but existing algorithms produce visible distortion artifacts and are computationally intensive
Solution Approach 1:
The patent extracts only the necessary depth information and geometric features from the stereo images through feature point detection and matching, rather than performing full 3D reconstruction of the entire scene. By taking out only the essential geometric data needed for virtual object positioning and scaling, the system avoids the distortion artifacts and computational burden of complete 3D reconstruction while still achieving accurate depth measurement for virtual rendering.
Data Source
AI summary
A head-mounted display device including a camera, a processor, and a displayer is disclosed. The camera is configured to capture a specific image of a specific object within an environment. The processor is coupled to the camera, and the processor is configured to: obtain several feature points of the specific image, and each of several feature points includes a world coordinate; generate a view coordinate according to the world coordinate of each of the plurality of feature points; and generate a projection coordinate of the environment according to the view coordinate of each of the feature points. The displayer is coupled to the processor, and the displayer is configured to display a virtual image including a transformed specific image.