Mixed Reality Depth Ordering via Segmented Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In mixed reality systems, accurately rendering virtual objects in a manner that respects the depth ordering of real and virtual objects is challenging, requiring effective acquisition and adaptation of depth information from real-world scenes.
Innovation Solution
A system comprising image capturing units, an image representation generation unit, a game engine, and an image processing unit that captures and processes depth map data and texture data to generate and adapt virtual scenarios, allowing for flexible mixing of real and virtual scenes from various viewpoints, enabling realistic and goal-oriented training environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth information is acquired from real-world scenes to render virtual objects correctly, then the depth ordering accuracy is improved, but the system complexity increases
Solution Approach 1:
The system segments the complex task of depth perception into multiple components: image capturing means for acquiring scene data, image representation generation unit for creating depth maps, and image processing unit for adapting depth information. This segmentation allows each component to handle a specific aspect of depth processing, improving accuracy while managing system complexity through modular design.
Solution Approach 2:
The patent introduces intermediate representations (depth maps, texture maps) as mediators between the real-world scene capture and the virtual object rendering. These intermediate data structures serve as a bridge that facilitates accurate depth ordering without requiring direct complex interactions between all system components, thus improving measurement precision while controlling system complexity.
2Adaptability or versatility
If virtual scenarios are mixed with real scenes from arbitrary viewpoints, then the adaptability is improved, but the image processing complexity increases
Solution Approach 1:
The system implements dynamic viewpoint adaptation by allowing the image processing unit to adapt image representations to arbitrary virtual viewpoints. The depth map and texture map are dynamically adjusted based on the desired viewpoint, enabling flexible mixing of virtual and real scenes from any angle while managing processing complexity through efficient algorithms.
Solution Approach 2:
The patent changes key parameters (viewpoint position, depth map perspective, texture map orientation) to achieve arbitrary viewpoint mixing. By systematically varying these parameters in the image processing unit, the system achieves high adaptability for different virtual viewing points while controlling complexity through parameter-based transformations rather than complete reprocessing.
3Reliability
If depth map data and texture data are adapted to create immersive virtual video sequences, then the training realism is improved, but the processing time increases
Solution Approach 1:
The system performs preliminary actions by pre-processing and adapting depth map data and texture map data before final rendering. The image processing unit prepares these data structures in advance, adapting them to the desired virtual scenario and viewpoint, which reduces processing time during actual training operations while maintaining high training realism.
Data Source
AI summary
A system and method for mixing a scene with a virtual scenario. An image capturing unit is arranged to capture at least one image so as to cover the scene from a first viewpoint. An image representation generation unit is arranged to generate at least one image representation based on the captured image. A game engine unit is arranged to generate a virtual scenario. An image processing unit is arranged to adapt the at least one image representation based on the generated virtual scenario so as to provide a virtual video sequence.