Mobile Robot Spatial Capture for HMD Immersion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for virtual reality, particularly in gaming, fail to effectively capture and render the user's local environment while wearing a head-mounted display (HMD), as they cannot see their surroundings, leading to a lack of immersion and interaction with the real space.
Innovation Solution
A mobile robot equipped with sensors, including image capture devices and depth cameras, captures spatial and texture data of the local environment by moving to various positions and adjusting lighting conditions, generating a spatial model and texture information that is used to render a virtual space corresponding to the real environment on the HMD, allowing the user to experience their surroundings virtually.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a head-mounted display is worn to provide immersive virtual reality, then the user experiences freedom of movement and 3D real-time view, but the user is unable to see the local environment in which they are situated
Solution Approach 1:
The system creates a digital twin or replica of the local environment by capturing spatial data, texture information, and lighting conditions with sensors, then rendering this copied environment on the HMD display. This allows the user to see a virtual representation of their surroundings while wearing the HMD, resolving the contradiction between immersion and environmental awareness
Solution Approach 2:
The HMD display acts as an intermediary between the user and the local environment, presenting a rendered view that combines virtual reality elements with real environmental data. This intermediary layer allows the user to maintain awareness of their surroundings while experiencing the immersive capabilities of VR
2Device complexity
If sensors are positioned statically to capture environment data, then the system structure is simple, but the quality and completeness of captured spatial and texture information is insufficient
Solution Approach 1:
The sensor system transitions from a static configuration to a dynamic one, where sensors are moved to multiple positions and orientations throughout the environment. This dynamic positioning allows comprehensive capture of spatial data and texture information from various viewpoints, significantly improving capture quality while accepting increased system complexity
Solution Approach 2:
The environment capture process is segmented into multiple discrete sensor positions and measurement phases. By dividing the capture task across multiple locations and orientations, the system achieves complete spatial and texture coverage that would be impossible from a single static position
3Productivity
If lighting conditions are not controlled during capture, then the capture process is simpler and faster, but the accuracy of texture information and spatial model is reduced
Solution Approach 1:
The system performs preliminary actions by controlling and standardizing lighting conditions before the actual capture process begins. This pre-adjustment of lighting ensures consistent illumination across all sensor positions, which improves the accuracy of texture and spatial measurements without significantly impacting overall capture productivity
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
A method is provided, including the following method operations: using a robot having a plurality of sensors to acquire sensor data about a local environment; processing the sensor data to generate a spatial model of the local environment, the spatial model defining virtual surfaces that correspond to real surfaces in the local environment; further processing the sensor data to generate texture information that is associated to the virtual surfaces defined by the spatial model; tracking a location and orientation of a head-mounted display (HMD) in the local environment; using the spatial model, the texture information, and the tracked location and orientation of the HMD to render a view of a virtual space that corresponds to the local environment; presenting the view of the virtual environment through the HMD.