Virtual Environment Display Using Automatic Layout Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current XR technologies require users to manually mark layout information of a real environment in a virtual environment, which is complex and inefficient, leading to low virtual environment building speed and immersive interaction experience.
Innovation Solution
A method and apparatus for automatically generating a panoramic image from multiple environment images captured from different angles, extracting layout information, and building a virtual environment simulation of the target place, using key point detection, camera pose determination, and feature extraction models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If users manually mark layout information of a real environment in a virtual environment, then the virtual environment can be constructed, but the operation process becomes complex and virtual environment building efficiency decreases
Solution Approach 1:
The system automatically captures environment images through the camera, performs key point detection, determines camera poses, and constructs the virtual environment without requiring manual user input for layout marking. The device serves itself by autonomously completing the entire virtual environment construction process from image capture to panoramic image generation and layout extraction.
2Productivity
If multiple environment images are captured and processed to generate panoramic image and extract layout information automatically, then virtual environment building speed increases, but the processing complexity increases
Solution Approach 1:
The processing pipeline is divided into distinct sequential stages: (1) capturing multiple environment images from different angles, (2) performing key point detection on each image, (3) determining camera poses based on key point locations, (4) projecting images to spherical coordinate system, (5) splicing projected images to form panoramic image, and (6) extracting layout information from the panoramic image. This segmentation enables automated high-speed processing while managing complexity through modular organization.
3Loss of time
If manual marking of layout information is used, then the virtual environment can be constructed, but the time required for virtual environment building increases
Solution Approach 1:
The manual mechanical process of users marking layout information is replaced with an automated computer vision system that performs key point detection, camera pose estimation, and layout extraction through image processing algorithms. This substitution eliminates the time-consuming manual marking process while automatically achieving the same virtual environment construction goal.
Data Source
AI summary
A panoramic image formed by projecting a target place to a virtual environment can be generated based on a plurality of environment images obtained by observing the target place from different angles of view. A machine can automatically recognize and intelligently extract layout information of the target place based on the panoramic image, and build, by using the layout information, a target virtual environment for simulating the target place.