Mixed Reality Camera Position Estimation via Polygon Mesh Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mixed reality systems face challenges in accurately determining the position and orientation of cameras in real spaces, especially when the imaging area differs from the captured keyframe, leading to reduced estimation accuracy.
Innovation Solution
An information processing apparatus that visualizes the mapped space by generating a polygon mesh representing the observed area, where vertices are colored based on their mapping status, allowing users to identify captured and uncaptured regions, and utilizing a spot area setting unit to section the observed space for improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If visual SLAM is used to determine camera position and orientation, then positioning accuracy is improved in captured areas, but accuracy deteriorates in areas not captured by keyframes
Solution Approach 1:
The space is segmented into mapped and unmapped regions using a polygon mesh visualization. The polygon mesh divides the three-dimensional space into discrete elements that can be individually colored to indicate whether they have been captured by keyframes, allowing users to clearly distinguish between areas with reliable position estimation and areas without sufficient map information.
Solution Approach 2:
Different colors are assigned to represent different mapping statuses. The polygon mesh elements are colored based on whether they fall within captured areas (mapped regions) or uncaptured areas (unmapped regions), providing an intuitive visual indicator of the reliability of position and orientation estimation in different spatial zones.
2Device complexity
If only virtual space images are presented to users, then device complexity is reduced, but ease of operation deteriorates as users must understand correspondence between real and virtual spaces
Solution Approach 1:
A polygon mesh visualization is introduced as an intermediary element that bridges the real and virtual spaces. The polygon mesh is rendered in the real space view and provides a visual mediator that helps users understand which areas have been mapped, making the correspondence between real physical space and virtual map data immediately apparent without requiring complex explanations.
3Adaptability or versatility
If the imaging area of the camera differs from the keyframe imaging area, then adaptability is improved for different viewing angles, but measurement precision deteriorates due to inability to acquire correspondence relation
Solution Approach 1:
The solution extends the visualization from two-dimensional images to three-dimensional spatial representation using a polygon mesh. By constructing a three-dimensional mesh that represents the spatial extent of mapped areas, the system can accurately indicate which three-dimensional regions have sufficient map information, even when the camera views the space from different angles and positions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An information processing apparatus includes an image acquisition unit configured to acquire an image from a camera that captures an image of a real space, a position and estimation unit configured to estimate a position or orientation of the camera in the real space, based on the image, a map creation unit configured to create a map including at least one keyframe, an observation space setting unit configured to set an observation space of a user, an analysis unit configured to analyze a relationship between the observation space set by the observation space setting unit and the map created by the map creation unit, and generate a model representing the keyframe included in the map, and a control unit configured to cause a display unit to display a combined image based on the generated model and the image.