Dual-Camera 3D Reconstruction With Absolute Scale From Facial Geometry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision methods for 3D reconstruction, such as SLAM and SfM, often result in reconstructions that are up to scale, lacking an absolute spatial scale factor, leading to inaccurate camera movements and inconsistent virtual object scaling in applications like navigation and Augmented Reality.
Innovation Solution
Utilize a user-facing camera to capture a human face, which has consistent spatial properties, and a world-facing camera to reconstruct the environment, determining the absolute scale by leveraging known facial features and spatial transformations between the cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If monocular capture apparatus is used for 3D reconstruction, then device complexity is reduced, but measurement precision of absolute spatial scale deteriorates
Solution Approach 1:
The patent introduces a second camera as an intermediary device to capture images of the first camera at different positions. This mediator camera enables the system to determine absolute spatial scale by analyzing the geometric relationship between multiple captured images, thereby resolving the scale ambiguity inherent in monocular reconstruction without requiring complex specialized sensors.
Solution Approach 2:
The patent transitions from analyzing images in a single camera's coordinate system to incorporating spatial information from multiple camera positions. By capturing images from different viewpoints and analyzing the geometric transformations between these views, the system recovers absolute scale information that was lost in the monocular projection.
2Measurement precision
If additional known objects or sensors are used to determine absolute scale, then measurement precision of absolute spatial scale is improved, but device complexity increases
Solution Approach 1:
The system uses the first camera itself as the object to be measured by the second camera. Instead of requiring external calibration objects or specialized sensors, the method leverages the camera's own images captured from different positions to determine absolute scale, making the system self-calibrating and eliminating the need for additional measurement tools.
Solution Approach 2:
The second camera serves multiple functions: it captures images of the first camera at different positions, provides geometric constraints for scale determination, and enables absolute scale recovery. This multi-functional approach avoids the need for separate calibration devices while achieving the same measurement precision.
3Ease of operation
If up-to-scale reconstruction is performed, then ease of operation is improved, but reliability of camera movement determination deteriorates
Solution Approach 1:
The patent replaces the need for physical measurement tools or mechanical calibration devices with a computational approach. By using geometric relationships and projective geometry calculations on captured images, the system determines absolute scale and reliable camera movements through mathematical processing rather than physical measurement, maintaining operational simplicity while improving reliability.
4Productivity
If arbitrary scale is assigned in reconstruction, then productivity is improved, but consistency of virtual object scaling deteriorates
Solution Approach 1:
The patent performs absolute scale determination as a preliminary step before virtual object placement and rendering. By calculating the scale factor from geometric relationships in the captured images beforehand, the system ensures that all subsequent virtual objects are scaled consistently with the real environment, preventing scale mismatches while maintaining efficient processing.
Data Source
Figure 1
Figure 2a~2b
Figure 3
AI summary
Determining spatial coordinates of a 3D reconstruction of at least part of a first real object comprising: obtaining, from a first and second camera system with known spatial relationship, a first and second image comprising at least part of a first and second real object, respectively, the latter object associated with known geometric properties; determining a scale of at least part of the second real object based on the second image and said known geometric properties; determining a pose of the second camera system based on the determined scale of the at least part of the second real object; determining a pose of the first camera system according to the pose of the second camera system and the known spatial relationship; and determining, based on the pose of the first camera system and the scale of the at least part of the second real object, spatial coordinates of a 3D reconstruction of the first real object; and superimposing a virtual object into a view of the first real object at a scale consistent with the first real object.