AR Object Overlay via 3D Plane Angle Minimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality technologies face challenges in merging real objects from one perspective view with a different perspective view of a real environment in a perspectively and dimensionally correct manner, due to difficulties in determining camera poses and spatial relationships between different environments.
Innovation Solution
A method that involves determining 3D planes and angles relative to camera positions, mapping object image areas to target spaces within the real environment to minimize angle differences, and using warping functions to achieve correct or approximate perspective alignment, allowing for the overlay of real objects in a visually realistic manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a first image of a real object captured from one perspective is merged with a second image of a real environment captured from a different perspective, then the virtual object can be displayed in the real environment, but the perspective and dimensional correctness of the merged view deteriorates due to difficulty in determining camera poses and spatial relationships
Solution Approach 1:
The patent introduces an intermediary coordinate system transformation process that mediates between the first camera's coordinate system (capturing the real object) and the second camera's coordinate system (capturing the real environment). By establishing transformation matrices and mapping relationships through intermediary steps, the system resolves the perspective mismatch while maintaining dimensional accuracy, thus enabling versatile image merging without sacrificing precision.
2Measurement precision
If camera poses in two different real environments are determined to enable correct merging, then the perspective accuracy improves, but the complexity of determining spatial relationships and camera positions increases
Solution Approach 1:
The patent segments the complex camera pose determination problem into multiple independent sub-problems: (1) determining the first camera pose relative to the real object, (2) determining the second camera pose relative to the real environment, and (3) establishing the transformation relationship between the two coordinate systems. This segmentation reduces the overall complexity while maintaining measurement precision through systematic step-by-step analysis.
3Adaptability or versatility
If the real object is represented in another perspective view different from the original capture, then the augmentation flexibility increases, but the accuracy of representing the real object in the new perspective deteriorates
Solution Approach 1:
The patent transitions from 2D image plane coordinates to 3D spatial coordinates and then back to 2D projection coordinates in the target perspective. By introducing the intermediate 3D dimension, the system can accurately represent the real object in different perspective views while maintaining measurement precision through proper projection transformations and coordinate system mappings.
Data Source
AI summary
The disclosure relates to a method and system for presenting at least part of an image of a real object in a view of a real environment, comprising providing a first image of at least part of a real object captured by a first camera, determining at least part of the real object in the first image as an object image area, determining a first 3D plane relative to the first camera, the first camera being at a position where the first image is captured, providing at least one image feature related to the real object in the first image, providing at least one first ray passing an optical center of the first camera being at a position where the first image is captured and the at least one image feature, determining, according to a first plane normal direction of the first 3D plane, at least one first angle between the first 3D plane and the at least one first ray, providing a second image of a real environment captured by a second camera, determining a second 3D plane relative to the second camera, the second camera being at a position where the second image is captured, mapping at least part of the object image area to a target space in a view of the real environment captured at a viewpoint, wherein the at least one image feature is mapped to at least one target position in a coordinate system of the view, determining at least one second angle between the second 3D plane and at least one second ray passing the viewpoint of capturing the view and the at least one target position according to a second plane normal direction of the second 3D plane, wherein the at least one image feature is mapped to the at least one target position such that a difference between the at least one first angle and the at least one second angle is minimized.