Augmenting Multi-View Images with Synthetic Objects via IMU Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for adding three-dimensional information to augmented reality are computationally expensive and typically restricted to static scenes, making them inefficient for dynamic environments.
Innovation Solution
The use of inertial measurement units (IMUs) and image data to efficiently compute and place synthetic objects within multi-view images, allowing them to move with the scene and be accurately positioned without requiring extensive 3D reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D reconstruction is used to add three-dimensional information to augmented reality, then the accuracy and realism of the augmented elements are improved, but the computational cost and processing time increase significantly
Solution Approach 1:
The patent uses synthetic 3D objects (computer-generated models) as copies or representations of real-world objects to be augmented. Instead of performing complex 3D reconstruction of the entire scene, the system places pre-defined synthetic objects at calculated positions in the image, significantly reducing computational requirements while maintaining visual realism of the augmented elements
Solution Approach 2:
The patent introduces an intermediary approach by using synthetic object placement as a middle ground between full 3D reconstruction and simple 2D overlay. The system calculates camera pose and projects 3D synthetic objects into the image space, creating an intermediate solution that provides sufficient depth perception without the full computational burden of complete scene reconstruction
2Measurement precision
If traditional 3D reconstruction methods are used, then accurate spatial information is obtained, but the method is restricted to static scenes and cannot handle dynamic environments
Solution Approach 1:
The patent implements dynamics by enabling the augmented reality system to handle moving cameras and dynamic scenes. The system continuously estimates camera pose from video frames and updates the projection of synthetic objects in real-time, allowing the augmented elements to correctly follow camera motion and maintain accurate spatial relationships in dynamic environments
Solution Approach 2:
The patent applies preliminary action by pre-defining synthetic 3D objects and their properties before augmentation. Instead of reconstructing 3D information from static scenes, the system prepares synthetic objects in advance and simply needs to calculate their correct projection positions based on camera pose estimation, enabling rapid adaptation to dynamic scenes without complex real-time reconstruction
Data Source
AI summary
Provided are mechanisms and processes for augmenting multi-view image data with synthetic objects using inertial measurement unit (IMU) and image data. In one example, a process includes receiving a selection of an anchor location in a reference image for a synthetic object to be placed within a multi-view image. Movements between the reference image and a target image are computed using visual tracking information associated with the multi-view image, device orientation corresponding to the multi-view image, and an estimate of the camera's intrinsic parameters. A first synthetic image is then generated by placing the synthetic object at the anchor location using visual tracking information in the multi-view image, orienting the synthetic object using the inverse of the movements computed between the reference image and the target image, and projecting the synthetic object along a ray into a target view associated with the target image. The first synthetic image is overlaid on the target image to generate an augmented image from the target view.