Mobile AR 3D Object Superimposition with Live Camera Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in accurately rendering and superimposing three-dimensional (3D) objects onto live images on mobile devices, particularly in augmented reality applications, due to difficulties in determining 3D geometry, recognizing objects, and aligning virtual views with the correct perspective and scale within the environment.
Innovation Solution
A method and system that utilize sensors on mobile devices to determine 3D geometry, recognize objects using a database, and render images with a given perspective, combining these with live images to create an augmented reality experience, including the use of geometry, recognition, position, rendering, and presentation components to ensure accurate alignment and interaction with the environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If 3D geometry information is determined using sensors on mobile devices, then augmented reality rendering is enabled, but measurement precision and reliability are challenging due to sensor limitations
Solution Approach 1:
The patent combines multiple sensor types (cameras, gyroscopes, accelerometers, magnetometers) to capture both 2D image data and 3D orientation data simultaneously. This merging of sensor inputs allows the system to determine accurate 3D geometry information by integrating data from multiple sources, thereby improving measurement precision while maintaining reliability in augmented reality rendering.
Solution Approach 2:
The patent introduces intermediate processing steps including image processing to extract 3D geometry information, perspective calculation based on device orientation, and coordinate transformation. These intermediary processes bridge the gap between raw sensor data and accurate 3D rendering, enhancing both measurement precision and rendering reliability by systematically transforming sensor inputs into accurate spatial representations.
2Manufacturing precision
If virtual views are rendered with correct perspective based on device position, then alignment with live images is improved, but device complexity increases
Solution Approach 1:
The patent implements dynamic perspective calculation that automatically adjusts the rendering perspective based on the mobile device's real-time position and orientation. The system continuously updates the viewpoint coordinates using sensor data, allowing virtual objects to maintain correct alignment with the live environment as the device moves. This dynamic adaptation achieves high alignment accuracy without requiring complex manual configuration.
Solution Approach 2:
The rendering system automatically determines its own perspective and alignment parameters by utilizing data from the device's built-in sensors. The system self-calibrates by calculating the viewpoint based on device position and orientation, eliminating the need for external calibration equipment or complex manual setup procedures. This self-service approach reduces system complexity while maintaining precise alignment.
3Measurement precision
If multiple sensors are used to determine 3D geometry, then measurement accuracy is improved, but use of energy and device complexity increase
Solution Approach 1:
The patent employs partial action by selectively activating sensor subsets based on the specific augmented reality task requirements. The system can use only the necessary sensors for the current operation (e.g., camera and gyroscope for basic AR, or additional sensors for specialized applications), avoiding continuous full-sensor operation. This approach maintains measurement precision when needed while reducing energy consumption during normal operation.
Data Source
AI summary
Methods and systems for superimposing virtual views of 3D objects with live images are provided. An example method may include determining 3D geometry information associated with an environment using one or more sensors of a mobile device, and recognizing 3D objects in the environment based on a database of 3D objects and the 3D geometry information. The method may also include determining a field of view of the environment based on a position of the mobile device in the environment and rendering an image of one or more of the 3D objects that are within the field of view. The rendered image may have a given perspective that is based on a viewpoint position from which the mobile device is viewed. The rendered image may also be combined with a live image of the field of view that is presented by the mobile device.