AR Overlay Using Image Segmentation for Precise 3D Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) technologies lack the precision and exactness to overlay three-dimensional (3D) virtual objects onto real-world objects, making it difficult to achieve accurate alignment and placement, which is crucial for applications involving modular objects like vehicles or houses.
Innovation Solution
A computing device analyzes images of a physical object using techniques like image segmentation and machine learning or guess-and-check methods to generate a digital outline, determining the object's position and scale, and then configures a 3D model to match these parameters, allowing for precise overlay of 3D virtual objects based on predefined locational relations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional AR techniques (SLAM, recognition-based, location-based) are used to overlay virtual objects, then the system can provide basic AR functionality, but the precision and exactness of overlay alignment with real-world objects deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-defining locational relations between virtual objects and physical objects in a 3D model. The system pre-establishes the spatial relationships and transformation parameters (rotation, scale, position) that will be used when overlaying virtual objects. This allows the system to achieve high precision overlay without complex real-time calculations, as the alignment parameters are determined in advance through image segmentation and 3D model configuration.
2Manufacturing precision
If image segmentation and 3D model configuration techniques are applied to determine position and scale, then overlay precision is improved, but the complexity of the processing system increases
Solution Approach 1:
The patent uses copying by creating a digital outline (2D representation) from the physical object through image segmentation, and then creating a corresponding 3D model that replicates the object's geometry. This 3D model serves as a virtual copy that can be precisely configured and manipulated. The system copies the essential geometric features of the physical object into the digital domain, allowing for precise measurement and configuration without directly measuring the physical object repeatedly.
Solution Approach 2:
The patent applies dimensionality change by transitioning from 2D image data to 3D model representation. Image segmentation produces a 2D digital outline, which is then used to configure a 3D model of the physical object. This dimensional transformation allows the system to leverage 3D spatial relationships and pre-defined locational relations for precise virtual object placement, improving accuracy beyond what 2D image analysis alone could achieve.
3Reliability
If 3D virtual objects are overlaid based on predefined locational relations, then alignment accuracy with physical objects is improved, but the time required for initial setup and model configuration increases
Solution Approach 1:
The patent applies preliminary action by pre-defining locational relations between virtual objects and physical objects in a 3D model. The system pre-establishes the spatial relationships and transformation parameters (rotation, scale, position) that will be used when overlaying virtual objects. This allows the system to achieve high precision overlay without complex real-time calculations, as the alignment parameters are determined in advance through image segmentation and 3D model configuration.
Data Source
AI summary
Various embodiments are generally directed to techniques of overlaying a virtual object on a physical object in augmented reality (AR). A computing device may receive one or more images of the physical object, perform analysis on the images (such as image segmentation) to generate a digital outline, and determine a position and a scale of the physical object based at least in part on the digital outline. The computing device may configure (e.g., rotate, scale) a 3D model of the physical object to match the determined position and scale of the physical object. The computing device may place or overlay a 3D virtual object on the physical object in AR based on a predefined location relation between the 3D virtual object and the 3D model of the physical object, and further, generate a composite view of the placement or overlay.