Augmented Reality Overlay Alignment for Masked Object Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems fail to effectively visualize information on real objects when parts of the object are masked, such as by the operator's hand or a robot arm, as the digital information overlays on the interposed elements rather than the object, reducing the system's usefulness.
Innovation Solution
A method that involves real-time adjustment processing of a three-dimensional model with a video capture system's reference point and displaying digital information superimposed on the captured image, including an initial step of recording the object's reference texture and analyzing images to generate a composite image by mixing the captured and three-dimensional model images, with optional transparency exceeding 50% for overlay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If digital information is overlaid on the captured image using conventional augmented reality systems, then information can be displayed on the real object, but the information incorrectly overlays on interposed elements (operator's hand, robot arm) instead of the object, reducing accuracy and usefulness
Solution Approach 1:
The patent creates a synthetic image that copies the appearance of the real object from the captured image, then overlays digital information on this synthetic copy rather than directly on the captured image. This allows the information to be correctly positioned on the object's geometry even when parts are masked by interposed elements, as the synthetic image reconstructs the object's surface where it should be.
Solution Approach 2:
The patent introduces a synthetic image as an intermediary between the captured image and the digital information overlay. This synthetic image acts as a mediator that represents the object's true geometry and surface, allowing accurate information placement without being affected by masking elements in the captured image.
2Measurement precision
If the system uses a three-dimensional model with real-time adjustment processing, then accurate visualization on the object is achieved, but the device complexity increases
Solution Approach 1:
The system creates a synthetic image that copies the object's appearance and geometry, then overlays information on this copy. This approach maintains accuracy while managing complexity by working with a simplified representation (the synthetic image) rather than complex real-time 3D model adjustments.
Solution Approach 2:
The patent replaces complex mechanical or computational 3D model adjustment mechanisms with an image processing approach that generates a synthetic image. This substitution simplifies the system by using image synthesis techniques instead of complex real-time 3D registration and adjustment mechanisms.
3Measurement precision
If the system processes and generates composite images in real-time, then accurate information display is achieved, but the processing time and computational load increase
Solution Approach 1:
The system performs preliminary processing to create the synthetic image from the captured image and three-dimensional model before overlaying the digital information. By preparing the synthetic image in advance, the system reduces the processing time required during the actual information display, as the complex image synthesis work is done beforehand.
Solution Approach 2:
The patent divides the processing into separate stages: first generating the synthetic image from the captured image and 3D model, then overlaying the digital information on the synthetic image. This segmentation of processing tasks allows optimization of each stage independently and reduces overall processing time by separating computationally intensive operations from the information display operation.
Data Source
AI summary
A method for visualizing an image combining an image (Ic) of a real object (200) originating from a video capture system (300) with digital information (In) originating from a three-dimensional model of the equipment, comprising: carrying out a processing operation to superimpose, in real time, a reference point (402) of the three-dimensional model with a reference point (302) of the video capture system and an object reference point (202) of the real object, and displaying at least some of the digital information superimposed on the image captured by the video capture system, further comprising: an initial step (Ei) of recording the reference texture (T200) of the real object, and a step (Ea) of analyzing the images transmitted by the video capture system, the analysis step comprising: generating a synthesis image from the captured image, and from the three-dimensional model of the equipment textured using the recorded texture; a step of calculating a composite image by mixing the synthesis image and the captured image.