Mixed Reality Self-Guidance via 3D Data Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality solutions for self-guidance in mixed reality environments lack the ability to provide effective three-dimensional registration and annotation in situations with limited data or moving targets, often requiring concurrent assistance and failing to perform three-dimensional data augmentation from two-dimensional data.
Innovation Solution
A method that performs continuous three-dimensional registration based on two-dimensional and three-dimensional relationships, using data augmentation to generate annotations in a mixed reality environment by determining a three-dimensional transform function between images and extrapolating three-dimensional information from two-dimensional data, even with a small amount of information available.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional augmented reality solutions are used for self-guidance, then users can receive guidance information, but the system fails to perform accurate three-dimensional registration and data augmentation from two-dimensional data when data is limited or targets are moving
Solution Approach 1:
The patent performs three-dimensional data augmentation by extrapolating three-dimensional information from two-dimensional image data. This dimensional transformation allows the system to generate sufficient three-dimensional point clouds for accurate registration even when direct three-dimensional data is limited, resolving the contradiction between data quantity and registration precision
Solution Approach 2:
The system uses two-dimensional image data as an intermediary to generate three-dimensional point clouds through data augmentation. This intermediary approach enables the system to bridge the gap between limited available data and the three-dimensional registration requirements, achieving accurate positioning without sufficient direct three-dimensional input
2Extent of automation
If conventional augmented reality solutions require concurrent expert assistance, then users can get help with complex tasks, but the system cannot provide independent self-guidance capability
Solution Approach 1:
The patent enables self-guidance by implementing automated three-dimensional registration and annotation generation from two-dimensional images. The system performs continuous tracking and data augmentation autonomously without requiring expert intervention, allowing users to independently complete complex maintenance and repair tasks while the system handles the computational complexity internally
3Reliability
If the system performs continuous three-dimensional registration with moving targets, then tracking accuracy is maintained, but the system struggles with target movement in conventional approaches
Solution Approach 1:
The patent performs continuous three-dimensional registration by continuously tracking the unit across multiple images and performing continuous data augmentation. This continuous process maintains reliable tracking of moving targets by constantly updating the three-dimensional point clouds and transform functions, ensuring accuracy even when targets move at varying speeds
4Ease of operation
If annotations are generated in a mixed reality environment with limited data, then user guidance is provided, but the system cannot generate accurate annotations without sufficient three-dimensional data
Solution Approach 1:
The system generates annotations by transforming two-dimensional image data into three-dimensional point clouds through data augmentation. This dimensional transformation enables annotation generation in mixed reality environments even when three-dimensional data is limited, as the system extrapolates the necessary three-dimensional information from available two-dimensional images
Data Source
AI summary
A method, a computer program product, and a computer system guide a user through dimensional relationships. The method includes receiving a plurality of images of a unit of a perspective of a user. When a number of corresponding three-dimensional points between a current image and a previous image is less than a registration threshold to perform a three-dimensional registration operation, the method includes performing a three-dimensional data augmentation operation based on two-dimensional data from the current image and the previous image to generate extended corresponding three-dimensional points. The method includes determining a three-dimensional transform function between the current image and the previous image based on the extended corresponding three-dimensional points. The method includes generating annotations to be shown for the unit in the current image based on the three-dimensional transform function, the annotations being shown as a virtual rendering in a mixed reality environment viewed by the user.