Markerless AR Manual Generation via Inside-Out Sensor Trajectory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) user manual generation methods require either pre-set markers or computationally intensive artificial intelligence (AI) training, making them impractical for many scenarios due to the need for marker placement or high data acquisition costs.
Innovation Solution
A method for generating AR user manuals in an electronic 3D viewing environment that uses an inside-out optical sensor and motion sensor to record the moving trajectory of the sensor, with a calibration process involving a predefined 3D model and user input to estimate the target object's pose without pre-set markers or intensive data computation, allowing for real-time superimposition of instructions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If marker-based approach is used for AR user manual generation, then object pose estimation can be achieved, but the need for placing markers on target objects makes it impractical in many situations
Solution Approach 1:
The patent extracts the marker placement requirement from the AR system by using inside-out sensing to capture the target object's appearance and features directly from the real-world scene, eliminating the need for artificial markers while maintaining pose estimation capability
Solution Approach 2:
The patent creates a 3D copy or digital twin of the target object by capturing its appearance, shape, and features through the inside-out optical sensor, allowing the system to recognize and track the object without physical markers
2Ease of operation
If AI-based approach is used for AR user manual generation, then object pose estimation can be performed without markers, but the computational intensity and data acquisition cost become very high
Solution Approach 1:
The patent segments the pose estimation process into distinct modules: inside-out sensing for object appearance capture, feature extraction from visual data, and pose calculation based on extracted features, making the system more efficient and less computationally intensive
Solution Approach 2:
The patent changes the approach from using complex AI neural networks with large training datasets to using geometric parameter extraction and feature-based methods, significantly reducing computational requirements while maintaining markerless operation
3Measurement precision
If AI-based approach is used for AR user manual generation, then object pose estimation can be achieved, but the accuracy depends largely on the amount and relevancy of training data which may be very high
Solution Approach 1:
The system performs self-calibration by automatically capturing images of the target object from multiple angles using the inside-out sensor, extracting features and computing pose parameters without requiring external training data or manual calibration procedures
Data Source
AI summary
A method of generating an AR user manual in an electronic 3D viewing environment, comprising: recording a moving trajectory of the 3D viewing environment's optical sensor; receiving a landmark location information; executing an iterative target object pose estimation comprising: estimating an estimated target object pose from each of the optical sensor poses in the recorded moving trajectory and the landmark location; calculating an estimation error from a 3D model being arranged in the estimated target object pose and projected onto the target object in the real-world scene; calculating a mean of the estimation errors; and reiterating the iterative target object pose estimation to optimize the estimated target object pose for a minimum mean estimation error; if the minimum mean estimation error is within a predefined estimation error threshold, then rendering the AR user manual onto the target object according to the optimized estimated target object pose.