Robot AR Calibration Using 3D Pose Alignment Without Markers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality applications in robotics face impracticalities with artificial markers for calibration, requiring manual effort and incurring additional costs, and suffer from inaccurate localization and tracking due to sensor inaccuracies and procedures.
Innovation Solution
A method that uses a machine-integrated measuring system to record position information, forming a three-dimensional machine model, and calibrates an augmented reality display environment by determining the relative pose between the machine and image capture device, enabling precise localization and tracking without artificial markers, using dynamic SLAM algorithms and machine simulation models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If artificial markers or test bodies are used for calibration, then the augmented reality display environment can be calibrated, but significant manual effort and time are required to measure the markers by manually moving the robot
Solution Approach 1:
The robot serves itself as the calibration object by using its own structured lighting projections and image capture capabilities. The robot automatically projects structured light patterns onto itself, captures images of these projections, and processes the data to determine its own pose and calibrate the AR environment, eliminating the need for external markers and manual measurement operations.
Solution Approach 2:
The patent replaces manual mechanical measurement operations with automated optical measurement systems. Instead of manually moving the robot and measuring markers with physical tools, the system uses structured light projection and image capture to automatically obtain measurement data, substituting mechanical interaction with optical field-based measurement.
2Measurement precision
If additional external sensors are used to track the AR display, then tracking accuracy can be improved, but disproportionate additional costs are incurred
Solution Approach 1:
The robot's existing imaging and control systems are made multi-functional by using them for both their primary operations and for AR calibration and tracking. The same cameras and processors used for robot navigation and manipulation are also employed to capture structured light patterns and calculate pose information, eliminating the need for dedicated external tracking sensors.
Solution Approach 2:
The robot uses its own built-in imaging systems to perform tracking functions that would traditionally require external sensors. By projecting structured light patterns and capturing their reflection or distortion, the robot self-generates the tracking signals needed to maintain accurate AR overlay positioning without external assistance.
3Ease of operation
If common localization and tracking functions are used, then basic AR display is achieved, but accuracy is insufficient because simulated and actual robot positions do not precisely match
Solution Approach 1:
The patent replaces conventional localization and tracking functions with a structured light-based optical measurement system. Instead of relying on standard sensor fusion and localization algorithms that produce approximate positions, the system uses projected structured light patterns and their interaction with the robot's own geometry to obtain precise pose information that accurately reflects the robot's actual position and orientation.
Solution Approach 2:
The system changes the measurement parameters by using structured light projection rather than passive marker detection or active sensor scanning. This parameter change enables more precise measurement of the robot's pose by utilizing the known geometry of the projected light patterns and their interaction with the robot's surfaces, providing higher accuracy position and orientation data.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
The invention relates to a method for use with a machine (20), in particular a mobile or stationary robot, especially an articulated robot. The method includes a calibration of an augmented reality display environment (22), namely by determining a relative pose between the machine (20) and an image acquisition device (18A) by recognizing the machine (20) and/or one or more parts of the machine (20) in a captured image based on a three-dimensional machine model. Furthermore, a coordinate alignment is performed between a machine coordinate system (28) of the machine (20) and a coordinate system (26) of the image acquisition device (18A) based on the determined relative pose.