AR Robot Calibration Using a 3D Digital Twin Pose
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Calibration of augmented reality (AR) devices for industrial robots is challenging due to the need for precise placement of visual targets, which is time-consuming and prone to errors, especially for robots with occluded axes or 'delta'-type robots, where single-axis methods are imprecise or impossible.
Innovation Solution
The method uses the robot itself as a 3D calibration target by providing a CAD model of the entire robot in its current pose, allowing the AR system to analyze images for precise calibration without requiring a conventional visual calibration target, enabling continuous tracking and self-correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a planar visual target or fiducial is placed at a precise fixed location in the robot work cell for calibration, then the position and orientation of the AR device can be determined, but the calibration process becomes time-consuming and costly due to the large effort required for precise placement
Solution Approach 1:
The robot serves itself as the calibration target. The system uses the robot's own 3D model and current pose information to perform self-calibration, eliminating the need for external operators to place and measure physical fiducials. The robot controller provides joint position data that automatically generates the expected 3D model configuration for comparison with camera images.
Solution Approach 2:
Instead of using a physical fiducial target, the system creates a virtual 3D model copy of the robot in its current pose. This digital twin is generated from CAD models and joint position data, then used for calibration by comparing it with images captured by the AR device camera. The virtual model replaces the need for physical calibration artifacts.
2Measurement precision
If a planar visual target is used for calibration, then the transformation between coordinate systems can be determined, but calibration imprecision occurs due to small tilts in the calibration plane leading to large errors at longer distances
Solution Approach 1:
The system transitions from 2D planar fiducial targets to 3D volumetric robot models for calibration. By using the robot's three-dimensional structure with multiple features at different depths and positions, the calibration becomes insensitive to camera angle and distance. The 3D model provides geometric constraints in all spatial dimensions, eliminating the sensitivity issues inherent in 2D planar targets.
3Ease of manufacture
If calibration is performed to a 3D model of the robot in a single predefined pose, then some calibration can be achieved, but precision is reduced because information is localized and some robot axes may be occluded
Solution Approach 1:
The calibration system dynamically adapts to the robot's current pose rather than requiring a fixed predefined pose. The 3D model is regenerated in real-time based on the robot controller's joint position data, allowing calibration to work effectively from any robot configuration. This dynamic approach ensures all robot axes are visible and provides comprehensive geometric information regardless of the robot's position.
Solution Approach 2:
The calibration method works universally for any robot pose and type, including delta-type robots with parallel kinematics. By using the robot's own dynamic 3D model rather than a static single-pose model, the system achieves comprehensive calibration coverage for all robot axes and configurations, making the calibration process universally applicable.
4Measurement precision
If a conventional visual calibration target is used, then calibration can be performed, but the device complexity and cost increase due to the need for precise fiducial placement and measurement equipment
Solution Approach 1:
The robot system performs its own calibration using its existing components (controller, joint position sensors, and CAD models) without requiring external calibration equipment. The robot controller already contains the necessary geometric information and joint data to generate the 3D model, eliminating the need for separate measurement devices and complex calibration toolkits.
Solution Approach 2:
The system uses virtual 3D model copies generated from existing CAD data and joint position information, replacing the need for physical calibration targets and complex measurement equipment. This digital copying approach simplifies the hardware requirements while maintaining calibration precision.
Data Source
AI summary
A method and system for calibration of an augmented reality (AR) device's position and orientation based on a robot's positional configuration. A conventional visual calibration target is not required for AR device calibration. Instead, the robot itself, in any pose, is used as a three dimensional (3D) calibration target. The AR system is provided with a CAD model of the entire robot to use as a reference frame, and 3D models of the individual robot arms are combined into a single object model based on joint positions known from the robot controller. The 3D surface model of the entire robot in the current pose is then used for visual calibration of the AR system by analyzing images from the AR device camera in comparison to the surface model of the robot in the current pose. The technique is applicable to initial AR device calibration and to ongoing device tracking.