Digital Twin Alignment Using 3D CAD and Camera Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital twin technologies face challenges in accurately expressing objects in a physical space due to the complexity of the site, leading to inaccuracies in camera parameter estimation, which hinders appropriate representation in a virtual space.
Innovation Solution
A digital twin management program that includes a digital twin management device for acquiring and processing image data, extracting feature lines, estimating camera parameters, and performing alignment processing to generate setting information for accurately positioning objects in a virtual space using three-dimensional CAD models and machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If camera parameters are estimated using feature lines extracted from objects in the environment, then the digital twin can be generated automatically, but the accuracy of camera parameters deteriorates due to site complexity
Solution Approach 1:
The patent introduces a reference object with known three-dimensional information as an intermediary between the camera and the environment. This reference object serves as a mediator that provides reliable geometric constraints for camera parameter estimation, independent of the complex environmental features. By using the reference object's known geometry as an intermediary reference frame, the system achieves accurate camera calibration without being affected by site complexity.
Solution Approach 2:
The patent uses a three-dimensional model (copy) of the reference object with known geometric information to establish accurate spatial relationships. Instead of relying on uncertain feature extraction from environmental objects, the system creates a digital copy of the reference object with precise known dimensions and uses this copy as the basis for camera parameter estimation, ensuring high accuracy.
2Measurement precision
If manual adjustment of camera parameters is performed to align 3D model ridge lines with object edges, then alignment accuracy improves, but the calibration process becomes complex and time-consuming
Solution Approach 1:
The patent enables the system to automatically determine camera parameters by itself without requiring manual intervention. The control device automatically extracts feature lines from the captured image, identifies correspondence with the three-dimensional model, calculates camera parameters through geometric computation, and generates the digital twin autonomously. This self-service approach eliminates manual calibration while maintaining high alignment accuracy.
Solution Approach 2:
The patent replaces the manual mechanical adjustment process with an automated computational system. Instead of manually adjusting camera parameters to align 3D models with object edges, the system uses computer vision algorithms to extract feature lines, geometric computation to calculate camera parameters, and automated processing to achieve alignment. This substitution of manual mechanical operations with automated computational methods reduces complexity while maintaining precision.
3Measurement precision
If multiple camera parameters are adjusted to match main ridge lines and unnecessary edges, then representation accuracy improves, but it becomes difficult to appropriately set all parameters
Solution Approach 1:
The patent extracts only the essential feature lines from the captured image that are necessary for accurate camera parameter determination. Instead of requiring adjustment of multiple parameters to match all edges including unnecessary ones, the system selectively extracts relevant feature lines that provide sufficient geometric constraints. This extraction of essential features simplifies the parameter setting process while maintaining accurate object representation.
Solution Approach 2:
The patent uses a partial approach by focusing on key feature lines and critical geometric correspondences rather than attempting to match all edges and features. By applying camera parameter estimation based on selected essential feature lines and their correspondence with the three-dimensional model, the system achieves sufficient accuracy without the complexity of adjusting multiple parameters for every detail.
Data Source
AI summary
A non-transitory computer-readable recording medium stores therein a digital twin management program that causes a computer to execute a process including acquiring a two-dimensional video in which an object is arranged in a physical space, the two-dimensional video being captured by a camera device, acquiring three-dimensional design data defining formation of the object in a virtual space, setting a position and a posture of the object in the three-dimensional design data with respect to the object in the two-dimensional video, and generating setting information defining a correspondence relationship between the physical space and the virtual space in digital twin based on feature values of the set position and the set posture.