Mixed Reality Digital Twin Inspection for 3D Anomaly Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D defect detection systems face challenges in large-scale manufacturing due to inadequate coverage, false positives, and the lack of high-quality training data, especially for complex and uniquely designed objects, limiting their effectiveness in enterprise settings.
Innovation Solution
A mixed reality system using enterprise digital twins and a recursively fused multimodal segmentation paradigm for 3D defect detection, which reduces the need for extensive training data by combining multiple pre-trained deep learning models to enhance accuracy and adaptability, enabling portable and robust 3D defect inspection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional 3D defect detection systems are used, then defect detection capability is provided, but measurement precision and reliability deteriorate due to false positives and inadequate coverage
Solution Approach 1:
The patent combines multiple pre-trained deep learning models (e.g., PointNet++, Point Transformer) into a fused multimodal segmentation paradigm. This merging of models allows the system to leverage diverse feature extraction capabilities, reducing false positives and improving both measurement precision and reliability through ensemble decision-making rather than relying on a single model's output
Solution Approach 2:
The patent applies recursive segmentation to divide the 3D defect detection task into multiple hierarchical levels. By segmenting the detection process into coarse-to-fine stages, the system achieves submillimeter precision while maintaining reliability through progressive refinement of defect boundaries and characteristics at each segmentation level
2Measurement precision
If extensive training data is collected for complex objects, then detection accuracy improves, but loss of time and productivity deteriorate due to data collection and processing requirements
Solution Approach 1:
The patent employs pre-trained deep learning models that have already been trained on extensive datasets beforehand. This preliminary action transfers learning from pre-training to the specific defect detection task, eliminating the need for extensive on-site data collection and processing while maintaining high detection accuracy through feature extraction learned during pre-training
Solution Approach 2:
The patent uses digital twins as virtual copies of physical objects to perform defect detection in the virtual space. By copying the object's geometry and properties into a digital model, the system can detect defects without physically inspecting the actual object, significantly reducing inspection time while maintaining detection accuracy through virtual-real alignment
3Ease of operation
If portable inspection systems are deployed, then ease of operation improves, but measurement precision deteriorates due to limited coverage and resolution
Solution Approach 1:
The patent transitions from 2D image-based inspection to 3D point cloud-based inspection using depth sensors and spatial coordinates. This dimensional change enables portable systems to capture comprehensive 3D geometry information, achieving submillimeter precision while maintaining portability by processing spatial data directly without requiring high-resolution 2D imagery
Data Source
AI summary
A system and method for 3D anomaly detection and tracking are provided that uses multimodal fusion, reduced training data, recursive segmentation and histogram statistic distance. The anomaly may be a defect or a configuration error.