RGB-Depth Defect Detection for Global and Local Anomaly Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 2D and 3D defect detection algorithms in the electronic manufacturing industry face limitations in detecting 3D defects, leading to poor defect detection performance.
Innovation Solution
A defect detection method and device that combines RGB and depth images using a feature extraction network for feature map fusion, followed by a global and local defect detection network to generate a comprehensive defect score map, enhancing defect detection capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If 2D AOI equipment and 2D defect detection algorithms are used, then the system is simple and easy to operate, but the defect detection capability is limited and cannot detect 3D defects
Solution Approach 1:
The patent transitions from 2D defect detection to 3D defect detection by incorporating depth information. A depth estimation module is introduced to generate depth maps from 2D images, adding the depth dimension to the detection process. This enables the system to detect 3D defects while maintaining compatibility with existing 2D AOI equipment, thus improving detection capability without significantly increasing system complexity.
Solution Approach 2:
The patent combines 2D image data with estimated depth information to create a composite 3D representation of the detection object. By fusing 2D visual features with 3D depth features, the system achieves enhanced defect detection capability that leverages the strengths of both 2D and 3D approaches while avoiding the full complexity of dedicated 3D sensing systems.
2Reliability
If existing 3D defect detection algorithms are used, then depth information is supported, but the defect detection performance is poor
Solution Approach 1:
The patent employs a nested network architecture where a shallow network and a deep network are hierarchically organized. The shallow network extracts local features and generates initial defect scores, while the deep network processes global features and refines the detection results. This nested structure allows the system to preserve both local and global feature information, improving defect detection performance while preventing feature information loss that occurs in single-network architectures.
3Measurement precision
If only global defect detection is performed, then the detection process is simple, but small local defects are missed
Solution Approach 1:
The patent segments the defect detection process into two distinct networks: a shallow network focused on local defect detection and a deep network focused on global defect detection. The shallow network uses smaller receptive fields to capture fine-grained local features, while the deep network uses larger receptive fields to capture global context. This segmentation allows the system to detect both small local defects and large global defects with high precision, while managing network complexity through specialized modular designs.
4Measurement precision
If only local defect detection is performed, then small defects are detected well, but large defects and global patterns are missed
Solution Approach 1:
The patent merges the outputs of the shallow local defect detection network and the deep global defect detection network through a fusion module. The defect score maps from both networks are combined to produce a final comprehensive defect detection result. This merging strategy ensures that both small local defects and large global defects are detected with high accuracy, while the modular fusion approach keeps the overall system complexity manageable.
Data Source
AI summary
Provided is a defect detection method and device, computer equipment and a storage medium. The method includes: acquiring an RGB image, a depth image and a sample label of a detection object sample; performing feature map extraction and feature map fusion on the RGB image and the depth image by a feature extraction network of the defect detection model, to obtain a fused feature map; performing defect detection based on the fused feature map by a feature reconstruction network of the defect detection model, to obtain a defect score map, wherein the defect score map being obtained by fusing a global defect score map which is generated based on a global defect detection network with a local defect score map which is generated by a local defect detection network; and updating parameters of the defect detection model based on the defect score map and the sample label.