A 3D spectral confocal scanning defect detection method based on a DFine model
The 3D spectral confocal scanning defect detection method based on the DFine model solves the problem that traditional algorithms cannot effectively utilize 3D spectral confocal scanning data, and achieves efficient detection of complex and weak defects. It improves detection accuracy and robustness and is suitable for high-speed online detection in industrial automated production lines.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHANGZHOU REECHI PRECISION MEASURETECH CO LTD
- Filing Date
- 2026-03-16
- Publication Date
- 2026-06-12
AI Technical Summary
In existing technologies, 3D spectral confocal scanning data is large in volume and high in dimensionality, and is easily affected by differences in material reflectivity, surface roughness and noise. Traditional algorithms have difficulty making full use of data features, and the accuracy of defect detection is low and the robustness is poor, especially the ability to detect complex and weak defects is limited.
A 3D spectral confocal scanning defect detection method based on the DFine model is adopted. By acquiring depth map and grayscale data, pseudo-color and pseudo-3D rendering are performed. Combined with DFine model training and post-processing algorithms, multi-scale features are extracted to identify defects. Post-processing is then performed to improve detection accuracy and robustness.
It improves the accuracy and robustness of defect detection, effectively detects complex shapes and weak feature defects, reduces reliance on human experience and rule thresholds, and is suitable for high-speed online detection scenarios.
Smart Images
Figure CN122199482A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial vision inspection and intelligent manufacturing technology, and more specifically, it relates to a 3D spectral confocal scanning defect detection method based on the DFine model. Background Technology
[0002] With the rapid development of artificial intelligence technology, especially deep learning technology, intelligent sensing and analysis methods based on large-scale data have been widely used in the field of industrial inspection. Deep learning models, through multi-layer nonlinear feature extraction structures, can automatically learn high-level semantic features from complex data, demonstrating significantly better performance than traditional algorithms in tasks such as target detection, defect recognition, and image segmentation.
[0003] Spectral confocal scanning technology, characterized by its non-contact nature, high precision, and high axial resolution, is widely used in precision manufacturing for the detection of workpiece surface morphology, thickness, and minute defects. This technology typically obtains high-resolution three-dimensional height data or spectral response data by scanning the surface of the object being measured point-by-point or line-by-line.
[0004] However, in practical industrial applications, the 3D data acquired by spectral confocal scanning has the following problems: 1. The data volume is large and the dimensionality is high, containing height information, spectral intensity information and spatial location information, which traditional algorithms cannot fully utilize; 2. The scanning process is easily affected by differences in material reflectivity, surface roughness, noise and environmental interference, resulting in a large number of false features in the data, which are difficult for traditional algorithms to distinguish. 3. Existing defect detection methods mostly rely on human experience to design features or simple threshold rules, which have limited ability to detect complex morphological defects, weak defects and multi-scale defects.
[0005] Therefore, there is an urgent need for a defect detection technology that can combine the advantages of artificial intelligence technology, fully explore the intrinsic feature correlations of 3D spectral confocal scanning data, and has good generalization ability and robustness. Summary of the Invention
[0006] The technical problem to be solved by the present invention is to provide a 3D spectral confocal scanning defect detection method based on the DFine model, so as to solve the problems of insufficient utilization of three-dimensional spectral confocal data, low accuracy of complex defect detection and poor robustness in the prior art.
[0007] The technical solution adopted by this invention to solve its technical problem is: A 3D spectral confocal scanning defect detection method based on the DFine model includes the following steps: S1, acquire 3D spectral confocal scanning data of the area to be detected, the scanning data including depth map data, grayscale map data and scanning interval; S2, based on the preprocessed depth map and grayscale data, performs pseudo-color and pseudo-3D rendering imaging to highlight the defect area; S3. Based on the defect information labeled in the rendered image, create a dataset and construct a DFine model that simultaneously inputs the depth map and grayscale image to extract multi-scale features and identify defects; S4. Based on the defect information, depth map, and grayscale image, train, validate, and deploy the DFine model; S5 inputs the scan data from S1 into the trained DFine model for inference, and combines it with the post-processing algorithm to obtain the final defect detection result.
[0008] Preferably, the pseudo-color rendering in step S2 is based on the color spectrum mapping of depth values, and the pseudo-3D rendering enhances the surface morphology differences through the lighting model.
[0009] Preferably, the DFine model constructed in step S3 includes an encoder and a decoder. The encoder simultaneously receives a depth map and a grayscale image as dual-channel inputs and fuses feature information through a cross-channel attention mechanism.
[0010] Preferably, the encoder of the DFine model adopts a multi-level convolution module, and the decoder adopts an upsampling and skip connection structure to generate the bounding box, category, and segmentation mask information of the defects.
[0011] Preferably, the training process in step S4 includes jointly optimizing model parameters using the cross-entropy loss function and Dice coefficients, and the validation process uses an independent test set to evaluate model performance.
[0012] Preferably, the post-processing in step S5 includes: S501, Nonmaximum Suppression Based on Class Consistency; S502, defect merging or filtering based on spatial inclusion relationships; S503: Construct a standard spatial coordinate system for the defect area, calculate the three-dimensional size information of the defect, and perform filtering.
[0013] Preferably, the three-dimensional dimension calculation in S503 includes the volume of the defect area, the maximum height difference, and the projected area, and the dimension is filtered by setting a threshold.
[0014] Preferably, the post-processing further includes ranking the detected defects by confidence level and visualizing them.
[0015] Preferably, the acquisition of the 3D spectral confocal scanning data adopts vertical scanning interferometry, and the spectral dimensions cover at least 3 wavelength channels.
[0016] Preferably, the 3D spectral confocal scanning defect detection method is applied to high-speed online defect detection scenarios in automated production lines.
[0017] The beneficial effects of this invention are: 1. It fully utilizes the height and spectral information of 3D spectral confocal scanning data to improve the accuracy of defect detection; 2. It achieves multi-scale feature modeling through the DFine model, which has a stronger detection capability for complex shapes and weak feature defects; 3. It reduces the dependence on human experience and rule thresholds, and improves the automation and versatility of the system; 4. It is suitable for high-speed online detection scenarios and has good industrial applicability. Attached Figure Description
[0018] Figure 1 This is a schematic diagram of the structure of a 3D spectral confocal scanning defect detection method based on the DFine model; Figure 2 This is a schematic diagram of defect detection and post-processing. Detailed Implementation
[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0020] It should be noted that, unless otherwise specified, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0021] In this invention, unless otherwise stated, the directional terms such as "up" and "down" generally refer to the directions shown in the accompanying drawings, or to the vertical, perpendicular, or gravitational direction; similarly, for ease of understanding and description, "left" and "right" generally refer to the left and right shown in the accompanying drawings; "inner" and "outer" refer to the inner and outer contours of each component itself, but the above directional terms are not intended to limit this invention.
[0022] Referring to Figure S1, it is an overall flowchart of an embodiment of the present invention.
[0023] A 3D spectral confocal scanning defect detection method based on the DFine model includes the following steps: Step 1: Acquire 3D spectral confocal scanning data A spectral confocal scanning sensor is used to scan the workpiece line by line to obtain two-dimensional grid data corresponding to the workpiece surface, including: 1. Depth map data: The surface height value is represented in the form of a two-dimensional array, and the pixel coordinates and the physical coordinates of the workpiece correspond one-to-one according to the scanning interval in the x and y directions; 2. Grayscale image data: Light reflection intensity image corresponding to the depth map space, used to supplement texture and reflection difference information.
[0024] In this application, the depth map resolution is W*H (e.g., 2048*2542), and the depth value is 32 bits; the grayscale image is an 8-bit single-channel image. The depth map and grayscale image are spatially aligned using parameters of the scanning sensor, including pixel spacing, coordinate zero point, scanning direction, etc., to ensure that the depth value and grayscale value at the same pixel location correspond to the same surface point.
[0025] Step S2: Depth Map-Based Pseudo-Color and Pseudo-3D Rendering Imaging To enhance defect visibility and facilitate annotation, this embodiment renders the depth map to generate a "rendered image" to highlight the defect area. The rendering process includes: 1. Handling invalid and outlier values: Mark invalid depths (such as fixed sentinel values, NaN, or heights exceeding the threshold range) in the depth map as invalid regions; 2. Normalization and pseudo-color mapping: The effective depth values are normalized according to the height range, and the normalized values are mapped to a preset pseudo-color lookup table (such as turbo / viridis, etc.) to form a pseudo-color image; 3. Pseudo-3D lighting enhancement: Based on the gradient or normal of the depth map, the local orientation of the surface is estimated, and the lighting intensity is calculated in combination with the preset lighting direction (e.g., incident from the upper left). The lighting is modulated on the pseudo-color image, thereby highlighting the local undulations and boundaries of defects while maintaining high continuity. 4. Background generation: Generate a background brightness distribution that creates a sense of distance from the foreground for invalid areas, in order to reduce abrupt boundaries and enhance overall readability.
[0026] In this application, the rendering parameters include: lighting direction vector, lighting intensity coefficient, pseudo-color mapping method, depth normalization interval (which can be global min / max or percentile range), and background grayscale and gradient coefficients, etc. The rendered image R is obtained through the above rendering process, and its size and depth... Figure 1 To.
[0027] Step S3: Perform defect annotation based on the rendered image and construct a customized DFine input. In this embodiment, a rendered image R is used for defect annotation to obtain defect annotation data. Annotation methods may include: 1. Target detection annotation: Defect bounding box (bbox); 2. Instance / Semantic Segmentation Annotation: Defect Mask; 3. And defect category labels (class).
[0028] We construct a DFine model input format that simultaneously inputs depth maps and grayscale images. We concatenate the depth maps and grayscale images by channel to form a two-channel tensor input. We extract features from the depth map branch and the grayscale image branch respectively, and then perform feature fusion to improve the learnability of defect features.
[0029] In this application, the input tensor can be represented as X = co Where D_norm is the normalized depth image and G_norm is the normalized grayscale image, with dimensions H×W×C (C is the number of channels).
[0030] Step S4: Training, validation, and deployment based on labeled data, depth maps, and grayscale images In this embodiment, the labeled data is used to construct a training set and a validation set (optionally a test set), and the DFine model is trained. The training process includes: 1. Data organization: Each sample contains a depth map, a grayscale image, and corresponding annotations (bbox / mask / class), and maintains spatial alignment between depth and grayscale. 2. Data Augmentation: Perform consistent geometric augmentation (flip, rotate, crop, scale) on both depth and grayscale, and apply amplitude perturbation or noise simulation to the depth map to enhance robustness; 3. Training hyperparameters: including but not limited to batch size, learning rate, number of iterations, input size (which can maintain the original resolution or use a slicing strategy), and loss function weights, etc. 4. Validation and Model Selection: Use validation set statistical metrics (e.g., mAP, IoU, recall, false negative rate, false positive rate, etc.) to select the best model; 5. Deployment and Export: Export the trained DFine model in deployment format (ONNX / TensorRT) and solidify the input specifications, normalized parameters, and post-processing parameters.
[0031] In this application, for ultra-high resolution input, a "sliding window slice reasoning" strategy is adopted: slices are made with a fixed window size (1024×1024) according to the overlap step, ensuring that defects can still be completely captured at the slice boundary.
[0032] Step S5: Reasoning and post-processing to obtain the final defect result In the actual inference and detection phase, the depth map and grayscale image of the sample to be tested are acquired. Following the same preprocessing method as in the training phase, these are used to form the model input. The trained DFine model is then used to perform inference, obtaining candidate defect results. Candidate results may include: 1. Defect candidate boxes and confidence levels; 2. Defect category; 3. Defect mask or pixel-level segmentation results.
[0033] To obtain stable and reliable final detection results, this embodiment further incorporates post-processing algorithms, including but not limited to: refer to Figure 2 Here is a flowchart of the post-processing algorithm, which includes the following steps: 1. Sliding slice result fusion: Map each slice result back to the original image coordinates, and perform deduplication and merging; 2. Same-category NMS suppression: Sort candidate boxes of the same category by confidence and perform IoU threshold suppression; 3. The filtered defective areas are cropped on the depth map, and the cropped local areas are standardized in three-dimensional coordinate system; 4. Calculate the length, width, and height of the region from the standardized 3D point set to obtain the size information of the defect, and make further defect judgments based on the size information.
[0034] The final output of the defect detection results includes, but is not limited to, the location, size, type, and corresponding mask of the defect in the workpiece coordinate system; and the detection results can be overlaid on the rendered image or the original grayscale image for manual review and traceability.
[0035] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0036] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.
[0037] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A 3D spectral confocal scanning defect detection method based on the DFine model, characterized by: Includes the following steps: S1, acquire 3D spectral confocal scanning data of the area to be detected, the scanning data including depth map data, grayscale map data and scanning interval; S2, based on the preprocessed depth map and grayscale data, performs pseudo-color and pseudo-3D rendering imaging to highlight the defect area; S3. Based on the defect information labeled in the rendered image, create a dataset and construct a DFine model that simultaneously inputs the depth map and grayscale image to extract multi-scale features and identify defects; S4. Based on the defect information, depth map, and grayscale image, train, validate, and deploy the DFine model; S5 inputs the scan data from S1 into the trained DFine model for inference, and combines it with the post-processing algorithm to obtain the final defect detection result.
2. The 3D spectral confocal scanning defect detection method based on the DFine model according to claim 1, characterized in that: In step S2, pseudo-color rendering is based on color mapping of depth values, and pseudo-3D rendering enhances surface morphology differences through lighting models.
3. The 3D spectral confocal scanning defect detection method based on the DFine model according to claim 1, characterized in that: The DFine model constructed in step S3 includes an encoder and a decoder. The encoder receives both depth maps and grayscale maps as dual-channel inputs and fuses feature information through a cross-channel attention mechanism.
4. The 3D spectral confocal scanning defect detection method based on the DFine model according to claim 3, characterized in that: The encoder of the DFine model uses a multi-level convolution module, and the decoder uses an upsampling and skip connection structure to generate the bounding box, category, and segmentation mask information of the defects.
5. The 3D spectral confocal scanning defect detection method based on the DFine model according to claim 1, characterized in that: The training process in step S4 includes jointly optimizing model parameters using the cross-entropy loss function and Dice coefficients, and the validation process uses an independent test set to evaluate model performance.
6. The 3D spectral confocal scanning defect detection method based on the DFine model according to claim 1, characterized in that: The post-processing in step S5 includes... S501, Nonmaximum Suppression Based on Class Consistency; S502, defect merging or filtering based on spatial inclusion relationships; S503: Construct a standard spatial coordinate system for the defect area, calculate the three-dimensional size information of the defect, and perform filtering.
7. The 3D spectral confocal scanning defect detection method based on the DFine model according to claim 6, characterized in that: The three-dimensional dimension calculation in S503 includes the volume of the defect area, the maximum height difference, and the projected area, and the dimension is filtered by setting a threshold.
8. The 3D spectral confocal scanning defect detection method based on the DFine model according to claim 7, characterized in that: The post-processing also includes ranking the detected defects by confidence level and visualizing them.
9. The 3D spectral confocal scanning defect detection method based on the DFine model according to claim 1, characterized in that: The 3D spectral confocal scanning data was acquired using vertical scanning interferometry, and the spectral dimensions covered at least three wavelength channels.
10. The 3D spectral confocal scanning defect detection method based on the DFine model according to claim 1, characterized in that: The 3D spectral confocal scanning defect detection method is applied to high-speed online defect detection scenarios in automated production lines.