A method for detecting surface defects of a metal plate
By employing multimodal data acquisition and deep fusion technology, the efficiency and accuracy issues of metal plate surface defect detection have been resolved, achieving fully automated defect detection and quantitative analysis, adapting to complex environments, reducing false alarm rates, and improving production efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU LEQIE METAL TECH CO LTD
- Filing Date
- 2026-02-27
- Publication Date
- 2026-06-16
AI Technical Summary
In existing technologies, the detection efficiency of metal plate surface defects is low, the false detection rate and the missed detection rate are high, and it is difficult to handle complex backgrounds and variable defect morphologies. Traditional methods have poor robustness, and manual detection is unstable.
By employing multimodal data acquisition, adaptive data alignment and preprocessing, a bi-branch feature extraction network, and deformable attention fusion technology, deep fusion of two-dimensional vision and three-dimensional morphological features is achieved, and defect detection and quantification are performed in conjunction with adaptive anchor boxes.
It achieves efficient and accurate defect detection and quantification, significantly reduces false alarm rate, improves production efficiency, provides comprehensive inspection reports, adapts to complex environments, and replaces manual inspection.
Smart Images
Figure CN122222912A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of product defect detection technology, and in particular to a method for detecting defects on the surface of metal plates. Background Technology
[0002] As a basic industrial material, the surface quality of sheet metal directly affects the performance and reliability of the final product. During the production process, due to differences in raw materials, fluctuations in process parameters, changes in equipment status, and interference from environmental factors, the surface of sheet metal is prone to various defects such as scratches, pits, inclusions, cracks, pitting, and scabs. Traditional inspection methods relying on manual visual inspection are not only inefficient and labor-intensive, but also susceptible to subjective factors and fatigue of personnel, resulting in unstable detection rates and failing to meet the quality control requirements of modern industry.
[0003] Traditional image processing methods (such as edge detection, thresholding, and texture analysis) are used for defect identification. These methods are computationally simple, but they are not robust enough to handle complex backgrounds, noise interference, and variable defect shapes, resulting in high false positive and false negative rates.
[0004] In recent years, deep convolutional neural networks have been widely used in surface defect detection due to their powerful feature learning capabilities. For example, object detection algorithms such as Faster R-CNN, YOLO series, and SSD can simultaneously locate defect positions and identify their categories. However, most algorithms are based on anchor box designs with regular shapes, which are insufficient for extracting features of small, irregular, or morphologically varied defects; complex backgrounds such as oxide scale, oil stains, and uneven lighting on metal plate surfaces can easily be confused with real defects, leading to misjudgments. Summary of the Invention
[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for detecting defects on the surface of metal plates.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: A method for detecting surface defects in metal plates, implemented based on a metal plate surface defect detection system, the system comprising: A multimodal data acquisition module, wherein the multimodal data acquisition module acquires multidimensional data from the surface of the metal plate; A data preprocessing module, which performs standardization and enhancement on the collected raw data; The feature extraction module extracts feature representations of different modalities; The fusion module achieves deep fusion of two-dimensional visual features and three-dimensional morphological features by introducing deformable attention. The defect identification and quantification module performs defect detection, classification, and quantification analysis based on the fused features.
[0007] Preferably, the multimodal data acquisition module acquires data including: High-resolution color images are used to acquire surface texture, color features, and two-dimensional morphological information; 3D line laser scanning data, through laser triangulation, obtains three-dimensional surface topography data, accurately characterizing the depth features of depressions and protrusions; Infrared thermal imaging data is used to identify internal defects and microcracks by utilizing differences in surface thermal conductivity.
[0008] Preferably, the multimodal data acquisition module, based on an adaptive calibration algorithm for feature points, achieves spatiotemporal alignment of various data, as detailed below:
[0009] in, Represents the optimal transformation matrix. and Let represent the matching feature points in the RGB image and the depth image, respectively, and T be the transformation matrix to be solved.
[0010] Preferably, the data preprocessing module performs standardization and enhancement through adaptive grayscale run-length matrix feature extraction and multi-scale adaptive smoothing filtering. Specifically, the adaptive grayscale run-length matrix feature extraction is as follows: Perform continuous gray-level dimensionality reduction on a grayscale image, obtain the gray-level run-length matrix of the image after each dimensionality reduction, calculate the dimensionality reduction confidence based on this matrix, and determine the maximum allowable degree of dimensionality reduction.
[0011] in, This represents the sum of the elements of the gray-level run-length matrix after the k-th dimensionality reduction. Indicates the credibility of dimensionality reduction. The confidence variance is used to determine the optimal dimensionality reduction level.
[0012] Preferably, the data preprocessing module performs multi-scale adaptive smoothing filtering as follows: The size of the smoothing filter window for each pixel is adaptively determined based on the maximum allowable dimensionality reduction and local features:
[0013] in, Indicates adaptive window size. This refers to the number of gray levels contained in each grayscale level under the maximum allowable dimensionality reduction. This represents the grayscale value of a pixel.
[0014] Preferably, the dual-branch feature extraction network, when performing feature extraction, specifically includes two-dimensional visual feature extraction and three-dimensional shape feature extraction. The two-dimensional visual feature extraction method is as follows: Based on the improved YOLOv5n architecture, the original CBS downsampling module is replaced with the ADown module in the backbone and neck network, which reduces the number of model parameters and improves computational efficiency. The ADown module first reduces the spatial dimension of the input feature map through the average pooling layer, then divides the feature map into two equal parts, processes them differently, and finally merges the feature maps.
[0015] Preferably, the dual-branch feature extraction network extracts 3D shape features in the following way: A deformable convolutional network is used to extract 3D surface topography features, and the ability to model irregularly shaped defects is enhanced by learningable sampling points. The calculation formula for deformable convolution is as follows:
[0016] Where p represents spatial location. The kernel size is [size]. and These represent the learnable offset and modulation coefficient, respectively.
[0017] Preferably, the fusion module operates as follows:
[0018] Where q is the query vector. A lightweight offset generation network. Where M is the spatial offset and M is the number of attention heads. Attention weights; The fusion module adopts a phased fusion strategy, performing feature fusion at the shallow, middle and deep layers respectively, taking into account both detailed and semantic information.
[0019] Preferably, the defect identification and quantification module performs defect detection, classification, and quantitative analysis in the following manner: Defect detection and classification: Adaptive anchor boxes are used, and the width and height of the labeled boxes in the training set are clustered using the K-means++ algorithm to generate initial anchor box sizes that are more suitable for the specific dataset. Defect Quantification Analysis: Accurately quantifies detected defects, including automatic measurement of defect area, depth, and length parameters.
[0020] Preferred method: The detection method includes the following steps: S1: Simultaneously acquires high-resolution RGB images, 3D line laser scanning data, and infrared thermal imaging data; employs a combined mechanism of synchronous triggering hard synchronization and timestamp-aligned soft synchronization to ensure that data of different modalities are aligned in time and space; S2: Standardize, denoise, and enhance the data for each modality; employ a nonlinear denoising algorithm based on singular value decomposition for ultrasound data; and apply multi-scale adaptive smoothing filtering and other techniques to image data to improve the signal-to-noise ratio. S3: Parallel extraction of two-dimensional visual features and three-dimensional morphological features; S4: Deeply fuse the extracted two-dimensional and three-dimensional features, and dynamically adjust the attention focus based on the deformable attention mechanism to achieve adaptive weighted fusion between features of different modalities; S5: Decode the fused features and output the defect category, location, size, and depth information; use an adaptive anchor frame mechanism to optimize the positioning accuracy, and finally generate a comprehensive inspection report containing various parameters of the defect.
[0021] The beneficial effects of this invention are as follows: 1. This invention proposes a spatiotemporally synchronized multimodal data acquisition scheme and solves the matching problem in multi-source data fusion based on an adaptive data alignment algorithm; it also designs an asymmetric dual-branch network structure to balance computational efficiency and feature extraction capability for different modal data characteristics.
[0022] 2. This invention introduces a deformable attention transformer into multimodal feature fusion, dynamically adjusts the attention focus, and significantly improves the detection capability for small and complex defects.
[0023] 3. This invention automates the entire process from data acquisition to defect quantification, providing a comprehensive analysis report that includes not only defect location and category, but also detailed quantification parameters.
[0024] 4. The multimodal data cross-validation of this invention can effectively suppress common interferences such as oil stains and reflections, significantly reduce the false alarm rate, and has a wide system environmental adaptability.
[0025] 5. This invention replaces manual visual inspection with fully automated detection, directly reducing labor costs and improving production efficiency due to the increased detection speed; by accurately identifying defects and linking them to production process data, it can promote quality control throughout the entire process, helping enterprises optimize processes and reduce scrap rates. Attached Figure Description
[0026] Figure 1 This is a flowchart of a method for detecting surface defects in metal plates proposed in this invention. Detailed Implementation
[0027] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.
[0028] Example 1: A method for detecting surface defects in a metal plate, implemented based on a metal plate surface defect detection system, the system comprising: A multimodal data acquisition module, wherein the multimodal data acquisition module acquires multidimensional data from the surface of the metal plate; A data preprocessing module, which performs standardization and enhancement on the collected raw data; The feature extraction module extracts feature representations of different modalities; The fusion module achieves deep fusion of two-dimensional visual features and three-dimensional morphological features by introducing deformable attention. The defect identification and quantification module performs defect detection, classification, and quantification analysis based on the fused features.
[0029] The multimodal data acquisition module acquires data including: High-resolution color images are used to acquire surface texture, color features, and two-dimensional morphological information; 3D line laser scanning data, through laser triangulation, obtains three-dimensional surface topography data, accurately characterizing depth features such as depressions and protrusions; Infrared thermal imaging data is used to identify internal defects and microcracks by utilizing differences in surface thermal conductivity.
[0030] The multimodal data acquisition module, based on an adaptive calibration algorithm for feature points, achieves spatiotemporal alignment of various data, as detailed below:
[0031] in, Represents the optimal transformation matrix. and Let represent the matching feature points in the RGB image and the depth image, respectively, and T be the transformation matrix to be solved.
[0032] The data preprocessing module performs standardization and enhancement through adaptive gray-scale run-length matrix feature extraction and multi-scale adaptive smoothing filtering. The adaptive gray-scale run-length matrix feature extraction is specifically as follows: Perform continuous gray-level dimensionality reduction on a grayscale image, obtain the gray-level run-length matrix of the image after each dimensionality reduction, calculate the dimensionality reduction confidence based on this matrix, and determine the maximum allowable degree of dimensionality reduction.
[0033] in, This represents the sum of the elements of the gray-level run-length matrix after the k-th dimensionality reduction. Indicates the credibility of dimensionality reduction. The confidence variance is used to determine the optimal dimensionality reduction level.
[0034] The data preprocessing module performs multi-scale adaptive smoothing filtering as follows: The size of the smoothing filter window for each pixel is adaptively determined based on the maximum allowable dimensionality reduction and local features:
[0035] in, Indicates adaptive window size. This refers to the number of gray levels contained in each grayscale level under the maximum allowable dimensionality reduction. This represents the grayscale value of a pixel.
[0036] The dual-branch feature extraction network, when performing feature extraction, specifically includes two-dimensional visual feature extraction and three-dimensional shape feature extraction. The two-dimensional visual feature extraction method is as follows: Based on the improved YOLOv5n architecture, the original CBS downsampling module is replaced with the ADown module in the backbone and neck network, which reduces the number of model parameters and improves computational efficiency. The ADown module first reduces the spatial dimension of the input feature map through the average pooling layer, then divides the feature map into two equal parts, processes them differently, and finally merges the feature maps.
[0037] The dual-branch feature extraction network extracts 3D shape features in the following way: A deformable convolutional network (DCNv2) is used to extract 3D surface topography features, and the ability to model irregularly shaped defects is enhanced by learningable sampling points. The calculation formula for deformable convolution is as follows:
[0038] Where p represents spatial location. The kernel size is [size]. and These represent the learnable offset and modulation coefficient, respectively.
[0039] The fusion module operates as follows:
[0040] Where q is the query vector. A lightweight offset generation network. Where M is the spatial offset and M is the number of attention heads. Attention weights; The fusion module adopts a phased fusion strategy, performing feature fusion at the shallow, middle and deep layers respectively, taking into account both detailed and semantic information.
[0041] The defect identification and quantification module performs defect detection, classification, and quantification analysis in the following ways: Defect detection and classification: Adaptive anchor boxes are used, and the width and height of the labeled boxes in the training set are clustered using the K-means++ algorithm to generate initial anchor box sizes that are more suitable for the specific dataset. Defect Quantification Analysis: Accurately quantifies detected defects, including automatic measurement of defect area, depth, and length parameters.
[0042] Example 2: A method for detecting surface defects in a metal plate, comprising the following steps:
[0043] S1: Simultaneously acquires high-resolution RGB images, 3D line laser scanning data, and infrared thermal imaging data; employs a combined mechanism of synchronous triggering hard synchronization and timestamp-aligned soft synchronization to ensure that data of different modalities are aligned in time and space; S2: Standardize, denoise, and enhance the data for each modality; employ a nonlinear denoising algorithm based on singular value decomposition for ultrasound data; and apply multi-scale adaptive smoothing filtering and other techniques to image data to improve the signal-to-noise ratio. S3: Parallel extraction of two-dimensional visual features and three-dimensional morphological features; S4: Deeply fuse the extracted two-dimensional and three-dimensional features, and dynamically adjust the attention focus based on the deformable attention mechanism to achieve adaptive weighted fusion between features of different modalities; S5: Decode the fused features and output the defect category, location, size, and depth information; use an adaptive anchor frame mechanism to optimize the positioning accuracy, and finally generate a comprehensive inspection report containing various parameters of the defect.
[0044] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for detecting surface defects in a metal plate, characterized in that, This is based on a metal plate surface defect detection system, which includes: A multimodal data acquisition module, wherein the multimodal data acquisition module acquires multidimensional data from the surface of the metal plate; A data preprocessing module, which performs standardization and enhancement on the collected raw data; The feature extraction module extracts feature representations of different modalities; The fusion module achieves deep fusion of two-dimensional visual features and three-dimensional morphological features by introducing deformable attention. The defect identification and quantification module performs defect detection, classification, and quantification analysis based on the fused features.
2. The method for detecting surface defects in a metal plate according to claim 1, characterized in that, The multimodal data acquisition module acquires data including: High-resolution color images are used to acquire surface texture, color features, and two-dimensional morphological information; 3D line laser scanning data, through laser triangulation, obtains three-dimensional surface topography data, accurately characterizing the depth features of depressions and protrusions; Infrared thermal imaging data is used to identify internal defects and microcracks by utilizing differences in surface thermal conductivity.
3. The method for detecting surface defects in a metal plate according to claim 2, characterized in that, The multimodal data acquisition module, based on an adaptive calibration algorithm using feature points, achieves spatiotemporal alignment of various data types, as detailed below: in, Represents the optimal transformation matrix. and Let represent the matching feature points in the RGB image and the depth image, respectively, and T be the transformation matrix to be solved.
4. The method for detecting surface defects in a metal plate according to claim 1, characterized in that, The data preprocessing module performs standardization and enhancement through adaptive gray-scale run-length matrix feature extraction and multi-scale adaptive smoothing filtering. The adaptive gray-scale run-length matrix feature extraction is specifically as follows: Perform continuous gray-level dimensionality reduction on a grayscale image, obtain the gray-level run-length matrix of the image after each dimensionality reduction, calculate the dimensionality reduction confidence based on this matrix, and determine the maximum allowable degree of dimensionality reduction. in, This represents the sum of the elements of the gray-level run-length matrix after the k-th dimensionality reduction. Indicates the credibility of dimensionality reduction. The confidence variance is used to determine the optimal dimensionality reduction level.
5. The method for detecting surface defects in a metal plate according to claim 4, characterized in that, The data preprocessing module performs multi-scale adaptive smoothing filtering as follows: The size of the smoothing filter window for each pixel is adaptively determined based on the maximum allowable dimensionality reduction and local features: in, Indicates adaptive window size. This refers to the number of gray levels contained in each grayscale level under the maximum allowable dimensionality reduction. This represents the grayscale value of a pixel.
6. The method for detecting surface defects in a metal plate according to claim 1, characterized in that, The dual-branch feature extraction network, when performing feature extraction, specifically includes two-dimensional visual feature extraction and three-dimensional shape feature extraction. The two-dimensional visual feature extraction method is as follows: Based on the improved YOLOv5n architecture, the original CBS downsampling module is replaced with the ADown module in the backbone and neck network, which reduces the number of model parameters and improves computational efficiency. The ADown module first reduces the spatial dimension of the input feature map through the average pooling layer, then divides the feature map into two equal parts, processes them differently, and finally merges the feature maps.
7. The method for detecting surface defects in a metal plate according to claim 6, characterized in that, The dual-branch feature extraction network extracts 3D shape features in the following way: A deformable convolutional network is used to extract 3D surface topography features, and the ability to model irregularly shaped defects is enhanced by learningable sampling points. The calculation formula for deformable convolution is as follows: Where p represents spatial location. The kernel size is [size]. and These represent the learnable offset and modulation coefficient, respectively.
8. The method for detecting surface defects in a metal plate according to claim 1, characterized in that, The fusion module operates as follows: Where q is the query vector. A lightweight offset generation network. Where M is the spatial offset and M is the number of attention heads. Attention weights; The fusion module adopts a phased fusion strategy, performing feature fusion at the shallow, middle and deep layers respectively, taking into account both detailed and semantic information.
9. The method for detecting surface defects in a metal plate according to claim 1, characterized in that, The defect identification and quantification module performs defect detection, classification, and quantification analysis in the following ways: Defect detection and classification: Adaptive anchor boxes are used, and the width and height of the labeled boxes in the training set are clustered using the K-means++ algorithm to generate initial anchor box sizes that are more suitable for the specific dataset. Defect Quantification Analysis: Accurately quantifies detected defects, including automatic measurement of defect area, depth, and length parameters.
10. A method for detecting surface defects in a metal plate according to any one of claims 1-9, characterized in that, Includes the following steps: S1: Simultaneously acquires high-resolution RGB images, 3D line laser scanning data, and infrared thermal imaging data; employs a combined mechanism of synchronous triggering hard synchronization and timestamp-aligned soft synchronization to ensure that data of different modalities are aligned in time and space; S2: Standardize, denoise, and enhance the data for each modality; employ a nonlinear denoising algorithm based on singular value decomposition for ultrasound data; and apply multi-scale adaptive smoothing filtering and other techniques to image data to improve the signal-to-noise ratio. S3: Parallel extraction of two-dimensional visual features and three-dimensional morphological features; S4: Deeply fuse the extracted two-dimensional and three-dimensional features, and dynamically adjust the attention focus based on the deformable attention mechanism to achieve adaptive weighted fusion between features of different modalities; S5: Decode the fused features and output the defect category, location, size, and depth information; use an adaptive anchor frame mechanism to optimize the positioning accuracy, and finally generate a comprehensive inspection report containing various parameters of the defect.