Metal defect detection method based on abnormal prior guidance and context awareness

Abstract

The invention discloses a metal defect detection method based on anomaly prior guidance and context perception, which comprises the following steps of: carrying out unsupervised learning on a normal image by utilizing a parallel lightweight anomaly prior module to ensure that the normal image masters a distribution mode of normal metal textures; and calculating a pixel-by-pixel reconstruction error of any input image, generating an anomaly thermodynamic diagram capable of representing an anomaly probability, taking the anomaly thermodynamic diagram as a spatial attention guidance signal, and deeply fusing the anomaly thermodynamic diagram into a feature fusion neck of a YOLO detection network to enhance feature response of the network to a potential defect area. And finally, taking the accurate position and category of the metal defect as output of supervised learning, and training an end-to-end defect detection model guided by abnormal priori knowledge. According to the method, various defects in the metal image can be accurately positioned and recognized only by inputting the metal image to be detected, and particularly, tiny and low-contrast hidden defects which are difficult to find by a traditional method can be accurately positioned and recognized.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, specifically to a method for detecting metal defects based on anomaly prior guidance and context awareness. Background Technology

[0002] The surface quality of metal components is crucial to the performance and safety of products in high-end manufacturing. Efficient and accurate detection of surface defects is a core aspect of industrial quality control. Traditional manual visual inspection is inefficient and prone to errors, while automated inspection (AVI) methods based on traditional image processing have poor robustness to complex textures and lighting variations, making it difficult to meet high standards.

[0003] Existing technology 1 (CN115797326A) provides a method for detecting surface defects of metal castings based on an improved YOLOX-S algorithm. The fundamental limitation of this method is that it can only identify defects that have been learned during the training phase. When there are insufficient defect samples or weak features, the model is very prone to missing detections.

[0004] Prior art 2 (CN111292305A) discloses an improved YOLO-V3 method for detecting defects on metal processing surfaces. In the backbone network, a pooling-free layer is used and the downsampling factor is smaller than that of Darknet-53 to retain more shallow features in order to detect small targets. However, it is still essentially within the scope of supervised learning and fails to fundamentally solve the problem that weak feature signals are easily submerged by background texture.

[0005] Prior art 3 (CN116977294A) discloses a method and device for detecting metal surface defects based on an improved YOLO model. It optimizes the feature extraction and fusion capabilities of the YOLO model by designing reparameterized DSC and CSP structures, but its detection logic is still end-to-end supervised learning, that is, directly mapping from the input image to the defect location.

[0006] In summary, while deep learning object detection algorithms, represented by YOLO, have made progress, they still face significant challenges in the field of metal defect detection. This is primarily due to their insufficient sensitivity to small, low-contrast defects; weak defect features are easily lost after multi-layer downsampling, leading to missed detections. Secondly, deep learning methods heavily rely on massive and balanced defect annotation data. However, in industrial settings, defect samples, especially rare defects, are scarce, making the acquisition of large-scale annotation data extremely costly and limiting the model's generalization ability. Furthermore, existing schemes that integrate anomaly detection employ a two-stage approach of anomaly detection followed by localization. This fragmented process and sequential reasoning result in poor real-time performance, and the anomaly information only serves as a rough guide, failing to deeply integrate with the features of the detection network, thus limiting accuracy improvement. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention proposes a metal defect detection method based on anomaly prior guidance and context awareness. An anomaly prior module generates an anomaly heatmap in real time and injects it as a spatial attention signal into the neck feature fusion process of the detection network. This utilizes normal texture knowledge to guide the detection network to actively focus on small, hidden anomaly regions, significantly improving the detection accuracy of various metal defects while maintaining efficient end-to-end detection.

[0008] A metal defect detection method based on anomaly prior guidance and context awareness, the specific steps of which are as follows: Step 1. Acquire and preprocess metal surface images, and divide them into defect-free datasets based on whether defects are present on the metal surface in the images. and defective datasets .

[0009] Step 2. Construct a convolutional autoencoder to reconstruct the input image using a defect-free dataset. Offline training was performed, and all parameters were fixed after training to serve as the anomaly prior module (APM).

[0010] Step 3. Construct an anomaly prior-guided target detection network, including a backbone network, a neck fusion network, and a detection head network.

[0011] The backbone network is used to extract features from the input image and output effective feature maps of different scales from different stages.

[0012] The neck fusion network first reconstructs the input image using the Anomaly Prior Module (APM), then calculates the error between the input and reconstructed images to generate a global anomaly heatmap. Next, it generates an attention map corresponding to the size of the effective feature map through downsampling, and spatial attention weights are applied to the effective feature maps at different scales. Then, a two-stage fusion process (top-down and bottom-up) is performed using the path aggregation network PANet to output a temporary feature map. Finally, the attention map is used to spatially weight the temporary feature maps at different scales to obtain the detection feature map.

[0013] The detection head network is used to regress the defect location coordinates, confidence level, and category label in the detection feature map.

[0014] Step 4. Using a defective dataset Train the target detection network guided by anomaly priors.

[0015] Step 6. Deploy the trained target detection network to the industrial site to perform defect detection on the metal surface image frames captured by the camera.

[0016] The present invention has the following beneficial effects: This invention proposes a unique end-to-end detection framework based on anomaly prior guidance. It utilizes an anomaly prior module (APM) trained unsupervised with a large number of readily available normal samples, combined with an innovative two-stage (1+M) residual-enhanced attention injection mechanism. This significantly improves the sensitivity for detecting small, low-contrast, and hidden defects while greatly reducing reliance on defect samples that are difficult to collect and label. This design not only endows the model with good generalization ability for detecting unknown or rare defect types but also integrates anomaly guidance and defect detection into a highly efficient end-to-end parallel network. This avoids the poor real-time performance and accuracy bottlenecks caused by process fragmentation and information loss in traditional two-stage schemes, thus making it more practical for industrial applications. Attached Figure Description

[0017] Figure 1 This is a flowchart of a metal defect detection method based on anomaly prior guidance and context awareness.

[0018] Figure 2 The image shown is the original image captured in this example.

[0019] Figure 3 This is a schematic diagram of the Anomaly Priority Module (APM).

[0020] Figure 4 This is a schematic diagram of the neck fusion network structure.

[0021] Figure 5 This is a schematic diagram of residual-type spatial attention enhancement.

[0022] Figure 6 The results of the defect-free detection are shown in the example.

[0023] Figure 7 This is a diagram showing the detection results for non-conductive defects.

[0024] Figure 8 This is a diagram showing the results of the scratch-type defect detection.

[0025] Figure 9 This is a diagram showing the results of the electrical defect detection for dirty points.

[0026] Figure 10 This is a diagram showing the results of the impact damage defect detection.

[0027] Figure 11 This is a diagram showing the results of defect detection for convex powder type. Detailed Implementation

[0028] The present invention will be further explained below with reference to the accompanying drawings; A metal defect detection method based on anomaly prior guidance and context awareness, such as Figure 1 As shown, the specific steps are as follows: Step 1. Acquire and preprocess images of metal surfaces A line scan industrial camera is vertically deployed above the metal sheet production line, 300mm above the sheet surface. To eliminate surface glare and uniformly illuminate the texture, an LED strip light source coaxial with the lens is used. As the sheet passes through the inspection station, continuous image acquisition is performed at a line frequency of 1000Hz, capturing the original images. like Figure 2 As shown.

[0029] High-resolution original image Non-overlapping shearing sub-image of pixels and sub-image The pixel values ​​are linearly scaled from [0, 255] to the floating-point range of [0, 1] as the input image. .

[0030] Filtering from preprocessed images The metal surface has no visible defects. Images, constructing a defect-free dataset .

[0031] Then filter from the preprocessed images The sheet contains four different types of defects: micro-scratch, low-contrast pit, hidden rust, and large patches. Images were collected, and professional quality inspectors used annotation software to label the location and category of all defects, thus constructing a defective dataset. .

[0032] Step 2. Construct and train the anomaly prior module offline. A lightweight reconstruction network is trained using unsupervised learning, enabling it to specifically learn from defect-free datasets. The distribution pattern of normal metal surface texture is used to reconstruct normal texture, which serves as the anomalous prior module (APM).

[0033] like Figure 3 As shown, the reconstructed network is designed as a lightweight convolutional autoencoder, which includes four convolutional layers and a negative slope. The LeakyReLU activation function is used for downsampling and feature compression, outputting latent features. The decoder structure is symmetrical to the encoder, consisting of four transposed convolutional layers and a sigmoid activation function, used for upsampling and image reconstruction, with the reconstructed image constrained to the range [0, 1].

[0034] Specifically, the encoder has 32, 64, 128, and 256 convolutional kernels in its four convolutional layers, respectively, and the kernel size is [missing information]. With a step size of 2 and padding of 1, output latent features. The size is 1 / 16 of the input image. The number of convolution kernels in the transposed convolutional layer of the decoder is the same as that in the convolutional layer of the same level in the encoder.

[0035] Mean squared error is used as the reconstruction loss. Minimize the input image Rather than reconstructing the image Differences between them:

[0036] in, These are the input images. Height, width, and number of channels; It is the first Zhang Input Image exist The value at; It is to reconstruct the image exist The value at that point. Using the Adam optimizer, set the learning rate to... , , The batch size is set to 16. The number of training epochs is set to 200.

[0037] After training, all network parameters of the reconstructed network are fixed and saved as the apm_frozen.pth weight file, which is used as the Anomalous Prior Module (APM) in subsequent steps.

[0038] Step 3. Construct an anomaly prior-guided detection network Based on the YOLO series network architecture, an anomaly prior-guided target detection network AP-YOLO is constructed, which includes a backbone network, a neck fusion network, and a detection head network.

[0039] The backbone network uses CSPDarknet53 as the main feature extraction network for the input image. The CSPDarknet53 outputs effective feature maps at three different scales from the 3rd, 4th, and 5th stages of the network. , , The sizes are as follows: , , .

[0040] like Figure 4 As shown, the neck fusion network uses the path aggregation network PANet for a two-stage fusion process, both top-down and bottom-up. It also loads the anomalous prior module APM obtained in step two, guiding the fusion process of the path aggregation network PANet through residual spatial attention enhancement operations. Specifically: Input image In the Synchronous Input Anomaly Prior Module (APM), the input image is calculated. Reconstructed image with output The original error map for a single channel is obtained by squared L2 norm differences (SSD) between pixels. Then to Perform min-max normalization to constrain its range to Generate a global anomaly heatmap within the specified range. .

[0041] Then, adaptive average pooling is used to analyze the global anomaly heatmap. Perform downsampling to obtain three scales and feature maps. , , Attention maps with the same spatial resolution , , .

[0042] Define an anomalous prior-guided residual-enhanced spatial attention operation, such as... Figure 5 As shown, using attention map M j For feature maps C of the same scale j Perform residual spatial attention enhancement:

[0043] Where j = 3, 4, 5, This indicates element-wise multiplication. This represents the output feature map after attention-guided processing. During the attention enhancement process, Design is a key improvement in this method. When attention maps... When the pixel value is close to 0, Feature map C j The basic information is preserved, ensuring the integrity of normal contextual information; when the attention graph... When the pixel value is close to 1, Feature map C jThe response is significantly amplified, forcing the network to focus on it, thus improving upon traditional attention enhancement methods. The problem is the suppression of normal areas.

[0044] Utilizing the path aggregation network PANet to process attention-guided output feature maps , , Perform a two-stage fusion process, combining top-down and bottom-up approaches, to output a temporary feature map. , , Using attention maps , , After further spatial attention enhancement, the output detection feature map is obtained. , , :

[0045]

[0046]

[0047]

[0048] in, This represents the C3 module structure in YOLOv5. Indicates feature splicing, , These represent downsampling and upsampling operations, respectively. Indicates use The CBL module for convolution.

[0049] In the neck fusion network, a two-stage anomaly prior guidance is applied to the input and output of the path aggregation network PANet to ensure that the focus on anomalous regions is not diluted after multi-layer fusion. This ensures that the final output detection feature map is a context-aware feature map that is guided by a two-stage anomaly prior and is highly focused on the anomaly region.

[0050] The detection head network uses the standard decoupled detection head of YOLOv5, and the detection head receives three-scale detection feature maps output from the neck fusion network. , , It predicts bounding boxes, confidence levels, and categories.

[0051] Step 4. End-to-end training of the ensemble detection network Using the defective dataset built in step 1 Train the object detection network AP-YOLO and set the loss function. for:

[0052] in, To determine the loss, CIoU (Complete Intersection over Union) Loss is used. For confidence loss, binary cross-entropy (BCE) loss is used; For classification loss, binary cross-entropy (BCE) loss is used, with weights... , , Set them to 0.05, 1.0, and 0.5 respectively.

[0053] Using the AdamW optimizer, the initial learning rate is set to... The weight decay is set to 0.0005, and the optimizer momentum is... The learning rate was set to 0.937. A cosine annealing strategy was used to dynamically adjust the learning rate. The batch size was set to 16, and the total number of training epochs was set to 300. After training, the integrated AP-YOLO model weight file apyolo.pth was saved.

[0054] Step 5. Deploy and execute metal defect detection The trained object detection network AP-YOLO was deployed in an industrial field to process single-frame images captured by a camera. Perform cropping and normalization operations to obtain the detection image. , detect image The input to the backbone network CSPDarknet53 and the anomaly prior module APM of the AP-YOLO object detection network is decoded from the output of the detection head to obtain a tensor containing thousands of raw prediction boxes.

[0055] Non-maximum suppression (NMS) is performed on the original predicted bounding box tensor to obtain the final defect detection result. The defect detection results It is a list containing zero or more detection boxes, each containing the location coordinates of the defect, confidence level, and defect category.

[0056] Defect detection results All detection boxes are superimposed on the detection image. The above information is available for on-site personnel to view in real time, such as... Figures 6-11The figures show examples of defect detection results for the following categories: no defects, non-conductive defects, scratches, dirt, dents, and raised powder defects. To quantitatively evaluate the effectiveness and superiority of this method, a series of comparative experiments were conducted based on the proprietary high-precision cold-rolled steel sheet dataset constructed using the method described in step one.

[0057] The comparison models selected were the current mainstream object detection models Faster R-CNN, YOLOv5-L, and YOLOX-S. All models were trained and tested under the same hardware and software environment. During training, the input image size was uniformly set. .

[0058] The standard mean accuracy (mAP@0.5) was used as the primary evaluation metric. Additionally, to focus on the model's ability to detect difficult-to-detect defects, the mean accuracy (AP) was specifically calculated for the two most challenging categories: "micro-scratch" and "low-contrast pit." The performance comparison data of each model on the test set of the self-built dataset is shown in Table 1 below: Table 1 method mAP@0.5 AP_micro-scratch AP_low-contrast-pit Faster R-CNN 75.3 68.1 65.4 YOLOv5-L 78.8 72.5 70.8 YOLOX-S 78.1 71.9 70.2 This invention 84.2 81.6 79.5 As can be seen, our method achieved an mAP of 84.2% at 0.5, which is 5.4 percentage points higher than the best-performing comparison model, YOLOv5-L, demonstrating a significant advantage. This proves that our method exhibits excellent robustness and accuracy in handling complex scenarios with severe imbalance between normal and defective samples and weak defect features.

[0059] Furthermore, for the detection of "minor scratches" and "low-contrast pits," the AP values ​​of this method reached 81.6% and 79.5%, respectively, representing significant improvements of 9.1 and 8.7 percentage points compared to YOLOv5-L. This demonstrates that when purely supervised learning models struggle to establish effective mappings due to weak features, this method, through its proposed Anomaly Prior Module (APM), can utilize the deviation from normal patterns to provide crucial and decisive discriminative information for the detection model. Simultaneously, the anomaly heatmap, through a (1+M) residual-enhanced attention mechanism, retrieves and amplifies these almost submerged defect signals from the complex background texture, enabling the detection model to accurately capture them and achieving a qualitative leap in detection performance.

[0060] In summary, this method overcomes the detection bottleneck of existing technologies when dealing with small, low-contrast defects, and is a high-precision, robust, and more industrially practical solution for detecting metal surface defects.

Claims

1. A metal defect detection method based on anomaly prior guidance and context awareness, characterized in that: The specific steps are as follows: Step 1. Divide the metal surface images into defect-free datasets based on whether there are defects on the metal surface in the images. and defective datasets ; Step 2. Construct an image reconstruction network using a defect-free dataset. Perform offline training, and after training is completed, fix all parameters as the anomaly prior module (APM). Step 3. Based on the YOLO series model structure, construct an anomaly prior-guided target detection network; The neck fusion network of the target detection network first reconstructs the input image through the anomaly prior module (APM), then calculates the error between the input image and the reconstructed image to generate a global anomaly heatmap. Next, it generates an attention map corresponding to the size of the multi-scale effective feature map output by the backbone network through downsampling. It then performs spatial attention weighting on the effective feature maps of different scales, and then performs two-stage fusion from top to bottom and bottom to top through the path aggregation network (PANet) to output a temporary feature map. Finally, it uses the attention map to perform spatial attention weighting on the temporary feature maps of different scales to obtain the detection feature map, which is then input into the detection head for regression prediction. Step 4. Using a defective dataset Train a target detection network guided by anomaly priors; Step 6. Use the trained target detection network to detect defects in the metal surface image.

2. The metal defect detection method based on anomaly prior guidance and context awareness as described in claim 1, characterized in that: The defective dataset In this context, the types of defects on metal surfaces include micro-scratches, low-contrast pits, hidden corrosion, and large-area patches.

3. The metal defect detection method based on anomaly prior guidance and context awareness as described in claim 1, characterized in that: The image reconstruction network is a lightweight convolutional autoencoder, which includes four convolutional layers and a LeakyReLU activation function for downsampling and feature compression, outputting latent features. ; The decoder structure is symmetrical to the encoder, consisting of four transposed convolutional layers and a sigmoid activation function.

4. The metal defect detection method based on anomaly prior guidance and context awareness as described in claim 3, characterized in that: The encoder has four convolutional layers with 32, 64, 128, and 256 convolutional kernels, respectively, and the kernel size is [missing information]. With a step size of 2 and padding of 1, output latent features. The size is 1 / 16 of the input image; The number of convolution kernels in the transposed convolutional layer of the decoder is the same as that in the convolutional layer of the same level in the encoder.

5. The metal defect detection method based on anomaly prior guidance and context awareness as described in claim 3, characterized in that: Mean squared error is used as the reconstruction loss. Train the image reconstruction network to minimize the input image. Rather than reconstructing the image Differences between them: in, These are the input images. Height, width, and number of channels; It is the first Zhang Input Image exist The value at; It is to reconstruct the image exist The value at the specified location; N represents the number of input images.

6. The metal defect detection method based on anomaly prior guidance and context awareness as described in claim 1, characterized in that: The backbone network uses CSPDarknet53 as the main feature extraction network for the input image. Output effective feature maps at three different scales from stages 3, 4, and 5. , , .

7. The metal defect detection method based on anomaly prior guidance and context awareness as described in claim 1, characterized in that: Calculate the pixel-wise L2 norm squared between the input image and the reconstructed image, then perform min-max normalization to generate a global anomaly heatmap, and finally use adaptive average pooling for downsampling to obtain an attention map with the same spatial resolution as the effective feature map. , , .

8. The metal defect detection method based on anomaly prior guidance and context awareness as described in claim 1, characterized in that: The spatial attention weighting method is as follows: Where M represents the attention map and C represents the feature map to be weighted. This indicates element-wise multiplication. This represents the output feature map after spatial attention weighting.

9. The metal defect detection method based on anomaly prior guidance and context awareness as described in claim 1, characterized in that: The detection head network uses the standard decoupled detection head of YOLOv5 to predict the bounding box, confidence level, and category of the defect location in the detection feature map.

10. The metal defect detection method based on anomaly prior guidance and context awareness as described in claim 1, characterized in that: Set the loss function Training a target detection network guided by anomaly priors: in, To pinpoint the loss, For confidence loss, For classifying losses, , , These are the corresponding weights.

Images