An industrial defect detection system based on iterative feedback optimization

Abstract

The present invention discloses an industrial defect detection system based on iterative feedback optimization, comprising a backbone network, an iteration module, a feedback optimization module, and a spatial attention module (SAM). The backbone network extracts feature maps at four scales, and feeds the feature maps at the second, third, and fourth scales into the iteration module, while the feature map at the first scale is fed into the SAM. The iteration module performs multiple rounds of iterative processing to obtain multiple iteration results, which are then fed into the feedback optimization module and the SAM. The feedback optimization module monitors the iteration effect using a loss function based on the multiple iteration results. The SAM merges the feature map at the first scale and the result of the last iteration to obtain an industrial defect feature segmentation result map. This invention can reduce the demand for computing resources and quickly and accurately complete defect detection tasks on production lines.

Description

Technical Field

[0001] The present invention relates to the field of industrial defect detection systems, and in particular to an industrial defect detection system based on iterative feedback optimization. Background Art

[0002] Current industrial defect detection methods can be divided into traditional industrial defect detection and automatic industrial defect detection based on deep learning technology.

[0003] Traditional industrial defect detection consists of manual inspection and traditional machine vision methods. Although manual visual inspection has certain flexibility and adaptability, it has significant limitations. Manual inspection is slow and cannot meet the real-time monitoring needs of large-scale, high-speed production lines. It is also prone to missed detections and misjudgments. At the same time, personal judgment standards are greatly influenced by subjectivity, which is not conducive to the accuracy and consistency of detection results. Traditional rule-based machine vision methods mainly rely on the basic principles of computer vision, image processing technology, and predefined rules and algorithms. They show good stability and controllability in defect detection under specific conditions, but they generally lack generalization capabilities and are not effective in identifying defects that have never been seen or in complex backgrounds. The programming process is usually complex, making it difficult to cope with product design changes or the emergence of new defect types.

[0004] Deep learning-based industrial defect detection technology is an advanced method developed in the context of modern industrial automation and quality control. It combines cutting-edge research in computer vision, machine learning, and particularly deep learning, to address the many challenges faced by traditional industrial defect detection methods.

[0005] Research on industrial defect detection methods based on deep learning offers broad technical advantages and research value. These methods boast high accuracy and adaptability, a high degree of automation, and robust continuous optimization capabilities. They are key technologies for achieving Industry 4.0 and digital transformation, and can bring significant economic benefits to enterprises. However, since the results of automated industrial defect detection directly impact production quality and efficiency, and production efficiency dictates the timeliness of the detection process—that is, the need for rapid and accurate judgments—the deep learning network models used must balance speed and accuracy. This is the primary focus of deep learning-based industrial defect detection research. However, most current deep learning-based industrial defect detection methods utilize models that stack numerous modules, resulting in complex structures and high computational complexity. This is detrimental to resource-constrained production environments. Summary of the Invention

[0006] The present invention provides an industrial defect detection system based on iterative feedback optimization to solve the problems of complex structure and high computational complexity of the model adopted by the existing industrial defect detection method based on deep learning.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] An industrial defect detection system based on iterative feedback optimization includes a backbone network, an iteration module, a feedback optimization module, and a spatial attention module (SAM). An industrial defect image to be detected is input into the backbone network, which extracts feature maps at a total of four scales. The feature maps at the second, third, and fourth scales are fed into the iteration module, and the feature map at the first scale is fed into the spatial attention module (SAM).

[0009] The iteration module performs multiple rounds of iterative processing to obtain multiple rounds of iterative results, and sends the multiple rounds of iterative results to the feedback optimization module respectively, and sends the last round of iterative results to the spatial attention module SAM. The process of each round of iterative processing is as follows:

[0010] In each round of iteration, the second-level feature map and the fourth-level feature map are first fused with the result of the previous iteration through different iterative feature fusion modules IFF. The fourth-level feature map fused with the result of the previous iteration is processed by an attention module SEM and then upsampled to the feature map of the third level to obtain a first spliced ​​feature map. The first spliced ​​feature map is then processed by another attention module SEM and then upsampled to the feature map of the second level fused with the result of the previous iteration to obtain a second spliced ​​feature map. The second spliced ​​feature map is finally processed by the third attention module SEM and then channel compressed to obtain the result of the current iteration.

[0011] The feedback optimization module applies a loss function to the obtained multiple rounds of iteration results, and supervises and feeds back the iterative effect of the iterative module based on the loss function calculation results;

[0012] The spatial attention module SAM fuses the feature map of the first level scale and the result of the last round of iteration to obtain the industrial defect feature segmentation result map.

[0013] Furthermore, the backbone network is a CSPDarknet network.

[0014] Furthermore, the iterative feature fusion module IFF in the iterative module resizes the input feature map of the corresponding level scale, upsamples the previous round of iteration results and the resized feature map of the corresponding level scale, splices them by channel, and then performs channel adjustment, thereby integrating the previous round of iteration results into the feature map of the corresponding level scale.

[0015] Furthermore, each attention module SEM in the iterative module compresses the spatial information of the input image through global average pooling and learns the importance weight of each channel through a fully connected layer.

[0016] Furthermore, the spatial attention module SAM first performs mean pooling and maximum pooling on the feature map of the first level scale, and performs mean pooling and maximum pooling on the results of the last round of iteration; then the spatial attention module SAM splices the mean pooling results and the maximum pooling results of the feature map of the first level scale and then performs convolution processing, and splices the mean pooling results and the maximum pooling results of the last round of iteration results and then performs convolution processing; then the spatial attention module SAM multiplies the feature map of the first level scale with the corresponding convolution processing result, and multiplies the last round of iteration results with the corresponding convolution processing result; finally, the spatial attention module SAM splices the two sets of multiplication results to obtain the industrial defect feature segmentation result map.

[0017] Inspired by the human cognitive review process of knowledge, this paper proposes an industrial defect detection system based on iterative feedback optimization. First, to address the problems of complex convolutional neural network model structure, severe module stacking, and high computational complexity, this paper chooses to use an iterative mechanism to reuse some modules to reduce the model size, and adds an attention module during feature fusion to focus on more important feature information; secondly, this paper applies a weighted loss function to the iterative results to provide feedback and supervise the iterative process, ensuring that the iterative results are optimized step by step; finally, in order to retain shallow spatial information and detailed features, this paper captures large target information with a larger receptive field, and fuses the underlying large-scale feature map with the iterative results, effectively improving the model's ability to learn features of different sizes.

[0018] Therefore, the present invention reuses some modules through an iterative mechanism, repeatedly analyzes and learns features of images, and continuously optimizes the results. While ensuring high performance, the size of the final neural network model is controlled not to be too large, thereby reducing the demand for computing resources and being able to quickly and accurately complete defect detection tasks on the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the overall structure of embodiment 1 of the present invention.

[0020] Figure 2 This is a schematic diagram of the CSP structure in Example 1 of the present invention.

[0021] Figure 3 This is a schematic diagram of the IFF structure in Example 1 of the present invention.

[0022] Figure 4 This is a schematic diagram of the SAM structure in Example 1 of the present invention.

[0023] Figure 5 This is a diagram of the detection results of IFRNet in the second embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described below with reference to the accompanying drawings and examples.

[0025] Example 1

[0026] like Figure 1 As shown, this embodiment discloses an industrial defect detection system based on iterative feedback optimization, named IFRNet, which includes a backbone network, an iteration module, a feedback optimization module, and a spatial attention module SAM. The input image is subjected to feature extraction by the backbone network to obtain feature maps of multiple levels and different scales as the input of the iteration module. The result after each round of iterative processing in the iteration module will be used as input again and fused with the original feature map for iterative processing, and the results of the iteration will be fed back to the supervision model through the feedback optimization module. After a total of four rounds of iteration, the iteration module generates four segmentation result maps as iterative results, all of which are input into the feedback optimization module for optimization processing. Finally, the spatial attention module SAM fuses the last round of iteration results and the feature map of the first level scale to obtain the final accurate industrial defect feature segmentation result map. The specific instructions are as follows:

[0027] (1) In this embodiment, the backbone network uses the open source CSPDarknet network to extract features from the input industrial defect detection image. The industrial defect detection image is extracted through the CSPDarknet network to obtain multi-level feature maps of different scales, namely, the first-level scale feature map X1, the second-level scale feature map X2, the third-level scale feature map X3, and the fourth-level scale feature map X4 for subsequent processing. The scales of the feature maps X1, X2, X3, and X4 are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image (i.e., the industrial defect detection image), respectively. The second-level, third-level, and fourth-level scale feature maps X2, X3, and X4 are fed into the iterative module, and the first-level scale feature map X1 is fed into the spatial attention module SAM.

[0028] The backbone network CSPDarknet reduces the problem of repeated calculation of feature maps by connecting the network part forward and cross-stage (CSP) network structure, significantly improving the efficiency of model operation. Figure 2 As shown in the figure, the cross-stage connection structure enhances feature flow by merging features at early stages, helping to alleviate the vanishing gradient problem common in deep learning and avoid feature redundancy. Each convolutional layer in CSP is followed by batch normalization and Silu activation functions, which improves the stability of the training process and facilitates the propagation of gradient flow.

[0029] (2) In this embodiment, the iteration module performs multiple rounds of iterative processing to obtain multiple rounds of iterative results, and respectively sends the multiple rounds of iterative results to the feedback optimization module, and sends the last round of iterative results to the spatial attention module SAM. The process of each round of iterative processing is as follows:

[0030] During each iteration of the iterative module, the feature map X2 of the second level scale is first fused with the result of the previous iteration through an iterative feature fusion module IFF, and the feature map X4 of the fourth level scale is fused with the result of the previous iteration through another iterative fusion module IFF, thereby obtaining the feature information of the iterative feedback of the previous iteration result before participating in the current iteration.

[0031] The fourth-level feature map X4, which incorporates the results of the previous iteration, is processed by an attention module SEM and then upsampled to the third-level feature map X3, which is then concatenated to form a first concatenated feature map. The first concatenated feature map is then processed by another attention module SEM and upsampled to the second-level feature map X2, which incorporates the results of the previous iteration, to form a second concatenated feature map. The second concatenated feature map is finally processed by a third attention module SEM and then channel compressed to form the result of the current iteration.

[0032] In this embodiment, the iteration module performs N=4 or four rounds of iterations, thereby obtaining the four-round iteration result Y i , i = 1, 2, 3, 4. The four-round iteration results Y1, Y2, Y3, and Y4 are respectively sent to the feedback optimization module, and the last round iteration result Y4 is sent to the spatial attention module SAM,

[0033] In this embodiment, Figure 3 As shown, each iterative feature fusion module IFF resizes the input feature map of the corresponding level scale to ensure that it is consistent with the image size of the previous round of iteration results. Then, the neighboring interpolation method is used to upsample the previous round of iteration results and the resized feature maps of the corresponding level scale and concat them by channel. In the first round of iteration, since there is no previous round of iteration results, the previous round of iteration results take null values ​​in the first round of iteration. Then, in order to enhance the selectivity and expression ability of the concat spliced ​​features, the concat spliced ​​features are input to the convolution layer for channel adjustment, thereby integrating the previous round of iteration results into the feature map of the corresponding level scale. The data processing process of the iterative feature fusion module IFF in the i-th round of iteration is shown in the following formula:

[0034] IFF(X k ,Y i-1 )=Conv(Concat(UP(X k ),Y i-1 ))

[0035] Among them, X k is the feature map of the k-th scale (k=2, 4); Y i-1 represents the result of the i-1th iteration (i=1, 2, 3); Conv represents the convolutional layer.

[0036] In this embodiment, each attention module (SEM) compresses the spatial information of the input image through global average pooling and learns the importance weights of each channel through two fully connected layers. This adaptive learning enables the network to pay more attention to feature channels that are useful for the current task and ignore unimportant information, thereby improving feature utilization and the model's discriminative power.

[0037] (3) In this embodiment, a loss function is applied to the obtained results of multiple rounds of iterations in the feedback optimization module, and the iterative effect of the iterative module is supervised and fed back based on the loss function calculation results. In this way, the data flow of each iteration can be fed back, and each feedback feature can be linked to the segmentation result to avoid feature damage caused by the iterative structure.

[0038] In the feedback optimization module, the loss function used for each iteration result of the iterative module is the open source CIOU loss function. Then the feedback optimization module assigns different weights to each iteration result according to the iteration round. The iterative feedback loss function is as follows:

[0039]

[0040] Where ω = 0.2 is the weight coefficient, and i is the iteration round.

[0041] (4) In this embodiment, Figure 4 As shown in the figure, the spatial attention module SAM fuses the first-level scale feature map X1 and the last round of iteration result Y4 to obtain the industrial defect feature prediction result.

[0042] Specifically, the spatial attention module SAM has two processing channels, which are used to process the feature map of the first level scale and the results of the last round of iterations respectively. In the first processing channel, the feature map of the first level scale is firstly subjected to mean pooling and maximum pooling, and then the mean pooling result and the maximum pooling result of the feature map of the first level scale are spliced ​​and then convolution processing is performed, and then the feature map of the first level scale is multiplied by the corresponding convolution processing result to obtain the multiplication result. In the second processing channel, the results of the last round of iterations are firstly subjected to mean pooling and maximum pooling, and then the mean pooling result and the maximum pooling result of the last round of iterations are spliced ​​and then convolution processing is performed, and then the results of the last round of iterations are multiplied by the corresponding convolution processing result to obtain the multiplication result. Finally, the spatial attention module SAM splices the multiplication results obtained from the two processing channels to obtain the industrial defect feature prediction results.

[0043] Example 2

[0044] The following is a comparative experimental example of the industrial defect detection system IFRNet based on iterative feedback optimization described in Example 1 with existing FCN, SSD, Mask-rcnn, U-Net, U-Net++, SegNet, and Yolov8-seg. The details are as follows:

[0045] (1) Experimental setup

[0046] In this experiment, an NVIDIA GTX 3090 GPU was used, and the model code was built using Python and the PyTorch framework. The input image size was set to 640×640, and the AdamW optimizer was used. The initial learning rate was approximately 0.01, and training was performed using the Early Stopping strategy. Due to the small dataset size, a total of 200 iterations were used during training.

[0047] (2) Dataset and evaluation metrics

[0048] This experiment uses the CDS2k dataset, which contains 2,492 samples from multiple industrial product databases. The dataset consists of 1,330 positive samples and 1,162 negative samples. The defect areas and backgrounds exhibit similar patterns, demonstrating hidden characteristics. The evaluation metrics used were precision, recall, and mAP50. Figure 5 Detection results of some images of IFRNet.

[0049] In this experiment, the CDS2k dataset was divided into a training set, a validation set, and a test set in a ratio of 6:2:2. The training set was fed into the IFRNet described in Example 1 to calculate the loss error and train the model parameters using the backpropagation algorithm. The validation set was fed into the trained IFRNet to verify the IFRNet and initially test its segmentation performance. The test set data was then fed into the trained IFRNet to test its accuracy, thus completing the training of the IFRNet described in Example 1. Figure 5 This is a diagram of the detection results of IFRNet described in Example 1.

[0050] At the same time, this experimental example inputs the above training set, validation set, and test set into other prediction models, namely FCN, SSD, Mask-Rcnn, U-Net, U-Net++, SegNet, and Yolov8-seg, for comparative experiments.

[0051] (3) Experimental performance comparison

[0052] By statistically analyzing the data from the comparative experiments, we compared the detection accuracy of the IFRNet described in Example 1, as well as FCN, SSD, Mask-RCNN, U-Net, U-Net++, SegNet, and Yolov8-seg. Table 1 shows the comparative experimental results of the eight models in this experimental example. It can be seen that the IFRNet described in Example 1 outperformed the other models in all indicators on the CDS2k dataset, achieving good detection results. Table 1 is as follows:

[0053] Table 1. Comparative experimental results of 8 models

[0054]

[0055] according to Figure 5 As can be seen from Table 1, the IFRNet described in Example 1 can achieve good defect detection results in industrial defect detection in different scenarios, and has better results than all comparison methods in various indicators.

[0056] The proposed IFRNet described in Example 1 uses CSPDarknet as the backbone network, designs an iterative mechanism to process feature maps, and forms an effective feedback optimization mechanism to ensure the effectiveness of iteration. A channel attention mechanism is added to ensure the effective acquisition of information during feature fusion. The model also finally fuses low-level large-scale feature maps to ensure that shallow-level spatial features and detail information are not lost. This experimental example evaluates the IFRNet described in Example 1 on the CDS2k dataset and conducts comparative experiments with multiple segmentation models. The comprehensive test results and comparative test results show that the IFRNet described in Example 1 has a good accuracy rate and can reach the state-of-the-art level in industrial defect detection tasks.

[0057] The preferred embodiments of the present invention are described in detail above with reference to the accompanying drawings. The embodiments described in the present invention are merely descriptions of the preferred embodiments of the present invention and do not limit the concept and scope of the present invention. The various specific technical features described in the above specific embodiments can be combined in any suitable manner unless there is any contradiction. Such combinations should also be regarded as the contents disclosed in this disclosure as long as they do not violate the concept of the present invention. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0058] The present invention is not limited to the specific details of the above-mentioned embodiments. Within the scope of the technical concept of the present invention and without departing from the design concept of the present invention, various modifications and improvements made to the technical solution of the present invention by those skilled in the art should fall within the scope of protection of the present invention. The technical contents for which protection is sought in the present invention have been fully recorded in the claims.

Claims

1. An industrial defect detection system based on iterative feedback optimization, characterized in that: The system includes a backbone network, an iterative module, a feedback optimization module, and a spatial attention module (SAM). An image of an industrial defect to be detected is input into the backbone network, and the backbone network extracts a total of four-level feature maps. The feature maps of the second, third, and fourth levels are fed into the iterative module, and the feature map of the first level is fed into the spatial attention module (SAM). The iteration module performs multiple rounds of iterative processing to obtain multiple rounds of iterative results, and sends the multiple rounds of iterative results to the feedback optimization module respectively, and sends the last round of iterative results to the spatial attention module SAM. The process of each round of iterative processing is as follows: In each round of iteration, the second-level feature map and the fourth-level feature map are first fused with the result of the previous iteration through different iterative feature fusion modules IFF. The fourth-level feature map fused with the result of the previous iteration is processed by an attention module SEM and then upsampled to the feature map of the third level to obtain a first spliced ​​feature map. The first spliced ​​feature map is then processed by another attention module SEM and then upsampled to the feature map of the second level fused with the result of the previous iteration to obtain a second spliced ​​feature map. The second spliced ​​feature map is finally processed by the third attention module SEM and then channel compressed to obtain the result of the current iteration. The feedback optimization module applies a loss function to the obtained multiple rounds of iteration results, and supervises and feeds back the iterative effect of the iterative module based on the loss function calculation results; The spatial attention module SAM fuses the feature map of the first level scale and the result of the last round of iteration to obtain the industrial defect feature segmentation result map.

2. The industrial defect detection system based on iterative feedback optimization according to claim 1, characterized in that: The backbone network is the CSPDarknet network.

3. The industrial defect detection system based on iterative feedback optimization according to claim 1, characterized in that: The iterative feature fusion module IFF in the iterative module resizes the input feature map of the corresponding level scale, upsamples the previous iteration result and the resized feature map of the corresponding level scale, splices them by channel, and then performs channel adjustment to integrate the previous iteration result into the feature map of the corresponding level scale.

4. The industrial defect detection system based on iterative feedback optimization according to claim 1, characterized in that: Each attention module SEM in the iterative module compresses the spatial information of the input image through global average pooling and learns the importance weight of each channel through a fully connected layer.

5. The industrial defect detection system based on iterative feedback optimization according to claim 1, characterized in that: The spatial attention module SAM first performs mean pooling and maximum pooling on the feature map of the first level scale, and also performs mean pooling and maximum pooling on the result of the last round of iteration; Then the spatial attention module SAM concatenates the mean pooling result and the maximum pooling result of the first-level feature map and performs convolution processing, and concatenates the mean pooling result and the maximum pooling result of the last round of iteration and performs convolution processing; then the spatial attention module SAM multiplies the first-level feature map with the corresponding convolution processing result, and multiplies the last round of iteration result with the corresponding convolution processing result; finally, the spatial attention module SAM concatenates the two sets of multiplication results to obtain the industrial defect feature segmentation result map.

Images