Industrial defect detection system based on iterative feedback optimization

Through the combination of the iterative feedback optimization mechanism and attention module, the problem of complex structure and large calculations of deep learning models is solved, and fast and accurate industrial defect detection is achieved, which is suitable for production environments with limited resources.

CN118941548AActive Publication Date: 2024-11-12THE ACAD OF TIANJIN UNIV HEFEI

Patent Information

Application Number
CN202411142509.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-12
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

The existing industrial defect detection method based on deep learning has complex model structure and high computational volume, making it difficult to complete the detection task quickly and accurately in a production environment with limited resources.

Method used

Using an industrial defect detection system based on iterative feedback optimization, multiple rounds of iterative processing and feature fusion are carried out through backbone network, iterative module, feedback optimization module and spatial attention module, and the iterative effect is monitored in combination with loss function, reducing the size of the model and paying attention to the importance. Feature information.

Benefits of technology

It realizes the reduction of computing resource requirements while ensuring high performance, and quickly and accurately completes defect detection tasks on the production line, improving the model's learning ability of different size features.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941548A_ABST
    Figure CN118941548A_ABST
Patent Text Reader

Abstract

The invention discloses an industrial defect detection system based on iterative feedback optimization. The industrial defect detection system comprises a backbone network, an iteration module, a feedback optimization module and a space attention module SAM, the backbone network extracts four-level-scale feature maps, the second-level-scale feature map, the third-level-scale feature map and the fourth-level-scale feature map are sent to an iteration module, and the first-level-scale feature map is sent to a space attention module SAM; the iteration module carries out multiple rounds of iteration processing to obtain multiple rounds of iteration results, and sends the multiple rounds of iteration results to the feedback optimization module and the space attention module SAM; the feedback optimization module supervises a feedback iteration effect for a loss function based on a multi-round iteration result; and a space attention module SAM fuses the feature map of the first-level scale and the last round of iteration result to obtain an industrial defect feature segmentation result map. According to the invention, the demand on computing resources can be reduced, and the defect detection task on the production line can be quickly and accurately completed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of industrial defect detection systems, and in particular to an industrial defect detection system based on iterative feedback optimization. Background Art

[0002] Current industrial defect detection methods can be divided into traditional industrial defect detection and automatic industrial defect detection based on deep learning technology.

[0003] Traditional industrial defect detection refers to manual inspection and traditional machine vision methods. Although manual visual inspection has certain flexibility and adaptability, it has significant limitations. Manual inspection is slow and cannot meet the real-time monitoring needs of large-scale, high-speed production lines. It is prone to missed inspections and misjudgments. At the same time, personal judgment standards are greatly affected by subjectivity, which is not conducive to the accuracy and consistency of the detection results. The traditional rule-based machine vision method mainly relies on the basic principles of computer vision, image processing technology, and predefined rules and algorithms. It shows good stability and controllability in defect detection under specific conditions, but the problem is that it usually lacks generalization ability, and the recognition effect of defects that have not been seen or in complex backgrounds is poor. In addition, the programming process is usually complicated and it is difficult to cope with product design changes or the emergence of new defect types.

[0004] Industrial defect detection technology based on deep learning is an advanced method developed in the context of modern industrial automation and quality control. It combines cutting-edge research results in computer vision, machine learning, especially deep learning, to solve many challenges faced by traditional industrial defect detection methods.

[0005] The research on industrial defect detection methods based on deep learning has extensive technical advantages and research value. It has the advantages of high accuracy and adaptability, high degree of automation, and good continuous optimization capabilities. It is one of the key technologies for realizing Industry 4.0 and digital transformation, and can bring significant economic benefits to enterprises. However, since the results of automatic industrial defect detection directly affect production quality and efficiency, and production efficiency determines the timeliness of the detection process, that is, it is necessary to make judgments quickly and accurately, the deep learning network model used must take into account both speed and accuracy. This is the main direction of research on industrial defect detection based on deep learning. At present, most of the models used in industrial defect detection methods based on deep learning have complex structures and high computational complexity by stacking a large number of modules, which is not conducive to production environments with limited resources. Summary of the invention

[0006] The present invention provides an industrial defect detection system based on iterative feedback optimization to solve the problems of complex structure and high computational complexity of the model adopted by the industrial defect detection method based on deep learning in the prior art.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] An industrial defect detection system based on iterative feedback optimization includes a backbone network, an iterative module, a feedback optimization module, and a spatial attention module SAM; an industrial defect image to be detected is input into the backbone network, a feature map of a total of four levels of scales is extracted by the backbone network, and the feature maps of the second level, the third level, and the fourth level of scales are sent to the iterative module, and the feature map of the first level of scale is sent to the spatial attention module SAM;

[0009] The iteration module performs multiple rounds of iterative processing to obtain multiple rounds of iterative results, and sends the multiple rounds of iterative results to the feedback optimization module respectively, and sends the last round of iterative results among the multiple rounds of iterative results to the spatial attention module SAM. The process of each round of iteration is as follows:

[0010] In each round of iteration, firstly, the feature map of the second level scale and the feature map of the fourth level scale are fused with the result of the previous round of iteration respectively through different iterative feature fusion modules IFF; the feature map of the fourth level scale fused with the result of the previous round of iteration is processed by an attention module SEM, and then upsampled to the feature map of the third level scale to obtain a first spliced ​​feature map, and the first spliced ​​feature map is processed by another attention module SEM, and then upsampled to the feature map of the second level scale fused with the result of the previous round of iteration to obtain a second spliced ​​feature map, and the second spliced ​​feature map is finally processed by the third attention module SEM and then channel compressed, thereby obtaining the result of the current round of iteration;

[0011] The feedback optimization module applies a loss function to the obtained multiple rounds of iteration results, and supervises and feeds back the iteration effect of the iteration module based on the loss function calculation result;

[0012] The spatial attention module SAM fuses the feature map of the first level scale and the result of the last round of iteration to obtain an industrial defect feature segmentation result map.

[0013] Furthermore, the backbone network is a CSPDarknet network.

[0014] Furthermore, the iterative feature fusion module IFF in the iterative module resizes the input feature map of the corresponding level scale, and upsamples the previous round of iteration results and the resized feature map of the corresponding level scale to splice them by channel, and then performs channel adjustment, thereby integrating the previous round of iteration results into the feature map of the corresponding level scale.

[0015] Furthermore, each attention module SEM in the iterative module compresses the spatial information of the input image through global average pooling, and learns the importance weight of each channel through a fully connected layer.

[0016] Furthermore, the spatial attention module SAM first performs mean pooling and maximum pooling on the feature map of the first level scale, and performs mean pooling and maximum pooling on the result of the last round of iteration; then the spatial attention module SAM splices the mean pooling result and the maximum pooling result of the feature map of the first level scale and then performs convolution processing, and splices the mean pooling result and the maximum pooling result of the last round of iteration results and then performs convolution processing; then the spatial attention module SAM multiplies the feature map of the first level scale with the corresponding convolution processing result, and multiplies the last round of iteration results with the corresponding convolution processing result; finally, the spatial attention module SAM splices the two sets of multiplication results to obtain the industrial defect feature segmentation result map.

[0017] Inspired by the human cognitive review process of knowledge, the present invention proposes an industrial defect detection system based on iterative feedback optimization. First, in order to solve the problems of complex structure, serious module stacking and large computational complexity of convolutional neural network models, the present invention chooses to use an iterative mechanism to reuse some modules to reduce the model size, and adds an attention module to focus on more important feature information during feature fusion; secondly, the present invention applies a weighted loss function to the iterative result to feedback and supervise the iterative process, ensuring that the iterative result is optimized successively; finally, in order to retain shallow spatial information and detail features, the present invention captures large target information with a larger receptive field, and fuses the underlying large-scale feature map with the iterative result, effectively improving the model's learning ability for features of different sizes.

[0018] Therefore, the present invention reuses some modules through an iterative mechanism, repeatedly analyzes and learns features of images, and continuously optimizes the results. While ensuring high performance, the size of the final neural network model is controlled not to be too large, thereby reducing the demand for computing resources and being able to quickly and accurately complete defect detection tasks on the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the overall structure of the first embodiment of the present invention.

[0020] Figure 2 It is a schematic diagram of the CSP structure in the first embodiment of the present invention.

[0021] Figure 3 It is a schematic diagram of the IFF structure in the first embodiment of the present invention.

[0022] Figure 4 It is a schematic diagram of the SAM structure in the first embodiment of the present invention.

[0023] Figure 5 This is a diagram of the detection results of IFRNet in the second embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0025] Embodiment 1

[0026] like Figure 1 As shown, this embodiment discloses an industrial defect detection system based on iterative feedback optimization, named IFRNet, including a backbone network, an iteration module, a feedback optimization module, and a spatial attention module SAM. The input image is subjected to feature extraction by the backbone network to obtain feature maps of multiple levels and different scales as the input of the iteration module. The result after each round of iterative processing in the iteration module will be used as input again, fused with the original feature map for iterative processing, and the results of the iteration will be fed back to the supervision model through the feedback optimization module. After a total of four rounds of iterations, the iterative module generates four segmentation result maps as iterative results, which are all input into the feedback optimization module for optimization processing. Finally, the spatial attention module SAM fuses the results of the last round of iterations and the feature map of the first level scale to obtain the final accurate industrial defect feature segmentation result map. The specific instructions are as follows:

[0027] (1) In this embodiment, the backbone network uses the open source CSPDarknet network to extract features from the input industrial defect detection image. The industrial defect detection image is extracted through the CSPDarknet network to obtain multi-level feature maps of different scales, namely, the first-level feature map X1, the second-level feature map X2, the third-level feature map X3, and the fourth-level feature map X4 for subsequent processing. The scales of the feature maps X1, X2, X3, and X4 are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image (i.e., the industrial defect detection image). The second-level, third-level, and fourth-level feature maps X2, X3, and X4 are sent to the iteration module, and the first-level feature map X1 is sent to the spatial attention module SAM.

[0028] The backbone network CSPDarknet reduces the problem of repeated calculation of feature maps by connecting the network part forward and cross-stage (CSP) network structure, significantly improving the model operation efficiency. Figure 2 As shown in the figure, the cross-stage connection structure enhances the fluidity of features by merging features in the early stages, which helps alleviate the common gradient vanishing problem in deep learning and avoids feature redundancy. Each convolutional layer in CSP is followed by batch normalization and Silu activation function, which improves the stability of the training process and helps the propagation of gradient flow.

[0029] (2) In this embodiment, the iteration module performs multiple rounds of iteration processing to obtain multiple rounds of iteration results, and sends the multiple rounds of iteration results to the feedback optimization module respectively, and sends the last round of iteration results among the multiple rounds of iteration results to the spatial attention module SAM. The process of each round of iteration is as follows:

[0030] During each iteration of the iterative module, the feature map X2 of the second level scale is first fused with the result of the previous iteration through an iterative feature fusion module IFF, and the feature map X4 of the fourth level scale is fused with the result of the previous iteration through another iterative fusion module IFF, thereby obtaining the feature information of the iterative feedback of the result of the previous iteration before participating in the current iteration.

[0031] The feature map X4 of the fourth level scale fused with the result of the previous round of iteration is processed by an attention module SEM and then upsampled to the feature map X3 of the third level scale for splicing to obtain the first spliced ​​feature map. The first spliced ​​feature map is then processed by another attention module SEM and then upsampled to the feature map X2 of the second level scale fused with the result of the previous round of iteration for splicing to obtain the second spliced ​​feature map. The second spliced ​​feature map is finally processed by the third attention module SEM and then channel compressed to obtain the result of the current round of iteration.

[0032] In this embodiment, the iteration module performs N=4, i.e., four rounds of iteration processing, thereby obtaining four rounds of iteration results Y i , i = 1, 2, 3, 4. The four-round iteration results Y1, Y2, Y3, and Y4 are respectively sent to the feedback optimization module, and the last round of iteration result Y4 is sent to the spatial attention module SAM,

[0033] In this embodiment, Figure 3 As shown, each iterative feature fusion module IFF resizes the input feature map of the corresponding level scale to ensure that it is consistent with the image size of the previous round of iteration results. Then, the neighboring interpolation method is used to upsample the previous round of iteration results and the resized feature maps of the corresponding level scales, and Concat splicing is performed by channel. In the first round of iteration, since there is no result of the previous round of iteration, the result of the previous round of iteration takes a null value in the first round of iteration. Then, in order to enhance the selectivity and expression ability of the concat spliced ​​features, the concat spliced ​​features are input into the convolution layer for channel adjustment, thereby integrating the results of the previous round of iteration into the feature map of the corresponding level scale. The data processing process of the iterative feature fusion module IFF in the i-th round of iteration is shown in the following formula:

[0034] IFF(X k ,Y i-1 )=Conv(Concat(UP(X k ),Y i-1 ))

[0035] Among them, X k Y is the feature map of the k-th scale (k=2, 4); i-1 represents the result of the i-1th iteration (i=1,2,3); Conv represents the convolutional layer.

[0036] In this embodiment, each attention module SEM compresses the spatial information of the input image through global average pooling, and learns the importance weight of each channel through two fully connected layers. This adaptive learning enables the network to pay more attention to the feature channels that are useful for the current task and ignore unimportant information, thereby improving the utilization of features and the discriminability of the model.

[0037] (3) In this embodiment, a loss function is applied to the obtained results of multiple rounds of iterations in the feedback optimization module, and the iterative effect of the iterative module is supervised and fed back based on the loss function calculation results. In this way, the data flow of each iteration can be fed back, and each feedback feature can be linked to the segmentation result to avoid feature damage caused by the iterative structure.

[0038] In the feedback optimization module, the loss function used for each round of iteration results of the iteration module is the open source CIOU loss function. Then the feedback optimization module assigns different weights to each round of iteration results according to the iteration rounds. The iterative feedback loss function is as follows:

[0039]

[0040] Where ω=0.2 is the weight coefficient and i is the iteration round.

[0041] (4) In this embodiment, Figure 4 As shown, the spatial attention module SAM fuses the first-level scale feature map X1 and the last round of iteration result Y4 to obtain the industrial defect feature prediction result.

[0042] Specifically, the spatial attention module SAM has two processing channels, which are used to process the feature map of the first-level scale and the last round of iteration results respectively. In the first processing channel, the feature map of the first-level scale is firstly subjected to mean pooling and maximum pooling, and then the mean pooling result and the maximum pooling result of the feature map of the first-level scale are spliced ​​and then convolution is performed, and then the feature map of the first-level scale is multiplied with the corresponding convolution processing result to obtain the multiplication result. In the second processing channel, the last round of iteration results are firstly subjected to mean pooling and maximum pooling, and then the mean pooling result and the maximum pooling result of the last round of iteration results are spliced ​​and then convolution is performed, and then the last round of iteration results are multiplied with the corresponding convolution processing result to obtain the multiplication result. Finally, the spatial attention module SAM splices the multiplication results obtained from the two processing channels to obtain the industrial defect feature prediction result.

[0043] Embodiment 2

[0044] The following is a comparative experimental example of the industrial defect detection system IFRNet based on iterative feedback optimization described in Example 1 and the existing FCN, SSD, Mask-rcnn, U-Net, U-Net++, SegNet and Yolov8-seg. The specific description is as follows:

[0045] (1) Experimental setup

[0046] In this experiment, NVIDIA GTX 3090 GPU is used, and the model code is built based on Python and pytorch framework. In addition, the input image size of this experiment is set to 640×640, the AdamW optimizer is used, the initial value of the learning rate is about 0.01, and the training uses the Early Stopping strategy. Due to the small data set, a total of 200 iterations are performed during training.

[0047] (2) Datasets and evaluation metrics

[0048] This experimental example uses the CDS2k dataset, which has a total of 2492 samples from multiple different industrial product databases. The dataset consists of 1330 positive samples and 1162 negative samples. The defect area and the background have similar patterns and have hidden features. The evaluation indicators selected are precision, recall, and mAP50. Figure 5 Detection results of some images of IFRNet.

[0049] In this experimental example, the CDS2k data set is divided into a training set, a validation set, and a test set in a ratio of 6:2:2. The training set is input into the IFRNet described in Example 1, the loss error is calculated, and the model parameters are trained through the back propagation algorithm; the validation data set is input into the trained IFRNet to verify the IFRNet and preliminarily test the segmentation effect of the IFRNet; and the test set data is input into the trained IFRNet to detect the accuracy of the IFRNet, thereby completing the training of the IFRNet described in Example 1. Figure 5 This is a diagram of the detection results of IFRNet described in Example 1.

[0050] At the same time, this experimental example inputs the above training set, validation set, and test set into other prediction models, namely FCN, SSD, Mask-Rcnn, U-Net, U-Net++, SegNet, and Yolov8-seg, for comparative experiments.

[0051] (3) Experimental performance comparison

[0052] By statistically comparing the experimental data, the detection accuracy of IFRNet described in Example 1, as well as FCN, SSD, Mask-Rcnn, U-Net, U-Net++, SegNet and Yolov8-seg are compared and evaluated. Table 1 shows the comparative experimental results of a total of 8 models in this experimental example. It can be seen that the IFRNet described in Example 1 performs better than other models in various indicators on the CDS2k dataset and achieves good detection results. Table 1 is as follows:

[0053] Table 1. Comparative experimental results of 8 models

[0054]

[0055] according to Figure 5 As can be seen from Table 1, the IFRNet described in Example 1 can achieve good defect detection effects in industrial defect detection in different scenarios, and has better results than all comparison methods in various indicators.

[0056] The proposed IFRNet described in Example 1 uses CSPDarknet as the backbone network, designs an iterative mechanism to process feature maps, forms an effective feedback optimization mechanism to ensure the effectiveness of iteration, and adds a channel attention mechanism to ensure the effective acquisition of information during feature fusion. The model also finally fuses low-level large-scale feature maps to ensure that shallow spatial features and detail information are not lost. This experimental example evaluates the IFRNet described in Example 1 on the CDS2k dataset, and conducts comparative experiments with multiple segmentation models. The comprehensive test results and comparative test results show that the IFRNet described in Example 1 has a good accuracy rate and can reach the most advanced level in industrial defect detection tasks.

[0057] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. The embodiments described in the present invention are merely descriptions of the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. The various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction, and such combinations should also be regarded as the contents disclosed in the present disclosure as long as they do not violate the concept of the present invention. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0058] The present invention is not limited to the specific details of the above-mentioned embodiments. Within the technical concept of the present invention and without departing from the design concept of the present invention, various modifications and improvements made to the technical solution of the present invention by technical personnel in this field should fall within the protection scope of the present invention. The technical contents for which protection is sought in the present invention have been fully recorded in the claims.

Claims

1. An industrial defect detection system based on iterative feedback optimization, It is characterized in that It includes a backbone network, an iteration module, a feedback optimization module, and a spatial attention module SAM; the industrial defect image to be detected is input into the backbone network, and a feature map of a total of four scales is extracted by the backbone network, and the feature maps of the second, third, and fourth scales are sent to the iteration module, and the feature map of the first scale is sent to the spatial attention module SAM; The iteration module performs multiple rounds of iterative processing to obtain multiple rounds of iterative results, and sends the multiple rounds of iterative results to the feedback optimization module respectively, and sends the last round of iterative results among the multiple rounds of iterative results to the spatial attention module SAM. The process of each round of iteration is as follows: In each round of iteration, firstly, the feature map of the second level scale and the feature map of the fourth level scale are fused with the result of the previous round of iteration respectively through different iterative feature fusion modules IFF; the feature map of the fourth level scale fused with the result of the previous round of iteration is processed by an attention module SEM, and then upsampled to the feature map of the third level scale to obtain a first spliced ​​feature map, and the first spliced ​​feature map is processed by another attention module SEM, and then upsampled to the feature map of the second level scale fused with the result of the previous round of iteration to obtain a second spliced ​​feature map, and the second spliced ​​feature map is finally processed by the third attention module SEM and then channel compressed, thereby obtaining the result of the current round of iteration; The feedback optimization module applies a loss function to the obtained multiple rounds of iteration results, and supervises and feeds back the iteration effect of the iteration module based on the loss function calculation result; The spatial attention module SAM fuses the feature map of the first level scale and the result of the last round of iteration to obtain an industrial defect feature segmentation result map.

2. The industrial defect detection system based on iterative feedback optimization according to claim 1, It is characterized in that The backbone network is the CSPDarknet network.

3. The industrial defect detection system based on iterative feedback optimization according to claim 1, It is characterized in that The iterative feature fusion module IFF in the iterative module resizes the input feature map of the corresponding level scale, upsamples the previous round of iteration results and splices the resized feature map of the corresponding level scale by channel, and then performs channel adjustment, thereby integrating the previous round of iteration results into the feature map of the corresponding level scale.

4. The industrial defect detection system based on iterative feedback optimization according to claim 1, It is characterized in that Each attention module SEM in the iterative module compresses the spatial information of the input image through global average pooling, and learns the importance weight of each channel through a fully connected layer.

5. The industrial defect detection system based on iterative feedback optimization according to claim 1, It is characterized in that The spatial attention module SAM first performs mean pooling and maximum pooling on the feature map of the first level scale, and performs mean pooling and maximum pooling on the result of the last round of iteration; Then the spatial attention module SAM concatenates the mean pooling result and the maximum pooling result of the feature map of the first scale and then performs convolution processing, and concatenates the mean pooling result and the maximum pooling result of the last round of iteration results and then performs convolution processing; then the spatial attention module SAM multiplies the feature map of the first scale with the corresponding convolution processing result, and multiplies the last round of iteration results with the corresponding convolution processing result; finally, the spatial attention module SAM concatenates the two sets of multiplication results to obtain the industrial defect feature segmentation result map.

Citation Information

Patent Citations

  • Deep learning-based skin lesion image segmentation method and device, and storage medium

    CN114066904A

  • Light-weight strip steel surface defect detection method based on multi-scale feature fusion attention

    CN115330729A

  • Wood board small target surface defect identification and detection method based on machine vision

    CN118396968A

  • Intelligent defect detection system for anode copper plate production

    CN118485650A

Cited By

  • Seed germination segmentation method and seed germination rate and germination rate identification method

    CN119339087A

  • Seed germination segmentation method, seed germination rate and germination rate identification method

    CN119339087B