Thin film defect detection method and device based on MSSA-YOLOv8 lightweight model

Abstract

The invention discloses a thin film defect detection method and device based on an MSSA-YOLOv8 lightweight model, and the method specifically comprises the following steps: S1, obtaining thin film image data containing thin film surface defect information, carrying out the preprocessing of the thin film image data, obtaining an annotation data set, and dividing the annotation data set into a training set and a verification set; s2, an initial MSSA-YOLOv8 lightweight model is constructed through a YOLOv8 network architecture and a multi-scale stripe attention module; s3, inputting the training set in the step S1 into the initial MSSA-YOLOv8 lightweight model to carry out model training, carrying out precision monitoring through a verification set, and stopping training until the model converges to obtain a final MSSA-YOLOv8 network model; and S4, inputting to-be-detected film image data into the final MSSA-YOLOv8 network model, and completing detection of film defects. According to the thin film defect detection method based on the lightweight MSSA-YOLOv8, a backbone network is ingeniously designed according to the scene of thin film defect detection and high-resolution detection speed and according to the characteristics of weak semantic defects of a thin film single-channel image, the depth and parameter quantity of a network model are reduced, and the model parameter quantity is only 0.4 M.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and image technology, and in particular to a thin film defect detection method and device based on an MSSA-YOLOv8 lightweight model. Background Art

[0002] Traditional methods for thin film defect detection mostly rely on manual visual inspection or methods based on traditional machine vision, but these methods have the problem of difficulty in stably extracting defect features, especially when thin film defects usually present characteristics such as low contrast, weak semantics, and fine stripes, the detection accuracy of traditional methods is limited. Existing deep learning methods such as the YOLOv8 model have made significant progress in the field of target detection, but due to the complexity of the model and the large number of parameters, it is difficult to deploy in real time in industrial production with limited resources. To solve this problem, the present invention combines the characteristics of thin film defect detection and proposes a lightweight YOLOv8 network structure based on multi-scale stripe attention (MSSA) to improve detection accuracy and speed. Summary of the invention

[0003] The purpose of the present invention is to provide a thin film defect detection method and device based on the MSSA-YOLOv8 lightweight model to overcome the shortcomings of the prior art.

[0004] To achieve the above object, the present invention provides the following technical solutions: This application discloses a thin film defect detection method based on the MSSA-YOLOv8 lightweight model, which specifically includes the following steps: S1, obtaining film image data containing film surface defect information, preprocessing the data to obtain a labeled data set, and dividing the labeled data set into a training set and a verification set; S2, build the initial MSSA-YOLOv8 lightweight model through the YOLOv8 network architecture and multi-scale stripe attention module; S3, input the training set in step S1 into the initial MSSA-YOLOv8 lightweight model for model training, and monitor the accuracy through the verification set until the model converges, stop training, and obtain the final MSSA-YOLOv8 network model; S4. Input the film image data to be detected into the final MSSA-YOLOv8 network model to complete the detection of film defects.

[0005] Preferably, the preprocessing in step S1 includes: S11, cropping and resizing the film image data to obtain a 1024*1024 image block; the image block contains film surface defect information; S12, marking the area where the film surface defect information is located in the image block of step S11, and assigning a category label according to the type of the film surface defect information to obtain a marked data set; the types of the film surface defect information include dirt, creases and bright spots.

[0006] Preferably, the ratio of the training set to the validation set is 8:2.

[0007] Preferably, step S2 specifically includes the following sub-steps: S21. Set the number of input channels of the first convolutional layer of the YOLOv8 network architecture to 1; the backbone network includes 5 stages, namely stage1, stage2, stage3, stage4 and stage5. The number of channels of stage1 is 8, which is composed of 1 CONV_BN_SILU module; the number of channels of stage2, stage3, stage4 and stage5 are 8, 16, 32 and 64 respectively, which are composed of CONV_BN_SILU modules connected to C2F modules; S22, insert the multi-scale stripe attention module after stage2, stage3, stage4, and stage5, S23, input the output feature maps of stage2, stage3, stage4, and stage5 into the FAPAN module; complete the construction of the initial MSSA-YOLOv8 lightweight model.

[0008] Preferably, the operation of the multi-scale stripe attention module is as follows: for a two-dimensional tensor , using convolution operations with stripe widths of 1, 3, and 5, the calculation in the horizontal direction is as follows: ; The calculation in the horizontal direction is as follows: ; The output of MSSA is as follows: ; in The convolution kernel size is , the step length is A two-dimensional convolution operation.

[0009] Preferably, step S3 specifically includes the following operations: S31, setting training parameters; S32, input the training set into the initial MSSA-YOLOv8 lightweight model for model training; S33: During the model training process, the accuracy of the model is monitored through the validation set to determine whether the stop training standard is reached; if so, proceed to step S34; otherwise, loop through step S33; S34. Stop training and obtain the final MSSA-YOLOv8 network model.

[0010] Preferably, the training parameters of step S31 include an input image size of 1024*1024, a batch size of 8, and an upper limit of a total training epoch of 500.

[0011] Preferably, the criterion for stopping training in step S33 is as follows: the accuracy of the model does not increase for 100 consecutive epochs or reaches the upper limit of the training epoch.

[0012] The present invention also discloses a thin film defect detection device based on the MSSA-YOLOv8 lightweight model, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned thin film defect detection method based on the MSSA-YOLOv8 lightweight model.

[0013] The present invention also discloses a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned thin film defect detection method based on the MSSA-YOLOv8 lightweight model is implemented.

[0014] Beneficial effects of the present invention: 1. The thin film defect detection method based on lightweight MSSA-YOLOv8 proposed in the present invention is aimed at the scene of thin film defect detection with high resolution and fast detection speed. According to the characteristics of weak semantic defects in the single-channel image of the thin film, the backbone network is cleverly designed to reduce the depth and parameter amount of the network model. The model parameter amount is only 0.4M.

[0015] 2. The present invention proposes a multi-scale stripe attention module (MSSA). In the thin film defect detection scenario, there are many thin strip defects, such as dirt, creases, bright spots, etc. Inspired by the StringPooling algorithm, the present invention designs a multi-scale stripe attention module, which enables the model to capture richer and more critical feature representations, and improves the model's detection effect on thin film defects with only a small increase in parameters.

[0016] The features and advantages of the present invention will be described in detail through embodiments in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1It is a flow chart of a thin film defect detection method cabinet based on the MSSA-YOLOv8 lightweight model of the present invention; Figure 2 This is the MSSA-YOLOv8 model architecture diagram of the present invention; Figure 3 This is a comparison table of different models in terms of parameter quantity and mAP@0.5 indicators; Figure 4 It is a structural schematic diagram of a thin film defect detection device based on the MSSA-YOLOv8 lightweight model of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below through the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the scope of the present invention. In addition, in the following description, the description of known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0019] See also Figure 1 The embodiment of the present invention provides a thin film defect detection method based on the MSSA-YOLOv8 lightweight model, which specifically includes the following steps: S1, obtaining film image data containing film surface defect information, preprocessing the data to obtain a labeled data set, and dividing the labeled data set into a training set and a verification set; Specifically, the preprocessing in step S1 includes: S11, cropping and resizing the film image data to obtain a 1024*1024 image block; the image block contains film surface defect information; S12, marking the area where the film surface defect information is located in the image block of step S11, and assigning a category label according to the type of the film surface defect information to obtain a marked data set; the types of the film surface defect information include dirt, creases and bright spots.

[0020] S2, build the initial MSSA-YOLOv8 lightweight model through the YOLOv8 network architecture and multi-scale stripe attention module; Specifically, step S2 includes: S21. Set the number of input channels of the first convolutional layer of the YOLOv8 network architecture to 1; the backbone network includes 5 stages, namely stage1, stage2, stage3, stage4 and stage5. The number of channels of stage1 is 8, which is composed of 1 CONV_BN_SILU module; the number of channels of stage2, stage3, stage4 and stage5 are 8, 16, 32 and 64 respectively, which are composed of CONV_BN_SILU modules connected to C2F modules; S22, insert the multi-scale stripe attention module after stage2, stage3, stage4, and stage5, S23, input the output feature maps of stage2, stage3, stage4, and stage5 into the FAPAN module; complete the construction of the initial MSSA-YOLOv8 lightweight model, such as Figure 2 As shown; The operation of the multi-scale stripe attention module is as follows: for a two-dimensional tensor , using convolution operations with stripe widths of 1, 3, and 5, the calculation in the horizontal direction is as follows: ; The calculation in the horizontal direction is as follows: ; The output of MSSA is as follows: ; in The convolution kernel size is , the step length is A two-dimensional convolution operation.

[0021] S3, input the training set in step S1 into the initial MSSA-YOLOv8 lightweight model for model training, and monitor the accuracy through the verification set until the model converges, stop training, and obtain the final MSSA-YOLOv8 network model; Specifically, step S3 includes the following operations: S31, setting training parameters; S32, input the training set into the initial MSSA-YOLOv8 lightweight model for model training; S33: During the model training process, the accuracy of the model is monitored through the validation set to determine whether the stop training standard is reached; if so, proceed to step S34; otherwise, loop through step S33; S34. Stop training and obtain the final MSSA-YOLOv8 network model.

[0022] S4. Input the film image data to be detected into the final MSSA-YOLOv8 network model to complete the detection of film defects.

[0023] In a feasible embodiment, the ratio of the training set to the validation set is 8:2.

[0024] In a feasible embodiment, the training parameters of step S31 include an input image size of 1024*1024, a batch size of 8, and an upper limit of a total training epoch of 500.

[0025] In a feasible embodiment, the criterion for stopping training in step S33 is as follows: the accuracy of the model does not increase for 100 consecutive epochs or reaches the upper limit of the training epoch.

[0026] Example: 1. Film image data collection: Collect film image data on industrial production lines to ensure that common defects such as dirt, creases, bright spots, etc. are included. Ensure image clarity and ensure that the details of small defects can be distinguished; 2. Image cropping and resizing: crop the original captured image into 1024x1024 blocks to meet the input requirements of YOLOv8. When cropping, ensure that each image block contains the film surface defect information.

[0027] 3. Use the labeling tool (LabelMe) to label the defect area and assign a category label to each type of defect (dirt, crease, bright spot, etc.). Convert the labeling format to a YOLO labeling file and divide the labeling data set into a training set (80%) and a validation set (20%) to ensure that the model performs well in both the training and validation stages.

[0028] 4. Build a lightweight backbone network and build a lightweight model based on the YOLOv8 architecture. The backbone network is divided into five stages (stage 1 to stage 5). The number of input channels of the first convolutional layer is set to 1 to adapt to single-channel image input. The characteristics of each stage are as follows: Stage 1: consists of a CONV_BN_SILU module for preliminary feature extraction.

[0029] Stage2 to Stage5: Each stage includes a CONV_BN_SILU module followed by a C2F module, and the number of channels is set to 8, 8, 16, 32, and 64.

[0030] 5. Add the multi-scale stripe attention module (MSSA) and insert the MSSA module after stage 2 to stage 5. The MSSA module uses convolution operations with stripe widths of 1, 3, and 5 to calculate feature information in the horizontal and vertical directions, combining multi-scale stripe features to enhance the model's feature capture capability.

[0031] The multi-scale stripe attention module (MSSA) is defined as follows: Given a two-dimensional tensor ,in is the stripe width (the stripe width used in the present invention is 1, 3, 5), where The convolution kernel size is , the step length is A two-dimensional convolution operation.

[0032] The calculation in the horizontal direction is as follows: The calculation in the vertical direction is as follows: For MSSA, for multiple stripe widths 1, 3, and 5, the output of MSSA is as follows: ; 6. Feature fusion: Input the feature map of stage 2 into the FAPAN module, and combine the feature maps of different scales to further improve the detection accuracy, especially in capturing the features of fine stripe defects; 7. Model training settings,Use the processed training data set, set the input image size to 1024x1024, the batch size to 8, and the total training epoch limit to 500, ensuring that there are enough training iterations to improve the stability of the model.

[0033] 8. Dynamic adjustment of adaptive learning rate: Adopting adaptive learning rate strategy, the learning rate is automatically adjusted according to the accuracy of the validation set, so that the model can reach the optimal performance state more quickly.

[0034] 9. Early Stopping: Monitor the validation set accuracy during training. When the validation set accuracy does not improve for 100 consecutive rounds, stop training to prevent overfitting.

[0035] 10. Monitor and evaluate the training process, monitor the training loss and the change in the accuracy of the validation set, and save the model with the best performance on the validation set for use in actual testing.

[0036] 11. Real-time detection system deployment, deploy the trained MSSA-YOLOv8 model to the film detection system. The system divides the film images on the assembly line into 1024x1024 blocks and inputs them into the model one by one for defect detection.

[0037] 12. Defect recognition and classification, the model performs defect detection on each image and outputs information such as defect location, type and confidence.

[0038] 13. Visualization of test results: Display the test results in real time on the monitoring interface, mark the defect location, category and confidence level, and facilitate operators to monitor film quality.

[0039] 14. Defect handling and alarm mechanism: the system can trigger an alarm for serious defects, remind operators to intervene, and save the detection records to facilitate subsequent quality analysis and tracking.

[0040] like Figure 3 The figure shows a comparison of parameters and mAP@0.5 indicators of different models. It can be seen from the figure that the MSSA-YOLOv8 proposed in the present invention has 87.7% fewer parameters than YOLOv8n, and the accuracy index has increased by 0.219 when mAP@0.5 is used as the accuracy evaluation index. That is, the MSSA-YOLOv8 lightweight model proposed in the present invention is fast and accurate in terms of speed and accuracy.

[0041] An embodiment of a thin film defect detection device based on the MSSA-YOLOv8 lightweight model of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From a hardware perspective, if Figure 4 As shown, it is a hardware structure diagram of a thin film defect detection device based on the MSSA-YOLOv8 lightweight model of the present invention, in which any device with data processing capability is located, except Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in the embodiment of the device may also include other hardware according to the actual function of the device with data processing capabilities, which will not be described in detail. The implementation process of the functions and effects of each unit in the above device is specifically detailed in the implementation process of the corresponding steps in the above method, which will not be described in detail here.

[0042] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0043] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a thin film defect detection device based on the MSSA-YOLOv8 lightweight model in the above embodiment is implemented.

[0044] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capability. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0045] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent substitution or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A thin film defect detection method based on the MSSA-YOLOv8 lightweight model, characterized in that: The specific steps include: S1, obtaining film image data containing film surface defect information, preprocessing the data to obtain a labeled data set, and dividing the labeled data set into a training set and a verification set; S2, build the initial MSSA-YOLOv8 lightweight model through the YOLOv8 network architecture and multi-scale stripe attention module; S3, input the training set in step S1 into the initial MSSA-YOLOv8 lightweight model for model training, and monitor the accuracy through the verification set until the model converges, stop training, and obtain the final MSSA-YOLOv8 network model; S4. Input the film image data to be detected into the final MSSA-YOLOv8 network model to complete the detection of film defects.

2. The thin film defect detection method based on the MSSA-YOLOv8 lightweight model according to claim 1, characterized in that: The preprocessing in step S1 includes: S11, cropping and resizing the film image data to obtain a 1024*1024 image block; the image block contains film surface defect information; S12, marking the area where the film surface defect information is located in the image block of step S11, and assigning a category label according to the type of the film surface defect information to obtain a marked data set; the types of the film surface defect information include dirt, creases and bright spots.

3. The thin film defect detection method based on the MSSA-YOLOv8 lightweight model according to claim 1, characterized in that: The ratio of the training set to the validation set is 8:

2.

4. The thin film defect detection method based on the MSSA-YOLOv8 lightweight model according to claim 1, characterized in that: Step S2 specifically includes the following sub-steps: S21. Set the number of input channels of the first convolutional layer of the YOLOv8 network architecture to 1; the backbone network includes 5 stages, namely stage1, stage2, stage3, stage4 and stage5. The number of channels of stage1 is 8, which is composed of 1 CONV_BN_SILU module; the number of channels of stage2, stage3, stage4 and stage5 are 8, 16, 32 and 64 respectively, which are composed of CONV_BN_SILU modules connected to C2F modules; S22, insert the multi-scale stripe attention module after stage2, stage3, stage4, and stage5, S23, input the output feature maps of stage2, stage3, stage4, and stage5 into the FAPAN module; complete the construction of the initial MSSA-YOLOv8 lightweight model.

5. The thin film defect detection method based on the MSSA-YOLOv8 lightweight model according to claim 1, characterized in that: The operation of the multi-scale stripe attention module is as follows: for a two-dimensional tensor , using convolution operations with stripe widths of 1, 3, and 5, the calculation in the horizontal direction is as follows: ； The calculation in the horizontal direction is as follows: ； The output of MSSA is as follows: ； in The convolution kernel size is , the step length is A two-dimensional convolution operation.

6. The thin film defect detection method based on the MSSA-YOLOv8 lightweight model according to claim 1, characterized in that: Step S3 specifically includes the following operations: S31, setting training parameters; S32, input the training set into the initial MSSA-YOLOv8 lightweight model for model training; S33: During the model training process, the accuracy of the model is monitored through the validation set to determine whether the stop training standard is reached; if so, proceed to step S34; otherwise, loop through step S33; S34. Stop training and obtain the final MSSA-YOLOv8 network model.

7. The thin film defect detection method based on the MSSA-YOLOv8 lightweight model according to claim 6, characterized in that: The training parameters of step S31 include input image size 1024*1024, batch size 8, and total training epoch limit 500.

8. The thin film defect detection method based on the MSSA-YOLOv8 lightweight model according to claim 6, characterized in that: The criterion for stopping training in step S33 is as follows: the accuracy of the model does not increase for 100 consecutive epochs or reaches the upper limit of the training epoch.

9. A thin film defect detection device based on the MSSA-YOLOv8 lightweight model, characterized in that: It includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement a thin film defect detection method based on the MSSA-YOLOv8 lightweight model as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a thin film defect detection method based on the MSSA-YOLOv8 lightweight model as described in any one of claims 1 to 8 is implemented.

Images