A surface defect detection method based on a semi-supervised training strategy

Abstract

A surface defect detection method based on a semi-supervised training strategy, surface image acquisition is carried out, and a surface defect data set is constructed; the surface defect data set is divided; the picture in the labeled data set is data labeled using the Labelimg tool; the labeled data set is divided, the yolov7 model is modified, and the yolov7-ea model is obtained; in the supervision stage, the yolov7-ea model is trained in the labeled data set, and the trained yolov7-ea model is used as a pseudo-label generator; in the semi-supervised stage, the yolov7-ea model is used as a detection model. Pseudo-labels are generated by using a pseudo-label generator to infer unlabeled data, and the detection model is trained by combining labeled data and unlabeled data with pseudo-labels, and the pseudo-label generator is updated in real time with the best detection model. The present application greatly reduces the workload of manual labeling while achieving good detection performance.

Description

Technical Field

[0001] This invention relates to the fields of semi-supervised learning and defect detection technology, and in particular to a surface defect detection method based on a semi-supervised training strategy. Background Technology

[0002] Surface defect inspection is a crucial part of industrial product manufacturing. Its purpose is to ensure that the quality and appearance of product surfaces meet standards, as surface defects can directly impact product quality, performance, and safety. However, surface defects are unavoidable due to environmental factors in the production workshop, equipment malfunctions, and operator errors. Therefore, strictly controlling product surface quality is a key focus in industrial product manufacturing.

[0003] Traditional surface defect detection relies primarily on manual inspection, which is susceptible to subjective human factors, leading to missed detections and false positives. Furthermore, its low efficiency makes it unsuitable for large-scale production. With the development of machine vision technology, new automated surface defect detection methods have emerged, largely overcoming the drawbacks of manual inspection methods such as susceptibility to subjective influences, missed detections, and false positives, thus improving detection efficiency. However, machine vision-based methods require manual feature design and extraction, heavily relying on the designer's experience. Moreover, given the complexity and diversity of surface defects, accurate feature extraction is difficult using manual methods, resulting in poor detection performance. Deep learning, with its ability to automatically learn features from raw data and its strong learning and expressive capabilities, is increasingly being used in surface defect detection, achieving better results than traditional methods. However, deep learning-based surface defect detection typically employs fully supervised models, which rely on a large number of accurately labeled samples for training. This requires significant time and human resources, and in practice, a lack of skilled and professional labelers often leads to large labeling errors, affecting the model's accuracy.

[0004] Based on the above analysis, the problems and shortcomings of existing technologies are as follows: 1. Existing fully supervised models rely on a large number of accurate manually labeled samples for training, which requires a significant amount of time and human resources. 2. Due to the lack of skilled and professional labelers, significant labeling errors occur, thus affecting the accuracy of the model. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention proposes a surface defect detection method based on a semi-supervised training strategy. This method can make full use of the information of unlabeled samples by generating pseudo-labels for unlabeled samples. Therefore, a small number of labeled and unlabeled samples can be used simultaneously for model training, which greatly reduces the workload of manual labeling while achieving good detection performance.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] A surface defect detection method based on a semi-supervised training strategy, the method comprising the following steps:

[0008] S1. Build a surface image acquisition system to acquire surface images and construct a surface defect dataset;

[0009] S2, divide the surface defect dataset into unlabeled and labeled datasets according to a set ratio;

[0010] S3, Use the Labelimg tool to annotate the images in the labeled dataset;

[0011] S4, Divide the labeled dataset into training and test sets according to a set ratio;

[0012] S5, Modify the yolov7 model to obtain the yolov7-ea model;

[0013] S6. In the supervised phase, the yolov7-ea model is trained on the labeled dataset and the trained yolov7-ea model is used as a pseudo-label generator.

[0014] In S7, during the semi-supervised stage, the YOLOv7-EA model is used as the detection model. A pseudo-label generator is used to infer pseudo-labels from unlabeled data. The detection model is trained using both labeled data and unlabeled data with pseudo-labels, and the pseudo-label generator is updated in real-time using the best-performing detection model.

[0015] Furthermore, in S1, the surface image acquisition system includes an LCD screen, a CCD camera, a sample to be tested, and a computer.

[0016] Preferably, the ratio set in S2 is 9:1; the ratio set in S4 is 7:3.

[0017] Furthermore, in S5, the yolov7-ea model includes Backbone, Neck, and Head, and includes the following sub-steps:

[0018] In Backbone, S51 uses deformable convolution to replace the 3×3 ordinary convolution in the ELAN module, that is, the original ELAN module is replaced by the newly formed DCN-ELAN.

[0019] In the Neck, S52 modifies the original SPPCSPC structure into an SPPFCSPC structure and introduces an external attention mechanism to form an SPPFEA structure, that is, replaces the original SPPCSPC structure with the newly formed SPPFEA.

[0020] S53 replaces the original CIoU in the loss function of the YOLOv7 model with WIoU. The formula for WIoU is as follows:

[0021] L WIoUv1 =R WIoU L IOU

[0022]

[0023]

[0024] Among them, R WIoU ∈[1,e), will significantly amplify the L of ordinary quality anchor boxes. IoU L IoU ∈[0,1], when a high-quality anchor box coincides with the target box, it will significantly reduce R. WIoU And its focus on the distance between the two center points; x and y are the center coordinates of the bounding box; x gt and y gt W represents the center coordinates of the bounding box. g and H g These are the width and height of the smallest package; W i and H i These are the width and height of the overlapping region of the bounding box and the target box, respectively; S u It is the area of ​​the combined region of the bounding box and the target box.

[0025] Furthermore, in S6, the supervised training phase uses labeled data to train the model, and the loss in this phase is the supervised loss, i.e., L. s The monitoring loss is defined as:

[0026]

[0027] Among them, L obj For object loss, L cls For classification loss, L box For bounding box loss.

[0028] In S7, the semi-supervised stage uses both labeled and unlabeled data to train the model. The loss in this stage is the semi-supervised loss, which is the sum of the supervised loss and the unsupervised loss. The semi-supervised loss is defined as follows:

[0029]

[0030] Among them, L s It is the monitoring loss, λ u It is the unsupervised loss weight, used to balance supervised loss and unsupervised loss.

[0031] The beneficial effects of this invention are: it greatly reduces the workload of manual labeling while achieving good detection performance. Attached Figure Description

[0032] Figure 1 This is a flowchart of the detection method of the present invention.

[0033] Figure 2 The network structure diagram for the improved yolov7-ea model.

[0034] Figure 3 This is a flowchart of the semi-supervised training strategy of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0036] The above-mentioned and other technical features and advantages of the present invention will be described in detail below with reference to the accompanying drawings.

[0037] Reference Figures 1-3 A surface defect detection method based on a semi-supervised training strategy includes the following steps:

[0038] S1 is used to build a surface image acquisition system to acquire surface images and construct a surface defect dataset. The surface image acquisition system includes an LCD screen, a CCD camera, the sample to be inspected, and a computer. During surface image acquisition, the LCD screen illuminates the surface of the sample, the CCD camera captures the reflected image, and the captured surface defect images are transmitted to the computer for processing.

[0039] S2 divides the surface defect dataset into an unlabeled dataset and a labeled dataset in a 9:1 ratio.

[0040] S3 uses the Labelimg tool to annotate images in the labeled dataset. There are five common types of surface defects: particles, shrinkage cavities, fibers, pits, and external damage.

[0041] S4 divides the labeled dataset into training and test sets in a 7:3 ratio.

[0042] S5 modifies the yolov7 model to obtain the yolov7-ea model.

[0043] The YOLOv7-EA model consists of three parts: Backbone, Neck, and Head. The YOLOv7-EA network structure diagram is shown below. Figure 2 As shown below:

[0044] In S51 Backbone, deformable convolutions replace the 3×3 ordinary convolutions in the ELAN module with deformable convolutions (DCN-ELAN), essentially replacing the original ELAN module with the newly formed DCN-ELAN. Deformable convolutions add direction vectors to the original convolution kernel, increasing the kernel's extension range and shape diversity. Because defects are diverse in type and shape, fixed-size convolution kernels struggle to capture all types of defects. Deformable convolutions can automatically adjust the scale or receptive field to better capture defect features. The feature matrix obtained from deformable convolutions is as follows:

[0045]

[0046] Where, Δp n Let be the offset matrix. During training, the convolutional kernels that generate the output features and the convolutional kernels that generate the offsets are learned simultaneously. The offsets are learned using a bilinear interpolation algorithm through backpropagation.

[0047] In S52's Neck module, the original SPPCSPC structure is modified to an SPPFCSPC structure, and an external attention mechanism is introduced, forming the SPPFEA structure. This SPPFEA replaces the original SPPCSPC structure. SPPFEA replaces the 5×5, 9×9, and 13×13 max-pooling layers in the SPPCSPC module with three cascaded 5x5 max-pooling layers. Each pooling layer extracts features from the layer above, reducing computational load and improving speed while preserving the original receptive field. Considering the significant differences in defect features and the small proportion of defect pixels, a memory strategy can be used to accumulate defect features from different image samples to enhance learning. The SPPFEA structure introduces an external attention mechanism to consider the correlation between all data samples and uses external memory instead of self-attention to mine potential relationships in the dataset and provide stronger regularization. The memory mechanism of the external attention mechanism is implemented by two learnable external memory units, which can accumulate defect features from different samples in the entire dataset and discover potential relationships between different defect features. The working principle of the external attention mechanism is as follows:

[0048]

[0049] F out =AMv

[0050] Among them, F in Feature maps extracted for spatial pyramid pooling modules (F in Belongs to R nxd There are n elements, each with d-dimensional features, and the memory key M. k and memory value M v The value is assigned by default using a random initialization method (M) k Belongs to R sxd M v Belongs to R sxd s is the number of features in the memory unit, which defaults to 64.

[0051] S53 replaces the original CIoU in the YOLOv7 model's loss function with WIoU (Wise-IoU). The formula for WIoU is as follows:

[0052] L WIoUv1 =R WIoU L IoU

[0053]

[0054]

[0055] Among them, R WIoU ∈[1,e), will significantly amplify the L of ordinary quality anchor boxes. IoU L IoU ∈[0,1], when a high-quality anchor box coincides with the target box, it will significantly reduce R. WIoU And its focus on the distance between the two center points. x and y are the center coordinates of the bounding box; x gt and y gt W represents the center coordinates of the bounding box. g and H g These are the width and height of the smallest package; W i and H i These are the width and height of the overlapping region of the bounding box and the target box, respectively; S u It is the area of ​​the combined region of the bounding box and the target box.

[0056] In the supervised phase, the YOLOv7-EA model is trained on the labeled dataset, and the trained YOLOv7-EA model is used as a pseudo-label generator. We will use the labeled samples D... sWeak augmentation is performed to obtain dataset D1, which is then input into the defect detector for training, resulting in the defect detector T1 with adjusted weights. The supervised training phase uses labeled data to train the model; the loss in this phase is the supervised loss, i.e., Lsupervised. s The monitoring loss is defined as:

[0057]

[0058] Among them, L obj For object loss, L cls For classification loss, L box For bounding box loss.

[0059] In the semi-supervised stage, S7 uses the YOLOv7-EA model as the detection model. A pseudo-label generator is used to infer pseudo-labels from unlabeled data. The detection model is trained using both labeled data and unlabeled data with pseudo-labels, and the pseudo-label generator is updated in real-time using the best-performing detection model.

[0060] First, T1 in S6 is used as a pseudo-label generator to infer unlabeled samples and obtain preliminary pseudo-labels. Then, NMS is used to remove redundant predicted boxes. Next, to prevent a large number of low-quality pseudo-labels from degrading model performance, a confidence-based bounding box filtering algorithm is introduced to remove low-quality pseudo-labels. Specifically, we define a confidence threshold τ; only pseudo-labels with a confidence higher than τ are retained. If a sample does not have a pseudo-label with a confidence higher than τ, it is temporarily not used in this round of model training. The filtered high-quality pseudo-label data is then strongly augmented to obtain dataset D2. Finally, datasets D1 and D2 are used together for model training, and the trained model is used to replace and update T1, resulting in a defect detector T2 with new weights. This process iterates through each step of the semi-supervised stage until a stopping condition is met, resulting in a surface defect detector T after n updates. n+1 The semi-supervised stage uses both labeled and unlabeled data to train the model. The loss in this stage is called the semi-supervised loss, which is the sum of the supervised and unsupervised losses. The semi-supervised loss is defined as follows:

[0061]

[0062] Among them, L s It is the monitoring loss, λ u It is the unsupervised loss weight, used to balance supervised loss and unsupervised loss.

[0063] The solution in this embodiment can make full use of the information of unlabeled samples by generating pseudo-labels for unlabeled samples. Therefore, a small number of labeled and unlabeled samples can be used for model training at the same time, which greatly reduces the workload of manual labeling while achieving good detection performance.

[0064] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A surface defect detection method based on a semi-supervised training strategy, characterized in that, The method includes the following steps: S1. Build a surface image acquisition system to acquire surface images and construct a surface defect dataset; S2, divide the surface defect dataset into unlabeled and labeled datasets according to a set ratio; S3, Use the Labelimg tool to annotate the images in the labeled dataset; S4, Divide the labeled dataset into training and test sets according to a set ratio; S5, Modify the YOLOv7 model to obtain the YOLOv7-EA model; the YOLOv7-EA model includes Backbone, Neck, and Head, and includes the following sub-steps: In Backbone, S51 uses deformable convolution to replace the 3 × 3 ordinary convolution in the ELAN module, that is, it replaces the original ELAN module with the newly formed DCN-ELAN. In S52, the original SPPCSPC structure is modified into an SPPFCSPC structure, and an external attention mechanism is introduced to form an SPPFEA structure. That is, the original SPPCSPC structure is replaced by the newly formed SPPFEA. SPPFEA replaces the 5×5, 9×9, and 13×13 max pooling layers in the SPPCSPC structure with three cascaded 5×5 max pooling layers. S53 replaces the original CIoU in the loss function of the YOLOv7 model with WIoU. The formula for WIoU is as follows: ； ； ； in, This will significantly amplify the size of ordinary quality anchor boxes. ; When a high-quality anchor box coincides with the target box, it will significantly reduce [the impact of the error]. And its focus on the distance between the two center points; and These are the center coordinates of the bounding box; and These are the center coordinates of the target bounding box; and These are the width and height of the smallest package box, respectively; and These are the width and height of the overlapping area between the bounding box and the target box, respectively. It is the area of ​​the combined region of the bounding box and the target box; S6. In the supervised phase, the yolov7-ea model is trained on the labeled dataset and the trained yolov7-ea model is used as a pseudo-label generator. In S7, during the semi-supervised stage, the yolov7-ea model is used as the detection model. A pseudo-label generator is used to infer pseudo-labels from unlabeled data. The detection model is trained by combining labeled data and unlabeled data with pseudo-labels, and the pseudo-label generator is updated in real time with the best-performing detection model.

2. The surface defect detection method based on a semi-supervised training strategy according to claim 1, characterized in that: In step S1, the surface image acquisition system includes an LCD screen, a CCD camera, a sample to be tested, and a computer.

3. A surface defect detection method based on a semi-supervised training strategy according to claim 1 or 2, characterized in that: The ratio set in S2 is 9:1; the ratio set in S4 is 7:

3.

4. A surface defect detection method based on a semi-supervised training strategy according to claim 1 or 2, characterized in that: In step S6, the supervised training phase uses labeled data to train the model, and the loss in this phase is the supervised loss, i.e. The monitoring loss is defined as: ； in, For object loss, For classifying losses, For bounding box loss.

5. The surface defect detection method based on a semi-supervised training strategy according to claim 4, characterized in that: In S7, the semi-supervised stage uses both labeled and unlabeled data to train the model. The loss in this stage is the semi-supervised loss, which is the sum of the supervised loss and the unsupervised loss. The semi-supervised loss is defined as follows: ； in, It is a loss of supervision. It is the unsupervised loss weight, used to balance supervised loss and unsupervised loss.

Images