Defect detection method combined with unsupervised image feature extraction technology

Abstract

The invention relates to a defect detection method combined with an unsupervised image feature extraction technology, and the method comprises the steps: collecting a qualified image of a product as training data of an unsupervised convolution auto-encoder, and obtaining a feature extractor; defect images are collected and marked; inputting the defect image in the training set into YOLOv8 for training to obtain a multi-scale feature map; inputting the same defect image into a feature extractor to obtain an unsupervised feature map; fusing the unsupervised feature map with the multi-scale feature map; the fused image is subjected to feature combination through a neck network, and a detection result is output through a head network; optimizing the model by using the test set, and evaluating a detection result output by the model by using the verification set; and carrying out product defect detection by utilizing the trained model. According to the method, the global features of the images are captured from a large number of defect-free images, effective feature representation under current imaging is formed, and the problems of model over-fitting and substandard model performance caused by defect data deficiency are avoided.

Description

Technical Field

[0001] The present invention relates to the field of defect detection technology, and in particular to a defect detection method combined with unsupervised image feature extraction technology. Background Art

[0002] With the transformation of the global energy structure and the enhancement of environmental protection awareness, solar power generation, as a clean and renewable energy source, has received widespread attention. As the core component of the solar power generation system, the manufacturing quality of solar cells is directly related to the power generation efficiency and component life, especially the surface quality of solar cells, which directly affects the overall performance of photovoltaic modules. However, due to the complex production process, solar cells are prone to various defects during the manufacturing process, such as edge collapse, hidden cracks, missing corners, decrystalline, etc. These defects will produce fragments in subsequent processes, leading to jamming, which will not only interrupt the production process, but also waste precious raw materials and increase production costs. Therefore, how to quickly and accurately detect surface defects of solar cells has become a key issue that needs to be solved in the photovoltaic industry.

[0003] In recent years, with the rapid development of computer vision technology and artificial intelligence algorithms, solar cell defect detection technology based on deep learning image processing has gradually emerged. By capturing images of the surface of the cell with a high-definition camera and analyzing them with advanced image processing algorithms, various types of defects can be effectively identified. This method not only improves the speed and accuracy of detection, but also greatly reduces labor costs, providing strong support for the large-scale production of solar cells. However, with the continuous improvement of detection requirements, existing defect detection methods face the problem of time-consuming and labor-intensive collection of defect sample data under traditional methods, resulting in a long online deployment cycle, which seriously delays the application speed required by enterprises.

[0004] At present, some literatures have expanded the data set based on the data enhancement algorithm of the GAN network, which alleviates the data dilemma of small defect data in industrial scenarios, and the high cost of collection, cleaning and labeling. However, in actual production, it is often difficult to collect training data sets that cover all possible defect categories and complexity for training GAN networks, and it is difficult to generate a sufficient number of diverse and high-quality defect image samples. Similarly, it is difficult to ensure the sample balance of the training data set, which makes the trained GAN network prone to "mode collapse", that is, the generator tends to generate only a few or one defect sample. Since the effect of defect detection depends on the quality of defect generation, the above problems can easily lead to overfitting of the detection model, that is, it performs well on training samples and has high indicators, but the effect is poor when tested in actual scenarios. Summary of the invention

[0005] The present invention provides a defect detection method combined with unsupervised image feature extraction technology to solve the above technical problems.

[0006] In order to solve the above technical problems, the present invention provides a defect detection method combined with unsupervised image feature extraction technology, comprising the following steps:

[0007] Step 1: Collect qualified images of defect-free products as training data for an unsupervised convolutional autoencoder, wherein the unsupervised convolutional autoencoder includes an encoder and a decoder; the encoder is used to encode the input qualified image into a low-dimensional feature vector, and the decoder is used to re-decode the low-dimensional feature vector into a reconstructed image, and the encoder after training is used as a feature extractor;

[0008] Step 2: Collect defect images and annotate them, and divide the defect images into a training set, a test set, and a validation set;

[0009] Step 3: Input the defect image in the training set into YOLOv8 for training, wherein the YOLOv8 includes a backbone network, a neck network, and a head network in sequence, and obtains a multi-scale feature map after the defect image passes through the backbone network; input the same defect image into the feature extractor to extract an unsupervised feature map of the defect image; and use an attention mechanism to fuse the unsupervised feature map with the multi-scale feature map;

[0010] Step 4: The fused image is subjected to feature combination by the neck network, and the detection result is outputted by the head network, wherein the detection result includes at least the bounding box and category of each defect;

[0011] Step 5: Optimize the model using the test set, and evaluate the detection results output by the model using the validation set;

[0012] Step 6: Use the trained model to detect product defects.

[0013] Preferably, step 1 further comprises preprocessing the collected qualified image, and step 2 further comprises preprocessing the collected defective image, wherein the preprocessing comprises at least boundary correction and scaling.

[0014] Preferably, during the training of the unsupervised convolutional autoencoder, L2 loss is used as the pixel-level reconstruction loss.

[0015] Preferably, in step 2, the defect location and category are labeled using the LabelImg labeling tool.

[0016] Preferably, in step 2, the training set, test set and validation set are divided into the ratio of 6:2:2 based on the category-based stratified sampling method.

[0017] Preferably, step 3 comprises:

[0018] Step 3.1: Input the defect image into the backbone network, and obtain feature maps of three different scales after 8, 16, and 32 times downsampling by convolution operation;

[0019] Step 3.2: For each scale feature map, perform global average pooling on its channel dimension to obtain the global feature vector of each scale;

[0020] Step 3.3: concatenate the feature vector extracted by the feature extractor with the global feature vector of each scale obtained in step 3.2 to form a fused global feature representation;

[0021] Step 3.4: Use a multi-layer perceptron to process the global feature representation obtained in step 3.3 and output an attention weight vector;

[0022] Step 3.5: Use the attention weight to perform weighted fusion of the unsupervised feature map with the feature map of each scale of YOLOv8.

[0023] Preferably, the parameters of YOLOv8 are set as follows: training size is 1280, initial learning rate is 0.01, momentum coefficient is 0.937, weight decay coefficient is 0.005, batch size is 128, number of iterations is 300, including 3 warm-up times, using stochastic gradient descent optimizer and cosine annealing learning rate adjustment strategy.

[0024] Preferably, step 4 also includes: freezing the encoder and optimizing model parameters using a standard target detection loss function of YOLOv8.

[0025] Preferably, in step 4, the feature combination of the fused image through the neck network includes: splicing feature maps of different scales.

[0026] Preferably, in step 4, the detection result also includes a confidence score.

[0027] Compared with the prior art, the defect detection method combined with unsupervised image feature extraction technology provided by the present invention has the following advantages:

[0028] 1. Aiming at the situation that there are more normal samples than defective samples in industrial scenes, the present invention adopts an unsupervised convolutional autoencoder to capture the global features of images from a large number of defect-free images, and extracts the imaging feature information of the scene through the encoder, thereby obtaining an effective feature representation under the current imaging;

[0029] 2. The present invention improves the accuracy and generalization ability of the model by integrating the features with YOLOv8, effectively responds to the defect detection needs in complex process flow environments, avoids the problems of model overfitting and substandard model performance caused by lack of defect data, reduces dependence on labeled data, and thus reduces training costs and time. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flowchart of a defect detection method combined with unsupervised image feature extraction technology in a specific embodiment of the present invention;

[0031] Figure 2a and 2b They are defect detection effect diagrams of the original YOLOv8 network model and the network model provided by the present invention on the test set. DETAILED DESCRIPTION

[0032] In order to describe the technical solution of the above invention in more detail, specific embodiments are listed below to demonstrate the technical effects; it should be emphasized that these embodiments are used to illustrate the present invention but not to limit the scope of the present invention.

[0033] The defect detection method provided by the present invention combines unsupervised image feature extraction technology, such as Figure 1 As shown, the following steps are included:

[0034] Step 1: Collect a large number of qualified images of defect-free products as training data for an unsupervised convolutional autoencoder, wherein the unsupervised convolutional autoencoder includes an encoder and a decoder; the encoder is used to encode the input qualified image into a low-dimensional feature vector, and the decoder is used to re-decode the low-dimensional feature vector into a reconstructed image of the input image, and the encoder after training is used as a feature extractor;

[0035] Step 2: Collect defect images containing various defects and accurately annotate each image, and divide the defect images into a training set, a test set, and a verification set;

[0036] Step 3: Input the defect image in the training set into YOLOv8 for training, wherein the YOLOv8 includes a backbone network, a neck network, and a head network in sequence, and obtains a multi-scale feature map after the defect image passes through the backbone network; input the same defect image into the feature extractor to extract an unsupervised feature map of the defect image; and use an attention mechanism to fuse the unsupervised feature map with the multi-scale feature map;

[0037] Step 4: The fused image is further subjected to feature combination by the neck network, and the detection result is outputted by the head network, wherein the detection result includes at least the bounding box and category of each defect;

[0038] Step 5: Optimize the model using the test set, and evaluate the detection results output by the model using the validation set;

[0039] Step 6: Use the trained model to detect product defects.

[0040] Aiming at the situation where there are more normal samples than defective samples in industrial scenarios, the present invention adopts an unsupervised convolutional autoencoder to capture the global features of images from a large number of defect-free images, and extracts the imaging feature information of the scene through the encoder, so as to obtain the effective feature representation under the current imaging; by fusion with the features of YOLOv8, the accuracy and generalization ability of the model are improved, which effectively meets the defect detection needs in complex process flow environments, avoids the problems of model overfitting and substandard model performance caused by the lack of defect data, reduces the dependence on labeled data, and thus reduces the training cost and time.

[0041] In some embodiments, step 1 further includes preprocessing the collected qualified images, and the preprocessing includes at least boundary correction and scaling. In this embodiment, the original image size collected by the camera is 3000*3000, which becomes 1280*1280 after scaling, thereby eliminating the influence of different postures of the product when shooting, making the collected images comparable, and image scaling reduces memory usage during training.

[0042] In some embodiments, during the training of the unsupervised convolutional autoencoder, L2 loss is used as the pixel-level reconstruction loss to ensure that the difference between the decoder output and the original image is minimized, thereby improving the generalization ability of the model.

[0043] In some embodiments, the defective image acquired in step 2 also needs to be preprocessed in the same manner as the qualified image in step 1.

[0044] In some embodiments, in step 2, the LabelImg annotation tool can be used to annotate the defect location and category, and precise annotation enables the model to more accurately understand and locate defects in the image, thereby improving the performance of the model.

[0045] In some embodiments, in step 2, the training set, test set, and validation set are divided into the training set, test set, and validation set in a ratio of 6:2:2 based on the category-based stratified sampling method. Based on the validation set and the test set, the accuracy, recall rate, and other indicators of the model can be calculated to understand the performance of the model, thereby optimizing the model more effectively.

[0046] In some embodiments, step 3 includes:

[0047] Step 3.1: Input the defect image into the backbone network, and obtain three feature maps of different scales after 8, 16, and 32 times downsampling through convolution operation. This application adopts a multi-scale prediction strategy to downsample the feature map at different ratios, which can better capture objects of different sizes on feature maps at different levels.

[0048] Step 3.2: For the feature map of each scale, perform global average pooling on its channel dimension to obtain the global feature vector of each scale, and compress its spatial dimension into a global feature vector.

[0049] Step 3.3: The feature vector extracted by the feature extractor is concatenated with the global feature vector of each scale obtained in step 3.2 to form a fused global feature representation.

[0050] Step 3.4: Use a multi-layer perceptron to process the global feature representation obtained in step 3.3 and output an attention weight vector. The attention weight can control the degree of fusion of the features at each scale with the YOLOv8 features.

[0051] Step 3.5: Use the attention weight to perform weighted fusion of the unsupervised feature map with the feature map of each scale of YOLOv8, and enhance the feature expression ability in a weighted manner.

[0052] This application balances the unsupervised feature learning and the feature expression of YOLOv8 through the dynamic weighted fusion of the attention mechanism, so that the model can adaptively adjust according to the features of the input image and further optimize the defect detection effect.

[0053] In some embodiments, the parameters of YOLOv8 are set as follows: training size is 1280, initial learning rate is 0.01, momentum coefficient is 0.937, weight decay coefficient is 0.005, batch size is 128, number of iterations is 300, of which the number of warm-up times is 3 times, and a stochastic gradient descent optimizer and a cosine annealing learning rate adjustment strategy are used. By setting the parameters during model training, the model can better converge to the optimal solution on the weighted fusion features.

[0054] In some embodiments, step 4 further includes: freezing the encoder and optimizing model parameters using a standard target detection loss function (including category loss and position loss) of YOLOv8, thereby preventing features extracted by the encoder from being changed during supervised training.

[0055] In some embodiments, in step 4, the fused image is subjected to feature combination through the neck network, including: splicing feature maps of different scales, generating detection results through the head network; the detection results may also include a confidence score, and then the difference between the predicted result and the true label is calculated based on the loss function of YOLOv8, so as to optimize the model parameters during the training process.

[0056] In some embodiments, in step 5, the trained model may be evaluated. In this embodiment, the performance of the model may be evaluated by the mAP performance indicator. After evaluating the detection performance of the model, the model may be actually tested and analyzed using a validation set.

[0057] The following takes the surface defect detection of solar cells as an example to introduce the implementation steps of the detection method in detail.

[0058] Step 1: Use a high-resolution infrared camera to collect about 5,000 images of defect-free solar cells on different production lines, use a contour extraction algorithm to obtain the boundaries of the solar cells in the image, and perform affine transformation correction. Then, the original image of size 3000*3000 collected by the camera is scaled to 1280*1280 for training the convolutional autoencoder to extract unsupervised features of the image. The encoder in the convolutional autoencoder compresses the input data into a low-dimensional hidden representation, and the decoder restores this hidden representation to the original data. By minimizing the reconstruction error, the convolutional autoencoder can learn the intrinsic structure and characteristics of the input data. In this embodiment, the data can be divided into a training set and a test set in a ratio of 8:2, the batch size is 128, the number of epochs is 100, and the L2 loss is used as the pixel-level reconstruction loss. After randomly initializing the network parameters of the encoder and decoder, the value of the L2 loss function is calculated, and finally the parameters of the model are updated through the back propagation algorithm. The above steps are repeated many times until the model converges or reaches the preset number of training rounds. After the training is completed, the encoder part is retained as a feature extractor, and any battery cell image is input, and the encoder can output the corresponding feature vector.

[0059] Step 2: Collect images of solar cells with various defects on different production lines, perform the same preprocessing as step 1, and then use the LabelImg annotation tool to annotate the location and category of all defects. After annotation, the images are divided into training set, test set and validation set according to the ratio of 6:2:2 using the category-based stratified sampling method, thus avoiding the imbalance of category distribution in the data set.

[0060] Step 3: Use the attention mechanism to fuse the feature map extracted by YOLOv8 with the unsupervised features extracted by the encoder, including:

[0061] Step 3.1: Set the YOLOv8 parameters to: training size is 1280, initial learning rate is 0.01, momentum coefficient is 0.937, weight decay coefficient is 0.005, batch size is 128, iteration number epochs is 300 times, of which warm-up epochs is 3 times, use stochastic gradient descent optimizer and cosine annealing learning rate adjustment strategy, input the defect image processed in step 2 into the backbone network of YOLOv8, obtain three different scales of feature maps after 8, 16, and 32 times downsampling through convolution operation, which are used to detect defective objects of different sizes in the data, and input the same image into the trained encoder (feature extractor) to extract the unsupervised feature vector of the image.

[0062] Step 3.2: Introduce the attention mechanism to let the model adaptively decide how to weight the features of the encoder and the features of YOLOv8. For each scale of the YOLOv8 feature map, perform global average pooling on its channel dimension to obtain the global feature vector of each scale, and compress its spatial dimension into a global feature vector.

[0063] Step 3.3: Concatenate the feature vector extracted by the feature extractor with the global feature vector of each scale obtained in step 3.2 to form a fused global feature representation.

[0064] Step 3.4: Use a multi-layer perceptron to process the global feature representation obtained in step 3.3 and output an attention weight vector. The attention weight can control the degree of fusion of unsupervised features and YOLOv8 features at each scale.

[0065] Step 3.5: Use the attention weights to fuse the global features extracted by the feature extractor with the feature maps of each scale of YOLOv8, and enhance the feature expression ability in a weighted manner.

[0066] Step 4: After freezing the features extracted by the encoder, the fused multi-scale feature map is sent to the neck network to splice the feature maps of different scales. Finally, the detection results are generated through the head network, including the bounding box, category label and confidence score of the target. The difference between the predicted result and the true label is then calculated based on the loss function of YOLOv8 to optimize the model parameters during the training process.

[0067] Step 5: Evaluate the trained model. In this embodiment, the performance of the model is evaluated by mAP. After evaluating the detection performance of the model, the model is actually tested and analyzed using the validation set until the model meets the preset requirements.

[0068] Step 6: Use the trained model to perform product defect detection. In this embodiment, the defect categories in the cell image at least include edge chipping, corner chipping, and decrystalline.

[0069] Depend on Figure 2a and 2b It can be seen that compared with the original YOLOv8 network model and the YOLOv8 network model combined with the unsupervised image feature extraction technology in the present invention, the method proposed in the present invention has a better defect detection effect on the test set.

[0070] In summary, the defect detection method combined with unsupervised image feature extraction technology provided by the present invention comprises the following steps: Step 1: Collect a large number of qualified images of defect-free products as training data of an unsupervised convolutional autoencoder, wherein the unsupervised convolutional autoencoder comprises an encoder and a decoder; use the encoder to encode the input qualified image into a low-dimensional feature vector, and then use the decoder to re-decode the low-dimensional feature vector into a reconstructed image of the input image, and use the encoder after training as a feature extractor; Step 2: Collect defective images containing various defects and accurately annotate each image, and divide the defective images into a training set, a test set and a verification set; Step 3: Input the defective images in the training set into YOLOv8 The YOLOv8 includes a backbone network, a neck network and a head network in sequence. After the defect image passes through the backbone network, a multi-scale feature map is obtained; the same defect image is input into the feature extractor to extract the unsupervised feature map of the defect image; the unsupervised feature map is fused with the multi-scale feature map using the attention mechanism; step 4: the fused image is further subjected to feature combination by the neck network, and the detection result is output through the head network, and the detection result at least includes the bounding box and category of each defect; step 5: the model is optimized using the test set, and the detection result output by the model is evaluated using the verification set; step 6: product defect detection is performed using the trained model. Aiming at the situation where there are more normal samples than defective samples in industrial scenarios, the present invention adopts an unsupervised convolutional autoencoder to capture the global features of images from a large number of defect-free images, and extracts the imaging feature information of the scene through the encoder, so as to obtain the effective feature representation under the current imaging; by fusion with the features of YOLOv8, the accuracy and generalization ability of the model are improved, which effectively meets the defect detection needs in complex process flow environments, avoids the problems of model overfitting and substandard model performance caused by the lack of defect data, reduces the dependence on labeled data, and thus reduces the training cost and time.

[0071] Obviously, those skilled in the art can make various changes and modifications to the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the invention fall within the scope of the claims of the invention and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A defect detection method combining unsupervised image feature extraction technology, characterized in that: The steps include: Step 1: Collect qualified images of defect-free products as training data for an unsupervised convolutional autoencoder, wherein the unsupervised convolutional autoencoder includes an encoder and a decoder; the encoder is used to encode the input qualified image into a low-dimensional feature vector, and the decoder is used to re-decode the low-dimensional feature vector into a reconstructed image, and the encoder after training is used as a feature extractor; Step 2: Collect defect images and annotate them, and divide the defect images into a training set, a test set, and a validation set; Step 3: Input the defect image in the training set into YOLOv8 for training, wherein the YOLOv8 includes a backbone network, a neck network, and a head network in sequence, and obtains a multi-scale feature map after the defect image passes through the backbone network; input the same defect image into the feature extractor to extract an unsupervised feature map of the defect image; and use an attention mechanism to fuse the unsupervised feature map with the multi-scale feature map; Step 4: The fused image is subjected to feature combination by the neck network, and the detection result is outputted by the head network, wherein the detection result includes at least the bounding box and category of each defect; Step 5: Optimize the model using the test set, and evaluate the detection results output by the model using the validation set; Step 6: Use the trained model to detect product defects.

2. The defect detection method combined with unsupervised image feature extraction technology as claimed in claim 1, characterized in that: Step 1 also includes preprocessing the collected qualified image, and step 2 also includes preprocessing the collected defective image, wherein the preprocessing at least includes boundary correction and scaling.

3. The defect detection method combined with unsupervised image feature extraction technology as claimed in claim 1, characterized in that: During the training of the unsupervised convolutional autoencoder, L2 loss is used as the pixel-level reconstruction loss.

4. The defect detection method combined with unsupervised image feature extraction technology as claimed in claim 1, characterized in that: In step 2, use the LabelImg labeling tool to label the defect location and category.

5. The defect detection method combined with unsupervised image feature extraction technology as claimed in claim 1, characterized in that: In step 2, the training set, test set, and validation set are divided into the ratio of 6:2:2 based on the category-based stratified sampling method.

6. The defect detection method combined with unsupervised image feature extraction technology as claimed in claim 1, characterized in that: Step 3 includes: Step 3.1: Input the defect image into the backbone network, and obtain feature maps of three different scales after 8, 16, and 32 times downsampling by convolution operation; Step 3.2: For each scale feature map, perform global average pooling on its channel dimension to obtain the global feature vector of each scale; Step 3.3: concatenate the feature vector extracted by the feature extractor with the global feature vector of each scale obtained in step 3.2 to form a fused global feature representation; Step 3.4: Use a multi-layer perceptron to process the global feature representation obtained in step 3.3 and output an attention weight vector; Step 3.5: Use the attention weight to perform weighted fusion of the unsupervised feature map with the feature map of each scale of YOLOv8.

7. The defect detection method combined with unsupervised image feature extraction technology as claimed in claim 1, characterized in that: The parameters of YOLOv8 are set as follows: training size is 1280, initial learning rate is 0.01, momentum coefficient is 0.937, weight decay coefficient is 0.005, batch size is 128, number of iterations is 300, including 3 warm-up times, using stochastic gradient descent optimizer and cosine annealing learning rate adjustment strategy.

8. The defect detection method combined with unsupervised image feature extraction technology as claimed in claim 1, characterized in that: Step 4 also includes: freezing the encoder and optimizing model parameters using a standard target detection loss function of YOLOv8.

9. The defect detection method combined with unsupervised image feature extraction technology as claimed in claim 1, characterized in that: In step 4, the fused image is subjected to feature combination by the neck network, including: splicing feature maps of different scales.

10. The defect detection method combined with unsupervised image feature extraction technology according to claim 1, characterized in that: In step 4, the detection result also includes a confidence score.

Images