A solar panel defect detection method based on improved yolov5

Abstract

The application relates to the technical field of digital images, in particular to a solar cell panel defect detection method based on an improved Yolov5, and mainly relates to the following technology: obtaining a target detection data set, pre-processing the target detection data set, and obtaining a research data set; dividing the research data set to obtain a training data set and a test data set; respectively improving a C3 module of a main network part of the Yolov5, a Conv network in the Yolov5 main network and a Yolov5 prediction layer network to obtain an improved Yolov5 model; putting the training data set into the improved Yolov5 model for training, obtaining the best weight after the training; putting the best weight into the improved Yolov5 model to obtain a solar cell panel defect detection model; and using the solar cell panel defect detection model for defect detection of a solar cell panel under real working conditions.

Description

Technical Field

[0001] This invention relates to the field of digital image technology, and more specifically to a defect detection method for solar panels based on an improved Yolov5 architecture. Background Technology

[0002] Due to the complex manufacturing process of solar panels, defects such as scratches, nicks, black spots, black edges, and lack of electricity are easily produced during the manufacturing process. The presence of these defects greatly reduces the conversion efficiency of solar panels and also shortens their lifespan. Therefore, it is particularly important to inspect the surface of solar panels for defects before they leave the factory.

[0003] Currently, deep learning-based object detection algorithms fall into two categories: two-stage and one-stage. Two-stage algorithms first generate candidate bounding boxes and then classify them using a convolutional neural network. Common two-stage algorithms include R-CNN and Faster R-CNN. One-stage algorithms directly transform the object localization problem into a regression problem. Unlike two-stage algorithms, one-stage algorithms do not require candidate bounding boxes. Common one-stage algorithms include the YOLO series and SSD. While current networks can be used for defect detection in solar panels, their accuracy still has significant room for improvement. Summary of the Invention

[0004] The purpose of this invention is to develop a defect detection method for solar panels based on the improved Yolov5, thereby solving the problems mentioned in the background art.

[0005] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0006] This invention provides a defect detection method for solar panels based on an improved Yolov5 solar cell, comprising the following steps:

[0007] Obtain the object detection dataset and preprocess it to obtain the research dataset;

[0008] The research dataset is divided into a training dataset and a test dataset;

[0009] The C3 module of the YOLOv5 backbone network, the Conv network in the YOLOv5 backbone, and the YOLOv5 prediction layer network are improved to obtain the improved YOLOv5 model.

[0010] The training dataset is fed into the improved Yolov5 model for training, and the best weights are obtained after training.

[0011] The optimal weights are then incorporated into the improved Yolov5 model to obtain the solar panel defect detection model.

[0012] The solar panel defect detection model is used for defect detection of solar cells under real working conditions.

[0013] Furthermore, the target detection dataset is preprocessed, including:

[0014] Select the required defect images from the target detection dataset and merge them into a new dataset; mirror the new dataset to increase the number of images, and annotate the defects in the images to obtain the research dataset.

[0015] Furthermore, the C3 module of the Yolov5 backbone network is improved by: adding the ParNet module to the Res2Net module to obtain the improved Res2Net module; and adding the improved Res2Net module to the C3 module.

[0016] Furthermore, the improved Res2Net module is added to the C3 module, including replacing the BottleNeck part in the C3 module with the improved Res2Net.

[0017] Furthermore, adding the ParNet module to the Res2Net module includes: changing the original ResNet feature map from one path to four paths for transmission: the first path x1 is transmitted directly to y1 without any processing; the second path x2 is processed by a 3*3 convolution, then two paths are selected, one path continues to propagate downwards to y2, and the other path is transmitted to x3, so that the third path receives information from the second path; the third path x3 is processed by a 3*3 convolution, then two paths are selected, one path continues to propagate downwards to y3, and the other path is transmitted to x4, so that the fourth path receives information from the third path; the fourth path x4 is processed by a 3*3 convolution and does not require processing, then is directly transmitted to y4; the ParNet module is added after the last convolution to obtain the improved Res2Net module.

[0018] Furthermore, the Conv network in the Yolov5 backbone is improved by replacing the ReLU activation function in the ODConv network with the RReLU activation function to obtain the improved ODConv; and replacing the Conv network in the Yolov5 backbone with the improved ODConv.

[0019] Furthermore, the YOLOv5 prediction layer network is improved by introducing an adaptive feature fusion structure before the YOLOv5 prediction network.

[0020] Furthermore, the introduced adaptive feature fusion structure includes identity scaling and adaptive fusion.

[0021] Furthermore, the introduction of the adaptive feature fusion structure includes the following steps: combining the semantic features of Level 1, Level 2, and Level 3 with the weights α, β, and γ from different layers according to the formula... The calculation yields adaptive feature fusion, where, Indicates the weights of different layers;

[0022] Adaptive feature fusion is added before the prediction layer of the Yolov5 prediction network, enabling each spatial location to automatically learn the fusion weights between features of different sizes.

[0023] Furthermore, the aforementioned The calculation method is as follows:

[0024]

[0025]

[0026]

[0027] In summary, the present invention has the following beneficial effects:

[0028] 1. By using a self-made dataset, the data required for the experiment can better meet the experimental environment. Furthermore, by enhancing the mirror flipping algorithm, the number of sample images is increased, which is beneficial to improving detection accuracy.

[0029] 2. This invention improves feature extraction and reduces the number of detection parameters by improving the ODConv network and replacing the Conv network in the Yolov5 backbone, thereby improving the detection accuracy by 4.6%.

[0030] 3. Improve Res2Ne by adding the ParNet module to the last convolutional part of Res2Ne, and replace the BottleNeck part in the C3 module of the Yolov5 backbone with the improved Res2Ne to increase the receptive field and improve the detection accuracy by 1.7%.

[0031] 4. By adding an adaptive fusion structure before the prediction layer, the feature fusion capability is enhanced, improving the detection accuracy by 5.5%.

[0032] 5. By adding the improved ODConv and the improved C3 to the Yolov5 backbone and adding the adaptive fusion structure before the Yolov5 prediction layer, the detection accuracy was improved to 8.3%. Attached Figure Description

[0033] Figure 1 This is a flowchart of the testing method;

[0034] Figure 2 A flowchart for obtaining the research dataset;

[0035] Figure 3 Original battery image

[0036] Figure 4 This is an introduction to the defects.

[0037] Figure 5 A schematic diagram of the improved Yolov5 structure;

[0038] Figure 6 To improve the flowchart of Yolov5;

[0039] Figure 7 This is a schematic diagram of the Res2Net module structure;

[0040] Figure 8 A schematic diagram of the structure after adding the ParNet module to Res2Net;

[0041] Figure 9 This is a schematic diagram of the improved C3 module structure;

[0042] Figure 10 A schematic diagram of the improved ODConv structure;

[0043] Figure 11 This is a schematic diagram of the adaptive fusion structure;

[0044] Figure 12 The image shows the comparison results of the detection, where a represents the detection result before the improvement and b represents the comparison result after the improvement. Detailed Implementation

[0045] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the specific implementation method, features and effects of a solar panel defect detection method based on the improved YOLOv5 proposed in accordance with the present invention are described in detail below.

[0046] This specific embodiment provides a defect detection method for solar panels based on an improved Yolov5 architecture, such as... Figure 1 As shown, the operation steps are as follows:

[0047] S1. Obtain the object detection dataset and preprocess it to obtain the research dataset;

[0048] S2. Divide the research dataset into training dataset and test dataset;

[0049] S3. Improve the C3 module of the YOLOv5 backbone network, the Conv network in the YOLOv5 backbone, and the YOLOv5 prediction layer network to obtain the improved YOLOv5 model.

[0050] S4. Feed the training dataset into the improved Yolov5 model for training, and obtain the best weights after training;

[0051] S5. Input the best weights into the improved Yolov5 model to obtain the solar panel defect detection model;

[0052] S6. Apply the solar panel defect detection model to detect defects in solar panels under real working conditions.

[0053] In this specific implementation, on the one hand, by improving the Conv and C3 modules of the Yolov5 backbone, the ability to extract feature information is improved, thereby improving the detection accuracy; on the other hand, by adding an adaptive feature fusion structure in front of the detection layer, the ability to fuse different features is enhanced, thereby improving the overall network's ability to detect defects; overall, the improved Yolov5 model improves the detection ability of solar panel defects by 8.3%.

[0054] In step S1, the target detection dataset is preprocessed to obtain the research dataset, such as... Figure 2 As shown, it includes the following steps:

[0055] S11. Select the required defect images from the target detection dataset and merge them into a new dataset;

[0056] S12. Mirror the new dataset to increase the number of images, and label the defects in the images to obtain the research dataset.

[0057] In this specific embodiment, an industrial camera is used to photograph the produced solar cells, and the original images of the solar cells are as follows: Figure 3As shown, the images are grouped together to create a target detection dataset. The images in the target detection dataset are cropped. The original image is a 4043 pixel * 643 pixel image of a solar panel module. Because the original image is rectangular and long, and contains some unwanted parts, the image is manually cropped to isolate the defective parts that need to be detected. The cropped images are uniformly 600 pixel * 600 pixel in size. The cropped images are then enhanced using a mirror flip algorithm to obtain the new dataset in this specific implementation. The LabelImg tool is used to label the images in the new dataset, highlighting the defects in the images, such as... Figure 4 As shown, the research dataset was obtained.

[0058] In this specific embodiment, the target detection dataset contains a total of 1203 images. After the above cropping and mirroring processes, the number of images increases to 2406, which is used as the research dataset. The research dataset is divided into a training set and a test set in an 8:2 ratio, resulting in a training set of 1925 images and a test set of 481 images.

[0059] In this specific embodiment, step S3 constructs an improved Yolov5, the structure of which is as follows: Figure 5 As shown. To improve the accuracy of solar panel defect detection, an improved ODConv (full-dimensional dynamic network) is added to layers 1, 3, 5, and 7 of the Backbone. Through its progressive manipulation of convolution along dimensions such as position, channel, filter, and kernel, it can better capture rich contextual information, thereby significantly improving the feature extraction capability of convolution.

[0060] The C3 module in BackBone was replaced with the improved C3_H module, which improved the accuracy of solar panel defect detection. Finally, ASFF adaptive feature fusion was introduced before the prediction network. This involves weighted fusion by setting self-learned weights for each fused feature map, thus making more rational use of high-level semantic information and low-level features. Figure 6 As shown, the specific operation of step S3 is as follows:

[0061] S31. Add the ParNet module to the Res2Net module to obtain the improved Res2Net module;

[0062] S32. Add the improved Res2Net module to the C3 module;

[0063] S33. Replace the ReLU activation function in the ONConv network with the RReLU activation function to obtain the improved ODConv;

[0064] S34. Replace the Conv network in the Yolov5 backbone with the improved ODConv;

[0065] S35. The semantic features of Level 1, Level 2, and Level 3 are calculated with the weights α, β, and γ from different layers to obtain adaptive feature fusion.

[0066] S36. Adaptive feature fusion is added before the prediction layer of the YOLOv5 prediction layer network, so that each spatial location automatically learns the fusion weights between features of different sizes, resulting in an improved YOLOv5 model.

[0067] The specific operation of step S31 is as follows: The feature map of the original ResNet is converted from one line to four lines for transmission: The first line x1 is transmitted directly to y1 without any processing; the second line x2 is processed by a 3*3 convolution and then two lines are selected, one line continues to propagate downwards to y2, and the other line is transmitted to x3, so that the third line receives the information of the second line; the third line x3 is processed by a 3*3 convolution and then two lines are selected, one line continues to propagate downwards to y3, and the other line is transmitted to x4, so that the fourth line receives the information of the third line; the fourth line x4 is processed by a 3*3 convolution and, like line 1, is transmitted directly to y4 without any processing; a ParNet module is added after the last convolution to obtain the improved Res2Net module, as shown below. Figure 8 As shown, the receptive field was increased, thus improving detection accuracy.

[0068] Specifically, such as Figure 7 As shown, suppose xi, where i∈{1,2,3,…,s}, then Figure 6 In the equation s = 4 (dividing x into four parts, so s = 4), K i () represents a 3x3 convolution, so the output yi is:

[0069]

[0070] Adding the ParNet module after the last 1*1 convolution increases the number of parameters in the improved module, but also further increases the receptive field, thereby improving the accuracy of defect detection. The improvement effect is shown in Table 1.

[0071] Table 1. Comparison of the improvement in detection accuracy brought about by the improved ODConv structure

[0072] Yolov5s 70.6 76.9 Yolov5s+ODConv 72.8 73.0 Yolov5s+ODConv(RRelu) 75.2 77.9 Yolov5s+ODConv(Softsign) 73.9 73.0 Yolov5s+ODConv(ELU) 74.4 73.6 Yolov5s+ODConv(Hardswish) 74.0 73.7 Yolov5s+ODConv(Sigmoid) 72.5 71.7

[0073] The specific operation in step S32 is as follows: replace the BottleNeck part in module C3 with the improved Res2Net, and the result is as follows. Figure 9 As shown, the left side is a schematic diagram of the improved C3 structure, where the improved Res2Net module replaces the BottleNeck part of the original network. The right side, within the dashed box, shows the improved Res2Net structure. After replacing the BottleNeck in the original Yolov5 C3 module, the receptive field is significantly increased compared to the original network, thereby optimizing the convolutional neural network's ability to detect targets and improving the accuracy of defect detection on solar panels.

[0074] In this specific implementation, such as Figure 10 As shown, ODConv utilizes a novel multidimensional attention mechanism to compute four types of attention a in parallel along all four dimensions of the kernel space. si a ci , afi and a wi , of which, (1)a si At k*k spatial locations, assign different amounts of attention to the convolution parameters (each filter); (2)a ci For each convolution filter W i m c in Channels are assigned different scalars of interest; (3)a fi For c out Convolutional filters are assigned different attention scalars; (4)a wi Distribute the attention scalar across the entire convolution kernel.

[0075] Full-Dimensional Dynamic Convolution (ODConv) primarily leverages the three dimensions (spatial size, number of input channels, and number of output channels) that other convolutions ignore. In any convolutional layer, ODConv utilizes a novel multidimensional attention mechanism to learn four types of attention for the convolutional kernel in parallel along all four dimensions of the kernel space.

[0076] ODConv learns these four types of attention in a complementary way. Gradually applying them to the corresponding convolutional kernels can enhance the feature extraction capability of basic convolutional operations in CNNs. In this specific implementation, the ReLU activation function in the original network is replaced with the RReLU activation function, which can better extract useful defect information in the detection of defects in solar cells.

[0077] In this specific embodiment, as in step S34, the original Yolov5 Conv network in the Yolov5 backbone is replaced with the improved ODConv, which can improve the model accuracy to varying degrees. The improvement effect is shown in Table 1.

[0078] In this specific embodiment, step S35 involves introducing an adaptive feature fusion structure before the Yolov5 prediction network, and the operation steps are as follows:

[0079] S351. Combine the semantic features of Level 1, Level 2, and Level 3 with the weights α, β, and γ from different levels according to the formula. The calculation yields adaptive feature fusion, where, Indicates the weights of different layers;

[0080] S352. Adaptive feature fusion is added before the prediction layer of the Yolov5 prediction layer network, so that each spatial location automatically learns the fusion weights between features of different sizes.

[0081] in, The calculation method is as follows:

[0082]

[0083]

[0084]

[0085] To fully utilize the semantic information of high-level features and the linear strength features of low-level features, many existing object detection algorithms typically employ the Feature Fusion Network (FPN) architecture, a relatively effective feature fusion method. The adaptive feature fusion mechanism used in this specific implementation allows each spatial location to automatically learn the fusion weights between features of different sizes during training, thereby making more rational use of multi-scale feature information. Its network structure is as follows: Figure 11 As shown.

[0086] In this specific embodiment, in step S4, the training set is put into the improved Yolov5 model and run in PyCharm. For multiple improvements, multiple training sessions are conducted separately to obtain ablation experiments. The experimental data is shown in Table 2. The best weights are obtained, and it can be seen from the ablation experiments in Table 2 that the detection accuracy of solar panel defects has been improved to a certain extent after the improvements.

[0087] Table 2. Ablation Experiment

[0088] Yolov5s 70.6 73.8 76.9 Yolov5s+ODConv 75.2 77.0 77.9 Yolov5s+CH_3 72.3 78.1 77.5 Yolov5s+ASFF 76.1 76.4 77.1 Yolov5s+ODConv+CH_3 71.3 78.2 77.3 Yolov5s+ODConv+ASFF 75.9 76.0 77.5 Yolov5s+CH_3+ASFF 72.6 78.3 78.5 This specific implementation method 78.9 78.3 80.6

[0089] Step S5 of this specific embodiment involves substituting the best weights obtained above into the improved YOLOv5 model to obtain the solar panel defect detection model of this specific embodiment.

[0090] Furthermore, in this specific embodiment, step S6 uses the solar panel defect detection model obtained above to detect the test set, and the detection results are as follows: Figure 12 As shown. The test set was then tested using the unmodified Yolov5 model, and the results are also shown in [the original text]. Figure 11 ;according to Figure 11 It can be seen that the improved network can detect defects that were missed in the original network as well as some less obvious defects. The improved network has improved the accuracy of detecting various defects under the same environment, and the accuracy of defect bounding box regression is also improved compared with the original network.

[0091] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been shown above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A defect detection method for solar panels based on an improved Yolov5 solar cell, characterized in that, The following steps are included: Obtain the object detection dataset and preprocess the object detection dataset to obtain the research dataset, wherein the object detection dataset is an image dataset; The research dataset is divided into a training dataset and a test dataset; The C3 module of the YOLOv5 backbone network, the Conv network in the YOLOv5 backbone, and the YOLOv5 prediction layer network are improved to obtain the improved YOLOv5 model. The training dataset is fed into the improved Yolov5 model for training, and the best weights are obtained after training. The optimal weights are then incorporated into the improved Yolov5 model to obtain the solar panel defect detection model. The solar panel defect detection model is used for defect detection of solar panels under real working conditions. The C3 module of the Yolov5 backbone network is improved by: adding the ParNet module to the Res2Net module to obtain the improved Res2Net module; and adding the improved Res2Net module to the C3 module. Adding the improved Res2Net module to the C3 module includes replacing the BottleNeck part in the C3 module with the improved Res2Net; Adding the ParNet module to the Res2Net module includes: splitting the original ResNet feature map from one line into four lines for transmission: the first line x1 is transmitted directly to y1 without any processing; the second line is x2 processed by... After convolution, two paths are selected: one path continues propagating downwards to y2, and the other path is passed to x3, allowing the third path to receive information from the second path; the third path x3 passes through... After convolution, two paths are selected: one path continues propagating downwards to y3, and the other path is propagated to x4, allowing the fourth path to receive information from the third path; the fourth path x4 passes through... No further processing is needed after the convolution; it is directly passed to y4. A ParNet module is added after the last convolution to obtain an improved Res2Net module. The Conv network in the Yolov5 backbone is improved by replacing the ReLU activation function in the ODConv network with the RReLU activation function to obtain the improved ODConv; and replacing the Conv network in the Yolov5 backbone with the improved ODConv.

2. The method for detecting defects in solar panels based on improved Yolov5 according to claim 1, characterized in that, Preprocessing of the target detection dataset includes: Select the required defect images from the target detection dataset and merge them into a new dataset; mirror the new dataset to increase the number of images, and annotate the defects in the images to obtain the research dataset.

3. The method for detecting defects in solar panels based on improved Yolov5 according to claim 1, characterized in that, The improvement to the YOLOv5 prediction layer network is to introduce an adaptive feature fusion structure before the YOLOv5 prediction network.

4. The method for detecting defects in solar panels based on the improved Yolov5 according to claim 3, characterized in that, The introduction of the adaptive feature fusion structure includes the following steps: combining the semantic features of Level 1, Level 2, and Level 3 with weights from different layers. According to the formula The calculation yields adaptive feature fusion, where, , , , representing the weights of different layers; Adaptive feature fusion is added before the prediction layer of the Yolov5 prediction network, enabling each spatial location to automatically learn the fusion weights between features of different sizes.

5. The method for detecting defects in solar panels based on the improved Yolov5 according to claim 1, characterized in that, The , , The calculation method is as follows: ； ； 。

Images