Photovoltaic panel EL defect detection method based on improved YOLOv8 model

By improving the YOLOv8 model, using VanillaNet and variable nuclear convolution module, combined with optimized loss function and data enhancement technology, the problem of photovoltaic panel defect detection accuracy and slow speed is solved, achieving more efficient detection results and better Lu Great.

CN118967625APending Publication Date: 2024-11-15CHANGZHOU UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411055272.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing photovoltaic panel defect detection methods are not accurate and slow in detection speed. Especially the methods based on the faster-rcnn and Yolov5m models consume a lot of computing resources and memory, are difficult to deploy on mobile devices, and have poor practicality.

Method used

Using the improved YOLOv8 model, by replacing the Backbone network with VanillaNet, using variable kernel convolution module and WIoU_v3 loss function, combined with data enhancement technologies such as Mosaic, Mixup, hsv transformation and scale transformation, the loss function and gradient gain allocation strategy are optimized, and the loss function and gradient gain allocation strategy are improved. Detection accuracy and speed.

Benefits of technology

It improves detection accuracy and speed, reduces network parameters and calculation amount, improves the expression ability and robustness of the model, adapts to input characteristics of different shapes and sizes, and significantly improves the accuracy and practicality of photovoltaic panel defect detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967625A_ABST
    Figure CN118967625A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of image recognition and deep learning, in particular to a photovoltaic panel EL defect detection method based on an improved YOLOv8 model, and the method comprises the steps: obtaining a photovoltaic panel defect image, and constructing a photovoltaic panel defect image data set; an improved YOLOv8 model is constructed, a Backbone network is replaced with a VanilaNet network, a convolutional layer with a convolution kernel of 4 * 4 and a stride of 4 is used in stem1 of the VanilaNet network, and a convolutional layer with a convolution kernel of 1 * 1 and a stride of 1 and three stage layers are used in stem2 of the VanilaNet network; and training the improved YOLOv8 model by using the photovoltaic panel defect image data set to realize photovoltaic panel defect detection. The existing method has the problems of low accuracy and low detection speed of photovoltaic panel defect detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image recognition and deep learning technology, and in particular to a photovoltaic panel EL defect detection method based on an improved YOLOv8 model. Background Art

[0002] Photovoltaic panels are devices that can convert sunlight into electrical energy. When there are defects in photovoltaic panels, the energy conversion efficiency of the panels may decrease, the life of the photovoltaic panels may be shortened, local overheating may occur, and the risk of fire may increase. In order to minimize these hazards, defect detection of photovoltaic panels is required to ensure efficient operation, extend the life of the panels, and ensure safe and reliable energy production.

[0003] Usually, defects in photovoltaic panels are not easy to distinguish with the naked eye, and special imaging technology is generally required to highlight the defects. The common photovoltaic panel defect detection imaging technology uses electroluminescence imaging EL technology. By applying a forward bias to the photocell, the intensity of the light (mainly near-infrared light) generated at this time is proportional to the voltage. The electrically inactive areas in the photovoltaic module will appear as shadows on the EL image. These shadows are the defective parts of the photovoltaic panel, but the use of EL detection technology requires professionalism and the cost of professional equipment is high.

[0004] Deep learning technology is used to detect battery defects using target detection, avoiding tedious and complex image preprocessing and manual feature extraction operations, and adapting to different scenarios and different types of defects, thereby improving detection accuracy, robustness and real-time performance.

[0005] The patent with publication number CN113989241A first pre-segments the photovoltaic components, then pre-classifies the pre-segmented photovoltaic components, and then loads the pre-classified candidate negative samples into the trained faster-rcnn model weight file for secondary defect detection; however, this method is complicated to operate in the detection process, and the faster-rcnn model has a slow inference speed, which is not conducive to application in actual production detection.

[0006] The patent with publication number CN116542962A uses Yolov5m weights for photovoltaic cell defect detection. The network structure is relatively complex, with more parameters and more calculations, so more computing resources and memory are required. In addition, by introducing the SimAM attention mechanism, there are also problems of increased calculation amount and high computing resource overhead, which is not conducive to deployment on devices with limited computing resources such as mobile devices, and has poor practicality. Summary of the invention

[0007] In view of the shortcomings of the existing methods, the present invention solves the problems of low accuracy and slow detection speed of photovoltaic panel defects in the existing methods.

[0008] The technical solution adopted by the present invention is: a photovoltaic panel EL defect detection method based on an improved YOLOv8 model comprises the following steps:

[0009] Step 1: Obtain defect images of photovoltaic panels and construct a photovoltaic panel defect image dataset;

[0010] As a preferred embodiment of the present invention, the defects of the photovoltaic panel include: cracks, broken grids, black cores and thick lines.

[0011] As a preferred implementation of the present invention, Mosaic, Mixup, HSV transformation, scale transformation and flip data enhancement are adopted for the photovoltaic panel defect image.

[0012] Step 2: Build an improved YOLOv8 model and replace the Backbone network with the VanillaNet network. The stem1 of the VanillaNet network uses a convolution layer with a convolution kernel of 4×4 and a stride of 4, and the stem2 uses a convolution layer with a convolution kernel of 1×1 and a stride of 1 and 3 stage layers.

[0013] As a preferred embodiment of the present invention, step 2 further includes:

[0014] Replace the Backbone network with the improved VanillaNet network: The improved VanillaNet network includes: stem1 layer and stem2 layer. The stem1 layer includes a convolution layer with a convolution kernel of 3×3 and a stride of 2, a normalization layer and a LeakyReLU activation function layer; the stem2 layer includes a convolution layer with a convolution kernel of 2×2 and a stride of 1, a normalization layer, a ReLU activation function layer, a depth-wise convolution layer and a normalization layer; the Body layer includes 4 stage parts, each stage layer includes two layers, the first layer of each stage layer includes: a convolution layer with a convolution kernel of 1×1, a normalization layer and a Leaky ReLU activation function layer; the second layer of each stage layer includes: a convolution layer with a convolution kernel of 1×1, a normalization layer, a maximum pooling layer with a stride of 2 and a ReLU activation function layer, a depth-wise convolution layer, and a normalization layer.

[0015] As a preferred implementation of the present invention, the Neck part Conv module of the improved YOLOv8 model is replaced with a variable kernel convolution module.

[0016] As a preferred embodiment of the present invention, the CIOU loss function of the improved YOLOv8 model is replaced by the WIoU_v3 loss function.

[0017] Step 3: Use the photovoltaic panel defect image dataset to train the improved YOLOv8 model to achieve photovoltaic panel defect detection;

[0018] As a preferred embodiment of the present invention, a photovoltaic panel EL defect detection system based on an improved YOLOv8 model includes: a memory for storing instructions executable by a processor; and a processor for executing instructions to implement a photovoltaic panel EL defect detection method based on the improved YOLOv8 model.

[0019] As a preferred embodiment of the present invention, a computer readable medium stores a computer program code, and when the computer program code is executed by a processor, a photovoltaic panel EL defect detection method based on an improved YOLOv8 model is implemented.

[0020] Beneficial effects of the present invention:

[0021] 1. Improve the YOLOv8 model, optimize the loss function, use the dynamic non-monotonic focusing mechanism, and use a more superior gradient gain allocation strategy to improve the quality of the frame and the detection accuracy;

[0022] 2. Use the improved VanillaNet network to reduce network parameters and computational complexity, speed up detection, capture finer local features, extract more detailed information, and improve the model's expressiveness. Introduce more nonlinear transformations to help regularize the model, reduce the risk of overfitting, and bring higher classification accuracy and detection performance;

[0023] 3. Use a variable kernel convolution module to replace the original Conv module. By dynamically calculating the offset and bilinear interpolation, the convolution kernel is allowed to dynamically adjust parameters and sampling shapes according to the input data, so that the model can better adapt to different input features, thereby improving the detection accuracy of the model; for the complex characteristics and diversity of photovoltaic panel defects, variable kernel convolution can significantly improve the performance and robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of photovoltaic panel EL defect detection based on the improved YOLOv8 model;

[0025] Figure 2 This is a diagram of the types of EL defects detected in photovoltaic panels;

[0026] Figure 3 This is the detailed structure diagram of the improved VanillaNet network;

[0027] Figure 4 It is a schematic diagram of the VanillaNet network and the improved VanillaNet network structure;

[0028] Figure 5 It is a schematic diagram of the structure of the variable kernel convolution module;

[0029] Figure 6 It is a schematic diagram of the structure of the improved YOLOv8 model;

[0030] Figure 7 This is the effect diagram of photovoltaic panel EL defect detection based on the improved YOLOv8 model. DETAILED DESCRIPTION

[0031] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0032] like Figure 1 ,The photovoltaic panel EL defect detection method based on the improved YOLOv8 model includes the following steps:

[0033] Step 1: Obtain defect images of photovoltaic panels and construct a photovoltaic panel defect image dataset;

[0034] The EL defect images of photovoltaic panels are obtained from the public data set and the data set is established by shooting and collecting the EL defect image data of photovoltaic panels; the collected images are uniformly cropped to 1024×1024 pixels, and the number of channels is 3; Figure 2 ,The acquired photovoltaic panel defect images can be divided into different types: cracks, broken grids, black cores, and thick lines, where one image contains one or more categories of defects.

[0035] The labelimg image annotation tool is used to annotate defect images according to the standard VOC data format. For images with multiple defects, multiple anchor frames are used to calibrate the defect areas. The crack is labeled as crack, the broken grid is labeled as finger, the black core is labeled as black_core, and the thick line is labeled as thick_line. After the annotation is completed, the generated XML format data is saved and converted into TXT format data.

[0036] The number of crack, broken grid, black core, and thick line categories is 1260, 2958, 981, and 1028. The images and corresponding txt files are transformed by flipping, rotating, changing brightness, and translating data enhancement methods, respectively, and the number of crack and thick line defect categories is increased by 200 each.

[0037] Mosaic, Mixup, HSV transformation, scale transformation, and flipping are combined to expand the data set without losing the original feature information;

[0038] Step 2: Build a photovoltaic panel defect image detection model based on the improved YOLOv8 model;

[0039] Use WIOU Loss as the loss function to replace the original loss function, and use the loss function WIoU_v3 to replace the loss function CIOU of the original YOLOv8 model;

[0040] The function value L of the loss function WIoU_v3 WIoUv3 The calculation formula is:

[0041]

[0042] in, is the gradient gain, which is the mapping of the outlier degree β and is controlled by the hyperparameters α and δ; R WIoU Represents the normalized distance between the predicted box and the center point of the real box, W g , H g is the size of the smallest enclosing box between the predicted box and the real box, (x, y) and (x gt ,y gt ) is the center point coordinate of the predicted box and the real box, B p is the prediction box, B gt It’s a real frame.

[0043] Using the WIoU_v3 loss function and a dynamic focusing mechanism, we use the outlier degree instead of IoU to evaluate the quality of the outlines. We use a gradient gain allocation strategy with better performance, dynamically adjust the gradient gain allocation during the training process, save high-quality outlines in the early stages of training, weaken the penalty of geometric metrics, and do not intervene in the training too much, which will enable the model to have better generalization ability. In the middle and late stages of training, we allocate smaller gradient gains to low-quality outlines, which will effectively prevent low-quality examples from generating large harmful gradients and increase the detection ability of the model.

[0044] Replace the Backbone network with the improved VanillaNet network;

[0045] like Figure 3 , the improved VanillaNet network includes the Stem module and the Body module. The Stem module is a shallow feature extraction operation, which includes the stem1 layer and the stem2 layer. Each layer consists of a conventional convolution layer. The stem1 layer includes a convolution layer with a convolution kernel of 3×3 and a stride of 2, a normalization layer and a Leaky ReLU activation function layer; the stem2 layer includes a convolution layer with a convolution kernel of 2×2 and a stride of 1, a normalization layer, a ReLU activation function layer, a depth-wise convolution layer and a normalization layer; the corresponding input image of size (3,640,640) is subjected to stem feature transformation to obtain a feature map of size (128,320,320);

[0046] The Body layer consists of 4 stages, each of which consists of two layers. The first layer of each stage includes: a convolution layer with a convolution kernel of 1×1, a normalization layer, and a Leaky ReLU activation function layer; the second layer of each stage layer includes: a convolution layer with a convolution kernel of 1×1, a normalization layer, a maximum pooling layer with a stride of 2 and a ReLU activation function layer, a depth-wise convolution layer, and a normalization layer; the feature map with an input size of (128,320,320) is subjected to four-layer stage operations to obtain feature maps of (256,160,160), (512,80,80), (1024,40,40), and (1024,20,20) in turn; among them, the feature map with a size of (1024,40,40) is used as the backbone extraction feature 1, and the feature map with a size of (512,80,80) is used as the backbone extraction feature 2; Concat splicing is performed with the feature maps of different layers in the Neck network, and the feature map with a size of (1024,20,20) is used as the input feature map of the spatial feature pyramid pooling feature module, as shown in Figure 6 shown.

[0047] As shown in Table 1, the improved VanillaNet network reduces network parameters and computational complexity compared to the backbone of the original YOLOv8 network. Figure 4 (a) is the original VanillaNet network Stem layer. In the module, stem1 uses a convolution layer with a convolution kernel of 4×4 and a stride of 4, and stem2 uses a convolution layer with a convolution kernel of 1×1 and a stride of 1. Figure 4 (b) In the improved VanillaNet network, the stem1 layer uses a convolution layer with a convolution kernel of 3×3 and a stride of 2, and the stem2 uses a convolution layer with a convolution kernel of 2×2 and a stride of 1. Using a 3×3 convolution kernel in stem1 can capture finer local features for defects such as broken grids, thick lines, and some cracks in the data set, which account for a small proportion of defects in the image, and help to extract more detail information, thereby improving the expressiveness of the model. The 3×3 convolution kernel has a smaller receptive field, which means that it can pay more attention to the local area of ​​the input image and avoid losing too much detail information in the first layer. A stride of 2 will cause the feature map size to decrease more smoothly, thereby retaining more spatial information and effectively reducing information loss. In stem2, the 2×2 convolution kernel can perform feature fusion in a larger range, which helps to further integrate local information.

[0048] The original VanillaNet network body module includes 3 stage layers, and the improved VanillaNet network body module includes 4 stage layers. Adding a stage means adding additional layers and nonlinear transformations, so that more complex and high-level features can be extracted, and more nonlinear transformations are introduced, which helps to regularize the model, reduce the risk of overfitting, and bring higher classification accuracy and detection performance.

[0049] Table 1 Comparison between the model of the present invention and the existing model

[0050]

[0051] like Figure 5 In the Neck part, the variable kernel convolution module is used instead of the Conv module. The variable kernel convolution is based on the traditional Conv, and the direction vector of the convolution kernel is adjusted to make the shape of the convolution kernel closer to the feature object. The variable kernel convolution is used for any point P on the input feature map. 0 , the convolution operation formula is:

[0052]

[0053] Among them, P n represents the offset of each point in the convolution kernel relative to the center point; R represents the set of all points in the convolution kernel; ω(P n ) represents the weight of the corresponding position of the convolution kernel; x(P 0 +P n +ΔP n ) represents the input feature map P 0 +P n +ΔP n The element value at the position, ΔP n Indicates the offset, y(P 0 ) represents the output feature map P 0 The element value at position.

[0054] The implementation process of the variable kernel convolution module is:

[0055] The feature map is taken as input and a convolution layer is applied to the feature map to obtain the offset of the deformable convolution. The offset layer is 2N, and a translation is performed on the plane, changing both the horizontal and vertical directions.

[0056] After adding an offset to a point in the feature map, it is highly unlikely that an integer coordinate point will be obtained. Through linear interpolation, the specific method is to find the four points closest to the offset point whose distance is less than 1, multiply their values ​​by the weight (the weight is measured by distance), and then add them up, which is the value of the point in the feature map after the final offset.

[0057] The obtained feature map is reshaped into a shape suitable for the convolution operation, and then the convolution operation is performed. The convolution result is batch normalized, and then the SiLU activation function is applied to obtain the output data.

[0058] Using the variable kernel convolution module allows the convolution kernel to dynamically adjust parameters and sampling shapes according to the input data, so that the model can better adapt to input features of different shapes and sizes, thereby improving the detection accuracy of the model; since the convolution kernel parameters are no longer static, variable kernel convolution can avoid overfitting and improve the generalization ability of the model. The convolution kernel with dynamic parameters can capture more features, thus performing better on both training data and test data. Dynamic adjustment of the convolution kernel can make the model more flexible and cope with diverse input data; for the complex features and diversity of the data set in this dataset, variable kernel convolution can significantly improve the performance and robustness of the model.

[0059] Experimental process:

[0060] Environment configuration, download the official source code of the YOLOv8 model from GitHub, and use Anaconda to create a virtual environment according to the dependent library files of the YOLOv8 source code, including matplotlib>=3.2.2, numpy>=1.18.5, opencv-python>=4.1.1, Pillow>=7.1.2, psutil, PyYAML>=5.3.1, requests>=2.23.0, scipy>=1.4.1, thop>=0.1.1, torch>=1.7.0, torchvision>=0.8.1, tqdm>=4.64.0, tensorboard>=2.4.1, pandas>=1.1.4, seaborn>=0.11.0; the configuration used is CPU: Intel Core i9-13900HX; CPU main frequency: 2.20GHz; memory: 16G; GPU: NVIDIA GeForce GTX 4060; video memory: 8G; deep learning framework is PyTorch, development environment is Pytoch1.13.1, Python3.8, Cuda11.7.

[0061] Divide the data set into a training set and a test set according to a preset ratio of 8:2; use the torch.utils.data function in the pytorch library to read the data set; hyperparameter configuration: set the learning rate lr = 0.1, batch size batchsize = 8, iterations epochs = 300 and learning rate momentum momentum = 0.937 to obtain the optimal hyperparameter combination.

[0062] The improved YOLOv8 model is used to detect and evaluate the actual photovoltaic panel defect images. The collected photovoltaic panel defect images are passed into the weight file generated after the training is completed for model testing to obtain the test results, including the defect location, defect type and confidence. The test results are as follows: Figure 7 shown.

[0063] In order to verify the effects of the improved VanillaNet network, variable kernel convolution module, and WIOU Loss loss function, experiments were conducted based on the YOLOv8 model. The training and validation sets of all models used the same set of data. The specific values ​​are shown in Table 2 below.

[0064] Table 2 Ablation experiment

[0065]

[0066]

[0067] By comparison, the improved YOLOv8 model of the present invention has improved the accuracy of all categories by 2.8%, the recall rate by 0.5%, the mAP@0.5 value by 3.5%, and the map@0.5:0.95 value by 3.8% compared with the YOLOv8 model. All of them have been improved to a certain extent, and the number of parameters has also been significantly reduced. Figure 7 Four defect detection effects.

[0068] Through data image enhancement, the robustness and generalization ability of the model are improved; the WIOU Loss loss function uses a gradient gain allocation strategy with superior performance to effectively prevent low-quality examples from generating large harmful gradients and increase the detection ability of the model; by improving the VanillaNet network, more refined local features are captured, more complex and high-level features are extracted, and by introducing more nonlinear transformations, it helps to regularize the model, reduce the risk of overfitting, and bring higher classification accuracy and detection performance; using the variable kernel convolution module, the model can better adapt to input features of different shapes and sizes. For images with complex features and diversity in the photovoltaic panel dataset, variable kernel convolution can significantly improve the performance and robustness of the model.

[0069] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. Photovoltaic panel EL defect detection method based on improved YOLOv8 model, It is characterized in that The following steps are involved: Step 1: Obtain defect images of photovoltaic panels and construct a photovoltaic panel defect image dataset; Step 2: Build an improved YOLOv8 model and replace the Backbone network with the VanillaNet network. The stem1 of the VanillaNet network uses a convolution layer with a convolution kernel of 4×4 and a stride of 4, and the stem2 uses a convolution layer with a convolution kernel of 1×1 and a stride of 1 and 3 stage layers. Step 3: Use the photovoltaic panel defect image dataset to train the improved YOLOv8 model to realize photovoltaic panel defect detection.

2. A photovoltaic panel EL defect detection method based on an improved YOLOv8 model as claimed in claim 1, It is characterized in that Step 2 also includes: Replace the Backbone network with the improved VanillaNet network: The improved VanillaNet network includes: stem1 layer and stem2 layer. The stem1 layer includes a convolution layer with a convolution kernel of 3×3, a normalization layer and a Leaky ReLU activation function layer; the stem2 layer includes a convolution layer with a convolution kernel of 2×2, a normalization layer, a ReLU activation function layer, a depth-wise convolution layer and a normalization layer; the Body layer includes 4 stage parts, each stage layer includes two layers, the first layer of each stage layer includes: a convolution layer with a convolution kernel of 1×1, a normalization layer and a Leaky ReLU activation function layer; the second layer of each stage layer includes: a convolution layer with a convolution kernel of 1×1, a normalization layer, a maximum pooling layer, a ReLU activation function layer, a depth-wise convolution layer and a normalization layer.

3. A photovoltaic panel EL defect detection method based on an improved YOLOv8 model as claimed in claim 1, It is characterized in that Replace the Neck part Conv module of the improved YOLOv8 model with a variable kernel convolution module.

4. A photovoltaic panel EL defect detection method based on an improved YOLOv8 model as claimed in claim 1, It is characterized in that Replace the CIOU loss function of the improved YOLOv8 model with the WIoU_v3 loss function.

5. A photovoltaic panel EL defect detection method based on an improved YOLOv8 model as claimed in claim 1, It is characterized in that Photovoltaic panel defects include: cracks, broken grids, black cores and thick wires.

6. A photovoltaic panel EL defect detection method based on an improved YOLOv8 model as claimed in claim 1, It is characterized in that Mosaic, Mixup, HSV transformation, scale transformation and flipping data enhancement are used for photovoltaic panel defect images.

7. Photovoltaic panel EL defect detection system based on improved YOLOv8 model, It is characterized in that include: a memory for storing instructions executable by a processor; A processor, used to execute instructions to implement the photovoltaic panel EL defect detection method based on the improved YOLOv8 model as described in any one of claims 1-6.

8. A computer readable medium storing computer program code, It is characterized in that When the computer program code is executed by a processor, the computer program code implements the photovoltaic panel EL defect detection method based on the improved YOLOv8 model as described in any one of claims 1 to 6.

Citation Information

Cited By

  • Lightweight photovoltaic cell panel defect detection method based on improved YOLOv8

    CN119693712A