Photovoltaic cell panel defect detection method based on improved YOLOv8s model

Abstract

The invention relates to a photovoltaic cell panel defect detection method based on an improved YOLOv8s model, and the method comprises the steps: inputting a solar cell panel image into the improved YOLOv8s model to detect the defects of a photovoltaic cell panel, and the improved YOLOv8s model comprises a backbone network, a neck network and a head network. The backbone network is a YOLOv8s model backbone network in which all Conv modules except the first Conv module are replaced by a Ghost-Conv module and an SPPF module is replaced by an SPPCSPC module; the neck network is a YOLOv8s model neck network in which a Concat module is replaced by a BiFPNConcat module, and the neck network is a YOLOv8s model neck network in which the Concat module is replaced by a BiFPNConcat module; the head network is a YOLOv8s model head network in which a P2 layer detection head is added to three detection heads and a CBAM module is added to the front ends of four detection heads. The detection method provided by the invention has the beneficial effects that the detection precision and speed of a fine defect target are higher under the background of a complex near-infrared image of the photovoltaic cell panel.

Description

Technical Field

[0001] The present invention relates to the technical field of solar photovoltaic panel detection, and particularly to an improved method for object detection algorithm based on deep learning, aiming to improve the detection accuracy and speed of small defect objects in the complex background of near-infrared images of photovoltaic panels by the YOLOv8s model. Background Art

[0002] There have been reports on detecting small defect objects of photovoltaic panels by improving the YOLOv8 model. For example, Chinese Patent CN119359632A discloses a method for detecting photovoltaic cell defects, which adds an SPDConv convolution module to the backbone network of the YOLOv8 model and adds a BoTNet attention mechanism after the SPPF layer. CN116883801A discloses a YOLOv8 object detection method based on attention mechanism and multi-scale feature fusion; combining the CBAM attention mechanism, BiFPN multi-scale feature fusion pyramid, SIoU loss function with the YOLOv8 model. However, the detection accuracy and speed of the above methods for small defect objects in the complex background of near-infrared images of photovoltaic panels still need to be improved. Summary of the Invention

[0003] To solve the technical problems of the present invention, one aspect of the present invention provides a method for detecting photovoltaic panel defects based on an improved YOLOv8s model, including the step of inputting a solar panel image into the improved YOLOv8s model to detect the photovoltaic panel defects. The improved YOLOv8s model includes a backbone network, a neck network, and a head network.

[0004] The backbone network is the backbone network of the YOLOv8s model in which all Conv modules except the first one are replaced with Ghost-Conv modules, and the SPPF module is replaced with an SPPCSPC module.

[0005] The neck network is the neck network of the YOLOv8s model in which the Concat module is replaced with a BiFPN_Concat module.

[0006] The head network is the head network of the YOLOv8s model in which one P2 layer detection head is added to the three detection heads, and a CBAM module is added to the front ends of the four detection heads; preferably, the resolution of the P2 layer detection head is 160x160 pixels.

[0007] The second aspect of the present invention also provides a system for detecting photovoltaic panel defects based on an improved YOLOv8s model. The system includes at least one processor; and a memory storing instructions, which, when executed by at least one processor, implement the steps of the method described above.

[0008] The beneficial effects of the present invention are as follows. The present invention provides a method for detecting defects in photovoltaic panels based on an improved YOLOv8s model. On the one hand, a small target detection head is added, and the CBAM module is added to all detection heads in the head network, significantly improving the detection accuracy of multi-targets, especially small targets. On the other hand, the bidirectional feature fusion module BiFPN_Concat is added to the neck network and the SPPCSPC module is introduced into the backbone network. Without adding too much computational complexity, it effectively integrates context information of different scales, improves the adaptability of the model to scale changes, and enhances the detection performance of the model for multi-scale targets. On the further hand, the Ghost-Conv lightweight convolution module is introduced, reducing the required number of parameters and computational complexity, enabling the improved YOLOv8s model to perform normal inference on mobile devices.

[0009] The method for detecting defects in photovoltaic panels based on the improved YOLOv8s model of the present invention has higher detection accuracy and speed for small defect targets in the complex background of near-infrared images of photovoltaic panels. Brief Description of the Drawings

[0010] Figure 1 Flowchart of the method for detecting defects in photovoltaic panels based on the improved YOLOv8s model in Embodiment 1.

[0011] Figure 2 : Overall architecture diagram of the improved YOLOv8s model in Embodiment 1.

[0012] Figure 3 : Structural schematic diagram of the bidirectional feature fusion module BiFPN_Concat in Embodiment 1.

[0013] Figure 4 : Structural schematic diagram of the target detection head CBAM-Detect in Embodiment 1.

[0014] Figure 5 : Schematic diagram of the principle of the SPPCSPC module in Embodiment 1. Detailed Embodiments

[0015] At present, the defect detection of photovoltaic panels is mainly carried out by manual observation. This method completely relies on the professional qualities of personnel and is also affected by the mental state and concentration of observers, resulting in low detection efficiency and poor stability. With the development of deep learning, object detection technology has gradually matured and has been widely applied in fields such as industrial inspection, autonomous driving, and security monitoring. The YOLO (You Only Look Once) series, as an efficient and real-time single-stage object detection algorithm, has achieved a good balance between performance and speed. Among them, the YOLOv8s version open-sourced by Ultralytics is more suitable for the automated detection of photovoltaic panel defects. However, in the task of detecting small defects in complex backgrounds such as near-infrared images of photovoltaic panels, the basic model of the YOLOv8s version still has the following deficiencies: 1. The small target defect features are easily submerged by the high-level feature information, resulting in missed or misdetected detections; 2. The complex background of the polysilicon image easily interferes with feature extraction, making the expression of small target features unclear; 3. Due to insufficient optimization for multi-scale targets, the detection accuracy of the targets is not high; 4. Due to the relatively complex model structure, the detection speed is not ideal in low-power systems such as embedded systems; To address the above problems, it is urgent to improve and optimize the model structure of YOLOv8s.

[0016] Chinese Patent CN119359632A discloses a method for detecting photovoltaic cell defects, which is to add an SPDConv convolutional module to the backbone network of the YOLOv8 model and add a BoTNet attention mechanism after the SPPF layer. CN116883801A discloses a YOLOv8 object detection method based on attention mechanism and multi-scale feature fusion; combining the CBAM attention mechanism, BiFPN multi-scale feature fusion pyramid, SIoU loss function with the YOLOv8 model. However, the detection accuracy and speed of the above methods for small defect targets in the complex background of near-infrared images of photovoltaic panels still need to be improved.

[0017] The present invention will be further described below in conjunction with some embodiments.

[0018] In some embodiments, a method for detecting photovoltaic panel defects based on an improved YOLOv8s model is provided, including the step of inputting a solar panel image into the improved YOLOv8s model to detect the photovoltaic panel defects. The improved YOLOv8s model includes a backbone network, a neck network, and a head network.

[0019] The backbone network is the backbone network of the YOLOv8s model in which all Conv modules except the first one are replaced with Ghost-Conv modules and the SPPF module is replaced with an SPPCSPC module.

[0020] The neck network is the neck network of the YOLOv8s model that replaces the Concat module with the BiFPN_Concat module;

[0021] The head network is the head network of the YOLOv8s model that adds one P2 layer detection head to the three detection heads and adds the CBAM module at the front end of the four detection heads; Preferably: the resolution of the P2 layer detection head is 160x160 pixels.

[0022] The term "YOLOv8s model" refers to the object detection model open-sourced by Ultralytics. It includes the backbone network Backbone, the neck network Neck, and the head network Head. The backbone network includes the Conv module, the C2f module, and the SPPF module, which are used to generate feature maps; the neck network includes the Concat module, the Ghost-Conv module, and the Upsample module, which are used to perform feature fusion on the feature maps; the head network includes three Detect detection heads.

[0023] The term "photovoltaic panel defect" includes, but is not limited to, panel cracks, cell scratches, cell edge breakages, EVA layer cracks, backplane warping, junction box water seepage, delamination, microcracks, and hot spot effects. They are called small target defects, small-scale target defects, medium-scale target defects, and large-scale target defects in solar panel images.

[0024] The term "solar panel image" includes, but is not limited to, images taken by devices such as near-infrared industrial cameras, infrared thermal imagers, and SWIR cameras of drones.

[0025] The term "adding the CBAM module at the front end of the four detection heads" means adding the CBAM (Convolutional Block Attention Module) module at the front end of the Detect detection heads corresponding to the P2, P3, P4, and P5 feature vectors of the improved YOLOv8s neural network model.

[0026] The CBAM module is a dual attention mechanism that enhances the performance of convolutional neural networks. It consists of a channel attention module and a spatial attention module, which are used to enhance the feature expression ability of the model. When combined with small target detection heads, it further improves the model's detection ability for small targets.

[0027] The SPPCSPC module is an improvement on the SPP (Spatial Pyramid Pooling) module, introducing the concept of Cross Stage Partial Networks (CSPN). The SPPCSPC module mainly enhances the multi-scale feature representation ability and computational efficiency of the model by adding parallel MaxPool operations multiple times in a series of convolutions and combining the CSPN structure, enabling the model to better distinguish large and small targets.

[0028] The Ghost-Conv module consists of three parts: a conventional convolution, a Ghost generation module, and a feature map concatenation. Compared with ordinary convolutional neural networks, Ghost-Conv generates more feature maps from existing feature maps using linear transformation, reducing the computational load while maintaining performance similar to that of conventional convolution. Experiments have proven that the Ghost-Conv module has excellent inference performance on mobile devices.

[0029] The bidirectional feature fusion module BiFPN_Concat is an improvement based on the original Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) structures. It can transfer high-level semantic information to the low level, enhancing the understanding of small target features, and then combines with the Concat module to concatenate feature maps of different scales, helping to detect small targets more accurately.

[0030] In these embodiments, aiming to solve the problem of insufficient small target detection performance of the existing YOLOv8s model in the complex background of photovoltaic panel detection images, a method for improving the YOLOv8s model structure is proposed. By introducing the bidirectional feature fusion module BiFPN_Concat, improving the object detection head, using an improved spatial pyramid pooling structure, and replacing the lightweight convolution module, etc., the accuracy, robustness, and detection speed of the model in small target and complex background detection tasks are improved.

[0031] In some embodiments, a method for detecting photovoltaic panel defects based on an improved YOLOv8s model is provided. Among them, the improved YOLOv8s neural network model includes a backbone network Backbone, a neck network Neck, and a head network Head;

[0032] The backbone network includes a Conv module, a Ghost-Conv module, a C2f module, and an SPPCSPC module, which are used to generate feature maps;

[0033] The neck network includes a BiFPN_Concat module, a Ghost-Conv module, and an Upsample module, which are used to perform feature fusion on the feature maps;

[0034] The head network includes four CBAM-Detect object detection heads, which are used to generate feature vectors P2, P3, P4, and P5 of different sizes, and are respectively used to detect small targets, small-scale targets, medium-scale targets, and large-scale targets. Specifically: the CBAM-Detect small target detection head for generating the feature vector P2 and detecting small targets, the CBAM-Detect small-scale target detection head for generating the feature vector P3 and detecting small-scale targets, the CBAM-Detect medium-scale target detection head for generating the feature vector P4 and detecting medium-scale targets, and the CBAM-Detect large-scale target detection head for generating the feature vector P5 and large-scale targets.

[0035] In some embodiments, a photovoltaic panel defect detection method based on an improved YOLOv8s model is provided. Among them, the Ghost-Conv module includes three parts: a conventional convolution, a Ghost generation module, and a feature map splicing.

[0036] In some embodiments, a photovoltaic panel defect detection method based on an improved YOLOv8s model is provided. Among them, the solar panel image is a near-infrared image of the solar panel.

[0037] In some embodiments, a photovoltaic panel defect detection method based on an improved YOLOv8s model is provided. Among them, the training method of the improved YOLOv8s model includes the following steps:

[0038] S1: Select the PVEL-AD dataset as the training sample;

[0039] S2: Divide the PVEL-AD dataset into a training set, a validation set, and a test set;

[0040] S3: Modify the YOLOv8s model to obtain a pre-trained improved YOLOv8s model;

[0041] S4: Input the images included in the training set into the pre-trained improved YOLOv8s model to train for several rounds to obtain the best weight file best.pt;

[0042] S5: Input the images included in the test set into the best weight file best.pt, compare the test results with the validation set, and use accuracy, recall, and mean average precision as the model evaluation metrics.

[0043] In some embodiments, a photovoltaic panel defect detection method based on an improved YOLOv8s model is provided. Among them, in step S3, the method for modifying the YOLOv8s model to obtain a pre-trained improved YOLOv8s model includes the following steps:

[0044] Add the CBAM module to all detection heads;

[0045] Replace all Conv modules except the first one with Ghost-Conv modules;

[0046] Replace the SPPF module with the SPPCSPC module;

[0047] Replace the Concat module with the BiFPN_Concat module.

[0048] In some embodiments, a method for detecting defects in photovoltaic panels based on an improved YOLOv8s model is provided. Among them, the method for modifying the YOLOv8s model to obtain a pre-trained improved YOLOv8s model further includes the following steps:

[0049] Add the P2 layer detection head; The target detection head includes 3 detection heads and 1 P2 layer detection head.

[0050] In some embodiments, a method for detecting defects in photovoltaic panels based on an improved YOLOv8s model is provided, wherein,

[0051] In step S1: Select 4500 labeled solar panel images in the PVEL-AD dataset as training samples;

[0052] In step S2: Divide the training samples into the training set, the validation set, and the test set according to the ratio of 7:2:1;

[0053] In some embodiments, a method for detecting defects in photovoltaic panels based on an improved YOLOv8s model is provided, wherein,

[0054] In step S4: Input the images included in the training set into the pre-trained improved YOLOv8s model and train for several rounds to obtain the best weight file best.pt, which includes the following steps:

[0055] Set the number of training rounds epochs to 200, the batch size batch-size to 4, use the lightweight model yolov8s.pt as the pre-trained weight file, and obtain the best weight file best.pt after training.

[0056] In some embodiments, a system for detecting defects in photovoltaic panels based on an improved YOLOv8s model is provided. The system includes at least one processor; and a memory that stores instructions, and when the instructions are executed by at least one processor, the steps of the foregoing method are implemented.

[0057] Embodiment 1

[0058] As Figure 1 shown, it is a flowchart of a photovoltaic panel defect detection method based on an improved YOLOv8s model provided by the present invention.

[0059] S1: Select 4,500 labeled solar panel images from the publicly available dataset PVEL-AD of solar panel defect images as training samples;

[0060] S2: Divide the dataset in step S1 into a training set (train), a validation set (val), and a test set (test) according to a ratio of 7:2:1;

[0061] S3: Improve the original YOLOv8s model by modifying files such as yolov8s.yaml, including improving the object detection head by adding a CBAM module, replacing all Conv modules except the first one with Ghost-Conv modules, replacing the SPPF module with an SPPCSPC module, and replacing the Concat module with a BiFPN_Concat module;

[0062] S4: Input the images included in the training set constructed in step S2 into the improved YOLOv8s network in step S3 for training for 200 epochs to obtain the best.pt weight file

[0063] Specifically, the training is carried out using the improved YOLOv8s model in step S3. The CPU model used in the experimental platform is AMD Ryzen 5 5600H 3.3GHz, the memory is 16GB, and the GPU model is NVIDIA GeForce RTX 3050 (4G). The 64-bit Windows 10 operating system is adopted, Pycharm is used as the development platform, Pytorch 1.13.1 is used as the deep learning framework, Python 3.9 is used as the programming language, and CUDA 11.6 version is used as the parallel computing framework. The number of training epochs epochs is set to 200, the batch size batch-size is set to 4, the SGD optimizer is set, the initial learning rate is set to lr0 = 0.003, and the final learning rate lrf = 0.05 (the default lr0 and lrf are 0.01). The training set (train), validation set (val), and test set (test) are set, the number of detection type quantities (nc) is set to 12, the lightweight model yolov8s.pt is used as the pre-training weight file, and the best weight file best.pt is obtained after training.

[0064] S5: Input the images contained in the test set into the best.pt obtained after training in step S4. After obtaining the test results, compare them with the information of the true standard, and use precision, recall, and mean average precision (mAP) as the evaluation metrics of the model.

[0065] S6: Use the best.pt weight file obtained in step S4 for the detection of near-infrared images of solar panels actually collected to verify the improvement effect of the model. For the results, see Test Example 1.

[0066] Combined with Figure 3 As shown, based on the Feature Pyramid Network (FPN) of YOLOv8s, a Bidirectional Feature Pyramid Network (BiFPN) module is introduced, which allows information to propagate bidirectionally between different resolution levels, enabling targets of different sizes to have appropriate feature representations at corresponding scales, effectively fusing multi-scale features, helping the model to more comprehensively understand targets of different sizes, and improving the detection ability for multi-scale objects. The BiFPN module is then combined with the Concat module, which concatenates feature maps of different levels, enabling YOLOv8s to more sensitively detect and locate small targets and improving the accuracy of small target detection.

[0067] Combined with Figure 2 As shown, a small target detection head is added to the YOLOv8s basic model. Specifically, a new detection head is introduced through the features of the P2 layer. The resolution of the P2 layer detection head is 160x160 pixels, which is equivalent to only 2 downsampling operations in the backbone network and contains richer underlying feature information of the target. The two P2 layer features obtained from top to bottom and bottom to top in the neck network are fused with the same-scale features in the backbone network, and the output feature is the fusion result of 3 input features. In this way, when the P2 layer detection head deals with tiny targets, it can detect them quickly and effectively. The P2 layer detection head plus the original 3 detection heads can effectively alleviate the negative impact brought by the scale variance.

[0068] Combined with Figure 4 As shown, optimization is carried out by adding a CBAM module to the front end of the four Detect detection heads. By adaptively learning the channel and spatial attention weights, the feature expression ability of the convolutional neural network is improved. By combining channel attention and spatial attention, the CBAM module can capture the correlation between features in different dimensions, thereby enhancing the performance of the image detection task.

[0069] Combined with Figure 5As shown in the figure, the SPPCSPC module is introduced. Specifically, the SPPCSPC module combines the Spatial Pyramid Pooling (SPP) module and the Cross Stage Partial Networks (CSPN) module. The SPPCSPC module mainly improves the multi-scale feature expression ability and computational efficiency of the model by adding parallel MaxPool operations multiple times in a series of convolutions. The CSPN module reduces the computational load by splitting and reusing feature maps while maintaining the model's expressive power, enabling the model to better distinguish large and small targets.

[0070] Experimental Example 1

[0071] Define the model of Example 1 as YOLOv8s-CBAM, and use the test set (test) of the publicly available dataset PVEL-AD for verification, and compare it with the original model YOLOv8s:

[0072]

[0073] Experimental Example 2

[0074] Test the detection accuracy and speed of different models for small defective targets in the complex background of near-infrared images of photovoltaic panels.

[0075] Experimental method: Use the test set (test) of the publicly available dataset PVEL-AD for verification, and test the detection accuracy and speed of small defective targets in the complex background of near-infrared images of photovoltaic panels with YOLOv8s-CBAM, as well as CN119359632A and CN116883801A respectively.

[0076] Experimental results: The YOLOv8s-CBAM of the present invention has higher detection accuracy and speed for small defective targets in the complex background of near-infrared images of photovoltaic panels compared to the models of CN119359632A and CN116883801A, with improvements of 25% / 11% and 17% / 22% respectively.

[0077] The above-described embodiments are merely described as the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A photovoltaic panel defect detection method based on an improved YOLOv8s model, comprising the step of inputting a solar panel image into the improved YOLOv8s model to detect the photovoltaic panel defects, characterized in that: The improved YOLOv8s model includes a backbone network, a neck network and a head network. The backbone network is a YOLOv8s model backbone network in which all Conv modules except the first one are replaced with Ghost-Conv modules, and the SPPF module is replaced with an SPPCSPC module; The neck network is a YOLOv8s model neck network in which the Concat module is replaced with the BiFPN_Concat module; The head network is a YOLOv8s model head network in which one P2 layer detection head is added to three detection heads and a CBAM module is added to the front end of four detection heads; preferably, the resolution of the P2 layer detection head is 160x 160 pixels.

2. The photovoltaic panel defect detection method based on the improved YOLOv8s model according to claim 1 is characterized in that: The backbone network includes a Conv module, a Ghost-Conv module, a C2f module, and a SPPCSPC module; The neck network includes a BiFPN_Concat module, a Ghost-Conv module, and an Upsample module; The head network includes four CBAM-Detect target detection heads, namely, a CBAM-Detect small target detection head for generating feature vector P2 and detecting small targets, a CBAM-Detect small-scale target detection head for generating feature vector P3 and detecting small-scale targets, a CBAM-Detect medium-scale target detection head for generating feature vector P4 and detecting medium-scale targets, and a CBAM-Detect large-scale target detection head for generating feature vector P5 and large-scale targets.

3. The photovoltaic panel defect detection method based on the improved YOLOv8s model according to claim 1 is characterized in that: The Ghost-Conv module includes three parts: conventional convolution, Ghost generation module and feature map splicing.

4. The photovoltaic panel defect detection method based on the improved YOLOv8s model according to any one of claims 1 to 3, characterized in that: The solar panel image is a near-infrared image of the solar panel.

5. The photovoltaic panel defect detection method based on the improved YOLOv8s model according to any one of claims 1 to 3, characterized in that: The training method of the improved YOLOv8s model includes the following steps: S1: Select the PVEL-AD dataset as training samples; S2: Divide the PVEL-AD dataset into a training set, a validation set, and a test set; S3: Modify the YOLOv8s model to obtain a pre-trained improved YOLOv8s model; S4: input the images included in the training set into the pre-trained improved YOLOv8s model for training for several rounds to obtain the best weight file best.pt; S5: Input the images contained in the test set into the best weight file best.pt, obtain the test results and compare them with the validation set, and use accuracy, recall rate and average precision as model evaluation indicators.

6. The photovoltaic panel defect detection method based on the improved YOLOv8s model according to claim 5 is characterized in that: In step S3, the method for modifying the YOLOv8s model to obtain a pre-trained improved YOLOv8s model includes the following steps: Add CBAM modules to all detection heads; Replace all Conv modules except the first one with Ghost-Conv modules; Replace the SPPF module with the SPPCSPC module; Replace the Concat module with the BiFPN_Concat module.

7. The photovoltaic panel defect detection method based on the improved YOLOv8s model according to claim 6 is characterized in that: The method for modifying the YOLOv8s model to obtain a pre-trained improved YOLOv8s model also includes the following steps: Add the P2 layer detection head; the target detection head includes 3 detection heads and 1 P2 layer detection head.

8. The photovoltaic panel defect detection method based on the improved YOLOv8s model according to claim 5 is characterized in that: In step S1: 4500 labeled solar panel images in the PVEL-AD dataset are selected as training samples; In step S2: the training samples are divided into the training set, the validation set and the test set in a ratio of 7:2:

1.

9. The photovoltaic panel defect detection method based on the improved YOLOv8s model according to claim 1, characterized in that: In step S4: inputting the images included in the training set into the pre-trained improved YOLOv8s model for training for several rounds, and obtaining the best weight file best.pt includes the following steps: The training round epochs is set to 200, the batch size batch-size is set to 4, and the lightweight model yolov8s.pt is used as the pre-training weight file. After the training is completed, the best weight file best.pt is obtained.

10. A photovoltaic panel defect detection system based on an improved YOLOv8s model, characterized in that: The system comprises at least one processor; and a memory storing instructions, which, when executed by the at least one processor, implement the steps of the method according to any one of claims 1 to 9.

Images