Lightweight photovoltaic cell panel defect detection method based on improved YOLOv8

Abstract

The invention belongs to the field of computer vision, and relates to an improved YOLOv8-based light-weight photovoltaic cell panel defect detection method, which comprises the following steps: acquiring a to-be-detected photovoltaic cell panel defect image, and inputting the image into a Faster Net backbone network for multi-scale feature extraction; inputting the multi-scale features into a feature fusion module for fusion to obtain a fused feature map; inputting the fused feature map into a target detection module to obtain a detection result; the output module is used for outputting the detection result; compared with the prior art, by improving the backbone network and the feature fusion module of the YOLOv8, the method has remarkable advantages in the aspects of calculation complexity, detection precision, real-time performance and the like, and particularly has outstanding performance in the aspects of processing multi-scale defects, improving precision and enhancing robustness; the problems of high calculation complexity, insufficient precision and poor multi-scale defect processing capability in the prior art can be effectively solved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and deep learning technology, and specifically relates to a lightweight photovoltaic panel defect detection method based on improved YOLOv8. Background Art

[0002] As an important part of the green energy field, the production quality of photovoltaic panels has a direct impact on power generation efficiency and equipment life. At present, photovoltaic panel defect detection methods are mainly divided into traditional image processing technology and deep learning-based detection technology. Traditional methods (such as edge detection, morphological operations, and image segmentation) usually rely on manually designed rules, which are not only inefficient, but also easily affected by light changes and noise interference.

[0003] Existing photovoltaic panel defect detection methods mainly rely on traditional image processing technology and some deep learning methods. Common image processing methods include edge detection, image segmentation, morphological processing and other methods. These methods usually require manual feature extraction and are sensitive to lighting changes and noise interference. Although these methods can work under certain specific conditions, they cannot cope well with the complexity and diversity of photovoltaic panel surface defects, and are inefficient and inaccurate in practical applications.

[0004] Deep learning, especially target detection technology, has made significant breakthroughs in industrial defect detection. The YOLO (You Only Look Once) series of algorithms are widely used in the industrial field due to their efficient target detection capabilities and real-time performance. However, although the latest YOLOv8 has improved detection accuracy and speed, it still faces the following major problems when dealing with complex defect scenes of photovoltaic panels: 1. The multi-scale feature fusion capability is limited, resulting in insufficient detection accuracy. The existing SPPF module has limitations in dealing with defects of different scales and is difficult to capture detailed information in complex scenes, especially the detection capability of small area and low contrast defects is weak, which is prone to missed detection and false detection; 2. High computational complexity: The amount of calculation is too large when running on embedded devices, which is difficult to meet real-time requirements; existing technologies usually detect a limited number of defect categories (such as cracks, scratches, etc.), but lack a unified detection framework and sufficient accuracy support for complex and diverse defect types (such as bubbles, stains, peeling, etc.). Summary of the invention

[0005] In order to solve the problems existing in the above prior art, the present invention proposes a lightweight photovoltaic panel defect detection method based on improved YOLOv8, the method comprising: obtaining a defect image of a photovoltaic panel to be detected, and preprocessing the image; inputting the preprocessed image into a lightweight photovoltaic panel defect detection model to obtain a defect detection result; the lightweight photovoltaic panel defect detection model is an improved YOLOv8 network, and the improved YOLOv8 network includes a FasterNet backbone network, a feature fusion module, a target detection module and an output module;

[0006] Training a lightweight photovoltaic panel defect detection model includes: obtaining an original image data set, preprocessing the images in the data set, and using the preprocessed images as a training set; inputting the data in the training set into the FasterNet backbone network for multi-scale feature extraction; inputting the multi-scale features into a feature fusion module for fusion to obtain a fused feature map; inputting the fused feature map into a target detection module to obtain a detection result; the output module is used to output the detection result; and calculating the loss function of the model according to the detection result, adjusting the parameters of the model, and completing the model training when the loss function converges.

[0007] Beneficial effects of the present invention:

[0008] By improving the backbone network and feature fusion module of YOLOv8, the present invention has significant advantages over the existing technology in terms of computational complexity, detection accuracy and real-time performance, especially in processing multi-scale defects, improving accuracy and enhancing robustness. It can effectively solve the problems of high computational complexity, insufficient accuracy and poor multi-scale defect processing capability in the existing technology, and provides strong technical support for efficient and accurate detection of photovoltaic panels. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is the overall network architecture diagram of the present invention;

[0010] Figure 2 It is a structural diagram of the FasterNet module of the present invention;

[0011] Figure 3 It is a structural diagram of the feature fusion module of the present invention;

[0012] Figure 4 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] A lightweight photovoltaic panel defect detection method based on improved YOLOv8, such as Figure 4 As shown, the method includes: obtaining a defect image of a photovoltaic panel to be detected, and preprocessing the image; inputting the preprocessed image into a lightweight photovoltaic panel defect detection model to obtain a defect detection result; the lightweight photovoltaic panel defect detection model is an improved YOLOv8 network, and the improved YOLOv8 network includes a FasterNet backbone network, a feature fusion module, a target detection module and an output module; training the lightweight photovoltaic panel defect detection model includes: obtaining an original image data set, preprocessing the images in the data set, and using the preprocessed images as a training set; inputting the data in the training set into the FasterNet backbone network for multi-scale feature extraction; inputting the multi-scale features into the feature fusion module for fusion to obtain a fused feature map; inputting the fused feature map into the target detection module to obtain a detection result; the output module is used to output the detection result; the loss function of the model is calculated according to the detection result, and the parameters of the model are adjusted. When the loss function converges, the model training is completed.

[0015] In this embodiment, making a data set includes: obtaining image data of defects on the surface of photovoltaic cells, first performing data processing and data enhancement on the obtained data set, and then labeling it; and then dividing the processed image data into a training set, a verification set, and a test set. The specific operation steps include:

[0016] Step 1.1: Obtain the photovoltaic cell surface defect image dataset PVEL-AD and perform data enhancement (cropping, horizontal flipping, adjusting saturation, etc.) on the categories with fewer numbers.

[0017] Step 1.2: Divide the data set into a training set: validation set: test set ratio of 7:1.5:1.5. The data in the training set is used to train the model, the data in the validation set is used to verify the trained model, and the data in the test set is used to test the trained model.

[0018] Step 1.3: Initialize the operating environment on the Windows system.

[0019] In this embodiment, if Figure 1As shown in the figure, the improved YOLOv8 network consists of Backbone (FasterNet), feature fusion module (GMSSPPCSPC module), Head (target detection module) and output module.

[0020] Backbone (FasterNet) is responsible for extracting features of the input image and is the core of the entire detection system. FasterNet is used to replace the original backbone network of YOLOv8 to achieve more efficient feature extraction and reduce computational complexity. The feature fusion module (GMSSPPCSPC module) is used to process the multi-scale features extracted from FasterNet. Through the improved SPPCSPC structure, the ability of multi-scale feature fusion is effectively enhanced, and the detection accuracy of small-area, low-contrast defects is improved. Head (target detection module) is responsible for performing the final defect detection task. Based on the detection framework of YOLOv8, this module can perform real-time target recognition and calibrate and locate the defective area of ​​the photovoltaic panel. The output module is responsible for converting the detection results into a user-readable format, which usually includes the type, location and corresponding confidence of the defect.

[0021] Specifically, the image data of the photovoltaic panel is passed to the backbone network (FasterNet) for processing after data enhancement. The backbone network is responsible for extracting low-level and high-level features from the input image. FasterNet uses a lightweight convolutional structure to reduce the amount of calculation while extracting useful visual information, which will be passed to the subsequent feature fusion module for further processing.

[0022] For the device and method of the present invention, the specific implementation schemes of the backbone network (FasterNet) and the feature fusion module (GMSSPPCSPC) are irreplaceable. The specific reasons are as follows:

[0023] Backbone network (FasterNet): FasterNet was selected as the backbone network for feature extraction. Its lightweight structure can significantly reduce the amount of computation and meet the needs of real-time detection. Although other lightweight neural networks (such as MobileNet, EfficientNet, etc.) can also be used for feature extraction, FasterNet performs better in this task and has better compatibility with other components. Therefore, there are limited alternatives to the backbone network.

[0024] GMSSPPCSPC module: The GMSSPPCSPC module improves the effect of multi-scale feature fusion by combining spatial pyramid pooling (SPP) and CSPC (CrossStage Partial Convolution), and can effectively extract features at different scales. Although SPPF and other pooling methods have similar functions, the GMSSPPCSPC module has higher accuracy and flexibility.

[0025] like Figure 2 As shown in the figure, the FasterNet backbone network consists of a PConv module, a FasterNetBlock module and a multi-stage feature extraction network; using the FasterNet backbone network to extract multi-scale features from an image includes: inputting the input image into the PConv module for feature extraction; inputting the extracted feature map into the FasterNetBlock module for deep feature extraction; and inputting the deep feature map into the multi-stage feature extraction network for multi-scale feature extraction.

[0026] Specifically, FasterNetBlock processes data in the following ways: Input stage: Input the input feature map to the PConv layer; Convolution layer: Add a standard convolution layer Conv to further extract features. Normalization and activation: Add Batch Normalization (BN) and ReLU activation functions after the convolution layer. Residual connection: Implement the jump connection between input and output to form a residual structure.

[0027] Building a multi-stage feature extraction network includes:

[0028] Step 1: Design a multi-stage structure: Divide the network into multiple stages, each of which contains several FasterNetBlocks. Use the Merging operation to fuse features between each stage. As the number of stages increases, the number of channels is gradually increased to capture more feature information.

[0029] Step 2: Build a global feature extraction module. After the last stage, apply Global Pool to perform global average pooling on the feature map. Use Conv1x1 convolution layer to compress the pooled features. Add a fully connected layer (FC) for the final classification output.

[0030] like Figure 3 As shown, build the GMSSPPCSPC network model; the operation steps include:

[0031] Step 1. Network input and initial convolution operation: The input feature map first passes through a grouped convolution layer GConv1x1 to achieve grouping and compression of feature channels. Use depthwise separable convolution to perform spatial-level convolution operations on the feature map to reduce computational complexity. After DWConv, the feature channels are rearranged through the Channel_Shuffle operation to improve information interaction between channels.

[0032] Step 2, residual connection module: add the output of Channel_Shuffle to the initial input features to form a residual connection.

[0033] Step 3: Convolutional feature extraction: Perform group convolution on the feature map after residual connection to extract deeper spatial and channel features.

[0034] Step 4, multi-scale pooling module: Use MaxPool2D, set the pooling kernel size to k = 5, k = 9 and k = 13 respectively, and extract multi-scale context features. The output of the three pooling operations is concatenated with the original feature map in the channel dimension through the Concat operation to form multi-scale features.

[0035] Step 5: Further feature processing: extract features through a combination of a GConv1x1 and a GConv3x3.

[0036] Step 6: Introduction of attention mechanism: Introduce the SimAM attention mechanism on the concatenated feature map to increase the weight of key features.

[0037] Step 7, output module: The last layer of grouped convolution processes the feature map into the final output features.

[0038] Replace the backbone of yolov8n with FasterNet network, and replace SPPF with GMSSPPCSPC. The head part still uses the structure of yolov8. Figure 1 As shown in the figure, the data set produced in step 1 is input into the improved YOLOv8 model for training to obtain a model for photovoltaic cell surface defect detection. The specific operation steps include:

[0039] Step 4.1: Create a file named guangfu.yaml in the cfg / datasets / folder, configure the path, the train training set path, and the val validation set path; then configure the names category, a total of 11 categories, corresponding to 11 types of anomalies: cracks (linear and star-shaped), broken grids, black cores, vertical dislocations, horizontal dislocations, thick lines, scratches, fragments, short circuits, printing errors, and material anomalies.

[0040] Step 4.2: Create a new file FasterNet.py in nn / modules and write the improved code into it; and import the FasterNetBlock module built in step 2 in ultralytics / nn / task.py.

[0041] Step 4.3: Create the GMSSPPCSPC.py file in the ultralytics / nn / module directory and put the improved code in it; and register the GMSSPPCSPC network model in step 3 in ultralytics / nn / tasks.py.

[0042] Step 4.4: Create a folder in ultralytics / cfg / models / v8

[0043] yolov8-FasterNet-GMSSPPCSPC.yaml configuration file completes the configuration of the improved yolov8 network model and obtains the yolov8-FasterNet-GMSSPPCSPC network model.

[0044] Step 4.5: Download the pre-trained weight yolov8.pt from the YOLOv8 open source website, write the train.py file, write the weight address and file address, set epochs = 200, batch = 16; the rest of the parameters are default.

[0045] The target detection module consists of three parallel detection branches, each of which is divided into a classification sub-branch and a regression sub-branch; the three detection branches are used to process features corresponding to different scales; the classification sub-branch consists of a ConvModule convolution module and a Conv2d convolution layer; the structure of the regression sub-branch is the same as that of the classification sub-branch. The target detection module processes the fused feature map by: receiving the three different scale feature maps output by the feature fusion module; inputting each scale feature map into two sub-branches in the corresponding three detection branches; each sub-branch extracts features through its own ConvModule, and uses Conv2d to transform features to obtain bounding box prediction results and category prediction results; calculates the loss function of the classification sub-branch and the loss function of the regression sub-branch; integrates the bounding box prediction and category prediction results of the three main detection branches; uses the non-maximum suppression algorithm to remove overlapping detection boxes; and screens according to the confidence threshold to obtain the final detection result.

[0046] The improved model is evaluated and compared according to the visualization chart in the runs file after running. According to the data set and parameters, ShuffleNetV2, the original YOLOv8n and YOLOV10n are used to compare with the improved model yolov8-FasterNet-GMSSPPCSPC model, and the comparison results are shown in Table 1.

[0047] Table 1 Algorithm comparison

[0048] Model mAP% Params / M FLOPs / G YOLOV8n 0.852 3.00 8.1 ShuffleNetV2 0.857 7.30 14.2 YOLO V10n 0.810 2.51 6.5 The present invention 0.862 2.04 5.4

[0049] This comparative experiment demonstrates the performance difference between the model of the present invention and YOLOV8n and ShuffleNetV2 in photovoltaic panel defect detection.

[0050] The results show that the model of the present invention performs best in mAP (mean average precision), reaching 0.862, which is better than YOLOV8n, YOLOV10n and ShuffleNetV2, indicating that it has higher accuracy in defect detection. At the same time, the number of parameters of the model of the present invention is only 2.04M, compared with 3.00M of YOLOV8n, 2.51M of YOLOV10n and 7.30M of ShuffleNetV2, which significantly reduces the computing resource consumption of the model and improves the deployment efficiency. In addition, the FLOPs / G of the model of the present invention is also relatively low, only 5.4, which is significantly lower than 8.1, 14.2 and 6.5 of the other three models, which means that it has higher computing efficiency and real-time performance. These advantages make the model of the present invention more suitable for deployment in embedded devices or real-time monitoring systems with limited computing resources in practical applications.

[0051] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A lightweight photovoltaic panel defect detection method based on improved YOLOv8, characterized in that: include: Obtain the defect image of the photovoltaic panel to be detected and preprocess the image; input the preprocessed image into the lightweight photovoltaic panel defect detection model to obtain the defect detection result; the lightweight photovoltaic panel defect detection model is an improved YOLOv8 network, and the improved YOLOv8 network includes a FasterNet backbone network, a feature fusion module, a target detection module and an output module; Training a lightweight photovoltaic panel defect detection model includes: obtaining an original image data set, preprocessing the images in the data set, and using the preprocessed images as a training set; The data in the training set is input into the FasterNet backbone network for multi-scale feature extraction; the multi-scale features are input into the feature fusion module for fusion to obtain a fused feature map; the fused feature map is input into the target detection module to obtain the detection result; the output module is used to output the detection result; the loss function of the model is calculated according to the detection result, the parameters of the model are adjusted, and when the loss function converges, the model training is completed.

2. According to a lightweight photovoltaic panel defect detection method based on improved YOLOv8 according to claim 1, it is characterized in that: Image preprocessing includes cropping, horizontal flipping, and saturation adjustment.

3. According to a lightweight photovoltaic panel defect detection method based on improved YOLOv8 according to claim 1, it is characterized in that: The FasterNet backbone network consists of a PConv module, a FasterNetBlock module, and a multi-stage feature extraction network. Using the FasterNet backbone network to extract multi-scale features from images includes: inputting the input image into the PConv module for feature extraction; inputting the extracted feature map into the FasterNetBlock module for deep feature extraction; and inputting the deep feature map into the multi-stage feature extraction network for multi-scale feature extraction.

4. According to claim 3, a lightweight photovoltaic panel defect detection method based on improved YOLOv8 is characterized in that: The FasterNetBlock module processes data by: inputting the feature map into the PConv layer for partial feature convolution; performing deep feature extraction on the feature map after partial convolution to obtain a deep feature map; normalizing and activating the deep feature map, performing residual connection between the processed feature map and the input feature map, and outputting a feature map with local information.

5. According to claim 3, a lightweight photovoltaic panel defect detection method based on improved YOLOv8 is characterized in that: The multi-stage feature extraction network consists of a Merging operation module and multiple Stage modules. The Merging operation module is set between two Stage modules; each Stage module contains multiple FasterNetBlock modules.

6. A lightweight photovoltaic panel defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The feature fusion module performs fusion processing on multi-scale features including: Step 1: Input the multi-scale feature map into a grouped convolutional layer GConv1x1 to obtain feature map 1; Step 2: Input feature map 1 into the depth-wise separable convolutional layer for spatial-level convolution operation to obtain feature map 2. Step 3, perform Channel_Shuffle operation on feature map 2 to obtain feature map 3; Step 4: Perform a residual connection between feature map 3 and the input multi-scale feature map to obtain feature map 4; Step 5: Perform group convolution on feature map 4 to obtain feature map 5 with deeper spatial information and channel information; Step 6: Perform multi-scale pooling on feature map 5, and concatenate the feature map after multi-scale pooling with the input feature map in the channel dimension to obtain a multi-scale feature map 1; Step 7: Extract features from feature map 5 through a GConv1x1 and a GConv3x3 combination to obtain feature map 6, and concatenate feature map 6 with multi-scale feature map 1; Step 8: Introduce the SimAM attention mechanism on the concatenated feature map to obtain feature map 7; Step 9: Perform group convolution processing on feature map 7 to obtain a fused feature map.

7. A lightweight photovoltaic panel defect detection method based on improved YOLOv8 according to claim 6, characterized in that: Multi-scale pooling includes pooling operations at three scales, where the pooling kernel sizes are k=5, k=9, and k=13, respectively.

8. The method for defect detection of lightweight photovoltaic panels based on improved YOLOv8 according to claim 1, characterized in that: The object detection module consists of three parallel detection branches, each of which is divided into a classification sub-branch and a regression sub-branch; the three detection branches are used to process features corresponding to different scales; The classification sub-branch consists of a ConvModule convolution module and a Conv2d convolution layer; the structure of the regression sub-branch is the same as that of the classification sub-branch.

9. A lightweight photovoltaic panel defect detection method based on improved YOLOv8 according to claim 8, characterized in that: The object detection module processes the fused feature map, including: receiving three feature maps of different scales output by the feature fusion module; inputting each scale feature map into two sub-branches in the corresponding three detection branches; each sub-branch extracts features through its own ConvModule, and uses Conv2d to transform features to obtain bounding box prediction results and category prediction results; calculating the loss function of the classification sub-branch and the loss function of the regression sub-branch; integrating the bounding box prediction and category prediction results of the three main detection branches; using the non-maximum suppression algorithm to remove overlapping detection boxes; and filtering according to the confidence threshold to obtain the final detection result.

Images