A multi-scale photovoltaic cell defect detection method based on complex background

Abstract

The present invention discloses a multi-scale photovoltaic cell defect detection method based on complex background, the method comprising: constructing a photovoltaic cell defect detection data set; designing and improving a YOLOv8 network model; inputting the constructed data set into the improved YOLOv8 model for training; ablation experiment to verify the improved model performance; performing comparative experiments on multiple models, comprehensively testing and evaluating model performance; and visualizing the model test results. The present invention adopts multi-scale sequence feature fusion technology, combines bidirectional information flow and learnable weights to enhance the feature expression ability of the network, avoids the loss of shallow semantic information, and thus significantly reduces the false detection and missed detection problems in photovoltaic cell defect detection caused by factors such as complex background interference, variable shapes and multi-scale defects, and can accurately detect photovoltaic cells in complex backgrounds.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a multi-scale photovoltaic cell defect detection method based on complex background. Background Art

[0002] As one of the key technologies to promote global energy transformation, the stability of photovoltaic cell performance and quality plays a vital role in improving the efficiency of solar power generation systems and ensuring the sustainability of energy supply. However, during the manufacturing and application of photovoltaic cells, various factors may cause defects such as cracks, broken grids, and short circuits. These defects will not only reduce the power generation efficiency of photovoltaic cells, but also shorten their service life.

[0003] In the past, photovoltaic cell defect detection mainly relied on manual visual inspection, but this method is not only time-consuming and labor-intensive, but also easily interfered by human factors, making it difficult to ensure the accuracy of the detection results. In recent years, with the rapid development of computer vision and artificial intelligence technology, automated detection methods based on these technologies have gradually been applied to the field of photovoltaic cell defect detection. However, this application process still faces many challenges. On the one hand, the complexity of the photovoltaic cell surface (covering multiple characteristics such as color, texture, and reflection) and the diversity of defect morphology (including crack length, width, direction, scale, etc.) have brought great difficulties to the design and implementation of detection algorithms. On the other hand, most of the current detection methods rely on specific image processing algorithms or models. When these algorithms or models process photovoltaic cell images under different lighting conditions and shooting angles, the accuracy and robustness of their detection results need to be further improved. Summary of the invention

[0004] In view of the problems existing in the prior art, the disclosed embodiment of the present invention provides a multi-scale photovoltaic cell defect detection method based on complex background. The method is based on the YOLOv8 model, adopts a bidirectional feature pyramid network (BiFPN) as a feature fusion mechanism, introduces a context aggregation module (CAM) in the Neck network, optimizes the loss function as Shape-IoU, and debugs the weight factor (scale), which significantly reduces the problem of false detection and missed detection of photovoltaic cell defects caused by complex background interference, variable shapes and multi-scale defects. At the same time, the constructed model is small in size and has a small number of parameters, which is suitable for practical application scenarios with limited resources.

[0005] The present invention provides the following technical solutions:

[0006] The present invention provides a multi-scale photovoltaic cell defect detection method based on complex background, comprising the following steps:

[0007] Step 1: Build a photovoltaic cell defect detection dataset;

[0008] Step 2: Based on the Pytorch deep learning framework, the YOLOv8 model is improved to build a multi-scale photovoltaic cell defect detection model based on complex backgrounds;

[0009] Step 3: Input the constructed photovoltaic cell defect detection dataset into the improved YOLOv8 algorithm model for training. Optimize the performance of the improved YOLOv8 model by optimizing the loss function and debugging parameters, and continuously improve the detection accuracy and generalization ability of the model.

[0010] Step 4: Test the original YOLOv8 model and the improved model, and verify the accuracy and efficiency of each module in detecting photovoltaic cell defects through ablation experiments, including evaluation of defect location, defect category, and confidence level.

[0011] Step 5: On the validation set, compare and analyze the improved YOLOv8 model with the current mainstream algorithm, and comprehensively evaluate the advantages and disadvantages of different models from the two dimensions of detection performance and real-time frame rate;

[0012] Step 6: Test the model on the test set, evaluate the actual application effect, and visualize the test results.

[0013] Among them, the improved YOLOv8 model structure consists of a Backbone module, a Neck module and a Head module connected in sequence. The Backbone module starts from the initial 64-channel convolution layer and deepens layer by layer until it reaches 1024 channels. Each layer of the convolution layer extracts features through a specific convolution module, and changes the scale of the feature map by adjusting the step size or the convolution kernel size. The Neck module fuses feature maps of different scales through BiFPN to form a series of feature maps with rich information. The Head module makes predictions based on the fused feature map, and outputs the detection results of the target through steps such as upsampling, BiFPN fusion and convolution operations.

[0014] The Backbone module includes a first CBS unit, a second CBS unit, a first C2f unit, a third CBS unit, a second C2f unit, a fourth CBS unit, a third C2f unit, a fifth CBS unit, a fourth C2f unit and a SPFF unit which are connected in sequence.

[0015] The output of the first C2f unit is connected to the input of the ninth CBS unit, the output of the second C2f unit is connected to the input of the tenth CBS unit, the output of the third C2f unit is connected to the input of the eleventh CBS unit, and the output of the SPPF is connected to the input of the sixth CBS unit.

[0016] The first CBS unit, the second CBS unit, the third CBS unit, the fourth CBS unit and the fifth CBS unit are all used for feature extraction, the first C2f unit, the second C2f unit and the third C2f unit are used for feature processing, and the SPFF unit is used for feature fusion of feature maps of different scales.

[0017] The Neck module includes a sixth CBS unit, a first upsampling unit, a first Fusion unit, a fifth C2f unit, a second upsampling unit, a second Fusion unit, a sixth C2f unit, a third Fusion unit, a seventh C2f unit, a first CAM unit, a seventh CBS unit, a fourth Fusion unit, an eighth C2f unit, an eighth CBS unit, a fifth Fusion unit, and a ninth C2f unit connected in sequence.

[0018] The output of the ninth CBS unit is connected to the input of the third Fusion unit, the output of the tenth CBS unit is connected to the inputs of the second Fusion unit and the third Fusion unit respectively, the output of the eleventh CBS unit is connected to the inputs of the first Fusion unit and the fourth Fusion unit respectively, and the output of the sixth CBS unit is connected to the input of the fifth Fusion unit.

[0019] The Head module includes a first Head unit, a second Head unit and a third Head unit.

[0020] The first Head unit is connected to the output of the first CAM unit, the second Head unit is connected to the output of the eighth C2f unit, and the third Head unit is connected to the output of the ninth C2f unit.

[0021] In step 1, a photovoltaic cell defect dataset is constructed, including:

[0022] 1-1) Perform data cleaning operations on the PVEL-AD dataset, remove images with incomplete labels and a small number of category samples, and retain five common defect images: cracks, star-shaped cracks, short circuits, broken grids, and black cores;

[0023] 1-2) Design and implement a random sampling strategy to select 6936 images from the cleaned data set to construct a data set, including 4855 positive samples and 1388 negative samples, which are divided into training set, validation set and test set in a ratio of 7:2:1.

[0024] In step 2, a multi-scale photovoltaic cell defect detection model based on complex background is constructed, including:

[0025] 2-1) Configure the environment: Use Anaconda to create a virtual environment for the source code of the YOLOv8 model;

[0026] 2-2) Data input: Input the constructed photovoltaic cell defect dataset into the YOLOv8 network structure model, including obtaining photovoltaic cell defect images, photovoltaic cell defect image labels, YOLOv8 network structure model pre-training weights, and YOLOv8 network structure model configuration file;

[0027] 2-3) Configure hyperparameters: Set the optimal hyperparameter combination of learning rate, batch size, number of iterations, number of image channels, image cropping size, and learning rate momentum. Specifically, use the Adam optimizer, set the batch size to 32, the initial learning rate to 0.01, the maximum number of training times to 100, and use the exponential decay method to dynamically adjust the learning rate to speed up the network convergence.

[0028] 2-4) Improve the YOLOv8 network structure model, including:

[0029] 2-4-1) Introduce the bidirectional feature pyramid network (BiFPN) as a feature fusion mechanism;

[0030] 2-4-2) Add a context aggregation module (CAM) to the Neck network.

[0031] In step 2-4-1), the bidirectional feature pyramid network (BiFPN) fusion method is adopted to introduce a jump connection between the input and output features of the same layer, which has more efficient feature fusion and stronger feature representation capabilities. The calculation formula of the two fused features is as follows:

[0032]

[0033] Among them, P i in represents the i-th input feature layer, P i td represents the i-th transition feature layer, P i out represents the i-th output feature layer; w1, w2 and w3 are automatically learned weights, corresponding to the weights of the input feature layer, transition feature layer and the output of the previous output feature layer respectively; ò is a very small constant introduced to avoid the denominator being zero, and Conv represents the convolution operation on the calculation result. In formula (1), the current input feature layer and the next higher resolution input feature layer are fused by weighted summation, and the transition feature layer is obtained by convolution. Formula (2) further fuses the current input feature layer, transition feature layer and the previous output feature layer to finally obtain the output feature layer.

[0034] In step 2-4-2), the introduction of the CAM module enables the improved model to effectively capture and integrate features of different scales from the input data, improve the model's ability to understand and parse complex scenes, and obtain more global contextual information while retaining local details. When performing defect detection on photovoltaic cells, it can better handle the scale changes and occlusion of the target, and provide richer semantic information through feature recalibration, reducing false detections and missed detections.

[0035] The overall network structure of the improved model is as follows:

[0036] First convolutional layer: convolution kernel size is 3x3, number of filters is 64, stride=2;

[0037] Second convolutional layer: convolution kernel size is 3x3, number of filters is 128, stride=2;

[0038] The C2f module is repeated three times: the number of filters is 128;

[0039] The third convolutional layer: the convolution kernel size is 3x3, the number of filters is 256, and the stride is 2;

[0040] The C2f module is repeated six times: the number of filters is 256;

[0041] Fourth convolutional layer: convolution kernel size is 3x3, number of filters is 512, stride=2;

[0042] The C2f module is repeated six times: number of filters 512;

[0043] Fifth convolutional layer: convolution kernel size is 3x3, number of filters is 1024, stride=2;

[0044] The C2f module is repeated three times: the number of filters is 1024;

[0045] SPPF module: number of filters / parameters is 1024, 5;

[0046] Upsampling layer: method nearest, scaling factor 2; belongs to the first layer of the head part;

[0047] BiFPN_Concat2 module: input comes from the current layer and one of the previous six layers; concatenates the backbone P4 feature map;

[0048] The C2f module is repeated three times: the number of filters is 512

[0049] Upsampling layer again: method nearest, scaling factor 2; belongs to the next layer of the head part;

[0050] BiFPN_Concat2 module: input comes from the current layer and one of the previous four layers; concatenates the backbone P3 feature map;

[0051] The C2f module is repeated three times: the number of filters is 256; the corresponding number of layers is immediately after the second BiFPN_Concat2 module;

[0052] CAM module: number of filters: 256;

[0053] Convolution layer: convolution kernel size 3x3, number of filters 256, stride 1; belongs to feature map downsampling preparation;

[0054] BiFPN_Concat3 module: input comes from the current layer, the previous twelve layers, and one of the previous six layers; concatenates the P4 feature map of the head;

[0055] The C2f module is repeated three times: the number of filters is 512;

[0056] Convolution layer: convolution kernel size 3x3, number of filters 512, stride 1; belongs to feature map downsampling preparation;

[0057] BiFPN_Concat2 module: input comes from the current layer and one of the previous nine layers; concatenates the P5 feature map of the head;

[0058] The C2f module is repeated three times: the number of filters is 1024;

[0059] Detect module: inputs the concatenated feature maps from the 15th, 18th, and 21st layers; performs target detection and outputs the detection results.

[0060] In step 3, the constructed dataset is input into the improved YOLOv8 model for training, including:

[0061] 3-1) Input the training set data into the model and adjust the model weights through the back propagation and gradient descent optimization algorithm;

[0062] 3-2) Use the validation set to evaluate model performance and adjust hyperparameters to avoid overfitting or underfitting;

[0063] 3-3) Optimize the loss function to Shape-IoU;

[0064] 3-4) Adjust the weight factor (scale) to different values ​​to optimize the model.

[0065] In step 3-3), Shape-IoU not only inherits CIoU's consideration of overlapping area and center point distance, but also emphasizes the shape and scale information of the bounding box, so that the model can more accurately capture the characteristics of the target during training, especially for some targets with complex shapes or large size changes. In addition, the improved model is more accurate in measuring the matching degree between the predicted box and the true box, reducing excessive reliance on specific shapes (such as sharp vertical lines), thereby improving the accuracy and stability of photovoltaic cell defect detection. The bounding box loss calculation formula of Shape-IoU is as follows:

[0066]

[0067] Among them, IoU is an indicator to measure the degree of overlap between the predicted bounding box and the true bounding box, and distance shape is the difference between the predicted value and the true value, 0.5×Ω shape It is to adjust the weight in the loss function.

[0068] In step 4, the trained YOLOv8 model and the improved YOLOv8 model are tested, and the weight file generated after the model training is loaded into the test file for model testing. The test results of the improved photovoltaic cell defect detection model on the test set are obtained, including defect location, defect category and confidence, and the effectiveness of each module is verified through ablation experiments.

[0069] In step 5, the detection performance of different models is comprehensively evaluated, and mAP@0.5%, Recall, FLOPs and number of parameters Params are selected as evaluation indicators.

[0070] In step 6, the training set is input into YOLOv5, YOLOv7-tiny, YOLOv8 and the model of the present invention for testing, and the test results are visualized.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] 1. The present invention designs a bidirectional feature pyramid network (BiFPN) as a feature fusion mechanism to enhance the semantic and position information of features. By introducing a dynamic weighting method, feature maps of different scales are differentiated and weighted fused, so that the algorithm can accurately identify and effectively utilize the criticality of features of each scale in a specific task, and assign reasonable weights to the detection task;

[0073] 2. The present invention designs a context aggregation module (CAM), which aggregates context information and uses dilated convolutions with different dilated convolution rates to obtain context information of different receptive fields, further improving the network's ability to capture subtle defects in complex backgrounds;

[0074] 3. Based on the YOLOv8 model provided, the present invention introduces Shape-IoU as a new loss metric, so that the model can more accurately capture subtle changes in the target shape during training, optimize the regression accuracy of the detection frame, and at the same time, by carefully debugging the parameters of the weight factor (scale), ensure that it can reach the optimal state when processing various types of defects. The present invention realizes the detection of photovoltaic cell defects under complex backgrounds, overcomes the problem that the detection effect of traditional methods is limited in complex environments, and significantly improves the efficiency of actual production. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a flow chart of the photovoltaic cell defect detection method based on improved YOLOv8 of the present invention;

[0076] Figure 2 This is the overall block diagram of the improved YOLOv8 model;

[0077] Figure 3 Schematic diagram of BiFPN structure;

[0078] Figure 4 It is a schematic diagram of CAM structure;

[0079] Figure 5 This is a schematic diagram of the intersection area parameters of the loss function prediction box and the true box;

[0080] Figure 6 This is the mAP@0.5 comparison chart of the ablation experiment;

[0081] Figure 7 This is a comparison chart of the detection results of different models. DETAILED DESCRIPTION

[0082] The specific implementation modes of the present invention are described in detail below with reference to the accompanying drawings.

[0083] The present invention introduces a bidirectional feature pyramid network (BiFPN) as the Neck part of the YOLOv8 model for multi-scale feature fusion, adds a context aggregation module (CAM) to enhance the mining capability of small target information, and changes the loss function to Shape-IoU to improve the detection accuracy. It effectively solves the problems of false detection and missed detection caused by complex background interference, variable shapes and multi-scale defects in photovoltaic cell defect detection.

[0084] The key point of the present invention is to improve the YOLOv8 model to meet the needs of photovoltaic cell defect detection. Specifically, the neck network adopts a bidirectional feature pyramid network (BiFPN) to achieve dynamic weighted fusion of multi-scale feature maps, thereby optimizing the importance learning of features and removing isolated nodes to reduce computational complexity. In addition, a context aggregation module (CAM) is designed and embedded, which integrates different receptive field features and local context information through an adaptive learning mechanism, significantly improving the detection ability of small targets. The Shape-IoU loss function is used to reduce the model's dependence on specific shapes and improve the matching accuracy between the predicted box and the real box. By adjusting the weight factor (scale), the optimization of multi-category detection is achieved, further improving the detection performance and robustness of the model. Under the combined effect of these improvements, the detection accuracy is significantly improved, the computational complexity is reduced, and the practicality and reliability of the model in practical applications are enhanced.

[0085] The innovations of the present invention include the application of Bidirectional Feature Pyramid Network (BiFPN) in the Neck part of the YOLOv8 model and the implementation of its dynamic weighted feature fusion, the design of Context Aggregation Module (CAM) and its application in enhancing the detection capability of small targets, and the introduction of Shape-IoU loss function and its role in improving detection accuracy. These innovations not only enrich the detection methods of the model, but also improve the robustness and adaptability of the model, enabling it to maintain high accuracy and stability in the complex and changeable photovoltaic cell defect detection environment.

[0086] Compared with the prior art, the model training proposed in the present invention is different from the prior art in that:

[0087] At the basic algorithm level, the present invention uses an optimized YOLOv8 model for training, which is significantly different from many patents that use other basic algorithms (such as YOLOv5, Faster R-CNN, etc.). These different basic algorithms have their own characteristics in model structure, feature extraction methods and processing procedures.

[0088] In terms of processing objects, the present invention focuses on the detection of electroluminescent defects in photovoltaic cells, while other existing patents may focus on the detection of surface defects of PCB boards or other types of defects. This difference in processing objects directly affects the pertinence of model design and the selection of optimization strategies.

[0089] In terms of the improved method, the present invention introduces the Bidirectional Feature Pyramid Network (BiFPN) as the core mechanism of feature fusion, and innovatively designs the Context Aggregation Module (CAM) and embeds it into the neck of the network. This approach is in sharp contrast to many patents that use other deep learning algorithms.

[0090] In terms of parameter fine-tuning, the present invention adopts the Adam optimizer and dynamically adjusts the learning rate through the exponential training strategy, which effectively accelerates the network convergence process. At the same time, during the training process, the present invention also performs multiple rounds of value adjustment on the weight factor (scale) in order to obtain better performance.

[0091] In terms of loss function, the present invention adopts Shape-IoU loss function as a new metric to reduce the model's dependence on specific shapes and improve the matching accuracy between the predicted box and the real box.

[0092] In summary, the present invention shows significant differences from existing patents in many key aspects. These innovations make the method of the present invention more in line with the actual needs of photovoltaic cell electroluminescent defect detection, and provide strong support for efficient and accurate defect detection.

[0093] Reference Figure 1 As shown, the present invention relates to a multi-scale photovoltaic cell defect detection method based on complex background, comprising:

[0094] Step 1: Build a photovoltaic cell defect detection dataset;

[0095] Step 2: Based on the YOLOv8 model, the Pytorch deep learning framework is improved, the bidirectional pyramid feature fusion network (BiFPN) is introduced as the feature fusion mechanism, and the context aggregation module (CAM) is introduced into the Neck network to build a multi-scale photovoltaic cell defect detection model based on complex background;

[0096] Step 3: Input the constructed photovoltaic cell defect detection dataset into the improved YOLOv8 algorithm model for training. By optimizing the loss function and debugging parameters, the performance of the improved YOLOv8 model is optimized to continuously improve the detection accuracy and generalization ability of the model.

[0097] Step 4: Test the original YOLOv8 model and the improved model, and verify the accuracy and efficiency of detecting photovoltaic cell defects through ablation experiments, including evaluation of defect location, defect category, and confidence level.

[0098] Step 5: On the validation set, compare and analyze the improved YOLOv8 model with the current mainstream algorithm, and comprehensively evaluate the advantages and disadvantages of different models from the two dimensions of detection performance and real-time frame rate;

[0099] Step 6: Test the model on the test set, evaluate the actual application effect, and visualize the test results.

[0100] Among them, Figure 2As shown in the figure, the improved YOLOv8 model structure consists of a Backbone module, a Neck module and a Head module connected in sequence. The Backbone module starts from the initial 64-channel convolution layer and deepens layer by layer until it reaches 1024 channels. Each layer of the convolution layer extracts features through a specific convolution module, and changes the scale of the feature map by adjusting the step size or the convolution kernel size. The Neck module fuses feature maps of different scales through BiFPN to form a series of feature maps with rich information. The Head module makes predictions based on the fused feature map, and outputs the detection results of the target through steps such as upsampling, BiFPN fusion and convolution operations.

[0101] The Backbone module includes a first CBS unit, a second CBS unit, a first C2f unit, a third CBS unit, a second C2f unit, a fourth CBS unit, a third C2f unit, a fifth CBS unit, a fourth C2f unit and an SPFF unit connected in sequence. The output of the first C2f unit is connected to the input of the ninth CBS unit, the output of the second C2f unit is connected to the input of the tenth CBS unit, the output of the third C2f unit is connected to the input of the eleventh CBS unit, and the output of the SPPF is connected to the input of the sixth CBS unit. The first CBS unit, the second CBS unit, the third CBS unit, the fourth CBS unit and the fifth CBS unit are all used for feature extraction, the first C2f unit, the second C2f unit and the third C2f unit are used for feature processing, and the SPFF unit is used to fuse the features of feature maps of different scales.

[0102] The Neck module includes a sixth CBS unit, a first upsampling unit, a first Fusion unit, a fifth C2f unit, a second upsampling unit, a second Fusion unit, a sixth C2f unit, a third Fusion unit, a seventh C2f unit, a first CAM unit, a seventh CBS unit, a fourth Fusion unit, an eighth C2f unit, an eighth CBS unit, a fifth Fusion unit, and a ninth C2f unit connected in sequence. The output of the ninth CBS unit is connected to the input of the third Fusion unit, the output of the tenth CBS unit is connected to the inputs of the second Fusion unit and the third Fusion unit respectively, the output of the eleventh CBS unit is connected to the inputs of the first Fusion unit and the fourth Fusion unit respectively, and the output of the sixth CBS unit is connected to the input of the fifth Fusion unit.

[0103] The Head module includes a first Head unit, a second Head unit and a third Head unit, wherein the first Head unit is connected to the output of the first CAM unit, the second Head unit is connected to the output of the eighth C2f unit, and the third Head unit is connected to the output of the ninth C2f unit.

[0104] The following is a detailed introduction to this embodiment:

[0105] The main process of the YOLOv8 network includes data set creation, model improvement, model training and other stages.

[0106] Step 1. Perform data cleaning operations on the PVEL-AD dataset, remove images with incomplete labels and a small number of category samples, retain five common defect images of cracks, star-shaped cracks, short circuits, broken grids and black cores, design and implement a random extraction strategy, and select 6936 images from the cleaned dataset to construct a dataset, including 4855 positive samples and 1388 negative samples. The training set, validation set and test set are divided into a ratio of 7:2:1 to produce a photovoltaic cell defect detection dataset.

[0107] Step 2: To address the problems of photovoltaic cell defects, such as complex background interference, uneven shapes, and multi-scale defects, this embodiment uses the YOLOv8 model as the basis, adopts the bidirectional feature pyramid network (BiFPN) as the feature fusion mechanism, and designs a context aggregation module (CAM) to introduce into the Neck network. The BiFPN module is as follows: Figure 3 As shown, the context aggregation module (CAM) is Figure 4 shown.

[0108] BiFPN is a bidirectional feature pyramid network that can fuse feature information of different scales. It uses the fusion method of BIFPN and introduces jump connections between the input and output features of the same layer, which has more efficient feature fusion and stronger feature representation capabilities. The calculation formula of the two fused features is as follows:

[0109]

[0110] Among them, P i in represents the i-th input feature layer, P i td represents the i-th transition feature layer, P i outrepresents the i-th output feature layer; w1, w2 and w3 are automatically learned weights, corresponding to the weights of the input feature layer, transition feature layer and the output of the previous output feature layer respectively; ò is a very small constant introduced to avoid the denominator being zero, and Conv represents the convolution operation on the calculation result. In formula (1), the current input feature layer and the next higher resolution input feature layer are fused by weighted summation, and the transition feature layer is obtained by convolution. Formula (2) further fuses the current input feature layer, transition feature layer and the previous output feature layer to finally obtain the output feature layer.

[0111] When integrating the multi-scale features of photovoltaic cell defects, shallow detail information is easily covered by deep semantic information, resulting in loss of spatial details and affecting the detection of small targets and dense areas. This embodiment introduces the CAM module, so that the improved model can effectively capture and integrate features of different scales from the input data, improve the model's ability to understand and analyze complex scenes, and obtain more global context information while retaining local details. When performing defect detection on photovoltaic cells, it can better handle the scale changes and occlusion of the target, and provide richer semantic information through feature recalibration, reducing false detections and missed detections.

[0112] The overall network structure of the improved model is as follows:

[0113] First convolutional layer: convolution kernel size is 3x3, number of filters is 64, stride=2;

[0114] Second convolutional layer: convolution kernel size is 3x3, number of filters is 128, stride=2;

[0115] The C2f module is repeated three times: the number of filters is 128;

[0116] The third convolutional layer: the convolution kernel size is 3x3, the number of filters is 256, and the stride is 2;

[0117] The C2f module is repeated six times: the number of filters is 256;

[0118] Fourth convolutional layer: convolution kernel size is 3x3, number of filters is 512, stride=2;

[0119] The C2f module is repeated six times: number of filters 512;

[0120] Fifth convolutional layer: convolution kernel size is 3x3, number of filters is 1024, stride=2;

[0121] The C2f module is repeated three times: the number of filters is 1024;

[0122] SPPF module: number of filters / parameters is 1024, 5;

[0123] Upsampling layer: method nearest, scaling factor 2; belongs to the first layer of the head part;

[0124] BiFPN_Concat2 module: input comes from the current layer and one of the previous six layers; concatenates the backbone P4 feature map;

[0125] The C2f module is repeated three times: the number of filters is 512

[0126] Upsampling layer again: method nearest, scaling factor 2; belongs to the next layer of the head part;

[0127] BiFPN_Concat2 module: input comes from the current layer and one of the previous four layers; concatenates the backbone P3 feature map;

[0128] The C2f module is repeated three times: the number of filters is 256; the corresponding number of layers is immediately after the second BiFPN_Concat2 module;

[0129] CAM module: number of filters: 256;

[0130] Convolution layer: convolution kernel size 3x3, number of filters 256, stride 1; belongs to feature map downsampling preparation;

[0131] BiFPN_Concat3 module: input comes from the current layer, the previous twelve layers, and one of the previous six layers; concatenates the P4 feature map of the head;

[0132] The C2f module is repeated three times: the number of filters is 512;

[0133] Convolution layer: convolution kernel size 3x3, number of filters 512, stride 1; belongs to feature map downsampling preparation;

[0134] BiFPN_Concat2 module: input comes from the current layer and one of the previous nine layers; concatenates the P5 feature map of the head;

[0135] The C2f module is repeated three times: the number of filters is 1024;

[0136] Detect module: inputs the concatenated feature maps from the 15th, 18th, and 21st layers; performs target detection and outputs the detection results.

[0137] Step 3: Input the constructed dataset into the improved YOLOv8 model for training, set model hyperparameters, such as Adam optimizer, batch size, initial learning rate, maximum number of training times, etc., input the training set data into the model, adjust the model weights through the back propagation and gradient descent optimization algorithm, use the validation set to evaluate the model performance, and adjust the hyperparameters to avoid overfitting or underfitting. The optimized loss function is Shape-IoU, such as Figure 5 , Shape-IoU not only inherits CIoU's consideration of overlapping area and center point distance, but also emphasizes the shape and scale information of the bounding box, which enables the model to capture the characteristics of the target more accurately during training, especially for some targets with complex shapes or large size changes. In addition, the improved model is more accurate in measuring the matching degree between the predicted box and the true box, reducing excessive reliance on specific shapes (such as sharp vertical lines), thereby improving the accuracy and stability of photovoltaic cell defect detection. The bounding box loss calculation formula of Shape-IoU is as follows:

[0138]

[0139] Among them, IoU is an indicator to measure the degree of overlap between the predicted bounding box and the true bounding box, and distance shape is the difference between the predicted value and the true value, 0.5×Ω shape It is to adjust the weight in the loss function.

[0140] Step 4. Load the weight file generated after the model training is completed into the validation set for model testing. During this process, it is necessary to debug the size of the weight factor (scale) to obtain the test results of the improved defect detection model. These results will include the specific location of the defect, the category of the defect, and the corresponding confidence. Next, the improved model is compared with the original YOLOv8 model in an ablation experiment. During this process, BiFPN, CAM, and the loss function are gradually added to the YOLOv8 model to observe their impact on model performance. The experimental results are shown in Table 1. At the same time, the comparison of the mAP@0.5% curve is shown in Table 1. Figure 6 shown.

[0141] Table 1:

[0142] Experimental group name BiFPN CAM Shape-IoU Precision / % Recall / % mAP@0.5 / % Parameters / MB Control group — — — 93.0 79.0 91.4 11.13 1 √ — — 88.3 85.6 91.9 11.19 2 √ √ — 91.3 82.4 92.7 12.29 3 √ √ √ 86.3 90.4 93.2 12.29

[0143] From the data in Table 1, we can see that after the introduction of the Bidirectional Feature Pyramid Network (BiFPN) in the Neck part, the model's ability to detect target objects is enhanced by effectively fusing multi-scale features, and the recall rate and average precision are improved. Furthermore, when the context aggregation module (CAM) is added to the model, the model can more accurately locate the key areas of the target object, thereby improving the detection accuracy and average precision. Finally, in order to further improve the regression performance of the model's bounding box, the loss function is optimized to Shape-IoU. By more finely measuring the shape difference of the bounding box, the model effectively improves the prediction accuracy of the target object's bounding box, thereby further improving the detection performance of the model.

[0144] Step 5: Comprehensively evaluate the detection performance of different models. Use the training set to train YOLOv5, YOLOv6, YOLOv7-tiny and YOLOv8 respectively, and verify the model in the validation set. Use mAP@0.5%, Recall, FLOPs and number of parameters as evaluation indicators. The comparison of experimental results of different models is shown in Table 2.

[0145] Table 2:

[0146] Experimental Algorithm Precision / % Recall / % mAP@0.5 / % Parameters / MB <![CDATA[FPS / (frames per second -1 )]]> YOLOv5 86.2 85.4 89.9 7.0 175 YOLOv6 84.8 82.6 83.2 16.3 258 YOLOv7-tiny 87.5 77.1 86.9 6.0 161 YOLOv8 93.0 79.0 91.4 11.13 278 Method of the present invention 86.3 90.4 93.2 12.30 270

[0147] As shown in Table 2, the average precision mean and recall rate of the method of the present invention are significantly higher than those of other algorithms. At the same time, compared with the original YOLOv8 algorithm, the recall rate is increased by 10.4% and the average precision mean is increased by 1.8%, indicating that the method of the present invention can detect the defects of photovoltaic cells more comprehensively, and the FPS is maintained at 270 frames / s, ensuring good real-time performance.

[0148] Step 6: Model testing and reasoning. The photovoltaic cell defect images to be identified are input into YOLOv5, YOLOv7-tiny, YOLOv8 and the model of the present invention respectively, and the detection results are output, such as Figure 7 As shown in the figure, it can be seen that the present invention can accurately detect various defects with high confidence.

[0149] The above content only shows the preferred embodiments of the present invention, but its protection scope is not limited thereto. Any person with professional knowledge in this technical field, within the technical framework disclosed by the present invention, replaces the data set or makes any form of adjustment, equivalent substitution and optimization improvement under the guidance of the core concept and basic principles of the present invention, shall be deemed to fall within the protection scope of the present invention.

Claims

1. A multi-scale photovoltaic cell defect detection method based on complex background, characterized in that: The steps include: Step 1: Build a photovoltaic cell defect detection dataset; Step 2: Based on the Pytorch deep learning framework, the YOLOv8 model is improved to build a multi-scale photovoltaic cell defect detection model based on complex backgrounds; Step 3: Input the constructed photovoltaic cell defect detection dataset into the improved YOLOv8 algorithm model for training. By optimizing the loss function and debugging parameters, the performance of the improved YOLOv8 model is optimized to continuously improve the detection accuracy and generalization ability of the model. Step 4: Test the original YOLOv8 model and the improved model, and verify the accuracy and efficiency of each module in detecting photovoltaic cell defects through ablation experiments, including evaluation of defect location, defect category, and confidence level. Step 5: On the validation set, compare and analyze the improved YOLOv8 model with the current mainstream model algorithm, and comprehensively evaluate the advantages and disadvantages of different models from the two dimensions of detection performance and real-time frame rate; Step 6: Test the model on the test set, evaluate the actual application effect, and visualize the test results. Among them, the improved YOLOv8 model structure consists of a Backbone module, a Neck module and a Head module connected in sequence. The Backbone module starts from the initial 64-channel convolution layer and deepens layer by layer until it reaches 1024 channels. Each layer of the convolution layer extracts features through a specific convolution module, and changes the scale of the feature map by adjusting the step size or the convolution kernel size. The Neck module fuses feature maps of different scales through BiFPN to form a series of feature maps with rich information. The Head module makes predictions based on the fused feature map, and outputs the detection results of the target through steps such as upsampling, BiFPN fusion and convolution operations. The Backbone module includes a first CBS unit, a second CBS unit, a first C2f unit, a third CBS unit, a second C2f unit, a fourth CBS unit, a third C2f unit, a fifth CBS unit, a fourth C2f unit and a SPFF unit which are connected in sequence. The output of the first C2f unit is connected to the input of the ninth CBS unit, the output of the second C2f unit is connected to the input of the tenth CBS unit, the output of the third C2f unit is connected to the input of the eleventh CBS unit, and the output of the SPPF is connected to the input of the sixth CBS unit. The first CBS unit, the second CBS unit, the third CBS unit, the fourth CBS unit and the fifth CBS unit are all used for feature extraction, the first C2f unit, the second C2f unit and the third C2f unit are used for feature processing, and the SPFF unit is used for feature fusion of feature maps of different scales. The Neck module includes a sixth CBS unit, a first upsampling unit, a first Fusion unit, a fifth C2f unit, a second upsampling unit, a second Fusion unit, a sixth C2f unit, a third Fusion unit, a seventh C2f unit, a first CAM unit, a seventh CBS unit, a fourth Fusion unit, an eighth C2f unit, an eighth CBS unit, a fifth Fusion unit, and a ninth C2f unit connected in sequence. The output of the ninth CBS unit is connected to the input of the third Fusion unit, the output of the tenth CBS unit is connected to the inputs of the second Fusion unit and the third Fusion unit respectively, the output of the eleventh CBS unit is connected to the inputs of the first Fusion unit and the fourth Fusion unit respectively, and the output of the sixth CBS unit is connected to the input of the fifth Fusion unit. The Head module includes a first Head unit, a second Head unit and a third Head unit. The first Head unit is connected to the output of the first CAM unit, the second Head unit is connected to the output of the eighth C2f unit, and the third Head unit is connected to the output of the ninth C2f unit.

2. The method according to claim 1, characterized in that: The step 1 of constructing a photovoltaic cell defect detection dataset includes: 1-1) Perform data cleaning operations on the PVEL-AD dataset, remove images with incomplete labels and a small number of category samples, and retain five common defect images: cracks, star-shaped cracks, short circuits, broken grids, and black cores; 1-2) Design and implement a random sampling strategy to select 6936 images from the cleaned data set to construct a data set, including 4855 positive samples and 1388 negative samples, which are divided into training set, validation set and test set in a ratio of 7:2:

1.

3. The method according to claim 1, characterized in that The construction of a multi-scale photovoltaic cell defect model based on a complex background described in step 2 includes: 2-1) Configure the environment: Use Anaconda to create a virtual environment for the source code of the YOLOv8 model; 2-2) Data input: Input the constructed photovoltaic cell defect dataset into the YOLOv8 network structure model, including obtaining photovoltaic cell defect images, photovoltaic cell defect image labels, YOLOv8 network structure model pre-training weights, and YOLOv8 network structure model configuration file; 2-3) Configure hyperparameters: Set the optimal hyperparameter combination of learning rate, batch size, number of iterations, number of image channels, image cropping size, and learning rate momentum. Specifically, use the Adam optimizer, set the batch size to 32, the initial learning rate to 0.01, the maximum number of training times to 100, and use the exponential decay method to dynamically adjust the learning rate to speed up the network convergence; 2-4) Improve the YOLOv8 network structure model, including: 2-4-1) Introduce the bidirectional feature pyramid network (BiFPN) as a feature fusion mechanism; 2-4-2) Add a context aggregation module (CAM) to the Neck network. In step 2-4-1), the bidirectional feature pyramid network (BiFPN) fusion method is adopted to introduce a jump connection between the input and output features of the same layer, which has more efficient feature fusion and stronger feature representation capabilities. The two fused features are calculated by the following formula: Among them, P i in represents the i-th input feature layer, P i td represents the i-th transition feature layer, P i out represents the i-th output feature layer; w1, w2 and w3 are automatically learned weights, corresponding to the weights of the input feature layer, transition feature layer and the output of the previous output feature layer respectively; ò is a very small constant introduced to avoid the denominator being zero, and Conv represents the convolution operation on the calculation result. In formula (1), the current input feature layer and the next higher resolution input feature layer are fused by weighted summation, and the transition feature layer is obtained by convolution. Formula (2) further fuses the current input feature layer, transition feature layer and the previous output feature layer to finally obtain the output feature layer. In step 2-4-2), the introduction of the CAM module enables the improved model to effectively capture and integrate features of different scales from the input data, improve the model's ability to understand and parse complex scenes, and obtain more global contextual information while retaining local details. When performing defect detection on photovoltaic cells, it can better handle the scale changes and occlusion of the target, and provide richer semantic information through feature recalibration, reducing false detections and missed detections. The overall network structure of the improved model is as follows: First convolutional layer: convolution kernel size is 3x3, number of filters is 64, stride=2; Second convolutional layer: convolution kernel size is 3x3, number of filters is 128, stride=2; The C2f module is repeated three times: the number of filters is 128; The third convolutional layer: the convolution kernel size is 3x3, the number of filters is 256, and the stride is 2; The C2f module is repeated six times: the number of filters is 256; Fourth convolutional layer: convolution kernel size is 3x3, number of filters is 512, stride=2; The C2f module is repeated six times: number of filters 512; Fifth convolutional layer: convolution kernel size is 3x3, number of filters is 1024, stride=2; The C2f module is repeated three times: the number of filters is 1024; SPPF module: number of filters / parameters is 1024, 5; Upsampling layer: method nearest, scaling factor 2; belongs to the first layer of the head part; BiFPN_Concat2 module: input comes from the current layer and one of the previous six layers; concatenates the backbone P4 feature map; The C2f module is repeated three times: the number of filters is 512 Upsampling layer again: method nearest, scaling factor 2; belongs to the next layer of the head part; BiFPN_Concat2 module: input comes from the current layer and one of the previous four layers; concatenates the backbone P3 feature map; The C2f module is repeated three times: the number of filters is 256; the corresponding number of layers is immediately after the second BiFPN_Concat2 module; CAM module: number of filters: 256; Convolution layer: convolution kernel size 3x3, number of filters 256, stride 1; belongs to feature map downsampling preparation; BiFPN_Concat3 module: input comes from the current layer, the previous twelve layers, and one of the previous six layers; concatenates the P4 feature map of the head; The C2f module is repeated three times: the number of filters is 512; Convolution layer: convolution kernel size 3x3, number of filters 512, stride 1; belongs to feature map downsampling preparation; BiFPN_Concat2 module: input comes from the current layer and one of the previous nine layers; concatenates the P5 feature map of the head; The C2f module is repeated three times: the number of filters is 1024; Detect module: inputs the concatenated feature maps from the 15th, 18th, and 21st layers; performs target detection and outputs the detection results.

4. The method according to claim 1, characterized in that: Step 3 describes inputting the constructed data set into the improved YOLOv8 model for training, including: 3-1) Input the training set data into the model and adjust the model weights through the back propagation and gradient descent optimization algorithm; 3-2) Use the validation set to evaluate model performance and adjust hyperparameters to avoid overfitting or underfitting; 3-3) Optimize the loss function to Shape-IoU; 3-4) Adjust the weight factor (scale) to different values ​​to optimize the model. In step 3-3), Shape-IoU not only inherits CIoU's consideration of overlapping area and center point distance, but also emphasizes the shape and scale information of the bounding box, so that the model can more accurately capture the characteristics of the target during training, especially for some targets with complex shapes or large size changes. In addition, the improved model is more accurate in measuring the matching degree between the predicted box and the true box, reducing excessive reliance on specific shapes (such as sharp vertical lines), thereby improving the accuracy and stability of photovoltaic cell defect detection. The bounding box loss of Shape-IoU is calculated by the following formula: Among them, IoU is an indicator to measure the degree of overlap between the predicted bounding box and the true bounding box, and distance shape is the difference between the predicted value and the true value, 0.5×Ω shape It is to adjust the weight in the loss function.

Images