A lightweight method for detecting defects in EL images of photovoltaic cells

Abstract

The present invention discloses a lightweight photovoltaic cell EL image defect detection method. First, the EL images of solar cells are collected to form a classification data set with only one type of defect and a detection data set containing multiple types of defects. Secondly, a dual residual mask snowflake Gaussian backbone network DRMSG-Net is constructed and trained using a classification head in combination with the classification data set. Then, a classification pruning head is constructed to replace the classification head, and based on the weights of the trained DRMSG-Net, the classification pruning head is trained using the classification data set to obtain the importance scores of each feature channel and prune the feature channels. Finally, the classification pruning head is removed, and a feature fusion network and a classification and regression network are constructed to obtain the defect type and the target box, and they are trained through the detection data set to obtain the defect detection result. The present invention can learn more detailed and complex representations for accurate photovoltaic cell image defect detection.

Description

Technical Field

[0001] The present invention belongs to the field of defect detection based on deep learning, and particularly relates to a lightweight photovoltaic cell EL image defect target detection method based on a selective state space model. Background Technique

[0002] As a basic component in the field of photovoltaic power generation, the performance of photovoltaic cells directly affects the power generation efficiency of photovoltaic modules. Defects in the cells may cause current loss and efficiency degradation, and may lead to the failure of the entire module after long-term use.

[0003] Electro-luminescence (EL) is a phenomenon based on the emission of light when current passes through a material. In the field of photovoltaic power generation, electro-luminescence technology can be used to detect minute defects in solar cells, especially latent defects such as cracks, broken grids, and black cores. Compared with traditional visible light detection methods, electro-luminescence technology has higher sensitivity and resolution, and can accurately identify internal defects in the cells without damaging the cells. By detecting and repairing these defects in a timely manner, the overall performance and service life of photovoltaic cells can be greatly improved. Therefore, efficient defect detection technology is crucial for the stable operation of photovoltaic modules. Currently, the commonly used methods for photovoltaic cell EL defect target detection in industry are one-stage target detection methods, usually including You Only Look Once (YOLO) and Single Shot MultiBox Detector (SSD). The one-stage model directly performs classification and regression tasks, and outputs the probability value and position coordinates of the target.

[0004] However, the above-mentioned general methods do not consider the optimization of specific tasks. In the defect detection of photovoltaic cells, photovoltaic cells contain complex backgrounds and small target features, and existing general networks and convolutional methods cannot well match the features of defects on photovoltaic cells. In addition, how to lightweight the backbone network in the defect target detection network is also a difficult problem in this direction. Summary of the Invention

[0005] Aiming at the problems existing in the prior art, the present invention proposes a lightweight photovoltaic cell EL image defect detection method, which includes the following steps:

[0006] S1. Collect the EL images of solar cells, intercept and label the defective parts to form two datasets. One dataset contains pure defective images with only one type of defect and the annotation labels of this type of defect. In the present invention, it is called the photovoltaic defect classification dataset (PV-CLS); the other dataset contains actual cell defect images with multiple types of defects, and each type of defect contains a true anchor box (Bbox) and the defect type label. In the present invention, it is called the detection dataset (PV-OD). Preprocess and augment the data of the two datasets, and divide them into training set, validation set and test set;

[0007] S2. Construct a dual-residual masked snowflake Gaussian backbone network (DRMSG-Net), use a classification head composed of global average pooling (GAP) and multi-layer perceptron (MLP), and train it in combination with the PV-CLS dataset.

[0008] S3. Construct a classification pruning head, replace the classification head in the trained DRMSG-Net, use the weights of the DRMSG-Net trained in step S2, and train the classification pruning head with the PV-CLS dataset.

[0009] S4. Obtain the importance scores of each feature channel after training the classification pruning head, set a threshold, perform pruning operations on the feature channels below the threshold to reduce the model parameters, and obtain the pruned DRMSG-Net;

[0010] S5. Based on the pruned DRMSG-Net, remove the classification pruning head, construct a feature fusion network and a classification and regression network to obtain the defect type and the target box. Use the pruned DRMSG-Net in step S4, and train the feature fusion network and the classification and regression network through the PV-OD dataset to obtain the final dual-residual masked snowflake Gaussian backbone network object detection network (DRMSG-OD);

[0011] S6. Use the trained DRMSG-OD model to predict the EL images of photovoltaic cells to obtain the prediction results.

[0012] Further, the dual-residual masked snowflake Gaussian backbone network DRMSG-Net in step S2 includes three sequentially connected stages. The first stage is sequentially composed of an image patch embedding layer, a downsampling layer and a dual-residual structure layer. The second stage and the third stage have the same structure, including a sequentially connected downsampling layer and a dual-residual structure layer;

[0013] S2.1. The image patch embedding layer divides the input image into image patches proportionally, and combines the image patches with equal width and length into one channel of the feature map without adding padding. Repeat the above operations in sequence from left to right and from top to bottom to obtain the input feature map.

[0014] S2.2. The downsampling layer downsamples the input feature map using global average pooling (GAP).

[0015] S2.3. The double residual structure layer processes the downsampled feature map. The double residual structure layer includes a residual connection, a masked snowflake Gaussian module (MSGB), a multi-layer perceptron (MLP), and batch normalization (BN) connected in sequence to obtain a reconstructed feature map with the same size as the input feature map.

[0016] Furthermore, the image patch embedding layer can be specifically embodied as: dividing the input image into small image patches according to a certain ratio, and combining a certain number of image patches with equal width and length into one channel of the feature map without adding padding. Repeat the above operations in sequence from left to right and from top to bottom to obtain the input feature map.

[0017] Furthermore, the MSGB in step S23 includes the following steps:

[0018] S231. Combine the small image patches in the input feature map into one-dimensional vectors in six directions in sequence to obtain a total of six one-dimensional vectors.

[0019] S232. Perform masked snowflake Gaussian convolution (MSGConv) on the image patches in the six obtained one-dimensional vectors respectively, and perform element-wise addition of the feature map vectors output by the convolution to the input one-dimensional vectors to obtain enhanced feature map vectors.

[0020] S233. Process the enhanced feature map vectors through a state space model (SSSM) to efficiently process long sequence inputs and extract effective features to obtain a state space output sequence.

[0021] S234. Output a reconstructed feature map with the same size as the input feature map according to the reverse operation of step S231 for the state space output sequence.

[0022] Furthermore, the MSGConv in step S232 is specifically implemented as multiplying the convolution kernel parameters by the two-dimensional Gaussian distribution weights, and then performing a masking operation on some of the convolution kernel parameters to finally form a convolution operation with a snowflake-shaped unmasked part, which can better extract features of star-shaped crack defects with a gray distribution similar to the Gaussian distribution.

[0023] Further, the classification pruning head described in step S3 is composed of a GAP operation, an MLP network, and a residual structure that directly connects the input feature vector to the output part. The residual structure performs a linear mapping on the input vector, maintains the same input domain output size, and corresponds one by one. The loss function of the residual structure uses the cross-entropy function and the norm to impose penalties on this residual layer, screen out effective features, and the size of the penalty coefficient can be set to define its importance. The specific process can be expressed as: establishing the relationship between the first hidden layer of the MLP network and the residual layer, and restricting the maximum absolute value of the weights of any input vector to the first hidden layer to be no greater than the weights of the corresponding residual layer, so as to screen the importance of the input features.

[0024] Further, the feature channel pruning operation described in step S4 includes the following steps:

[0025] S41, obtaining the weights of the residual layer in the trained classification pruning head as the importance score of this feature channel;

[0026] S42, normalizing the importance score;

[0027] S43, setting a threshold K, and pruning the feature channels corresponding to the normalized importance score less than the threshold K.

[0028] Compared with the prior art, the present invention has the following characteristics and beneficial effects:

[0029] (1) The present invention proposes a double-residual mask snowflake Gaussian backbone network, which uses a double-residual structure, can separately process the spatial information and channel information of the feature map, allows richer information flow and interaction, and enables the network to learn more detailed and complex representations.

[0030] (2) Through various order permutations, it is possible to fuse image features from different perspectives, which helps to capture context information at different positions and directions in the image. Multi-directional feature extraction can help the model process various textures, object shapes, and variations in different directions, improving the performance of the model in complex environments. In addition, multi-directional permutation can reduce the impact of translational transformation, making the model more translation-invariant and able to recognize the same object at different positions and scales.

[0031] (3) Through a unique design of the convolution kernel parameters, MSGConv can better match small target defects that conform to a two-dimensional Gaussian distribution. By using large convolution kernels and masking the weights of some convolution kernels, without significantly increasing the number of model parameters, the receptive field of the model is greatly increased.

[0032] (4) By introducing a selective state space model, it integrates the dual capabilities of global information processing and sequential order processing. Compared with traditional methods, the selective state space model can overcome the problem that ordinary sequential visual models (such as DETR) have weak capabilities in processing long image sequences.

[0033] (5) Through pruning strategies and batch training, by sparsifying the weights, it can reduce the number of parameters in the backbone network, thereby reducing the computational amount and memory occupancy of the entire network. Brief Description of the Drawings

[0034] Figure 1 is the overall design flow chart of the present invention;

[0035] Figure 2 is the DRMSG-Net structure diagram of the present invention;

[0036] Figure 3 is the double residual structure diagram of the present invention;

[0037] Figure 4 is the schematic diagram of the multi-directional feature map sequence combination of the present invention;

[0038] Figure 5 is the convolution kernel structure diagram of the masked snowflake Gaussian convolution of the present invention;

[0039] Figure 6 is the classification pruning head structure diagram of the present invention;

[0040] Figure 7 is the DRMSG-OD structure diagram of the present invention;

[0041] Figure 8 is the comparison diagram of the detection effects between the present invention and the YOLOv8 algorithm. Detailed Embodiments

[0042] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. To make the purpose, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings. On the contrary, the present invention covers any alternatives, modifications, equivalent methods and solutions made within the essence and scope of the present invention defined by the claims. Further, in order to enable the public to have a better understanding of the present invention, in the following detailed description of the present invention, some specific details are described in detail. Those skilled in the art can fully understand the present invention without the description of these details.

[0043] Embodiment 1:

[0044] As Figure 1 shown, a lightweight photovoltaic cell EL image defect target detection method based on a selective state space model includes the following steps:

[0045] Step S1: Collect the EL images of solar cells, intercept and label the defective parts to form two datasets. One dataset contains pure defective images with only one type of defect in the image and the annotation labels of this type of defect. In the present invention, it is called the photovoltaic defect classification dataset (PV-CLS); the other dataset contains actual cell defect images with multiple defects in the image, and each defect contains a true anchor box (Bbox) and a defect type label. In the present invention, it is called the detection dataset (PV-OD). Preprocess and augment the data of the two datasets, and divide them into training set, validation set and test set. Among them, the resolution of the collected EL images of photovoltaic cells is 1024×1024, including 6 types of defects, namely linear crack (LC), finger defect (FD), black core (BC), false grid (TL), star crack (SC) and fragment (Fr). The resolution of PV-CLS images is 160×160, including 6 defect labels. The resolution of PV-OD images is 640×640, including 6 defect labels and corresponding true anchor boxes.

[0046] Step S2: The background of photovoltaic cells is complex and there are multiple small defect targets. Existing backbone networks cannot solve the above problems well. The double residual mask snowflake Gaussian backbone network (DRMSG-Net) proposed in the present invention uses a double residual structure, which allows richer information flow and interaction, enabling the network to learn more detailed and complex representations. As Figures 2 to 5 shown, construct the double residual mask snowflake Gaussian backbone network (DRMSG-Net), which includes an image patch embedding layer, a double residual structure layer, a downsampling layer and a three-stage backbone network body.

[0047] The image patch embedding layer can be specifically embodied as: dividing the input image into small patch images according to a certain ratio, and without adding padding, combining a certain number of image patches with the same width and length into one channel of the feature map. Repeat the above operations in sequence from left to right and from top to bottom to obtain the input feature map.

[0048] The double residual structure layer, as Figure 3 shown, mainly includes a residual connection, a mask snowflake Gaussian module (MSGB), a multi-layer perceptron (MLP) and batch normalization (BN).

[0049] The mask snowflake Gaussian module includes: combining the small patch images in the input feature map into one-dimensional vectors in six directions in sequence, as Figure 4As shown, these six directions are as follows: (1) Starting from the upper left corner, arranged row by row from left to right in sequence; (2) Starting from the lower right corner, arranged row by row from right to left in sequence; (3) Starting from the upper left corner, arranged column by column from top to bottom in sequence; (4) Starting from the lower right corner, arranged column by column from bottom to top in sequence; (5) Starting from the upper left corner, arranged clockwise from the outside to the inside in sequence; (6) Starting from the lower right corner, arranged counterclockwise from the outside to the inside in sequence. Then, perform masked snowflake Gaussian convolution (MSGConv) on the image patches in the feature map vector, and perform element-wise addition of the feature map vector output by the convolution and the input feature map vector to obtain a feature map vector with enhanced small targets. Among them, MSGConv is a convolution operation with a convolution kernel size of 7×7, where the convolution kernel parameters are multiplied by the weights of the two-dimensional Gaussian distribution, and then a masking operation is performed on some of the convolution kernel parameters, finally forming a convolution operation with a snowflake-shaped unmasked part. This operation can better extract features of star-shaped crack defects with a gray-scale distribution similar to the Gaussian distribution.

[0050] Process the above feature map vector by selecting a state space model (SSSM) to efficiently process long sequence inputs and extract effective features. Among them, the continuous state space can be represented by three consecutive parameter matrices , , .

[0051]

[0052] Among them, is the input feature at time , is the intermediate state at time ,

[0053]

[0054] Among them, and represent the state vectors at time and time respectively, and represent the output feature vectors at time and are the parameter matrices after discretization, is the time scale parameter for the discretization parameters.

[0055] Then, perform the reverse operation of step 2 on the output sequence to reconstruct the feature map as the output feature map, and the output feature map does not change the length, width, and number of channels in step S1.

[0056] The three-stage backbone network body sequentially includes an image patch embedding layer, a first group of downsampling layers and a double residual structure layer, a second group of downsampling layers and a double residual structure layer, and a third group of downsampling layers and a double residual structure layer.

[0057] Finally, a classification head composed of global average pooling (GAP) and a multi-layer perceptron is used for training in combination with the PV-CLS dataset.

[0058] The gray images of some small defect targets conform to the characteristics of two-dimensional Gaussian distribution. The mask snowflake Gaussian convolution (MSGConv) proposed by the present invention can better match the defect targets. By increasing the convolution kernel and mask operation, the receptive field of the model is greatly increased without significantly increasing the number of model parameters.

[0059] Step S3: Since there are redundant feature channels in the backbone network during training, and these redundant feature channels contribute less to the results during inference, pruning these redundant feature channels can reduce the number of network parameters and computational amount without basically losing network performance. By introducing a residual layer and L1 regularization in the detection head, the weights of some neurons can be made zero, thus effectively performing feature selection. This enables the model to automatically reduce irrelevant or redundant features during the training process, improving the efficiency and interpretability of the model. As Figure 6 shown, a classification pruning head is constructed, which consists of a GAP operation, an MLP network, and a residual structure directly connecting the input feature vector to the output part. The dimension of the residual structure is the same as the dimension of the input vector and corresponds one by one. Its loss function uses the cross-entropy function and the norm to impose a penalty on this residual layer, screening effective features, and the magnitude of the penalty coefficient can be set to define its importance. The specific process can be expressed as: Let the dimension of the input feature vector be , and the number of neurons in the first hidden layer of the MLP network be , the minimized structural risk of the residual path and the neurons in the first hidden layer of the MLP network can be defined as:

[0060]

[0061] Among them, represents the weight of the residual path at the position, represents the weight from the position of the MLP input layer to the first hidden layer of the MLP network, represents the norm penalty term coefficient, and the following restrictions are imposed on it:

[0062]

[0063] Among them, represents the non-linear control coefficient. By controlling the parameter , the relationship between the first hidden layer and the residual layer of the MLP network is established, and the maximum absolute value of the weights of any input vector to the first hidden layer is restricted not to be greater than the weights of the corresponding residual layer, so as to screen the importance of the input features.

[0064] Replace the classification head in DRMSG-Net with a classification pruning head, use the weights of the trained DRMSG-Net in step S2, and use the PV-CLS dataset to train the classification pruning head.

[0065] Step S4: Obtain the importance scores of each feature channel after training, set a threshold, perform pruning operations on the feature channels below the threshold, reduce the model parameters, and obtain the pruned DRMSG-Net; the feature channel pruning operation can be specifically described as: (1) Obtain the weights of the residual layer in the trained classification pruning head; (2) Scale it to between [0,1] through normalization; (3) Set a threshold K∈[0,1], extract the part of the residual layer weights that is less than K, obtain its position information, and the feature channels at the same position of the corresponding input feature vector will be pruned to achieve the purpose of reducing the model parameter quantity and calculation amount.

[0066] Step S5: As Figure 7 shown, based on the pruned DRMSG-Net, remove the classification pruning head, and construct a feature fusion network and a classification and regression network. Use the pruned DRMSG-Net in step S4, and through the PV-OD dataset, train the feature fusion network and the classification and regression network to obtain the final dual-residual mask snowflake Gaussian backbone network object detection network (DRMSG-OD). The feature fusion network consists of upsampling, Concat splicing, and convolution (Conv) with a convolution kernel size of 3×3; the classification and regression network consists of convolution with a convolution kernel size of 3×3 and convolution with a convolution kernel size of 1×1. The classification loss function selects the binary cross-entropy loss function (BCE), and the regression loss function selects CIOU. Among them, the formula for BCE is:

[0067]

[0068] Among them, is the binary label 0 or 1, is the probability of the output label, is the number of groups of prediction objects.

[0069] The formula for CIOU is:

[0070]

[0071] Among them, is the Intersection over Union (IoU), which represents the ratio of the overlapping area between the predicted bounding box and the ground truth bounding box to the union area. is the square of the Euclidean distance between the centers of the predicted bounding box and the ground truth bounding box. are the width of the predicted bounding box, the height of the predicted bounding box, the width of the ground truth bounding box, and the height of the ground truth bounding box, respectively. is the diagonal distance of the minimum bounding rectangle. is a coefficient used to adjust the influence of the aspect ratio error. is the difference in aspect ratio, which represents the difference in aspect ratio between the predicted bounding box and the ground truth bounding box, and measures the deviation of the shape of the target bounding box.

[0072] Step S6: Use the trained DRMSG-OD model to predict the EL image of the photovoltaic cell to obtain the prediction result, as Figure 8 shown is the detection effect of the YOLOv8 model and the detection effect of the model of the present invention. It can be seen that the model of the present invention detects more defects.

[0073] Table 1

[0074]

[0075] mAP is the average of the average precision (AP) over all classes, and it is used to measure the overall performance of the model at different classes and different IoU thresholds. mAP_50 refers to the average precision calculated at an IoU threshold of 0.5. mAP@50:95 refers to the mAP value calculated at IoU thresholds ranging from 0.5 to 0.95 with a step size of 0.05. As can be seen from Table 1, compared with the Yolov8s model, the present invention not only reduces the number of parameters and computational complexity, but also improves the detection accuracy. Among them, mAP_50 is improved by 4.3 percentage points, and mAP_50:95 is improved by 3.5 percentage points. The number of parameters is reduced by 1.8M, and the computational amount is reduced by 7.5 GFLOPs.

Claims

1. A lightweight photovoltaic cell EL image defect detection method, characterized in that, It includes the following steps: S1. Collect the EL images of solar cells, and form a photovoltaic defect classification dataset PV-CLS with only one type of defect and a detection dataset PV-OD with multiple types of defects; S2. Construct a dual-residual masked snowflake Gaussian backbone network DRMSG-Net, use a classification head composed of global average pooling GAP and a multi-layer perceptron MLP, and train it in combination with the PV-CLS dataset; The dual-residual masked snowflake Gaussian backbone network DRMSG-Net includes three sequentially connected stages. The first stage is composed of an image patch embedding layer, a downsampling layer, and a dual-residual structure layer connected in sequence. The second stage has the same structure as the third stage, including a downsampling layer and a dual-residual structure layer connected in sequence; The dual-residual structure layer processes the downsampled feature map. The dual-residual structure layer includes a residual connection, a masked snowflake Gaussian module MSGB, a multi-layer perceptron MLP, and batch normalization BN connected in sequence to obtain a reconstructed feature map with the same size as the input feature map; The specific implementation process of the masked snowflake Gaussian module MSGB is as follows: S221. Combine the image patches in the input feature map into one-dimensional vectors in six directions in sequence, and a total of six one-dimensional vectors are obtained; S222. Perform masked snowflake Gaussian convolution MSGConv on the image patches in the six obtained one-dimensional vectors respectively, and add the feature map vectors output by the convolution element by element to the input one-dimensional vectors to obtain enhanced feature map vectors; S223. Process the enhanced feature map vectors through a selected state space model SSSM to obtain a state space output sequence; S224. Perform the reverse operation of step S221 on the state space output sequence, and output a reconstructed feature map with the same size as the input feature map; The masked snowflake Gaussian convolution MSGConv is implemented as: multiplying the convolution kernel parameters by the two-dimensional Gaussian distribution weights, and then performing a masking operation on the convolution kernel parameters, and finally forming a convolution operation with a snowflake-shaped unmasked part; S3. Construct a classification pruning head, replace the classification head in the trained DRMSG-Net, and train the classification pruning head based on the trained DRMSG-Net weights using the PV-CLS dataset; The classification pruning head is composed of a GAP operation, an MLP network, and a residual structure that directly connects the input feature vector to the output part. The residual structure performs a linear mapping on the input vector, keeps the input and output sizes the same, and corresponds one by one; S4. Obtain the importance scores of each feature channel after training the classification pruning head, and prune the feature channels; S5. Remove the classification pruning head after pruning, construct a feature fusion network and a classification and regression network, obtain the defect type and the target box, and train the feature fusion network and the classification and regression network through the PV-OD dataset to obtain the defect detection result.

2. The lightweight photovoltaic cell EL image defect detection method according to claim 1, wherein, The specific implementation of the step S1 is as follows: Collect the EL images of solar cells, intercept and label the defective parts to form two data sets. One data set contains pure defective images, in which there is only one type of defect and the annotation labels of this type of defect are included, and it is called the photovoltaic defect classification data set PV-CLS; the other data set contains actual cell defect images, in which there are multiple types of defects, and each defect contains a true anchor box Bbox and a defect type label, and it is called the detection data set PV-OD; preprocess and augment the two data sets.

3. The lightweight photovoltaic cell EL image defect detection method according to claim 2, wherein The image patch embedding layer divides the input image into image patches in proportion, and combines the image patches with equal width and length into one channel of the feature map without adding padding; repeat the above operations in sequence from left to right and from top to bottom to obtain the input feature map; The downsampling layer downsamples the input feature map using global average pooling GAP.

4. The lightweight photovoltaic cell EL image defect detection method according to claim 3, characterized in that The loss function of the residual structure uses the cross-entropy function and the L1 norm to impose penalties on the residual layer. The specific implementation of the penalty is as follows: Establish the relationship between the first hidden layer of the MLP network and the residual layer, and limit the maximum absolute value of the weights of any input vector to the first hidden layer to be not greater than the weights of the corresponding residual layer, so as to screen the importance of the input features.

5. The lightweight photovoltaic cell EL image defect detection method according to claim 4, wherein, The specific implementation process of the step S4 is as follows: S41, Obtain the weights of the residual layer in the trained classification pruning head as the importance score of the corresponding feature channel; S42, Normalize the importance score; S43, Set a threshold K, and prune the feature channels corresponding to the normalized importance score less than the threshold K.

Images