Light-weight photovoltaic cell EL image defect detection method

Abstract

The invention discloses a lightweight photovoltaic cell EL image defect detection method, which comprises the following steps: firstly, collecting EL images of solar cells, and forming a classification data set only having one defect and a detection data set containing a plurality of defects; secondly, constructing a double-residual mask snowflake Gaussian backbone network DRMSG-Net, and training by using a classification head and combining a classification data set; and then constructing a classification pruning head to replace the classification head, training the classification pruning head by using the classification data set based on the trained DRMSG-Net weight, and obtaining the importance score of each feature channel to prune the feature channel. And finally, removing a classification pruning head, constructing a feature fusion network and a classification and regression network, obtaining a defect type and a target frame, and training through a detection data set to obtain a defect detection result. According to the invention, more detailed and complex representation can be learned, and accurate photovoltaic cell image defect detection can be carried out.

Description

Technical Field

[0001] The present invention belongs to the field of defect detection based on deep learning, and particularly relates to a lightweight photovoltaic cell EL image defect target detection method based on a selective state space model. Background Art

[0002] As a basic component in the field of photovoltaic power generation, the performance of photovoltaic cells directly affects the power generation efficiency of photovoltaic modules. Defects in the cells may lead to current loss and efficiency decline, and may cause the failure of the entire module after long-term use.

[0003] Electro-luminescence (EL) is a phenomenon based on the emission of light when current passes through a material. In the field of photovoltaic power generation, electro-luminescence technology can be used to detect minute defects in solar cells, especially hidden defects such as cracks, broken grids, and black cores. Compared with traditional visible light detection methods, electro-luminescence technology has higher sensitivity and resolution, and can accurately identify internal defects in the cells without damaging the cells. By detecting and repairing these defects in a timely manner, the overall performance and service life of photovoltaic cells can be greatly improved. Therefore, efficient defect detection technology is crucial for the stable operation of photovoltaic modules. Currently, the commonly used method for photovoltaic cell EL defect target detection in industry is the one-stage target detection method, which usually includes the You Only Look Once (YOLO) algorithm and the Single Shot MultiBox Detector (SSD) algorithm. The one-stage model directly performs classification and regression tasks, and outputs the probability value and position coordinates of the target.

[0004] However, the above-mentioned general methods do not consider the optimization of specific tasks. In the defect detection of photovoltaic cells, photovoltaic cells contain complex backgrounds and small target features, and existing general networks and convolutional methods cannot well match the features of defects on photovoltaic cells. In addition, how to lightweight the backbone network in the defect target detection network is also a difficult problem in this direction. Summary of the Invention

[0005] Aiming at the problems existing in the prior art, the present invention proposes a lightweight photovoltaic cell EL image defect detection method, which includes the following steps:

[0006] S1. Collect the EL images of solar cells, intercept and label the defective parts to form two datasets. One dataset contains pure defective images with only one type of defect and the annotation labels of this type of defect, which is called the photovoltaic defect classification dataset (PV-CLS) in the present invention. The other dataset contains actual cell defective images with multiple types of defects, and each type of defect contains a true anchor box (Bbox) and a defect type label, which is called the detection dataset (PV-OD) in the present invention. Preprocess and augment the two datasets, and divide them into training set, validation set and test set;

[0007] S2. Construct a dual-residual mask snowflake Gaussian backbone network (DRMSG-Net), use a classification head composed of global average pooling (GAP) and multi-layer perceptron (MLP), and train it in combination with the PV-CLS dataset.

[0008] S3. Construct a classification pruning head, replace the classification head in the trained DRMSG-Net, use the weights of the DRMSG-Net trained in step S2, and train the classification pruning head with the PV-CLS dataset.

[0009] S4. Obtain the importance scores of each feature channel after training the classification pruning head, set a threshold, perform pruning operations on the feature channels below the threshold to reduce the model parameters, and obtain the pruned DRMSG-Net;

[0010] S5. On the basis of the pruned DRMSG-Net, remove the classification pruning head, construct a feature fusion network and a classification and regression network to obtain the defect type and the target box. Use the pruned DRMSG-Net in step S4, and train the feature fusion network and the classification and regression network through the PV-OD dataset to obtain the final dual-residual mask snowflake Gaussian backbone network object detection network (DRMSG-OD);

[0011] S6. Use the trained DRMSG-OD model to predict the EL images of photovoltaic cells to obtain the prediction results.

[0012] Further, the dual-residual mask snowflake Gaussian backbone network DRMSG-Net in step S2 includes three sequentially connected stages. The first stage is composed of an image patch embedding layer, a downsampling layer and a dual-residual structure layer connected in sequence. The second stage and the third stage have the same structure, including a downsampling layer and a dual-residual structure layer connected in sequence;

[0013] S2.1. The image patch embedding layer divides the input image into image patches proportionally, and without adding padding, combines the image patches with equal width and length into one channel of the feature map. Repeat the above operations in sequence from left to right and from top to bottom to obtain the input feature map.

[0014] S2.2. The downsampling layer downsamples the input feature map using global average pooling (GAP).

[0015] S2.3. The double residual structure layer processes the downsampled feature map. The double residual structure layer includes a residual connection, a masked snowflake Gaussian module (MSGB), a multi-layer perceptron (MLP), and batch normalization (BN) connected in sequence to obtain a reconstructed feature map with the same size as the input feature map.

[0016] Further, the image patch embedding layer can be specifically embodied as: dividing the input image into small image patches according to a certain ratio, and without adding padding, combining a certain number of image patches with equal width and length into one channel of the feature map. Repeat the above operations in sequence from left to right and from top to bottom to obtain the input feature map.

[0017] Further, the MSGB in step S23 includes the following steps:

[0018] S231. Combine the small image patches in the input feature map into one-dimensional vectors in six directions in sequence to obtain a total of six one-dimensional vectors.

[0019] S232. Perform masked snowflake Gaussian convolution (MSGConv) on the image patches in the six obtained one-dimensional vectors respectively, and add the feature map vectors output by the convolution element-wise to the input one-dimensional vectors to obtain enhanced feature map vectors.

[0020] S233. Process the enhanced feature map vectors through a state space model (SSSM) to efficiently process long sequence inputs and extract effective features to obtain a state space output sequence.

[0021] S234. Output a reconstructed feature map with the same size as the input feature map according to the reverse operation of step S231 for the state space output sequence.

[0022] Further, the MSGConv in step S232 is specifically implemented as multiplying the convolution kernel parameters by the two-dimensional Gaussian distribution weights, and then performing a masking operation on some of the convolution kernel parameters to finally form a convolution operation with a snowflake-shaped unmasked part. This operation can better extract features of star-shaped crack defects with a gray distribution similar to the Gaussian distribution.

[0023] Further, the classification pruning head described in step S3 is composed of a GAP operation, an MLP network, and a residual structure that directly connects the input feature vector to the output part. The residual structure performs a linear mapping on the input vector, keeps the input domain and output size the same, and corresponds one by one. The loss function of the residual structure uses the cross-entropy function and the norm to impose penalties on this residual layer, screen out effective features, and the size of the penalty coefficient can be set to define its importance. The specific process can be expressed as: establishing the relationship between the first hidden layer of the MLP network and the residual layer, and restricting the maximum absolute value of the weights of any input vector to the first hidden layer not to be greater than the weights of the corresponding residual layer, so as to screen the importance of the input features.

[0024] Further, the feature channel pruning operation described in step S4 includes the following steps:

[0025] S41, obtaining the weights of the residual layer in the trained classification pruning head as the importance score of this feature channel;

[0026] S42, normalizing the importance score;

[0027] S43, setting a threshold K, and pruning the feature channels corresponding to the normalized importance score less than the threshold K.

[0028] Compared with the prior art, the present invention has the following characteristics and beneficial effects:

[0029] (1) The present invention proposes a dual-residual mask snowflake Gaussian backbone network, which uses a dual-residual structure, can separately process the spatial information and channel information of the feature map, allows richer information flow and interaction, and enables the network to learn more detailed and complex representations.

[0030] (2) Through various order permutations, it is possible to fuse image features from different perspectives, which helps to capture the context information at different positions and directions in the image. Multi-directional feature extraction can help the model process various textures, object shapes and changes in different directions, and improve the performance of the model in complex environments. In addition, multi-directional permutation can reduce the impact of translational transformation, making the model more translation-invariant and able to recognize the same object at different positions and scales.

[0031] (3) Through the unique design of the convolutional kernel parameters, MSGConv can better match small target defects that conform to the two-dimensional Gaussian distribution. By using large convolutional kernels and masking the weights of some convolutional kernels, without significantly increasing the number of model parameters, the receptive field of the model is greatly increased.

[0032] (4) By introducing the selective state space model, it integrates the dual capabilities of global information processing and sequential order processing. Compared with traditional methods, the selective state space model can overcome the problem that ordinary serialized vision models (such as DETR) have weak processing capabilities for long image sequences.

[0033] (5) Through pruning strategies and batch training, by sparsifying the weights, it can reduce the number of parameters in the backbone network, thereby reducing the computational amount and memory occupancy of the entire network. Brief Description of the Drawings

[0034] Figure 1 is the overall design flow chart of the present invention;

[0035] Figure 2 is the DRMSG-Net structure diagram of the present invention;

[0036] Figure 3 is the double residual structure diagram of the present invention;

[0037] Figure 4 is the schematic diagram of the sequence combination of multi-directional feature maps of the present invention;

[0038] Figure 5 is the convolution kernel structure diagram of the masked snowflake Gaussian convolution of the present invention;

[0039] Figure 6 is the classification pruning head structure diagram of the present invention;

[0040] Figure 7 is the DRMSG-OD structure diagram of the present invention;

[0041] Figure 8 is the comparison diagram of the detection effects between the present invention and the YOLOv8 algorithm. Detailed Embodiments

[0042] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. In order to make the purpose, technical solutions and advantages of the present invention clearer, the following further details the present invention with reference to the accompanying drawings. On the contrary, the present invention covers any alternatives, modifications, equivalent methods and solutions made within the essence and scope defined by the claims. Further, in order to enable the public to have a better understanding of the present invention, in the following detailed description of the present invention, some specific details are described in detail. Those skilled in the art can fully understand the present invention without the description of these details.

[0043] Embodiment 1:

[0044] As Figure 1 shown, a lightweight photovoltaic cell EL image defect target detection method based on a selective state space model includes the following steps:

[0045] Step S1: Collect the EL images of solar cells, intercept and label the defective parts to form two datasets. One dataset contains pure defective images with only one type of defect in the image and the annotation labels of this type of defect. In the present invention, it is called the photovoltaic defect classification dataset (PV-CLS); the other dataset contains actual cell defect images with multiple defects in the image, and each defect contains a true anchor box (Bbox) and the defect type label. In the present invention, it is called the detection dataset (PV-OD). Preprocess and augment the data of the two datasets, and divide them into training set, validation set and test set. Among them, the resolution of the collected EL images of photovoltaic cells is 1024×1024, including 6 types of defects, namely linear crack (LC), finger defect (FD), black core (BC), false grid (TL), star crack (SC) and fragment (Fr). The resolution of PV-CLS images is 160×160, including 6 types of defect labels. The resolution of PV-OD images is 640×640, including 6 types of defect labels and the corresponding true anchor boxes.

[0046] Step S2: The background of photovoltaic cells is complex and there are multiple small defect targets. The existing backbone networks cannot solve the above problems well. The dual residual mask snowflake Gaussian backbone network (DRMSG-Net) proposed in the present invention uses a dual residual structure, which can allow richer information flow and interaction, enabling the network to learn more detailed and complex representations. As Figures 2 to 5 shown, construct the dual residual mask snowflake Gaussian backbone network (DRMSG-Net), which includes an image patch embedding layer, a dual residual structure layer, a downsampling layer and a three-stage backbone network body.

[0047] The image patch embedding layer can be specifically embodied as: dividing the input image into small patch images according to a certain ratio, and combining a certain number of image patches with equal width and length into one channel of the feature map without adding padding. Repeat the above operations in sequence from left to right and from top to bottom to obtain the input feature map.

[0048] The dual residual structure layer, as Figure 3 shown, mainly includes a residual connection, a mask snowflake Gaussian module (MSGB), a multi-layer perceptron (MLP) and batch normalization (BN).

[0049] The mask snowflake Gaussian module includes: combining the small patch images in the input feature map into one-dimensional vectors in six directions in sequence, as Figure 4As shown, these six directions are as follows: (1) Starting from the upper left corner, arranged row by row from left to right in sequence; (2) Starting from the lower right corner, arranged row by row from right to left in sequence; (3) Starting from the upper left corner, arranged column by column from top to bottom in sequence; (4) Starting from the lower right corner, arranged column by column from bottom to top in sequence; (5) Starting from the upper left corner, arranged clockwise from the outside to the inside in sequence; (6) Starting from the lower right corner, arranged counterclockwise from the outside to the inside in sequence. Then, perform masked snowflake Gaussian convolution (MSGConv) on the image patches in the feature map vector, and perform element-by-element superposition on the feature map vector output by the convolution and the input feature map vector to obtain a feature map vector with enhanced small targets. Among them, MSGConv is a convolution operation with a convolution kernel size of 7×7, where the convolution kernel parameters are multiplied by the weights of the two-dimensional Gaussian distribution, and then a masking operation is performed on some of the convolution kernel parameters, finally forming a convolution operation with a snowflake-shaped unmasked part. This operation can better extract features of star-shaped crack defects with a gray distribution similar to the Gaussian distribution.

[0050] Process the above feature map vector by selecting a state space model (SSSM) to efficiently process long sequence inputs and extract effective features. Among them, the continuous state space can be represented by three consecutive parameter matrices , , .

[0051]

[0052] Among them, is the input feature at time , is the intermediate state at time , is the output feature at time

[0053]

[0054] Among them, and respectively represent the state vectors at time and time , and represent the output feature vectors at time and are the parameter matrices after discretization, is the time scale parameter for the discretization parameters.

[0055] Then, reverse the operation in step 2 on the output sequence to reconstruct the feature map as the output feature map, and the output feature map does not change the length, width, and number of channels in step S1.

[0056] The three-stage backbone network body sequentially includes an image patch embedding layer, a first group of downsampling layers and double residual structure layers, a second group of downsampling layers and double residual structure layers, and a third group of downsampling layers and double residual structure layers.

[0057] Finally, a classification head composed of global average pooling (GAP) and a multi-layer perceptron is used for training in combination with the PV-CLS dataset.

[0058] The gray images of some small defect targets conform to the characteristics of two-dimensional Gaussian distribution. The mask snowflake Gaussian convolution (MSGConv) proposed in the present invention can better match the defect targets. By increasing the convolution kernel and mask operation, the receptive field of the model is greatly increased without significantly increasing the number of model parameters.

[0059] Step S3: Since there are redundant feature channels in the backbone network during training, and these redundant feature channels contribute less to the results during inference, pruning these redundant feature channels can reduce the number of network parameters and computational volume without basically losing the network performance. By introducing a residual layer and L1 regularization in the detection head, the weights of some neurons can be made zero, thereby effectively performing feature selection. This enables the model to automatically reduce irrelevant or redundant features during the training process, improving the efficiency and interpretability of the model. As Figure 6 shown, a classification pruning head is constructed, which consists of a GAP operation, an MLP network, and a residual structure directly connecting the input feature vector to the output part. The dimension of the residual structure is the same as the dimension of the input vector and corresponds one by one. Its loss function uses the cross-entropy function and the norm to impose penalties on this residual layer, and effective features can be screened. The size of the penalty coefficient can be set to define its importance. The specific process can be expressed as: Let the dimension of the input feature vector be , and the number of neurons in the first hidden layer of the MLP network be , the minimized structural risk of the residual path and the neurons in the first hidden layer of the MLP network can be defined as:

[0060]

[0061] Among them, represents the weight of the residual path at the position, represents the weight from the position of the MLP input layer to the first hidden layer of the MLP network, represents the norm penalty term coefficient, and the following restrictions are imposed on it:

[0062]

[0063] Among them, represents the non - linear control coefficient. By controlling the parameter , the relationship between the first hidden layer and the residual layer of the MLP network is established, and the maximum value of the absolute value of all weights of any input vector to the first hidden layer is restricted not to be greater than the weights of the corresponding residual layer, so as to screen the importance of input features.

[0064] Replace the classification head in DRMSG - Net with a classification pruning head, use the weights of the trained DRMSG - Net in step S2, and use the PV - CLS dataset to train the classification pruning head.

[0065] Step S4: Obtain the importance scores of each feature channel after training, set a threshold, perform pruning operations on the feature channels below the threshold to reduce the model parameters, and obtain the pruned DRMSG - Net; the feature channel pruning operation can be specifically described as: (1) Obtain the weights of the residual layer in the trained classification pruning head; (2) Scale it to between [0, 1] through normalization; (3) Set the threshold K ∈ [0, 1], extract the part of the residual layer weights that is less than K, obtain its position information, and the feature channels at the same position of the corresponding input feature vector will be pruned to achieve the purpose of reducing the model parameter quantity and computational complexity.

[0066] Step S5: As Figure 7 shown, based on the pruned DRMSG - Net, remove the classification pruning head, and construct a feature fusion network and a classification and regression network. Use the pruned DRMSG - Net in step S4, and through the PV - OD dataset, train the feature fusion network and the classification and regression network to obtain the final dual - residual mask snow - flake Gaussian backbone network object detection network (DRMSG - OD). The feature fusion network consists of upsampling, Concat splicing, and a convolution (Conv) with a kernel size of 3×3; the classification and regression network consists of a convolution with a kernel size of 3×3 and a convolution with a kernel size of 1×1. The classification loss function selects the binary cross - entropy loss function (BCE), and the regression loss function selects CIOU. Among them, the formula for BCE is:

[0067]

[0068] Among them, is the binary label 0 or 1, is the probability of the output label, is the number of groups of prediction objects.

[0069] The formula for CIOU is:

[0070]

[0071] Among them, is the Intersection over Union (IoU), which represents the ratio of the overlapping area between the predicted bounding box and the ground truth bounding box to the union area. is the square of the Euclidean distance between the centers of the predicted bounding box and the ground truth bounding box. are the width of the predicted bounding box, the height of the predicted bounding box, the width of the ground truth bounding box, and the height of the ground truth bounding box respectively. is the diagonal distance of the minimum bounding rectangle. is a coefficient used to adjust the influence of the aspect ratio error. is the difference in aspect ratio, which represents the difference in aspect ratio between the predicted bounding box and the ground truth bounding box, and measures the deviation of the target box shape.

[0072] Step S6: Use the trained DRMSG-OD model to predict the EL image of the photovoltaic cell to obtain the prediction result, as Figure 8 shown is the detection effect of the YOLOv8 model and the detection effect of the model of the present invention. It can be seen that the model of the present invention detects more defects.

[0073] Table 1

[0074]

[0075] mAP is the average of the average precision (AP) over all classes, which is used to measure the overall performance of the model at different classes and different IoU thresholds. mAP_50 refers to the average precision calculated at an IoU threshold of 0.5. mAP@50:95 refers to the mAP value calculated at IoU thresholds ranging from 0.5 to 0.95 with a step size of 0.05. As can be seen from Table 1, compared with the Yolov8s model, the present invention not only reduces the number of parameters and computational complexity, but also improves the detection accuracy. Among them, mAP_50 is improved by 4.3 percentage points, and mAP_50:95 is improved by 3.5 percentage points. The number of parameters is reduced by 1.8M, and the computational amount is reduced by 7.5 GFLOPs.

Claims

1. A lightweight photovoltaic cell EL image defect detection method, characterized in that, It includes the following steps: S1. Collect the EL images of solar cells, and form a photovoltaic defect classification dataset PV-CLS with only one type of defect, and a detection dataset PV-OD containing multiple types of defects; S2. Construct a dual residual masked snowflake Gaussian backbone network DRMSG-Net, use a classification head composed of global average pooling GAP and multi-layer perceptron MLP, and train it in combination with the PV-CLS dataset; S3. Construct a classification pruning head, replace the classification head in the trained DRMSG-Net, and train the classification pruning head based on the weights of the trained DRMSG-Net using the PV-CLS dataset; S4. Obtain the importance scores of each feature channel after training the classification pruning head, and prune the feature channels; S5. Remove the classification pruning head after pruning, construct a feature fusion network and a classification and regression network, obtain the defect type and target box, and train the feature fusion network and the classification and regression network through the PV-OD dataset to obtain the defect detection result.

2. The lightweight photovoltaic cell EL image defect detection method according to claim 1, characterized in that, The specific implementation of step S1 is as follows: Collect the EL images of solar cells, intercept and label the defective parts to form two datasets. One dataset contains pure defect images with only one type of defect in the image and the annotation labels of this type of defect, which is called the photovoltaic defect classification dataset PV-CLS; the other dataset contains actual cell defect images with multiple types of defects in the image, and each defect contains a true anchor box Bbox and defect type label, which is called the detection dataset PV-OD; preprocess and augment the two datasets.

3. The lightweight photovoltaic cell EL image defect detection method according to claim 2, characterized in that, The dual residual masked snowflake Gaussian backbone network DRMSG-Net includes three sequentially connected stages. The first stage is composed of an image patch embedding layer, a downsampling layer and a dual residual structure layer connected in sequence. The second stage and the third stage have the same structure, including a downsampling layer and a dual residual structure layer connected in sequence; The image patch embedding layer divides the input image into image patches according to a ratio, and combines the image patches with equal width and length into one channel of the feature map without adding padding; repeat the above operations in sequence from left to right and from top to bottom to obtain the input feature map; The downsampling layer downsamples the input feature map using global average pooling GAP; The dual residual structure layer processes the downsampled feature map. The dual residual structure layer includes a residual connection, a masked snowflake Gaussian module MSGB, a multi-layer perceptron MLP and batch normalization BN connected in sequence to obtain a reconstructed feature map with the same size as the input feature map.

4. The lightweight photovoltaic cell EL image defect detection method according to claim 3, characterized in that, The specific implementation process of the masked snowflake Gaussian module MSGB is as follows: S221. Combine the image patches in the input feature map into one-dimensional vectors in six directions in sequence to obtain a total of six one-dimensional vectors; S222. Perform masked snowflake Gaussian convolution MSGConv on the image patches in the six obtained one-dimensional vectors respectively, and add the feature map vectors output by the convolution element-wise to the input one-dimensional vectors to obtain enhanced feature map vectors; S223. Process the enhanced feature map vector through the selected state space model SSSM to obtain a state space output sequence; S224. Perform the reverse operation of step S221 on the state space output sequence and output a reconstructed feature map with the same size as the input feature map.

5. The lightweight photovoltaic cell EL image defect detection method according to claim 4, characterized in that, The masked snowflake Gaussian convolution MSGConv is implemented as follows: Multiply the convolution kernel parameters by the two-dimensional Gaussian distribution weights, then perform a masking operation on the convolution kernel parameters, and finally form a convolution operation with a snowflake-shaped unmasked part.

6. The lightweight photovoltaic cell EL image defect detection method according to claim 5, characterized in that, The described classification pruning head consists of a GAP operation, an MLP network, and a residual structure that directly connects the input feature vector to the output part. The residual structure performs a linear mapping on the input vector, keeps the input domain and output size the same, and corresponds one by one. The loss function of the residual structure uses the cross-entropy function and the norm imposes a penalty on this residual layer. The specific penalty is as follows: establish the relationship between the first hidden layer of the MLP network and the residual layer, and limit the maximum absolute value of the weights of any input vector to the first hidden layer to be no greater than the weights of the corresponding residual layer, so as to screen the importance of the input features.

7. The lightweight photovoltaic cell EL image defect detection method according to claim 6, characterized in that, The specific implementation process of step S4 is as follows: S41. Obtain the residual layer weights in the trained classification pruning head as the importance scores of the corresponding feature channels; S42. Normalize the importance scores; S43. Set a threshold K, and prune the feature channels corresponding to the normalized importance scores less than the threshold K.

Images