Solar cell defect detection method combining multi-scale feature enhancement and staged metric learning

Abstract

The invention provides a solar cell defect detection method combining multi-scale feature enhancement and staged metric learning, and relates to the technical field of photovoltaic module intelligent detection and defect identification, and the method comprises the following steps: constructing a multi-scale feature enhanced deep residual network embedded with a channel attention module and a multi-scale feature fusion module; performing staged progressive optimization on parameters of the multi-scale feature enhanced deep residual network, and constructing a twin network structure; using the high-dimensional depth feature vector to calculate a dual metric index used for constructing a fusion loss function so as to optimize the twin network structure; and performing defect detection on a to-be-detected sample by using the optimized twin network structure in combination with a pre-constructed standard sample support set. The method has the advantage that under the condition that the number of defect samples is limited or the types are extremely unbalanced, the distinguishing capability of the model on rare defects is remarkably improved through a metric learning mechanism.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection and defect identification technology for photovoltaic modules, and more specifically, to a solar cell defect detection method that combines multi-scale feature enhancement and staged metric learning. Background Technology

[0002] Solar energy, as a clean and renewable energy source, is rapidly expanding its application globally. Solar cells, as the core component of photovoltaic power generation systems, directly determine the power generation efficiency and lifespan of the entire system through their manufacturing quality and operational status. However, various types of defects inevitably occur during the production and long-term operation of solar cells. These defects, such as fingerprints, broken lines, dark spots / patches, and main / sub-grid detachment, exhibit significant unique characteristics in terms of morphology, scale, contrast, and location. For example, main / sub-grid detachment manifests as fine linear breaks, appearing as low-contrast, variable-shape dark lines in images; fingerprint defects often appear as locally blurred areas with little difference in grayscale from the background; dark spots / patches vary greatly in scale and are often confused with the shadows at the edges of the solar cells. These characteristics make it difficult to directly apply general defect detection methods; especially rare defects occurring at the end of the production line or during field maintenance, due to the scarcity of samples and the difficulty in collection, make their detection and identification extremely challenging.

[0003] Currently, deep learning-based visual inspection methods are widely used in industrial quality inspection. Traditional convolutional neural network models, such as ResNet and VGG, can achieve excellent classification performance when trained on large, balanced datasets. However, these methods often perform poorly in real-world scenarios with extremely imbalanced sample sizes and a scarcity of rare defect samples. The models tend to overfit to common defect categories with large sample sizes, while their ability to distinguish rare defects is severely insufficient, leading to increased false negative and false positive rates.

[0004] Existing research has employed strategies such as data augmentation, cost-sensitive learning, or hard sample mining to alleviate class imbalance. However, these methods offer limited improvement when the defective features are extremely similar or when the background interference is complex. Furthermore, some methods attempt to address data scarcity through meta-learning or few-shot learning frameworks, but their model structures are complex, the training process is unstable, and they are difficult to integrate effectively with existing high-performance backbone networks.

[0005] A review and analysis of existing literature revealed the following: "A Solar Cell Defect Detection Method Based on Transfer Learning and ResNet" demonstrates an effective approach to improving detection performance using pre-trained models; "Solar Cell Surface Defect Detection Integrating Multi-Scale Features and Attention" presents the CMFAnet algorithm, which integrates multi-scale features and attention mechanisms, achieving an average detection accuracy (mAP) of 91.4%; and "A Solar Cell Defect Detection Method Based on Deep Convolutional Neural Networks" proposes a solar cell defect detection method based on deep convolutional neural networks (DCNNs), achieving a defect detection accuracy of over 97%. However, these methods may still have limitations such as insufficient feature discrimination or high model complexity when facing rare defect detection scenarios with extremely imbalanced samples.

[0006] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0007] In view of this, the present invention provides a solar cell defect detection method that combines multi-scale feature enhancement and staged metric learning to solve the aforementioned problems.

[0008] To solve the above problems, the specific technical solution adopted by the present invention is as follows:

[0009] A solar cell defect detection method combining multi-scale feature enhancement and staged metric learning includes the following steps:

[0010] S1. An improved deep feature extraction network based on the ResNet-50 architecture is constructed, and a deep residual network with multi-scale feature enhancement embedded with a channel attention module and a multi-scale feature fusion module is built by combining pre-trained ResNet-50 architecture weights.

[0011] S2. The parameters of the deep residual network are progressively optimized in stages by adopting the parameter freezing method, and a twin network structure containing two parallel sub-networks with real-time parameter synchronization is constructed based on the optimized deep residual network.

[0012] S3. Using the Siamese network structure, high-dimensional deep feature vectors are extracted from the training sample pairs pre-constructed using solar cell electroluminescence images. Based on the dual-metric calculation strategy, dual metrics are calculated using the high-dimensional deep feature vectors to construct the fusion loss function, so as to optimize the Siamese network structure.

[0013] S4. Using the optimized twin network structure and combined with the pre-built standard sample support set, defect detection is performed on the sample to be tested.

[0014] Preferably, a deep residual network for multi-scale feature enhancement, based on an improved deep feature extraction network of the ResNet-50 architecture and combined with pre-trained ResNet-50 architecture weights, and incorporating a channel attention module and a multi-scale feature fusion module, includes the following steps:

[0015] S11. Using a transfer learning strategy, the backbone network of the ResNet-50 architecture is initialized using weights pre-trained on a visualization image dataset, resulting in a preliminary ResNet-50 architecture.

[0016] S12. Construct a feature enhancement structure consisting of a channel attention module and a multi-scale feature fusion module connected in series, which is used for adaptive calibration of channel feature responses and fusion of multi-scale defect information;

[0017] S13. Remove the global average pooling layer and classification head at the end of ResNet-50, retaining only the convolutional part as the backbone network. Embed the feature enhancement structure and the preset mapping module into the backbone network. The feature enhancement structure consists of a channel attention module and a multi-scale feature fusion module connected in series and is embedded at the end of the backbone network. The mapping module contains a newly added global average pooling layer and two fully connected layers, which are followed by the feature enhancement structure to obtain a deep residual network with multi-scale feature enhancement.

[0018] Preferably, the adaptive calibration channel characteristic response and fused multi-scale defect information include:

[0019] Obtain the feature map output by the initialized ResNet-50 architecture, and use global average pooling to obtain global information for each feature channel;

[0020] Based on the global information of each feature channel, the attention weight of each feature channel is calculated, and the attention weight of each feature channel is multiplied with the feature map output by the initialized ResNet-50 architecture to obtain the calibrated feature map.

[0021] The calibrated feature map is input into the multi-scale feature fusion module. The parallel branching structure of the multi-scale feature fusion module is used to perform feature extraction and feature fusion to obtain an enhanced feature map that is robust to scale changes.

[0022] Preferably, the parameters of the deep residual network are progressively optimized in stages using a parameter freezing method. Based on the optimized deep residual network, a twin network structure containing two parallel sub-networks with real-time parameter synchronization is constructed, including the following steps:

[0023] S21. An adaptive channel-aware initialization method based on Kaiming normal distribution is adopted to initialize the parameters of the channel attention module, the multi-scale feature fusion module, and the fully connected layer in the embedding layer of the deep residual network with multi-scale feature enhancement, so as to obtain the initialized deep residual network.

[0024] S22. Freeze all parameters of the backbone network of the initialized deep residual network, and unfreeze the parameters of the channel attention module, the multi-scale feature fusion module and the embedding layer.

[0025] S23. Using the preset solar cell defect data, drive the channel attention module, multi-scale feature fusion module and embedding layer to learn parameter representations suitable for defect feature enhancement and measurement, and obtain the learned deep residual network.

[0026] S24. Unfreeze the learned deep residual network and perform end-to-end joint training on the unfrozen deep residual network using a preset learning rate to obtain an optimized deep residual network.

[0027] S25. Using the optimized deep residual network as a weight-sharing subnetwork, construct a twin network architecture containing two parallel subnetworks with real-time parameter synchronization.

[0028] Preferably, the Siamese network structure is optimized by extracting high-dimensional deep feature vectors from training sample pairs pre-constructed using electroluminescence images of solar cells, and then using a dual-metric calculation strategy to calculate dual metrics for constructing the fusion loss function based on the high-dimensional deep feature vectors. This optimization process includes the following steps:

[0029] S31. Collect images of the electroluminescence of solar cells, perform data optimization processing, and combine a class-balanced weighted sampling strategy to construct training sample pairs for metric learning.

[0030] S32. Input the training sample pairs into the Siamese network structure, extract features through the Siamese network structure, and output two sets of high-dimensional deep feature vectors;

[0031] S33. Using a dual-metric collaborative mechanism, the cosine similarity and Euclidean distance between two sets of high-dimensional deep feature vectors are calculated to obtain the dual-metric results.

[0032] S34. Calculate the weighted fusion loss function based on the dual metric results, and optimize the network parameters of the Siamese network structure through the backpropagation algorithm to obtain the optimized Siamese network structure.

[0033] Preferably, collecting electroluminescence images of solar cells, performing data optimization processing, and constructing training sample pairs for metric learning using a class-balanced weighted sampling strategy includes the following steps:

[0034] S311. The LANCZOS resampling algorithm is used to normalize the size of the collected solar cell electroluminescence images to obtain normalized images.

[0035] S312. Using the mean and standard deviation of the visualized image dataset, the normalized image is pixel-standardized to obtain a standardized image.

[0036] S313. Perform channel expansion and data augmentation on the standardized image to obtain the training dataset;

[0037] S314. Traverse the training dataset, adopt a class-balanced weighted sampling strategy to generate positive and negative sample pairs, and integrate the positive and negative sample pairs to obtain training sample pairs for metric learning.

[0038] Preferably, the process of traversing the training dataset, employing a class-balanced weighted sampling strategy to generate positive and negative sample pairs, and then integrating these pairs to obtain training sample pairs for metric learning includes the following steps:

[0039] S3141. By traversing the training dataset, count the number of samples for each defect category and calculate the sampling weight for each category;

[0040] S3142. Based on sampling weights and biased negative sampling, a random number generation method is used to generate positive and negative sample pairs until a preset batch size of positive and negative sample pairs is generated.

[0041] S3143. By integrating sample pairs, training sample pairs for metric learning are obtained.

[0042] Preferably, a dual-metric collaborative mechanism is used to calculate the cosine similarity and Euclidean distance between two sets of high-dimensional deep feature vectors. The dual-metric results are obtained by the following steps:

[0043] S331. The cosine similarity and Euclidean distance between two sets of high-dimensional deep feature vectors, respectively;

[0044] S332. Transform the cosine similarity into a preset interval using a linear mapping function to obtain the similarity score;

[0045] S333. Use the Euclidean distance and similarity score between the two sets of high-dimensional deep feature vectors as the dual measurement results.

[0046] Preferably, the process of calculating the weighted fusion loss function based on the dual-metric results and optimizing the network parameters of the Siamese network structure using the backpropagation algorithm to obtain the optimized Siamese network structure includes the following steps:

[0047] S341. Calculate the binary cross-entropy loss based on the similarity score in the dual-measure results;

[0048] S342. Calculate the contrast loss based on the Euclidean distance in the dual-metric results;

[0049] S343. Based on the preset weight coefficients, a weighted fusion loss function is constructed by combining the binary cross-entropy loss and the contrast loss.

[0050] S344. Based on the weighted fusion loss function, and combined with the iterative optimization of all parameters of the Siamese network structure through backpropagation until the preset iteration termination condition is reached, the optimized Siamese network structure is obtained.

[0051] Preferably, using the optimized twin network structure, combined with a pre-built standard sample support set, defect detection of the sample to be tested includes the following steps:

[0052] S41. Pre-build a standard sample library covering all categories of defects to be detected as a support set;

[0053] S42. Input the image of the solar cell to be tested into one of the subnetworks in the optimized twin network structure, and extract the high-dimensional deep feature vector of the image of the solar cell to be tested.

[0054] S43. Input each standard sample image in the support set into another sub-network in the Siamese network structure and extract the feature vector of each standard sample image.

[0055] S44. Calculate the similarity score between the high-dimensional depth feature vector of the solar cell image to be tested and the feature vector of each standard sample image, and select the defect category to which the standard sample image in the support set with the highest similarity score belongs.

[0056] S45. The selected defect category is used as the final defect type identification result of the solar cell image under test.

[0057] The beneficial effects of this invention are as follows:

[0058] 1. This invention provides a solar cell defect detection method that combines multi-scale feature enhancement and staged metric learning. It has the advantage of significantly improving the model's ability to distinguish rare defects through metric learning mechanism when the number of defect samples is limited or the categories are extremely imbalanced. This solves the problems of poor generalization ability and low recognition accuracy of existing models in such scenarios.

[0059] 2. The strategy of combining metric learning and phased training adopted in this invention transforms the model's learning objective from a traditional classification task to a similarity metric between samples, and implements phased, progressive optimization. This helps the model learn more discriminative feature representations in scenarios with scarce samples or class imbalance. This design enables the model to better adapt to the detection requirements of rare defects in solar cells, providing a feasible technical path to alleviate the model overfitting problem caused by insufficient samples.

[0060] 3. The multi-scale feature enhancement network structure constructed in this invention, by integrating the ECA channel attention module and the multi-scale feature fusion module, can achieve adaptive calibration of feature channels and effective fusion of multi-scale contextual information while controlling model complexity. This structural design helps the model better capture key features related to defects and adapt to the detection needs of defects at different scales, providing technical support for improving the discriminative power of feature representation.

[0061] 4. The dual metric and fusion loss mechanism designed in this invention combines the complementary characteristics of cosine similarity and Euclidean distance, and adopts a weighted fusion loss function, which helps guide the model to learn a more discriminative feature embedding space. This mechanism provides methodological support for improving the clustering of similar samples and the separation of dissimilar samples, and is conducive to improving the accuracy of solar cell defect identification. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart of the solar cell rare defect detection method based on metric learning and improved residual network in the solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to an embodiment of the present invention.

[0064] Figure 2 This is a twin network structure diagram in the solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to an embodiment of the present invention;

[0065] Figure 3 This is a structural diagram of the ASPP feature fusion module in the solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to an embodiment of the present invention.

[0066] Figure 4This is a structural diagram of the ECA module in a solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to an embodiment of the present invention.

[0067] Figure 5 This is an image of a solar cell defect, specifically a primary and secondary grid detachment type defect, in a solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to an embodiment of the present invention.

[0068] Figure 6 This refers to fingerprint-type solar cell defect images in a solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to an embodiment of the present invention.

[0069] Figure 7 This refers to the image of black spot / black patch type solar cell defects in the solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to an embodiment of the present invention.

[0070] Figure 8 This invention relates to a solar cell defect detection method combining multi-scale feature enhancement and staged metric learning, which detects broken line imprints in solar cells.

[0071] Figure 9 This is a schematic diagram of the training and validation loss curves in the solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to an embodiment of the present invention.

[0072] Figure 10 This is a schematic diagram of the training and validation accuracy curves in a solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to an embodiment of the present invention. Detailed Implementation

[0073] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0074] According to embodiments of the present invention, a solar cell defect detection method combining multi-scale feature enhancement and staged metric learning is provided.

[0075] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, according to an embodiment of the present invention, a solar cell defect detection method combining multi-scale feature enhancement and staged metric learning is provided. The method includes the following steps:

[0076] S1. An improved deep feature extraction network based on the ResNet-50 architecture is constructed, and a deep residual network with multi-scale feature enhancement embedded with a channel attention module and a multi-scale feature fusion module is built by combining pre-trained ResNet-50 architecture weights.

[0077] It should be noted that, to overcome the problems of overfitting and weak generalization ability caused by the limited number of solar cell defect samples, especially rare defect samples, an improved deep feature extraction network based on the ResNet-50 architecture is constructed. This network adopts a transfer learning strategy, using the weights of a ResNet-50 model pre-trained on the large ImageNet dataset for parameter initialization of the backbone network. Based on this, a feature enhancement structure consisting of an efficient channel attention (ECA) module and a multi-scale feature fusion (ASPP) module is embedded at the end of the network, before the global average pooling layer. The ECA module adaptively calibrates the channel weights of high-level feature maps through a local cross-channel interaction mechanism, significantly enhancing the feature channel responses related to solar cell defect regions (such as the fine line features of main and sub-gate separation, and the regional features of black spots and patches), while effectively suppressing background interference features. This structure adaptively strengthens the feature channel responses related to defect regions through the channel attention mechanism and fuses feature information from different receptive fields through a multi-branch parallel architecture to generate enhanced feature representations robust to scale changes. At the end of the network, an embedding mapping module is set up to finally map the 2048-dimensional features to a 512-dimensional normalized embedding vector through two fully connected layers.

[0078] As a preferred implementation, a deep residual network for multi-scale feature enhancement, which incorporates a channel attention module and a multi-scale feature fusion module, is constructed based on an improved deep feature extraction network of the ResNet-50 architecture and combined with pre-trained ResNet-50 architecture weights. The steps include:

[0079] S11. Using a transfer learning strategy, the backbone network of the ResNet-50 architecture is initialized using weights pre-trained on a visualization image dataset, resulting in a preliminary ResNet-50 architecture.

[0080] S12. Construct a feature enhancement structure consisting of a channel attention module and a multi-scale feature fusion module connected in series, which is used for adaptive calibration of channel feature responses and fusion of multi-scale defect information;

[0081] As a preferred embodiment, the adaptive calibration channel characteristic response and fusion of multi-scale defect information include:

[0082] Obtain the feature map output by the initialized ResNet-50 architecture, and use global average pooling to obtain global information for each feature channel;

[0083] Based on the global information of each feature channel, the attention weight of each feature channel is calculated, and the attention weight of each feature channel is used in conjunction with the feature map output by the initialized ResNet-50 architecture.

[0084] The calibrated feature map is input into the multi-scale feature fusion module. The parallel branching structure of the multi-scale feature fusion module is used to perform feature extraction and feature fusion to obtain an enhanced feature map that is robust to scale changes.

[0085] S13. Remove the global average pooling layer and classification head at the end of ResNet-50, retaining only the convolutional part as the backbone network. Embed the feature enhancement structure and the preset mapping module into the backbone network. The feature enhancement structure consists of a channel attention module (ECA) and a multi-scale feature fusion module (ASPP) connected in series and is embedded at the end of the backbone network. The mapping module contains a newly added global average pooling layer and two fully connected layers, which are followed by the feature enhancement structure to obtain a deep residual network with multi-scale feature enhancement.

[0086] It should be noted that a deep feature extraction network is constructed as the core feature extractor. This network adopts a transfer learning strategy, using ResNet-50 weights pre-trained on large datasets such as ImageNet for parameter initialization, but removing the global average pooling layer and classification head at its end, retaining only the convolutional feature extraction part as the backbone network. On this basis, at the end of this backbone network (i.e., before the original global average pooling layer), a feature enhancement structure consisting of an efficient channel attention (ECA) module and a multi-scale feature fusion module is embedded to adaptively calibrate the channel feature response and fuse multi-scale defect information.

[0087] Specifically, at the end of the standard ResNet-50 architecture, before the global average pooling layer, a feature enhancement structure is embedded, consisting of an efficient channel attention (ECA) module and a multi-scale feature fusion (ASPP) module connected in series. The feature map output by the ResNet-50 backbone network has a size of 7×7×2048, which serves as the input to the ECA module. The ECA module first performs adaptive calibration of the channel weights on the input features, and its specific implementation includes:

[0088] For the input feature map H, W, and C represent the height, width, and number of channels of the feature map, respectively, and R represents the real number field, meaning that each element in the feature map is a real number. ECA uses Global Average Pooling (GAP) to capture global information for each channel. The GAP operation process can be represented as:

[0089] ;

[0090] In the formula, A GAP (F) represents global information, F cij Let represent the (i,j)th element of channel c in the input feature map F. After the GAP operation, a Conv1D(·) convolution is performed. Subsequently, a one-dimensional Conv1D convolutional layer captures cross-channel interaction information. The kernel size k is automatically determined according to the number of channels, and can be expressed as:

[0091] ;

[0092] In the formula, c represents the number of channels, and γ and b represent empirical constants, which are set to 2 and 1 respectively here; |t| odd This indicates selecting the odd number closest to the value t. Attention weight W for each channel. c The implementation process can be represented as follows:

[0093] ;

[0094] In the formula, σ represents the Sigmoid activation function.

[0095] The final output of the ECA module is determined by weight W. c The product of the input F to the ECA module can be expressed as:

[0096] ;

[0097] In the formula, W c Point-to-point multiplication between F and F.

[0098] The feature maps calibrated by the ECA module are then fed into the ASPP module; the ASPP module employs four branches in parallel to capture multi-scale contextual information at a single input scale, and its specific structure includes:

[0099] Branch 1: A 1×1 convolutional layer used to preserve the spatial details of the original feature map;

[0100] Branch 2: A 3×3 convolutional layer with an atrous rate r1 set to 6;

[0101] Branch 3: A 3×3 convolutional layer with a porosity r2 set to 12;

[0102] Branch 4: A global average pooling layer followed by a 1×1 convolutional layer for feature transformation, and then upsampling to the spatial size of the input feature map (7×7) through bilinear interpolation, used to introduce image-level global contextual features;

[0103] The calculation and setting principle of the void ratio r is as follows: In deep convolutional networks, the effective receptive field is related to the output stride. For the ResNet-50 backbone network used in this method (with an output stride of 32 relative to the input), the effective receptive field of its terminal feature maps is already relatively large. In order to capture a wider range and more multi-scale contextual information without further increasing the number of parameters, in order to cope with the variability of solar cell defect morphology, the void ratio of the ASPP branch needs to grow exponentially to generate multi-scale features. Its setting follows the following empirical formula:

[0104] ;

[0105] Where i is the branch index, in this method, the base void ratio is set to base_rate=6 and the multiplier is set to multiplier=2, therefore:

[0106] ;

[0107] This setup ensures that different branches can capture features ranging from local details (branch one) to contextual information of different ranges (branch two and three), and finally merge them with global features (branch four). The feature maps output by the four branches are concatenated along the channel dimension, and then channel fusion and dimensionality reduction are performed through a 1×1 convolutional layer to finally generate an enhanced feature map that is robust to scale changes.

[0108] S2. The parameters of the deep residual network are progressively optimized in stages by adopting the parameter freezing method, and a twin network structure containing two parallel sub-networks with real-time parameter synchronization is constructed based on the optimized deep residual network.

[0109] It should be noted that the training of the deep residual network with multi-scale feature enhancement adopts a phased optimization strategy: in the first phase, the pre-trained backbone network is frozen, and only the feature enhancement pipeline and subsequent layers are trained; in the second phase, the freeze is lifted, and the entire network is jointly fine-tuned end-to-end.

[0110] To evaluate model performance and monitor the training process, the entire solar cell electroluminescence image dataset was divided into training, validation, and test sets in a 6.4:1.6:2 ratio. During training, the loss and accuracy values ​​on the training and validation sets were recorded, and corresponding change curves were plotted (e.g., ...). Figure 9 , Figure 10 (As shown).

[0111] As a preferred implementation, a parameter freezing method is used to progressively optimize the parameters of the deep residual network in stages. Based on the optimized deep residual network, a twin network structure containing two parallel sub-networks with real-time parameter synchronization is constructed, including the following steps:

[0112] S21. An adaptive channel-aware initialization method based on Kaiming normal distribution is adopted to initialize the parameters of the channel attention module, the multi-scale feature fusion module, and the fully connected layer in the embedding layer of the deep residual network with multi-scale feature enhancement, so as to obtain the initialized deep residual network.

[0113] S22. Freeze all parameters of the backbone network of the initialized deep residual network, and unfreeze the parameters of the channel attention module, the multi-scale feature fusion module and the embedding layer.

[0114] S23. Using the preset solar cell defect data, drive the channel attention module, multi-scale feature fusion module and embedding layer to learn parameter representations suitable for defect feature enhancement and measurement, and obtain the learned deep residual network.

[0115] S24. Unfreeze the learned deep residual network and perform end-to-end joint training on the unfrozen deep residual network using a preset learning rate to obtain an optimized deep residual network.

[0116] S25. Using the optimized deep residual network as a weight-sharing sub-network, construct a Siamese network architecture containing two parallel sub-networks with real-time parameter synchronization. Specific implementation includes:

[0117] S251. The optimized deep residual network is defined as two logically parallel sub-networks. These two logical sub-networks share the same network structure and parameter configuration, and are used to process the two images in the input image pair respectively, forming a twin processing logic architecture with real-time parameter synchronization.

[0118] S252. An optimization scheme using multiple forward calls of a single network is adopted: the deep residual network is instantiated as a single feature extractor. Through two forward propagation calls of this single instance, features are extracted from the two images in the image pair respectively. Since the two calls use the same set of parameters, the consistency of parameters is naturally guaranteed, and no additional synchronization mechanism is required.

[0119] S253. Define a forward propagation method for a twin network, which receives paired solar cell images. As input, the corresponding feature vectors are calculated by calling the single feature extractor twice. and The feature extraction function This represents the optimized deep residual network. These are normalized feature vectors;

[0120] S254. Establish a batch-dimensional feature alignment mechanism to ensure that when processing image pairs in batches, the feature vectors extracted from each pair of images maintain the correct correspondence in the batch, providing structured feature output for subsequent similarity calculation and loss optimization.

[0121] It should be noted that for the newly embedded efficient channel attention module (ECA) and multi-scale feature fusion module (ASPP), such as Figure 3-4 As shown, an adaptive initialization method combining the Kaiming normal distribution and the channel sensing factor is used for random initialization. The specific implementation process is as follows:

[0122] For each newly added convolutional and fully connected layer in the network, its weight parameter tensor W param Sampling is performed from a normal distribution with a mean of 0:

[0123] ;

[0124] Among them, standard deviation Determined by the following formula:

[0125] ;

[0126] In the formula, for a convolutional layer, f in This represents the product of the number of input channels and the kernel size of the layer; for fully connected layers, This represents the number of input connections. This is the standard definition of the Kaiming normal distribution.

[0127] Based on this, a channel-aware scaling factor is introduced. Adaptive adjustments are made to the standard initialization method:

[0128] ;

[0129] In the formula, This represents the Kaiming gain coefficient. This represents the channel-aware scaling factor, used to adaptively adjust the initial response intensity of different modules or branches. N represents a normal distribution. This represents the parameters of the weight tensor.

[0130] For a one-dimensional convolutional layer of the ECA module, the channel-aware scaling factor γ eca Defined as:

[0131] ;

[0132] Where C represents the number of channels, k represents the kernel size, and α represents the adjustment coefficient (with a value of 0.1). This design adaptively adjusts the initial variance based on the channel size, ensuring that high-channel features maintain strong responsiveness, thereby balancing the contributions of features at different scales.

[0133] For the four parallel branches of the ASPP module, different channel awareness factors are set for each:

[0134] ;

[0135] Among them, branch one (γ) asp1 ): 1×1 convolution, enhancing detail preservation; branch two (γ asp2 ): Void ratio 6, maintain standard initialization; Branch three (γ) asp3 ): Hollowness ratio 12, suppressing the initial response of large receptive field branches; Branch four (γ asp4 Global pooling enhances global features;

[0136] Based on the above branch initialization methods, the overall initialization distribution of the ASPP module is W asp It can be represented as:

[0137] ;

[0138] Simultaneously, all bias terms are uniformly initialized to a zero vector. The technical advantage of this initialization method lies in…

[0139] (1) To address the multi-scale characteristics of solar cell defects, a differentiated initialization strategy is adopted for branches of different receptive fields to avoid feature learning bias caused by uniform initialization.

[0140] (2) By using a scaling mechanism that adapts the number of channels, the contribution of different channels in the ECA module can be better balanced in the early stage of training;

[0141] (3) Improve the feature learning effect of rare defects (such as main gate and sub-gate separation) and enhance the model’s sensitivity to subtle defects through targeted initialization strategies.

[0142] Simultaneously, all bias terms are uniformly initialized to zero vectors. After parameter initialization, a core feature extractor integrating the ECA and ASPP modules is formed at the end of the improved residual network and before the global average pooling layer, providing multi-scale enhanced feature representations for subsequent defect metric learning.

[0143] Specifically, to achieve stable convergence and efficient optimization of the model, a phased, progressive training strategy is adopted, and the parameters of newly added modules are scientifically initialized:

[0144] Phase 1: Initialization and constrained training of the feature enhancement module;

[0145] (1) Parameter initialization: For the newly embedded ECA module, multi-scale feature fusion module, and fully connected layer in the embedding layer, an adaptive channel-aware initialization method based on the Kaiming normal distribution is adopted. Its weight parameters are sampled from the following distribution expression:

[0146] ;

[0147] In the formula, for convolutional layers, f represents the product of the number of input channels and the kernel size. For fully connected layers, f in Indicates the number of input connections; Let N represent the parameters of the weight tensor, and let N represent the normal distribution. γ represents the Kaiming gain coefficient, and γ is the channel sensing scaling factor, used to adaptively adjust the initial response amplitude of different channels and branches at different scales; all bias terms are uniformly initialized to zero. This initialization method, while maintaining the stability of standard Kaiming, introduces channel structure and receptive field information, enabling the network to have multi-scale adaptive sensing capability for solar cell defects in the early stages of training, effectively accelerating the convergence speed and improving the detection accuracy of rare defects.

[0148] (2) Parameter state configuration: Freeze all parameters of the ResNet-50 backbone network, and only unfreeze the parameters of the newly embedded feature enhancement module and the embedding layer.

[0149] Specifically, all parameters of the ResNet-50 backbone network (covering all layers from the initial convolutional layer, batch normalization layer to the Layer4 output) are set to an untrainable state, and its weight updates are frozen; all parameters of the newly embedded ECA module, the multi-scale feature fusion module (ASPP), and the fully connected embedding layer (feature dimension mapping layer) after the global average pooling layer are set to a trainable state.

[0150] (3) Training objective: Under this constraint, the parameters of the feature enhancement module are specifically optimized using solar cell defect data so that it learns a transformation mapping suitable for defect feature representation and measurement, while avoiding damage to the general feature extraction capability of the pre-trained backbone network.

[0151] Phase Two: End-to-End Joint Fine-Tuning;

[0152] (1) Fully unfreeze: Unfreeze all the ResNet-50 backbone network and set all parameters of the entire improved feature extraction network (including the pre-trained backbone, ECA module, ASPP module) and the embedding layer to trainable state.

[0153] (2) Refined training: The entire network is jointly trained end-to-end with a small learning rate to achieve deep collaborative optimization between the pre-trained backbone network and the newly added feature enhancement module, thereby further improving the discriminative power of feature representation for solar cell defects.

[0154] In addition, such as Figure 2 As shown (pool is 3*3 max pool, stride is 2), when building the Siamese network structure, a deep residual network with multi-scale feature enhancement is used as a sub-network with weight sharing. This constructs a Siamese network architecture containing two parallel sub-networks with real-time parameter synchronization. This architecture achieves simultaneous processing and feature extraction of paired input images by forward propagating the same set of network parameters twice, ensuring that the two branches are completely consistent in structure and parameters.

[0155] S3. Using the Siamese network structure, high-dimensional deep feature vectors are extracted from the training sample pairs pre-constructed using solar cell electroluminescence images. Based on the dual-metric calculation strategy, dual metrics are calculated using the high-dimensional deep feature vectors to construct the fusion loss function, so as to optimize the Siamese network structure.

[0156] It should be noted that the training dataset consists of solar cell EL images containing four types of defects: fingerprints, black spots and patches, broken lines and dotted marks, and separation of main and sub-grids. Training sample pairs for metric learning are then constructed. Positive sample pairs consist of two images belonging to the same defect category, while negative sample pairs consist of two images belonging to different defect categories. Each image is assigned a corresponding binary label (1 for positive sample pairs and 0 for negative sample pairs).

[0157] At the same time, such as Figure 5-8As shown, to address the extreme sample imbalance problem caused by the coexistence of common defects such as fingerprints, black spots, black patches, and broken lines in the dataset, as well as rare defects such as main and sub-gate detachment, an enhanced balanced sampling strategy for solar cell defects is adopted. This strategy calculates the sampling weight based on the number of samples in each category, with the weight being inversely proportional to the number of samples. This ensures that rare defect categories such as main and sub-gate detachment are assigned the highest sampling weight and are fully learned during training.

[0158] In the construction of negative sample pairs, a biased negative sampling mechanism is further introduced. It deliberately selects the two different categories with the fewest and most samples (i.e., primary and secondary grid detachment and fingerprint) with a significant probability to form negative sample pairs. This forces the model to strengthen its ability to distinguish the most challenging rare defects, thereby effectively alleviating the problem of imbalance in the number of class samples in the dataset. This lays the foundation for the model to learn robust feature representations on all defect categories, especially primary and secondary grid detachment.

[0159] As a preferred implementation, the Siamese network structure is used to extract high-dimensional deep feature vectors from training sample pairs pre-constructed using electroluminescence images of solar cells. Based on a dual-metric calculation strategy, dual metrics are calculated using these high-dimensional deep feature vectors to construct a fusion loss function, thereby optimizing the Siamese network structure. This includes the following steps:

[0160] S31. Collect images of the electroluminescence of solar cells, perform data optimization processing, and combine a class-balanced weighted sampling strategy to construct training sample pairs for metric learning.

[0161] In a preferred embodiment, collecting electroluminescence images of solar cells, performing data optimization processing, and constructing training sample pairs for metric learning using a class-balanced weighted sampling strategy includes the following steps:

[0162] S311. The LANCZOS resampling algorithm is used to normalize the size of the collected solar cell electroluminescence images to obtain normalized images.

[0163] S312. Using the mean and standard deviation of the visualized image dataset, the normalized image is pixel-standardized to obtain a standardized image.

[0164] S313. Perform channel expansion and data augmentation on the standardized image to obtain the training dataset;

[0165] S314. Traverse the training dataset, employ a class-balanced weighted sampling strategy to generate positive and negative sample pairs, and integrate these pairs to obtain training sample pairs for metric learning. Specifically, the LANCZOS resampling algorithm is used to uniformly scale the image resolution to 224×224 pixels. The LANCZOS kernel function is defined as:

[0166] ;

[0167] In the formula, L (x) This represents the LANCZOS resampling kernel function, with parameter a=3. The resampling process is implemented through convolution:

[0168] ;

[0169] In the formula, (u,v) represents the coordinates of the target image, (x,y) represents the corresponding coordinates of the original image, and S x S y Indicates the horizontal and vertical scaling ratio, I resized I represents the normalized image. original The image represents the original electroluminescent image of the solar cell, L represents the LANCZOS resampling kernel function, and i and j represent the summation indices.

[0170] After scaling the image spatial resolution to a uniform 224×224 pixels, pixel standardization is performed using the mean and standard deviation based on the ImageNet dataset. The calculation formula is as follows:

[0171] ;

[0172] In the formula, I norm Represents a normalized image, I raw This represents the original pixel value image after size normalization. , .

[0173] The expression for expanding a pixel-normalized single-channel image into a three-channel image is:

[0174] ;

[0175] In the formula, I RGB (x,y) represents the expanded three-channel pseudo-color image, I gray This represents the original single-channel grayscale EL image.

[0176] Geometric enhancement is achieved through affine transformation:

[0177] ;

[0178] In the formula, the scaling factor s∈[0.8,1.2], the rotation angle θ∈[-20°,20°], and the translation amount t x , t y . , The original image pixel coordinates are represented by u′ and v′, respectively. The homogeneous coordinate extended component is represented by l and is used for matrix operations.

[0179] As a preferred implementation, the training dataset is traversed, a class-balanced weighted sampling strategy is used to generate positive and negative sample pairs, and the positive and negative sample pairs are integrated to obtain training sample pairs for metric learning, including the following steps:

[0180] S3141. By traversing the training dataset, count the number of samples for each defect category and calculate the sampling weight for each category;

[0181] S3142. Based on sampling weights and biased negative sampling, a random number generation method is used to generate positive and negative sample pairs until a preset batch size of positive and negative sample pairs is generated.

[0182] S3143. By integrating sample pairs, training sample pairs for metric learning are obtained.

[0183] It should be noted that, considering the extreme class imbalance distribution in this solar cell defect dataset, the number of samples for common planar contamination defects such as fingerprints (636 images) and black spots / patches (425 images) differs significantly from that of key but rare linear defects such as main / sub-grid detachment (only 24 images). Therefore, this method employs a weighted sampling strategy based on class balance. Specifically, the four types of defects have distinct physical characteristics and manifestations, and their impact on the photoelectric conversion efficiency of solar cells varies: main / sub-grid detachment, as a key linear defect directly affecting carrier collection efficiency, appears as a fine, low-contrast dark line in the EL image, severely disrupting the current transport path; broken lines and imprints, as discontinuous linear defects, exhibit discontinuous linear features, leading to increased local series resistance; black spots / patches, as common planar defects, appear as dark areas of different scales, directly reducing the effective power generation area; and fingerprints, as common planar contamination defects, appear as gray-scale variation areas with blurred boundaries, mainly causing surface light trapping effect loss.

[0184] Specifically, a weighted sampling strategy based on category balance is used to construct sample pairs, specifically addressing the imbalanced sample distribution problem in solar cell EL images for four types of defects: fingerprints, black spots / masks, broken lines / virtual imprints, and separation of main and sub-grids. The number of samples N for each defect category c is counted. c Based on the number of samples N c Calculate the sampling weight P for each category cBased on the number of samples N in the training set for each defect category c. c Calculate its sampling weight P c The weights are inversely proportional to the number of samples, and the calculation formula is:

[0185] ;

[0186] In the formula, C represents the total number of categories, and N... k This represents the number of samples in the k-th class.

[0187] Based on the above formula, the weights of fingerprints, broken lines, black spots and patches, and separation of main and secondary grids are calculated to be 0.0356, 0.0558, 0.0487, and 0.8629, respectively. Among them, the sampling weight of separation of main and secondary grids, which has a small sample size, is 26.5 times that of fingerprints.

[0188] For each sample pair to be generated, first generate a random number uniformly distributed in the range [0,1). If the random number is greater than 0.5, decide to generate a positive sample pair; if the random number is less than or equal to 0.5, decide to generate a negative sample pair.

[0189] If the decision generates positive sample pairs, then according to the sampling weight P c First, randomly select a category c from all defect categories. Then, randomly select two different samples from the sample set belonging to category c, combine them into a positive sample pair, and assign them similarity labels. ;

[0190] If the decision generates negative sample pairs, then biased negative sampling is performed: with a first preset probability (its value is 0.7), the two extreme categories with the fewest sample numbers (24 images of primary and secondary grid detachment) and the most sample numbers (636 images of fingerprints) are deliberately selected as the first category and the second category, respectively, as they have significant differences in defect morphology, impact on efficiency, and environmental sensitivity; with a second preset probability (its value is 0.3), based on the obtained sampling weight P... c Two distinct categories are independently and randomly selected as the first and second categories, respectively. Then, one sample is randomly drawn from each category, and these are combined to form a negative sample pair, which is then assigned a similarity label. ;

[0191] Repeatedly perform random number generation and balanced sampling and construction until the number of generated sample pairs reaches the preset batch capacity, thereby forming a balanced solar cell defect training batch that fully covers linear defects to planar defects, from minor efficiency impacts to severe damage, and from sufficient samples to scarce samples.

[0192] In practical implementation, for a dataset containing fingerprints (636 images), black spots / patches (425 images), broken lines / dummy prints (371 images), and main / sub-gate detachment (24 images), this weighting mechanism calculates the sampling weight for the main / sub-gate detachment category to be approximately 26.5 times that of the fingerprint category, ensuring that defect categories with very few samples receive a higher sampling probability. Each sample is randomly assigned a weight corresponding to its category. For example, when constructing training batches, samples with main / sub-gate detachment defects have a significantly higher probability of being selected than those of other categories. This alleviates the inherent class imbalance problem in solar cell defect datasets, ensuring that rare defects such as main / sub-gate detachment receive sufficient learning opportunities during training, thereby improving the model's ability to identify defects across the entire solar cell spectrum.

[0193] S32. Input the training sample pairs into the Siamese network structure, extract features through the Siamese network structure, and output two sets of high-dimensional deep feature vectors;

[0194] It should be noted that the constructed sample pairs are input into the built Siamese network, and a forward propagation process with a hierarchical feature fusion mechanism is performed. Two sub-networks perform progressive multi-level processing on the input image: first, basic features are extracted through the ResNet-50 backbone network; then, channel adaptive calibration is performed through the ECA module to highlight defect-related feature channels; next, the four parallel branches of the multi-scale feature fusion module simultaneously extract contextual information under different receptive fields and perform deep fusion; finally, the fused multi-scale features are mapped to a 512-dimensional normalized embedding space through the embedding mapping module. This multi-level feature processing mechanism, through channel reweighting, multi-scale fusion, and adaptive embedding mapping, significantly improves the discriminativeness and specificity of feature representation for solar cell defects (such as blurred areas of fingerprints and discontinuous features of broken lines).

[0195] Specifically, the obtained sample pairs (x1, x2) are input into the Siamese network. A forward propagation process based on multi-scale feature enhancement is performed, and parameter optimization is carried out based on dual-metric fusion loss, including the following steps:

[0196] 1. Multi-scale feature enhancement forward propagation: Two sub-networks process the input image separately, extracting high-dimensional deep feature vectors through a serialized multi-scale feature enhancement mechanism. The specific implementation is as follows:

[0197] For feature maps input from a ResNet-50 network ,in , , These represent the height, width, and number of channels of the feature map, respectively; the specific operation of the ECA module is as follows:

[0198] (1) Compress each channel using global average pooling to obtain a channel statistical description:

[0199] ;

[0200] In the formula, F cij Let represent the (i,j)th element of channel c in the input feature map F.

[0201] (2) One-dimensional convolution (ConvlD(·)) is used to capture cross-channel interaction information. The obtained channel statistical description vectors are processed by one-dimensional convolution to capture the dependencies between adjacent channels. The kernel size k of the one-dimensional convolution is automatically determined based on the number of channels and can be expressed as:

[0202] ;

[0203] In the formula, c represents the number of channels, and γ and b represent empirical constants, which are set to 2 and 1 respectively here; |t| odd This indicates taking the odd number closest to the value t.

[0204] (3) Generate attention weights for each channel using the Sigmoid activation function. :

[0205] The cross-channel interaction information output by one-dimensional convolution is non-linearly transformed by the Sigmoid activation function to generate attention weights for each channel.

[0206] ;

[0207] In the formula, σ represents the Sigmoid activation function. This step converts the captured cross-channel interaction information into actual channel importance weights.

[0208] (4) The final output of the ECA module is determined by the weights. The product of the input F to the ECA module can be expressed as:

[0209] ;

[0210] In the formula, express Point-to-point multiplication between F and F.

[0211] The feature maps calibrated by the ECA module are then fed into the ASPP module; the ASPP module employs four branches in parallel to capture multi-scale contextual information at a single input scale, and its specific structure includes:

[0212] Branch 1: A 1×1 convolutional layer used to preserve the spatial details of the original feature map;

[0213] Branch 2: A 3×3 convolutional layer with an atrous rate r1 set to 6;

[0214] Branch 3: A 3×3 convolutional layer with a porosity r2 set to 12;

[0215] Branch 4: A global average pooling layer followed by a 1×1 convolutional layer for feature transformation, and then upsampling to the spatial size of the input feature map (7×7) through bilinear interpolation, used to introduce image-level global contextual features;

[0216] void ratio The calculation and setup principles are as follows: In deep convolutional networks, the effective receptive field is related to the output stride. For the ResNet-50 backbone network used in this method (with an output stride of 32 relative to the input), the effective receptive field of its terminal feature maps is already relatively large. In order to capture a wider range and more multi-scale contextual information without further increasing the number of parameters, in order to cope with the variability of solar cell defect morphology, the porosity of the ASPP branch needs to increase exponentially to generate multi-scale features. Its setup follows the following empirical formula:

[0217] ;

[0218] Where i is the branch index, in this method, the base void ratio is set to base_rate=6 and the multiplier is set to multiplier=2, therefore:

[0219] ;

[0220] This setup ensures that different branches can capture features ranging from local details (branch one) to contextual information of different ranges (branch two and three), and ultimately fuse them with global features (branch four). The feature maps output by the four branches are concatenated along the channel dimension, and then fused and reduced in dimensionality by a 1×1 convolutional layer, ultimately generating an enhanced feature map robust to scale changes. The output consists of two corresponding high-dimensional deep feature vectors v1=f(x1:θ) and v2=f(x2:θ). Here, v1=f(x1:θ) is the feature extraction function defined by the improved residual network, and θ is the set of all trainable parameters of the network. Its final value is determined through iterative optimization on the training dataset using the subsequent loss function and backpropagation algorithm.

[0221] S33. Using a dual-metric collaborative mechanism, the cosine similarity and Euclidean distance between two sets of high-dimensional deep feature vectors are calculated to obtain the dual-metric results.

[0222] As a preferred implementation, a dual-metric collaborative mechanism is used to calculate the cosine similarity and Euclidean distance between two sets of high-dimensional deep feature vectors. The dual-metric result is obtained by the following steps:

[0223] S331. The cosine similarity and Euclidean distance between two sets of high-dimensional deep feature vectors, respectively;

[0224] S332. Transform the cosine similarity into a preset interval using a linear mapping function to obtain the similarity score;

[0225] S333. Use the Euclidean distance and similarity score between the two sets of high-dimensional deep feature vectors as the dual measurement results.

[0226] Specifically, a dual-metric calculation strategy is adopted to simultaneously calculate the cosine similarity and Euclidean distance between the two output feature vectors:

[0227] (1) Calculate the similarity in the direction of two feature vectors to measure the class similarity of the input images. For example, two images of fingerprint defects should have a high cosine similarity. The formula is:

[0228] ;

[0229] In the formula, This indicates the degree of similarity between feature vectors v1 and v2 in the direction of their orientation.

[0230] (2) Quantify the absolute difference between two feature vectors in the embedding space to distinguish different categories of defects. For example, the feature vectors of black spots and patches and broken lines and dotted imprints should maintain a large Euclidean distance. The formula is:

[0231] ;

[0232] The cosine similarity is transformed to the interval between 0 and 1 using a linear mapping function to obtain the final similarity score, which serves as a probability estimate of whether two input images belong to the same category (including fingerprints, black spots and patches, broken lines and dotted marks, and main and sub-gate separation). This score directly reflects the probability that the image under test and the support set samples belong to the same defect category (such as both being main and sub-gate separation).

[0233] The expression for the vector dot product is:

[0234] ;

[0235] The expression for the L2 norm of a vector is:

[0236] ;

[0237] In the formula, n represents the dimension of the feature vector, and v 1i Represents the first eigenvector The i-th component (element value), v 2i Represents the second eigenvector The i-th component (element value). This distance is used to quantify the absolute difference between two feature vectors in n-dimensional space. The cosine similarity is transformed to the interval 0 to 1 using a linear mapping function:

[0238] ;

[0239] In the formula, This represents the similarity score, with a value range of [0,1].

[0240] S34. Calculate the weighted fusion loss function based on the dual metric results, and optimize the network parameters of the Siamese network structure through the backpropagation algorithm to obtain the optimized Siamese network structure.

[0241] As a preferred implementation, the weighted fusion loss function is calculated based on the dual-metric results, and the network parameters of the Siamese network structure are optimized using the backpropagation algorithm to obtain the optimized Siamese network structure, including the following steps:

[0242] S341. Calculate the binary cross-entropy loss based on the similarity score in the dual-measure results;

[0243] S342. Calculate the contrast loss based on the Euclidean distance in the dual-metric results;

[0244] S343. Based on the preset weight coefficients, a weighted fusion loss function is constructed by combining the binary cross-entropy loss and the contrast loss.

[0245] S344. Based on the weighted fusion loss function, and combined with the iterative optimization of all parameters of the Siamese network structure through backpropagation until the preset iteration termination condition is reached, the optimized Siamese network structure is obtained.

[0246] It should be noted that the weighted fusion loss is calculated based on the results of the two metrics, and the network parameters are optimized through the backpropagation algorithm. The fusion loss function L total The binary cross-entropy loss L bce Compared with the loss L cont It is composed of a weighted sum based on preset weights, and its calculation formula is as follows:

[0247] ;

[0248] In the formula, α and β represent the preset weighting coefficients, and α+β=1, α=0.7, β=0.3;

[0249] Binary cross-entropy loss L bce The input is the similarity score (i.e., the similarity score). The calculation formula is as follows:

[0250] ;

[0251] in, T represents the total number of sample pairs in a training batch. i T represents the true label of the i-th sample pair, where T represents a positive sample pair. i =1, T for negative sample pairs i =0, This represents the similarity score predicted by the model for the i-th sample pair;

[0252] Comparison loss L cont The input is the obtained Euclidean distance d(v1,v2), which is calculated using the following formula:

[0253] ;

[0254] in, Let represent the Euclidean distance between the feature vectors of the i-th sample pair, and m represent a preset marginal hyperparameter that is greater than zero. T represents the total number of sample pairs in a training batch. i T represents the true label of the i-th sample pair, where T represents a positive sample pair. i =1, T for negative sample pairs i =0; Through backpropagation, iteratively optimize all parameters of the Siamese network so that the similarity score of positive sample pairs approaches 1 and the Euclidean distance is minimized, while the similarity score of negative sample pairs approaches 0 and the Euclidean distance is greater than the preset margin.

[0255] S4. Using the optimized twin network structure and combined with the pre-built standard sample support set, defect detection is performed on the sample to be tested.

[0256] As a preferred implementation, the defect detection of the sample to be tested, using the optimized twin network structure and combined with a pre-built standard sample support set, includes the following steps:

[0257] S41. Pre-build a standard sample library covering all categories of defects to be detected as a support set;

[0258] Specifically, a standard sample library covering all categories of defects to be detected is pre-built as a support set. , among which, S i Represents a standard sample image, l i This represents the corresponding defect category label, and M represents the total number of samples in the support set. The support set contains one or more manually annotated samples that are located in the typical region of that category in the feature space.

[0259] S42. Input the image of the solar cell to be tested into one of the subnetworks in the optimized twin network structure, and extract the high-dimensional deep feature vector of the image of the solar cell to be tested.

[0260] S43. Input each standard sample image in the support set into another sub-network in the Siamese network structure and extract the feature vector of each standard sample image.

[0261] S44. Calculate the similarity score between the high-dimensional depth feature vector of the solar cell image to be tested and the feature vector of each standard sample image, and select the defect category to which the standard sample image in the support set with the highest similarity score belongs.

[0262] S45. The selected defect category is used as the final defect type identification result of the solar cell image under test.

[0263] It should be noted that the image of the solar cell under test is x test Input a subnetwork from a pre-trained Siamese network and extract its high-dimensional deep feature vector v. test =f(x test ;θ). Simultaneously, it will support each standard sample image S in the set. j Input each sub-network sequentially and extract its respective feature vector. .

[0264] Calculate the feature vector v of the sample to be tested test With each sample feature vector in the support set Similarity score between :

[0265] ;

[0266] By comparing all similarity scores, the defect category of the support set sample corresponding to the highest score is determined as the final defect type identification result of the solar cell image under test. :

[0267] ;

[0268] ;

[0269] In the formula, Let represent the feature extraction mapping function performed by the subnetwork in the trained Siamese network, j represent the index value of the sample in the support set, and M represent the total number of samples in the support set.

[0270] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A solar cell defect detection method combining multi-scale feature enhancement and staged metric learning, characterized in that, The method includes the following steps: S1. An improved deep feature extraction network based on the ResNet-50 architecture is constructed, and a deep residual network with multi-scale feature enhancement embedded with a channel attention module and a multi-scale feature fusion module is built by combining pre-trained ResNet-50 architecture weights. S2. The parameters of the deep residual network are progressively optimized in stages by adopting the parameter freezing method, and a twin network structure containing two parallel sub-networks with real-time parameter synchronization is constructed based on the optimized deep residual network. S3. Using the Siamese network structure, high-dimensional deep feature vectors are extracted from the training sample pairs pre-constructed using solar cell electroluminescence images. Based on the dual-metric calculation strategy, dual metrics are calculated using the high-dimensional deep feature vectors to construct the fusion loss function, so as to optimize the Siamese network structure. S4. Using the optimized twin network structure and combined with the pre-built standard sample support set, defect detection is performed on the sample to be tested.

2. The solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to claim 1, characterized in that, The improved deep feature extraction network based on the ResNet-50 architecture, combined with pre-trained ResNet-50 architecture weights, constructs a multi-scale feature enhancement deep residual network embedding a channel attention module and a multi-scale feature fusion module, comprising the following steps: S11. Using a transfer learning strategy, the backbone network of the ResNet-50 architecture is initialized using weights pre-trained on a visualization image dataset, resulting in a preliminary ResNet-50 architecture. S12. Construct a feature enhancement structure consisting of a channel attention module and a multi-scale feature fusion module connected in series, which is used for adaptive calibration of channel feature responses and fusion of multi-scale defect information; S13. Remove the global average pooling layer and classification head at the end of ResNet-50, retaining only the convolutional part as the backbone network. Embed the feature enhancement structure and the preset mapping module into the backbone network. The feature enhancement structure consists of a channel attention module and a multi-scale feature fusion module connected in series and is embedded at the end of the backbone network. The mapping module contains a newly added global average pooling layer and two fully connected layers, which are followed by the feature enhancement structure to obtain a deep residual network with multi-scale feature enhancement.

3. The solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to claim 2, characterized in that, The adaptive calibration channel characteristic response and fused multi-scale defect information include: Obtain the feature map output by the initialized ResNet-50 architecture, and use global average pooling to obtain global information for each feature channel; Based on the global information of each feature channel, the attention weight of each feature channel is calculated, and the attention weight of each feature channel is multiplied with the feature map output by the initialized ResNet-50 architecture to obtain the calibrated feature map. The calibrated feature map is input into the multi-scale feature fusion module. The parallel branching structure of the multi-scale feature fusion module is used to perform feature extraction and feature fusion to obtain an enhanced feature map that is robust to scale changes.

4. The solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to claim 1, characterized in that, The method of progressively optimizing the parameters of the deep residual network in stages using parameter freezing, and constructing a twin network structure containing two parallel sub-networks with real-time parameter synchronization based on the optimized deep residual network, includes the following steps: S21. An adaptive channel-aware initialization method based on Kaiming normal distribution is adopted to initialize the parameters of the channel attention module, the multi-scale feature fusion module, and the fully connected layer in the embedding layer of the deep residual network with multi-scale feature enhancement, so as to obtain the initialized deep residual network. S22. Freeze all parameters of the backbone network of the initialized deep residual network, and unfreeze the parameters of the channel attention module, the multi-scale feature fusion module and the embedding layer. S23. Using the preset solar cell defect data, drive the channel attention module, multi-scale feature fusion module and embedding layer to learn parameter representations suitable for defect feature enhancement and measurement, and obtain the learned deep residual network. S24. Unfreeze the learned deep residual network and perform end-to-end joint training on the unfrozen deep residual network using a preset learning rate to obtain an optimized deep residual network. S25. Using the optimized deep residual network as a weight-sharing subnetwork, construct a twin network architecture containing two parallel subnetworks with real-time parameter synchronization.

5. The solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to claim 1, characterized in that, The process of using a Siamese network structure to extract high-dimensional deep feature vectors from training sample pairs pre-constructed using electroluminescence images of solar cells, and then using a dual-metric calculation strategy to calculate dual metrics for constructing a fusion loss function based on the high-dimensional deep feature vectors to optimize the Siamese network structure includes the following steps: S31. Collect images of the electroluminescence of solar cells, perform data optimization processing, and combine a class-balanced weighted sampling strategy to construct training sample pairs for metric learning. S32. Input the training sample pairs into the Siamese network structure, extract features through the Siamese network structure, and output two sets of high-dimensional deep feature vectors; S33. Using a dual-metric collaborative mechanism, the cosine similarity and Euclidean distance between two sets of high-dimensional deep feature vectors are calculated to obtain the dual-metric results. S34. Calculate the weighted fusion loss function based on the dual metric results, and optimize the network parameters of the Siamese network structure through the backpropagation algorithm to obtain the optimized Siamese network structure.

6. The solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to claim 5, characterized in that, The process of collecting electroluminescence images from solar cells, performing data optimization processing, and constructing training sample pairs for metric learning using a class-balanced weighted sampling strategy includes the following steps: S311. The LANCZOS resampling algorithm is used to normalize the size of the collected solar cell electroluminescence images to obtain normalized images. S312. Using the mean and standard deviation of the visualized image dataset, the normalized image is pixel-standardized to obtain a standardized image. S313. Perform channel expansion and data augmentation on the standardized image to obtain the training dataset; S314. Traverse the training dataset, adopt a class-balanced weighted sampling strategy to generate positive and negative sample pairs, and integrate the positive and negative sample pairs to obtain training sample pairs for metric learning.

7. The solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to claim 6, characterized in that, The process of traversing the training dataset, employing a class-balanced weighted sampling strategy to generate positive and negative sample pairs, and then integrating these pairs to obtain training sample pairs for metric learning includes the following steps: S3141. By traversing the training dataset, count the number of samples for each defect category and calculate the sampling weight for each category; S3142. Based on sampling weights and biased negative sampling, a random number generation method is used to generate positive and negative sample pairs until a preset batch size of positive and negative sample pairs is generated. S3143. By integrating sample pairs, training sample pairs for metric learning are obtained.

8. The solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to claim 5, characterized in that, The method of employing a dual-metric collaborative mechanism to calculate the cosine similarity and Euclidean distance between two sets of high-dimensional deep feature vectors and obtain the dual-metric result includes the following steps: S331. The cosine similarity and Euclidean distance between two sets of high-dimensional deep feature vectors, respectively; S332. Transform the cosine similarity into a preset interval using a linear mapping function to obtain the similarity score; S333. Use the Euclidean distance and similarity score between the two sets of high-dimensional deep feature vectors as the dual measurement results.

9. The solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to claim 5, characterized in that, The process of calculating the weighted fusion loss function based on the dual-metric results and optimizing the network parameters of the Siamese network structure using the backpropagation algorithm to obtain the optimized Siamese network structure includes the following steps: S341. Calculate the binary cross-entropy loss based on the similarity score in the dual-measure results; S342. Calculate the contrast loss based on the Euclidean distance in the dual-metric results; S343. Based on the preset weight coefficients, a weighted fusion loss function is constructed by combining the binary cross-entropy loss and the contrast loss. S344. Based on the weighted fusion loss function, and combined with the iterative optimization of all parameters of the Siamese network structure through backpropagation until the preset iteration termination condition is reached, the optimized Siamese network structure is obtained.

10. The solar cell defect detection method combining multi-scale feature enhancement and staged metric learning according to claim 1, characterized in that, The process of using an optimized twin network structure, combined with a pre-built standard sample support set, to perform defect detection on the sample to be tested includes the following steps: S41. Pre-build a standard sample library covering all categories of defects to be detected as a support set; S42. Input the image of the solar cell to be tested into one of the subnetworks in the optimized twin network structure, and extract the high-dimensional deep feature vector of the image of the solar cell to be tested. S43. Input each standard sample image in the support set into another sub-network in the Siamese network structure and extract the feature vector of each standard sample image. S44. Calculate the similarity score between the high-dimensional depth feature vector of the solar cell image to be tested and the feature vector of each standard sample image, and select the defect category to which the standard sample image in the support set with the highest similarity score belongs. S45. The selected defect category is used as the final defect type identification result of the solar cell image under test.

Images