A semi-supervised microcrack detection method based on SSMD

Abstract

The present application relates to a semi-supervised microcrack detection method based on SSMD, including: constructing a student model and a teacher model; conducting supervised training on the student model based on a labeled photovoltaic cell image set; initializing the parameters of the teacher model based on the parameters of the student model after supervised training; the initialized teacher model generates pseudo labels and microcrack images with pseudo labels based on an unlabeled photovoltaic cell image set, and a plurality of microcrack images with pseudo labels constitute a microcrack image data set; the student model after supervised training is unsupervisedly trained based on the pseudo labels, and the teacher model is updated based on the feature information output by the student model in supervised training; the student model after unsupervised training is semi-supervisedly trained based on the microcrack data set after data enhancement; the student model is optimized based on the total loss function, the photovoltaic cell image to be detected is input into the optimized student model, and the microcrack detection result is output. The method realizes accurate microcrack detection.

Description

Technical Field

[0001] The present application relates to the technical field of microcrack detection, and in particular to a semi-supervised microcrack detection method based on SSMD. Background Art

[0002] During the manufacturing process of photovoltaic cells, microcracks may occur inside the cell due to mechanical collision, thermal expansion and contraction, etc. The existence of microcracks will have a significant impact on the economic benefits of the entire solar power generation project. In view of this, it is particularly important to regularly detect microcracks in photovoltaic cells.

[0003] Early microcrack detection mainly relied on manual inspection, which was time-consuming and laborious and easily influenced by subjective judgment. With the rapid development of image processing technology, especially the breakthrough in deep learning, automatic microcrack detection has become possible. By training a deep neural network model, it is possible to quickly scan and analyze the surface of photovoltaic cells and identify microcracks that are difficult to detect with the naked eye. This method not only greatly improves the detection efficiency and reduces labor costs, but also continuously optimizes the detection accuracy through the continuous iteration of machine learning, providing strong technical support for product quality control in the photovoltaic industry. However, deep learning methods rely on data and annotations. Therefore, a large amount of manual annotation results are required to train the model, which makes it difficult for deep learning methods to adapt to the microcrack detection of new photovoltaic electroluminescent images; in addition, photovoltaic cells are divided into two types according to core components: single crystal cells and polycrystalline cells. Due to the interference of gratings and impurities, it is very difficult to accurately mark microcracks, and it is even more difficult to obtain enough labeled data for training models. Summary of the invention

[0004] Based on this, it is necessary to provide a semi-supervised microcrack detection method based on SSMD, which includes:

[0005] S1: Construct a student model and a teacher model. Both the student model and the teacher model are based on the Faster-R-CNN architecture, are equipped with a feature pyramid network, and use the ImageNet pre-trained ResNet50 as the backbone network;

[0006] S2: obtaining a set of annotated photovoltaic cell images, and performing supervised training on the student model based on the set of annotated photovoltaic cell images;

[0007] S3: Initialize the parameters of the teacher model based on the parameters of the student model after supervised training;

[0008] S4: Obtain an unlabeled photovoltaic cell image set, and the initialized teacher model generates pseudo labels and microcrack images with pseudo labels based on the unlabeled photovoltaic cell image set. Multiple microcrack images with pseudo labels constitute a microcrack image dataset;

[0009] S5: the student model after supervised training is subjected to unsupervised training based on pseudo labels, and the teacher model is updated based on feature information output by the student model in supervised training;

[0010] S6: combining the microcrack image dataset and the annotated photovoltaic cell image dataset to obtain a microcrack dataset, and performing semi-supervised training on the student model after unsupervised training based on the microcrack dataset after data enhancement;

[0011] S7: Combine the loss functions of supervised training, unsupervised training and semi-supervised training to calculate a total loss function; optimize the student model based on the total loss function, input the photovoltaic cell image to be detected into the optimized student model, and output the microcrack detection result.

[0012] Beneficial effects: This method uses a small amount of labeled data and makes full use of a large amount of available but unlabeled data in industrial scenarios to achieve accurate microcrack detection; SSMD is based on a teacher-student framework, in which the teacher model generates pseudo labels and the student model uses these pseudo information for training; the classification weights of the teacher model are updated using the microcrack features output by the student model, and the information obtained by the student model is transferred to the teacher model; in addition, a data enhancement method is provided to significantly improve the accuracy and efficiency of microcrack detection in photovoltaic cells. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 This is a flow chart of a semi-supervised microcrack detection method based on SSMD in an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0016] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0017] like Figure 1 As shown, this embodiment provides a semi-supervised microcrack detection method based on SSMD, the method comprising:

[0018] S1: Construct a student model and a teacher model. Both the student model and the teacher model are Faster-R-CNN architectures, are equipped with feature pyramid networks, and use ImageNet pre-trained ResNet50 as the backbone network.

[0019] Photovoltaic cell image set with annotations during training and unlabeled photovoltaic cell image sets ,in, y Represents the real information of the labeled image, including the details of the bounding box. represents the total number of annotated images, Indicates the number of unlabeled images.

[0020] The present invention first needs to initialize the model. Both the teacher model and the student model are initialized to the same Faster-R-CNN architecture, equipped with Feature Pyramid Network (FPN), and use ImageNet pre-trained ResNet50 as the backbone network. Faster-R-CNN is selected as the basic model, which is one of the more advanced target detection models. It uses Region Proposal Network (RPN) to generate candidate regions, and then classifies and regresses these regions into bounding boxes.

[0021] Faster-R-CNN can be divided into two main parts: shared convolutional layers and region proposal networks (RPN). Shared convolutional layers usually use pre-trained deep convolutional neural networks (such as VGG, ResNet, etc.) as the backbone network, responsible for extracting image features; RPN is one of the key innovations of Faster R-CNN, which uses a sliding window method to generate candidate regions (Region Proposals) on anchor boxes (Anchors) of different scales and aspect ratios, replacing the previous time-consuming Selective Search method.

[0022] In order to process targets of different scales, the present invention combines Faster-R-CNN with FPN, which can capture crack features of different scales. No matter how large or small the crack is, it can be effectively detected, which can further improve the accuracy and robustness of detection. The main purpose of FPN is to solve the problem of scale changes in target detection, that is, to effectively detect targets at different scales. Traditional target detection models often only use the deepest feature maps. Although these features are rich in semantic information, their position information is relatively vague, and their detection ability for small targets is weak. FPN constructs a pyramid structure to fuse feature maps of different levels, so that the model can effectively detect targets of different scales. In FPN, starting from the top feature map, each layer of feature maps After 1×1 convolution, it becomes , and then upsampled and combined with the feature map of the next level Perform lateral connection. This process can be expressed by the following formula:

[0023] ;

[0024] ;

[0025] in, represents a 1×1 convolution operation, Represents a bilinear interpolation upsampling operation. arrive The process is repeated until all required feature layers have been processed.

[0026] In this embodiment, ResNet50 is used as the backbone network, and this network is pre-trained on the ImageNet dataset, which means that the network has learned rich visual features, which is helpful for subsequent transfer learning. The last step of the initialization phase is parameter synchronization. Before starting semi-supervised learning, the parameters of the teacher model and the student model are exactly the same. This step is to ensure that the two have the same initial state so that the subsequent semi-supervised learning process can proceed smoothly.

[0027] S2: Obtain a set of labeled photovoltaic cell images, and perform supervised training on the student model based on the set of labeled photovoltaic cell images.

[0028] The supervised training phase is a crucial step that lays the foundation for subsequent semi-supervised learning and microcrack detection. This phase involves training the student model using an annotated dataset so that the model can learn basic classification and localization capabilities.

[0029] First, we need a set of annotated photovoltaic cell images. The image set contains images of photovoltaic cells, where the images have been annotated, that is, the location and category of microcracks in the images are known. The data set is divided into training set, validation set and test set. The training set is used for model training, the validation set is used to adjust hyperparameters, and the test set is used to evaluate the final performance of the model. Secondly, image preprocessing is required. Before entering the model, the image is usually preprocessed by standardization and normalization to optimize the training process.

[0030] Specifically, the supervised training process includes:

[0031] Step 1: Input the annotated photovoltaic cell image set into the student model, and the backbone network and feature pyramid network in the student model perform feature extraction to generate multi-scale feature maps;

[0032] Step 2: The region proposal network in the student model generates candidate regions based on the feature map;

[0033] Step 3: Use RoI Pooling to crop the features of each candidate region from the feature map and resize it to a fixed size;

[0034] Step 4: Classify and regress the features of each adjusted candidate region through the fully connected layer in the student model to determine the category and location of the microcracks;

[0035] Step 5: Calculate the classification loss based on the predicted microcrack category and the category in the real annotation, calculate the positioning loss based on the predicted microcrack position and the position in the real annotation, and calculate the loss function of supervised learning based on the classification loss and positioning loss;

[0036] Step 6: Back-propagate the loss function of the supervised learning into the learning model, and update the parameters of the student model by gradient descent.

[0037] In this embodiment, the gradient descent method includes Adam or SGD.

[0038] During the supervised training phase, it may be necessary to adjust hyperparameters such as learning rate, batch size, and optimizer parameters to optimize the model training process. At the same time, the validation set obtained during data preprocessing is used to monitor the performance of the model during training to avoid overfitting, and to adjust hyperparameters based on the performance on the validation set. When the performance of the model on the validation set reaches a satisfactory level or the training reaches a predetermined number of iterations, the supervised training ends. At this point, the student model already has basic classification and positioning capabilities, and can transition to the unsupervised training and semi-supervised training stages, using unlabeled photovoltaic cell image sets to further optimize the model.

[0039] S3: Initialize the parameters of the teacher model based on the parameters of the student model after supervised training.

[0040] The teacher model is initialized using the student model obtained through supervised training. This means that the teacher model and the student model have the same weight parameters at the beginning. Such initialization helps ensure that both models have a certain degree of recognition ability.

[0041] S4: Obtain an unlabeled photovoltaic cell image set. The initialized teacher model generates pseudo labels and microcrack images with pseudo labels based on the unlabeled photovoltaic cell image set. Multiple microcrack images with pseudo labels constitute a microcrack image dataset.

[0042] In this embodiment, the pseudo-label includes a bounding box and a category, and the pseudo-label is screened by a non-maximum suppression method to reduce redundant bounding boxes; a plurality of microcrack images with the screened pseudo-labels are combined into a microcrack image dataset. The non-maximum suppression method merges as many overlapping pseudo-label bounding boxes as possible to reduce redundant bounding boxes and retain the prediction results that are most likely to represent microcracks.

[0043] S5: The student model after supervised training is subjected to unsupervised training based on pseudo labels, and the teacher model is updated based on feature information output by the student model in supervised training.

[0044] Specifically, the unsupervised training process includes:

[0045] The student model after supervised training simulates the classification and regression process of the supervised detector according to the category and bounding box provided by the pseudo-label, and calculates the loss function of unsupervised training based on the pseudo-label, unlabeled photovoltaic cell image set; the optimization direction of the student model after supervised training is guided by the loss function of unsupervised training.

[0046] Furthermore, in the unsupervised training process, this embodiment provides a feature memory bank for storing the latest feature information output by the student model, which is used to gradually update the teacher model. As the training process progresses, the offline training method prevents the new information learned by the student model from being passed to the teacher model, resulting in a large amount of information being lost during the training process. Previous semi-supervised target detection methods directly use the parameters of the student model to update the teacher model through the exponential moving average (EMA), which may introduce noise to the teacher model due to unstable training.

[0047] Therefore, a method of using Memory Bank to store the output features of the student model for a long time is proposed, and this feature Memory Bank is used to gradually update the teacher model. Using this method, the teacher model not only retains the prototype information in the original classification, but also obtains the latest feature information of the student model. Long-term storage can weaken the impact of unstable training. The information stored in Memory Bank is the feature of the student model, and the updated information is the prototype in the teacher model.

[0048] During the prototype training of the teacher model, the final step involves classification and regression of the output bounding box. In the classification step, the commonly used classifier is the softmax classifier. The result of the softmax operation calculated on the output of the classification layer (i.e., the last fully connected layer FC) is as follows:

[0049] ;

[0050] in represents the weight of the classification layer, and yes No. Column, representing the corresponding The weight of the class. represents the value of the output feature, The corresponding The number of categories is determined by In this task, , which means and Corresponding to the microcrack category and the background respectively. Following the concept of prototype learning in the face recognition task, we define: is the prototype of all categories, where Representative The prototype of the class. In other words, All Therefore, The calculated result represents the similarity between the sample output feature and the prototype of each category. Since the goal of softmax loss is to make the corresponding Classification score This means that the similarity between the output features of the sample and the prototype of the true category is getting higher and higher during the training process. Therefore, in most trained object detection models, the prototype trained by softmax loss can well summarize the information of each category.

[0051] The updating of the teacher model based on the feature information output by the student model in the supervised training comprises:

[0052] Step 1: Construct a feature memory bank to store the feature information output by the student model in supervised training for a long time.

[0053] Step 2: Update the weight of the last fully connected layer in the teacher model based on the feature information; in MemoryBank, is defined as a matrix where each column Represents the average value of the features of the corresponding category. Here, only the features calculated by the student model in the supervised learning stage are recorded; the weight update formula is:

[0054] ;

[0055] in, is the updated weight, It is a hyperparameter that controls the extent to which the information in the Memory Bank is utilized.

[0056] Step 3: For the microcrack detection task, there are two categories in classification: background and microcracks. The number of samples in these two categories is very unbalanced. Therefore, after obtaining the updated prototype When , the updated weights need to be normalized, and the output features of the corresponding categories in the fully connected layer that updates the weights need to be normalized to ensure that the entire feature vector set has the same L2 norm. In this way, all features are maintained on a hypersphere and are better aggregated and separated by category. The calculation formula is:

[0057] ;

[0058] ;

[0059] in, and Represent the updated classification layer weights and the output features of the corresponding categories, respectively. For the i The value of the output feature of the class.

[0060] Step 4: The feature information is continuously updated with the supervised training, and steps 2-3 are repeatedly executed as the feature information is updated to update the teacher model in real time.

[0061] In this way, the knowledge learned by the student model from new data can be effectively used to update the teacher model, so that the teacher model can better generate high-quality pseudo labels for unlabeled images.

[0062] In the process of pseudo-label generation, Memory Bank is introduced to use the features output by the student model from labeled images during supervised learning to help the teacher model generate pseudo-labels for unlabeled images. Specifically, the student model is continuously updated during the supervised learning phase, and its output features are stored in Memory Bank. When the teacher model detects unlabeled images and generates pseudo-labels, these features stored in Memory Bank can be used for assistance. This approach helps the teacher model better understand the characteristics of microcracks and improves its ability to generate accurate pseudo-labels for unlabeled images.

[0063] S6: The microcrack image dataset and the annotated photovoltaic cell image dataset are combined to obtain a microcrack dataset, and the student model after unsupervised training is semi-supervised trained based on the microcrack dataset after data enhancement.

[0064] Specifically, this embodiment provides a Crack-copy data enhancement technique, which is a method specifically designed to improve the performance of microcrack detection models. Microcrack detection is a challenging task because microcracks usually appear as small and irregular shapes in images, which makes them difficult to detect. In order to enhance the generalization ability and robustness of the model, the Crack-copy technique is introduced as part of the data enhancement.

[0065] The process of Crack-copy data enhancement includes:

[0066] Step 1: Generate a binary mask for each image in the microcrack dataset, where the crack area is marked as 1 and the rest is marked as 0;

[0067] Step 2: Extract the crack area from the binary mask and copy the crack area. The copy formula is:

[0068] ;

[0069] Among them, C represents the crack area, I represents the image, and M represents the binary mask;

[0070] Step 3: To further enhance the diversity of cracks, replicate the crack instances Deformation operations such as rotation, scaling and translation can be performed. These transformations help simulate the different forms that cracks may present in actual scenes; in this embodiment, the copied crack area is rotated and scaled, and the transformation formula is:

[0071] ;

[0072] in, represents the crack area after transformation; Represents a zoom operation; Represents a rotation operation;

[0073] Step 4: To make the copied cracks look more natural, you can apply color and brightness adjustments. This can be done by adjusting the pixel values ​​to ensure that the cracks blend more naturally with the background. You can adjust the pixel values ​​in the following ways:

[0074] ;

[0075] in, represents the adjusted crack area; Indicates the brightness adjustment coefficient; Indicates the color adjustment coefficient;

[0076] Step 5: Paste the adjusted crack area onto another image in the microcrack dataset. The pasting formula is:

[0077] ;

[0078] in, represents the image after pasting the cracks, represents the location of the crack area, Represents another image from the microcrack dataset.

[0079] In this embodiment, the data enhancement further includes adding Gaussian noise to the crack region to simulate noise interference that may occur in an actual environment, which is expressed as:

[0080] ;

[0081] in, represents the crack area with noise added, represents Gaussian noise.

[0082] This data enhancement method can effectively increase the diversity of cracks in the training dataset, thereby improving the generalization and robustness of the model. After obtaining the microcracks in the unlabeled images through the teacher model, a microcrack database can be established based on these microcracks and the microcracks in the labeled images. According to the Crack-copy method, the microcrack database is further enhanced for the labeled images. Due to the addition of microcracks in the unlabeled images, the microcrack information in the dataset is richer, which helps to improve the effectiveness of microcrack detection. The training process using this dataset belongs to semi-supervised learning.

[0083] S7: Combine the loss functions of supervised training, unsupervised training and semi-supervised training to calculate a total loss function; optimize the student model based on the total loss function, input the photovoltaic cell image to be detected into the optimized student model, and output the microcrack detection result.

[0084] In this embodiment, the expression of the total loss function is:

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] in, L represents the total loss; represents the loss function of supervised training; represents the loss function for unsupervised training; represents the loss function of semi-supervised training; express The corresponding hyperparameters; express The corresponding hyperparameters; express The corresponding hyperparameters; represents the classification loss in supervised training; represents the localization loss in supervised training; represents the classification loss in unsupervised training; represents the localization loss in unsupervised training; represents the classification loss in semi-supervised training; represents the localization loss in semi-supervised training.

[0090] In this embodiment, since the Faster-R-CNN model is used in the three stages of supervised training, unsupervised training, and semi-supervised training, the calculation formula of the loss function is the same. The classification loss in each stage is calculated using the cross entropy loss, and the positioning loss in each stage is calculated using the L1 loss or the IoU loss. The only difference is that the parameters in different stages are different.

[0091] This embodiment shows the calculation formula of the loss function in Faster-R-CNN, which is:

[0092] ;

[0093] in, It is u The predicted probability distribution of candidate regions is It is u The true category labels of the candidate regions, and They are RPN and detection network for theu Bounding box regression predictions and true values ​​for each candidate region. Represents classification loss, usually using cross entropy loss, Represents the positioning loss, usually using smooth L1 loss or IoU loss. is a hyperparameter that balances classification loss and localization loss, is an indicator function that returns 1 if the condition is true, otherwise it returns 0;

[0094] This embodiment also shows the calculation formulas for classification loss and positioning loss.

[0095] The classification loss uses cross entropy loss, which is usually used in classification tasks to measure the difference between the predicted probability distribution and the true label. Taking the supervised training stage as an example, the calculation formula for the classification loss is:

[0096] ;

[0097] in, It is u The predicted probability distribution of candidate regions is It is u The true category labels of the candidate regions, N Indicates the number of candidate regions;

[0098] The positioning loss uses IoU loss or L1 loss. IoU loss is also called Jaccard index, which is used to measure the degree of overlap between two bounding boxes (predicted box and real box). The calculation formula of IoU loss is:

[0099] ;

[0100] ;

[0101] in, Represents the area of ​​the prediction box; Represents the area of ​​the real box; Represents the intersection area of ​​two boxes; represents the union area of ​​two boxes;

[0102] L1 loss is used for regression tasks, especially when the error distribution is asymmetric or there are large outliers. The calculation formula of L1 loss is:

[0103] ;

[0104] Where m represents the number of samples; Represents the true value of the qth sample; Represents the predicted value of the qth sample;

[0105] In Faster R-CNN, L1 loss is often used in the bounding box regression part, which adjusts the position of the candidate box to more accurately match the position of the real object.

[0106] In practical applications, these loss functions may be used in combination, for example, cross entropy loss is used for classification loss, while L1 loss or IoU loss is used for bounding box regression loss. During training, these losses are weighted and combined in some way to form the final total loss function.

[0107] The semi-supervised microcrack detection method based on SSMD provided in this embodiment has the following beneficial effects:

[0108] This method introduces semi-supervised learning, uses a small amount of labeled data, and makes full use of a large amount of acquired but unlabeled data in industrial scenarios to achieve accurate microcrack detection. SSMD is based on a teacher-student framework, in which the teacher model generates pseudo labels and the student model uses these pseudo information for training. The present invention proposes two innovations: Memory Bank and Crack-copy. Memory Bank uses the microcrack features output by the student model to update the classification weights of the teacher model and transfer the information obtained by the student model to the teacher model; Crack-copy is a data enhancement method that uses the irregular shape and position characteristics of microcracks and generates pseudo frames to introduce microcrack information from unlabeled images into labeled data sets. Thereby significantly improving the accuracy and efficiency of microcrack detection in photovoltaic cells.

[0109] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A semi-supervised microcrack detection method based on SSMD, characterized in that: include: S1: Construct a student model and a teacher model. Both the student model and the teacher model are based on the Faster-R-CNN architecture, are equipped with a feature pyramid network, and use the ImageNet pre-trained ResNet50 as the backbone network. S2: obtaining a set of annotated photovoltaic cell images, and performing supervised training on the student model based on the set of annotated photovoltaic cell images; S3: Initialize the parameters of the teacher model based on the parameters of the student model after supervised training; S4: Obtain an unlabeled photovoltaic cell image set, and the initialized teacher model generates pseudo labels and microcrack images with pseudo labels based on the unlabeled photovoltaic cell image set. Multiple microcrack images with pseudo labels constitute a microcrack image dataset; S5: the student model after supervised training is subjected to unsupervised training based on pseudo labels, and the teacher model is updated based on feature information output by the student model in supervised training; S6: combining the microcrack image dataset and the annotated photovoltaic cell image dataset to obtain a microcrack dataset, and performing semi-supervised training on the student model after unsupervised training based on the microcrack dataset after data enhancement; The process of data augmentation includes: Step 1: Generate a binary mask for each image in the microcrack dataset, where the crack area is marked as 1 and the rest is marked as 0; Step 2: extracting the crack region from the binary mask and copying the crack region; Step 3: rotating and scaling the copied crack area; Step 4: adjusting the pixel values ​​in the crack area after transformation; Step 5: Paste the adjusted crack region onto another image in the microcrack dataset; S7: Combine the loss functions of supervised training, unsupervised training and semi-supervised training to calculate a total loss function; optimize the student model based on the total loss function, input the photovoltaic cell image to be detected into the optimized student model, and output the microcrack detection result.

2. The semi-supervised microcrack detection method based on SSMD according to claim 1 is characterized in that: The supervised training process includes: Step 1: Input the annotated photovoltaic cell image set into the student model, and the backbone network and feature pyramid network in the student model perform feature extraction to generate multi-scale feature maps; Step 2: The region proposal network in the student model generates candidate regions based on the feature map; Step 3: Use RoI Pooling to crop the features of each candidate region from the feature map and resize it to a fixed size; Step 4: Classify and regress the features of each adjusted candidate region through the fully connected layer in the student model to determine the category and location of the microcracks; Step 5: Calculate the classification loss based on the predicted microcrack category and the category in the real annotation, calculate the positioning loss based on the predicted microcrack position and the position in the real annotation, and calculate the loss function of supervised learning based on the classification loss and positioning loss; Step 6: Back-propagate the loss function of the supervised learning into the learning model, and update the parameters of the student model by gradient descent.

3. The semi-supervised microcrack detection method based on SSMD according to claim 1 is characterized in that: The pseudo-labels include bounding boxes and categories. The pseudo-labels are screened using a non-maximum suppression method to reduce redundant bounding boxes. Multiple microcrack images with the screened pseudo-labels are combined into a microcrack image data set.

4. The semi-supervised microcrack detection method based on SSMD according to claim 1 is characterized in that: The unsupervised training process includes: The student model after supervised training simulates the classification and regression process of the supervised detector according to the category and bounding box provided by the pseudo-label, and calculates the loss function of unsupervised training based on the pseudo-label, unlabeled photovoltaic cell image set; the optimization direction of the student model after supervised training is guided by the loss function of unsupervised training.

5. The semi-supervised microcrack detection method based on SSMD according to claim 1, characterized in that: The updating of the teacher model based on the feature information output by the student model in the supervised training comprises: Step 1: Construct a feature memory bank to store the feature information output by the student model in supervised training for a long time. Step 2: Update the weight of the last fully connected layer in the teacher model based on the feature information; Step 3: Normalize the updated weights and normalize the output features of the corresponding categories in the fully connected layer with updated weights; Step 4: The feature information is continuously updated with the supervised training, and steps 2-3 are repeatedly executed as the feature information is updated to update the teacher model in real time.

6. The semi-supervised microcrack detection method based on SSMD according to claim 1, characterized in that: The data enhancement also includes adding Gaussian noise on the crack area.

7. The semi-supervised microcrack detection method based on SSMD according to claim 1 is characterized in that: The expression of the total loss function is: ； ； ； ； in, L represents the total loss; represents the loss function of supervised training; represents the loss function for unsupervised training; represents the loss function of semi-supervised training; express The corresponding hyperparameters; express The corresponding hyperparameters; express The corresponding hyperparameters; represents the classification loss in supervised training; represents the localization loss in supervised training; represents the classification loss in unsupervised training; represents the localization loss in unsupervised training; represents the classification loss in semi-supervised training; Represents the positioning loss in semi-supervised training; each classification loss is calculated using the cross entropy loss, and each positioning loss is calculated using the L1 loss or the IoU loss.

Images