A fully automatic and rapid detection and segmentation method for photovoltaic cell defects

Abstract

The present invention discloses a fully automatic and fast detection and segmentation method for photovoltaic cell defects. The present invention adopts a lightweight SSA-ViT image encoder to replace the heavyweight ViT image encoder in SAM, and adopts a separable self-attention mechanism (SSA) to replace a multi-head attention mechanism (MHA), which greatly reduces the complexity of the attention mechanism and the amount of calculation, and reduces the inference time of a single-frame image from an average of 4099ms to an average of 11.7ms (350 times), thereby greatly improving the detection efficiency of the model. Moreover, the prediction result of the YOLOv10 model is used as a new segmentation paradigm for YoMo-SAM prompt information, and the human-computer interaction segmentation paradigm of the original SAM is abandoned. Under the premise of ensuring the segmentation accuracy, the fully automatic target detection and segmentation tasks are truly realized; at the same time, the task generalization of zero samples is realized, and the problem of data scarcity is overcome while avoiding manual segmentation, thereby improving the detection and segmentation accuracy of the model.

Description

Technical Field

[0001] The invention belongs to the field of detection and segmentation methods, and in particular, relates to a fully automatic and rapid detection and segmentation method for photovoltaic cell defects. Background Art

[0002] Climate change is a common challenge facing human survival and development, and has received great attention from people and governments around the world. Renewable energy such as wind power, photovoltaic power generation, and hydropower are natural green energy. my country's photovoltaic power generation has achieved the ambitious goals of carbon peak and carbon neutrality at an unprecedented speed. According to the my country Renewable Energy Report for the first half of 2024 released by the International Energy Agency, by the end of 2023, my country's cumulative installed capacity of renewable energy will reach 1.517 billion kilowatts, accounting for more than 50% of the global renewable energy power generation installed capacity. In particular, wind power and photovoltaic power generation added about 303 million kilowatts of installed capacity last year, exceeding the total installed capacity of new renewable energy in other countries in the world. The data shows that my country has become an indispensable backbone force in the development of global renewable energy. With the large-scale application of photovoltaic power generation, how to ensure the stability and sustainable development of photovoltaic power supply has become a top priority for industry research. Photovoltaic cells (PV), as one of the important components of photovoltaic power generation systems, are affected by various complex factors during production and use, and are prone to defects such as module scratches, breakage, hidden cracks, broken grids, and dirt, which affect the performance and life of the overall battery array. If defects are not discovered in time, it is very likely to cause the power of the entire photovoltaic power supply system to decrease, burying fire safety hazards and causing great losses to the national economy. Accurate defect detection of photovoltaic cells is an important means to ensure the normal operation of photovoltaic power generation systems. Therefore, scholars at home and abroad have carried out extensive research. In recent years, due to its high detection precision and accuracy, photovoltaic cell detection methods based on deep learning have developed rapidly and made important progress.

[0003] Jumaboev et al. first used Feature Pyramid Networks (FPN), U-Net model and DeepLab model to segment and detect defects in photovoltaic cell arrays. Experiments have shown that this method has high detection accuracy and better segmentation effect than other algorithms. Wang et al. proposed an ERDCF-Net (Efficient and Refined Deep Convolutional Feature Networks) method to achieve accurate detection of solar cell cracks under background interference.

[0004] The BPGA-Detector defect detection method integrates the dual-channel feature pyramid into the YOLOv5 method, which improves the accuracy of small target defect detection under complex backgrounds. Using Yolov5 as the model backbone network, the attention module and deformable convolution are combined to extract defect features of different scales and improve the accuracy of defect recognition. Sohail et al. integrated the fusion method AttentionU-Net, U-Net, LinkNet and FPN method together to significantly improve the detection effect of cracks. Otamendi et al. improved the existing method and proposed an end-to-end defect detection framework to realize defect detection, positioning and segmentation of electroluminescence (EL) images.

[0005] SachinMehta et al. proposed a contamination detection method for photovoltaic cells based on convolutional neural networks. The method takes images of photovoltaic components and environmental factors as input to predict power loss, contamination location and type, and introduces a bidirectional perception fusion module to learn defect types. The rapid development of PV has put higher and higher requirements on PV field maintenance and fault diagnosis. However, the parameters of the existing deep learning model-based algorithms are often very large, the performance of the deep model is usually proportional to the number of its parameters, and the hardware resource requirements of the edge or mobile device are very strict, which makes it difficult to meet the deployment requirements on mobile devices. In order to meet the requirements of both accuracy and speed of defect detection in the field of photovoltaic power generation, a fast and lightweight detection method is needed. Based on the above difficulties, Liu et al. from Wuhan University of Science and Technology proposed a feature-enhanced lightweight CNN model to detect small targets with high missed detection rates in photovoltaic panels. The method designed a multi-scale feature extraction module to enhance the recognition ability of the small target prediction layer, thereby achieving the purpose of multi-scale defect prediction. Chen et al. improved the model based on the ShuffleNetV2 network for the type of photovoltaic module fault detection. The improved method has a good effect on defect diagnosis, but cannot accurately identify deformation and small target defects. Shi Ce et al. incorporated the balance factor into the InceptionV3 network, combined with transfer learning to build a photovoltaic cell defect detection model, and introduced a new loss function to correct the detection results. ZHANG et al. designed a lightweight neural framework search network to detect photovoltaic panel defects, using multi-scale features and a knowledge distillation model to complete the migration learning of lightweight networks. In summary, the existing photovoltaic cell defect detection methods generally have the following problems:

[0006] 1) Existing deep models for photovoltaic cell defect detection are mostly medium or large models with many parameters and slow inference speed, which makes it difficult to meet the efficiency requirements of deployment.

[0007] 2) Due to the diversity and complexity of photovoltaic panel defects, existing methods mostly use deep learning models to perform semi-automatic detection of defect features. However, they lack a large number of labeled sample sets, cannot achieve zero-sample task generalization, and have low detection and segmentation accuracy.

[0008] 3) The current model cannot guarantee the detection accuracy for deformation and small target defects, and is prone to missed detection and false detection, lacking generalization and robustness.

[0009] In view of this, the present invention is proposed. Summary of the invention

[0010] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a method for fully automatic and rapid detection and segmentation of photovoltaic cell defects, thereby solving the problems raised in the above-mentioned background technology.

[0011] In order to solve the above technical problems, the basic concept of the technical solution adopted by the present invention is:

[0012] A photovoltaic cell defect fully automatic rapid detection and segmentation method, comprising the following steps:

[0013] Step S1: acquiring a high-resolution image of a photovoltaic cell through an image acquisition device;

[0014] Step S2: inputting the acquired photovoltaic cell image into a lightweight SSA-ViT image encoder, and encoding the image features using SSA-ViT to obtain an image feature vector;

[0015] Step S3: Based on the image feature vector obtained in the above steps, the YOLOv10 model is used to predict and classify the target position;

[0016] Step S4: using the prediction result of the YOLOv10 model as segmentation prompt information, inputting it into the mask decoder of the YoMo-SAM model to generate a defect segmentation mask;

[0017] Step S5: Use separable convolution (SC) to downsample and upsample the segmentation mask to increase the channel dimension, further improve the detection and segmentation accuracy of small targets and deformed targets, and obtain the final optimized segmentation result.

[0018] Optionally, the SSA-ViT image encoder uses a separable self-attention mechanism (SSA) instead of a multi-head attention mechanism (MHA). In the separable self-attention mechanism, first, the image X is input into three branches: input I, keyword K, and value V for processing;

[0019] Input I uses a weight of The linear layer maps each d-dimensional image embedding in image X to a scalar with weight W I As the potential node L in the graph, the inner product operation is used to calculate the linear projection, and the distance between the potential image node L and the input image X is calculated to obtain a k-dimensional vector; then, the Softmax operation is applied to the above k-dimensional vector to generate the context factor SSA reduces the computational cost from Reduce to

[0020] The input image X is passed through the keyword branch K with weights The linear layer is projected into the d-dimensional space to generate the corresponding output Then, using the context factor c s Calculate the context vector As the weighted sum of the input image X, its calculation formula is: Among them, c v Encode all image embeddings in the input image X, c v The context factor encoded in shares label information with all images in X. Finally, the weight of the value branch V Linearly project the input image X into the d-dimensional space and generate the output through the Softmax activation function;

[0021] The context vector c v The convolution operation applied to the image through broadcast multiplication operation finally calculates the feature output Y by the following formula: Where σ represents the activation function, ReLU represents the rectified linear unit, and W o is the output weight.

[0022] Optionally, the steps of inputting the image feature vector output by the above preprocessing into the YOLOv10 model for target position prediction and classification are as follows:

[0023] The separable convolutional attention mechanism (SSA) in the lightweight image encoder SSA-ViT is integrated into the classifier of the YOLOv10 model. Subsequently, separable convolution is used for spatial downsampling, which reduces the computational cost to At the same time, the number of parameters is reduced to

[0024] Based on the above SSA-ViT fused feature vector, the YOLOv10 model performs target detection on the input image. In target detection, a consistency matching metric strategy is adopted to ensure that the prediction results between the two detection heads can achieve accurate matching. The expression is: Among them, p is the classification score, and b represent the predicted target bounding box and the target ground-truth bounding box respectively, s represents the spatial prior information, and α and β are hyperparameters used to balance the influence of semantic prediction and position regression;

[0025] In the dual-label assignment process, the two branches are first initialized the same way to ensure that the predicted classification scores p and the corresponding IoU values ​​generated by the two branches are consistent;

[0026] For each object, calculate its maximum IoU value and denote it as u * , and at the same time, calculate the maximum one-to-many matching score and one-to-one matching scores In this process, the one-to-many branch generates a positive sample set Ω, while the one-to-one branch selects the i-th prediction sample;

[0027] In the prediction and matching stage, m is used o2o,i = m as the metric, the classification target and task alignment losses are calculated, which are expressed by the following formula: Among them, (j∈Ω).

[0028] Optionally, the supervision gap between the two branches in step S2 is calculated by the Wasserstein distance of different classification targets, which is expressed as: With t o2m，i As t increases, the gap between the classification target and the prediction target gradually decreases, that is, when t o2m，i =u * , indicating that the i-th sample has the highest score in Ω, corresponding to the optimal match, which means that m o2o = r·m o2m , α o2o = r·α o2m and β o2o = r·β o2m , which means that m o2o =m Γ o2m Therefore, the best positive sample of a pair of multi-heads corresponds to the best positive sample of a pair of one-heads, and the two branches have achieved consistent optimization.

[0029] Optionally, the YoMo-SAM model includes a backbone network (Backbone), a neck (Neck) and a head (Head). The backbone part uses a lightweight image encoder SSA-ViT as the backbone network to extract deep features of the image. The neck part uses a YOLOv10 model as the neck, which is responsible for processing the feature vector extracted by the backbone network and making a preliminary prediction of the position and category of the target. The head of the YoMo-SAM model contains a segmentation mask branch and a detection hint branch. The segmentation mask branch is used to generate a segmentation mask of the target area, and the detection hint branch is used for the final prediction of the target classification and position regression.

[0030] Optionally, the prediction result of the YOLOv10 model is used as segmentation prompt information and input into the mask decoder of the YoMo-SAM model to generate a defect segmentation mask as follows:

[0031] Perform object detection on the input image through the YOLOv10 model to obtain the location and classification information of the target and input the prediction result generated by the YOLOv10 model as prompt information into the mask decoder of the YoMo-SAM model;

[0032] In the mask decoder, the input prompt information is further processed using deep convolution and point convolution to generate a preliminary segmentation mask. The preliminary generated segmentation mask is optimized and refined by combining the context factor with the convolution operation. Finally, an accurate defect segmentation mask is obtained and output to identify the boundaries of the defective area in the image.

[0033] Optionally, separable convolution (SC) is used to downsample and upsample the segmentation mask to increase the channel dimension and further improve the detection and segmentation accuracy of small targets and deformed targets. The steps of obtaining the final optimized segmentation result are:

[0034] Separable convolution is used to downsample the initially generated segmentation mask. Downsampling reduces the spatial dimension while increasing the channel dimension, thereby extracting more representative features.

[0035] After extracting more representative features, separable convolution is applied again to upsample the segmentation mask to restore its original spatial resolution;

[0036] Through the joint operation of downsampling and upsampling, the optimized segmentation mask can more accurately represent the boundary and shape of the target, thereby obtaining the final optimized segmentation result.

[0037] After adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art. Of course, any product implementing the present invention does not necessarily need to achieve all the advantages described below at the same time:

[0038] The present invention adopts a lightweight SSA-ViT image encoder to replace the heavyweight ViT image encoder in SAM, and adopts a separable self-attention mechanism (SSA) to replace the multi-head attention mechanism (MHA), which greatly reduces the complexity of the attention mechanism and the amount of calculation, and reduces the inference time of a single frame image from an average of 4099ms to an average of 11.7ms (350 times), greatly improving the detection efficiency of the model, and uses the prediction result of the YOLOv10 model as a new segmentation paradigm for YoMo-SAM prompt information, abandoning the original SAM human-computer interaction segmentation paradigm, and truly realizing the fully automatic target detection and segmentation task while ensuring the segmentation accuracy (the average accuracy is 92.7%); at the same time, it realizes the generalization of zero-sample tasks, overcomes the problem of data scarcity while avoiding manual segmentation, and thus improves the detection and segmentation accuracy of the model.

[0039] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described below are only some embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0041] Figure 1 This is the YOLOv10 framework diagram;

[0042] Figure 2 Schematic diagram of SSA-ViT image encoder;

[0043] Figure 3 A diagram of the separable self-attention mechanism (SSA) and the process of solving the context factor in SSA;

[0044] Figure 4 This is the overall framework diagram of the YoMo-SAM model;

[0045] Figure 5 The detection and segmentation results of three defects of photovoltaic cells are shown in Figure 2.

[0046] Figure 6 The comparison chart of detection and segmentation results of three defects;

[0047] Figure 7 It is a comparison curve chart of P, R and F1 values.

[0048] It should be noted that these drawings and textual descriptions are not intended to limit the conceptual scope of the present invention in any way, but are intended to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0049] The present invention will now be described in further detail with reference to the accompanying drawings.

[0050] See also Figure 1-7 As shown, in this embodiment, a method for fully automatic and rapid detection and segmentation of photovoltaic cell defects is provided, comprising the following steps:

[0051] Step S1: acquiring a high-resolution image of a photovoltaic cell through an image acquisition device;

[0052] Step S2: Input the acquired photovoltaic cell image into a lightweight SSA-ViT image encoder ( Figure 2 ), SSA-ViT is used to encode image features to obtain image feature vectors;

[0053] Step S3: Based on the image feature vector obtained in the above steps, the YOLOv10 model is used to predict and classify the target position;

[0054] Step S4: The YOLOv10 model ( Figure 1 ) is used as the segmentation hint information and input into the mask decoder of the YoMo-SAM model to generate a defect segmentation mask;

[0055] Step S5: Use separable convolution (SC) to downsample and upsample the segmentation mask to increase the channel dimension, further improve the detection and segmentation accuracy of small targets and deformed targets, and obtain the final optimized segmentation result.

[0056] In this embodiment, the SSA-ViT image encoder uses a separable self-attention mechanism (SSA) instead of a multi-head attention mechanism (MHA) ( Figure 3 ), in the separable self-attention mechanism, first, the image X is input into the three branches of input I, keyword K, and value V for processing;

[0057] Input I uses a weight of The linear layer maps each d-dimensional image embedding in image X to a scalar with weight W I As the potential node L in the graph, the inner product operation is used to calculate the linear projection, and the distance between the potential image node L and the input image X is calculated to obtain a k-dimensional vector; then, the Softmax operation is applied to the above k-dimensional vector to generate the context factor SSA reduces the computational cost from Reduce to

[0058] The input image X is passed through the keyword branch K with weights The linear layer is projected into the d-dimensional space to generate the corresponding output Then, using the context factor c s Calculate the context vector As the weighted sum of the input image X, its calculation formula is: Among them, c v Encode all image embeddings in the input image X, c v The context factor encoded in shares label information with all images in X. Finally, the weight of the value branch V Linearly project the input image X into the d-dimensional space and generate the output through the Softmax activation function;

[0059] The context vector c v The convolution operation applied to the image through broadcast multiplication operation finally calculates the feature output Y by the following formula: Where σ represents the activation function, ReLU represents the rectified linear unit, and W o is the output weight.

[0060] In this embodiment, the steps of inputting the image feature vector outputted by the above preprocessing into the YOLOv10 model for target position prediction and classification are as follows:

[0061] The separable convolutional attention mechanism (SSA) in the lightweight image encoder SSA-ViT is integrated into the classifier of the YOLOv10 model. Subsequently, separable convolution is used for spatial downsampling, which reduces the computational cost to At the same time, the number of parameters is reduced to

[0062] Based on the above SSA-ViT fused feature vector, the YOLOv10 model performs target detection on the input image. In target detection, a consistency matching metric strategy is adopted to ensure that the prediction results between the two detection heads can achieve accurate matching. The expression is: Among them, p is the classification score, and b represent the predicted target bounding box and the target ground-truth bounding box respectively, s represents the spatial prior information, and α and β are hyperparameters used to balance the influence of semantic prediction and position regression;

[0063] In the dual-label assignment process, the two branches are first initialized the same way to ensure that the predicted classification scores p and the corresponding IoU values ​​generated by the two branches are consistent;

[0064] For each object, calculate its maximum IoU value and denote it as u * , and at the same time, calculate the maximum one-to-many matching score and one-to-one matching scores In this process, the one-to-many branch generates a positive sample set Ω, while the one-to-one branch selects the i-th prediction sample;

[0065] In the prediction and matching stage, m is used o2o,i = m as the metric, the classification target and task alignment losses are calculated, which are expressed by the following formula: Among them, (j∈Ω).

[0066] In this embodiment, the supervision gap between the two branches in step S2 is calculated by the Wasserstein distance of different classification targets, and its expression is: With t o2m，i As t increases, the gap between the classification target and the prediction target gradually decreases, that is, when t o2m，i =u * , indicating that the i-th sample has the highest score in Ω, corresponding to the optimal match, which means that m o2o = r·m o2m , α o2o = r·α o2m and β o2o = r·β o2m , which means that m o2o =m Γ o2m Therefore, the best positive sample of a pair of multi-heads corresponds to the best positive sample of a pair of one-heads, and the two branches have achieved consistent optimization.

[0067] In this embodiment, the YoMo-SAM model ( Figure 4 ) includes a backbone network (Backbone), a neck (Neck) and a head (Head). The backbone part uses the lightweight image encoder SSA-ViT as the backbone network to extract deep features of the image. The neck part uses the YOLOv10 model as the neck, which is responsible for processing the feature vector extracted by the backbone network and making preliminary predictions on the location and category of the target. The head of the YoMo-SAM model contains a segmentation mask branch and a detection hint branch. The segmentation mask branch is used to generate a segmentation mask of the target area, while the detection hint branch is used for the final prediction of target classification and position regression.

[0068] In this embodiment, the prediction result of the YOLOv10 model is used as segmentation prompt information and input into the mask decoder of the YoMo-SAM model to generate a defect segmentation mask as follows:

[0069] Perform object detection on the input image through the YOLOv10 model to obtain the location and classification information of the target and input the prediction result generated by the YOLOv10 model as prompt information into the mask decoder of the YoMo-SAM model;

[0070] In the mask decoder, the input prompt information is further processed using deep convolution and point convolution to generate a preliminary segmentation mask. The preliminary generated segmentation mask is optimized and refined by combining the context factor with the convolution operation, and finally an accurate defect segmentation mask is obtained.

[0071] The defect segmentation mask is output to identify the boundary of the defect area in the image.

[0072] In this embodiment, separable convolution (SC) is used to perform downsampling and upsampling operations on the segmentation mask to increase the channel dimension and further improve the detection and segmentation accuracy of small targets and deformed targets. The steps of obtaining the final optimized segmentation result are:

[0073] Separable convolution is used to downsample the initially generated segmentation mask. Downsampling reduces the spatial dimension while increasing the channel dimension, thereby extracting more representative features;

[0074] After extracting more representative features, separable convolution is applied again to upsample the segmentation mask to restore its original spatial resolution;

[0075] Through the joint operation of downsampling and upsampling, the optimized segmentation mask can more accurately represent the boundary and shape of the target, thereby obtaining the final optimized segmentation result.

[0076] The experimental example introduces the dataset, data enhancement process, evaluation indicators and implementation details used in the experiment of the present invention; then the ablation experiment is introduced to verify the effectiveness of the backbone module, detection module and improved loss function of the proposed innovative part; finally, from the quantitative and qualitative perspectives, the model proposed in the present invention is compared with the most advanced model, proving the superiority of the proposed model.

[0077] The data set used in the experiment of this invention is a public photovoltaic cell data set, with a total of 2400 frames of images, a resolution of 877×945, and Labels software is used for image annotation processing. The total number of labels is 4566. The detection and segmentation algorithm is mainly studied for three defects of photovoltaic cell scratches, broken grids, and dirt. Among them: the training set consists of 1920 frames of images, and the test set consists of 240 frames of images.

[0078] Without increasing the number of images in large quantities, data augmentation randomly selected 800 frames of images from the original data set and performed horizontal flipping, vertical flipping, random rotation {0o, 90o, 180o, 270o} and affine transformation operations.

[0079] The present invention uses five evaluation indicators, namely precision (P), recall (R), F1 value, mean intersection over union (mIoU) and model reasoning speed (FPS), to measure the performance of the model proposed in the present invention.

[0080] Precision, recall and F1 value:

[0081] Precision (P) and recall (R) are used to measure the false alarm rate and false negative rate of the classifier. The higher the precision, the lower the false alarm rate; the higher the recall, the lower the false alarm rate. The F1 value combines the precision and recall into one value, considering both to be equally important. Therefore, the larger the area under the F1 curve, the better the performance of the model. The calculation formula for F1 is

[0082] Average Intersection-Over-Union ratio in segmentation:

[0083] mIoU is one of the commonly used metrics in instance segmentation methods, which is used to calculate the average ratio of the intersection and union of all categories. Let P (Prediction) represent the category prediction value, G (GroundTruth) represent the true value, and k represent the number of categories, then:

[0084] Model parameters and model inference speed:

[0085] Parameters refer to the number of parameters that can be adjusted during model training. It is one of the important factors affecting model performance. The unit is megabytes (M). The more parameters, the larger the model and the more training time required. The inference speed of the model refers to the time the model uses to infer one frame of an image. The unit is milliseconds (ms). The faster the inference speed, the less hardware resources the model occupies and the faster the model detection and segmentation speed.

[0086] The present invention conducts experiments on the MS-COCO dataset and the public photovoltaic cell image dataset, verifying the effectiveness of the proposed model and the superiority of the most advanced method (State-of-theArt, SOTA). Among them, the MS-COCO dataset contains 80 categories and more than 1.5 million object instances. A large number of small targets make it a widely used dataset. The present invention uses train2017 as the training dataset, test2017 as the test set, and val2017 for ablation experiments and visualization experiments. The experimental environment and parameter configuration are shown in Table 1.

[0087] Table 1 Experimental environment configuration and parameters

[0088]

[0089] Ablation experiment:

[0090] In order to verify the effectiveness of the three innovative methods proposed in this paper, this paper conducts ablation experiments on the data set introduced in Section 3.1. When the innovative method is introduced and the model converges, the ablation experiment comparison results are shown in Table 2. Among them, "√" in the table indicates that this method is used for innovation, and SSA-ViT, YAP and SC in the table represent the three improved methods proposed by this paper.

[0091] Table 2. Comparison of ablation experiments

[0092]

[0093]

[0094] 1. The impact of the SSA-ViT image encoder. The present invention uses a lightweight SSA-ViT image encoder to replace the heavyweight ViT image encoder in SAM, and uses a separable self-attention mechanism to replace the multi-head attention mechanism, which can effectively and significantly reduce the complexity of attention and the amount of calculation. From Table 2-rows 2, 5, 6, and 8, it can be seen that when SSA-ViT is used for image encoding, the inference time of the model is reduced from 456ms to 10ms, and the inference speed is also significantly improved (45.6 times).

[0095] 2. The present invention uses the predicted results of the YOLO model as the prompt information of YoMo-SAM, and truly realizes the fully automatic target detection and segmentation task while ensuring the segmentation accuracy (the average accuracy is 92.7%). At the same time, it realizes the generalization of zero-sample tasks, overcomes the problem of data scarcity while avoiding manual segmentation, and thus improves the detection and segmentation accuracy of the model. It can be observed from the 3rd, 5th, 7th and 8th rows of Table 2 that in addition to improving the overall recognition accuracy P of the model (average 88.43%, an increase of 13.3%), the improved module also increases its recall rate (average 90.0%) and F1 value (average 92.7%) by 14.3% and 10.8%, respectively.

[0096] 3. In target position prediction, the present invention uses separable convolution (SC) to replace the CNN convolution in the original model to downsample the image to increase the channel dimension, improve the detection ability of small targets and deformed targets, and improve the robustness and generalization of the model. It can be observed from the 4th, 6th, 7th and 8th rows of Table 2 that the improved module improves the P, R and F1 values ​​by 17.1%, 13.7% and 9.7% respectively.

[0097] In summary, through the comparison of ablation experiments, we can see that when the three innovative solutions SSA-ViT, YAP and SC are introduced in sequence, the P, R and F1 values ​​are greatly improved (by 18.8%, 16.0% and 10.5% respectively), and the reasoning speed is greatly improved (45.6 times). It can be seen that when the algorithm introduces three innovative solutions, the accuracy and speed of model detection and segmentation can be greatly improved, further verifying the feasibility of the solution of the present invention.

[0098] Model performance comparison experiment:

[0099] On the MC-COCO dataset, the segmentation performance of the proposed algorithm is compared with that of the SAM, FastSAM, and YOLOv8-seg models with ViT image encoder, and the detection performance is compared with the YOLOv10 model. The comparative experimental results are shown in Table 3.

[0100] Table 3 Performance comparison of different algorithms

[0101]

[0102] Table 3 is divided into two parts. The first part is the first 3 rows, which are the classic SAM segmentation model and the improved model FastSAM segmentation model. The FastSAM segmentation model is a SAM improved instance segmentation model proposed by the Chinese Academy of Sciences. It uses 1 / 50 of the SA-1B dataset released by SAM for training and achieves performance comparable to that of the SAM method at a speed 50 times faster. Yolov8n-seg refers to the use of Yolov8's nano model for instance segmentation tasks;

[0103] Yolov10n refers to the nano-model of Yolov10 for object detection tasks. Table 3 gives a quantitative comparison with the most advanced method (SOTA) on the MS-COCO dataset.

[0104] The results show that the model proposed in this paper outperforms SOTA in terms of accuracy and speed. In general, the fully automatic and lightweight YoMo-SAM model proposed in this paper has an accuracy of 92.7% on the test set, far exceeding other models. At the same time, the model proposed in this paper has smaller parameters and requires fewer hardware resources than other classic large models, making it easier to deploy on actual edge terminals or mobile devices.

[0105] Visualization results and analysis:

[0106] On a public photovoltaic cell image dataset, a comparative experiment was conducted between the Yolov10 model and the model proposed in the present invention, and the performance of the two models was compared and analyzed through curves and detection and segmentation visualization results of test images.

[0107] Visualization and analysis of photovoltaic cell defect detection and segmentation results

[0108] In order to verify the effectiveness of the improved algorithm for photovoltaic cell defect detection, the present invention visualizes the detection and segmentation images of photovoltaic cell scratches, broken grids, and dirt to verify the performance of the algorithm proposed in the present invention. Figure 5 They are the visualization results of several defect detection and segmentation.

[0109] from Figure 5 Analyzing the visualization results, the model proposed in the present invention has a high accuracy rate in identifying photovoltaic cell defects, with an average of more than 93%, and is insensitive to scale changes and deformations, showing extremely high detection and segmentation performance.

[0110] Comparative experimental visualization results and analysis:

[0111] Figure 6 The figure is a comparison of the detection and segmentation results of three types of defects in photovoltaic cells, where the first column is the original image, the second column is the detection result using YOLOV10, and the third column is the detection and segmentation result using the YoMo-SAM model proposed in the present invention. It can be seen from the figure that the detection confidence of the algorithm of the present invention is higher and the recognition effect of defects is better.

[0112] from Figure 6 From the visual comparison results, compared with other classic models, the proposed model can not only accurately detect segmentation defects, but also reduce the amount of calculation, and at the same time achieve higher performance under a relatively lightweight architecture, which proves the robustness and generalization of the proposed model.

[0113] Comparison of algorithm precision, recall rate and F1 value curves

[0114] Figure 7 It is a curve chart comparing the three indicators of precision, recall rate and FI value of the model proposed in the present invention and the YOLOv10 model in the MS-COCO public dataset.

[0115] The results show that the proposed model exhibits better defect recognition capabilities while balancing accuracy and speed. It can be seen that in the early stage of network training, compared with the YOLOv10 algorithm, the proposed model has higher accuracy and faster improvement speed under the same number of training times, and the model training is more stable. At the same time, in the later stage of network training, when the model reaches convergence, the number of training iterations of the proposed algorithm is significantly less than that of the original YOLOv10 algorithm, and the proposed algorithm accelerates the convergence speed of the model.

[0116] The present invention is not limited to the above-mentioned embodiments. Anyone should be aware that any structural changes made under the enlightenment of the present invention, and any technical solutions that are the same or similar to the present invention, fall within the protection scope of the present invention. The technology, shape, and structural parts not described in detail in the present invention are all well-known technologies.

Claims

1. A method for automatic and rapid detection and segmentation of photovoltaic cell defects, characterized in that: The following steps are involved: Step S1: acquiring a high-resolution image of a photovoltaic cell through an image acquisition device; Step S2: inputting the acquired photovoltaic cell image into a lightweight SSA-ViT image encoder, and encoding the image features using SSA-ViT to obtain an image feature vector; Step S3: Based on the image feature vector obtained in the above steps, the YOLOv10 model is used to predict and classify the target position; Step S4: using the prediction result of the YOLOv10 model as segmentation prompt information, inputting it into the mask decoder of the YoMo-SAM model to generate a defect segmentation mask; Step S5: using separable convolution (SC) to downsample and upsample the segmentation mask to increase the channel dimension, further improve the detection and segmentation accuracy of small objects and deformed objects, and obtain the final optimized segmentation result; The SSA-ViT image encoder uses the separable self-attention mechanism (SSA) instead of the multi-head attention mechanism (MHA). In the separable self-attention mechanism, first, the image Input to Input 、Keywords ,value Processing is done in three branches; enter Use a weight of The linear layer maps each d-dimensional image embedding in image X to a scalar, with weights As a potential node in the graph , and use the inner product operation to calculate the linear projection and calculate the potential image nodes and the input image The distance between dimensional vector; then, the Softmax operation is applied to the above k-dimensional vector to generate the context factor , SSA reduces the computational cost from Reduce to ; The input image X is passed through the keyword branch K with weights The linear layer is projected onto dimensional space, generating the corresponding output , and then use the context factor Calculate the context vector , as the input image The weighted sum of is calculated as: ,in, For the input image All images in the image embedding are encoded, The context factor encoded in shares label information with all images in X. Finally, the value branch Weight Linearly project the input image X onto dimensional space, and generates output through the Softmax activation function; The context vector The convolution operation applied to the image through broadcast multiplication operation finally calculates the feature output by the following formula : ,in, represents the activation function, ReLU represents the rectified linear unit, is the output weight.

2. A photovoltaic cell defect fully automatic rapid detection and segmentation method according to claim 1, characterized in that: Based on the image feature vector output by the above preprocessing, the steps of inputting it into the YOLOv10 model for target location prediction and classification are as follows: The separable convolutional attention mechanism (SSA) in the lightweight image encoder SSA-ViT is integrated into the classifier of the YOLOv10 model. Subsequently, separable convolution is used for spatial downsampling, which reduces the computational cost to , while reducing the number of parameters to ; Based on the above SSA-ViT fused feature vector, the YOLOv10 model performs target detection on the input image. In target detection, a consistency matching metric strategy is adopted to ensure that the prediction results between the two detection heads can achieve accurate matching. The expression is: ,in, is the classification score, and Represent the predicted target bounding box and the target true value bounding box respectively, represents spatial prior information, and is a hyperparameter used to balance the impact of semantic prediction and position regression; In the dual-label assignment process, the two branches are first initialized identically to ensure that the predicted classification scores generated by the two branches are Keep consistent with the corresponding IoU value; For each object, its maximum IoU value is calculated and expressed as , and at the same time, calculate the maximum one-to-many matching score and one-to-one matching scores In this process, one-to-many branches generate a set of positive samples , and the one-to-one branch selects prediction samples; In the prediction matching stage, use As metrics, the classification objective and task alignment losses are calculated, which are expressed by the following formulas: , ,in, .

3. The photovoltaic cell defect fully automatic rapid detection and segmentation method according to claim 1, characterized in that: The supervision gap between the two branches in step S2 is calculated by the Wasserstein distance of different classification targets, which is expressed as: ,along with As increases, the gap between the classification target and the prediction target gradually decreases, that is, when When Samples in The highest score in corresponds to the best match, which means , and , which means ， Therefore, the best positive sample of a pair of multi-heads corresponds to the best positive sample of a pair of one-heads, and the two branches achieve consistent optimization.

4. A photovoltaic cell defect fully automatic rapid detection and segmentation method according to claim 1, characterized in that: The YoMo-SAM model includes a backbone network (Backbone), a neck (Neck) and a head (Head). The backbone part uses the lightweight image encoder SSA-ViT as the backbone network to extract deep features of the image. The neck part uses the YOLOv10 model as the neck, which is responsible for processing the feature vector extracted by the backbone network and making a preliminary prediction of the location and category of the target. The head of the YoMo-SAM model contains a segmentation mask branch and a detection hint branch. The segmentation mask branch is used to generate a segmentation mask of the target area, while the detection hint branch is used for the final prediction of the target classification and position regression.

5. The photovoltaic cell defect fully automatic rapid detection and segmentation method according to claim 1, characterized in that: The prediction results of the YOLOv10 model are used as segmentation hint information and input into the mask decoder of the YoMo-SAM model to generate defect segmentation masks as follows: Perform object detection on the input image through the YOLOv10 model to obtain the location and classification information of the target and input the prediction result generated by the YOLOv10 model as prompt information into the mask decoder of the YoMo-SAM model; In the mask decoder, the input prompt information is further processed using deep convolution and point convolution to generate a preliminary segmentation mask. The preliminary generated segmentation mask is optimized and refined by combining the context factor with the convolution operation. Finally, an accurate defect segmentation mask is obtained and output to identify the boundaries of the defective area in the image.

6. A photovoltaic cell defect fully automatic rapid detection and segmentation method according to claim 1, characterized in that: Separable convolution (SC) is used to downsample and upsample the segmentation mask to increase the channel dimension and further improve the detection and segmentation accuracy of small objects and deformed objects. The steps to obtain the final optimized segmentation result are: Separable convolution is used to downsample the initially generated segmentation mask. Downsampling reduces the spatial dimension while increasing the channel dimension, thereby extracting more representative features. After extracting more representative features, separable convolution is applied again to upsample the segmentation mask to restore its original spatial resolution; Through the joint operation of downsampling and upsampling, the optimized segmentation mask can more accurately represent the boundary and shape of the target, thereby obtaining the final optimized segmentation result.

Images