Photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation

Abstract

The invention relates to a photovoltaic module defect continuous segmentation method based on an attention mechanism and auxiliary multi-label knowledge distillation, and the method specifically comprises the following steps: setting a model of a previous stage, i.e., a (t-1) stage, as an old model, and setting a model of a current stage, i.e., a t stage, as a new model; according to calculation of attention maps in the new model and the old model, through an attention mechanism based on rapid normalization feature fusion, the injection of the model to fine or difficult-to-detect defect features is enhanced; generating a pseudo label; on the basis of feature knowledge distillation of an attention map and auxiliary multi-class pseudo labels, feature matching is carried out in a shallow layer and a deep layer of the network according to attention map knowledge and auxiliary multi-label knowledge at the same time, so that a new model can learn knowledge about old-class defects in an old model. When the new class of defects occur, the model can be quickly and finely adjusted only based on the newly added new class of defect data, so that the model can effectively segment the hidden crack defects of the old class and the new class at the same time, and continuous segmentation of the photovoltaic hidden crack defects is realized.

Description

Technical Field

[0001] The present invention relates to a continuous learning segmentation method, in particular to a photovoltaic module defect continuous segmentation method based on attention and auxiliary multi-label recognition, belonging to the technical field of continuous learning. Background Art

[0002] Photovoltaic power generation is a clean, environmentally friendly and sustainable form of energy, and has therefore developed rapidly around the world. As a core component of a photovoltaic power generation system, the reliability of photovoltaic modules is directly related to the performance and efficiency of the power generation system. However, during the assembly and use of photovoltaic modules, they are inevitably affected by external forces or the environment, resulting in defects such as hidden cracks, oxidation, and hot spots. These defects may affect its service life and power generation efficiency, and thus have a negative impact on the photovoltaic power generation system. Therefore, it is necessary to detect defects in photovoltaic module cells to ensure the normal operation of photovoltaic modules.

[0003] Recently, some researchers have tried to perform automated and intelligent detection of hidden crack defects based on electroluminescent images of solar cells. However, these methods usually assume that the database already contains all defect categories on site, so they only need to be trained on a database with fixed categories. However, as time goes by, changes in temperature and humidity will lead to an increase in the number of defect types in photovoltaic modules. In addition, a photovoltaic power station may contain hidden crack defects that other photovoltaic power stations do not have. When a new type of hidden crack defect appears, the existing defect detection method needs to retrain the model based on all new and old data, which adds additional time costs. In addition, as the amount of data grows, the required storage space is also continuously expanding, which will cause a huge waste of space resources.

[0004] The existing general-purpose continuous learning methods can process images with obvious texture features and high distinction between background and subject. However, the electroluminescent image texture of the cell is fuzzy and the hidden crack defects are easily confused with background noise such as dust, grid lines and background flocs. If the general-purpose continuous learning technology is directly applied to the field of photovoltaic module defect detection, the model cannot effectively learn new categories and correctly identify old categories. Therefore, a new solution is urgently needed to solve the above technical problems. Summary of the invention

[0005] The present invention aims at the problems existing in the prior art and provides a method for continuous segmentation of photovoltaic module defects based on attention and auxiliary multi-label recognition. When a new category of defects appears, the technical solution can quickly fine-tune the model using only the new category data to effectively segment the old and new categories of hidden crack defects at the same time.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows: a method for continuous segmentation of photovoltaic module defects based on attention and auxiliary multi-label recognition, the method comprising the following steps:

[0007] S1: Set the model of the previous stage, i.e., stage (t-1) as the old model, and the model of the current stage, i.e., stage t, as

[0008] is the new model, and t represents the step of continuous learning;

[0009] S2: Computation of attention maps in the new and old models, via an attention mechanism based on fast normalized feature fusion,

[0010] Enhance the model's attention to subtle or hard-to-detect defect features;

[0011] S3: Generate pseudo labels by fusing the prediction results of the old model with the true values ​​containing only the defects of the new category to obtain pseudo labels;

[0013] S4: Based on the feature knowledge distillation of attention map and auxiliary multi-class pseudo-labels, when new types of defects that are not in the original database appear, the model can be quickly fine-tuned based on the newly added new types of defect data, so that the model can effectively segment old and new types of hidden crack defects at the same time, realizing the continuous segmentation of photovoltaic hidden crack defects;

[0014] Furthermore, in step S1, in the class incremental continuous learning stage, the defect segmentation model will go through the incremental learning step t, by performing knowledge distillation between the old segmentation model in the previous stage (t-1) and the new model in the current stage, so that the new model inherits the old category defect knowledge in the old model, thereby avoiding forgetting the old category defects while learning the new category defects, and realizing the continuous segmentation of hidden crack defects.

[0015] In step S2, the multi-scale feature map tensor F corresponding to the convolutional layer in the old model and the new model is extracted respectively. t-1 and F t . F t-1 and F t Each of them contains N feature maps, where N is the number of scales. Each feature map enhances the attention to subtle or difficult-to-detect defect features through an attention mechanism based on fast normalized feature fusion, thereby improving the accuracy of defect segmentation. The attention mechanism is:

[0016]

[0017] Where x is a single feature map. i or j is the index of each point in the image block R, x i or x j is the eigenvalue of the corresponding point. ris the attention value obtained for the rth image patch. The size of the image patch R is k×k, where k is 2. To avoid numerical instability, ∈ is set to 0.001.

[0018] Furthermore, in step S3, the prediction results of the old model and the true value labels of the new defects are used to realize pseudo labels. If the pixel belongs to the defect in the current true value, the true value label is used directly; if the pixel is both the current background and the old class defect in the old model prediction result, and the probability of belonging to the old class defect is greater than a fixed threshold of 0.55, the prediction result is used directly; the remaining pixels are regarded as background.

[0019] Furthermore, in step S4, the feature map group F of the new model is grouped according to the cosine similarity between the attention maps of the two models. t The feature map group F of the old model t-1 Alignment:

[0020]

[0021] Among them, A(·) is the above attention mechanism, and l indicates the scale from which the feature map comes.

[0022] In step S4, the multi-scale feature map tensor is further processed by a multi-scale auxiliary classifier c l (·) Perform auxiliary multi-label recognition to deeply explore defect category information at various scales in the image:

[0023]

[0024] The weighted BCE loss function L weighted-BCE The weights are calculated according to the pixel ratio of each category of defects in the dataset to alleviate the class imbalance problem.

[0025] In step S4, in addition to auxiliary multi-label recognition, we also use the KL divergence and the following formula to quantify the difference between the feature classification logit of the new and old models at multiple scales, and further align the multi-category information of the new and old models.

[0026]

[0027] Based on the above knowledge distillation, the new type of photovoltaic defect dataset is input into the teacher model and the student model at the same time, and the new model is quickly fine-tuned through the following loss function, so that the new model can achieve continuous segmentation of new types of defects.

[0028]

[0029] The loss between the student model prediction result and the pseudo label; is the characteristic distillation loss, which is calculated as follows:

[0030]

[0031] λ a , l and λ m are their respective balancing hyperparameters, which are 1, 20, and 1 respectively.

[0032] In step S4, the original defect segmentation model is any semantic segmentation model, including one of the Deep Lab series models.

[0033] Compared with the prior art, the present invention has the following advantages: when learning new defect knowledge that is not in the original database, the technical solution performs feature matching based on the attention map knowledge and auxiliary multi-label knowledge at both the shallow and deep layers of the network, inherits the relevant knowledge about old defects in the old model, and can quickly fine-tune the model using only new data to effectively segment old and new hidden crack defects at the same time. Compared with other existing photovoltaic cell defect segmentation technical solutions, this method can meet the needs of rapid iterative updates of defect segmentation models in photovoltaic power station operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the overall process of the embodiment,

[0035] Figure 2 This is the feature knowledge distillation based on attention map and auxiliary multi-class pseudo labels in step 4. DETAILED DESCRIPTION

[0036] In order to deepen the understanding of the present invention, the present embodiment is described in detail below with reference to the accompanying drawings.

[0037] Example 1: See Figure 1 , Figure 2 , a photovoltaic module defect continuous segmentation method based on attention and auxiliary multi-label recognition, comprising the following steps in sequence:

[0038] S1: The model of the previous stage, i.e., stage (t-1), is set as the old model, and the model of the current stage, i.e., stage t, is set as the new model, where t represents the step of continuous learning. In this embodiment, knowledge distillation is performed between the blue old model and the green new model of the current stage so that the new model inherits the old category crack defect knowledge in the old model, thereby avoiding forgetting crack defects while learning new category contact defects, and realizing continuous segmentation of hidden crack defects.

[0039] S2: Calculation of attention maps in the new model and the old model. Through the attention mechanism based on fast normalized feature fusion, the model's attention to subtle or difficult-to-detect defect features is enhanced. In this embodiment, the multi-scale feature map tensor F corresponding to the convolutional layer in the old model and the new model is extracted respectively. t-1 and F t , F t-1 and F t Each feature map contains 4 feature maps. Each feature map enhances the attention to subtle or difficult-to-detect defect features through an attention mechanism based on fast normalized feature fusion, thereby improving the accuracy of defect segmentation. The attention mechanism is:

[0040]

[0041] Where x is a single feature map. i or j is the index of each point in the image block R, x i or x j is the eigenvalue of the corresponding point. r is the attention value obtained for the rth image patch. The size of the image patch R is k×k, where k is 2. To avoid numerical instability, ∈ is set to 0.001.

[0042] S3: Generation of pseudo labels, fusing the prediction results of the old model with the true value containing only new category defects to obtain pseudo labels; in this embodiment, if the pixel belongs to a contact defect in the current true value, the true value label is used directly; if the pixel is both the current background and a crack defect in the prediction result of the old model, and the probability of belonging to a crack defect is greater than a fixed threshold, the prediction result is directly used; the remaining pixels are regarded as background.

[0043] S4: Feature knowledge distillation based on attention maps and auxiliary multi-class pseudo labels; in this embodiment, the feature map group F of the new model is combined according to the cosine similarity between the attention maps of the two models. t The feature map group F of the old model t-1 Alignment:

[0044]

[0045] Among them, A(·) is the above attention mechanism.

[0046] In this embodiment, the multi-scale feature map tensor is further processed by a multi-scale auxiliary classifier c l (·) Perform auxiliary multi-label recognition to deeply explore defect category information at various scales in the image:

[0047]

[0048] The weighted BCE loss function L Weighted-BCEThe weights are calculated according to the pixel ratio of each category of defects in the dataset to alleviate the class imbalance problem.

[0049] In this embodiment, in addition to assisting multi-label recognition, we will also use the KL divergence to quantify the difference between the feature classification logit of the new and old models at multiple scales using the following formula to further align the multi-category information of the new and old models:

[0050]

[0051] Based on the above knowledge distillation, the data of the new type of photovoltaic crack defects are simultaneously input into the teacher model and the student model, and the new model is quickly fine-tuned through the following loss function, so that the new model can achieve continuous segmentation of new types of defects.

[0052]

[0053] The loss between the student model prediction result and the pseudo label; is the characteristic distillation loss, which is calculated as follows:

[0054]

[0055] λ a , l and λ m are their respective balancing hyperparameters, which are 1, 20, and 1 respectively.

[0056] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions all fall within the protection scope of the claims of the present invention.

Claims

1. A photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation, characterized in that: The method comprises the following steps: S1: Set the model of the previous stage, i.e., stage (t-1), as the old model, and the model of the current stage, i.e., stage t, as the new model, where t represents the step of continuous learning; S2: Calculation of attention maps in the new and old models, using an attention mechanism based on fast normalized feature fusion to enhance the model’s attention to subtle or hard-to-detect defect features; S3: Generate pseudo labels by fusing the prediction results of the old model with the true values ​​containing only the defects of the new category to obtain pseudo labels; S4: Feature knowledge distillation based on attention maps and auxiliary multi-class pseudo-labels allows the model to be quickly fine-tuned based on the newly added new defect data when new types of defects that are not in the original database appear, so that the model can effectively segment old and new types of hidden crack defects at the same time, thereby achieving continuous segmentation of photovoltaic hidden crack defects.

2. According to claim 1, the photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation is characterized in that: In step S1, knowledge distillation is performed between the old model and the new model of this stage so that the new model inherits the knowledge of old category defects in the old model, thereby avoiding forgetting old category defects while learning new category defects, and achieving continuous segmentation of hidden crack defects.

3. The photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation according to claim 1 is characterized in that: In step S2, the multi-scale feature map tensor F corresponding to the convolutional layer in the old model and the new model is extracted respectively. t-1 and F t , F t-1 and F t Each of them contains N feature maps, where N is the number of scales 4. Each feature map enhances the attention to subtle or difficult-to-detect defect features through an attention mechanism based on fast normalized feature fusion, thereby improving the accuracy of defect segmentation. The attention mechanism is: Among them, x is a single feature map, i or j is the index of each point in the image block R, and x i or x j is the eigenvalue of the corresponding point, A r is the attention value obtained for the rth image block. The size of the image block R is k×k, k is 2, and ∈ is set to 0.001 to avoid numerical instability.

4. The photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation according to claim 2 is characterized in that: In step S3, the prediction results of the old model and the true value labels of the new defects are used to realize pseudo labels. If the pixel belongs to the defect in the current true value, the true value label is used directly; if the pixel is both the current background and the old class defect in the old model prediction result, and the probability of belonging to the old class defect is greater than the fixed threshold of 0.55, the prediction result is used directly; the remaining pixels are regarded as background.

5. The photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation according to claim 1 is characterized in that: In step S4, the feature map group F of the new model is grouped according to the cosine similarity between the attention maps of the two models. t The feature map group F of the old model t-1 Alignment: Among them, L cos is the cosine similarity, A(·) is the attention mechanism, and l indicates the scale from which the feature map comes.

6. The photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation according to claim 1 is characterized in that: In step S4, the multi-scale feature map tensor is further processed by a multi-scale auxiliary classifier c l (·) Perform auxiliary multi-label recognition to deeply explore defect category information at various scales in the image: The weighted BCE loss function L Weighted-BCE The weight is calculated according to the pixel ratio of each category of defects in the data set to alleviate the problem of imbalanced distribution of defect classes.

7. The photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation according to claim 1 is characterized in that: In step S4, in addition to auxiliary multi-label recognition, the difference between the feature classification logits of the new and old models is quantified at multiple scales using the following formula through KL divergence to further align the multi-category information of the new and old models:

8. The photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation according to claim 1 is characterized in that: In step S4, the new type of photovoltaic defect data set is simultaneously input into the teacher model and the student model, and fast fine-tuning is performed based on the following loss function: The loss between the student model prediction result and the pseudo label; is the characteristic distillation loss, which is calculated as follows: λ a , l and λ m are their respective balancing hyperparameters, which are 1, 20, and 1 respectively.

9. The photovoltaic module defect continuous segmentation method based on attention mechanism and auxiliary multi-label knowledge distillation according to claim 1 is characterized in that: In step S4, the original defect segmentation model is any semantic segmentation model, including one of the Deep Lab series models.

Images