A photovoltaic image defect classification method based on a transfer learning and unsupervised learning method

Abstract

The application discloses a photovoltaic image defect classification method based on a transfer learning and unsupervised learning method. The method performs transfer learning through a pre-training model obtained on a general image dataset to establish an initial defect classification model; then a large number of collected unlabeled images are mapped into a feature space through the initial model, clustering is performed according to L2 distance in the feature space, and corresponding labels are obtained, so that the initial model is continuously retrained, and the accuracy index of the photovoltaic image defect classification model is improved. The application takes collected images of a certain photovoltaic power station in Hainan as test data, gives a detailed algorithm description, and designs test examples to evaluate and verify the defect classification performance of the photovoltaic image.

Description

Technical Field

[0001] This invention relates to the field of power system defect detection, specifically to a photovoltaic image defect classification method based on transfer learning and unsupervised learning. Background Technology

[0002] Over the past few decades, with the shift from fossil fuels to green and sustainable energy supplies, the energy output of global photovoltaic (PV) power plants has increased dramatically. The efficiency of PV modules determines the performance of a power plant, and the performance of a power plant is largely affected by high-temperature conditions. Identifying early defects in PV modules is an effective and economical method to improve the long-term efficiency of PV power plants and prevent sudden failure of PV modules. Extensive research has been conducted in this field, using electrical signals obtained from contact sensors to detect electrical faults in PV modules. As a reactive maintenance strategy, such detection faces several challenges, such as deployment costs, sensor misreading, and others.

[0003] Automated systems can perform non-destructive and non-contact inspections of large photovoltaic (PV) systems to identify visual defects without interrupting system operation. With the help of such a system, a series of images of PV modules can be collected sequentially and timely, and stored for further processing. These images can then be processed offline using deep learning-based methods to extract defect features, defect maps, and summary statistics. To further improve detection accuracy and reduce manual annotation work, the need for label-less or unlabeled PV defect classification has been put on the agenda. Transfer learning, as an emerging deep learning paradigm, reduces the model's learning of basic feature extraction capabilities, making it a promising method for achieving label-less or unlabeled PV defect classification.

[0004] Furthermore, data annotation is a time-consuming and labor-intensive process, requiring manual screening by domain experts and potentially involving subjective judgment. If photovoltaic module defect categories could be automatically labeled without human intervention, the efficiency of the detection system could be further improved. Considering all these factors, a novel approach is proposed, combining a pre-processed deep convolutional neural network model with unsupervised learning to construct our defect classification system. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a photovoltaic image defect classification method based on transfer learning and unsupervised learning. This invention presents an efficient and innovative pre-training initialization model method and a similarity evaluation method for unlabeled image samples. It also achieves automatic labeling of unlabeled images by improving the k-means algorithm and using pseudo-labels. Finally, it enhances the model's fitting ability by fine-tuning the pre-trained model.

[0006] The objective of this invention is achieved through the following technical solution: a photovoltaic image defect classification method based on transfer learning and unsupervised learning, comprising the following steps:

[0007] 1) An initial photovoltaic defect classification model is established based on a general image pre-trained model on ImageNet through transfer learning;

[0008] 2) Obtain the initial class center and initial class spacing threshold in the feature space of the initial photovoltaic defect classification model using example images of the classes;

[0009] 3) Collect unlabeled photovoltaic images and map them into the feature space through an initial photovoltaic defect classification model; classify the unlabeled photovoltaic images into categories based on the improved k-means algorithm to obtain a set of pseudo-labeled images, and update the category center and category distance thresholds;

[0010] 4) Retrain the initial photovoltaic defect classification model based on the pseudo-label image set obtained in step 3) to obtain a trained photovoltaic defect classification model;

[0011] 5) Input the photovoltaic image to be classified into the trained photovoltaic defect classification model to complete the photovoltaic image defect classification.

[0012] As a preferred embodiment of the present invention, step 1) specifically involves: based on a general image pre-training model on ImageNet, transferring its weight parameters to a deep learning network to obtain an initialized photovoltaic defect classification model M. O =F(x, θ); where x is the input image and θ is the weight parameter for initializing the photovoltaic defect classification model.

[0013] As a preferred embodiment of the present invention, step 2) specifically involves: processing the category example image D containing n defect category examples. n The features are embedded into the feature space of the initial photovoltaic defect classification model to obtain n initial class centers P0.

[0014] D n ={(x1,y1),...,(x n ,y n )}

[0015] p i =F(x) i ,θ)

[0016] P0={(P i ,y i )}

[0017] Where x1…x n Example images for n categories; y1…y nθ represents the labels for the n types of defects that the model needs to identify; θ represents the weight parameters for initializing the photovoltaic defect classification model; p i Example image x for category i Initialize the photovoltaic defect classification model M o Mapping of feature space, x i ∈{x1…x n};y i ∈{y1…y n}

[0018] The initial class distance threshold H0 is obtained based on the L2 distance between the initial class centers, and its calculation method is as follows:

[0019]

[0020] Where x i and x j Here are two example images from different categories.

[0021] As a preferred embodiment of the present invention, step 3) specifically includes:

[0022] a) For the acquired unlabeled photovoltaic images D u ={x1…x m}, by initializing model M O Obtain the mapping points of the image in the feature space:

[0023] p k =F(x) k ,θ)

[0024] Where x k ∈D u θ represents the weight parameters for initializing the photovoltaic defect classification model.

[0025] b) Calculate the mapping point p based on the improved k-means algorithm k Calculate the distances to the n initial class centers P0 and assign them to the class corresponding to the initial class center with the smallest L2 distance; then, based on the initial class distance threshold H... O Filter each mapping point p k Can we obtain the pseudo-label for this category? If p k The distance to the nearest initial class center is less than or equal to H O Then, it obtains the label and places the unlabeled photovoltaic image it represents into the pseudo-labeled image set D. p Otherwise, remain in D. u middle;

[0026] c) Based on pseudo-label image set D p Recalculate the category center P for n categories:

[0027]

[0028] P={(p n ,y n )}

[0029] Where x k For the pseudo-label image set D p Images grouped into the same label, p n For label y n The updated value of the corresponding nth category center;

[0030] Simultaneously, the initial category spacing threshold H is set. O Updated to the category spacing threshold H:

[0031]

[0032] Where p i and p j These are the updated category centers for category i and category j, respectively;

[0033] d) For the remaining unlabeled photovoltaic image D u Repeat steps b) and c) until the labelless photovoltaic image D is satisfied. u There are no samples, or the maximum number of iterations has been reached.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] (1) This invention creatively establishes a photovoltaic image defect classification method based on transfer learning and unsupervised learning, forming a low-cost, high-performance learning paradigm for photovoltaic image defect recognition and classification models. Firstly, based on the transfer learning method, an initial model is constructed through pre-trained weights, reducing the learning process for the model's ability to extract basic photovoltaic image features, thus saving significant data requirements and learning time. Simultaneously, addressing the issues of long annotation times and high professional requirements for photovoltaic image data, an improved k-means clustering method is used to achieve automated, high-quality annotation of photovoltaic images. This further improves the model's defect classification performance and further controls the cost of acquiring photovoltaic image data.

[0036] (2) The method of this invention differs from existing conventional deep learning photovoltaic defect classification methods in that it involves different data acquisition costs and model training methods. Conventional deep learning photovoltaic defect classification methods often require a large number of labeled photovoltaic images to improve model performance, and the model's feature extraction capabilities need to be trained from scratch over a long period of time. In contrast, the improved k-means algorithm based on unsupervised learning in this invention can achieve high-quality labeling of unlabeled images, and at the same time, it enables the model to acquire basic feature extraction capabilities by transferring pre-trained weights. Therefore, the method adopted in this invention can reduce data labeling and model training costs and improve the model's defect classification performance. Attached Figure Description

[0037] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0038] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.

[0039] The present invention provides a photovoltaic image defect classification method based on transfer learning and unsupervised learning, comprising the following steps:

[0040] 1) An initial photovoltaic defect classification model is established based on a general image pre-trained model on ImageNet through transfer learning;

[0041] (2) Obtain the initial category center and initial category spacing threshold in the feature space of the initial photovoltaic defect classification model by using category example images;

[0042] (3) Collect unlabeled photovoltaic images and map them into the feature space through the initial photovoltaic defect classification model. Based on the improved k-means algorithm, classify the unlabeled photovoltaic images to obtain pseudo-labeled images and update the category center and category distance thresholds.

[0043] (4) Retrain the initial photovoltaic defect classification model based on the pseudo-label image obtained in step (3) to obtain the trained photovoltaic defect classification model;

[0044] (5) Input the photovoltaic image to be classified into the trained photovoltaic defect classification model to complete the photovoltaic image defect classification.

[0045] In one specific embodiment of the present invention, the various steps of the present invention will be described in more detail.

[0046] A large number of unlabeled photovoltaic images, totaling 1128, were collected from a photovoltaic power plant in Hainan. Four fault categories were selected for identification: normal operation, hot spot defects, contact overheating, and diode short circuit. One example image was chosen for each category.

[0047] EfficientNet-B0 was chosen as the framework for the photovoltaic image defect classification model. Pre-trained weight parameters θ based on ImageNet were loaded into the model and transferred to a deep learning network to obtain the initial model M. O =F(x, θ), where x is the input image, and the structural framework of the model is shown in the table below.

[0048] Table 1 EfficientNet-B0 Network Structure

[0049]

[0050] The set of four category example images D, containing examples of different defect categories. n ={(x1,y1),...,(x4,y4)}, embedding to initialize the feature space, we obtain the class center P0 = {(p1,y1),...,(p4,y4)}. Where p i =F(x) i ,θ), is the image x i By initializing the mapping of model M0 in the feature space, y1,...,y4 are the labels for the four types of defects that the model needs to identify, representing the four categories: normal, hot spot defect, contact overheating, and diode short circuit. Based on the L2 distance between the category center points, the category spacing threshold H0 is obtained, calculated as follows:

[0051]

[0052] Where x i and x j Here are two example images from different categories.

[0053] The 1128 unlabeled photovoltaic images collected were used for D u ={x1,...,x 1128 The mapping points of the image in the feature space are obtained by initializing model M0:

[0054] p k =F(x) k ,θ)

[0055] P u ={p1,...,p 1128}

[0056] Where x k ∈D uθ represents the weight parameters for initializing the photovoltaic defect classification model.

[0057] For the mapping point set P u Each mapping point P in k Calculate the 4 category centers P n The distance is calculated and the cells are assigned to the class corresponding to the class center with the smallest L2 distance.

[0058]

[0059] Based on the category spacing threshold H O Filter each mapping point P k Is it possible to obtain pseudo-labels for this category, if P k To the nearest category center P i The distance is less than or equal to H O Then it obtains the label and puts it into D. p Otherwise, remain in D u middle.

[0060] Based on pseudo-label image set D p Recalculate the category center P for the four categories:

[0061]

[0062] P={(p n ,y n )};n∈{1,…,4}

[0063] Where x k For pseudo-label set D p The middle is divided into the same label y i The image, p n For label y n The updated value of the corresponding nth category center;

[0064] The category spacing threshold H is also updated:

[0065]

[0066] Where p i and p j These are the updated category centers for category i and category j, respectively;

[0067] The pseudo-label image set D obtained based on the improved k-means algorithm p The initial model M0 is trained to obtain an enhanced photovoltaic image defect classification model M. The training process uses classification cross-entropy as the objective loss function, calculated as follows:

[0068]

[0069] Where y i This represents the pseudocode label of the sample, where y represents the pseudocode label of the sample as class i. i =1 otherwise y i =0; t i This represents the probability that the model predicts a sample belongs to the i-th class. Based on this, the parameters in the model are updated using gradient propagation, calculated as follows:

[0070]

[0071]

[0072] Where W represents the model's weight parameters, b represents the model's bias parameters, and W + b represents the update amount of W. + Let η represent the update amount of b, and let η represent the learning rate of Adam as the backpropagation optimizer. For the partial derivative of the target loss function, For the partial derivatives of the weight parameters, This is the partial derivative of the offset parameter. In this implementation, the Adam optimizer with a batch size of 32 was used to optimize the model, with the learning rate initialized to 0.0005 and momentum to 0.9. We trained the model for 400 epochs.

[0073] A test label image set was constructed, consisting of 350 pseudo-label images and 150 real label images in a 7:3 ratio. The model's performance improvement was validated using precision, recall, and accuracy. The calculation methods for these three metrics are as follows:

[0074]

[0075]

[0076]

[0077] Where: TP is the number of correctly segmented pseudo-labeled images; FP is the number of correctly segmented real-labeled images; FN is the number of incorrectly segmented pseudo-labeled image samples; TN is the number of incorrectly segmented real-labeled images.

[0078] The performance results of this model on the test set are shown in the table below.

[0079] Table 2 compares the defect classification performance and training time of our method and conventional supervised deep learning.

[0080]

[0081] The experimental environment used in this implementation is shown in the table below:

[0082] Table 3. Experimental Environment Description

[0083]

[0084]

[0085] The results show that, compared to conventional supervised deep learning methods with the same model structure, our proposed method exhibits significant advantages in precision, recall, and accuracy: achieving 94.2% precision, outperforming conventional supervised methods by 9 percentage points; 88.3% recall, also outperforming conventional supervised methods by 9 percentage points; and 11 percentage points accuracy. Furthermore, in terms of training time, supervised methods require 1000 epochs to converge to the aforementioned performance levels, while our proposed method converges in only 400 epochs. This demonstrates that our proposed method offers significant improvements over conventional deep learning methods in both defect classification performance and training time cost, without requiring a large amount of expensive labeled image data. This method plays a crucial role in achieving low-cost, high-performance photovoltaic image defect classification.

[0086] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A photovoltaic image defect classification method based on a transfer learning and unsupervised learning method, characterized in that, The method comprises the following steps: 1) An initial photovoltaic defect classification model is established by transfer learning based on a general image pre-training model on ImageNet; 2) An initial class center and an initial class distance threshold in a feature space of the initial photovoltaic defect classification model are obtained through a class example image; 3) An unlabeled photovoltaic image is collected and mapped into the feature space through the initial photovoltaic defect classification model; A pseudo-labeled image set is obtained by classifying the unlabeled photovoltaic image based on an improved k-means algorithm, and the class center and the class distance threshold are updated; In the step 3), the improved k-means algorithm calculation formula is as follows: a) collecting a no-label photovoltaic image D u = {x1...x m}, and passing it through an initialized photovoltaic defect classification model M O to obtain a mapping point of the image in the feature space: p k = F(x k , θ); where x k ∈D u ; θ are weight parameters for initializing the photovoltaic defect classification model; b) computing the mapping points p based on an improved k-means algorithm k the distance to the n initial cluster centers P0 and put it in the class corresponding to the initial cluster center with the smallest L2 distance; according to the initial cluster distance threshold H O filtering each mapping point p k whether the pseudo label of this class can be obtained, if p k the distance to the nearest initial cluster center is less than or equal to H O , then make it obtain the label and put the unlabeled photovoltaic image represented by it into the pseudo-labeled image set D p , otherwise it still remains in D u . c) based on the pseudo-labeled image set D p , recalculate the class centers P of the n classes: P = {(p n ,y n )}; where x k is the pseudo-labeled image set D p images divided into the same label, p n is the label y n the updated value of the corresponding nth class center; At the same time, the initial inter-class distance threshold H is updated as O the inter-class distance threshold H: where p i and p j are the updated class centers for class i and class j, respectively. d) for the remaining unlabeled photovoltaic images D u The operations of steps b) and c) are repeated until either no samples are left in the unlabeled photovoltaic images D u or a maximum number of iterations is reached. 4) The initial photovoltaic defect classification model is retrained according to the pseudo-labeled image set obtained in the step 3), and a trained photovoltaic defect classification model is obtained; where y k is the mapping point P k the label type obtained by dividing 5) A photovoltaic image to be classified is input into the trained photovoltaic defect classification model, and photovoltaic image defect classification is completed. The step 1) is specifically as follows:

2. The photovoltaic image defect classification method based on the transfer learning and unsupervised learning method according to claim 1, characterized in that, The step 2) is specifically as follows: Based on the general image pre-training model on ImageNet, the weight parameters are migrated to the deep learning network to obtain an initialized photovoltaic defect classification model M O = F(x, θ); wherein x is an input image, and θ is the weight parameter of the initialized photovoltaic defect classification model. 3.The photovoltaic image defect classification method based on the transfer learning and unsupervised learning method according to claim 1, characterized in that, An initial class distance threshold H0 is obtained according to an L2 distance of the initial class center point, and the calculation method is as follows: a category example image D including n defect category examples n embedding into a feature space for initializing a photovoltaic defect classification model, obtaining n initial category centers P0: D n = {(xl,yl),..., (x n ,y n )}; p i = F(x i , θ); P0 = {(p i ,y i )}; Where x1…x n Example images for n categories; y1…y n θ represents the labels for the n types of defects that the model needs to identify; θ represents the weight parameters for initializing the photovoltaic defect classification model; p i Example image x for category i Initialize the photovoltaic defect classification model M O Mapping of feature space, x i ∈{x1…x n };y i ∈{y1…y n }; The step 4) is specifically as follows: where x i and x j are two different class of example images. 4.The photovoltaic image defect classification method based on the migration learning and unsupervised learning method according to claim 1, characterized in that, On this basis, the parameters in the model are updated by gradient transmission, and the calculation method is as follows: Based on the pseudo label image set D obtained in step 3) p The initialized model M0 is trained to obtain an enhanced photovoltaic image defect classification model M; the training process takes classification cross-entropy as the target loss function, and the calculation formula is: where y i represents the sample pseudo-code label, y i = 1 if the sample is of the i-th class, otherwise y i = 0; t i represents the probability that the model predicts the sample belongs to the i-th class; ​ where W denotes a weight parameter of the model, b denotes a model bias parameter, W + denotes an update amount of W, b + denotes an update amount of b, and η denotes a learning rate of the optimizer, is a partial derivative of the target loss function, is a partial derivative of the weight parameter, is a partial derivative of the bias parameter.

Images