Photovoltaic panel defect identification method based on knowledge distillation and pruning technology

Abstract

The invention relates to the technical field of image recognition, and particularly discloses a photovoltaic panel defect recognition method based on knowledge distillation and pruning technologies, which comprises the following steps: establishing defect image data and annotation image data, dividing the annotation image data into a training set, a verification set and a test set for SRGAN model training, verification and testing to obtain super-resolution image data; dividing the step annotation image data and the super-resolution image data into a training set, a verification set and a test set, respectively training, verifying and testing the teacher model and the student model to obtain optimal performance weights, obtaining a student model after distillation training, obtaining a student model after constraint training, obtaining a student model after pruning, and obtaining the optimal performance weights of the teacher model, the student model and the student model after pruning; and carrying out callback training on the pruned student model. The method can reduce time consumption, high manual detection cost and the like, can improve productivity and actual industrial application value, has the advantages of high detection precision and light weight, and is more suitable for photovoltaic panel image defect identification.

Description

Technical Field

[0001] The invention relates to the technical field of image recognition, in particular to a defect recognition method for designing photovoltaic panels. Background Art

[0002] Solar energy is a clean and renewable energy source. Photovoltaic power generation systems can convert solar energy into electrical energy. They have the advantages of no pollution, safe power generation, noiseless operation and low installation cost. However, due to the uncertainty of outdoor environmental conditions and potential damage to photovoltaic panels during manufacturing, transportation or installation, various photovoltaic failures may occur, resulting in varying degrees of degradation, power loss and even fire risks. Therefore, it is of great significance to effectively identify defects in photovoltaic panels to ensure the reliability and durability of power generation. Generally speaking, defect identification of photovoltaic panels requires manual work by experts, which is very time-consuming, costly and requires extensive expertise. In addition, long hours of work may cause people to experience problems such as fatigue, perceptual or cognitive biases, leading to identification errors.

[0003] Chinese patent 202210528398.0 discloses a two-stage photovoltaic panel defect detection method based on deep learning, which mainly uses the YOLOv5 model to detect defects on infrared images of photovoltaic panels, trains YOLOv5 models for foreground extraction and defect detection respectively, and inputs the small unit panel image output by the foreground extraction YOLOv5 model into the defect detection YOLOv5 model to obtain the final defect detection result. This method directly uses the original network structure of the YOLOv5 model, does not improve the complexity of the model, and has limited cost control capabilities in actual industrial applications.

[0004] Chinese patent 202210209242.6 discloses a photovoltaic panel defect detection method based on visible light and infrared vision, which uses the YOLOv3 model to detect surface stains on visible light photovoltaic panel images, and uses an adaptive minimum threshold segmentation algorithm to detect hot spot effect areas on infrared photovoltaic panel images. This method can only detect surface stains for visible light images, and only adds the SE attention module to the YOLOv3 model and proposes an improved anchor frame clustering algorithm for the problem of errors in the size of the anchor frame. The complexity of the model has not been improved, and the defect category detection capability is also relatively single. Summary of the invention

[0005] The purpose of the present invention is to provide a photovoltaic panel defect recognition method based on knowledge distillation and pruning technology, which has the advantages of high detection accuracy and lightweight and is more suitable for photovoltaic panel image defect recognition.

[0006] To achieve the above purpose, the technical solution of the present invention is: a photovoltaic panel defect recognition method based on knowledge distillation and pruning technology, the recognition method steps are as follows: S1. Establishing photovoltaic panel defect image data, which includes defect image data and annotated image data; S2, dividing the annotated image data obtained in step S1 into a training set, a validation set, and a test set according to a certain ratio, wherein the training set is used to train the SRGAN model, the validation set is used to verify the performance of the SRGAN model during training, and the test set is used to test the performance of the model after training. The weight with the best performance is used to perform super-resolution reconstruction on all images to obtain super-resolution image data; S3, dividing the annotated image data obtained in step S1 and the super-resolution image data obtained in step S2 into a training set, a validation set, and a test set according to a certain ratio, and respectively training, validating, and testing the teacher model and the student model on the divided data sets, and obtaining the optimal performance weights of the teacher model and the student model after the training, validation, and testing are completed; S4, using the optimal performance weights of the teacher model and the student model in step S3 to perform distillation training on the student model to obtain a student model after distillation training; S5, performing constraint training on the student model after distillation training to obtain a constrained student model; then, performing a pruning operation on the student model after constraint training to obtain a pruned student model; S6. Perform callback training on the pruned student model.

[0007] The step S1 establishes defect image data of photovoltaic panels by collecting multiple defect images including three defect categories of scratches, broken grids and dirt, and obtains defect image data by rotating the defect images at multiple angles to obtain extended images as defect images; all defect images in the defect image data are labeled, and the labeling includes labeling the defect type and enclosing the defect location area in the defect image with a rectangular frame to obtain the labeled image data.

[0008] In the SRGAN model in step S2, the generator G is used to output a high-resolution image, and the discriminator D is used to determine whether an image is a high-resolution image. The training obtains the G with the best performance. In the training, the perceptual loss function is used. , the calculation formula is:

[0009] in, Indicates content loss, represents the adversarial loss, and the calculation formulas are:

[0010]

[0011] In the formula, represents the mean square error, represents a high-resolution image, represents the pixel value at position (x,y) in the high-resolution image, represents a low-resolution image, W and H represent the width and height of the feature map in the network, r represents the scaling factor, X represents the horizontal coordinate position of the pixel in the image, and Y represents the vertical coordinate position of the pixel in the image. represents the generator responsible for generating images, represents the discriminator responsible for determining whether the image is a generated image or a real image, and n represents the nth sample in the total number of samples N.

[0012] In step S2, super-resolution reconstruction is performed, and the result of super-resolution reconstruction is evaluated by three indicators: PSNR, MSE and SSIM, so as to obtain super-resolution image data; wherein, the calculation formulas of PSNR and MSE are respectively: ,

[0013] Where R is the maximum pixel value in the image. and There are two images respectively, N is the total number of pixels in the image, and the calculation formula of SSIM is:

[0014] Where x and y are two images to be compared. and are the average brightness values ​​of x and y respectively, and are the mean brightness variances of x and y, respectively, is the covariance of x and y, and C1 and C2 are constants.

[0015] In step S3, the YOLOv11x model is selected as the teacher model, and the YOLOv11n model is selected as the student model; In the teacher model of step S4, the probability of each category is outputted by the improved Softmax calculation. The improved Softmax introduces the distillation temperature variable T on the original Softmax, and its calculation formula is:

[0016] In the formula, represents the probability of the i-th category, j represents the j-th category among all categories, represents the original probability that the teacher model outputs the i-th category, Represents the original probability that the teacher model output is the jth class; In step S4, the student model is distilled and trained, and the total loss calculation formula of the distillation training is:

[0017] Among them, γ represents the loss weighting coefficient, represents the distillation loss between the teacher model and the student model, Represents the classification loss between the output of the student model and the true label.

[0018] In step S5, the constraint training is performed by adding L1 regularization to the Batch Normalization layer of the student model. The calculation formula of L1 regularization is:

[0019] In the formula, w represents the weight vector of the model, i represents the i-th weight among n weights, and n represents the length of the weight vector, that is, the number of weights. Represents the error between the predicted value of the model and the true label; during the pruning operation in step S5, the matrix with values ​​close to zero in the weight matrix generated by L1 regularization is removed.

[0020] By adopting the above technical scheme, the beneficial effect of the present invention is as follows: the present invention aims at the problem that the performance of solar photovoltaic panels is reduced or the life is reduced due to defects, and proposes a photovoltaic panel defect recognition method based on knowledge distillation and pruning technology with high detection accuracy and lightweight advantages, which is more suitable for photovoltaic panel image defect recognition. The method uses the YOLOv11 model to identify three defect categories of scratches, broken grids and dirt on the image, which can reduce the resource consumption of the defect recognition model in actual industrial applications, and transfers the knowledge learned in the data by the teacher model with strong learning ability but complex model structure to the student model with relatively weak learning ability but simple model structure, thereby enhancing the generalization ability of the student model, and then pruning the student model to make it have both high detection accuracy and lightweight advantages. In addition, in order to solve the problem that the model is difficult to identify smaller defects, the SRGAN model is used to perform super-resolution reconstruction preprocessing on the image, thereby increasing the image details extracted by the model and realizing the improvement of the detection accuracy of smaller defects. Through the defect recognition method proposed by the present invention, the time-consuming and expensive manual inspection costs can be reduced, the production capacity can be increased, and the actual industrial application value can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of a photovoltaic panel defect identification method based on knowledge distillation and pruning technology involved in the present invention.

[0022] Figure 2 This is a schematic diagram of labeling defect images of three defect categories in a photovoltaic panel defect recognition method based on knowledge distillation and pruning technology involved in the present invention.

[0023] Figure 3This is a schematic diagram of the comparison of images of three defect categories before and after super-resolution reconstruction in a photovoltaic panel defect recognition method based on knowledge distillation and pruning technology involved in the present invention.

[0024] Figure 4 It is a schematic diagram of the distillation training learning process in a photovoltaic panel defect recognition method based on knowledge distillation and pruning technology involved in the present invention.

[0025] Figure 5 It is a comparative schematic diagram before and after pruning operation in a photovoltaic panel defect recognition method based on knowledge distillation and pruning technology involved in the present invention. DETAILED DESCRIPTION

[0026] In order to further explain the technical solution of the present invention, the present invention is described in detail below through specific embodiments.

[0027] This embodiment discloses a photovoltaic panel defect recognition method based on knowledge distillation and pruning technology, and the recognition method steps are as follows: Figure 1 As shown: S1. Establishing photovoltaic panel defect image data; In this embodiment, multiple (preferably more, 600 images were collected in the embodiment test) defect images including three defect categories: scratches (Crack), broken grids (Grid) and dirt (Spot). Then, the defect images are rotated at multiple angles (such as 90°, 180° and 270° rotation operations in the embodiment test) to obtain expanded images, thereby expanding the defect images (the final data set after expansion in the embodiment test contains 2400 images) to obtain defect image data. All defect images in the defect image data are manually labeled using a labeling tool (such as LabelImg software). The labeling includes labeling the defect type and enclosing the defect location area in the defect image with a rectangular frame (x, y, w, h). The labeling results are shown in the figure. Figure 2 As shown, the labeled image data is obtained after manual labeling.

[0028] S2. Divide the labeled image data obtained in step S1 into a training set, a validation set, and a test set according to a certain ratio (7:2:1 in the embodiment test), wherein the training set is used to train the SRGAN model (Super-ResolutionGenerative Adversarial Network is a technology for image super-resolution using a generative adversarial network (GAN)), set the initial learning rate of the model to 0.0004, the minimum learning rate to 0.00004, the learning rate reduction method to cos, use the Adam optimizer, set the momentum to 0.9, set the upsampling multiple to 4, and train 200 epochs with a batch size of 16. The validation set is used to verify the performance of the SRGAN model during training, and the test set is used to test the performance of the model after training. The weight with the best performance is used to perform super-resolution reconstruction on all images to obtain super-resolution image data.

[0029] In this step, SRGAN is a super-resolution reconstruction model based on the GAN framework. In GAN, the generator G is usually used to randomly generate images, and the discriminator D is usually used to determine whether an image belongs to a certain data set. In SRGAN, the generator G is used to output high-resolution images, and the discriminator D is used to determine whether an image is a high-resolution image. Specifically, during the model training process, D will be continuously trained, and G will be continuously and iteratively updated according to the results of D. D and G compete with each other and grow with each other. Finally, G with the best performance is obtained through training. In this embodiment of the present invention, the perceptual loss function is used to train G. , the calculation formula is:

[0030] in, Indicates content loss, represents the adversarial loss, and the calculation formulas are:

[0031]

[0032] In the formula, represents the mean square error, and uses the mean square error (MSE) as the content loss. represents a high-resolution image, represents the pixel value at position (x,y) in the high-resolution image, represents a low-resolution image, W and H represent the width and height of the feature map in the network, r represents the scaling factor, X represents the horizontal coordinate position of the pixel in the image, and Y represents the vertical coordinate position of the pixel in the image. represents the generator responsible for generating images, The discriminator is responsible for determining whether the image is a generated image or a real image, and n represents the nth sample in the total number of samples N.

[0033] After the model training is completed, the super-resolution reconstruction results are evaluated by three indicators: PSNR (Peak Signal-to-Noise Ratio), MSE (mean-square error) and SSIM (Mean Structural Similarity Index Measure), thereby obtaining super-resolution image data. The calculation formulas for PSNR and MSE are:

[0034]

[0035] Where R is the maximum pixel value in the image. and There are two images respectively, N is the total number of pixels in the image, and the calculation formula of SSIM is:

[0036] Where x and y are two images to be compared. and are the average brightness values ​​of x and y respectively, and are the mean brightness variances of x and y, respectively, is the covariance of x and y, C1 and C2 are constants used to stabilize the calculation and prevent the denominator from being zero. , , L is the dynamic range of pixels in the image, and examples of the comparison of images of three defect categories before and after super-resolution reconstruction are shown in Figure 3 shown.

[0037] S3, dividing the annotated image data obtained in step S1 and the super-resolution image data obtained in step S2 into a training set, a validation set, and a test set according to a certain ratio (7:2:1 in the embodiment test), and training, validating, and testing the teacher model and the student model on the divided data sets (Teacher-Student Training Method TrainingParadigm) generally refers to a knowledge distillation technique in deep learning, in which a fully trained and well-performing model (teacher model) guides the learning process of another to-be-trained or simpler model (student model). In this embodiment, the YOLOv11x model (existing model) is selected as the teacher model, and the YOLOv11n model (existing model) is selected as the student model. The training settings of the teacher model and the student model in the embodiment are the same, wherein the initial learning rate is set to 0.01, the minimum learning rate is set to 0.01, the optimizer is selected using the "auto" mode, the momentum is set to 0.937, the weight decay is set to 0.0005, and 200 epochs are trained with a batch size of 16. After training, verification and testing are completed, the optimal performance weights of the teacher model and the student model are obtained, and the optimal performance weights of the teacher model and the student model are used as the basis for distillation learning.

[0038] S4, using the optimal performance weights of the teacher model and the student model in step S3 to perform distillation training on the student model to obtain a student model after distillation training. The distillation training learning process is as follows Figure 4 As shown in the figure, the teacher model (TeacherModel) outputs the probability of each category through the improved Softmax calculation, that is, the soft labels (Soft labels). The soft labels have high information entropy, that is, they contain a large amount of information, which is used to guide the learning of the student model (Student Model). It is worth noting that the improved Softmax mainly introduces the distillation temperature variable T on the original Softmax, and its calculation formula is:

[0039] In the formula, represents the probability of the i-th category, j represents the j-th category among all categories, represents the original probability that the teacher model outputs the i-th category, It indicates that the teacher model outputs the original probability of the jth class. When the distillation temperature variable T=1, the function is the same as the original Softmax. As T increases, the probability values ​​of all categories output by Softmax become more average. In other words, the size of T affects the degree of attention paid to negative samples during training. The higher T is, the more attention is paid to negative samples. Therefore, when it is necessary to learn information from negative samples, T needs to be increased. When you do not want to be disturbed by the information in negative samples, T needs to be lowered. Figure 4 It can be seen that the teacher model and the student model output soft labels and soft predictions respectively at T=t, and the distillation loss can be calculated through soft labels and soft predictions; the student model outputs hard predictions at T=1, and the student loss of the student model training can be calculated through hard predictions and hard labels. The total loss calculation formula for the final distillation training is:

[0040] in, The role of the distillation temperature variable T is to compensate for the gradient size when calculating soft labels. Scaling, and γ represents the loss weighting coefficient, γ needs to be as large as possible so that The weight of is greater, that is, the distillation effect is greater. represents the distillation loss between the teacher model and the student model, Represents the classification loss between the output of the student model and the true label.

[0041] S5. Constrain the student model after distillation training to obtain a constrained student model; then, prune the student model after constraint training to obtain a pruned student model. After the student model is distilled and trained in step S4, in order to make the model more lightweight, it is necessary to perform pruning on the model in this step; since pruning operations usually involve removing some weights or neurons in the network, thereby reducing the size and computing requirements of the model, and the removed weights may contain information that is important to the performance of the model, pruning operations may have a negative impact on the generalization ability of the model (such as performance degradation). To alleviate this problem, this step also performs constraint training on the student model before pruning. Constraint training mainly adds additional constraints to guide model learning during model training, thereby reducing the negative impact that pruning may have on the model.

[0042] Specifically, in this embodiment, L1 regularization is added to the Batch Normalization layer of the model to implement constrained training for the model. The calculation formula of L1 regularization is:

[0043] In the formula, w represents the weight vector of the model, i represents the i-th weight among n weights, and n represents the length of the weight vector, that is, the number of weights. Represents the error between the model's predicted value and the true label.

[0044] Since L1 regularization tends to produce sparse weight matrices, that is, the weight matrix of the Batch Normalization layer of the model will produce many values ​​close to zero, such as Figure 5 The above is a schematic diagram of comparison before and after the pruning operation. During the pruning operation, the matrices with values ​​close to zero can be removed.

[0045] S6. Perform callback training on the pruned student model. Callback training is to train the pruned model again to restore the performance of the model or even improve the performance of the model.

[0046] In summary, the present invention combines super-resolution reconstruction technology with knowledge distillation and pruning technology to train the target detection model, so that it has the advantages of high detection accuracy and lightweight, and can realize the automatic identification of defects in solar photovoltaic panels. Compared with the prior art, the use of the SRGAN model to pre-process the image for super-resolution reconstruction increases the texture detail features in the image that can be extracted by the model, and can still effectively identify some less obvious defects, thereby improving the detection accuracy of smaller defects. In addition, lightweight models are often required in actual industrial applications. The present invention is based on knowledge distillation and pruning technology, and uses distillation learning to transfer the knowledge learned in the data of the teacher model with strong learning ability but complex model structure to the student model with relatively weak learning ability but simple model structure, and then prunes the student model, thereby reducing the complexity of the model while improving the accuracy, thereby improving the value of actual industrial applications.

[0047] The above embodiments and drawings do not limit the product form and style of the present invention. Any appropriate changes or modifications made by ordinary technicians in the relevant technical field should be deemed to be within the patent scope of the present invention.

Claims

1. A photovoltaic panel defect recognition method based on knowledge distillation and pruning technology, characterized in that: The identification method steps are as follows: S1. Establishing photovoltaic panel defect image data, which includes defect image data and annotated image data; S2, dividing the annotated image data obtained in step S1 into a training set, a validation set, and a test set according to a certain ratio, wherein the training set is used to train the SRGAN model, the validation set is used to verify the performance of the SRGAN model during training, and the test set is used to test the performance of the model after training. The weight with the best performance is used to perform super-resolution reconstruction on all images to obtain super-resolution image data; S3, dividing the annotated image data obtained in step S1 and the super-resolution image data obtained in step S2 into a training set, a validation set, and a test set according to a certain ratio, and respectively training, validating, and testing the teacher model and the student model on the divided data sets, and obtaining the optimal performance weights of the teacher model and the student model after the training, validation, and testing are completed; S4, using the optimal performance weights of the teacher model and the student model in step S3 to perform distillation training on the student model to obtain a student model after distillation training; S5, performing constraint training on the student model after distillation training to obtain a constrained student model; then, performing a pruning operation on the student model after constraint training to obtain a pruned student model; S6. Perform callback training on the pruned student model.

2. The photovoltaic panel defect identification method based on knowledge distillation and pruning technology according to claim 1, characterized in that: The step S1 establishes defect image data of photovoltaic panels by collecting multiple defect images including three defect categories of scratches, broken grids and dirt, and obtains defect image data by rotating the defect images at multiple angles to obtain extended images as defect images; all defect images in the defect image data are labeled, and the labeling includes labeling the defect type and enclosing the defect location area in the defect image with a rectangular frame to obtain the labeled image data.

3. The photovoltaic panel defect identification method based on knowledge distillation and pruning technology according to claim 1, characterized in that: In the SRGAN model in step S2, the generator G is used to output a high-resolution image, and the discriminator D is used to determine whether an image is a high-resolution image. The training obtains the G with the best performance. In the training, the perceptual loss function is used. , the calculation formula is: in, Indicates content loss, Denotes adversarial loss, and the calculation formulas are: In the formula, represents the mean square error, represents a high-resolution image, represents the pixel value at position (x,y) in the high-resolution image, represents a low-resolution image, W and H represent the width and height of the feature map in the network, r represents the scaling factor, X represents the horizontal coordinate position of the pixel in the image, and Y represents the vertical coordinate position of the pixel in the image. represents the generator responsible for generating images, represents the discriminator responsible for determining whether the image is a generated image or a real image, and n represents the nth sample in the total number of samples N.

4. The photovoltaic panel defect identification method based on knowledge distillation and pruning technology as claimed in claim 3, characterized in that: In step S2, super-resolution reconstruction is performed, and the result of super-resolution reconstruction is evaluated by three indicators: PSNR, MSE and SSIM, so as to obtain super-resolution image data; wherein, the calculation formulas of PSNR and MSE are respectively: Where R is the maximum pixel value in the image. and There are two images respectively, N is the total number of pixels in the image, and the calculation formula of SSIM is: Where x and y are two images to be compared. and are the average brightness values ​​of x and y respectively, and are the mean brightness variances of x and y, respectively, is the covariance of x and y, and C1 and C2 are constants.

5. The photovoltaic panel defect identification method based on knowledge distillation and pruning technology as claimed in claim 1, characterized in that: In step S3, the YOLOv11x model is selected as the teacher model, and the YOLOv11n model is selected as the student model.

6. A photovoltaic panel defect recognition method based on knowledge distillation and pruning technology as described in any one of claims 1 to 5, characterized in that: In the teacher model of step S4, the probability of each category is outputted by the improved Softmax calculation. The improved Softmax introduces the distillation temperature variable T on the original Softmax, and its calculation formula is: In the formula, represents the probability of the i-th category, j represents the j-th category among all categories, represents the original probability that the teacher model outputs the i-th category, Represents the original probability that the teacher model output is the jth class; In step S4, the student model is distilled and trained, and the total loss calculation formula of the distillation training is: Among them, γ represents the loss weighting coefficient, represents the distillation loss between the teacher model and the student model, Represents the classification loss between the output of the student model and the true label.

7. A photovoltaic panel defect recognition method based on knowledge distillation and pruning technology as described in any one of claims 1 to 5, characterized in that: In step S5, the constraint training is performed by adding L1 regularization to the Batch Normalization layer of the student model. The calculation formula of L1 regularization is: In the formula, w represents the weight vector of the model, i represents the i-th weight among n weights, and n represents the length of the weight vector, that is, the number of weights. Represents the error between the predicted value of the model and the true label; during the pruning operation in step S5, the matrix with values ​​close to zero in the weight matrix generated by L1 regularization is removed.

Images