A method for generating defects in solar panels based on generative adversarial networks

By using the multi-attention module generation adversarial network and mixup algorithm in solar panel detection, the overfitting problem caused by insufficient sample size is solved, the detection efficiency and accuracy are improved, and high-quality image generation and model performance are achieved. promote.

CN113935977BInactive Publication Date: 2024-11-19HEBEI UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111234513.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-22
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In solar panel defect detection, due to insufficient sample size, deep learning models are prone to overfitting, resulting in low detection efficiency and accuracy. Traditional data augmentation methods like similar enhancement have limitations and cannot completely solve the problem of small samples.

Method used

A solar panel defect generation method based on a generative adversarial network was designed, and an improved generative adversarial network with multiple attention modules was adopted, and combined with the mixup algorithm, the image details were generated and fused through random noise, and the image details was improved. Solve the problem of model collapse.

Benefits of technology

Through the combination of improved generative adversarial network and mixup algorithm, the quality of image generation and detail clarity are significantly improved, the performance of the model is improved, the overfitting problem caused by insufficient sample size is solved, and the detection efficiency and accuracy are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113935977B_ABST
    Figure CN113935977B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for generating solar panel defects based on a generative adversarial network. The method uses a self-designed improved generative adversarial network containing a multi-attention module to train the network using a solar defect image data set. The images in the data set can all be resized to the same size, and a number of images are input as a batch into the generator and discriminator in the generative adversarial network at the corresponding stage of the training process for training. When the loss function value of the discriminator tends to be balanced, the training is terminated, and the generator finally obtained can generate high-quality images. The improved generative adversarial network designed by the present invention can improve the problem of mode collapse and improve the clarity of image details.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and relates to a method for generating defects of a solar panel based on a generative adversarial network. Background Art

[0002] The sun can continuously deliver abundant, clean, and free energy to the earth. How to effectively utilize solar energy has become an important direction of human research. The main carrier of solar power generation is the solar panel. Most solar panels are made of crystalline silicon materials because of the relatively low equipment cost, high photoelectric conversion efficiency, and the suitability for outdoor power generation under the sun. However, crystalline silicon slices are usually relatively thin, which can easily cause defects such as broken grids, open welding, solid black, and hidden cracks. At present, most of them adopt the battery array installation method, and a defective solar panel will seriously affect the overall performance. Therefore, it is very important to detect solar panel defects during the production process.

[0003] In the past, most solar cell manufacturers still detected defective solar panels through manual observation. This method has many problems, such as low manual detection efficiency and easy misjudgment. In recent years, with the increasing maturity of computer technology, more and more solar panel manufacturers have begun to use machine vision technology to replace manual detection technology to reduce labor costs and improve the detection efficiency and accuracy of solar panel defects. However, traditional machine learning is far from enough. The emergence of a new type of feature learning method has rapidly made breakthrough progress in many fields. This type of feature representation method is based on artificial neural networks, mainly using deep neural network models, combined with new training methods, called deep learning, but the advantage of deep learning depends on big data. Driven by big data, machines can learn effectively. If there are not enough training samples, neural networks with many parameters will have the risk of overfitting and show very poor results on test images. However, when studying solar panel defects, there is often a problem of small sample size. Too small a sample size will lead to low sample diversity, and the image data set is not enough for the model to capture enough data. Therefore, when the amount of data is limited, data enhancement can be used to improve sample diversity. Data augmentation refers to the expansion of data or feature enhancement of the original small data set with the help of auxiliary data or auxiliary information.

[0004] In early work, fixed transformation rules are usually applied to existing data sets to generate new samples. This can be done around a certain sample, called homogeneous enhancement, such as shift, rotation, scaling, cropping, flipping, etc., or it can be done on multiple samples at the same time, called mixed-class enhancement. However, artificially constructing these fixed data transformation rules requires a lot of knowledge in related fields. At the same time, these transformation rules are not general, and some only perform well when acting on specific data sets. Therefore, the use of fixed transformation rules cannot completely solve the problem of few samples. A more advanced and effective method is to use generative models to generate new samples or generate new features. Generative models refer to generating expected data distribution samples based on the data distribution of training samples by training the data set. At present, the most representative generative model is the generative adversarial network, because the generative adversarial network has significant advantages in the field of image generation and has also made rapid development in recent years. It can provide more generated images for training samples and increase the diversity of training samples. The generative adversarial network consists of a generator and a discriminator. The generator generates images by inputting noise, and the discriminator needs to classify the generated images and the real images into two categories. The two compete with each other until the discriminator can no longer correctly distinguish the difference between the images generated by the generator and the real images, thereby producing excellent output.

[0005] Previous generative adversarial networks were prone to mode collapse, with single sample modes lacking diversity. Specifically, generative adversarial networks use adversarial training to bring the data distribution of images generated by the generator closer to the data distribution of real images, and use real image data to train the discriminator to obtain a distribution of image data close to the real one. The original generative adversarial network technology describes the distance between two image data distributions as cross entropy, or JS divergence. When the two distributions do not intersect at the beginning of training, the training results using JS divergence are poor, and cannot provide effective gradients for the training of the generator, resulting in mode collapse. Summary of the invention

[0006] The purpose of the present invention is to provide a method for generating solar panel defects based on a generative adversarial network. The method is based on a generative adversarial network and incorporates a multi-attention module in the network to solve the problem that it is difficult to collect enough data to train the model for solar panel defects, resulting in overfitting of the deep network and substandard model performance.

[0007] To achieve the above object, the present invention provides the following technical solution: a method for generating defects of solar panels based on a generative adversarial network is designed, characterized in that the method comprises the following steps:

[0008] S1: Collect solar panel image samples and divide them into qualified samples and defective samples, and select defective sample images as training set;

[0009] S2: Design an improved generative adversarial network with multiple attention modules, and use random noise variables and training set images to train the parameters of the improved generative adversarial network;

[0010] S3: Input random noise into the improved generative adversarial network containing a multi-attention module whose parameters have been trained in step S2 to generate a solar panel defect image (A);

[0011] S4: The solar panel defect image (A) generated in step S3 is superimposed and fused with the training set image using a mixup algorithm to make the background grid lines of the solar panel clearer and the defects more diverse, thereby generating a high-quality defect image (B).

[0012] Compared with the prior art, the beneficial effect of the present invention is that: the network is trained using a solar defect image dataset through a self-designed improved generative adversarial network containing a multi-attention module. The images in the dataset can all be resized to the same size, and a number of images are input as a batch into the generator and discriminator in the generative adversarial network for training at the corresponding stage of the training process. When the loss function value of the discriminator tends to be balanced, the training is terminated, and the final generator can generate high-quality images. The improved generative adversarial network designed by the present invention can improve the problem of mode collapse and improve the clarity of image details. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0014] Figure 1 The principle and structural block diagram of an improved generative adversarial network including a multi-attention module according to an embodiment of the method of the present invention;

[0015] Figure 2 This is a schematic diagram of the principle of superimposing and fusing a solar panel defect image A generated by an embodiment of the method of the present invention and a training set image (real defect image) using a mixup algorithm;

[0016] Figure 3 This is a principle and structural block diagram of the self-attention layer and channel attention layer of an embodiment of the method of the present invention. DETAILED DESCRIPTION

[0017] The present invention is further described below in conjunction with specific embodiments.

[0018] The present invention provides a method for generating defects of a solar panel based on a generative adversarial network, the method comprising the following steps:

[0019] S1: Collect solar panel image samples and divide them into qualified samples and defective samples, and select defective sample images as training sets.

[0020] S2: Design an improved generative adversarial network with multiple attention modules, and use random noise variables and training set images to train the parameters of the improved generative adversarial network.

[0021] The generator of the improved generative adversarial network includes seven neural network modules, and the discriminator includes five neural network modules. After initializing the generator and the discriminator, the random noise variable z is input into the first neural network module of the generator, and is processed by the seven neural network modules of the generator in turn to generate the first generation of false defect pictures; the first generation of false defect pictures and training set pictures are input into the discriminator, and are processed by the five neural network modules of the discriminator in turn to judge the difference between the initial false defect pictures and the training set pictures. The discriminator uses Wasserstein distance as the loss function, and then writes the Wasserstein distance into a solvable form through mathematical transformation, and uses a discriminator neural network with a limited parameter value range to maximize this form to achieve approximate fitting of Wasserstein distance. sserstein distance, at this time the parameter training of the discriminator is completed; under this approximately optimal discriminator, the generator is optimized to reduce the Wasserstein distance, which can effectively bring the generated distribution closer to the real distribution. At this time, the parameters of the discriminator are kept unchanged, and the parameters of the generator are adjusted. The random noise variable z passes through the generator to generate the second-generation false defect pictures, and the second-generation false defect pictures and the training set pictures are input into the discriminator. Finally, the discriminator calculates the difference. When the loss function Wasserstein distance is the smallest, the generator parameter training is completed, and the second-generation false defect pictures are the network outputs that meet the requirements. Otherwise, the parameters of the generator are continued to be adjusted until the false defect pictures obtained by the current parameters have the smallest loss function Wasserstein distance after being processed by the discriminator.

[0022] The Wasserstein distance is defined as follows:

[0023]

[0024] II(P1,P2) is the set of all possible joint distributions of P1 and P2. For each possible joint distribution γ, a real sample x and a generated sample y can be sampled from it, and the distance between the two samples ||xy|| can be calculated. Therefore, the expected value of the distance between the samples under the joint distribution γ can be calculated. The lower bound of this expected value in all possible joint distributions is defined as the Wasserstein distance. Then, the Wasserstein distance is written into a solvable form through mathematical transformation, and a discriminator neural network with a limited parameter value range is used to maximize this form, which is to impose an additional restriction on a continuous function f, that is, to limit the maximum local variation of a continuous function with the Lipschitz continuity condition. Therefore, a discriminator network fw with parameter w is constructed, and the calculation formula of L is as follows:

[0025]

[0026] Under the condition that the Wasserstein distance is limited to a certain range, L is maximized as much as possible. At this time, L will approximate the Wasserstein distance between the true distribution and the generated distribution. Under this approximate optimal discriminator, the generator is optimized to reduce the Wasserstein distance, which can effectively bring the generated distribution closer to the true distribution.

[0027] The first neural network module G1 of the generator is a fully connected layer, which is a fully connected structure used for feature extraction of all nodes; the second neural network module G2 includes a fully connected layer, a BN layer (batch normalization layer) and a Leaky ReLU activation function. Each node of the fully connected layer is connected to all nodes of the previous layer to integrate the features extracted from the previous layer. The BN layer adjusts the features processed by the fully connected layer to a standard normal distribution with a mean of 0 and a variance of 1. The Leaky ReLU activation function gives a non-zero slope to all negative values ​​obtained after processing by the BN layer.

[0028] The third neural network module G3 of the generator is the self-attention layer. The self-attention layer is to improve the correlation between the pixel areas of the generated image. It performs a 1×1 convolution operation on the feature map extracted from the previous layer, performs dimensionality reduction, and converts the feature map into three specific feature spaces f, g, and h. The transpose of f(x) is matrix multiplied with g(x), and normalized by the softmax activation function to calculate the attention value. Then, the self-attention map is obtained by dot multiplication with h(x), and then weighted with the initial feature map to obtain the final output feature map y under the self-attention mechanism. The calculation formula is as follows:

[0029]

[0030] The fourth neural network module G4 of the generator is the channel attention layer, which focuses on the meaningful input image. It passes the result of the feature map y of the previous layer through the average pooling layer (AvgPool) and the maximum pooling layer (MaxPool) to a shared network (share MLP) to generate a channel attention map, and then multiplies the feature map y element by element to obtain the final attention map.

[0031] The channel attention map generation formula is as follows:

[0032] M c (y)=σ(MLP(AvgPool(y))+MLP(MaxPool(y))) (4)AvgPool(y) represents the average pooling feature, and MaxPool(y) represents the maximum pooling feature. Then, the results after passing through the average pooling layer and the maximum pooling layer are passed to a shared network (share MLP) to generate a channel attention map. The shared network consists of a multi-layer perceptron with one hidden layer. The output feature vectors are then merged using element-wise summation (corresponding elements one by one). σ represents the sigmoid function. Finally, the channel attention map Mc(y) is element-wise multiplied with the feature map y to obtain the final attention map.

[0033] The fifth neural network module G5 of the generator includes a fully connected layer, a BN layer (batch normalization layer) and a LeakyReLU activation function. The fully connected layer connects each node to all nodes in the previous layer and integrates the features extracted from the previous layer. The BN layer adjusts the features processed by the fully connected layer to a standard normal distribution with a mean of 0 and a variance of 1. The Leaky ReLU activation function gives a non-zero slope to all negative values ​​processed by the BN layer.

[0034] The sixth neural network module G6 of the generator includes a fully connected layer, a BN layer (batch normalization layer) and a LeakyReLU activation function. The fully connected layer connects each node to all nodes in the previous layer and integrates the features extracted from the previous layer. The BN layer adjusts the features processed by the fully connected layer to a standard normal distribution with a mean of 0 and a variance of 1. The Leaky ReLU activation function gives a non-zero slope to all negative values ​​obtained after processing by the BN layer, so that the neural network can arbitrarily approximate any nonlinear function.

[0035] The seventh neural network module G7 of the generator includes a fully connected layer and a tanh activation function. The tanh activation function is a hyperbolic tangent function. The output of the tanh function is centered at 0 and has a range of [-1, 1].

[0036] The role of the BN layer is to increase the training speed, prevent overfitting, and prevent the activation function from entering the nonlinear saturation region, thereby causing the gradient diffusion problem. The BN layer is added before the activation function to normalize the input of the activation function, which solves the impact of input data offset and increase. The role of the activation function is to introduce nonlinear factors to neurons, so that the neural network can arbitrarily approximate any nonlinear function and learn more complex mapping relationships. The added Leaky ReLU activation function gives a non-zero slope to all negative values;

[0037] The first neural network module D1 of the discriminator includes a fully connected layer and a Leaky ReLU activation function. The fully connected layer extracts features from all nodes, and the Leaky ReLU activation function assigns a non-zero slope to all negative values ​​in the features extracted by the fully connected layer. The second neural network module D2 includes a fully connected layer and a Leaky ReLU activation function. Each node of the fully connected layer is connected to all nodes of the previous layer, which is used to integrate the features extracted previously, and then pass through the Leaky ReLU activation function to assign a non-zero slope to all negative values ​​in the extracted features. The third neural network module D3 is a self-attention layer, and the structure of this network module is the same as that of the third neural network module of the generator. The fourth neural network module D4 is a channel attention layer, and the structure of this network module is the same as that of the fourth neural network module of the generator. The fifth neural network module D5 is a fully connected layer, which further integrates the features extracted from the previous layer.

[0038] S3: Input the random noise into the improved generative adversarial network including the multi-attention module whose parameters have been trained in step S2 to generate a solar panel defect image A.

[0039] S4: The solar panel defect image A generated in step S3 is superimposed and fused with the training set image using a mixup algorithm to make the background grid lines of the solar panel clearer and the defects more diverse, thereby generating a high-quality defect image B.

[0040] Mixup is an algorithm used in computer vision to enhance mixed-class images. It can mix images of different types to expand the training data set. That is, the defect image A is generated and mixed and superimposed with the real defect image, and the original image is used to make up for the problem of uneven grid lines in the generated image. After superposition, the defect positions are more diverse.

[0041] The software environment of the experiment built in the embodiment is a learning framework with pytorch as the backend under the Win10 system, and the experimental hardware environment built is an NVIDIA 2070s graphics card.

[0042] The generated data of this experiment is judged by two evaluation indicators, SSIM and PSNR. SSIM is structural similarity, which is also a full-reference image quality evaluation indicator. It measures image similarity from three aspects: brightness, contrast, and structure. PSNR is peak signal-to-noise ratio, which is the most common and widely used objective image evaluation indicator. It is based on the error between corresponding pixels, that is, based on error-sensitive image quality evaluation.

[0043] Taking the linear defects of solar panels as an example, the experimental data set is 52 pictures. After the generative adversarial network generates 6000 pictures, the comparative experimental results are as follows: the evaluation index SSIM of the pictures generated by the original generative adversarial network (GAN) is 0.187, and the PSNR is 8.621; the evaluation index SSIM of the pictures generated by the WGAN network model is 0.461, and the PSNR is 14.386; the evaluation index SSIM of the pictures generated by adding the self-attention mechanism to the generator and discriminator of the WGAN network model and performing the mixup algorithm to fuse the true and false is 0.598, and the PSNR is 17.973; The evaluation index SSIM of the pictures generated by adding the self-attention mechanism to the generator and discriminator of the GAN network model is 0.638, and the PSNR is 19.339; the evaluation index SSIM of the pictures generated by adding the multi-attention module combining the self-attention mechanism and the channel attention mechanism to the generator and discriminator of the WGAN network model is 0.671, and the PSNR is 19.556; the evaluation index SSIM of the pictures generated by adding the multi-attention module combining the self-attention mechanism and the channel attention mechanism to the generator and discriminator of the WGAN network model and performing the mixup algorithm for true and false fusion is 0.72, and the PSNR is 20.086.

[0044] Experimental results show that the improved generative adversarial network designed in the present invention has greatly improved the generation effect of solar panel defects, proving the feasibility of this invention.

[0045] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A method for generating defects in solar panels based on a generative adversarial network, characterized in that: The method comprises the following steps: S1: Collect solar panel image samples and divide them into qualified samples and defective samples, and select defective sample images as training set; S2: Design an improved generative adversarial network with multiple attention modules, and use random noise variables and training set images to train the parameters of the improved generative adversarial network; The generator of the improved generative adversarial network includes seven neural network modules; the third neural network module of the generator is the self-attention layer, which is used to improve the correlation between pixel areas of the generated image. It performs a 1×1 convolution operation on the feature map extracted from the previous layer, performs dimensionality reduction, converts the feature map into three specific feature spaces f, g and h, and performs matrix multiplication of the transpose of f(x) and g(x). It is normalized by the softmax activation function to calculate the attention value, and then the self-attention map is obtained by point multiplication h(x), and then the final output feature map (y) under the self-attention mechanism is obtained by weighting with the initial feature map. The calculation formula is as follows: The fourth neural network module of the generator is the channel attention layer, which focuses on the meaningful input image. It passes the result of the feature map (y) of the previous layer through the average pooling layer and the maximum pooling layer to a shared network to generate a channel attention map, which is then element-wise multiplied with the feature map (y) to obtain the final attention map. The channel attention map generation formula is as follows: Mc(y)=σ(MLP(AvgPool(y))+MLP(MaxPool(y)))4 AvgPool(y) represents the average pooling feature, and MaxPool(y) represents the maximum pooling feature. Then, the results after the average pooling layer and the maximum pooling layer are passed to a shared network to generate a channel attention map. The shared network consists of a multi-layer perceptron with one hidden layer. Then, the output feature vectors are merged using element-wise summation. σ represents the sigmoid function. Finally, the channel attention map Mc(y) is element-wise multiplied with the feature map (y) to obtain the final attention map. S3: Input random noise into the improved generative adversarial network containing a multi-attention module whose parameters have been trained in step S2 to generate a solar panel defect image (A); S4: The solar panel defect image (A) generated in step S3 is superimposed and fused with the training set image using a mixup algorithm to make the background grid lines of the solar panel clearer and the defects more diverse, thereby generating a high-quality defect image (B).

2. A method for generating defects in solar panels based on a generative adversarial network according to claim 1, characterized in that: The discriminator includes five neural network modules. After initializing the generator and the discriminator, the random noise variable (z) is input into the first neural network module of the generator, and processed by the seven neural network modules of the generator in turn to generate the first generation of false defect images; the first generation of false defect images and training set images are input into the discriminator, and processed by the five neural network modules of the discriminator in turn to judge the difference between the initial false defect images and the training set images. The discriminator uses Wasserstein distance as the loss function, and then writes the Wasserstein distance into a solvable form through mathematical transformation. A discriminator neural network with a limited parameter value range is used to maximize this form to achieve an approximate fitting of the Wasserstein distance. When the parameter training of the discriminator is completed; under this approximately optimal discriminator, the generator is optimized to reduce the Wasserstein distance, which can effectively bring the generated distribution and the real distribution closer. At this time, the parameters of the discriminator are kept unchanged, and the generator parameters are adjusted. The random noise variable (z) passes through the generator to generate the second-generation false defect pictures. The second-generation false defect pictures and the training set pictures are input into the discriminator. Finally, the discriminator calculates the difference. When the loss function Wasserstein distance is the smallest, the generator parameter training is completed, and the second-generation false defect pictures are the network outputs that meet the requirements. Otherwise, the generator parameters are adjusted until the false defect pictures obtained by the current parameters have the smallest loss function Wasserstein distance after being processed by the discriminator.

3. The method for generating solar panel defects based on a generative adversarial network according to claim 2, characterized in that: The first neural network module of the generator is a fully connected layer, which is a fully connected structure used for feature extraction of all nodes; the second neural network module includes a fully connected layer, a BN layer and a Leaky ReLU activation function. Each node of the fully connected layer is connected to all nodes of the previous layer to integrate the features extracted from the previous layer. The BN layer adjusts the features processed by the fully connected layer to a standard normal distribution with a mean of 0 and a variance of 1. The Leaky ReLU activation function gives a non-zero slope to all negative values ​​obtained after processing by the BN layer. The fifth neural network module of the generator includes a fully connected layer, a BN layer, and a Leaky ReLU activation function. The fully connected layer connects each node to all nodes in the previous layer and combines the features extracted from the previous layer. The BN layer adjusts the features processed by the fully connected layer to a standard normal distribution with a mean of 0 and a variance of 1. The Leaky ReLU activation function assigns a non-zero slope to all negative values ​​processed by the BN layer. The sixth neural network module of the generator includes a fully connected layer, a BN layer, and a Leaky ReLU activation function. The fully connected layer connects each node to all nodes in the previous layer and combines the features extracted from the previous layer. The BN layer adjusts the features processed by the fully connected layer to a standard normal distribution with a mean of 0 and a variance of 1. The Leaky ReLU activation function assigns a non-zero slope to all negative values ​​processed by the BN layer, so that the neural network can arbitrarily approximate any nonlinear function. The seventh neural network module of the generator includes a fully connected layer and a tanh activation function. The tanh activation function is a hyperbolic tangent function. The output of the tanh function is centered at 0 and has a range of [-1, 1].

4. The method for generating solar panel defects based on a generative adversarial network according to claim 2, characterized in that: The first neural network module of the discriminator includes a fully connected layer and a Leaky ReLU activation function. The fully connected layer extracts features from all nodes, and the Leaky ReLU activation function assigns a non-zero slope to all negative values ​​in the features extracted by the fully connected layer. The second neural network module includes a fully connected layer and a Leaky ReLU activation function. Each node of the fully connected layer is connected to all nodes of the previous layer, which is used to integrate the features extracted previously, and then pass through the Leaky ReLU activation function to assign a non-zero slope to all negative values ​​in the extracted features. The third neural network module is a self-attention layer, and the structure of this network module is the same as that of the third neural network module of the generator. The fourth neural network module is a channel attention layer, and the structure of this network module is the same as that of the fourth neural network module of the generator. The fifth neural network module is a fully connected layer, which further integrates the features extracted from the previous layer.