A solar screen plate defect detection method based on a generative adversarial network

Abstract

The application discloses a solar screen plate defect detection method based on a generative adversarial network, which takes qualified images as a defect-free data set and real defect samples as a test set, generates artificial defect images from the qualified images to expand the defect samples, and transmits the artificial defect images into a generator as actual training images; the generator generates reconstructed images and transmits the reconstructed images and the defect-free samples into a discriminator for training; the generator and the discriminator are mutually opposed; the trained discriminator identifies defects of a to-be-tested image; the generator generates a reconstructed image, compares a gray scale difference between the reconstructed image and the to-be-tested image, and performs defect positioning, so that the solar screen plate defects are detected. The detection method provided by the application effectively solves the problems of insufficient real defect samples and long time consumption of manual label marking, improves the generalization capability of the model for unknown defect types, thereby improving the defect detection efficiency and accuracy and reducing the missed detection rate.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of defect detection, and particularly relates to a solar screen plate defect detection method based on a generative adversarial network. BACKGROUND

[0002] With the proposal of the strategic objectives of 'carbon peak' and 'carbon neutrality' in China, the application of clean energy represented by solar energy is becoming increasingly widespread. Solar photoelectric power generation refers to a power generation mode in which light energy is directly converted into electric energy without a thermal process, photovoltaic power generation is one of the main ways to realize solar power generation, and a solar cell is a key component of a photovoltaic power generation system; batch manufacturing of the solar cell is mostly performed by using a solar screen plate as a mold, and the quality of the solar screen plate is an important factor affecting the photoelectric conversion efficiency and service life of the solar cell.

[0003] At present, defect detection of the solar screen plate is mainly performed by human eye observation, and the accuracy and work efficiency are low, and the cost is high. By using machine vision technology, the defect detection and identification of the solar screen plate can be completed by using an industrial camera and computer software. However, the traditional machine vision detection has high requirements for the detection environment, and has the defects of low fault tolerance, poor compatibility, and high missing detection rate for unknown defects, which leads to the decrease of the accuracy of the detection result, thereby affecting the reliability of the product. By using a defect detection method based on deep learning, high detection accuracy can be achieved and the detection environment can be adapted to be more complex, but the network model generally needs a large amount of labeled data set for training, and the labeling of the label needs to consume more manpower and time. Therefore, it is necessary to improve the existing solar screen plate defect detection method, improve the accuracy and work efficiency of the detection, so as to ensure the production quality of the solar screen plate and the cell, and reduce the production cost. SUMMARY

[0004] In view of the problems of high missing detection rate, low training data set construction, and low detection efficiency in the existing solar screen plate defect detection method, the application aims to provide a solar screen plate defect detection method based on a generative adversarial network (GAN).

[0005] The technical scheme adopted by the application to solve the above technical problems is:

[0006] In combination with Figures 1 to 4 , the method comprises the following steps:

[0007] Step 1, collecting a solar screen plate gray image, and setting the resolution of each image to a fixed value.

[0008] Step 2, screening the image in step 1, selecting a qualified screen plate image for image preprocessing to eliminate noise interference, taking the processed qualified image x as a defect-free data set, and taking a real defect image as a test set.

[0009] Step 3, adding artificial random defects to qualified images x and expanding the data volume, the qualified images x are automatically generated into multiple artificial defect images x with random defects by the artificial defect module M ～ , and the number of random defects is 4.

[0010] Further, the artificial defect module M generates artificial defect images x ～ The specific process is as follows:

[0011] Step 3.1, randomly generate four groups of cutting boxes, and the length and width of the cutting boxes are randomly selected respectively, and the length and width of each group of cutting boxes are not completely the same.

[0012] Step 3.2, each group of cutting boxes is cut from a random position in the qualified image x, and is pasted to any position of the qualified image x after being scaled by a random ratio, thereby generating an artificial defect image x ～ .

[0013] Step 3.3, for each qualified image x, steps 3.1 and 3.2 can be repeated multiple times to generate multiple artificial defect images x ～ , thereby expanding the data volume of the training samples.

[0014] Step 4, the artificial defect images x ～ generated in step 3 are used as actual training images, and an actual training data set is constructed; the artificial defect images x ～ in the actual training data set are input into the generator G of the GAN network model, and the generator G is trained, and the qualified images x and the output results of the generator G are compared; the qualified images x and the reconstructed images X generated by the generator G are used to train the discriminator, and through the mutual confrontation of the generator G and the discriminator D, the artificial defect images x ～ are restored to reconstructed images X highly similar to the qualified images x, and the discriminator D has the ability to identify images with defects and images without defects.

[0015] Further, the network structure of the generator G of the GAN network model is based on the U-Net network model structure, including one encoding path, multiple residual modules, and one decoding path; wherein the encoding path is composed of 6 convolutional layers and 5 down-sampling layers, each residual module includes 2 convolutional layers, and the decoding path is composed of 6 deconvolutional layers and 5 up-sampling layers; the network structure of the discriminator D of the GAN network model is composed of 6 convolutional layers and 5 down-sampling layers;

[0016] Further, the output layer of the generator G adopts a Tanh activation function; except for the output layer, each convolutional layer and deconvolutional layer in the encoding-decoding path of the generator G adopts Batchnorm normalization processing and a Mish nonlinear activation function, and the convolutional layer of the residual module adopts Batchnorm normalization processing and a LeakReLU nonlinear activation function;

[0017] Further, the output layer of the discriminator D adopts a Sigmoid activation function; except for the output layer, each convolutional layer of the discriminator D adopts Batchnorm normalization processing and a Mish nonlinear activation function.

[0018] Further, the specific process of training the GAN network model is as follows:

[0019] Step 4.1, inputting an artificial defect image x ～ Training the generator G network, the artificial defect image x ～ After the encoding path, output the features to the residual module, and then return to the image space through the decoding path, and output the reconstructed image X.

[0020] Step 4.2, connecting the feature information of the down-sampling layer of the encoding path and the up-sampling layer of the corresponding decoding path, so that the generator G can retain more image feature information.

[0021] Step 4.3, establishing a generation loss function L c , so that the reconstructed image X is closer to the qualified image x.

[0022] Step 4.4, inputting the reconstructed image X and the qualified image x to train the discriminator D network.

[0023] Step 4.5, fine-tuning the trained GAN network; wherein the fine-tuning adopts an Adam optimizer, the initial learning rate is set to 0.008, the learning rate is reduced to half of the original after every 100 training rounds, and the number of residual modules can be adjusted according to the depth of the required network.

[0024] Further, in order to enable the model to maximize the reconstruction of the qualified image x, an adversarial loss function L a is established by minimizing the output of the generator G and maximizing the output of the discriminator D, so as to ensure that the reconstructed image X generated by the generator G is close to the qualified image x, and the discriminator D can distinguish the reconstructed image X from the qualified image x.

[0025] Further, in step 4.3, in order to ensure that the generator G can fully obtain the pixel distribution of the qualified image, the distance between the pixels of the qualified image x and the reconstructed image X is calculated by using the L1 norm as the generation loss function L c .

[0026] Further, the overall loss function L of the GAN network model loss by the adversarial loss function L a and the generation loss function L c Two loss functions are weighted, and the network parameters of the generator G and the discriminator D are adjusted according to the value of the overall loss function L loss , improve the efficiency and accuracy of model training.

[0027] Step 5, the recognition and positioning of screen defects are performed.

[0028] Further, in step 5, the specific steps of the trained generator G and the discriminator D for defect recognition and positioning are:

[0029] Step 5.1, the discriminator D performs image-level defect discrimination on the test image z, and if the discrimination result is "defect", the test image z is sent to the generator G.

[0030] Step 5.2, the generator G generates a reconstructed image Z through the test image z, and the pixel points of the test image z and the reconstructed image Z are subtracted in gray value, to obtain n gray difference regions composed of adjacent points with gray difference, and each gray difference region contains m regions with the same gray difference.

[0031] Step 5.3, according to the results calculated in step 5.2, the ratio of the average gray difference of each gray difference region to the total gray value 255, and the ratio of the area value of the largest area region with the same gray difference in the region to the area value of the region are calculated, and the above two ratios are weighted and added to obtain the defect credibility value P n .

[0032] Step 5.4, for the defect credibility value P n obtained in step 5.3, set its threshold value to 0.6; the gray difference region with the defect credibility value P n greater than 0.6 is determined as a defect region and a positioning frame is drawn, and the defect recognition and positioning are completed.

[0033] Compared with the prior art, the beneficial effects of the present application are:

[0034] (1) The model proposed in the application belongs to an unsupervised learning model, and training samples do not need to be manually labeled with defect labels, only a small amount of real defect samples and defect-free samples are needed; the required defect samples for training can be automatically expanded through artificial defects, and real defect samples are used for testing, thereby effectively solving the problems of insufficient number of real defect samples and long time consumption of manual label labeling; in addition, the trained generator and discriminator network are directly used for defect recognition and positioning, and only one model needs to be trained to complete the detection of defects; through the above improvements, the defect detection method proposed in the application reduces the labor cost and improves the detection efficiency.

[0035] (2) The model proposed in the application uses artificial defects, and due to the randomness of the artificial defects, the generalization ability of the model to unknown defect types is improved; at the same time, the generator adopts a multi-layer network structure, and as the generator network level deepens, the overfitting degree of the discriminator is greatly reduced, thereby improving the accuracy of defect recognition and reducing the defect omission rate. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 for the solar screen defect detection method based on the generative adversarial network described in the application;

[0037] Figure 2 for the deep convolutional generative adversarial network structure described in the application;

[0038] Figure 3 for the residual module structure described in the application;

[0039] Figure 4 for the specific steps of the solar screen defect recognition and positioning described in the application. DETAILED DESCRIPTION

[0040] Referring to the accompanying Figures 1 to 4 A solar screen defect detection method based on a generative adversarial network, comprising the following steps:

[0041] Step 1, collect solar screen gray scale images, and set the resolution of each image to 512x512. Step 2, screen the images in step 1, select qualified screen images for image preprocessing to eliminate noise interference, and take the processed qualified images x as a defect-free data set, and real defect images as a test set.

[0042] Step 3, add artificial random defects to the qualified images x and expand the data volume, and the qualified images x automatically generate multiple artificial defect images x with random defects through an artificial defect module M ～ , and the number of random defects is 4.

[0043] Further, the artificial defect module M generates artificial defect images x ～The specific process is:

[0044] Step 3.1, randomly generate four groups of clipping frames, and the maximum length and width (pixel number) of each group of clipping frames are (10, 3), (5, 4), (20, 5), and (12, 12) respectively.

[0045] Step 3.2, each group of clipping frames is clipped from a random position in the qualified image x, and after random scaling, it is pasted to any position of the qualified image x, thereby generating an artificial defect image x. ～ .

[0046] Step 3.3, for each qualified image x, steps 3.1 and 3.2 can be repeated multiple times to generate multiple artificial defect images x. ～ , thereby expanding the data volume of the training sample.

[0047] Step 4, the artificial defect image x ～ generated in step 3 is used as an actual training image, and an actual training data set is constructed; the artificial defect image x ～ in the actual training data set is input into the generator G of the GAN network model, and the generator G is trained, and the qualified image x and the output result of the generator G are compared; the qualified image x and the reconstructed image X generated by the generator G are used to train the discriminator, and through the mutual confrontation of the generator G and the discriminator D, the artificial defect image x ～ is restored to a reconstructed image X highly similar to the qualified image x, and the discriminator D has the ability to identify images with defects and images without defects.

[0048] Further, the network structure of the generator G of the GAN network model is based on the U-Net network model structure, including one encoding path, multiple residual modules, and one decoding path; wherein the encoding path is composed of 6 convolutional layers and 5 down-sampling layers, each residual module includes 2 convolutional layers, and the decoding path is composed of 6 deconvolutional layers and 5 up-sampling layers; the network structure of the discriminator D of the GAN network model is composed of 6 convolutional layers and 5 down-sampling layers.

[0049] Further, the output layer of the generator G adopts a Tanh activation function; except for the output layer, each convolutional layer and deconvolutional layer in the encoding-decoding path of the generator G adopts Batchnorm normalization processing and a Mish nonlinear activation function, and the convolutional layer of the residual module adopts Batchnorm normalization processing and a LeakReLU nonlinear activation function.

[0050] Further, the output layer of the discriminator D adopts a Sigmoid activation function; except for the output layer, each convolutional layer of the discriminator D adopts Batchnorm normalization processing and a Mish nonlinear activation function.

[0051] Further, in combination with Figure 2 The specific process of training the GAN network model is as follows:

[0052] Step 4.1, input the artificial defect image x ～ Train the generator G network, the artificial defect image x ～ Output its features to the residual module through the encoding path, and return to the image space through the decoding path, and output the reconstructed image X.

[0053] Step 4.2, the feature information of the down-sampling layer of the encoding path is connected with the up-sampling layer of the corresponding decoding path, so that the generator G can retain more image feature information.

[0054] Step 4.3, according to the reconstructed image X output by step 4.1 and the qualified image x, establish a generation loss function L c Make the reconstructed image X closer to the qualified image x.

[0055] Step 4.4, input the reconstructed image X and the qualified image x to train the discriminator D network.

[0056] Step 4.5, fine-tune the trained GAN network; wherein the fine-tuning adopts the Adam optimizer, the initial learning rate is set to 0.008, the learning rate is reduced to half of the original after every 100 training rounds, and the number of residual modules is set to 10.

[0057] Further, in order to enable the model to maximize the reconstruction of the qualified image x, by minimizing the output of the generator G and maximizing the output of the discriminator D, an adversarial loss function L a Is established to ensure that the reconstructed image X generated by the generator G is close to the qualified image x, and the discriminator D can distinguish the reconstructed image X from the qualified image x, as shown in formula (1):

[0058]

[0059] Where P x represents the feature distribution of the qualified image, D(x) represents the discrimination result of the qualified image x, and D(X) represents the discrimination result of the reconstructed image X.

[0060] Further, in step 4.3, in order to ensure that the generator G can fully obtain the pixel distribution of the qualified image, the L1 norm is used to calculate the distance between the pixels of the qualified image x and the reconstructed image X as the generation loss function L c , as shown in formula (2):

[0061]

[0062] Further, the overall loss function Lloss by the adversarial loss function L a and the generation loss function L c The two loss functions are weighted and combined, and the network parameters of the generator G and the discriminator D are adjusted according to the value of the overall loss function L loss , which improves the efficiency and accuracy of model training, as shown in equation (3):

[0063] L loss = λ a L a + λ c L c (3)

[0064] where λ a and λ c are weight parameters that can be adjusted according to the training effect to adjust the influence of each part of the loss function on the model.

[0065] Step 5, identify and locate the screen defects.

[0066] Further, in step 5, the trained generator G and discriminator D are used for the specific steps of defect identification and positioning:

[0067] Step 5.1, the discriminator D performs image-level defect discrimination on the test image z, and if the discrimination result is "defective", the test image z is sent to the generator G.

[0068] Step 5.2, the generator G generates a reconstructed image Z from the test image z. The pixel points of the test image z and the reconstructed image Z are subtracted in gray value, and n gray difference regions composed of adjacent points with gray difference are obtained, and each gray difference region contains m regions with the same gray difference, as shown in equations (4) and (5):

[0069] θ n = Area | Z(i,j) - z(i,j) |, Z,z ∈ (0,255) (4)

[0070] η nm = Area_same (Z(i,j) - z(i,j) ), Z,z ∈ (0,255) (5)

[0071] where θ n represents the area of the nth gray difference region, η nm represents the area of the mth gray difference region in the nth gray difference region, Z(i,j) and z(i,j) represent the gray values of the corresponding pixel points (i,j) of the reconstructed image Z and the test image z, respectively, Area represents the area of the region composed of adjacent gray difference points, and Area_same represents the area of the region composed of points with the same gray difference.

[0072] Step 5.3, according to the results calculated in step 5.2, calculate the ratio of the average gray difference of each gray difference area to the total gray value 255, and the ratio of the area value of the area with the maximum gray difference in the area to the area value of the area, and then add the two ratios weightedly to obtain the defect credibility value P n As shown in formula (6):

[0073]

[0074] Wherein, ω, π are adjustable weight parameters.

[0075] Step 5.4, for the defect credibility value P obtained in step 5.3 n , set its threshold value to 0.6; the gray difference area with the defect credibility value P n greater than 0.6 is determined as a defect area and a positioning frame is drawn to complete defect recognition and positioning.

[0076] The content explained in the above specific embodiment should be understood as the specific embodiment only for more clearly illustrating the present application, and not for limiting the scope of the present application, after reading the present application, the various equivalent forms of the present application modified by the person skilled in the art all fall within the scope defined by the claims attached to the present application.

Claims

1. A solar plate defect detection method based on a generative adversarial network, characterized in that: The method comprises the following steps: Step 1, collect solar screen gray images, and set the resolution of each image to a fixed value; Step 2, screen the images in step 1, select qualified screen images for image preprocessing, eliminate noise interference, and take the processed qualified images x as a defect-free data set and real defect images as a test set; Step 3, adding artificial random defects to qualified images x and expanding the data volume, qualified images x automatically generate multiple artificial defect images x with random defects through the artificial defect module M ～ , and the number of random defects is 4; In step 3, the artificial defect module M generates an artificial defect image x ～ The specific process is as follows: Step 3.1, randomly generate four groups of clipping boxes, and randomly take values for the length and width of the clipping boxes, and the value range of the length and width of each group of clipping boxes is not completely the same; Step 3.2, each group of cut frames is cut from a random position in the qualified image x, and is pasted to an arbitrary position in the qualified image x after scaling by a random scale, thereby generating an artificial defect image x ～ ; Step 3.3, for each qualified image x, repeat steps 3.1, 3.2 multiple times to generate multiple artificial defect images x ～ , thereby expanding the data volume of the training samples; Step 4, generating an artificial defect image x from the artificial defect image x ～ as actual training images, and construct an actual training dataset; The artificial defect image x from the actual training dataset ～ The generator G of the GAN network model is input and trained. The output of the generator G is compared with that of a qualified image x. The discriminator is trained using the qualified image x and the reconstructed image X generated by the generator G. Through the adversarial interaction between the generator G and the discriminator D, the artificially defective image x is identified. ～ The reconstructed image X is restored to a highly approximate qualified image x, and the discriminator D is enabled to distinguish between defective and defect-free images; The network structure of the generator G of the GAN network model is based on the U-Net network model structure, and comprises one encoding path, multiple residual modules and one decoding path; wherein the encoding path is composed of six convolutional layers and five down-sampling layers, each residual module comprises two convolutional layers, and the decoding path is composed of six deconvolutional layers and five up-sampling layers; the network structure of the discriminator D of the GAN network model is composed of six convolutional layers and five down-sampling layers; The output layer of the generator G adopts a Tanh activation function; except the output layer, each convolutional layer and deconvolutional layer in the encoding-decoding path of the generator G adopts Batchnorm normalization processing and a Mish nonlinear activation function, and the convolutional layer of the residual module adopts Batchnorm normalization processing and a LeakReLU nonlinear activation function; The output layer of the discriminator D adopts a Sigmoid activation function; except the output layer, each convolutional layer of the discriminator D adopts Batchnorm normalization processing and a Mish nonlinear activation function; Step 5, screen and locate the defects of the screen.

2. The solar screen defect detection method based on the generative adversarial network according to claim 1, wherein: In step 4, the specific process of training the GAN network model is as follows: Step 4.1, input artificial defect image x ～ The generator G network is trained, and the artificial defect image x ～ The encoded path outputs its features to the residual module, and then returns to the image space through the decoding path and outputs the reconstructed image X. Step 4.2, connect the feature information of the down-sampling layer of the encoding path and the up-sampling layer of the corresponding decoding path, so that the generator G can retain more image feature information; Step 4.

3. Generating a loss function L based on the reconstructed image X output from Step 4.1 and the qualified image x c to make the reconstructed image X closer to the qualified image x; Step 4.4, input the reconstructed image X and the qualified image x to train the discriminator D network; Step 4.5, fine-tune the trained GAN network; wherein the fine-tuning adopts an Adam optimizer, the initial learning rate is set to 0.008, the learning rate is reduced to half of the original value after every 100 training rounds, and the number of residual modules can be adjusted according to the depth of the required network; To enable the model to maximize the reconstruction of the qualified image x, an adversarial loss function L is established by minimizing the output of the generator G and maximizing the output of the discriminator D a , to ensure that the reconstructed image X generated by the generator G is close to the qualified image x, and the discriminator D can distinguish the reconstructed image X from the qualified image x. In step 4.3, to ensure that the generator G can fully obtain the pixel distribution of the qualified image, the distance between the qualified image x and the reconstructed image X is calculated by using the L1 norm as the generation loss function L c ; The overall loss function L of the GAN network model loss The overall loss function L of the GAN network model a The overall loss function L of the GAN network model c The two loss functions are weighted and composed, and the network parameters of the generator G and the discriminator D are adjusted according to the value of the overall loss function L loss The efficiency and accuracy of model training are improved.

3. The solar screen defect detection method based on the generative adversarial network according to claim 1, wherein: In step 5, the specific steps of the trained generator G and discriminator D for defect identification and positioning are as follows: Step 5.1, the discriminator D performs image-level defect identification on the test image z, and if the identification result is "defective", the test image z is input into the generator G; Step 5.2, the generator G generates a reconstructed image Z from the test image z, and the pixel points of the test image z and the reconstructed image Z are subtracted to obtain n gray difference regions composed of adjacent points with gray differences, and each gray difference region contains m regions with the same gray difference. Step 5.

3. According to the result calculated in step 5.2, the ratio of the average gray scale difference of each gray scale difference area to the total gray scale value 255 and the ratio of the area value of the area with the largest gray scale difference in the area to the area value of the area are calculated, and the two ratios are weighted and added to obtain the defect credibility value P n ; Step 5.4, for the defect reliability value P obtained in step 5.3 n , set its threshold value to 0.6; The defect credibility value P n The gray difference area greater than 0.6 is determined as a defect area and a positioning frame is drawn, and defect recognition and positioning are completed.

Images