Single-stage photovoltaic module surface defect detection method based on improved RetinaNet

Abstract

The invention discloses a single-stage photovoltaic module surface defect detection method based on improved RetinaNet, belongs to the technical field of image recognition, and changes a network feature fusion method by optimizing a feature pyramid structure and a residual network structure of the RetinaNet. A bidirectional fusion structure from top to bottom and a bidirectional fusion structure from bottom to top are used for replacing an original unidirectional feature fusion structure, and the bidirectional feature fusion structure improves the capability of extracting small target information in different stages of the network; in order to suppress background noise and improve the feature extraction capability, a channel attention mechanism module ECA is added in a residual structure, the improved residual block improves the feature extraction capability of the network and enhances the attention to small targets, and therefore the optimized network shows excellent performance in photovoltaic module surface defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a single-stage photovoltaic module surface defect detection method based on improved RetinaNet. Background Art

[0002] Surface defect detection of photovoltaic modules is a crucial part of the photovoltaic industry, which directly affects the performance and long-term stable operation of photovoltaic power generation systems. As existing large-scale photovoltaic power stations are often located in harsh environments, solar photovoltaic modules may have problems such as cracks and hot spots. External environmental factors may cause cracks to expand, thereby aggravating the heating of the entire photovoltaic module. The presence of hot spots and cracks will significantly reduce the heating efficiency and may even cause serious harm and huge economic losses. Therefore, with the rapid development of new energy today, surface defect detection of solar photovoltaic modules has become extremely important.

[0003] The photovoltaic module surface defect detection method based on image processing and computer vision technology has been widely used. By collecting image data on the surface of photovoltaic modules and analyzing and processing these data using image processing technology, cracks, stains, ablation and other defects on the surface of photovoltaic modules can be effectively detected. This method has the characteristics of non-contact and high efficiency, and has become one of the mainstream methods for photovoltaic module surface defect detection. However, due to the uncertainty of defect types, different defects require different parameters to be adjusted, so this method cannot be widely used in most photovoltaic module surface defect detection occasions.

[0004] As machine learning and artificial intelligence technologies are increasingly used in photovoltaic module surface defect detection, machine learning algorithms can be used to train and learn a large amount of photovoltaic module data, thereby achieving automatic recognition and classification of photovoltaic module surface defects. The development of deep learning technology has further improved the accuracy and efficiency of photovoltaic module surface defect detection, making the detection results more accurate and reliable. For example, Chinese patent CN202210528398.0 discloses a two-stage photovoltaic module surface defect detection based on deep learning, including the following steps: (1) collecting original photovoltaic module images and making a sample data set; (2) dividing the sample data set to construct a training sample set, a test sample set and a verification sample set; (3) performing data enhancement preprocessing on the sample data set; (4) using the enhanced training sample set and the verification sample set to train the target detection YOLOv5 network model; (5) the target detection YOLOv5 network model includes a foreground extraction model and a defect detection model, and the output of the foreground extraction model is the input of the defect detection model; (6) inputting the enhanced test sample set into the trained target detection YOLOv5 network model, and detecting the output through the foreground extraction stage and the defect detection stage to obtain the defect detection result; although this method can effectively detect the surface defect area of ​​the photovoltaic module, the recognition accuracy of small target defects is low.

[0005] In view of this, the present invention proposes a single-stage photovoltaic module surface defect detection method based on improved RetinaNet.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The following steps are involved:

[0008] A1. Use industrial cameras to collect PV module images and create a PV module image dataset;

[0009] A2. Divide the sample data set into training set, test set and validation set;

[0010] A3. Perform data enhancement preprocessing on the sample data set;

[0011] A4. Based on the enhanced defective photovoltaic module image, the surface defect detection is performed through the RetinaNet deep learning network model to obtain the detection results. The improved RetinaNet single-stage target detection network includes a feature extraction network and a bidirectional feature fusion structure;

[0012] A5. Input the enhanced test sample set into the trained target detection RetinaNet network model, detect it through the feature extraction stage and feature fusion stage, and finally output it through the classification and regression module to obtain the defect detection result.

[0013] Preferably, in step A1, collecting original photovoltaic module images and preparing a sample data set includes the following steps:

[0014] A11. Use photovoltaic module surface defect detection machinery and the industrial camera on board to photograph photovoltaic modules and collect original images of photovoltaic module surface defects;

[0015] A12. Annotate the collected original images, use different defect labels to annotate small units and defects, and generate an .xml document;

[0016] A13. Bind the generated .xml document containing the annotation information and the original image and save them in a txt file as a sample data set.

[0017] Preferably, in step A3, the data enhancement preprocessing uses random brightness enhancement, image inversion, cropping and splicing methods to perform data enhancement on the labeled image to achieve expansion of the sample data set.

[0018] Preferably, in step A4, the target detection RetinaNet network model is trained using the enhanced training sample set and the verification sample set, and the specific steps are: inputting the enhanced training sample set into the improved RetinaNet network model, iteratively training the defect detection model in turn, and saving the trained network parameters; using the verification sample set to adjust the parameters of the trained defect detection model, and finally obtaining the defect detection RetinaNet model weights.

[0019] The specific steps of the defect detection stage are as follows: the enhanced photovoltaic module image is input into the defect detection RetinaNet model, firstly, the input image is feature extracted, and after passing through the PAN bidirectional feature fusion structure, it enters the classification module and the regression module to obtain the training result, outputs the candidate box, and then uses the maximum suppression algorithm to screen the candidate box to obtain the final defect detection result, and the defect detection result is marked in the original photovoltaic module image.

[0020] Preferably, in step A4, the improved RetinaNet network model is obtained by optimizing the existing RetinaNet network model, comprising the following steps:

[0021] A41. Obtain the loss function of the existing RetinaNet network model;

[0022] A42, access the bottom-up pathway in the existing RetinaNet network model feature fusion FPN structure to form a bidirectional feature fusion result;

[0023] A43, improve the ResNet structure in the feature extraction network, add the ECA attention mechanism, strengthen the ability to transmit deep semantic information, and improve the accuracy of small target detection;

[0024] A44. Update the existing RetinaNet network parameters to reduce the total loss function, and continuously iterate this process until the network converges to obtain the target detection RetinaNet network model.

[0025] Preferably, the loss function L of the existing RetinaNet network model is Z It consists of two parts: classification loss function L cls And the regression loss function L reg Add and we get

[0026] L Z =L cls +L reg

[0027] The classification loss uses FocalLoss as the loss function to calculate the classification loss of all positive and negative samples. The regression loss uses smooth L1 loss as the loss function to calculate the regression loss of all positive samples.

[0028]

[0029] The beneficial effects of the present invention are:

[0030] 1. In the present invention, the single-stage photovoltaic module surface defect detection method based on improved RetinaNet improves the network's ability to extract small target information by improving the RetinaNet feature fusion mechanism and introducing a bidirectional feature fusion path;

[0031] 2. In the present invention, the single-stage photovoltaic module surface defect detection method based on improved RetinaNet improves the ResNet structure of RetinaNet and uses the ECA attention mechanism to enhance the deep semantic information transmission capability and improve the small target detection accuracy;

[0032] 3. In the present invention, the single-stage photovoltaic module surface defect detection method based on improved RetinaNet, compared with the traditional method of detecting defects by human eyes, uses a single-stage target detection network to realize automatic detection of photovoltaic module surface defects, and can still achieve good detection effects for small targets, greatly improving the recognition accuracy and detection efficiency of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1A flow chart of a single-stage photovoltaic module surface defect detection method based on improved RetinaNet proposed by the present invention;

[0034] Figure 2 This is a network structure diagram of a single-stage photovoltaic module surface defect detection method based on improved RetinaNet proposed by the present invention;

[0035] Figure 3 The network structure diagram of the ECA attention mechanism introduced in the present invention;

[0036] Figure 4 This is the improved ResNet network structure diagram of the present invention. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0038] See also Figures 1 to 3 The present invention discloses a single-stage photovoltaic module surface defect detection method based on improved RetinaNet.

[0039] A single-stage photovoltaic module surface defect detection method based on improved RetinaNet includes the following steps:

[0040] A1. Collect original images of photovoltaic modules and build a sample data set;

[0041] The specific steps are as follows:

[0042] A11. Use the industrial camera above the photovoltaic module surface defect detection mechanical equipment to photograph the solar photovoltaic module to obtain its original image;

[0043] A12. Use the labeling software labelImg to label the original image. Label the different types of defects on the surface of the photovoltaic module and finally save it as a .xml format document;

[0044] In order to improve the accuracy and robustness of the target detection algorithm, a large number of image data sets are required for annotation. The annotation process is: use annotation software to open the collected original image, use a rectangular box to mark the surface defects of the photovoltaic module, and name the rectangular box as the defect type. After the annotation is completed, save it as an .xml format document. Each document corresponds to an image, and the document contains the category information ki of each defect, the horizontal and vertical coordinates xi and yi of the center point of the rectangular box, and the width and height wi and hi of the rectangular box;

[0045] This can improve detection accuracy, data diversity, annotation information richness and scalability. By using high-resolution industrial cameras to collect original images of photovoltaic modules, the clarity and detail richness of image data can be ensured, which helps subsequent algorithms to more accurately identify and locate surface defects. Collecting images at different times, angles and lighting conditions can increase the diversity of data sets, thereby improving the robustness of algorithms in various practical application scenarios. Using annotation software to annotate defects in detail, including defect type, location, size, etc., provides accurate supervision information for training deep learning or machine learning models, which is the key to achieving high-precision automatic defect detection. Saving annotation information in a structured XML format facilitates data storage, retrieval and expansion, and also facilitates other researchers or engineers to reuse data sets.

[0046] A13. Bind the original image and the .xml annotation file to each other, and give them the same name;

[0047] A2. Divide the prepared data set to construct a photovoltaic module surface defect detection training set, a photovoltaic module surface defect detection verification set, and a photovoltaic module surface defect detection test set.

[0048] The specific steps are as follows: In this embodiment, the data set, validation set, and test set are divided into 7:3:1 (which can be adjusted appropriately during actual operation). Among them, the training set is used to train the neural network to obtain the weights of each neuron in the network. The validation set is used to verify the detection effect of the network after each round of network training, and the hyperparameters in the model are adjusted according to the detection effect. The test set is used to evaluate the final prediction effect of the network. By reserving a part of the data as the test set, it can be used to measure the performance of the model on unseen data, which helps to evaluate the generalization ability of the model, that is, the prediction ability of the model for new samples. The test set does not participate in the training process and can be used to conduct a fair performance evaluation after the model is finally finalized, avoiding performance evaluation errors caused by overfitting of the model to the training data. The validation set can be used for hyperparameter selection and model early stopping, which can avoid overfitting of the model on the training set and find the optimal model complexity and hyperparameter settings.

[0049] A3. Perform data enhancement preprocessing on the sample data set;

[0050] Specifically, random brightness enhancement, image inversion, cropping and splicing methods are used to perform data enhancement on the annotated images to expand the sample data set, improve the generalization of the neural network, and make the trained network parameters more suitable for real scenarios. Data enhancement creates diverse data variations, forcing the model to learn a wider range of features, making it more robust when encountering novel data in the real world. By artificially expanding the training set, data enhancement reduces the model's dependence on a small number of specific samples and reduces the model's tendency to overfit the training data. More training samples usually provide richer information and help the model capture more complex patterns, thereby improving its prediction accuracy. The additional samples generated by data enhancement increase the diversity of the original data set, which helps to simulate various situations that may be encountered in actual applications. Enhancement methods such as brightness adjustment and image inversion can allow the model to maintain a good recognition rate when facing lighting changes, perspective differences, and other situations.

[0051] A4. Use the data-enhanced training set samples and validation set samples to train and validate the RetinaNet target detection network model.

[0052] Among them, the target detection RetinaNet network model includes a feature extraction structure and a feature fusion structure. The network uses the feature extraction output as the input of the feature fusion model.

[0053] The specific operation is to input the enhanced training set samples into the target detection RetinaNet network model, perform iterative training of feature extraction and classification on the defect targets in the data set in turn, and save the results of each round of training. After each round of training, the validation set samples will be tested using the network weights saved in this round, and the test results will be analyzed and the network model hyperparameters will be adjusted to finally obtain the optimized photovoltaic module surface defect detection neural network RetinaNet. As a deep learning model, RetinaNet can more accurately identify and locate defects on the surface of photovoltaic modules by learning a large number of data-enhanced samples. Using data-enhanced samples for training can help the model learn more types of defect features and enhance its generalization detection ability for new samples. The feature extraction structure and feature fusion structure of RetinaNet enable it to efficiently process image information, while ensuring high accuracy, it also maintains a faster detection speed. Using the validation set for each round of testing can timely discover problems in the model during training and adjust hyperparameters to optimize model performance.

[0054] The training steps are as follows: Input the enhanced training sample set into the target detection RetinaNet network model, refer to Figure 2, is a schematic diagram of the RetinaNet network structure, which is a classic Backbone+Neck+Head single-stage network structure, where Backbone uses Resnet, Neck uses FPN, and the Head part consists of a classification subnet (classsubnet) and a box regression subnet (boxsubnet). The Resnet structure consists of three convolution kernels, with sizes of 1×1, 3×3, and 1×1, which are used for compression dimension, convolution processing, and restoration dimension, respectively.

[0055] Specifically, obtaining the optimized target detection RetinaNet network model includes the following steps:

[0056] A41. Introduce the ECA (Efficient Channel Attention Network) attention mechanism to improve the network's attention to small targets. The core idea of ​​ECANet is to introduce the channel attention mechanism in the convolution operation to effectively capture the relationship between channels in the image and improve the feature representation ability. The goal of the channel attention mechanism is to adaptively adjust the weights of channel features so that the network can better focus on important features and suppress unimportant features. Through this mechanism, ECANet can effectively enhance the network's representation ability without adding too many parameters and computational costs. The structure of ECANet is as follows: Figure 3 As shown, the input feature map size is

[0057] [H, W, C]. Through the global average pooling operation, the two-dimensional features of each channel are compressed into a real number, so that the feature map becomes a vector of [1, 1, C]. Next, the size of the adaptive one-dimensional convolution kernel is calculated according to the number of channels in the feature map.

[0058]

[0059] Where |t|odd represents the odd number closest to k, b=1, y=2, and C is the number of input feature channels. The weights are obtained through a convolution kernel of size k, and the obtained weights are normalized and combined with the input feature map to generate the output feature map.

[0060] A42. When an image is input, the feature extraction network outputs the Conv3Block, Conv4Block, and Conv5Block layers, which are then passed to the FPN module for feature fusion. C5 is the high-level output of the feature extraction network, which directly outputs the feature map P5 after two convolutions. However, at this time, P5 has been convolved many times and contains very little target information, especially small targets. Therefore, in this study, the FPN structure of the RetinaNet algorithm was improved with the help of the PANet structure, and a top-down feature fusion path was added to the FPN structure to improve the information extraction capability. The feature maps C3_1, C4_1, and C5_1 output by FPN are subsampled and weighted. According to this fusion method, the output results of each layer are convolved to obtain the FPN optimized output P3-P7. Finally, the five feature maps P3-P7 are output to the classification regression module for target prediction.

[0061] A43. Obtain the loss function of the existing RetinaNet network model;

[0062] In this embodiment, the loss function of the existing RetinaNet network model is obtained by adding the classification loss and the regression loss, that is, L Z =L cls +L reg

[0063] Among them, the classification loss uses FocalLoss as the loss function, and the calculation formula of the classification loss function Lcls is:

[0064] L cls (p t )=-α(1-p t ) γ logp t

[0065]

[0066] Where pt is the score of the prediction box classification; α is a parameter used to control the balance of positive and negative samples, and its value is 0.25; y is a modulation factor parameter, which is a parameter that reduces the proportion of easy-to-classify samples in learning, and its value is 2.

[0067] The regression loss uses smooth L1 loss as the loss function.

[0068]

[0069] Among them, x and y are the output and label of the model respectively, and |xy| represents the difference between them. When |xy| is less than 1, the square error is used; otherwise, the linear error is used.

[0070] A44. Update the existing RetinaNett network parameters to reduce the total loss function, and continuously iterate this process until the network converges to obtain the target detection RetinaNet network model.

[0071] A5. Input the enhanced test sample set into the trained target detection RetinaNet network model, detect it through the feature extraction stage and feature fusion stage, and finally output it through the classification and regression module to obtain the defect detection result.

[0072] Specifically, the enhanced test sample set is input into the trained improved RetinaNet network model for defect detection. The detection is divided into three stages, namely feature extraction, feature fusion, classification and regression module.

[0073] (1) Feature extraction module: The enhanced test sample set is input into the RetinaNet feature extraction model to output the defect feature information of each layer.

[0074] (2) Feature fusion module: It fuses the bottom-up semantic data with the top-down semantic data and utilizes the bottom-level positioning detail information.

[0075] (3) The fused feature layer is sent to the classification subnet (ClassSubnet) and the bounding box subnet (BoxSubnet). The classification subnet first performs four 3×3 convolution operations with a kernel of 256, and uses a Relu activation function after each convolution. Then, a 3×3 convolution operation with a kernel of K×A is performed, where K is the number of target categories and A is the number of anchor boxes. Finally, the Softmax function is used to obtain the confidence of each anchor box. During training, the anchor boxes that reach the confidence threshold are selected and the classification loss is calculated using FocalLoss. The network is then optimized for learning. The classification subnet and the bounding box subnet do not share parameters, but parameter sharing is implemented between feature layers.

[0076] In summary, compared with the traditional manual detection method, the present invention improves the RetinaNet network, adopts a bidirectional fusion backbone structure to enhance the extraction capability of small targets, introduces the ECA attention mechanism, and proposes an improved ResNet structure, which improves the network's recognition accuracy and detection efficiency of photovoltaic module surface defects.

[0077] The above description is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and improved concepts of the present invention within the technical scope disclosed by the present invention, and they should be covered by the protection scope of the present invention.

Claims

1. A single-stage photovoltaic module surface defect detection method based on improved RetinaNet, characterized in that: The method comprises the following steps: A1. Use industrial cameras to collect PV module images and create a PV module image dataset; A2. Divide the sample data set into training set, test set and validation set; A3. Perform data enhancement preprocessing on the sample data set; A4. Based on the enhanced defective photovoltaic module images, the surface defect detection is performed through the RetinaNet deep learning network model to obtain the detection results. The improved RetinaNet single-stage target detection network includes a feature extraction network and a bidirectional feature fusion structure. A5. Input the enhanced test sample set into the trained target detection RetinaNet network model, detect it through the feature extraction stage and feature fusion stage, and finally output it through the classification and regression module to obtain the defect detection result.

2. The single-stage photovoltaic module surface defect detection method based on improved RetinaNet according to claim 1, characterized in that: In step A1, the acquisition of original photovoltaic module images and the preparation of sample data sets include the following steps: A11. Use photovoltaic module surface defect detection machinery and industrial cameras to photograph photovoltaic modules and collect original images of photovoltaic modules; A12. Annotate the collected original images, use different defect labels to annotate small units and defects, and generate an .xml document; A13. Bind the generated .xml document containing the annotation information and the original image and save them in a txt file as a sample data set.

3. The single-stage photovoltaic module surface defect detection method based on improved RetinaNet according to claim 1, characterized in that: In step A3, the data enhancement preprocessing uses random brightness enhancement, image inversion, cropping and splicing methods to perform data enhancement on the labeled images to achieve the expansion of the sample data set.

4. The single-stage photovoltaic module surface defect detection method based on improved RetinaNet according to claim 2 is characterized in that: In step A4, the enhanced training set and validation set are used to train the target detection RetinaNet network model. The specific steps are: inputting the enhanced training sample set into the improved RetinaNet network model, iteratively training the defect detection model in turn, and saving the trained network parameters; The validation sample set is used to adjust the parameters of the trained defect detection model to obtain the defect detection RetinaNet model.

5. The single-stage photovoltaic module surface defect detection method based on improved RetinaNet according to claim 4 is characterized in that: In step A5, the specific steps of the defect detection stage are: input the enhanced photovoltaic module image into the defect detection RetinaNet model, first extract features from the input image, enter the classification module and regression module after passing through the PAN bidirectional feature fusion structure to obtain the training results, output the candidate boxes, and then use the maximum suppression algorithm to screen the candidate boxes to obtain the final defect detection results, and mark the defect detection results in the original photovoltaic module image.

6. The single-stage photovoltaic module surface defect detection method based on improved RetinaNet according to claim 4 is characterized in that: In step A4, the improved RetinaNet network model is obtained by optimizing the existing RetinaNet network model, including the following steps: Get the loss function of the existing RetinaNet network model; A bottom-up path is connected to the existing RetinaNet network model feature fusion FPN structure to form a bidirectional feature fusion structure; Improve the ResNet structure in the feature extraction network, add the ECA attention mechanism, strengthen the ability to transmit deep semantic information, and improve the accuracy of small target detection; Update the existing RetinaNet network parameters to reduce the total loss function, and continuously iterate this process until the network converges to obtain the target detection RetinaNet network model.

7. The single-stage photovoltaic module surface defect detection method based on improved RetinaNet according to claim 6, characterized in that: The loss function consists of two parts: classification loss function L cls And the regression loss function L reg Add and we get L Z =L cls +L reg The classification loss uses Focal Loss as the loss function to calculate the classification loss of all positive and negative samples. The regression loss uses smooth L1 loss as the loss function to calculate the regression loss of all positive samples.

Images