A photovoltaic panel defect detection method based on dual-scale strategy and improved YOLOV5 network

Abstract

The present invention discloses a photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network, comprising: screening samples in EL electroluminescent images and marking them; filtering the screened sample images using the Otsu binarization method and removing the black background noise in the photovoltaic panel area; obtaining a trained weight file by training a dual-scale MET_YOLOV5 model; verifying the trained MET_YOLOV5 network model; and using the model to predict the collected photovoltaic panel EL image after verification to obtain a dual-scale prediction result. By applying the embodiments of the present invention, not only can more obvious hidden crack defects be detected in large-scale images, but small-scale defect detection can be performed for each grid unit in the photovoltaic panel to identify more subtle hidden cracks. Moreover, the fusion of the dual-scale defect prediction results can make the final prediction result more accurate and have a higher confidence level.

Description

Technical Field

[0001] The present invention relates to the technical field of solar panel defect recognition, and in particular to a photovoltaic panel defect detection method and device based on a dual-scale strategy and an improved YOLOV5 network. Background Art

[0002] In recent years, deep learning technology has achieved numerous results and applications in machine vision. Deep learning has become an important technical means for image detection and image processing. Machine vision defect detection methods have gradually replaced manual visual inspection methods.

[0003] Machine vision defect detection relies on the powerful computing power of computers. By feeding a massive amount of positive and negative defect sample image data into the model for learning, the trained model can then be used to directly detect whether the samples contain defects. Machine vision defect detection systems generally consist of hardware and software systems. The hardware system includes image acquisition devices and display devices, while the software system primarily uses deep neural network models and image segmentation algorithms to process and analyze image features.

[0004] In related research in this field, Demant et al. proposed a cell inspection method based on support vector machines (SVMs). This method uses extracted defect features from positive and negative samples to determine the presence of cells. Wang Xianbao et al. reconstructed cell images through unsupervised learning, trained deep belief networks (DBNs), and compared and analyzed the reconstructed images with real images, performing sequential inspections on the input data.

[0005] While deep learning-based defect detection methods have significantly improved efficiency compared to manual methods, they still face several challenges in practical industrial scenarios. For photovoltaic panel defect detection, the main challenges are: 1) The defect sample size is too small, which hinders feature extraction in the network model and leads to overfitting. 2) Defects vary in scale and morphology, and fine-grained defect features are easily lost in the network layer or masked by other noise, affecting the robustness and generalization of the model. 3) As the network depth and width increase, the parameters in the network increase exponentially, significantly increasing model training time, reducing detection efficiency, and placing higher demands on storage performance. Therefore, network model parameter optimization is necessary. 4) Due to the large size of EL images, existing methods have low accuracy in end-to-end whole-image defect detection, and the network lacks adaptability to images of different scales. 5) In tasks based on sliding window detection of small-scale images, existing methods cannot effectively aggregate the same defect in adjacent grids and lack the ability to understand global semantic information. Summary of the Invention

[0006] The purpose of this invention is to provide a photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network, aiming to realize the end-to-end photovoltaic panel defect identification task, achieve good recognition accuracy for minor defects, and the algorithm has strong real-time performance and a wide range of application scenarios.

[0007] In order to achieve the above objectives, the present invention provides a photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network, characterized in that the method includes:

[0008] Collect EL electroluminescence images of photovoltaic panels;

[0009] Screening sample images from the collected EL electroluminescence images and marking images with electroluminescence defects;

[0010] The screened sample images are filtered using the Otsu binarization method to obtain filtered sample images, and the black background noise in the photovoltaic panel area is removed;

[0011] Construct training sample sets and validation sets based on filtered sample images and labeled sample images;

[0012] Based on the training sample set, the MET_YOLOV5 network model is used and a dual-scale strategy is adopted to perform training to obtain a trained network MET_YOLOV5 network model;

[0013] The trained dual-scale MET_YOLOV5 network model was used for verification, and the verified MET_YOLOV5 model was used to predict the collected photovoltaic panel EL images to obtain the dual-scale prediction results.

[0014] In one implementation, the step of collecting the EL electroluminescence image of the photovoltaic panel includes:

[0015] Use a high-resolution CCD camera to capture near-infrared images of photovoltaic panel components;

[0016] The near-infrared image is used as the EL electroluminescence image of the photovoltaic panel.

[0017] In one implementation, the step of filtering the screened sample image using the Otsu binarization method to obtain a filtered sample image and removing the black background noise in the photovoltaic panel area includes:

[0018] The screened sample image is gray-scaled, and the threshold is obtained using the Otsu algorithm on the gray-scale image, and the threshold is used to perform image threshold binarization;

[0019] Use closing operation to find the maximum connected domain and obtain the filtered image;

[0020] The Canny operator is used for edge detection to remove the original edge texture in the filtered image that interferes with feature extraction.

[0021] In one implementation, the step of constructing a training sample set and a validation set based on the filtered sample images and the labeled sample images includes:

[0022] The Mosaic data augmentation method is used to expand the dataset by randomly scaling, cropping, and arranging the filtered sample images and the labeled sample images to perform image splicing.

[0023] The expanded dataset is fed into the network model for training.

[0024] In one implementation, the step of training the MET_YOLOV5 network model based on the training sample set to obtain a trained weight file includes:

[0025] The MET_YOLOV5 network model structure includes:

[0026] Input: Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling;

[0027] Backbone network: CSPNet, neck: SPP, FPN, PAN;

[0028] Prediction head: YoloHead; among them, CSPNet is used to extract features, SPP, FPN, and PAN are used to enhance features, and the prediction head YoloHead uses GIOU_Loss as the loss function for calculating the Boundingbox; its default input image size is 640*640*3.

[0029] In one implementation, MET_YOLOV5 adds a Swin-Transformer module to the C3 module in the backbone network CSPNet to perform targeted optimization and improvement on the YoloV5 network. The improved C3 module is called C3STR, which uses a self-attention structure to enhance the semantic information and feature extraction capabilities of small targets, and implements cross-window information interaction by dividing local windows to reduce the amount of computation.

[0030] MET_YOLOV5 adds an ECA attention module at the end of the neck SPP layer and performs average pooling on all feature channels without dimensionality reduction;

[0031] The ECA module uses one-dimensional convolution to capture information between different channels, multiplying the channel attention feature map and the input feature map channel by channel and outputting them;

[0032] The ECA module is able to optimize the feature map so that the network focuses on photovoltaic panel hidden crack defects of different sizes and shapes.

[0033] In one implementation, the step of using the verified MET_YOLOV5 model to predict the collected photovoltaic panel EL image to obtain a dual-scale prediction result includes:

[0034] For an EL image, the entire large-scale EL image is preprocessed and data enhanced before being fed into the MET_YOLOV5 network for prediction to obtain the large-scale prediction result F1. At the same time, the single patch image after the EL image is sliced ​​according to the photovoltaic panel grid is fed into the MET_YOLOV5 network for prediction. The defect prediction boxes that satisfy the nearest neighbor relationship are merged, and finally the prediction result of the patch scale is mapped to the original image scale to obtain the prediction result F2. Finally, F2 is fused with the full-scale prediction result F1 to obtain F.

[0035] In one implementation, the step of fusing F2 with the full-scale prediction result F1 to obtain F includes:

[0036] For an EL panel image, a large-scale prediction result set F1 and a small-scale prediction result set F2 are given; F1 and F2 contain i and j prediction boxes respectively, and a single prediction box is represented by F1 i , F2 j ,Each prediction box information includes: prediction box position Px, confidence Pc, and fusion using anchor box matching;

[0037] For any prediction box F1 i And the prediction box F2 j By calculating the intersection over union (IoU), we can determine whether the predicted box is the same defect. IoU measures the relative size of the overlap between two bounding boxes. For F1 i and F2 j , and its calculation formula is:

[0038]

[0039] If F1 i and F2 j If the threshold condition of IoU ≥ 0.5 is met, it is determined to be the same defect;

[0040] Set the final prediction box position Px to the prediction box F1 i And the prediction box F2 j The predicted box position with higher probability:

[0041] Px=[max(Pc i ,Pc j ),Px]

[0042] Then, by calculating the joint probability, the final confidence Pc of the prediction box is given. The calculation formula is:

[0043]

[0044] PC i and PC j Represents the prediction box F1 respectively i And the prediction box F2 j The confidence level Pc of the fused prediction result set F contains the m fused prediction boxes, and a single prediction box is represented by F m , each prediction box information includes: prediction box position Px, confidence Pc, i, j represents the sequence number in the prediction result set to which it belongs.

[0045] In one implementation, the method further comprises: for small-scale prediction results, using a fusion algorithm based on nearest neighbor relationship to merge cross-grid defects; using a dual-scale image fusion strategy algorithm to fuse the prediction results;

[0046] The prediction result F2 of each grid in the small-scale prediction result F2 j , calculate the prediction result F2 between it and the adjacent grid j+1 The shortest distance is filtered using the threshold judgment method (the shortest distance between any two prediction box boundary points is less than 1 / 4 of the longest diagonal average value), and the prediction boxes that meet the requirements are merged; for the merged prediction box F2 j ′, set its prediction box position Px to the minimum bounding rectangle of the union of the two prediction boxes; by calculating their joint probability, the final confidence Pc of the prediction box is obtained, and the calculation formula is:

[0047]

[0048] Among them, the threshold judgment method is used for filtering to ensure that the shortest distance between any two prediction box boundary points is less than 1 / 4 of the average value of the longest diagonal.

[0049] In one implementation, the step of verifying the trained MET_YOLOV5 network model using the validation set includes:

[0050] Set the number of images and iterations for each training session, adjust the ratio of the test set to the validation set, perform multiple validations, and select the optimal weight parameter file for defect prediction.

[0051] The photovoltaic panel defect detection method and device based on the dual-scale strategy and the improved YOLOV5 network provided in the embodiment of the present invention are applied. According to the characteristics of the sample image, the present invention adopts the Otsu binarization method to filter the background of the original image, and adopts the Canny operator to extract the edge of the image, thereby improving the network's ability to extract effective features.

[0052] The improvements of the MET_YOLOV5 network model described in the present invention are: by adding Swin-Block and using multi-head self-attention to enhance the semantic information and feature extraction capabilities of small targets, cross-window information interaction is achieved by dividing local windows; by adding the attention mechanism ECA module in the SPP layer to perform multi-scale feature weighting adjustment, the network's ability to extract features across channels is enhanced, and the accuracy of tiny defect detection is improved.

[0053] The end-to-end photovoltaic panel defect recognition task was realized, and a good recognition accuracy rate was achieved for minor defects. The algorithm has strong real-time performance and a wide range of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a structural diagram of a photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network according to an embodiment of the present invention.

[0055] Figure 2 This is a schematic diagram of processing results of a photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network according to an embodiment of the present invention.

[0056] Figure 3 This is another schematic diagram of processing results of the photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to an embodiment of the present invention.

[0057] Figure 4 This is another schematic diagram of processing results of the photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to an embodiment of the present invention.

[0058] Figure 5 Schematic diagram of the dual-scale processing flow of the photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network in an embodiment of the present invention.

[0059] The labels are: 1. Loading platform; 2. Industrial detection camera for process; 3. Modular air outlet; 4. Industrial detection camera for process; 5. Temperature sensor DETAILED DESCRIPTION

[0060] The following describes the embodiments of the present invention through specific examples. Those skilled in the art will readily understand the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention.

[0061] See also Figure 1-5 It should be noted that the diagrams provided in this embodiment are merely schematic illustrations of the basic concept of the present invention. Therefore, the diagrams only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0062] With the dramatic increase in global energy demand, the development of new energy, especially clean energy, has received significant attention. Photovoltaic power generation technology has been widely adopted both domestically and internationally. According to statistics, the cumulative installed capacity of photovoltaic power generation reached 400 GW in 2017, and as of April 2018, China's grid-connected photovoltaic capacity exceeded 140 million kW. Photovoltaic power generation technology is used in a wide range of scenarios and locations. As the core component of photovoltaic power generation, photovoltaic panels can face problems and defects at every stage of manufacturing, transportation, installation, and use due to environmental factors, physical factors, and material properties, leading to serious failures.

[0063] There are three main types of defects in photovoltaic solar panels: contamination of silicon wafer raw materials, damage during cell production, and defects generated during use. Methods used to detect these defects include electrical parameter measurement, laser beam induced current, lock-in infrared thermal imaging, and electroluminescence. Of these methods, electrical parameter measurement and laser beam induced current cannot directly obtain images of sample defects. Infrared thermal imaging, on the other hand, requires applying a reverse bias to the cell and then capturing the infrared light emitted from the panel surface. Because heating the cell surface takes time and an infrared camera is required for detection, this method is inefficient and relatively expensive. Electroluminescence (EL) is based on the principle that when a forward bias is applied to a photovoltaic panel, the light intensity generated is proportional to the voltage. This allows detection of electrically inactive shadowed areas within the photovoltaic panel. EL technology can quickly and efficiently detect defects such as broken grids, hidden cracks, poor solder joints, debris, concentric circles, leakage currents, and contamination in photovoltaic panels. EL technology is currently widely used, improving the efficiency of photovoltaic panel operation, maintenance, and inspection in large-scale power plants. However, defect identification and troubleshooting in EL images is primarily performed manually, with a low degree of automation. Furthermore, due to manual limitations, there is a significant risk of missed or false positives for even minor defects. In recent years, with the continuous development of deep learning and convolutional neural network technologies, methods such as support vector machines (SVMs), deep belief networks (DBNs), convolutional neural networks (CNNs), and generative adversarial networks (GANs) have been extensively and intensively studied and applied in areas such as image classification, object recognition, and semantic segmentation. The use of deep learning-based computer methods to achieve fully automated defect detection in photovoltaic panel EL images has also become a research area for many scholars.

[0064] As the scale of the photovoltaic industry continues to expand, the use of deep learning-based computer methods to achieve fully automatic defect detection of photovoltaic panel EL images can improve detection efficiency while reducing the false positive rate and missed detection rate, greatly reducing labor costs in photovoltaic panel production, operation and maintenance, and other links, and has high practical value and significance.

[0065] In the existing technology, in the field of target detection algorithm research based on deep learning, in the background technology, in order to address the problem 1) in the background technology, a method of data enhancement for training images is usually adopted. For example, K. Simonyan et al. increased the diversity of training images by performing operations such as flipping, scaling, and random cropping on images. Srivastava et al. also alleviated the overfitting of the network to a certain extent by using Dropout and regularization methods. Adamu et al. used generative adversarial networks (GANs) to generate more training data, which helped to alleviate model overfitting, but GANs still have problems such as unstable training and gradient disappearance.

[0066] In response to problem 2) in the background technology, the feature extraction network is usually optimized based on different target detection tasks. For example, He et al. proposed the Spatial Pyramid Pooling (SPP) structure in 2014, which achieved the fusion of multi-scale local features and global features, and enhanced the expressive power of the feature map. In addition, He Guozhong et al. introduced a global spatial attention module for fine-grained features in the YOLOV4 network, which reduced the possibility of fine-grained features disappearing due to downsampling through global connection channels, and improved the accuracy in PCB detection tasks. However, this method does not take into account the differences in the importance of each channel of the image and still needs to be improved.

[0067] Regarding issue 3) in the background technology, network models are typically lightweighted by optimizing the computational methods of convolutional and fully connected layers, compressing overall network parameters, and implementing depthwise separable convolution. For example, Gong Ming et al. replaced the YOLO-V3 backbone network with a simplified Darknet-53, retaining only 10 convolutional layers and 6 pooling layers, without using any pooling layers. This reduced network parameters without sacrificing network performance. However, this method lacks universality and can result in more false detections in image data with complex backgrounds.

[0068] The advantages of the method described in the present invention are: 1) In order to address the problem that insufficient number of samples may lead to insufficient generalization ability of the model, the Canny operator is used to perform first-order gradient processing on the defective part to make the defect more obvious, and the processed feature map and the original image are used as multi-channel inputs of the model. The data enhancement method based on the optimization of the underlying feature image proposed in the present invention can improve the generalization ability of the model and alleviate the problem of sample imbalance. 2) In order to address the problem of interference areas in the sample image, the Otsu threshold method is used to filter the background noise. 3) Since the defect morphology and scale of the EL image are quite different, and the image boundaries are often blurred and noisy, some subtle defect features are easily ignored by the model. The present invention combines the Transformer structure, the channel attention mechanism ECA module and the multi-scale feature fusion strategy on the Yolo-v5 network. By adding the Swin-Transformer module to the C3 module in the Yolo-v5 backbone network, the self-attention structure of the Transformer is introduced to enhance the semantic information and feature extraction capabilities of small targets, and cross-window information interaction is achieved by dividing local windows; by adding the channel attention mechanism ECA at the end of the SPP (Spatial Pyramid Pooling) layer of the backbone network, the feature weights of different channels are weighted and adjusted, the network's efficiency in extracting cross-channel features is optimized, and the model's recognition accuracy for tiny defects is improved. 4) Through the dual-scale network prediction and prediction box fusion method, defect prediction boxes based on different scales of vision are provided, enhancing the accuracy and efficiency of defect detection. 5) Through the cross-grid defect prediction box merging method based on nearest neighbor relationships, cross-grid defects in small-scale prediction results are eliminated, improving the semantic accuracy of defect prediction results.

[0069] Compared with existing deep learning-based photovoltaic panel defect detection methods, this method fully considers the characteristics of photovoltaic panel EL images in data preprocessing, filtering background noise and enhancing defect saliency on the input image, thereby improving the robustness of the model. In terms of network structure, based on research on improvements to the Yolo object detection algorithm, the network incorporates a Transformer module, an attention mechanism, and a multi-scale feature fusion strategy to enhance the network's ability to extract features at different scales and in different image regions, thereby improving object detection accuracy.

[0070] like Figure 1 The present invention provides a photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network, comprising:

[0071] S101, collecting EL electroluminescence images of the photovoltaic panel.

[0072] In the embodiments of the present invention, specialized equipment is used to capture EL images of photovoltaic panels. Specifically, an EL detector is used to capture EL images of photovoltaic panels. This instrument utilizes the electroluminescence principle of crystalline silicon and uses a high-resolution CCD camera to capture near-infrared images of photovoltaic panel components.

[0073] S102 , screening sample images from the collected EL electroluminescence images, and marking images with electroluminescence defects.

[0074] The collected photovoltaic panel EL images are screened to obtain images of better quality; images with defects are marked and the marking information is saved in the form of XML files.

[0075] In one specific implementation, sample images with high imaging quality are screened, and experts perform fine annotation on the screened images. The number of screened images with fine annotations should be no less than 400.

[0076] S103 , filtering the screened sample image using the Otsu binarization method to obtain a filtered sample image, and removing the black background noise in the photovoltaic panel area.

[0077] The Otsu binarization method is used to filter the image after S102 screening to remove the black background noise around the photovoltaic panel area.

[0078] Specific processing steps may include:

[0079] S3.1 grayscale processing is performed on the sample image, and the threshold is obtained by using the Otsu algorithm on the grayscale image. The threshold is used to perform image threshold binarization. The closed operation is used to find the maximum connected domain, and the filtered image is obtained as follows Figure 2 The image shown.

[0080] The Otsu algorithm first performs grayscale processing on the input RBG channel image, calculates the threshold value based on the maximum inter-class variance of the grayscale image histogram, and finally separates the foreground and background based on the threshold value.

[0081] S3.2: Use the Canny operator to perform edge detection and filter out the original edge texture in the image that interferes with feature extraction. After edge detection, the following is obtained: Figure 3 The image shown.

[0082] The edge detection algorithm based on the Canny operator first performs Gaussian smoothing on the input image, calculates the gradient change amplitude and direction to estimate the edge strength and direction at each point, and then performs non-maximum suppression on the gradient amplitude according to the gradient direction. Finally, double threshold processing is used to connect the detected edges.

[0083] The present invention adopts the Otsu binarization method to filter the background of the original image and adopts the Canny operator to extract the edge of the image, thereby improving the network's ability to extract effective features.

[0084] S104: Construct a training sample set and a validation set based on the filtered sample images and the labeled sample images.

[0085] The training sample set and validation set are constructed by combining the binarized filtered images and the annotated data generated in S102. Due to the limited number of samples, the embodiment of the present invention adopts the Mosaic data enhancement method to expand the data set by randomly scaling, randomly cropping, and randomly arranging images for image splicing, thereby enhancing the generalization ability of the model and improving the recognition accuracy of the model.

[0086] S105, based on the training sample set, using the MET_YOLOV5 network model, adopting a dual-scale strategy for training to obtain a trained network MET_YOLOV5 network model.

[0087] Use the MET_YOLOV5 network for training to obtain the training weight file. You can use the following steps:

[0088] S5.1: Set the training parameters, including epoch = 100 and input image size to 1280*1280.

[0089] S5.2: The MET_YOLOV5 backbone network uses a Focus structure to slice the feature image into two parts. It fuses the feature maps using a CSP structure to reduce computational overhead. Neck is used to generate a feature pyramid, enhancing the model's ability to detect features at different scales.

[0090] S5.3: Add the ECA module at the end of the SPP layer of the MET_YOLOV5 backbone network, use 1*1 convolution to capture information between different channels, multiply the channel attention feature map and the input feature map channel by channel and output them. For the aggregated feature y∈RC without dimensionality reduction, its channel attention feature map is:

[0091] ω=σ(C1D k (y))

[0092] Where C1D represents one-dimensional convolution; for features of different dimensions, the one-dimensional convolution kernel size k is determined by an adaptive method:

[0093]

[0094] The convolution kernel size is k, the channel dimension is C, γ and b are parameters of the ECA module, γ = 2, b = 1; |t|odd represents the odd number closest to t. The ECA module can focus on hidden cracks in photovoltaic panels of different sizes and shapes.

[0095] MET_YOLOV5 adds an improved ECA module at the end of the backbone network SPP layer to perform average pooling on all feature channels without dimensionality reduction. The ECA module uses one-dimensional convolution to capture information between different channels, multiplying the channel attention feature map and the input feature map channel by channel and outputting it. It is divided into two branches: Branch 1 performs global average pooling on the input feature map and then calculates the channel attention weight. avg ; Branch 2 performs global maximum pooling on the input feature map and calculates the channel attention weight Weight max The two attention weight tensors are added together and then input into the sigmoid function to calculate the weight value in the range of [0,1].

[0096] S5.4: The prediction output uses adaptive anchor box regression and GIOU_Loss for Bounding Box loss calculation. A weighted nms non-maximum suppression method based on the overlap area ratio (IOU) is used.

[0097] Set the number of images and iterations for each training session, and adjust the ratio of the test set to the validation set for multiple experiments to obtain the optimal weight parameters.

[0098] S5.5: Use Figure 5 The dual-scale defect detection strategy shown in the figure detects global defects in the large-scale network branch; detects subtle defects within the photovoltaic panel grid in the small-scale network branch, and merges the small-scale network prediction frames using a cross-grid defect prediction frame merging method based on nearest neighbor relationships; finally, a dual-scale prediction result fusion algorithm is used to fuse the prediction results of the two network branches.

[0099] S106, using the trained dual-scale MET_YOLOV5 network model for verification, and using the verified MET_YOLOV5 model to predict the collected photovoltaic panel EL image to obtain a dual-scale prediction result.

[0100] Based on the trained weight file, the test set is used for testing. The trained MET_YOLOV5 network model is used to predict the collected photovoltaic panel EL image, and the following is obtained: Figure 4 The prediction results are shown.

[0101] This invention improves upon the MET_YOLOV5 network model by adding a Swin-Transformer block to the backbone network feature extraction module C3 of the MET_YOLOV5 network model, using a self-attention structure to enhance the semantic information and feature extraction capabilities of small targets. Furthermore, an attention mechanism (ECA) is added to the end of the backbone network to perform multi-scale feature weighting adjustments, enhancing the network's cross-channel feature extraction capabilities and improving the accuracy of small defect detection. Based on the improved Yolo-v5 network model, the end-to-end photovoltaic solar panel EL image detection task can be completed. Furthermore, the network employs a dual-scale strategy for training and prediction, and proposes a small-scale cross-grid defect merging algorithm and a dual-scale prediction result fusion algorithm.

[0102] Based on the improved MET_YOLOV5 network model, the input image is first normalized to 1280*1280 pixels. After processing using the Mosaic data augmentation method and adaptive anchor box calculation, it is fed into the backbone network for feature extraction. The backbone layer uses the Focus and CSP-Net architectures for feature extraction. The Focus architecture slices the image and performs convolution operations with 32 kernels to produce a 320*320*32 feature map. The MET_YOLOV5 network model incorporates two CSP architectures, the CSP1_X architecture in the backbone network. Furthermore, an ECA channel attention mechanism is added at the end of the SPP layer of the backbone network to adjust features across different channels without dimensionality reduction. In the Neck layer, an FPN+PAN architecture is used to fuse image features to generate a feature pyramid. At the output, GIOU_Loss is used to calculate the BoundingBox loss, while cross-entropy and Logits loss functions are used to calculate class probabilities and objectness scores.

[0103] In the photovoltaic panel EL image defect recognition task, this embodiment of the present invention first uses the first stage of Otsu binarization to filter out the black background and original texture features of the photovoltaic panel in the original image, improving the efficiency and accuracy of feature extraction by the network. Furthermore, using the Mosaic data augmentation method, the generalization performance of the MET_YOLOV5 network model is improved with limited sample and annotation information. This achieves end-to-end photovoltaic panel defect recognition, achieving high recognition accuracy for minor defects. The algorithm also exhibits strong real-time performance and is suitable for a wide range of applications.

[0104] Secondly, by adding Swin-Transformerblock to the backbone network C3 structure, an advanced method based on sliding window multi-head self-attention is used to improve the network feature extraction capability and enhance the semantic information of small objects in the network, thereby improving the performance baseline.

[0105] Finally, by adding a channel attention mechanism (ECA) to the end of the backbone network's SPP layer, we captured the dependencies between feature channels while maintaining the dimension of the feature map. This captures cross-channel interactions, focuses on defect characteristics, and improves the detection accuracy of the MET_YOLOV5 network model. Through these three phases of collaboration, we achieved end-to-end photovoltaic panel EL image defect detection, paving the way for practical production applications.

[0106] Finally, the present invention uses a dual-scale target detection strategy, combines a dual-scale prediction result fusion algorithm, and a cross-grid defect prediction box merging algorithm based on nearest neighbor relationships, so that the network can pay attention to global defect features and local defect features, while improving the defect semantic accuracy in the prediction results.

[0107] In this application, in response to the problems existing in the above studies, the present invention proposes targeted solutions: 1) By preprocessing the image, the significance of the features is improved. By data enhancement operations, the sample diversity is improved. 2) By adding Swin-Transformerblock to the feature extraction module C3 of the backbone network, the feature extraction capability of the network is enhanced. 3) By introducing the channel attention module ECA into the network, the global semantic information of the feature channel is fused, thereby further improving the performance of the network. 4) Through the dual-scale target detection strategy, the defect prediction results under different detection fields of view are obtained and fused to improve the confidence of the prediction results. 5) By calculating the cross-grid defect distance with nearest neighbor relationship, the cross-grid defects at a small scale are merged to improve the semantic accuracy of defect detection.

[0108] In computer vision research, image quality directly affects the efficiency of the algorithm and the accuracy of the effect. Therefore, image preprocessing is very important for improving recognition accuracy and algorithm robustness. The main purpose of image preprocessing is to eliminate irrelevant information in the image, retain and restore useful real information, and simplify the data to the greatest extent possible. General image preprocessing operations include: grayscale, geometric transformation, image denoising, edge extraction, etc. Based on the characteristics of the input image, the present invention uses the Otsu threshold method to filter the black background that does not contain semantic information in the image. At the same time, in order to optimize the visual effect of the defect, the Canny operator edge detection algorithm is used to improve the significance of the defect.

[0109] The MET_YOLOV5 network model used in this paper uses the Mosaic method to expand the dataset and adopts operations such as flipping, brightness adjustment, and cropping. Therefore, it has good generalization ability for sample sets with smaller data volumes. Its backbone network uses a network structure that combines CSP-Darknet (Cross Stage Partial Networks Darknet), Swin-Transformer, and Spatial Pyramid Pooling (SPP), based on a sliding window multi-head self-attention mechanism. The gradient changes of the feature layer are integrated into the feature map, which greatly reduces the amount of parameter calculations, thereby achieving comparable accuracy and recognition speed while reducing the model scale. The network first divides each image in the training set into S×S (S=19, 38, 76) grids. Each grid has candidate boxes of different sizes after adaptive anchor box calculation. The grid where the object center is located is responsible for object detection. Then, the convolutional layer of the backbone network extracts features. Finally, the prediction layer is used for multi-scale prediction. The predicted feature map has multiple scales and can predict objects of different sizes. The multi-scale feature map fusion is obtained by the feature pyramid structure. At the same time, the present invention adopts a dual-scale target detection strategy, that is, training and predicting the entire image and small-scale raster image respectively; merging the defect frames of the prediction results of the small-scale network, and fusing the prediction frame result set obtained by the dual-scale network.

[0110] Specifically, for an EL image, the entire large-size EL image is preprocessed and data enhanced before being fed into the MET_YOLOV5 network for prediction to obtain the large-scale prediction result F1. At the same time, the single patch image after the EL image is sliced ​​according to the photovoltaic panel grid is fed into the MET_YOLOV5 network for prediction, and the defect prediction boxes that satisfy the nearest neighbor relationship are merged. Finally, the prediction result at the patch scale is mapped to the original image scale to obtain the prediction result F2, and finally F2 is fused with the full-scale prediction result F1 to obtain F.

[0111] The steps of fusing F2 with the full-scale prediction result F1 to obtain F include:

[0112] For an EL panel image, a large-scale prediction result set F1 and a small-scale prediction result set F2 are given; F1 and F2 contain i and j prediction boxes respectively, and a single prediction box is represented by F1 i , F2 j ,Each prediction box information includes: prediction box position Px, confidence Pc, and fusion using anchor box matching;

[0113] For any prediction box F1 i And the prediction box F2 jBy calculating the intersection over union (IoU), we can determine whether the predicted box is the same defect. IoU measures the relative size of the overlap between two bounding boxes. For F1 i and F2 j , and its calculation formula is:

[0114]

[0115] If F1 i and F2 j If the threshold condition of IoU ≥ 0.5 is met, it is determined to be the same defect;

[0116] Set the final prediction box position Px to the prediction box F1 i And the prediction box F2 j The predicted box position with higher probability:

[0117] Px=[max(Pc j ,Pc j ),px]

[0118] Then, by calculating the joint probability, the final confidence Pc of the prediction box is given. The calculation formula is:

[0119]

[0120] PC i and PC j Represents the prediction box F1 respectively i And the prediction box F2 j The confidence Pc of the fused prediction result set F contains the m fused prediction boxes, and a single prediction box is represented by F m , each prediction box information includes: prediction box position Px, confidence Pc, i, j represents the sequence number in the prediction result set to which it belongs.

[0121] For small-scale prediction results, a fusion algorithm based on nearest neighbor relationship is used to merge cross-grid defects; a dual-scale image fusion strategy algorithm is used to fuse the prediction results;

[0122] The prediction result F2 of each grid in the small-scale prediction result F2 j , calculate the prediction result F2 between it and the adjacent grid j+1 The shortest distance is filtered using the threshold judgment method (the shortest distance between any two prediction box boundary points is less than 1 / 4 of the longest diagonal average value), and the prediction boxes that meet the requirements are merged; for the merged prediction box F2 j ′, set its prediction box position Px to the minimum bounding rectangle of the union of the two prediction boxes; by calculating their joint probability, the final confidence Pc of the prediction box is obtained, and the calculation formula is:

[0123]

[0124] Among them, the threshold judgment method is used for filtering to ensure that the shortest distance between any two prediction box boundary points is less than 1 / 4 of the average value of the longest diagonal.

[0125] In the embodiment of the present invention, specifically, F formed by fusing the prediction results F1 and F2 is also a prediction result set. F contains several prediction boxes. Then, the prediction boxes in F1 and F2 are compared, duplicate prediction boxes are removed, and prediction boxes with the same semantic information are retained to obtain the final fusion result, that is, the prediction result.

[0126] The Yolo series of network models boasts relatively high performance and a remarkably lightweight model size, achieving outstanding results in public dataset object detection competitions both domestically and internationally. Their accuracy, speed, and storage requirements are well-suited to the demands of industrial product defect detection.

[0127] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network, characterized in that: The method comprises: Collect EL electroluminescence images of photovoltaic panels; Screening sample images from the collected EL electroluminescence images and marking images with electroluminescence defects; The screened sample images are filtered using the Otsu binarization method to obtain filtered sample images, and the black background noise in the photovoltaic panel area is removed; Construct training sample sets and validation sets based on filtered sample images and labeled sample images; Based on the training sample set, the MET_YOLOV5 network model is used and a dual-scale strategy is adopted to perform training to obtain a trained network MET_YOLOV5 network model; The trained dual-scale MET_YOLOV5 network model was used for verification, and the verified MET_YOLOV5 model was used to predict the collected photovoltaic panel EL images to obtain the dual-scale prediction results; The MET_YOLOV5 network model structure includes: Input: Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling; Backbone network: CSPNet, neck: SPP, FPN, PAN; Prediction head: Yolo Head; CSPNet is used to extract features, SPP, FPN, and PAN are used to enhance features, and the prediction head Yolo Head uses GIOU_Loss as the loss function for calculating the Boundingbox; its default input image size is 640*640*3; The steps of using the verified MET_YOLOV5 model to predict the collected photovoltaic panel EL image to obtain a dual-scale prediction result include: For an EL image, the entire large-scale EL image is preprocessed and data enhanced before being fed into the MET_YOLOV5 network for prediction to obtain the large-scale prediction result F1. At the same time, the single patch image after the EL image is sliced ​​according to the photovoltaic panel grid is fed into the MET_YOLOV5 network for prediction. The defect prediction boxes that meet the nearest neighbor relationship are merged, and finally the prediction result of the patch scale is mapped to the original image scale to obtain the prediction result F2. Finally, F2 is fused with the full-scale prediction result F1 to obtain F. For an EL panel image, a large-scale prediction result set F1 and a small-scale prediction result set F2 are given; F1 and F2 contain i and j prediction boxes respectively, and a single prediction box is represented by F1 i , F2 j , each prediction box information includes: prediction box position Px, confidence Pc, and fusion using anchor box matching.

2. The photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to claim 1 is characterized in that: The step of collecting the EL electroluminescence image of the photovoltaic panel includes: Use a high-resolution CCD camera to capture near-infrared images of photovoltaic panel components; The near-infrared image is used as the EL electroluminescence image of the photovoltaic panel.

3. The photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to claim 1 is characterized in that: The step of filtering the screened sample image using the Otsu binarization method to obtain a filtered sample image and removing the black background noise in the photovoltaic panel area includes: The screened sample image is gray-scaled, and the threshold is obtained using the Otsu algorithm on the gray-scale image, and the threshold is used to perform image threshold binarization; Use closing operation to find the maximum connected domain and obtain the filtered image; The Canny operator is used for edge detection to remove the original edge texture in the filtered image that interferes with feature extraction.

4. The photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to claim 1 is characterized in that: The step of constructing a training sample set and a validation set based on the filtered sample images and the labeled sample images includes: The Mosaic data augmentation method is used to expand the dataset by randomly scaling, cropping, and arranging the filtered sample images and the labeled sample images to perform image splicing. The expanded dataset is fed into the network model for training.

5. The photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to claim 1 is characterized in that: MET_YOLOV5 adds a Swin-Transformer module to the C3 module in the backbone network CSPNet to perform targeted optimization and improvement on the YoloV5 network. The improved C3 module is called C3STR, which uses a self-attention structure to enhance the semantic information and feature extraction capabilities of small targets, and implements cross-window information interaction by dividing local windows to reduce the amount of computation. MET_YOLOV5 adds an ECA attention module at the end of the neck SPP layer and performs average pooling on all feature channels without dimensionality reduction; The ECA module uses one-dimensional convolution to capture information between different channels, multiplying the channel attention feature map and the input feature map channel by channel and outputting them; The ECA module is able to optimize the feature map so that the network focuses on photovoltaic panel hidden crack defects of different sizes and shapes.

6. The photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to claim 1, characterized in that: The steps of fusing F2 with the full-scale prediction result F1 to obtain F include: For any prediction box F1 i And the prediction box F2 j By calculating the intersection over union (IoU), we can determine whether the predicted box is the same defect. IoU measures the relative size of the overlap between two bounding boxes. For F1 i and F2 j , and its calculation formula is: If F1 i and F2 j If the threshold condition of IoU ≥ 0.5 is met, it is determined to be the same defect; Set the final prediction box position Px to the prediction box F1 i And the prediction box F2 j The predicted box position with higher probability: Px=[max(Pc i ,Pc j ),Px] Then, by calculating the joint probability, the final confidence Pc of the prediction box is given. The calculation formula is: PC i and PC j Represents the prediction box F1 respectively i And the prediction box F2 j The confidence Pc of the fused prediction result set F contains the m fused prediction boxes, and a single prediction box is represented by F m , each prediction box information includes: prediction box position Px, confidence Pc, i, j represents the sequence number in the prediction result set to which it belongs.

7. The photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to claim 6 is characterized in that: The method further comprises: for small-scale prediction results, using a fusion algorithm based on nearest neighbor relationship to merge cross-grid defects; using a dual-scale image fusion strategy algorithm to fuse the prediction results; The prediction result F2 of each grid in the small-scale prediction result F2 j , calculate the prediction result F2 between it and the adjacent grid j+1 The shortest distance is filtered using the threshold judgment method, and the prediction frames that meet the requirements are merged; for the merged prediction frame F2 j ', set its prediction box position Px to the minimum bounding rectangle of the union of the two prediction boxes; by calculating their joint probability, the final confidence Pc of the prediction box is obtained, and the calculation formula is: Among them, the threshold judgment method is used for filtering to ensure that the shortest distance between any two prediction box boundary points is less than 1 / 4 of the average value of the longest diagonal.

8. The photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to claim 1, characterized in that: The step of verifying using the trained dual-scale MET_YOLOV5 network model includes: Set the number of images and iterations for each training session, adjust the ratio of the test set to the validation set for multiple validations, and select the optimal weight parameter file for defect prediction.

Images