Solar photovoltaic panel defect detection method based on improved convolutional neural network

Abstract

A solar photovoltaic panel defect detection method based on an improved convolutional neural network belongs to the technical field of photovoltaic power generation, aims to solve the problems of poor defect identification effect and low detection precision of a solar photovoltaic panel detection method, and comprises the steps of extracting a photovoltaic panel defect image data set, marking defects, and marking the defects. Naming and classifying the defects, generating XML (Extensive Makeup Language) files, and converting defect images and the corresponding XML files into a VOC (Volatile Organic Compounds) data set; constructing a photovoltaic panel defect detection model to detect defects; the improved convolutional neural network adopts a YOLOv8 neural network, a detection head adopts an Ancher-Free detection head, a loss function adopts a HIoU loss function, an attention mechanism is embedded behind an SPPF module of a Backbone layer, an attention mechanism is embedded behind a C2f module of a Head layer, and an attention mechanism is embedded behind a C2f module in the middle of a Neck layer. The method is used for a photovoltaic power generation system.

Description

Technical Field

[0001] The invention relates to a solar photovoltaic panel defect detection method, belonging to the technical field of photovoltaic power generation. Background Art

[0002] In the photovoltaic power generation system, the photovoltaic module is the most important component, which mainly uses silicon semiconductor materials to convert solar energy into electrical energy. Therefore, the photoelectric conversion efficiency of the photovoltaic system is mainly affected by the intensity of the solar radiation it receives. Most photovoltaic modules are installed outdoors. During transportation and use, they will inevitably suffer from collisions, wind and sand erosion and other damages, and thus inevitably produce defects such as cracks, broken grids and spots. In order to improve the conversion rate, utilization rate and service life of photovoltaic modules, these defects need to be monitored and effectively protected in actual work. Therefore, a stable and reliable monitoring system and defect detection technology play a vital role.

[0003] There are several methods for detecting defects in photovoltaic modules:

[0004] 1. The traditional operation and maintenance of photovoltaic power stations mainly rely on manual inspection methods, where workers directly observe the solar photovoltaic panels and detect the specific location and type of surface defects of the solar photovoltaic panels by visual observation or experience. The advantage of this method is that it does not require too high technical content and is simple to operate. The disadvantage is that it consumes manual time and energy. In addition, due to the increase in the scale of photovoltaic power generation systems and the constraints of different terrains, their distribution is relatively messy. The manual inspection method is inefficient, and workers are prone to mental fatigue during long-term inspections, which leads to detection errors.

[0005] 2. The laser scanning method utilizes the distribution characteristics of the battery silicon wafer and detects defects in the solar photovoltaic panel by laser scanning the battery. This method requires adding external current to the battery to achieve forward bias, and its accuracy and efficiency are not high.

[0006] 3. The acoustic wave method uses the mechanical excitation mode of the solar photovoltaic panel vibration to achieve acoustic measurement results, but it can only detect crack defects through frequency analysis of the sound waves. At the same time, this method may also damage the battery.

[0007] 4. The Hertz spectrum method analyzes the spectrum of polysilicon crystals in solar photovoltaic panels to determine changes in the distribution of battery materials and thus detect defects. However, this method is difficult and has low detection efficiency, and it is also unable to specifically determine the type of defects.

[0008] 5. The electrical detection method uses the conductive property of the material used to make solar photovoltaic panels. Various circuits are designed on the solar photovoltaic panels to test the current (or voltage) passing through them. The defects of the solar photovoltaic panels are detected by observing their changes. However, due to external factors added during the process, such as unreasonable circuit design, improper parameter settings, and even manual operation errors, the solar photovoltaic panels may be damaged, and this external damage is destructive.

[0009] 6. Obtain image data of photovoltaic panels through drones, and use machine vision methods to detect defects on solar photovoltaic panels. From the perspective of mathematical modeling, visual inspection methods can be divided into several categories: transform domain analysis, image domain analysis, and deep learning. Image domain analysis methods include clustering, matrix decomposition, gradient feature methods, etc., and transform domain analysis methods include wavelet transform, Fourier transform, etc. Research on physical inspection methods includes: designing unique circuits to analyze the changes in current (or voltage) passing through solar photovoltaic panels to detect defects on solar photovoltaic panels. Use different frequencies of sound waves to detect crack defects on solar photovoltaic panels, but this algorithm can only be used to detect crack defects and has great limitations. Use the distortion of the light intensity reflection image to detect crack defects. Based on the principle that different noises will be generated when the sample has defects, the difference in noise characteristics is compared to determine whether there are defects on the surface of the solar photovoltaic panel. However, the effect of this method is not ideal. It can only determine whether the defect exists, but not the specific defect type.

[0010] 7. With the explosive development of deep learning algorithms, the research on photovoltaic panel defect detection methods based on deep learning has also gradually deepened. By combining random forests with convolutional neural networks, a method for identifying defects in solar photovoltaic panels is used. This method effectively improves the accuracy and speed of recognition and marks the defects on the input image. The DenseNet network model is used to detect defects in outdoor solar photovoltaic panels, and a series of improvement measures are carried out on this basis, such as adjusting the Batch Normalization layer and adding L2 regularization to solve the overfitting problem, etc., so as to achieve better detection results. In the detection, different degrees of damage to photovoltaic panels can be effectively distinguished. The transfer learning method has a better effect on the defect recognition of solar photovoltaic panels. Summary of the invention

[0011] The purpose of the present invention is to solve the problems of poor defect recognition effect and low detection accuracy in existing methods for detecting solar photovoltaic panels, and to provide a solar photovoltaic panel defect detection method based on an improved convolutional neural network.

[0012] The present invention discloses a method for detecting defects in solar photovoltaic panels based on an improved convolutional neural network, comprising:

[0013] S1. Extract the photovoltaic panel defect image dataset from the public dataset, mark the defects in the image dataset, annotate the defects with boxes, name and classify the defects, generate an XML file corresponding to the entire image dataset, and then convert the photovoltaic panel defect images and the corresponding XML files into a VOC dataset;

[0014] S2, enhance the VOC data set;

[0015] S3, using an improved convolutional neural network to build a photovoltaic panel defect detection model to detect defects in photovoltaic panel images;

[0016] The improved convolutional neural network adopts the YOLOv8 neural network, the detection head adopts the Ancher-Free detection head, and the loss function adopts the HIoU loss function.

[0017] The attention mechanism is embedded in the back of the SPPF module of the Backbone layer, the attention mechanism is embedded in the back of the C2f module of the Head layer, and the attention mechanism is embedded in the back of the C2f module in the middle of the Neck layer.

[0018] Preferably, the defects in the image data set in S1 include cracks, broken grids and spots.

[0019] Preferably, the ratio of the three defects in the image data set is 1:1:1.

[0020] Preferably, the enhancement processing of the VOC data set described in S2 includes: random scaling, random rotation, random shift and Mosaic data enhancement.

[0021] Preferably, the specific method of random scaling includes:

[0022] Outward scaling: randomly crop a portion of the original image and then stretch it to the original size;

[0023] Inward scaling compresses the initial image and fills the pixels beyond the boundary area with a single pixel. The single pixel used for filling has nothing to do with the defect image dataset.

[0024] Preferably, the specific method of random rotation includes:

[0025] Random flipping: using the central axis of the image as the flipping center, flip the initial image horizontally or vertically;

[0026] Randomly rotate the image clockwise or counterclockwise with the center of the image as the rotation center.

[0027] Preferably, the specific method of random shifting includes:

[0028] Move along the X-axis, Y-axis, or both, while making assumptions about pixels outside the boundary area.

[0029] Preferably, the specific method of Mosaic data enhancement includes:

[0030] Four different initial images are randomly scaled and then spliced ​​into new images for training.

[0031] Preferably, the HIoU loss function specifically includes:

[0032]

[0033] Among them, L HIoU represents the HIoU loss function, L IoU represents the IoU loss function, R WIoU represents the penalty function, x represents the x-axis coordinate of the original prediction box, x gt represents the x-axis coordinate of the new target prediction box, y represents the y-axis coordinate of the original target function, gt Represents the y-axis coordinate of the new target prediction box, W g Indicates the width of the minimum prediction box, H g Represents the height of the minimum prediction box, v represents the proportional consistency coefficient between the width and height of the prediction box, and α represents the adjustment coefficient of v.

[0034] Preferably, the specific method of embedding the attention mechanism after the SPPF module of the Backbone layer includes:

[0035] The SPPF module is placed between the Backbone layer and the Neck layer, and the attention mechanism is embedded in the back side of the SPPF module;

[0036] The input image extracts feature information through the Backbone layer and is input into the SPPF module after multiple layers of convolution. The SPPF module integrates the feature information obtained by convolution in the Backbone layer, fuses features of different scales, obtains a feature vector of fixed length, and then inputs it into the Neck layer.

[0037] Advantages of the present invention: The present invention proposes a solar photovoltaic panel defect detection method based on an improved convolutional neural network, introduces deep learning technology into the monitoring of traditional photovoltaic power stations, and completes the detection of photovoltaic panel defects based on the YOLOv8 model of the improved convolutional neural network to improve the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the basic architecture of the YOLOv8 neural network described in the present invention;

[0039] Figure 2 It is a schematic diagram of the basic architecture of embedding the attention mechanism at the back side of the SPPF module of the Backbone layer according to the present invention;

[0040] Figure 3 It is a basic architecture diagram of embedding the attention mechanism at the back side of the C2f module of the Head layer according to the present invention;

[0041] Figure 4 It is a schematic diagram of the basic architecture of embedding the attention mechanism at the back side of the C2f module in the middle of the Neck layer according to the present invention;

[0042] Figure 5 This is a comparison chart of the accuracy before and after the dataset expansion;

[0043] Figure 6 This is a comparison chart of the convergence speed of model training using the original IoU loss function and the HIoU loss function. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0046] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0047] Embodiment 1:

[0048] Combine the following Figure 1-Figure 4 The present embodiment is described. The solar photovoltaic panel defect detection method based on the improved convolutional neural network described in the present embodiment comprises:

[0049] S1. Extract the photovoltaic panel defect image dataset from the public dataset, mark the defects in the image dataset, annotate the defects with boxes, name and classify the defects, generate an XML file corresponding to the entire image dataset, and then convert the photovoltaic panel defect images and the corresponding XML files into a VOC dataset;

[0050] S2, enhance the VOC data set;

[0051] S3, using an improved convolutional neural network to build a photovoltaic panel defect detection model to detect defects in photovoltaic panel images;

[0052] The improved convolutional neural network adopts the YOLOv8 neural network, the detection head adopts the Ancher-Free detection head, and the loss function adopts the HIoU loss function.

[0053] The attention mechanism is embedded in the back of the SPPF module of the Backbone layer, the attention mechanism is embedded in the back of the C2f module of the Head layer, and the attention mechanism is embedded in the back of the C2f module in the middle of the Neck layer.

[0054] Furthermore, the defects in the image data set in S1 include cracks, broken grids and spots;

[0055] The ratio of the three defects in the image dataset is 1:1:1.

[0056] Furthermore, the enhancement processing of the VOC data set described in S2 includes: random scaling, random rotation, random shift and Mosaic data enhancement.

[0057] Furthermore, the specific method of random scaling includes:

[0058] Outward scaling: randomly crop a portion of the original image and then stretch it to the original size;

[0059] Inward scaling compresses the initial image and fills the pixels beyond the boundary area with a single pixel. The single pixel used for filling has nothing to do with the defect image dataset.

[0060] Furthermore, the specific method of the random rotation includes:

[0061] Random flipping: using the central axis of the image as the flipping center, flip the initial image horizontally or vertically;

[0062] Randomly rotate the image clockwise or counterclockwise with the center of the image as the rotation center.

[0063] Furthermore, the specific method of the random shift includes:

[0064] Move along the X-axis, Y-axis, or both, while making assumptions about pixels outside the boundary area.

[0065] Furthermore, the specific method of the Mosaic data enhancement includes:

[0066] Four different initial images are randomly scaled and then spliced ​​into new images for training.

[0067] Furthermore, the HIoU loss function specifically includes:

[0068]

[0069] Among them, L HIoU represents the HIoU loss function, L IoU represents the IoU loss function, R WIoU represents the penalty function, x represents the x-axis coordinate of the original prediction box, x gt represents the x-axis coordinate of the new target prediction box, y represents the y-axis coordinate of the original target function, gt Represents the y-axis coordinate of the new target prediction box, W g Indicates the width of the minimum prediction box, H g Represents the height of the minimum prediction box, v represents the proportional consistency coefficient between the width and height of the prediction box, and α represents the adjustment coefficient of v.

[0070] Furthermore, the specific method of embedding the attention mechanism after the SPPF module of the Backbone layer includes:

[0071] The SPPF module is placed between the Backbone layer and the Neck layer, and the attention mechanism is embedded in the back side of the SPPF module;

[0072] The input image extracts feature information through the Backbone layer and is input into the SPPF module after multiple layers of convolution. The SPPF module integrates the feature information obtained by convolution in the Backbone layer, fuses features of different scales, obtains a feature vector of fixed length, and then inputs it into the Neck layer.

[0073] In the present invention, the photovoltaic panel defect image dataset used is derived from the AI ​​Studio public dataset. The defects of the photovoltaic panels in the dataset are mainly divided into three types: cracks, broken grids, and spots. The ratio of the three defects in the dataset is about 1:1:1. There are about 9,600 pictures in total. Since the defects are not labeled in the dataset, this design uses labelImg labeling software to manually mark the image defects, and divide the types into three categories: "cracks", "broken grids", and "spots". Use boxes to label the target defects to be detected, and name and classify the target defects. After using labelImg to label the defects of the target image, an XML file corresponding to all the dataset images will be generated. This file contains data such as the location of the labeled defects, the type of labeling, and the size of the defect image. The photovoltaic panel defect image and its corresponding XML file are converted into a VOC dataset.

[0074] Convolutional neural networks are robust and require a large number of training samples to avoid overfitting problems due to insufficient training models. Data enhancement can improve the robustness of the model and make the model more adaptable. If the convolutional neural network model does not have good generalization, then the model will not have practical application value. Convolutional neural network sample enhancement can improve robustness and accuracy. It is an indispensable part of convolutional neural network model training. Given that manual collection is currently impossible, this article will use random processing (random scaling, random rotation, random shift) and Mosaic data enhancement to enhance the data set to expand the original data set.

[0075] Random scaling operations are divided into two types: outward and inward scaling. Outward scaling is essentially to randomly crop a part from the initial image and stretch it to the size of the original image. The processed image size is larger than the initial image. Conversely, inward scaling compresses the initial image and fills the pixels beyond the boundary area with single pixels that are unrelated to the data set. The scaling ratio is generally taken as 0.5 to 2.

[0076] The random flip operation uses the central axis of the image as its flip center. Depending on the axis selected, it can be divided into two flipping methods: horizontal and vertical. Random rotation uses the center of the image as its rotation center and rotates the initial image clockwise or counterclockwise by a random angle. Whether it is actually used as an enhancement of the data set depends on the angle of image rotation and the initial information it carries. In the actual application of photovoltaic power generation systems, the layout is usually longitudinal and transverse. To ensure that the layout within the image does not change too much, it can be rotated by 90°, 180° or 270°.

[0077] Random shifting is not completely random. The shifting only includes moving the image along the X-axis or Y-axis direction or both directions. When performing the shifting, assumptions must also be made about the pixels outside the boundary area.

[0078] The Mosaic method randomly scales four different initial images and splices them into new images for training. After splicing, the images still contain the labels of the original images, which effectively increases the label density in the images and increases the batch size in disguise, which can reduce the batch size requirements of the model itself. The use of only a single GPU can meet the stable training of the model.

[0079] YOLOv8 improves the performance of the model, making it the best choice for tasks such as image segmentation, target detection, and pose estimation. Through a series of innovative improvements, YOLOv8 not only optimizes the performance of the model, but also expands its functional range, making it excel in processing tasks such as image segmentation, target detection, and pose estimation. It can run smoothly on a variety of platforms such as CPU and GPU, fully demonstrating its flexibility and adaptability. In terms of technological innovation, YOLOv8 introduces a new Ancher-Free detection head, backbone network, and loss function. These improvements together improve the accuracy and efficiency of the model. These new features not only make YOLOv8 perform well in the field of target detection, but also provide a solid foundation for its application in other fields. The YOLOv8 model structure diagram is shown below. Figure 1 shown.

[0080] In target detection tasks, IoU (Intersection over Union) is often used to measure the similarity between the model's prediction and the object's annotation. It calculates the union and intersection between the model's prediction and the object's annotation area, and then divides the intersection by the union to obtain a ratio value, and the obtained ratio is between 0 and 1. The larger the ratio of the two, the higher the similarity between the model's prediction and the object's annotation. If the ratio is greater than the predetermined value, the predicted box and the true box can be considered to be matched successfully. During network training, IoU is also often used as part of the loss function to improve the network's performance for target detection tasks. IoU is defined as follows:

[0081]

[0082] In actual network training, IoU has an unignorable defect, when two boxes do not overlap with each other, that is: W i =0,or,H i = 0, the gradient calculation result of IoU during back propagation is 0. This will result in the width W of the area where the two boxes overlap during training. i Unable to get updates.

[0083] WIoU (Wise-IoU) takes a completely different approach, using a dynamic method to calculate the prediction loss, defined as follows:

[0084] L WIoU =R WIoU L IoU

[0085]

[0086] Where R WI o U ∈[1,e), which makes L of the normal quality anchor box IoUSignificantly enlarged. I o U ∈[0,1], the R WIoU It is significantly reduced, and when the target box overlaps well with the anchor box, it pays less attention to the distance to the center point. WIoU Hinder the convergence of the model, W g ,H g By separating them, we can effectively remove the factors that hinder the convergence of the model.

[0087] In the actual training set, due to factors such as the shooting environment and weather, it is inevitable that there will be low-quality annotations in the data set, and geometric metrics such as aspect ratio and distance may increase the penalty for them, thus affecting the generalization performance of the model. When the prediction box and the target box have a good overlap, the loss function should weaken the penalty from the geometric metric to avoid excessive intervention in the training, so that the model can have better generalization performance.

[0088] In view of the idea of ​​dynamic processing of WloU, a new loss function based on IoU is proposed, named HIoU, which is defined as:

[0089]

[0090] Compared with CioU and WioU, the HIoU proposed in this application can first dynamically adjust the regression loss between bounding boxes like WIoU. In the early stage of training, the IoU between the predicted box and the labeled box is relatively small. At this time, the model should focus on improving the IoU between the predicted box and the labeled box. In the first term R of the HIoU loss above, WIoU ×L IoU R WIoU The penalty imposed by the model on smaller IoU can be amplified. When the training is in the later stage, the overlap of IoU between the predicted box and the labeled box is basically maintained at a high level and will not change much. WIoU ×L IoU L IoU It will decay to a smaller value. At this time, the center of gravity of the candidate box regression will be automatically placed on the regression of the center point and aspect ratio by the model, further making the predicted box regress to the labeled box, thereby improving the performance of the model.

[0091] As the network structure in the target detection task continues to deepen, the characteristics of the target are gradually weakened. Therefore, in the target detection task, it is easy to cause problems such as missed detection and wrong detection of the target. Considering that the defects of solar photovoltaic panels vary in type and have no fixed size, different attention mechanisms are embedded in different positions of the neural network to improve the accuracy of the network. However, in practical applications, there is no mature theory for which position of the attention mechanism is most effective for different positions of the network. This application selects three positions of the neural network and inserts the attention mechanism respectively.

[0092] After the attention mechanism is embedded in the SPPF module of the Backbone layer: In YOLOv8, the SPPF module is deployed between the Backbone network and the Neck network. After the input image passes through the Backbone network, the feature information of the image is extracted and input into the SPPF module through multiple layers of convolution. In the SPPF module, the features obtained by convolution in the Backbone network are integrated, and the features of different scales are fused to obtain a feature vector of fixed length, which is then input into the Neck network. The improved network has the following advantages: (1) The SPPF module fuses features of different scales, which is essentially a spatial pyramid pooling. After the attention mechanism is embedded in the SPPF module, the pooled features can be redistributed with weights to better capture the features of the image. (2) The image processed by the SPPF module is directly input into the Neck network. After the attention mechanism is embedded, the input image features will receive more attention, thereby achieving the purpose of enhancing the Neck network's processing of image features. (3) The size scale of the image features after the SPPF module is unified. Embedding the attention mechanism at this position can reduce the time consumed by training. The improved network is as follows: Figure 2 shown.

[0093] The Head network is connected to different positions of the Backbone network. This is because the image features input to the Head network come from different positions of the Backbone, and the Head network is directly connected to the three detection heads. Therefore, after the convolution operation is performed on the input features, the scales of the image features input to the detection heads are also different. The three C2fs in the Neck network have undergone different layers of convolution, and their output image features have different sizes. Combining the attention mechanism with the C2f module can redistribute the weights of features of different sizes. In the initial network, C2f is directly connected to the detection head. Embedding the attention mechanism at this position can make the image features input to the three detection heads receive more attention. The improved network structure is as follows: Figure 3 shown.

[0094] Although YOLOv8 officially divides the network into only two parts, the middle part can still be regarded as the Neck network. Since it is between the Backbone network and the output head, the Neck network plays a connecting role. The Neck network has a special structure from bottom to top and from top to bottom. Features of different scales can be fully integrated in the network, so the structure of the Neck network greatly affects the performance of the algorithm. Therefore, on the basis of combining the attention mechanism with the three C2f of the Head network, the attention mechanism is embedded after the C2f in the middle of the Neck network. The Neck network inputs the image features input from the Backbone network into the Head network through different layers of convolution. Therefore, on the basis of combining the attention mechanism with the three C2f of the Head network, the attention mechanism is embedded after the C2f of Neck, which can ensure that each convolution layer from the Neck network to the Head network can redistribute weights through the attention mechanism, thereby further improving the performance of the model. However, since too many attention mechanisms are inserted into the model, the training speed of the model may be affected. Therefore, the attention mechanism is combined with the three C2fs of the Head network, and on this basis, the attention mechanism is embedded after the C2f of the Neck and trained separately to ensure that the model training has sufficient speed. The improved network structure is as follows Figure 4 shown.

[0095] In the present invention, the training and testing of all convolutional neural network models in this application are carried out on computers with Microsoft Windows 11 operating system. The hardware facilities of the experimental computer are CPU model Intel (R) Core (TM) i7-9750H, GPU model GeForce RTX 1660Ti, GPU available video memory is 6144MiB, and the deep learning programming frameworks used in the experiment are TensorFlow and Keras. Based on the deep learning framework, Python language is used, code is written using the PyCharm platform, and the improved model is built for experiment. The specific information of the hardware and software environment configuration of the experimental machine is shown in Table 1.

[0096] Table 1

[0097] Configuration items Item Value operating system Microsoft Windows 11 Home Chinese Edition CPU Intel(R)Core(TM)i7-9750H GPU GeForce RTX 1660Ti Available video memory 6144MiB Deep Learning Programming Framework TensorFlow, Keras

[0098] In order to better evaluate the performance of the improved photovoltaic panel defect model compared with other models, accuracy (Precision, P), recall (Recall, R) and mAP are selected as indicators. Among them, mAP is applied to the evaluation of multi-label image classification tasks and has become one of the important criteria for measuring the performance of models in the field of target detection. This application uses the PR curve to evaluate the performance of the model, with P as the ordinate and R as the abscissa. The larger the area under the two curves, the better the model performance.

[0099] In order to solve the problem of too few original data sets, this application adopts data enhancement technology. Through digital image processing algorithms, this application will use random processing methods (random scaling, random rotation, random shift) and Mosaic data enhancement to enhance the data set to achieve the expansion of the original data set. This method not only increases the amount of model training data, but also avoids overfitting of the model. At the same time, this data enhancement can also simulate the shooting of defective images in different scenes. For example, changing the brightness can simulate the impact of sunlight on the image, and rotation and translation can simulate the impact of different shooting angles on the image. This method improves the anti-interference performance of the model, and can better adapt to different image changes, thereby improving the accuracy of classification and recognition. In this experiment, a comparative experiment will be used to verify the impact of data enhancement on YOLOv8. The accuracy comparison chart before and after data set expansion is as follows Figure 5 shown.

[0100] The YOLOv8 network uses the unexpanded training set for accuracy testing. The test curve is unstable and oscillating. The final accuracy of the test set test is only 89.05%. During the training process of the expanded training set, the curve is relatively flat, there is no curve oscillation, the model converges quickly, and the recognition accuracy reaches 93.17%. The accuracy is also improved compared to the unexpanded data set. Data enhancement can make the model more sensitive to the diversity and variability of the input data, thereby improving the robustness of the model. Secondly, it avoids the phenomenon of overfitting due to too few original data sets. Increasing the number of training samples and enriching the diversity can avoid the intelligence of overfitting in the model. Data collection and labeling are important steps in building a deep learning model. Data enhancement technology can save the cost of data collection and labeling and reduce the demand for new data.

[0101] This application proposes HIoU as a new loss function, which can effectively improve the training convergence speed of the model. When training the model, the original IoU loss function and HIoU loss function are used to compare the convergence speed of model training. Figure 6As shown, the left figure is the convergence curve of the initial model training. It can be seen from the figure that the initial model completed convergence after about one hundred and fifty training times. The right figure is the convergence curve of the model training using HIoU as the loss function. It can be seen that the model completed convergence after about one hundred training times. From the comparison of experimental results, it can be seen that using HIoU as the new loss function of the model can effectively improve the convergence speed of the model, reduce the training time of the model, and realize the rapid establishment of the model.

[0102] In order to improve the YOLOv8 model, this application selects a variety of different attention mechanisms and embeds them in different positions of the neural network. By training the annotated data set in the improved model, the accuracy of the model can be improved. The performance of the improved model is evaluated on the test set and compared with the initial model YOLOv8n used. In the following experimental results analysis, the three different positions in the previous text are marked as Attention-A, Attention-B, and Attention-C.

[0103] Polarized self-attention,The data of using Polarized self-attention to improve the convolutional neural network is shown in Table 2.

[0104] Table 2

[0105] Model crack Broken fence spot average YOLOv8n 82.9％ 88.3％ 93.7％ 88.3％ Polarized-A 83.2％ 87.7％ 92.7％ 87.9％ Polarized-B 88.3％ 88.8％ 90.5％ 89.2％ Polarized-C 86.9％ 87.8％ 90.0％ 88.3％

[0106] Compared with the YOLOv8n model, embedding Polarized self-attention in the convolutional neural network effectively improves the accuracy of the model. Among them, the combination of the attention mechanism and the three C2fs in the Head network has the largest increase, with the average accuracy increased by 0.9%. Although the accuracy of detecting "spot" defects has slightly decreased, the detection accuracy of "crack" defects has increased by 6.4%, and the accuracy of "broken gate" defects has also increased by 0.5%.

[0107] However, when the attention mechanism is embedded in the middle of the Neck network, the training effect is not ideal. Even after the attention mechanism is embedded in the SPPF of the Backbone network, the accuracy is reduced. The reason is speculated that the dot product of Polarized self-attention and the vector at each position is different from the image feature scale that needs to be processed in the convolution layer in the middle of the Neck after SPPF pooling, which cannot improve the performance of the model.

[0108] CoTAttention, using CoTAttention to improve the convolutional neural network data as shown in Table 3:

[0109] Table 3

[0110] Model crack Broken fence spot average YOLOv8n 82.9％ 88.3％ 93.7％ 88.3％ CoTAttention-A 86.9％ 87.5％ 93.4％ 89.3％ CoTAttention-B 86.2％ 88.1％ 90.1％ 88.1％ CoTAttention-C 88.2％ 87.4％ 91.1％ 88.9％

[0111] By comparing with the YOLOv8n model, it is found that embedding the CoTAttention mechanism in the convolutional neural network can significantly improve the performance of the model. After embedding the attention mechanism into the SPPF of the Backbone network, the average accuracy of the initial model increased by 1%. While ensuring that the high accuracy of "spots" and "broken fences" does not decay much, the accuracy of "cracks" is improved by 3.3%. Embedding the attention mechanism in the middle of the Neck network also improves the average accuracy by 0.6%. Although the accuracy decreases after combining the attention mechanism with the three C2fs in the Head network, it can be seen that embedding CoTAttention in the convolutional neural network can effectively improve the performance of the model. It is speculated that the reason is that the CoTAttention mechanism achieves better exchange and integration of information by calculating the interactive attention between features, which has a good effect on small target detection.

[0112] CoordAttention, using CoordAttention to improve the convolutional neural network data as shown in Table 4:

[0113] Table 4

[0114] Model crack Broken fence spot average YOLOv8n 82.9％ 88.3％ 93.7％ 88.3％ CoordAttention-A 83.7％ 88.2％ 93.5％ 88.5％ CoordAttention-B 85.4％ 85.7％ 55.8％ 75.6％ CoordAttention-C 81.5％ 88.6％ 91.8％ 87.3％

[0115] By comparing with the YOLOv8n model, it was found that the model detection accuracy was improved by 0.2% only after the attention mechanism was embedded in the SPPF of the Backbone network. When CoordAttention was embedded in other positions selected in this application, the detection accuracy dropped significantly. It can be seen that embedding CoordAttention in the convolutional neural network cannot effectively improve the performance of the model. It is speculated that the main reason is that while considering the input feature information, it also considers the position information of each pixel, which may ignore small targets and is not suitable for the detection of small targets such as photovoltaic panel defects.

[0116] In summary, among the positions selected for embedding the attention mechanism in this application, embedding the attention mechanism in the SPPF of the Backbone network and combining it with the three C2fs in the Head network have a greater impact on the model performance, while combining the attention mechanism with the three C2fs in the Head network and embedding it in the C2f of the neck at the same time have no significant impact on the model performance. Although in practical applications, it is not certain which position of the neural network is most effective to embed the attention mechanism, for embedding the attention mechanism in the neural network, considering that deep learning takes a long time, it can be given priority to embed the attention mechanism in the first two positions proposed in this design, and then select the third position to embed the attention mechanism.

[0117] For the different attention mechanisms selected in this application, embedding Polarized self-attention and CoTAttention mechanisms have significantly improved the performance of the model, while embedding NAM and CoordAttention mechanisms has no or insignificant improvement on the performance of the model. By comparing YOLOv8n with the improved model, it can be seen that the average accuracy of embedding CoTAttention into the SPPF of the Backbone layer of the neural network is the highest, which is 1% higher than the initial model. For the detection of "cracks", Polarized-B has the highest improvement, which is 5.4% higher than the initial model. Comparing the three different positions, the average accuracy of the attention mechanism embedded after the SPPF of the backbone layer is the highest, and the combination of the attention mechanism with the 3 C2f in the head is lower. It is speculated that the reason may be that there is more shallow feature information in the backbone layer, and this information plays a very important role in the target recognition of photovoltaic panel defects.

[0118] This application proposes a solar photovoltaic panel defect detection method with an improved convolutional neural network model. This method optimizes the data enhancement method at the input of the original model and improves the bounding box regression loss function HIoU. In terms of model improvement, the attention mechanism is combined at different positions, and corresponding defect detection models are proposed for different defect characteristics. Through the above improvements, the feature extraction and training convergence speed of the model are improved, the model overfitting problem is optimized, and the detection effect of different categories of defect targets is enhanced, thereby improving the detection accuracy and efficiency of surface defects of solar photovoltaic panels. It can detect defects and faults of photovoltaic panels in a timely manner, which has practical significance for the quality assurance of solar photovoltaic panels in industrial production. At the same time, it provides a good guarantee for the normal operation of photovoltaic power plants.

[0119] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. A solar photovoltaic panel defect detection method based on an improved convolutional neural network, a panel defect detection method, characterized in that: It includes: S1. Extract the photovoltaic panel defect image dataset from the public dataset, mark the defects in the image dataset, annotate the defects with boxes, name and classify the defects, generate an XML file corresponding to the entire image dataset, and then convert the photovoltaic panel defect images and the corresponding XML files into a VOC dataset; S2, enhance the VOC data set; S3, using an improved convolutional neural network to build a photovoltaic panel defect detection model to detect defects in photovoltaic panel images; The improved convolutional neural network adopts the YOLOv8 neural network, the detection head adopts the Ancher-Free detection head, and the loss function adopts the HIoU loss function. The attention mechanism is embedded in the back of the SPPF module of the Backbone layer, the attention mechanism is embedded in the back of the C2f module of the Head layer, and the attention mechanism is embedded in the back of the C2f module in the middle of the Neck layer.

2. The solar photovoltaic panel defect detection method based on improved convolutional neural network according to claim 1 is characterized in that: The defects in the image data set S1 include cracks, broken grids and spots.

3. The solar photovoltaic panel defect detection method based on improved convolutional neural network according to claim 2 is characterized in that: The ratio of the three defects in the image dataset is 1:1:

1.

4. The solar photovoltaic panel defect detection method based on improved convolutional neural network according to claim 1 is characterized in that: The enhancement processing of the VOC data set described in S2 includes: random scaling, random rotation, random shift and Mosaic data enhancement.

5. The solar photovoltaic panel defect detection method based on improved convolutional neural network according to claim 4 is characterized in that: The specific method of random scaling includes: Outward scaling: randomly crop a portion of the original image and then stretch it to the original size; Inward scaling compresses the initial image and fills the pixels beyond the boundary area with a single pixel. The single pixel used for filling has nothing to do with the defect image dataset.

6. The solar photovoltaic panel defect detection method based on improved convolutional neural network according to claim 4 is characterized in that: The specific method of random rotation includes: Random flipping: using the central axis of the image as the flipping center, flip the initial image horizontally or vertically; Randomly rotate the image clockwise or counterclockwise with the center of the image as the rotation center.

7. The solar photovoltaic panel defect detection method based on improved convolutional neural network according to claim 4 is characterized in that: The specific method of random shifting includes: Move along the X-axis, Y-axis, or both, while making assumptions about pixels outside the boundary area.

8. The solar photovoltaic panel defect detection method based on improved convolutional neural network according to claim 1 is characterized in that: The specific method of Mosaic data enhancement includes: Four different initial images are randomly scaled and then spliced ​​into new images for training.

9. The solar photovoltaic panel defect detection method based on improved convolutional neural network according to claim 1, characterized in that: The HIoU loss function specifically includes: Among them, L HIoU represents the HIoU loss function, L IoU represents the IoU loss function, R WIoU represents the penalty function, x represents the x-axis coordinate of the original prediction box, x gt represents the x-axis coordinate of the new target prediction box, y represents the y-axis coordinate of the original target function, gt Represents the y-axis coordinate of the new target prediction box, W g Indicates the width of the minimum prediction box, H g represents the height of the minimum prediction box, v represents the proportional consistency coefficient between the width and height of the prediction box, and α represents the adjustment coefficient of v.

10. The solar photovoltaic panel defect detection method based on improved convolutional neural network according to claim 1, characterized in that: The specific method of embedding the attention mechanism after the SPPF module of the Backbone layer includes: The SPPF module is placed between the Backbone layer and the Neck layer, and the attention mechanism is embedded in the back side of the SPPF module; The input image extracts feature information through the Backbone layer and is input into the SPPF module after multiple layers of convolution. The SPPF module integrates the feature information obtained by convolution in the Backbone layer, fuses features of different scales, obtains a feature vector of fixed length, and then inputs it into the Neck layer.

Images