Photovoltaic power station equipment defect detection method based on digital twinning

Abstract

The invention discloses a photovoltaic power station equipment defect detection method based on digital twinning, and the method is characterized in that the method comprises the steps: image data collection, data set construction, model feature extraction network improvement, model loss function improvement, model training and verification, model testing, and equipment defect detection when a model is applied to a photovoltaic power station. According to the method, the abnormal equipment in the photovoltaic power station can be accurately detected according to the real-time image obtained in the digital twinborn system, and simulation and analysis data are provided for the digital twinborn system of the photovoltaic power station.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and specifically relates to a photovoltaic power station equipment defect detection method based on digital twins. Background Art

[0002] Photovoltaic power stations include physical equipment such as photovoltaic panels, inverters, junction boxes, and weather stations. The equipment may age or malfunction due to bad weather and daily maintenance damage. In particular, defects in photovoltaic panels will lead to a decrease in power generation efficiency, which will directly affect the economy and reliability of photovoltaic power station operation. Photovoltaic power stations usually have a large number of photovoltaic modules deployed in large areas. The traditional manual inspection method of naked eye observation and manual recording is affected by human factors and natural factors such as weather and environment, resulting in low detection efficiency, low accuracy, and insufficient ability to analyze detection results.

[0003] Object detection is a computer vision technology based on deep learning. Its task is to find all the objects of interest in the image and determine their location and category. It has been widely used in many fields such as power systems, smart cities, high-end equipment manufacturing, and smart medical care. Most of the existing deep learning object detection methods use deep convolutional neural networks to extract image features, so the huge amount of parameters generated reduces the detection speed. In addition, excessive downsampling of feature maps may cause small targets to not occupy a pixel on the deep feature map, and their detailed features may disappear completely, which will affect the detection accuracy of the algorithm. In addition, the object detection method based solely on deep learning relies on a limited number of image datasets to train the model. When the number of images in the dataset is insufficient or the scene and target categories are single, the model training may suffer from underfitting or overfitting problems, thereby limiting the improvement of the model detection accuracy.

[0004] Digital twin is a simulation process that integrates multiple disciplines, multiple physical quantities, multiple scales, and multiple probabilities. In the digital twin system combined with the deep learning target detection algorithm, by making full use of physical models, sensor data, historical operation data, and real-time status data to model, deduce, and simulate in virtual space, it can map the entire life cycle of physical equipment, thereby determining the best maintenance and operation plan for photovoltaic power plants. The digital twin system can provide richer and continuously updated image data for the target detection model, thereby improving the learnable resources of the target detection model. Furthermore, the target detection model can provide accurate and reliable detection result data for the digital twin system, thereby improving the visualization, data analysis, and mapping simulation capabilities of the digital twin system. Summary of the invention

[0005] The purpose of the present invention is to address the shortcomings of existing target detection algorithms and propose a photovoltaic power station equipment defect detection method based on digital twins.

[0006] To achieve the above-mentioned purpose of the invention, the present invention provides a photovoltaic power station equipment defect detection method based on digital twin, characterized in that: the method specifically comprises the following steps:

[0007] Step 1, image data acquisition: Photovoltaic panels and photovoltaic power station equipment are photographed by an EL detector and a high-definition camera respectively, to obtain a photovoltaic power station equipment image mainly based on a near-infrared image of the photovoltaic panel;

[0008] Step 2, build a data set: annotate the abnormal targets in the image and obtain a label file in XML format, divide the annotated images and label files into a training data set, a verification data set, and a test data set, with a ratio of 6:2:2;

[0009] Step 3: Improvement of the model feature extraction network: Design two residual blocks as components of the feature extraction network, which reduces the number of parameters of the feature extraction network and the number of downsampling of the feature map, thereby improving the feature extraction and expression capabilities of the model while ensuring the detection speed;

[0010] Step 4: Improvement of model loss function: Based on the VFNet target detection network model, a regression loss function involving multiple geometric factors is designed;

[0011] Step 5, model training and verification: The dataset images are used as the input of the algorithm model to train the model. The purpose of training is to continuously optimize and correct the model parameters, and finally make the loss function converge to the global optimum, so that the model can be trained every certain number of rounds. The validation set is used to evaluate and verify the current performance of the model. mAP and recall rate are used as evaluation indicators of model detection accuracy. If mAP and recall rate reach the upper limit, it means that the model has been fully trained under the current hyperparameters.

[0012] Step 6, model testing: Use the test set images to perform a visual test on the model. When the model can quickly and accurately detect abnormal targets in the image, that is, the accuracy and detection speed meet the actual detection requirements, the model training is completed; otherwise, after adjusting the hyperparameters, repeat step 5 to continue training;

[0013] Step 7: Apply the model to the digital twin system of the photovoltaic power station: deploy the trained algorithm model on the target detection server of the digital twin system, which can effectively detect black spots, cracks, short circuits and other equipment abnormalities inside the photovoltaic panels; the digital twin system combines target detection data, real-time equipment status data and historical power station operation data to simulate and deduce the entire life cycle process of the photovoltaic power station.

[0014] Furthermore, the model feature extraction network in step 3 is improved as follows:

[0015] Two residual blocks, Bottleneck_s and Bottleneck_d, are designed to improve the feature extraction network. Bottleneck_s splits the single-layer 3×3 convolution in the original Bottleneck into two convolution layers: 1×3 convolution and 3×1 convolution. Taking the fourth convolution group in ResNet-101 as an example, when the bias is not considered, the number of parameters generated by the 3×3 standard convolution layer in the middle of the original Bottleneck module is 3×3×256×256, or 589,824. In the Bottleneck_s residual block, the parameters of the two middle convolution layers are 1×3×256×256+3×1×256×256, or 393,216. Compared with the original Bottleneck residual block, Bottleneck_s reduces the number of parameters of the 3×3 standard convolution layer by 33.33% through convolution splitting. In the feature extraction network, after replacing all Bottleneck modules in the fourth convolution group with Bottleneck_s, the number of parameters of the feature extraction network is significantly reduced.

[0016] The designed Bottleneck_d uses a 3×3 dilated convolution with a convolution kernel stride of 1, 2 padding layers, and a dilation rate of 2 to replace the original 3×3 standard convolution with a stride of 2 and a padding rate of 1 in Bottleneck. For the 3×3 dilated convolution layer under this parameter configuration, the feature map does not change in size after passing through Bottleneck_d. The convolution group using Bottleneck_d does not downsample the feature map as before. Bottleneck_d is applied to the fifth convolution group of a 101-layer deep residual network (Resnet-101), and Bottleneck_d is used to replace the first residual block in the fifth convolution group. The size of the feature map does not halve after passing through the fifth convolution group, ensuring that the feature map finally output by the convolutional neural network has a larger size and receptive field. The backbone network can extract more semantic information, which is conducive to improving the model's detection performance for small objects in the image; Bottleneck_d can reduce one upsampling operation, thereby reducing the loss of fine-grained features caused by pixel interpolation during the upsampling process of the feature map.

[0017] Furthermore, the model loss function in step 4 is improved as follows:

[0018] Design a regression loss function that can more comprehensively reflect the quality of prediction box positioning, the MIoU regression loss function. The expression of the MIoU regression loss function is shown in (1):

[0019]

[0020] In the formula, It is the GIoU regression loss, which is used to simply reflect the degree of overlap between the predicted box and the true box. It is used to reflect the normalized distance between the center points of the predicted box and the true box. The closer the predicted box is to the true box, the smaller the center point distance d0 is. Therefore, this item can provide accurate moving direction for the predicted box by directly minimizing the center point distance, thereby accelerating the network parameter optimization and the regression process of the learnable suggestion box. D is the offset variation coefficient, which is numerically equal to the ratio of the standard deviation to the mean of d0, d1, and d2, and can reflect the degree of discreteness of d0, d1, and d2. The expression of the offset variation coefficient is as follows:

[0021]

[0022] In the formula, is the mean of d0, d1, and d2; the closer the size and aspect ratio of the predicted box and the real box are, the smaller the discreteness of d0, d1, and d2; therefore, the MIoU regression loss function can optimize the predicted box by minimizing the coefficient of variation, making the predicted box closer to the real box in terms of size and aspect ratio; the role of the denominator in the penalty term where D is located is to adjust the contribution of the penalty term to the loss through normalization; the MIoU loss function not only retains the advantages of the GIoU loss function, but also takes into account the two factors of center point distance and offset coefficient of variation. More importantly, the MIoU loss function can reflect the degree of closeness of the bounding box in size and aspect ratio with only one penalty term, avoiding the problem of slow or even difficult convergence of the loss function due to too many penalty terms; in the process of minimizing the MIoU loss function, each penalty term increases the overlapping area of ​​the predicted box and the real box, decreases the distance between the center points, and gradually approaches the size and aspect ratio. In the most ideal case, when the predicted box and the real box are infinitely close to overlap, the MIoU regression loss value approaches 0 infinitely.

[0023] The improved total loss consists of two parts: classification loss and regression loss. The expression of the total loss function is shown in (3):

[0024] Loss = λ cls ·L cls +λ reg ·L MIoU (3)

[0025] Where, L cls is the classification loss generated by the fully connected head, using the Focal Loss function; L MIoU is the regression loss generated by the two paths in the fully connected head and the convolutional head, and the MIoU regression loss function proposed in this invention is adopted. cls and λ regare the weights of classification loss and regression loss, respectively, which are used to balance the loss contribution of each loss item. The experimental results are shown in λ cls and λ reg The proposed method shows the best performance when 2 and 4.8 respectively.

[0026] Table 1 Performance comparison of the proposed method and other algorithms

[0027]

[0028] The present invention provides a photovoltaic power station equipment defect detection method based on digital twins, which can accurately detect equipment defects in photovoltaic power stations. The digital twin system can provide richer and continuously updated on-site image data for the target detection model, thereby improving the learning ability of the target detection model. The target detection model can provide accurate and reliable detection result data for the digital twin system, thereby improving the visualization performance, data analysis ability and mapping simulation ability of the digital twin system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of a method for defect detection of photovoltaic power station equipment based on digital twins of the present invention;

[0030] Figure 2 It is a feature extraction network component of an embodiment of the present invention, a Bottleneck_s residual block;

[0031] Figure 3 It is a feature extraction network component of an embodiment of the present invention, a Bottleneck_d residual block;

[0032] Figure 4 Schematic diagram of a feature extraction network structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in detail below using the accompanying drawings and embodiments.

[0034] See also Figure 1-Figure 4 , a photovoltaic power station equipment defect detection method based on digital twins proposed in an embodiment of the present invention is used to accurately detect abnormal targets in a photovoltaic power station. The method specifically includes the following steps:

[0035] Step 1, image data acquisition: Photovoltaic panels and photovoltaic power station equipment are photographed respectively by an EL detector and a high-definition camera to obtain a photovoltaic power station equipment image mainly based on the near-infrared image of the photovoltaic panel.

[0036] Step 2, build a data set: mark the abnormal targets in the image and obtain the label file in XML format, divide the marked images and label files into training data set, verification data set and test data set in a ratio of 6:2:2, and make a data set in Pascal VOC format.

[0037] Step 3, model feature extraction network improvement: Based on the VFNet target detection network model, two residual blocks are designed as components of the feature extraction network, which reduces the number of parameters of the feature extraction network and the number of downsampling times of the feature map, thereby improving the feature extraction and expression capabilities of the model while ensuring the detection speed.

[0038] Step 4, model loss function improvement: Based on the VFNet target detection network model, an improved regression loss function is designed; the new regression loss function can comprehensively reflect the positioning quality of the prediction box through a variety of geometric factors, thereby supervising the model to reasonably and effectively train the network parameters, and ultimately improve the detection accuracy.

[0039] Step 5, model training and verification: The dataset images are used as the input of the algorithm model to train the model. The purpose of training is to continuously optimize and correct the model parameters, and finally make the loss function converge to the global optimum. After a certain number of training rounds, the model's current performance is evaluated and verified using the validation set. mAP and recall rate are used as evaluation indicators of model detection accuracy. The limit of mAP and recall rate indicates that the model has been fully trained under the current hyperparameters.

[0040] Step 6, model testing: Use the test set images to perform a visual test on the model. When the model can quickly and accurately detect abnormal targets in the image, that is, the accuracy and detection speed meet the actual detection requirements, the model training is completed. Otherwise, after adjusting the hyperparameters, repeat step 5 to continue training.

[0041] Step 7: Apply the model to the digital twin system of the photovoltaic power station: Deploy the trained algorithm model on the target detection server of the digital twin system, which can effectively detect black spots, cracks, short circuits and other equipment abnormalities inside the photovoltaic panels. The digital twin system combines target detection data, real-time equipment status data and historical power station operation data to simulate and interpret the entire life cycle of the photovoltaic power station.

[0042] The model feature extraction network improvement in step 3 is expressed as:

[0043] The feature extraction network is improved by designing two residual blocks, Bottleneck_s and Bottleneck_d. Bottleneck_s splits the single-layer 3×3 convolution in the original Bottleneck into two convolution layers: 1×3 convolution and 3×1 convolution. In the feature extraction network, after replacing all Bottleneck modules in the fourth convolution group with Bottleneck_s, the number of parameters of the feature extraction network is significantly reduced.

[0044] In the designed Bottleneck_d, a 3×3 dilated convolution with a kernel stride of 1, a padding layer of 2, and a dilation rate of 2 is used to replace the original 3×3 standard convolution with a stride of 2 and a padding rate of 1 in Bottleneck. For the 3×3 dilated convolution layer with this parameter configuration, the feature map does not change in size after passing through Bottleneck_d. The convolution group using Bottleneck_d does not downsample the feature map as before. Bottleneck_d is applied to the fifth convolution group of the 101-layer deep residual network (Resnet-101). Bottleneck_d is used to replace the first residual block in the fifth convolution group. The size of the feature map will not be halved after passing through the fifth convolution group, ensuring that the feature map finally output by the convolutional neural network has a larger size and receptive field. The backbone network can extract more semantic information, which is conducive to improving the model's detection performance for small targets in the image. Bottleneck_d can reduce one upsampling operation, thereby reducing the loss of fine-grained features of the feature map caused by pixel interpolation during the upsampling process.

[0045] The improvement of the model loss function in step 4 is expressed as:

[0046] Design a regression loss function that can more comprehensively reflect the quality of prediction box positioning, the MIoU regression loss function. The expression of the MIoU regression loss function is shown in (1):

[0047]

[0048] In the formula, It is the GIoU regression loss, which is used to simply reflect the degree of overlap between the predicted box and the true box. It is used to reflect the normalized distance between the center points of the predicted box and the true box. The closer the predicted box is to the true box, the smaller the center point distance d0 is. Therefore, this item can provide an accurate moving direction for the predicted box by directly minimizing the center point distance, thereby accelerating the network parameter optimization and the regression process of the learnable suggestion box. D is the offset variation coefficient, which is numerically equal to the ratio of the standard deviation to the mean of d0, d1, and d2, and can reflect the degree of discreteness of d0, d1, and d2. The expression of the offset variation coefficient is as follows:

[0049]

[0050] Where d is the mean of d0, d1, and d2. The closer the size and aspect ratio of the predicted box and the true box are, the smaller the discreteness of d0, d1, and d2. Therefore, the MIoU regression loss function can optimize the predicted box by minimizing the coefficient of variation, making the predicted box closer to the true box in terms of size and aspect ratio. The role of the denominator in the penalty term where D is located is to adjust the contribution of the penalty term to the loss through normalization. The MIoU loss function not only retains the advantages of the GIoU loss function, but also takes into account the two factors of center point distance and offset coefficient of variation. More importantly, the MIoU loss function can reflect the degree of closeness of the bounding box in size and aspect ratio with only one penalty term, avoiding the problem of slow or even difficult convergence of the loss function due to too many penalty terms. In the process of minimizing the MIoU loss function, each penalty term increases the overlapping area of ​​the predicted box and the true box, continuously reduces the distance between the center points, and gradually approaches the size and aspect ratio. In the most ideal case, when the predicted box and the true box are infinitely close to overlap, the MIoU regression loss value approaches 0 infinitely.

[0051] The improved total loss consists of two parts: classification loss and regression loss. The expression of the total loss function is shown in (3):

[0052] Loss = λ cls ·L cls +λ reg ·L MIoU (3)

[0053] Where, L cls is the classification loss generated by the fully connected head, using the Focal Loss function; L MIoU It is the regression loss generated by the two paths in the fully connected head and the convolutional head, and adopts the MIoU regression loss function proposed in this invention. cls and λ reg They are the weights of classification loss and regression loss, respectively, used to balance the loss contribution of each loss item.

Claims

1. A photovoltaic power station equipment defect detection method based on digital twins, characterized in that The following steps are involved: Step 1, image data acquisition: Photovoltaic panels and photovoltaic power station equipment are photographed by an EL detector and a high-definition camera respectively, to obtain a photovoltaic power station equipment image mainly based on a near-infrared image of the photovoltaic panel; Step 2, build a data set: annotate the abnormal targets in the image and obtain a label file in XML format, divide the annotated images and label files into a training data set, a verification data set, and a test data set, with a ratio of 6:2:2; Step 3: Improvement of model feature extraction network: Based on the VFNet target detection network model, two residual blocks are designed as components of the feature extraction network, which reduces the number of parameters of the feature extraction network and the number of downsampling of feature maps, thereby improving the feature extraction and expression capabilities of the model while ensuring the detection speed; Step 4: Improvement of model loss function: Based on the VFNet target detection network model, a regression loss function involving multiple geometric factors is designed; Step 5, model training and verification: The dataset images are used as the input of the algorithm model to train the model. The purpose of training is to continuously optimize and correct the model parameters. After a certain number of training rounds, the current performance of the model is evaluated and verified using the verification set. mAP and recall rate are used as evaluation indicators of model detection accuracy. If mAP and recall rate reach their upper limit, it means that the model has been fully trained under the current hyperparameters. Step 6, model testing: Use the test set images to perform a visual test on the model. When the model can quickly and accurately detect abnormal targets in the image, that is, the accuracy and detection speed meet the actual detection requirements, the model training is completed. Otherwise, after adjusting the hyperparameters, repeat step 5 to continue training. Step 7: Apply the model to the digital twin system of the photovoltaic power station: Deploy the trained algorithm model on the target detection server of the digital twin system, which can effectively detect black spots, cracks, short circuits and other equipment abnormalities inside the photovoltaic panels. The digital twin system combines target detection data, real-time equipment status data and historical power station operation data to simulate and deduce the entire life cycle of the photovoltaic power station.

2. A photovoltaic power station equipment defect detection method based on digital twin according to claim 1, characterized in that: The model feature extraction network improvement in step 3 is expressed as: The designed Bottleneck_s splits the single-layer 3×3 convolution in the original Bottleneck into two convolution layers: 1×3 convolution and 3×1 convolution. Finally, Bottleneck_s replaces all Bottleneck modules in the fourth convolution group of the 101-layer deep residual network. The designed Bottleneck_d uses a 3×3 dilated convolution with a convolution kernel stride of 1, a padding layer of 2, and a dilation rate of 2 to replace the original 3×3 standard convolution with a stride of 2 and a padding rate of 1 in Bottleneck. Finally, Bottleneck_d is applied to the fifth convolution group of the 101-layer deep residual network.

3. The photovoltaic power station equipment defect detection method based on digital twin according to claim 1 is characterized in that: The improvement of the model loss function in step 4 is expressed as: The MIoU regression loss function expression is shown in (1): In the formula, is the GIoU regression loss, is the normalized distance between the center points of the predicted box and the true box, D is the offset variation coefficient, which is numerically equal to the ratio of the standard deviation of d0, d1, and d2 to the mean. The expression of the offset variation coefficient is as follows: In the formula, is the mean of d0, d1, and d2. The improved total loss consists of classification loss and regression loss. The expression of the total loss function is shown in (3): Loss=λ cls ·L cls +λ reg ·L MI oh (3) Where, L cls is the classification loss generated by the fully connected head, using the Focal Loss function; L MIoU is the regression loss generated by the two paths in the fully connected head and the convolutional head, and the MIoU regression loss function proposed in this invention is adopted. cls and λ reg They are the weights of classification loss and regression loss, respectively, used to balance the loss contribution of each loss item.

Images