A photovoltaic panel inspection method based on deep learning

Abstract

The present invention discloses a photovoltaic panel inspection method based on deep learning, which relates to the field of power inspection and image processing, and the steps are as follows: prepare network input data, and use a drone carrying an imaging device to collect an image data set of a photovoltaic panel; enhance and expand a training set and a test set by flipping, rotating, mirroring, adjusting brightness and other operations to increase data diversity; the present invention designs a detection algorithm for defective areas of photovoltaic panels through an improved Unet segmentation model, improves the network structure of the original Unet model according to the characteristics of drone image collection and the requirements for detecting tiny areas, introduces a dual attention feature fusion module, and sets appropriate network parameters such as activation functions, thereby realizing feature extraction of tiny defects such as cracks, suppressing background interference, and improving the robustness and accuracy of the network, which can effectively solve the problem of photovoltaic panel inspection and provide a guarantee for the stable operation of photovoltaic substations.

Description

Technical Field

[0001] The present invention relates to the technical field of power inspection and image processing, and in particular to a photovoltaic panel inspection method based on deep learning. Background Art

[0002] As the energy crisis and environmental pollution intensify, people are gradually seeking new green energy to replace traditional fossil energy. As a renewable energy, solar energy has the advantages of easy access, sustainable use, and green environmental protection. At the same time, the photovoltaic industry and technology are also constantly developing, and photovoltaic power stations are gradually being built in various places to utilize solar energy. There are a large number of solar panels distributed in these power stations, and each panel is one of the most important components in solar power generation. Its conversion rate and service life are important factors that determine whether the solar cell has use value. As a device that converts light energy into electrical energy, the components of solar panels include solar panels, EVA materials, aluminum alloys, TP backplanes, and tempered glass. Any unexpected errors in the production process will lead to defects. At the same time, solar panels are exposed to harsh environments such as exposure to the sun and rain for a long time, which will inevitably cause varying degrees of damage to the panels. How to discover and deal with these problems in a timely manner directly affects the photoelectric conversion efficiency and service life of solar panels. Traditional detection methods mainly use artificial visual inspection, electroluminescence detection EL method, ultrasonic detection method, etc. However, these detection methods require a lot of manpower, material resources and materials and are extremely inefficient, and cannot meet the actual needs of industrial detection. With the rapid development of drone technology, it has been gradually used in power inspection and other fields. This series of problems can be effectively solved by using industrial drones equipped with visible light or infrared cameras to inspect large photovoltaic power stations. First, drones are used to collect photovoltaic images to be inspected according to the planned inspection route, and then image processing and other technologies are used to perform regional segmentation, separating the foreground area from the background area, and finally fault detection and analysis are performed to draw conclusions.

[0003] At present, when inspecting photovoltaic panels, it is impossible to accurately and quickly inspect the target photovoltaic panel area, and it is even more impossible to obtain the true status of the equipment in time and complete the maintenance task in the faulty area, thus failing to provide guarantee for the stable operation of the entire photovoltaic substation. To this end, we propose a photovoltaic panel inspection method based on deep learning to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to solve the problem in the prior art that when inspecting photovoltaic panels, it is impossible to accurately and quickly inspect the target photovoltaic panel area, and it is impossible to obtain the true status of the equipment in time, find the faulty area and complete the maintenance task, thus failing to provide guarantee for the stable operation of the entire photovoltaic substation. A photovoltaic panel inspection method based on deep learning is proposed to solve the problem that

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A photovoltaic panel inspection method based on deep learning, the steps are as follows:

[0007] Step 1: Prepare network input data and use drones to carry imaging equipment to collect image datasets of photovoltaic panels;

[0008] Step 2: Enhance and expand the training set and test set by flipping, rotating, mirroring, adjusting brightness, etc. to increase the diversity of the data;

[0009] Step 3: Build a parallel attention mechanism optimized segmentation network U-Net;

[0010] Step 4: Build a parallel attention module;

[0011] Step 5: Use the built model to train on the dataset constructed in step 1;

[0012] Step 6: Use the trained network to test the images taken by the drone and fine-tune the network.

[0013] Preferably, prepare network input data, use drones to carry imaging equipment to collect image data sets of photovoltaic panels, including different damage types: breakage, cracks, covering, etc., all images are resized to eight-bit 256pixel × 256pixel, and then use Labelme software to annotate the collected images with true values ​​and output the annotated data set and the corresponding true value map.

[0014] Preferably, the data set is enhanced by rotation, mirroring, scaling, and brightness adjustment to enrich the type and quantity of the data set, and the enhanced data set is divided into a training set, a test set, and a validation set in a ratio of 6:3:1.

[0015] Preferably, a segmentation network U-Net optimized by a parallel attention mechanism is constructed. The segmentation network U-Net optimized by a parallel attention mechanism mainly includes two core parts, upsampling and downsampling. The upsampling part uses a VGG16 network for feature extraction. The first layer of the network first uses two groups of 3×3 convolutions, a batch normalization layer, and a ReLu function layer to form a standard convolution module. The second layer uses maximum pooling for downsampling and then repeats the above standard convolution. The third to fifth layers of the network repeat three groups of the above convolution operations to extract deep semantic features.

[0016] During downsampling, the size of the feature map is halved and the number of channels is doubled for each additional layer, while the number of channels in the fifth layer remains unchanged. During upsampling, the fifth layer of the network is Dropout with a probability of 0.5 to prevent overfitting of the network. Bilinear interpolation is used for upsampling in the model, and then it is sent to the parallel attention mechanism module together with the feature map of the same scale from the fourth layer in the downsampling stage to enhance deep feature extraction. After obtaining the feature map of the ninth layer, a 1×1 convolution combined with a Sigmoid operation is used to output the segmentation result map of the network.

[0017] Preferably, a parallel attention module is constructed. Since there is a large amount of background interference in the photovoltaic panel detection task, a parallel dual attention mechanism is used to optimize the U-Net network for feature extraction; the parallel dual attention module includes the following three parts: channel attention module, spatial attention module, and feature fusion.

[0018] Preferably, the built model is used for training on the data set constructed in step 1; for the input image data X, the feature map is obtained after encoding processing The calculation formula is as follows:

[0019]

[0020] In the formula: Conv represents the convolution operation; function C represents the feature aggregation mechanism: convolution layer + normalization; function Down represents downsampling; L represents the number of network layers; then the obtained feature map is decoded, and the feature map obtained after decoding is The calculation formula is as follows:

[0021]

[0022] Where: [] represents the concatenation of dimensions; Up represents the upsampling of the function; Bp represents the parallel attention mechanism operation; Sigmoid is performed on the final feature map to obtain the final prediction result:

[0023]

[0024] The loss function is defined by the binary cross entropy loss function, which is defined as follows:

[0025]

[0026] Preferably, the channel attention module functions as follows:

[0027] First, define the feature map of the input module as F, and then get a 1×1×C global feature F1 after passing the global maximum pooling layer; then use a one-dimensional convolution of size k to convolve the global feature F1 to obtain the attention weight map G; the weight μ of the i-th channel in the attention weight map G is i It can be obtained by the following formula:

[0028]

[0029] Where: represents the jth weight used to learn the i-th channel weight, 1≤j≤k; represents the set of global features of the kth adjacent channel in the i-th channel;

[0030] The convolution kernel size k of one-dimensional convolution is calculated by the following formula:

[0031]

[0032] Where: C is the number of channels; γ and b represent hyperparameters, usually γ = 2, b = 1; |A| odd Represents the odd number closest to A; the feature map F is weighted by the weight map G to obtain the feature map R; the feature map R increases the channel weights related to the damaged area to be inspected, thereby reducing the weights of other channels.

[0033] Preferably, the spatial attention module functions as follows:

[0034] In the feature map F, average pooling and maximum pooling are performed in the channel dimension to generate two single-channel feature maps: average feature map And the maximum feature map Then, the two single-channel feature maps are combined to generate the weight map H and the feature map F is weighted to generate the feature map S;

[0035] The spatial attention module is calculated as follows:

[0036]

[0037] Where, σ represents the ReLu function; Represents the dot product of corresponding pixels in two feature maps.

[0038] Preferably, the feature fusion function is as follows:

[0039] After adding the feature map R to the feature map S, the ReLu activation function is used to obtain the weight map G; the obtained weight map integrates the weight distribution of the channel dimension and the weight distribution of the spatial dimension, thereby obtaining more complementary features and focusing on characteristic areas such as damage and cracks.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. The present invention designs a detection algorithm for defective areas of photovoltaic panels through an improved Unet segmentation model. According to the characteristics of drone image acquisition and the requirements for detecting tiny areas, the network structure of the original Unet model is improved, a dual-attention feature fusion module is introduced, and appropriate network parameters such as activation functions are set. This realizes feature extraction of tiny defects such as cracks, suppresses background interference, improves the robustness and accuracy of the network, and can effectively solve the problem of photovoltaic panel inspection, providing a guarantee for the stable operation of photovoltaic substations.

[0042] 2. The present invention introduces a parallel dual attention mechanism. In order to accurately distinguish tiny areas under complex backgrounds, a parallel dual attention feature fusion module is designed and applied to the cross-layer connection of the same scale between upsampling and downsampling, so that the network pays more attention to the detection of tiny damaged and cracked areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic diagram of the overall process of a photovoltaic panel inspection method based on deep learning proposed by the present invention;

[0044] Figure 2 This is a schematic diagram of the overall network structure of a photovoltaic panel inspection method based on deep learning proposed by the present invention;

[0045] Figure 3 This is a schematic diagram of the network structure of a dual attention mechanism for a photovoltaic panel inspection method based on deep learning proposed in the present invention. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0047] Reference Figure 1-3 , a photovoltaic panel inspection method based on deep learning, the steps are as follows:

[0048] Step 1: Prepare network input data. Use drones to carry imaging equipment to collect image datasets of photovoltaic panels, including different damage types: breakage, cracks, and covering. All images are resized to eight-bit 256 pixels × 256 pixels. Then use Labelme software to annotate the collected images and output the annotated dataset and the corresponding true value map.

[0049] Step 2: Enhance the dataset by rotation, mirroring, scaling, and brightness adjustment to enrich the type and quantity of the dataset, and divide the enhanced dataset into training set, test set, and validation set in a ratio of 6:3:1;

[0050] Step 3: Construct a segmentation network U-Net optimized by the parallel attention mechanism; the segmentation network U-Net optimized by the parallel attention mechanism mainly includes two core parts: upsampling and downsampling. The upsampling part uses the VGG16 network for feature extraction. The first layer of the network first uses two groups of 3×3 convolutions, batch normalization layers, and ReLu function layers to form a standard convolution module. The second layer uses the maximum pooling for downsampling and repeats the above standard convolution. The third to fifth layers of the network repeat the three groups of the above convolution operations to extract deep semantic features.

[0051] When downsampling, the size of the feature map is halved and the number of channels is doubled for each additional layer, while the number of channels in the fifth layer remains unchanged. During upsampling, the fifth layer network is Dropout with a probability of 0.5 to prevent overfitting of the network. The model uses bilinear interpolation for upsampling, and then sends it to the parallel attention mechanism module with the feature map of the same scale from the fourth layer in the downsampling stage to enhance deep feature extraction. After obtaining the feature map of the ninth layer, a 1×1 convolution combined with a Sigmoid operation is used to output the segmentation result map of the network.

[0052] Step 4: Construct a parallel attention module. Since there is a lot of background interference in the photovoltaic panel detection task, this paper adopts a parallel dual attention mechanism to optimize the U-Net network for feature extraction. The parallel dual attention module consists of the following three parts: channel attention module, spatial attention module, and feature fusion.

[0053] The channel attention module functions as follows:

[0054] First, define the feature map of the input module as F, and then get a 1×1×C global feature F1 through the global maximum pooling layer; then use a one-dimensional convolution of size k to convolve the global feature F1 to obtain an attention-enhanced weight map G; the weight μ of the i-th channel in the attention weight map G is i It can be obtained by the following formula:

[0055]

[0056] Where: represents the jth weight used to learn the i-th channel weight, 1≤j≤k; represents the set of global features of the kth adjacent channel in the i-th channel;

[0057] The convolution kernel size k of one-dimensional convolution is calculated by the following formula:

[0058]

[0059] Where: C is the number of channels; γ and b represent hyperparameters, usually γ = 2, b = 1; |A| odd Represents the odd number closest to A; the feature map F is weighted by the weight map G to obtain the feature map R; the feature map R increases the channel weights related to the damaged area to be inspected, thereby reducing the weights of other channels.

[0060] The spatial attention module functions as follows:

[0061] In the feature map F, average pooling and maximum pooling are performed in the channel dimension to generate two single-channel feature maps: average feature map And the maximum feature map Then, the two single-channel feature maps are combined to generate the weight map H and the feature map F is weighted to generate the feature map S;

[0062] The spatial attention module is calculated as follows:

[0063]

[0064] Where, σ represents the ReLu function; Represents the dot product of corresponding pixels in two feature maps.

[0065] The feature fusion function is as follows:

[0066] After adding the feature map R to the feature map S, the ReLu activation function is used to obtain the weight map G; the final weight map integrates the weight distribution of the channel dimension and the weight distribution of the spatial dimension, thereby obtaining more complementary features, which can suppress the interference caused by various backgrounds and focus on characteristic areas such as damage and cracks.

[0067] Step 5: Use the built model to train on the data set constructed in step 1; for the input image data X, the feature map is obtained after encoding The calculation formula is as follows:

[0068]

[0069] In the formula: Conv represents the convolution operation; function C represents the feature aggregation mechanism: convolution layer + normalization; function Down represents downsampling; L represents the number of network layers; then the obtained feature map is decoded, and the feature map obtained after decoding is The calculation formula is as follows:

[0070]

[0071] Where: [] represents the concatenation of dimensions; Up represents the upsampling of the function; Bp represents the parallel attention mechanism operation; Sigmoid is performed on the final feature map to obtain the final prediction result:

[0072]

[0073] In terms of loss function, the present invention selects the binary cross entropy loss function, which is defined as follows:

[0074]

[0075] Step 6: Use the trained network to test the images taken by the drone and fine-tune the network.

[0076] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A photovoltaic panel inspection method based on deep learning, characterized in that: The steps are as follows: Step 1: Prepare network input data and use drones to carry imaging equipment to collect image datasets of photovoltaic panels; Step 2: Enhance and expand the training set and test set by flipping, rotating, mirroring, and adjusting brightness to increase data diversity; Step 3: Build a parallel attention mechanism optimized segmentation network U-Net; Step 4: Build a parallel attention module; Step 5: Use the built model to train on the dataset constructed in step 1; Step 6: Use the trained network to test the images taken by the drone and fine-tune the network; Construct a parallel attention module. Since there is a lot of background interference in the photovoltaic panel detection task, a parallel dual attention mechanism is used to optimize the U-Net network for feature extraction. The parallel dual attention module consists of the following three parts: channel attention module, spatial attention module, and feature fusion. Use the built model to train on the data set constructed in step 1; for the input image data X, the feature map is obtained after encoding The calculation formula is as follows: In the formula: Conv represents the convolution operation; function C represents the feature aggregation mechanism: convolution layer + normalization; function Down represents downsampling; L represents the number of network layers; then the obtained feature map is decoded, and the feature map obtained after decoding is The calculation formula is as follows: Where: [] represents the concatenation of dimensions; Up represents the upsampling of the function; Bp represents the parallel attention mechanism operation; Sigmoid is performed on the final feature map to obtain the final prediction result: The loss function is defined by the binary cross entropy loss function, which is defined as follows: The channel attention module functions as follows: First, define the feature map of the input module as F, and then get a 1×1×C global feature F1 after passing the global maximum pooling layer; then use a one-dimensional convolution of size k to convolve the global feature F1 to obtain the attention weight map G; the weight μ of the i-th channel in the attention weight map G is i It can be obtained by the following formula: Where: represents the jth weight used to learn the i-th channel weight, 1≤j≤k; represents the set of global features of the kth adjacent channel in the i-th channel; The convolution kernel size k of one-dimensional convolution is calculated by the following formula: Where: C is the number of channels; γ and b represent hyperparameters, usually γ = 2, b = 1; |A| odd represents the odd number closest to A; the feature map F is weighted by the weight map G to obtain the feature map R; in the feature map R, the weight of the channel related to the damaged area to be inspected is increased, thereby reducing the weight of other channels; The spatial attention module functions as follows: In the feature map F, average pooling and maximum pooling are performed in the channel dimension to generate two single-channel feature maps: average feature map And the maximum feature map Then, the two single-channel feature maps are combined to generate the weight map H and the feature map F is weighted to generate the feature map S; The spatial attention module is calculated as follows: Where, σ represents the ReLu function; Represents the dot product of corresponding pixels in two feature maps.

2. A photovoltaic panel inspection method based on deep learning according to claim 1, characterized in that: Prepare network input data, use drones to carry imaging equipment to collect image datasets of photovoltaic panels, including different damage types: breakage, cracks, and covering. All images are resized to eight-bit 256 pixels × 256 pixels. Then, use Labelme software to annotate the collected images with true values ​​and output the annotated dataset and the corresponding true value map.

3. The photovoltaic panel inspection method based on deep learning according to claim 1 is characterized in that: The dataset is enhanced by rotation, mirroring, scaling, and brightness adjustment to enrich the type and quantity of the dataset. The enhanced dataset is divided into training set, test set, and validation set in a ratio of 6:3:

1.

4. The photovoltaic panel inspection method based on deep learning according to claim 1 is characterized in that: Construct a segmentation network U-Net optimized by the parallel attention mechanism. The segmentation network U-Net optimized by the parallel attention mechanism mainly includes two core parts: upsampling and downsampling. The upsampling part uses the VGG16 network for feature extraction. The first layer of the network first uses two groups of 3×3 convolutions, batch normalization layers, and ReLu function layers to form a standard convolution module. The second layer uses maximum pooling for downsampling and repeats the above standard convolution. The third to fifth layers of the network repeat three groups of the above convolution operations to extract deep semantic features. When downsampling, the size of the feature map is halved and the number of channels is doubled for each additional layer, while the number of channels in the fifth layer remains unchanged. During the upsampling process, the fifth layer of the network is Dropout with a probability of 0.5 to prevent overfitting of the network; bilinear interpolation is used for upsampling in the model, and then it is sent to the parallel attention mechanism module together with the feature map of the same scale from the fourth layer of the downsampling stage to enhance the deep feature extraction; after obtaining the feature map of the ninth layer, a 1×1 convolution combined with a Sigmoid operation is used to output the segmentation result map of the network.

5. The photovoltaic panel inspection method based on deep learning according to claim 1 is characterized in that: The feature fusion function is as follows: After adding the feature map R to the feature map S, the ReLu activation function is used to obtain the weight map G; the obtained weight map integrates the weight distribution of the channel dimension and the weight distribution of the spatial dimension, thereby obtaining more complementary features and focusing on the damage and crack feature areas.

Images