Matrix photovoltaic panel fault detection positioning method and system based on deep learning bidirectional three-channel visual state space

Abstract

The invention discloses a matrix photovoltaic panel fault detection positioning method and system based on a deep learning bidirectional three-channel visual state space, which are suitable for fault detection of a photovoltaic panel in photovoltaic power generation equipment. The method mainly comprises the following steps: preprocessing data, inputting a photovoltaic power station infrared image shot by an unmanned aerial vehicle into a Mask Rcnn model, segmenting a single photovoltaic module and obtaining a corresponding mask; a photovoltaic power station infrared image shot by an unmanned aerial vehicle is input into the bidirectional three-channel visual state space model, string fault features are extracted, and string fault types and pixel coordinates are obtained; inputting the outputs of the first two modules into a geographic coordinate calculation module to obtain longitude and latitude information of the photovoltaic fault string; and finally, the position information is issued, and the maintenance task can be activated.

Description

Technical Field

[0001] A matrix photovoltaic panel fault detection and positioning method and system based on deep learning bidirectional three-channel visual state space is suitable for fault detection of photovoltaic panels in photovoltaic power generation equipment. Background Art

[0002] my country's photovoltaic industry is developing very rapidly. Whether from the perspective of development policies or development trends, the future development of my country's photovoltaic power generation industry is worthy of attention. In order to allow photovoltaic panels to have more sunshine time and reduce the occupation of arable land, large-scale photovoltaic power stations are generally built in areas with harsh environments and difficult conditions. This makes photovoltaic panels exposed to the natural environment for a long time and are easily blocked by floating dust, bird droppings, leaves, etc. If the operation and maintenance are not timely, the photovoltaic panels will suffer from hot spot failure. The impact of hot spot failure is very serious. On the one hand, it will reduce the power generation efficiency and make the photovoltaic power station unable to generate enough electricity; on the other hand, serious hot spot failure may cause fire and cause property loss.

[0003] The common method of hot spot detection is for workers to use small thermal imagers to detect photovoltaic arrays. This manual method will cause great problems. First, due to the geographical location of photovoltaic power stations, manual inspection is time-consuming and laborious. Second, due to the difficult terrain of photovoltaic power stations, manual inspections may be more dangerous, and manual methods may have accuracy problems in data transmission and other aspects.

[0004] In order to solve the many problems existing in manual inspections, drones have begun to be used to inspect photovoltaic power stations. Moreover, with the rapid development of deep learning technology, the accuracy and practicality of target detection have been continuously improved, and the use of deep learning for hot spot detection has become possible.

[0005] Therefore, this study used drones to collect infrared images of photovoltaic panels, and then used the hot spot detection model of this study to realize real-time detection of hot spots on photovoltaic panels, greatly improving the detection efficiency, solving the problems of low accuracy and high risk of manual inspections, improving the intelligence level of photovoltaic power stations, and improving the reliability of photovoltaic power generation systems, which is of great significance to the intelligent operation and maintenance of photovoltaic power stations. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a matrix photovoltaic panel fault detection and positioning method and system based on deep learning bidirectional three-channel visual state space, which can accurately detect the number of faults, accurately classify fault categories and accurately identify the geographical location of faults, reduce labor costs, extend equipment service life and improve power generation efficiency.

[0007] In order to achieve the above object, the technical solution of the present invention is as follows: comprising the following steps:

[0008] A. Data Preprocessing

[0009] Use DJI Jingwei UAV to collect infrared images of photovoltaic power stations, set the shooting height, and filter the pictures to remove meaningless pictures without photovoltaic panels. Use OpenCV to rotate the image at any angle to simulate the change of the drone shooting angle and shooting route; secondly, use pictures with different pixels to simulate the noise interference caused by shooting at different heights; finally, change the contrast to simulate the data feature effects under different weather conditions.

[0010] B. Model Training

[0011] B1. Train MaskRCNN based on the infrared view of photovoltaic panels to obtain a trained MaskRCNN model, wherein the MaskRCNN model includes a ResNet-FPN network layer, an RPN network layer, an ROI Align pooling layer and a fully connected layer, wherein: the ResNet-FPN network layer extracts features from the photovoltaic panel image to obtain a feature image; the RPN network layer divides and filters the feature image into target areas to obtain a filtered target area; the ROI Align pooling layer maps the filtered target area, the infrared image of the photovoltaic panel and the feature image to obtain a fixed-size target area feature map; the fully connected layer classifies and regresses the fixed-size target area feature map to obtain a classification result and a target bounding box, thereby completing the recognition of the number of photovoltaic panels in the target image. Furthermore, the RPN network layer of the MaskRCNN model adopts the NMS non-maximum suppression algorithm, which is expressed as follows: N M :=max(N t ,d M ), Among them, N = {N1, N2, ..., Nm} represents the initial detection box, G = {g1, g2, ..., gn} represents the corresponding detection score, N M represents the NMS threshold, g i' Represents the detection box with the highest detection score. Through this step, the detection box far away from the highest detection score will not be affected, while the detection box very close will be assigned a larger penalty, that is, when the detection box graph overlaps in the detection of photovoltaic panels, it will not cause false filtering. Further, the RPN network layer of the MaskRCNN model includes a set of 64×64 anchor boxes and 10 anchor points. Through this step, the network can detect more small targets and identify incomplete photovoltaic panels in the target image. Furthermore, the fully connected layer of the MaskRCNN model uses a classifier based on cosine Softmax loss, and its loss function formula is expressed as follows: Among them, K represents the number of categories, N represents the number of training samples, and y ic Indicates the unique hot encoding of the sample target value, true is 1, otherwise it is 0, h θ (x i )c represents the observed sample x i The predicted probability of belonging to class c. Through this step, separable features can be learned, intra-class variance can be reduced, and the detection accuracy of new classes can be improved.

[0012] B2. A matrix photovoltaic panel fault detection model based on a bidirectional three-channel visual state space in deep learning is trained to obtain a converged model. The bidirectional three-channel visual state space matrix photovoltaic panel fault detection model based on deep learning includes a head data generator, a backbone network, and a classification head, wherein: the head data generator is used to expand the original infrared image into a one-dimensional vector. In the present invention, the original two-dimensional image is expanded into a 1×36 vector, wherein 36 represents that a standard photovoltaic module has 3 strings, each string has 2 columns of battery cells, and 6 is taken as the number of times the original image is cut, so 36 small samples are obtained, which are flattened into a one-dimensional vector; the backbone network includes a bidirectional three-channel state space network, the network includes a U-shaped scanning layer and a naive state space layer, and the three channels represent the three string feature vectors used to process the standard photovoltaic module respectively. After bidirectional processing, the network can remain sensitive to the features of the long-distance sequence before and after, ensuring that information is not lost; the classification head is a multi-layer perception machine layer, and its state space model is expressed as follows: h′(t)=Ah(t)+Bx(t) ① y(t)=Ch(t)+Dx(t) ② Formula ① is the state transformation function, and formula ② is the output function. In the state space model, as a continuous time-invariant system, it is discretized through a zero-order holder, and we can get: Where A∈C N×N ,B,C∈C N , B=(e ΔA-I )A -1 B≈(ΔA)(ΔA) -1 ΔB=ΔB.

[0013] B3. A matrix photovoltaic panel fault detection and positioning method and system based on a bidirectional three-channel visual state space in deep learning is obtained, and a positioning method suitable for photovoltaic detection is obtained. The positioning method includes Euclidean distance clustering and an image geographic coordinate calculation algorithm. The Euclidean distance clustering method is expressed as follows: Where x represents the detected fault data point, x={x1,x2,...,x j} represents the two-dimensional coordinates of the center point pixels of the j faults on the photovoltaic panel, c ij Represents the two-dimensional coordinates of the center pixel of the photovoltaic panel with a fault. This formula can be used to calculate the Euclidean distance dist of the detected fault on the photovoltaic panel. Di represents the c i By comparing the Euclidean distances to the centers of all photovoltaic panels, we can classify which photovoltaic panel the fault belongs to.

[0014] The image geographic coordinate calculation algorithm calculates the size of each pixel in the actual geographic location through the longitude and latitude coordinates of the drone aerial photography center according to camera parameters, deflection angle, and true north direction angle to obtain accurate longitude and latitude information. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of matrix photovoltaic panel fault detection and positioning method and system steps based on deep learning bidirectional three-channel visual state space

[0016] Figure 2 Schematic diagram of the MaskRcnn model framework of a matrix photovoltaic panel fault detection and positioning method and system based on deep learning bidirectional three-channel visual state space

[0017] Figure 3 Schematic diagram of the bidirectional three-channel state space model framework of the matrix photovoltaic panel fault detection and positioning method and system based on deep learning bidirectional three-channel visual state space

[0018] Figure 4 Schematic diagram of bidirectional three-channel U-shaped scanning operation of matrix photovoltaic panel fault detection and positioning method and system based on deep learning bidirectional three-channel visual state space

[0019] Figure 5 Photovoltaic fault classification diagram of matrix photovoltaic panel fault detection and positioning method based on deep learning bidirectional three-channel visual state space and system DETAILED DESCRIPTION

[0015] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only provided for the convenience of explanation and description, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0016] Referring to the schematic diagram of the matrix photovoltaic panel fault detection and positioning method and system steps based on deep learning bidirectional three-channel visual state space, the present invention provides a matrix photovoltaic panel fault detection and positioning method and system based on bidirectional three-channel visual state space in deep learning, the method comprising the following steps:

[0017] S1. Collect data from operating photovoltaic power stations to obtain infrared imaging data sets of photovoltaic panels. Specifically, use DJI drones, storage devices, cameras and other equipment to collect multiple batches of data from the photovoltaic panel area of ​​the distributed power station to obtain infrared images.

[0018] S2. Train the MaskRCNN model based on the infrared image of the photovoltaic panel to obtain the trained MaskRCNN model; specifically, referring to the MaskRCNN model framework diagram of the matrix photovoltaic panel fault detection and positioning method and system based on deep learning bidirectional three-channel visual state space, the MaskRCNN model includes a ResNet-FPN network layer, an RPN network layer, an ROIAlign pooling layer and a fully connected layer.

[0019] S2.1. Use the network parameters trained on the COCO dataset to initialize the parameters of the MaskRCNN network.

[0020] S2.2. Perform data enhancement and other preprocessing on the acquired infrared images of photovoltaic panels, and adjust the size of the images used for training to 1024×1024.

[0021] S2.3. Input the preprocessed infrared image into the ResNet-FPN network layer of the MaskRCNN model. The ResNet-FPN network layer uses the ResNet101-FPN network. The ResNet101-FPN network extracts the features of the photovoltaic panel under infrared imaging to obtain the feature image. After segmentation, the 114×114 single-component image data required for training the detection model can be obtained.

[0022] S2.4. The RPN network layer in the specific embodiment of the present invention adds a set of 64×64 anchor frames to the traditional RPN network layer structure so that the network can detect more small targets; the RPN network layer in the specific embodiment uses 10 anchor points, and each sliding window generates 10 candidate regions of different scales and different aspect ratios, with scale sizes of 64×64, 128×128, 256×256, and 512×512 pixels, and three aspect ratios of 1:1, 1:2, and 2:1. The feature image is input into the RPN network layer, and each pixel of the feature image is generated using the anchor point to generate 10 corresponding candidate region frames. The RPN network layer uses the NMS non-maximum suppression algorithm, and its formula is expressed as follows: N M:=max(N t ,d M ), Among them, N = {N1, N2, ..., Nm} represents the initial detection box, G = {g1, g2, ..., gn} represents the corresponding detection score, N M represents the NMS threshold, g i' Represents the detection box with the highest detection score.

[0016] S2.5. Input the filtered target area into the ROIAlign pooling layer, and simultaneously obtain the mapping relationship between the photovoltaic panel infrared image and the feature image, and the mapping relationship between the feature image and the filtered target area. Through the mapping relationship, a fixed-size target area feature map is obtained.

[0017] S2.6. Input the fixed-size target region feature map into the fully connected layer, classify and regress the fixed-size target region feature map, obtain the classification result and target bounding box, and use the canny edge detection algorithm. The formula expression is as follows: d x (x,y)=f(x,y)S x , d y (x,y)=f(x,y)S y Where: S x , S y is the Sobel operator, f(x,y) represents the image pixel value, and the expressions of the magnitude and phase angle of the image gradient can be obtained as follows:

[0018] S2.7. Calculate the loss function of the MaskRCNN model and adjust the weight parameters of the MaskRCNN model.

[0019] S2.8. Repeat the above steps S2.1 to S2.7 until the preset number of iterations is reached, stop training, save the MaskRCNN model, and obtain the trained MaskRCNN model.

[0020] S3. Train the matrix photovoltaic panel fault detection model based on the bidirectional three-channel visual state space in deep learning to obtain a converged model.

[0021] S3.1 Referring to the bidirectional three-channel state space model framework diagram of the matrix photovoltaic panel fault detection and positioning method and system based on deep learning bidirectional three-channel visual state space, the bidirectional three-channel visual state space matrix photovoltaic panel fault detection model based on deep learning includes a head data generator, a backbone network, and a classification head, wherein: the head data generator is used to expand the original infrared image into a one-dimensional vector. In the present invention, the original two-dimensional image is expanded into a 1×36 vector, wherein 36 represents that a standard photovoltaic module has 3 strings, each string has 2 columns of battery cells, and 6 is taken as the number of times the original image is cut, so 36 small samples are obtained, which are flattened into a one-dimensional vector; the backbone network includes a bidirectional three-channel state space network, the network includes a U-shaped scanning layer, a naive state space layer, and the three channels are respectively used to process the three string feature vectors under the standard photovoltaic module. After front and back bidirectional processing, the network can remain sensitive to the features of the front and back long-distance sequences to ensure that information is not lost; the classification head is a multi-layer perception machine layer, which is expressed as follows: h′(t)=Ah(t)+Bx(t) ① y(t)=Ch(t)+Dx(t) ② Formula ① is the state transformation function, and formula ② is the output function. In the bidirectional state space model, as a continuous time-invariant system, it is discretized by a zero-order holder, and we can get: Where A∈C N×N ,B,C∈C N , B=(e ΔA-I )A -1 B≈(ΔA)(ΔA) -1 ΔB=ΔB. In formula ①, h′(t) represents the hidden layer state. By discretizing the state space model, h k , which enables the entire model to have a richer understanding of the contextual information of photovoltaic infrared images.

[0022] S3.2. Build a bidirectional three-channel visual state space matrix photovoltaic panel fault detection model based on deep learning, train the model and initialize the parameters.

[0023] S3.3 performs preprocessing such as data enhancement on the acquired infrared images of photovoltaic panels.

[0024] S3.4 inputs the preprocessed infrared image into the deep learning based bidirectional three-channel visual state space matrix photovoltaic panel fault detection model.

[0025] S3.5 In the head network of the bidirectional three-channel visual state space matrix photovoltaic panel fault detection model, the original data is divided into 1×36 one-dimensional vectors. The one-dimensional vectors are linearly mapped and spatially projected to generate samples and category label fusion. The completed sequence is input into the decoder network for three-channel front and back convolution layers, and then U-shaped scanning features are performed. The generated feature map is enhanced by the features of the naive state space matrix and recombined into a feature map. Finally, the features are fused into the detection head and the category is output. In the present invention, photovoltaic fault categories are divided into three categories, referring to the photovoltaic fault classification diagram of the matrix photovoltaic panel fault detection and positioning method and system based on deep learning bidirectional three-channel visual state space, which are group string open circuit, group string short circuit, and ordinary hot spot fault.

[0026] S3.6 calculates the loss function of the model based on the classification results and the target bounding box, and adjusts the weight parameters of the model.

[0027] S3.7 Repeat the above steps S2.2 to S2.6 until the preset number of iterations is reached, stop training, and save the bidirectional three-channel visual state space matrix photovoltaic panel fault detection model.

[0028] S4 is a bidirectional three-channel visual state space matrix photovoltaic panel fault detection and positioning method and system based on deep learning, which obtains a positioning method suitable for photovoltaic detection, specifically including a Euclidean distance clustering algorithm and an image geographic coordinate calculation algorithm.

[0029] The Euclidean distance clustering algorithm described in S4.1 is as follows. According to the application scenario of the present invention, the mask inferred by the MaskRCNN model is the pixel coordinates of the photovoltaic panel, and the two-way three-channel visual state space matrix photovoltaic panel fault detection model infers the block diagram pixel coordinates of the fault on the unknown photovoltaic panel. First, the center pixel coordinates of the segmented independent photovoltaic panel are calculated according to the mask inferred by the MaskRCNN model, and the block diagram center coordinates of the fault on the unknown photovoltaic panel inferred by the two-way three-channel visual state space matrix photovoltaic panel fault detection model are used as the board code and the fault code respectively; secondly, the Euclidean distance between the fault point and the photovoltaic panel on each picture is calculated, and it is judged that the code of the fault to which the minimum Euclidean distance belongs and the code of the photovoltaic panel are a pair, that is, the fault belongs to the photovoltaic panel. The expression of the Euclidean distance clustering method is as follows: Where x represents the detected fault data point, x={x1,x2,...,x j} represents the two-dimensional coordinates of the center point pixels of the j faults on the photovoltaic panel, c ijRepresents the two-dimensional coordinates of the center pixel of the photovoltaic panel with a fault. This formula can be used to calculate the Euclidean distance dist of the detected fault on the photovoltaic panel. Di represents the c i Euclidean distance to the center of all photovoltaic panels

[0030] The image geographic coordinate calculation algorithm described in S4.2 is to calculate the size of each pixel in the actual geographic location based on the latitude and longitude coordinates of the drone aerial photography center, camera parameters, deflection angle, and true north direction angle to obtain accurate latitude and longitude information.

[0031] S5 data is uploaded to the server. The server indexes the photovoltaic panels that need to be repaired on site based on the returned longitude and latitude information and assigns tasks.

Claims

1. A bidirectional three-channel visual state space matrix photovoltaic panel fault detection and positioning method based on deep learning, characterized in that: The following steps are involved: A. Collect data from photovoltaic panels of a 95MW agricultural photovoltaic power station in Guangdong Province and construct an infrared image dataset; B. Train the MaskRCNN model based on the data set images to obtain the trained MaskRCNN model; C. Train the bidirectional three-channel visual state space matrix model based on deep learning to obtain the trained model; D. Adjust the parameters of the positioning algorithm and train the data inferred after the model converges to obtain a trained positioning model; E. Obtain the test photovoltaic panel infrared data set and input it into the trained deep learning-based bidirectional three-channel visual state space matrix model and positioning algorithm to obtain the number and location information of the faulty photovoltaic panels; F. Use cloud servers to assign staff to the fault area to perform maintenance work.

2. According to the deep learning-based bidirectional three-channel visual state space matrix photovoltaic panel fault detection and positioning method according to claim 1, it is characterized in that: The MaskRCNN model includes a ResNet-FPN network layer, an RPN network layer, a ROI Align pooling layer and a fully connected layer, wherein: the ResNet-FPN network layer extracts features from a photovoltaic panel image to obtain a feature image; the RPN network layer divides and filters the feature image into target areas to obtain a filtered target area; the ROIAlign pooling layer maps the filtered target area, the faulty photovoltaic component and the feature image to obtain a fixed-size target area feature map; the fully connected layer classifies and regresses the fixed-size target area feature map to obtain a classification result and a target boundary box.

3. According to claim 2, the bidirectional three-channel visual state space matrix photovoltaic panel fault detection and positioning method based on deep learning is characterized in that: The MaskRCNN model uses the canny edge detection algorithm in photovoltaic module detection. Under infrared imaging, the grayscale value of the photovoltaic module is quite different from the background. The image is binarized and the front and back are separated. The formula is as follows: d x (x,y)=f(x,y)S x ,d y (x,y)=f(x,y)S y Where: S x , S y is the Sobel operator, f(x,y) represents the image pixel value, and the expressions of the magnitude and phase angle of the image gradient can be obtained as follows:

4. According to claim 2, the bidirectional three-channel visual state space matrix photovoltaic panel fault detection and positioning method based on deep learning is characterized in that: The RPN network layer of the MaskRCNN model includes a set of 64×64 anchor boxes and 12 anchor points.

5. According to claim 2, the bidirectional three-channel visual state space matrix photovoltaic panel fault detection and positioning method based on deep learning is characterized in that: The fully connected layer of the MaskRCNN model uses a classifier based on cosine loss, and its loss function formula is expressed as follows: Among them, y i represents the label of the input vector, L represents the number of training samples, β represents the scaling factor, and θ j Represents the angle between the weight vector and the input vector.

6. According to claim 1, the matrix photovoltaic panel fault detection and positioning method based on the bidirectional three-channel visual state space in deep learning is characterized in that: The bidirectional three-channel visual state space matrix photovoltaic panel fault detection model based on deep learning includes a head data generator, a backbone network, and a classification head, wherein: the head data generator is used to expand the original infrared image into a one-dimensional vector. In the present invention, the original two-dimensional image is expanded into a 1×36 vector, wherein 36 represents that a standard photovoltaic module has 3 strings, each string has 2 columns of battery cells, and 6 is taken as the number of times the original image is cut, so 36 small samples are obtained, which are flattened into a one-dimensional vector; the backbone network includes a bidirectional three-channel state space network, the network includes a U-shaped scanning layer and a naive state space layer, and the three channels are respectively used to process the three string feature vectors under the standard photovoltaic module. After front and back bidirectional processing, the network can remain sensitive to the features of the front and back long-distance sequences to ensure that information is not lost; the classification head is a multi-layer perception machine layer.

7. According to claim 1, the matrix photovoltaic panel fault detection and positioning method based on the bidirectional three-channel visual state space in deep learning is characterized in that: The backbone network layer of the visual state space matrix model (SSM) includes a pure mamba layer and a selective scanning layer (SS2D). The pure state space formula is expressed as follows: h ′ (t)=Ah(t)+Bx(t) ① y(t)=Ch(t)+Dx(t) ② Formula ① is the state transformation function, and formula ② is the output function. In the state space model, as a continuous time-invariant system, it is discretized through a zero-order holder, and we can get: among them A∈C N×N 、B,C∈C N 、B=(e ΔA-I )A -1 B≈(ΔA)(ΔA) -1 ΔB = ΔB.

8. The selective scanning layer (SS2D) performs a U-shaped scan on the photovoltaic strings. It is used three times in the present invention to perform three bidirectional scans in the forward and reverse directions on the standard three-string photovoltaic components. The specific scanning method is to use a 3×3 convolution kernel to perform a deep separable convolution operation on the segmented 112×112 pixel single photovoltaic component image; wherein the bidirectional scanning, the forward convolution layer, and the reverse convolution layer are all one-dimensional, and the bidirectional convolution can be realized by transposing the convolution kernel weight parameters to extract high-order features.

9. A matrix photovoltaic panel fault detection and positioning system based on a bidirectional three-channel visual state space in deep learning constructed according to the method described in claims 1-7, characterized in that: include: A data acquisition subsystem for acquiring infrared data sets of photovoltaic panels in photovoltaic power plants; A data processing subsystem is used to obtain infrared data sets of photovoltaic panels of photovoltaic power stations for training and obtain training results; Feedback maintenance system, The training results of the data processing subsystem are fed back to the cloud server to assign maintenance tasks.

Images