Photovoltaic fault diagnosis and positioning method for unmanned aerial vehicles based on attention neural network
Through the drone photovoltaic fault diagnosis method based on attention neural network, the FPT module and the CSPDarknet53 backbone network are used to solve the problem of insufficient positioning accuracy in photovoltaic power station fault detection, and high-precision and efficient fault detection are achieved, suitable for photovoltaic power stations drone inspection.
Patent Information
- Application Number
- CN202210156492.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-21
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-02-21
AI Technical Summary
The existing photovoltaic power station fault detection methods have problems such as insufficient positioning accuracy, large calculation volume and poor real-time performance, which are difficult to meet the needs of high-frequency unmanned patrols.
Using the drone photovoltaic fault diagnosis and positioning method based on attention neural network, the global information interaction within the feature layer and local information interaction between layers are constructed by constructing the FPT module, and combining the CSPDarknet53 backbone network and semantic segmentation network, the optimal binary matching is designed. Loss function realizes high-precision defect detection and positioning of aerial images.
It improves the positioning accuracy and real-time performance of photovoltaic power station fault detection, reduces the calculation amount, and realizes high-precision real-time monitoring of component defects.
Smart Images

Figure CN114529817B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of intelligent inspection of solar photovoltaic power station faults, and specifically provides a method for diagnosing and locating photovoltaic faults using an unmanned aerial vehicle based on an attention neural network. Background Art
[0002] As society's demand for green and clean energy increases, the photovoltaic power station industry based on solar power generation technology has developed rapidly. Photovoltaic power stations occupy a large area and are mainly distributed in wild natural environments such as deserts, wastelands, and water surfaces. Photovoltaic modules are placed in harsh outdoor environments and are exposed to wind and sun all year round, resulting in serious problems of failures and defects. Real-time control, detection, and daily maintenance of power generation systems often require high labor costs, and there are disadvantages such as strong subjectivity and single inspection methods, which makes it difficult to meet the growing inspection needs. In order to ensure the efficient operation of photovoltaic power stations, unmanned intelligent inspection methods are urgently needed. Intelligent defect detection and positioning technology can realize the diversification of inspection modes, while giving full play to the advantages of high precision, flexible response, and all-weather of robots to meet the requirements of high-frequency unmanned inspections. It is of great significance to improve the power generation efficiency of photovoltaic power stations and ensure the safe and efficient operation of large photovoltaic power stations.
[0003] In recent years, the defect detection of photovoltaic modules based on images mainly includes two categories: traditional signal processing algorithms and artificial intelligence algorithms. Traditional signal processing algorithms include defect location and segmentation algorithms based on anisotropic diffusion filtering, matched filtering, vascular filtering and other methods. They can only target a certain defect and are difficult to handle the various types of defects with obvious appearance differences in the actual working environment, so their application value is not high. Artificial intelligence algorithms usually use deep learning convolutional neural networks for target detection and instance segmentation to achieve defect detection and judgment. By repeating the training of input data enough times to minimize the global loss, the training effect is achieved. Considering the wide variety of defects, the large differences in their manifestations, and the uneven distribution of samples in each category, the reasonable design of the structure of the neural network and the selection of an efficient loss function calculation method have become the key to building a neural network, and are also the main factors that determine the accuracy of network detection and positioning accuracy.
[0004] The acquisition of location information of aerial images mainly includes methods based on photovoltaic string CAD drawings and methods based on the fusion of mobile vehicles and visual location information. The method based on photovoltaic string CAD drawings relies on satellite remote sensing data such as Google Earth, and its positioning accuracy is low, which affects the inspection process of drones. The method based on the fusion of mobile vehicles and visual location information mostly restores high-precision maps through two-dimensional and three-dimensional reconstruction, but does not contain semantic information, and cannot automatically obtain the location distribution of photovoltaic areas. In addition, the recognition accuracy is not high and still requires manual modification, and cannot be fully automated. Summary of the invention
[0005] In order to solve the above technical problems, the present invention proposes a UAV photovoltaic fault diagnosis and positioning method based on attention neural network. This method alleviates the problem of insufficient accuracy of deep learning target detection and positioning, is suitable for photovoltaic power station defect detection based on aerial images, and can realize real-time positioning of defective components with small calculation amount and good real-time performance, thereby improving the efficiency of intelligent photovoltaic power station defect detection and positioning.
[0006] The photovoltaic fault diagnosis and positioning method of UAV based on attention neural network includes the following steps:
[0007] (1) Obtain infrared images of photovoltaic modules taken by inspection drones and read the real-time location information and attitude data of the drones. The flight altitude of the drone is h, and the GPS coordinates are (x D ,y D ), camera field of view angle γ, pitch angle θ ZD ,, heading angle ψ ZD .
[0008] (2) The FPT module is constructed based on the attention mechanism and FPN structure, which includes three transformer structures: self-transformer ST, implanted transformer GT, and rendered transformer RT, which respectively realize the global information interaction within the feature layer and the local information interaction between top-down and bottom-up layers:
[0009] The ST structure performs three different convolution operations on the current feature layer to obtain values, keys, and queries, and divides queries and keys into N equal parts to obtain q i , k j , use point multiplication to calculate the similarity s of each pair i,j Then use MoS normalization to get the weight coefficient w i,j , perform weighted summation on the values to get the feature map:
[0010]
[0011] Among them, F sim is the dot product operation, F mul is the matrix multiplication operation, F MoS The normalization operation is defined as follows, is a trainable parameter for normalization, For all keys i,j The arithmetic mean of :
[0012]
[0013] GT is a top-down feature interaction that implants the semantic information of the deep feature map into the high-resolution information of the shallow feature map to enhance the shallow feature map. Considering the differences in semantic information extracted from feature maps of different sizes, the negative Euclidean distance is used to calculate the correlation. The formula is as follows:
[0014]
[0015] where q i Shallow feature map, k j ,v j From the deep feature map, F eud represents the Euclidean distance, that is, every pair of q i and k j The closer the distance, the greater the weight:
[0016] F eud (q i ,k j )=-||q i --k i || 2 (4)
[0017] RT is a bottom-up feature fusion, which aims to render high semantic information through high-resolution pixel information and realize the rendering of deep feature maps. Q comes from high-level feature maps, and K and V come from low-level feature maps. First, the weight w is obtained by global average pooling of K to weight Q, and then the size of V is reduced by convolution with stride. Finally, K and Q with the same size after processing are added and refined to obtain the output feature map.
[0018]
[0019] Where GAP represents global average pooling, aligning the sizes of K and Q; F att is the outer product function, which realizes the weighting of the feature map Q; F sconv represents a 3×3 convolution with a stride, which is used to reduce the size of the low-level feature map V; F conv Represents a 3×3 convolution, which is used to weight the result Q att Optimize; F add It means adding first and then refining through 3×3 convolution;
[0020] (3) A single-stage target detection BP neural network was constructed, CSPDarknet53 was used as the backbone network for feature extraction, the FPT structure was used for information fusion, the output module used the anchor frame idea for prediction and regression, and the optimal binary matching idea was used to design the loss function; the aerial image size was unified to 416×416×3 as the network input, and the pixel coordinates of the center of the photovoltaic module where the defect was located were used as the output. The constructed neural network was trained to obtain a neural network for infrared image defect detection;
[0021] The output module divides the feature map into S×S grids, each of which is responsible for predicting nine anchor boxes of different scales, that is, learning the relative offset of the anchor box in the grid and the probability of each category. x , t y , t w , t h , p obj , p cls ) six parameters, among which (t x , t y ) represents the offset of the center coordinate of the anchor box relative to the grid point, (t w , t h ) represents the ratio of the width and height of the bounding box to the width and height of the preset bounding box, p obj Represents the probability that the bounding box contains the detected target, p cls Indicates the probability that the detected target belongs to each category. Finally, all prediction results are screened by the non-maximum suppression algorithm based on DIOU improvement, and the obtained target bounding box coordinates and category information are used as network output;
[0022] The loss function uses a bipartite graph matching algorithm. After each generation of training, the optimal bipartite graph matching between the predicted result and the true value is found through the loss function minimization strategy, and the loss function is only calculated for the predicted target box that is successfully matched. The loss function is defined as the weighted sum of the classification loss and the positioning loss:
[0023]
[0024] in Represents classification loss, calculated using Focal loss, Represents the normalized Euclidean distance between the center points of two target boxes, represents the normalized Manhattan distance between the upper left corner and the lower right corner of the two target boxes, λ cls , λ Manhattan is the weighting coefficient;
[0025] During the training phase, a one-to-one matching strategy between the prediction results and the true target frames is used to screen the target frames, and the loss function is calculated only for the successfully matched target frames. This strengthens the network's learning of the distribution of true target frames in the process of learning features, improves the calculation efficiency of the loss function, and can increase the model convergence speed. It can also indirectly reduce the probability of similar target frames being suppressed in the prediction results, thereby improving the detection accuracy.
[0026] (4) Use the semantic segmentation network LEDNet to segment the original image and obtain the photovoltaic module string mask; perform binary segmentation on the photovoltaic module string mask to obtain the photovoltaic module mask, then perform median filtering to eliminate noise points, and perform corrosion and expansion operations to optimize the photovoltaic module segmentation results, and take its minimum circumscribed rectangle as the optimized photovoltaic module mask; finally, determine the faulty photovoltaic module based on the positioning results of the target detection network, and obtain the pixel coordinates of the corner points of the target module;
[0027] (5) Using the target positioning method based on the UAV POS data, the camera attitude angle, field of view angle, flight altitude, GPS coordinates and other information of the UAV during image capture are obtained through the airborne GPS / INS system, and the GPS coordinates of the target pixel point are calculated according to the aerial triangulation geometry relationship;
[0028] The target detection module obtains the coordinates of the four corner points (x1, y1, x2, y2) and the defect category of the defective component, and calculates the pixel coordinates (x, y) of the center point of the photovoltaic component, that is, the coordinates of the target pixel point, according to the following formula:
[0029]
[0030] The flight altitude of the drone is h, and the GPS coordinates are (x D ,y D ), camera field of view angle γ, pitch angle θ ZD , heading angle ψ ZD , the camera's field of view is (y f0 ,y f1 ), (x f0 , x f1 ), the GPS coordinates of the target pixel are (X, Y), and the size of the infrared image is (W, H);
[0031] First, calculate the four corner points of the camera's field of view, that is, the GPS coordinates of the four vertices of the image. According to the aerial triangulation relationship, we have:
[0032]
[0033] According to the similarity relationship between the infrared image pixel coordinates and the actual GPS coordinates, the GPS coordinates of the center point of the target photovoltaic module are calculated by formula (9);
[0034]
[0035] As a further improvement of the present invention, the BP neural network constructed in step (3) adopts the idea of optimal binary matching to design the loss function. After each generation of training, the detection result is optimally binary matched with the true value, and the loss function is only calculated for the predicted target box that is successfully matched. The loss function includes three parts: classification loss, center point loss and corner point loss. Compared with IOUloss, this design guides the positioning of the target box from the center point positioning and corner point positioning, which can improve the model convergence rate.
[0036] As a further improvement of the present invention, CSPDarknet53 as the backbone network adopts Mish as the neural network activation function, which slightly allows better gradient flow for negative values, rather than a hard zero boundary like in the ReLU function:
[0037] Mish(x) = x·tanh(ln(1+e x )) (10)
[0038] As a further improvement of the present invention, the steps of training the constructed BP neural network are as follows:
[0039] (1) Gaussian noise is added to the dataset and operations such as contrast, brightness, and sharpness are adjusted to alleviate the category imbalance problem and achieve expansion; 65% of the expanded dataset is randomly selected as the training dataset, 15% of the images constitute the validation dataset, and the remaining 20% constitute the test dataset;
[0040] (2) The neck part of the BP network is randomly initialized, and the backbone part uses the pre-trained weights on the COCO dataset for transfer learning; in order to prevent the weights of the feature extraction network from being destroyed in the early stage of training, the backbone network parameters in the first 25 generations of training are frozen and do not participate in gradient updates;
[0041] (3) According to the error back propagation algorithm, the Adam optimizer and small batch stochastic gradient descent method are used. The learning rate descent curve adopts the StepLR fixed step size attenuation strategy, and gamma is 0.9. The weights of the backbone and neck structures are fine-tuned and updated respectively.
[0042] Beneficial effects:
[0043] The present invention discloses a method for diagnosing and locating photovoltaic faults of unmanned aerial vehicles based on an attention neural network. The method uses an inspection unmanned aerial vehicle to obtain infrared images of photovoltaic components and real-time position information and attitude data of the unmanned aerial vehicle, uses CSPDarknet53 as a backbone network for feature extraction, adopts an FPT structure based on an attention mechanism and an FPN structure for information fusion, takes an aerial image as input, and outputs the pixel coordinates of the photovoltaic component where the defect is located; the original image is segmented to obtain a photovoltaic component mask, and then the faulty photovoltaic component is determined according to the positioning result of the target detection network to obtain the pixel coordinates of the corner point of the target component; a coordinate conversion model is established according to the real-time coordinates and attitude angles of the unmanned aerial vehicle shooting, and the pixel coordinates output by the neural network are converted into position coordinates under geodetic coordinates according to the geometric relationship of aerial triangulation to obtain the position information of the photovoltaic component where the defect is located. The method alleviates the problem of low positioning accuracy of the target detection algorithm based on deep learning, reduces the calculation amount of aerial image positioning, greatly improves the real-time performance of defect positioning, and can realize high-precision real-time monitoring of component defects. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 is a flow chart of the method disclosed in the present invention;
[0045] Figure 2 Schematic diagram of the flight parameters and camera field of view during drone inspection. DETAILED DESCRIPTION
[0046] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:
[0047] The present invention discloses a method for diagnosing and locating photovoltaic faults of unmanned aerial vehicles based on an attention neural network, comprising the following steps:
[0048] Step 1: Obtain infrared images of photovoltaic modules taken by the inspection drone, and read the real-time location information and attitude data of the drone. The flight altitude of the drone is h, and the GPS coordinates are (x D ,y D ), camera field of view angle γ, pitch angle θ ZD , heading angle ψ ZD .
[0049] Step 2: Construct the FPT module based on the attention mechanism and FPN structure, which includes three structures: self-transformer (ST), implanted transformer (GT) and rendering transformer (RT), which respectively realize the global information interaction within the feature layer and the local information interaction between top-down and bottom-up layers:
[0050] The ST structure performs three different convolution operations on the current feature layer to obtain values, keys, and queries. Divide the queries and keys into N equal parts to obtain qi , k j , use point multiplication to calculate the similarity s of each pair i,j Then use MoS normalization to get the weight coefficient w i,j , perform weighted summation on the values to get the feature map:
[0051]
[0052] Among them, F sim is the dot product operation, F mul is the matrix multiplication operation, F MoS The normalization operation is defined as follows, is a trainable parameter for normalization, For all keys i,j The arithmetic mean of :
[0053]
[0054] GT is a top-down feature interaction that embeds the semantic information of deep feature maps into the high-resolution information of shallow feature maps to enhance shallow feature maps. Considering the differences in semantic information extracted from feature maps of different sizes, negative Euclidean distance is used to calculate the correlation, and the formula is as follows:
[0055]
[0056] where q i Shallow feature map, k j , v j From the deep feature map, F eud represents the Euclidean distance, that is, every pair of q i and k j The closer the distance, the greater the weight:
[0057] F eud (q i , k j )=-||q i -k j || 2 (4)
[0058] RT is a bottom-up feature fusion, which aims to render high semantic information through high-resolution pixel information and realize the rendering of deep feature maps. Q comes from high-level feature maps, and K and V come from low-level feature maps. First, the weight w is obtained by global average pooling of K to weight Q, and then the size of V is reduced by convolution with stride. Finally, K and Q of the same size are added and refined to obtain the output feature map.
[0059]
[0060] Where GAP represents global average pooling, aligning the sizes of K and Q; F att is the outer product function, which realizes the weighting of the feature map Q; F sconv represents a 3×3 convolution with a stride, which is used to reduce the size of the low-level feature map V; F conv Represents a 3×3 convolution, which is used to weight the result Q att Optimize; F add It means adding first and then refining through 3×3 convolution.
[0061] Step 3: Construct a single-stage target detection BP neural network, use CSPDarknet53 as the backbone network for feature extraction, use FPT structure for information fusion, use the anchor frame idea for prediction and regression in the output module, and use the optimal binary matching idea to design the loss function; unify the size of the aerial image to 416×416×3 as the network input, and use the pixel coordinates of the center of the photovoltaic module where the defect is located as the output. Train the constructed neural network to obtain a neural network for infrared image defect detection.
[0062] The output module divides the feature map into S×S grids, each of which is responsible for predicting nine anchor boxes of different scales, that is, learning the relative offset of the anchor box in the grid and the probability of each category. x , t y , t w , t h , p obj , p cls ) six parameters, among which (t x , t y ) represents the offset of the center coordinate of the anchor box relative to the grid point, (t w , t n ) represents the ratio of the width and height of the bounding box to the width and height of the preset bounding box, p obj Represents the probability that the bounding box contains the detected target, p cls Indicates the probability that the detected target belongs to each category. Finally, all prediction results are screened by the non-maximum suppression algorithm NMS based on DIOU improvement, and the obtained target bounding box coordinates and category information are used as network output.
[0063] The loss function uses a bipartite graph matching algorithm. After each generation of training, the optimal bipartite graph matching between the predicted result and the true value is found through the loss function minimization strategy, and the loss function is only calculated for the predicted target box that matches successfully. The loss function is defined as the weighted sum of the classification loss and the positioning loss:
[0064]
[0065] in Represents classification loss, calculated using Focal loss, Represents the normalized Euclidean distance between the center points of two target boxes, represents the normalized Manhattan distance between the upper left corner and the lower right corner of the two target boxes, λ cls , λ Manhattan is the weighting coefficient.
[0066] During the training phase, a one-to-one matching strategy between the prediction results and the true target frames is used to screen the target frames, and the loss function is calculated only for the successfully matched target frames. This strengthens the network's learning of the distribution of true target frames in the process of learning features, improves the calculation efficiency of the loss function, and can increase the model convergence speed. It can also indirectly reduce the probability of similar target frames being suppressed in the prediction results, thereby improving the detection accuracy.
[0067] The steps for training the constructed BP neural network are as follows:
[0068] (3-1) Adding Gaussian noise and adjusting contrast, brightness, and sharpness to the dataset to alleviate the class imbalance problem and achieve expansion; randomly selecting 65% of the expanded dataset as the training dataset, 15% of the images as the validation dataset, and the remaining 20% as the test dataset;
[0069] (3-2) The neck part of the BP network is randomly initialized, and the backbone part uses the pre-trained weights on the COCO dataset for transfer learning; in order to prevent the weights of the feature extraction network from being destroyed in the early stage of training, the backbone network parameters in the first 25 generations of training are frozen and do not participate in gradient updates;
[0070] (3-3) According to the error back propagation algorithm, the Adam optimizer and small batch stochastic gradient descent method are used. The learning rate descent curve adopts the StepLR fixed step size attenuation strategy, and gamma is 0.9. The weights of the backbone and neck structures are fine-tuned and updated respectively.
[0071] Step 4: Use the semantic segmentation network LEDNet to segment the original image and obtain the PV module string mask; perform binary segmentation on the PV module string mask to obtain the PV module mask, then perform median filtering to eliminate noise points, and perform corrosion and expansion operations to optimize the PV module segmentation results, and take its minimum circumscribed rectangle as the optimized PV module mask; finally, determine the faulty PV module based on the positioning results of the target detection network, and obtain the pixel coordinates of the corner points of the target module.
[0072] Step 5: Use the target positioning method based on the UAV POS data to obtain the camera attitude angle, field of view angle, flight altitude, GPS coordinates and other information during image capture through the onboard GPS / INS system, and calculate the GPS coordinates of the target pixel point based on the aerial triangulation geometry relationship.
[0073] The target detection module obtains the coordinates of the four corner points (x1, y1, x2, y2) and the defect category of the defective component, and calculates the pixel coordinates (x, y) of the center point of the photovoltaic component, that is, the coordinates of the target pixel point, according to the following formula:
[0074]
[0075] The schematic diagram of the flight parameters and camera field of view during drone inspection is as follows: Figure 2 As shown. The flight altitude of the drone is h, and the GPS coordinates are (x D ,y D ), camera field of view angle γ, pitch angle θ ZD , heading angle ψ ZD , the camera's field of view is (y f0 ,y f1 ), (x f0 , x f1 ), the GPS coordinates of the target pixel are (X, Y), and the size of the infrared image is (W, H).
[0076] First, calculate the four corner points of the camera's field of view, that is, the GPS coordinates of the four vertices of the image. According to the aerial triangulation relationship, we have:
[0077]
[0078] According to the similarity between the infrared image pixel coordinates and the actual GPS coordinates, the GPS coordinates of the center point of the target photovoltaic module can be calculated by the following formula:
[0079]
[0080] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.
Claims
1. UAV photovoltaic fault diagnosis and positioning method based on attention neural network, It is characterized in that The steps include: (1) Obtain infrared images of photovoltaic modules taken by inspection drones, and read the real-time location information and attitude data of the drones, where the flight altitude of the drone is h and the GPS coordinates are (x D ,y D ), camera field of view angle γ, pitch angle θ ZD , heading angle ψ ZD ; (2) Based on the attention mechanism and FPN structure, the FPT module is constructed, which includes three transformer structures: self-transformer ST, implanted transformer GT, and rendered transformer RT, which respectively realize the global information interaction within the feature layer and the local information interaction between top-down and bottom-up layers: The ST structure performs three different convolution operations on the current feature layer to obtain values, keys, and queries, and divides queries and keys into N equal parts to obtain q i , k j , use point multiplication to calculate the similarity s of each pair i,j Then use MoS normalization to get the weight coefficient w i,j , perform weighted summation on the values to get the feature map: Among them, F sim is the dot product operation, F mul is the matrix multiplication operation, F MoS The normalization operation is defined as follows, is a trainable parameter for normalization, For all keys i,j The arithmetic mean is: GT is a top-down feature interaction that implants the semantic information of the deep feature map into the high-resolution information of the shallow feature map to enhance the shallow feature map. Considering the differences in semantic information extracted from feature maps of different sizes, the negative Euclidean distance is used to calculate the correlation. The formula is as follows: where q i Shallow feature map, k j , v j From the deep feature map, F eud represents the Euclidean distance, that is, every pair of q i and k j The closer the distance, the greater the weight: F eud (q i ,k j )=-||q i -k j || 2 (4) RT is a bottom-up feature fusion, which aims to render high semantic information through high-resolution pixel information and realize the rendering of deep feature maps. Q comes from high-level feature maps, and K and V come from low-level feature maps. First, the weight w is obtained by global average pooling of K to weight Q, and then the size of V is reduced by convolution with stride. Finally, K and Q with the same size after processing are added and refined to obtain the output feature map. Where GAP represents global average pooling, aligning the sizes of K and Q; F att is the outer product function, which realizes the weighting of the feature map Q; F sconv represents a 3×3 convolution with a stride, which is used to reduce the size of the low-level feature map V; F conv Represents a 3×3 convolution, which is used to weight the result Q att Optimize; F add It means adding first and then refining through 3×3 convolution; (3) A single-stage target detection BP neural network was constructed, CSPDarknet53 was used as the backbone network for feature extraction, the FPT structure was used for information fusion, the output module used the anchor frame idea for prediction and regression, and the optimal binary matching idea was used to design the loss function; the aerial image size was unified to 416×416×3 as the network input, and the pixel coordinates of the center of the photovoltaic module where the defect was located were used as the output. The constructed neural network was trained to obtain a neural network for infrared image defect detection; The output module divides the feature map into S×S grids, each of which is responsible for predicting nine anchor boxes of different scales, that is, learning the relative offset of the anchor box in the grid and the probability of each category. x , t y , t w , t h , p obj , p cls ) six parameters, among which (t x , t y ) represents the offset of the center coordinate of the anchor box relative to the grid point, (t w , t h ) represents the ratio of the width and height of the bounding box to the width and height of the preset bounding box, p obj Represents the probability that the bounding box contains the detected target, p cls Indicates the probability that the detected target belongs to each category. Finally, all prediction results are screened by the non-maximum suppression algorithm NMS based on DIOU improvement, and the obtained target bounding box coordinates and category information are used as network output; The loss function uses a bipartite graph matching algorithm. After each generation of training, the optimal bipartite graph matching between the predicted result and the true value is found through the loss function minimization strategy, and the loss function is only calculated for the predicted target box that is successfully matched. The loss function is defined as the weighted sum of the classification loss and the positioning loss: in Represents classification loss, calculated using Focal loss, Represents the normalized Euclidean distance between the center points of two target boxes, represents the normalized Manhattan distance between the upper left corner and the lower right corner of the two target boxes, λ cls , λ Manhattan is the weighting coefficient; (4) Use the semantic segmentation network LEDNet to segment the original image and obtain the photovoltaic module string mask; perform binary segmentation on the photovoltaic module string mask to obtain the photovoltaic module mask, then perform median filtering to eliminate noise points, and perform corrosion and expansion operations to optimize the photovoltaic module segmentation results, and take its minimum circumscribed rectangle as the optimized photovoltaic module mask; finally, determine the faulty photovoltaic module based on the positioning results of the target detection network, and obtain the pixel coordinates of the corner points of the target module; (5) Using the target positioning method based on the UAV POS data, the camera attitude angle, field of view angle, flight altitude and GPS coordinate information of the UAV during image capture are obtained through the airborne GPS / INS system, and the GPS coordinates of the target pixel point are calculated according to the aerial triangulation geometry relationship; The target detection module obtains the coordinates of the four corner points of the defective component (x 1 ,y 1 , x 2 ,y 2 ) and defect category, the pixel coordinates (x, y) of the center point of the photovoltaic module, i.e. the target pixel coordinates, are calculated according to the following formula: The flight altitude of the drone is h, and the GPS coordinates are (x D ,y D ), camera field of view angle γ, pitch angle θ ZD , heading angle ψ ZD , the camera's field of view is (y f0 ,y f1 ),(x f0 ,x f1 ), the GPS coordinates of the target pixel are (X, Y), and the size of the infrared image is (W, H); First, calculate the four corner points of the camera's field of view, that is, the GPS coordinates of the four vertices of the image. According to the aerial triangulation relationship, we have: According to the similarity relationship between the infrared image pixel coordinates and the actual GPS coordinates, the GPS coordinates of the center point of the target photovoltaic module are calculated by formula (9); 2. The method for photovoltaic fault diagnosis and positioning of unmanned aerial vehicles based on attention neural network according to claim 1, It is characterized in that The BP neural network constructed in step (3) uses the idea of optimal binary matching to design the loss function. After each generation of training, the detection result is optimally binary matched with the true value, and the loss function is only calculated for the predicted target box that is successfully matched; the loss function includes three parts: classification loss, center point loss and corner point loss.
3. The method for photovoltaic fault diagnosis and positioning of unmanned aerial vehicles based on attention neural network according to claim 1, It is characterized in that CSPDarknet53, the backbone network, uses Mish as the neural network activation function, which allows better gradient flow for slightly negative values, rather than hard zero boundaries like in the ReLU function: Mish(x)=x·tanh(ln(1+e x )) (10)。 4. The method for photovoltaic fault diagnosis and positioning of unmanned aerial vehicles based on attention neural network according to claim 2, It is characterized in that The steps for training the constructed BP neural network are as follows: (1) Gaussian noise is added to the dataset and contrast, brightness, and sharpness are adjusted to alleviate the class imbalance problem and achieve expansion; 65% of the expanded dataset is randomly selected as the training dataset, 15% of the images constitute the validation dataset, and the remaining 20% constitute the test dataset; (2) The neck part of the BP network is randomly initialized, and the backbone part uses the pre-trained weights on the COCO dataset for transfer learning; in order to prevent the weights of the feature extraction network from being destroyed in the early stage of training, the backbone network parameters in the first 25 generations of training are frozen and do not participate in gradient updates; (3) According to the error back propagation algorithm, the Adam optimizer and small batch stochastic gradient descent method are used. The learning rate descent curve adopts the StepLR fixed step size attenuation strategy, and gamma is 0.
9. The weights of the backbone and neck structures are fine-tuned and updated respectively.
5. The method for photovoltaic fault diagnosis and positioning of unmanned aerial vehicles based on attention neural network according to claim 1, It is characterized in that In step (5), the aerial triangulation geometric relationship of the drone's aerial photography posture is used to locate the target in the aerial image.
Citation Information
Cited By
Photovoltaic string operation state identification method and system
CN115546579A
