Photovoltaic panel hot spot positioning method based on drone and thermal imaging

Abstract

The present invention discloses a method for locating hot spots of photovoltaic panels based on unmanned aerial vehicles and thermal imaging. The location information of photovoltaic modules is measured by a GPS positioning instrument and stored in a database; the total number of rows of photovoltaic panels in the photovoltaic module is calculated based on the collected location information of the photovoltaic modules; the flight of the unmanned aerial vehicle is controlled, and during the flight, the flight trajectory of the unmanned aerial vehicle is corrected using the location information of the four corner points of the photovoltaic module; an image algorithm is used to identify and segment a single photovoltaic panel in an aerial image, and the number of columns of photovoltaic panels in the photovoltaic module is located; the signal receiving device on the unmanned aerial vehicle determines whether to turn on the infrared camera, detects hot spots in real time, and processes to obtain the location information of the hot spots. The present invention makes full use of the automatic inspection of unmanned aerial vehicles to realize automatic positioning of hot spots and achieve correction of the flight trajectory of unmanned aerial vehicles. It is not only convenient to calculate but also has high detection efficiency, so that the photovoltaic power station can achieve more efficient and intelligent operation.

Description

Technical Field

[0001] The present invention belongs to the technical field of photovoltaic power generation, and specifically relates to the efficiency of photovoltaic power generation and the device durability of photovoltaic components, and relates to a method for locating hot spots of photovoltaic panels based on drones and thermal imaging. Background Art

[0002] In order to cope with global climate change and reduce carbon emissions, China has increased its development and investment in renewable energy. Solar energy has the advantages of universality, harmlessness, large reserves, and long-term use, and plays an important role in long-term energy strategies.

[0003] The solar photovoltaic power generation system is composed of a series of solar module cells connected in series. During the operation of the photovoltaic power station, it is inevitable that bird excrement, floating soil, fallen leaves and other obstructions will cover the photovoltaic modules. These obstructions will form local shadows on the photovoltaic modules, causing the current and voltage of some cells in the photovoltaic modules to change, causing the local temperature of the photovoltaic modules to rise. This phenomenon is called the hot spot effect.

[0004] The hot spot effect has a great impact on photovoltaic power stations. It not only affects the power generation efficiency of photovoltaic modules, but also may cause fires, posing a great threat to the safety of photovoltaic power stations. The hot spot effect is harmful to photovoltaic modules, so it is necessary to promptly detect photovoltaic modules that produce hot spot effects through inspections. The most popular solution is to use drones to shoot, transmit videos, and then manually calibrate the hot spot position. Due to certain visual errors of humans, it is easy to calibrate incorrectly, and it also wastes human resources. Summary of the invention

[0005] In order to overcome the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a method for locating hot spots of photovoltaic panels using drones and thermal imaging. By performing drone inspections, the GPS positions of hot spots are automatically recorded, thereby realizing automatic positioning of hot spots and enabling photovoltaic power stations to achieve efficient and intelligent operation.

[0006] In order to achieve the above object, the technical solution of the present invention comprises the following steps:

[0007] The method comprises the following steps:

[0008] Step 1: Measure the location information of the photovoltaic module through the GPS positioning instrument and store it in the database;

[0009] The location information is latitude and longitude location information.

[0010] Step 2: Based on the collected location information of the photovoltaic modules, the location information of the beginning and end of the photovoltaic module row and the location information of the beginning and end of the photovoltaic module row of a single photovoltaic panel are used to calculate the total number of rows of photovoltaic panels in the photovoltaic module;

[0011] Step 3: Control the drone to fly. During the flight:

[0012] The flight trajectory of the UAV is corrected using the position information of the four corner points of the photovoltaic module;

[0013] Use image algorithms to identify and segment individual photovoltaic panels in aerial images, and locate the number of photovoltaic panel columns in a photovoltaic module;

[0014] The signal receiving device on the drone determines whether to turn on the infrared camera, detect the hot spot in real time, and process the location information of the hot spot.

[0015] In the step 1, the details are as follows: the photovoltaic assembly is mainly composed of multiple photovoltaic panels tightly arranged in the same plane, and the longitude and latitude information of the four corner points of the photovoltaic assembly and the longitude and latitude information of the four corner points of a single photovoltaic panel are obtained by using a GPS positioning instrument arranged at the four corners of the photovoltaic panel, and the collected longitude and latitude information is stored in a database.

[0016] In step 2, the specific method for calculating the number of photovoltaic panels is as follows:

[0017] Since the location information collected by the photovoltaic components is on the same side, the upper left corner is taken as the beginning and the lower left corner is taken as the end, and the photovoltaic panel data is obtained according to the following formula:

[0018] num=(A1-A2) / (A3-A4)

[0019] Among them, A1 is the position information of the beginning of the photovoltaic module, A2 is the position information of the end of the photovoltaic module, A3 is the position information of the beginning of a single photovoltaic panel in any column of the photovoltaic module, A4 is the position information of the end of a single photovoltaic panel in any column of the photovoltaic module, and num represents the total number of rows of photovoltaic panels in the photovoltaic module.

[0020] In step 3, the specific method for correcting the flight trajectory of the drone is as follows:

[0021] The position information of the four corner points of the photovoltaic module are B1, B2, B3, and B4 respectively. The center positions of the two opposite sides of the photovoltaic module parallel to the column direction are calculated as follows:

[0022] B5=(B1+B2) / 2

[0023] B6=(B3+B4) / 2

[0024] The line connecting the center position B5 and the center position B6 is taken as the ideal trajectory of the drone;

[0025] The drone uses its own GPS positioning device to obtain its current position and determine whether its current position is in the ideal trajectory. If not, the flight trajectory is adjusted so that the drone flies horizontally over the photovoltaic panels along the ideal trajectory.

[0026] In a specific implementation, the drone is controlled to fly from the left side to the right side of the middle row of the photovoltaic modules along an ideal trajectory, so that the photovoltaic panels on the left side column of the photovoltaic modules enter the drone's infrared image first, and the photovoltaic panels on the middle row of the left side column of the photovoltaic modules reach the middle position in the infrared image first.

[0027] In step 3, the specific method of judging to turn on the infrared camera through the signal receiving device on the drone is as follows:

[0028] A signal receiving device is placed on the drone, and a signal sending device is placed at the corner point of each photovoltaic module. The signal sending device is connected to the GPS positioning instruments at the four corners of the photovoltaic module. The signal sending device collects the position information of the GPS positioning instruments at the four corners of the photovoltaic module in real time and sends it to the signal receiving device. The drone receives the position information of the GPS positioning instruments at the four corners of the photovoltaic module from the signal receiving device in real time and determines the area above the photovoltaic module. When the drone flies to the area above the photovoltaic module, the infrared camera is turned on to detect hot spots towards the photovoltaic module directly below.

[0029] In step 3, the specific method for locating the number of columns of photovoltaic panels in the photovoltaic assembly is as follows:

[0030] The camera on the drone looks down to shoot the photovoltaic panels to obtain video, and each image in the video is analyzed and processed in real time:

[0031] The image is first processed by the Otsu threshold segmentation algorithm to maximize the inter-class variance of each pixel class after segmentation, and then the image is processed by Gaussian filtering to complete the preprocessing;

[0032] The preprocessed image is sent to the Sobel edge detection algorithm to extract the edge, and the LSD line detection algorithm is used to segment and extract the image area where the single photovoltaic panel in the image is located.

[0033] Then, during the control of the UAV flight, the column cumulative number of the photovoltaic panel is initially set to 0. If the photovoltaic panel in the middle of the image leaves the middle of the image, the column cumulative number of the photovoltaic panel is increased by 1; the column cumulative number obtained in real time is used as the column number of a column of photovoltaic panels detected by the current UAV in the photovoltaic module.

[0034] The Otsu threshold segmentation process is as follows: for the image I (x, y), the segmentation threshold of the foreground and background is denoted as T, the proportion of the number of pixels belonging to the foreground in the whole image is denoted as ω0, and its average grayscale is μ0; the proportion of the number of background pixels in the whole image is ω1, and its average grayscale is μ1. The total average grayscale of the image is denoted as μ, and the inter-class variance is denoted as g.

[0035] The background of the image is dark, and the size of the image is M×N. The number of pixels whose grayscale values ​​are less than the threshold T in the image is recorded as N0, and the number of pixels whose grayscale values ​​are greater than the threshold T is recorded as N1. Then:

[0036] ω0=N0 / M×N, ω1=N1 / M×N, N0+N1=M×N, ω0+ω1=1

[0037] g=ω0ω1(μ0-μ1) 2

[0038] Among them, g represents the between-class variance.

[0039] Finally, the traversal method is used to obtain the threshold T that maximizes the inter-class variance g.

[0040] In the specific implementation, the image segmentation of the smallest photovoltaic panel adopts the opening and closing operations based on threshold segmentation, and uses the Sobel edge detection algorithm and the LSD line detection algorithm to extract the horizontal and vertical lines. The extracted vertical lines and vertical lines form a rectangle, and the smallest rectangular frame is extracted in the specified area to segment and extract the single photovoltaic panel area in the image.

[0041] The input of the LSD line detection algorithm is a grayscale image, and the output is the coordinates of the starting point and end point of the line segment and the attributes of the line segment, etc. The detected line segments are extracted by image downsampling, gradient descending sorting, small gradient value suppression, region growth, and rectangle estimation, so as to segment and extract the single photovoltaic panel area in the image.

[0042] In step 3, the hot spots are detected in real time and processed to obtain the location information of the hot spots. The specific method is as follows:

[0043] The infrared image collected by the infrared camera of the drone in real time is used to pre-process a row of photovoltaic panels in the middle vertical bar area of ​​the infrared image using Gaussian filtering, adaptive binarization, and morphological filtering image algorithms. The size of the middle vertical bar area is smaller than the size of the photovoltaic panel. The pre-processed infrared image is then used with the improved Faster RCNN model for target detection and identification of hot spots.

[0044] When a hot spot is detected, the current real-time accumulated column number is used as the column number of the photovoltaic panel where the hot spot is located in the photovoltaic module, and then the row number of the photovoltaic panel in the photovoltaic module is determined according to the positional relationship of the photovoltaic panel where the hot spot is located in the image, and then the position information of the four corner points of the photovoltaic panel where the hot spot is located is extracted. Finally, the position information of the hot spot is calculated based on the position information of the four corner points of the photovoltaic panel where the hot spot is located, the pixel position of the hot spot in the image and the pixel positions of the four corner points of the photovoltaic panel where the hot spot is located.

[0045] The improved Faster RCNN model replaces the CNN feature extraction network in the Faster RCNN model with the VGG16 network, and replaces the last two fully connected layers in the VGG16 network with a fully connected layer with 1024 neurons. The improved VGG16 network is combined with the RPN network to extract candidate box information, and hot spot detection is implemented according to the candidate box information. The specific steps are as follows:

[0046] (1) Input infrared image;

[0047] (2) Input the infrared image into the improved VGG16 network for feature extraction;

[0048] (3) The extracted features are input into the RPN network to generate candidate frame windows. Multiple candidate frame windows are generated for each infrared image. Specifically, 300 candidate frame windows can be generated.

[0049] (4) Map the extracted candidate box window to the convolutional feature map finally output by the VGG16 network;

[0050] (5) The convolution feature map is processed into a feature map of fixed size through the pooling layer;

[0051] (6) On a feature map of a fixed size, the classification probability and bounding box regression are jointly trained on the candidate box window using Softmax Loss (detection classification probability) and Smooth L1 Loss (detection bounding box regression), and the position of the detection box is continuously corrected. The detection box is a candidate box window that completely contains the hot spot, so as to achieve recognition and detection of the hot spot area.

[0052] The present invention combines the image preprocessing process, improves and optimizes the feature extraction network, continuously corrects the position of the detection frame, and obtains a model for identifying hot spot areas. This can increase the rate of hot spot defect detection, improve the problems of poor generalization ability and low accuracy in daily hot spot defect detection algorithms, and achieve real-time recognition and detection of hot spot areas of photovoltaic modules.

[0053] Compared with the prior art, the method provided by the embodiment of the present invention has the following beneficial effects:

[0054] The photovoltaic panel hot spot positioning method based on drone and thermal imaging provided in the embodiment of the present invention realizes automatic positioning of the hot spot by automatically recording the GPS position of the hot spot through drone inspection; the flight trajectory of the drone is corrected through the longitude and latitude information of the photovoltaic module, thus avoiding the problem of yaw, and realizing efficient and intelligent operation of the photovoltaic power station.

[0055] Compared with the existing photovoltaic panel hot spot positioning technology, the present invention makes full use of the automatic inspection of drones to realize the automatic positioning of hot spots, and corrects the flight trajectory of the drone through the longitude and latitude information of the photovoltaic components. It is not only convenient to calculate and has high detection efficiency, but also enables photovoltaic power stations to achieve more efficient and intelligent operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of a preferred embodiment of the method of the present invention;

[0057] Figure 2 A schematic diagram of the longitude and latitude of a photovoltaic module in a preferred embodiment of the method of the present invention.

[0058] Figure 3 Schematic diagram of an ideal trajectory of a drone in a preferred embodiment of the method of the present invention.

[0059] Figure 4 A hot spot positioning startup flow chart of a preferred embodiment of the method of the present invention. DETAILED DESCRIPTION

[0060] The preferred embodiments of the present invention are described in detail below so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0061] The embodiments of the present invention and their implementation process specifically include the following steps:

[0062] Step 1: Use GPS positioning equipment to measure the longitude and latitude information of the photovoltaic module and store it in the database;

[0063] The specific process of measuring the longitude and latitude information of the photovoltaic module by using the GPS positioning instrument and storing it in the database is as follows:

[0064] 1) Obtain the longitude and latitude information of the four corner points of the photovoltaic module and the longitude and latitude information of the four corner points of a single photovoltaic panel through a GPS positioning instrument;

[0065] 2) Store the collected latitude and longitude information into the database.

[0066] Step 2: According to the collected location information of the head end of the photovoltaic module, the location information of the tail end of the photovoltaic module and the location information of the head end and the tail end of a single photovoltaic panel, the number of photovoltaic panels in the photovoltaic module can be calculated;

[0067] Step 3: Use the signal receiving device on the drone to determine whether the infrared camera is turned on;

[0068] Step 4: Use the Otsu threshold segmentation algorithm to maximize the inter-class variance of each pixel class after segmentation, and then perform Gaussian filtering on the image. Send the preprocessed image to the Sobel edge detection algorithm, and use the LSD line detection algorithm for the extracted edges to segment and extract the single photovoltaic panel area in the image. Accurately identify and segment a single photovoltaic panel in the middle area of ​​the image, and accumulate the number of photovoltaic panels;

[0069] Step 5: During the flight of the drone, the flight trajectory of the drone is corrected using the position information of the four corner points of the photovoltaic module;

[0070] Step 6: Use the optimized Faster RCNN target detection network to detect the hot spot, and calculate the GPS position of the hot spot based on the longitude and latitude information of the photovoltaic module and the cumulative number of photovoltaic panels;

[0071] The specific process of detecting hot spots through the optimized Faster RCNN target detection network is as follows:

[0072] 1) Input infrared image and extract its features through the improved VGG16 network;

[0073] 2) Input the feature extraction results into the RPN network to generate candidate frame windows. 300 candidate frame windows are generated for each infrared image.

[0074] 3) Map the extracted candidate box window to the convolutional feature map finally output by the VGG16 network;

[0075] 4) The convolution feature map is processed through the pooling layer to generate a feature map of fixed size 60×40;

[0076] 5) On a feature map of a fixed size of 60×40, the classification probability and bounding box regression are jointly trained on the candidate box window using Softmax Loss (detection classification probability) and Smooth L1 Loss (detection bounding box regression), and the position of the detection box is continuously corrected. The detection box is a candidate box window that completely contains the hot spot, so as to achieve recognition and detection of the hot spot area.

[0077] In summary, the embodiment of the present invention provides a method for locating hot spots of photovoltaic panels based on drones and thermal imaging, and performs image processing on aerial images through drone inspections. In combination with the image preprocessing process, the Faster RCNN target detection network is improved and optimized, and the improved VGG16 network is used as a feature extraction network. The network depth is moderate, which can not only reduce network parameters and improve the rate of hot spot defect detection, but also improve the learning ability of the entire network, achieve real-time recognition and detection of hot spot areas of photovoltaic components, and automatically record the GPS position of hot spots. Automatic positioning of hot spots is achieved; the flight trajectory of the drone is corrected through the latitude and longitude information of the photovoltaic components, which avoids the problem of yaw and enables the photovoltaic power station to achieve efficient and intelligent operation.

[0078] The above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for locating hot spots of photovoltaic panels based on drones and thermal imaging, characterized in that: The method comprises the following steps: Step 1: Measure the location information of the photovoltaic module through the GPS positioning instrument and store it in the database; Step 2: Based on the collected location information of the photovoltaic modules, the location information of the beginning and end of the photovoltaic module row and the location information of the beginning and end of the photovoltaic module row of a single photovoltaic panel are used to calculate the total number of rows of photovoltaic panels in the photovoltaic module; Step 3: Control the drone to fly. During the flight: The flight trajectory of the UAV is corrected using the position information of the four corner points of the photovoltaic module; Use image algorithms to identify and segment individual photovoltaic panels in aerial images, and locate the number of photovoltaic panel columns in a photovoltaic module; The infrared camera is turned on through the signal receiving device on the drone, and the hot spots are detected in real time and processed to obtain the location information of the hot spots; In step 3, the specific method for locating the number of columns of photovoltaic panels in the photovoltaic assembly is as follows: The camera on the drone looks down to shoot the photovoltaic panels to obtain video, and each image in the video is analyzed and processed in real time: The image is first processed by the Otsu threshold segmentation algorithm to maximize the inter-class variance of each pixel class after segmentation, and then the image is processed by Gaussian filtering to complete the preprocessing; The preprocessed image is sent to the Sobel edge detection algorithm to extract the edge, and the LSD line detection algorithm is used to segment and extract the image area where the single photovoltaic panel in the image is located. Then, during the control of the UAV flight, the column cumulative number of the photovoltaic panel is initially set to 0. If the photovoltaic panel in the middle of the image leaves the middle of the image, the column cumulative number of the photovoltaic panel is increased by 1; the column cumulative number obtained in real time is used as the column number of a column of photovoltaic panels detected by the current UAV in the photovoltaic module.

2. A method for locating hot spots of photovoltaic panels based on drones and thermal imaging as claimed in claim 1, characterized in that: In the step 1, the details are as follows: the photovoltaic assembly is mainly composed of multiple photovoltaic panels tightly arranged in the same plane, and the longitude and latitude information of the four corner points of the photovoltaic assembly and the longitude and latitude information of the four corner points of a single photovoltaic panel are obtained by using a GPS positioning instrument arranged at the four corners of the photovoltaic panel, and the collected longitude and latitude information is stored in a database.

3. The method for locating hot spots of photovoltaic panels based on drones and thermal imaging as claimed in claim 1, characterized in that: In step 2, the specific method for calculating the number of photovoltaic panels is as follows: Taking the upper left corner as the beginning and the lower left corner as the end, the photovoltaic panel data is obtained according to the following formula: num=(A1-A2) / (A3-A4) Among them, A1 is the location information of the beginning of the photovoltaic module, A2 is the location information of the end of the photovoltaic module, A3 is the location information of the beginning of a single photovoltaic panel, A4 is the location information of the end of a single photovoltaic panel, and num represents the total number of rows of photovoltaic panels in the photovoltaic module.

4. The method for locating hot spots of photovoltaic panels based on drones and thermal imaging as claimed in claim 1, characterized in that: In step 3, the specific method for correcting the flight trajectory of the drone is as follows: The position information of the four corner points of the photovoltaic module are B1, B2, B3, and B4 respectively. The center positions of the two opposite sides of the photovoltaic module parallel to the column direction are calculated as follows: B5=(B1+B2) / 2 B6=(B3+B4) / 2 The line connecting the center position B5 and the center position B6 is taken as the ideal trajectory of the drone; The drone uses its own GPS positioning device to obtain its current position and determine whether the drone's current position is in the ideal trajectory. If not, the flight trajectory is adjusted so that the drone flies horizontally along the ideal trajectory.

5. The method for locating hot spots of photovoltaic panels based on drones and thermal imaging as claimed in claim 1, characterized in that: In the step 3, the specific method of judging whether to turn on the infrared camera by the signal receiving device on the drone is as follows: a signal receiving device is placed on the drone, a signal sending device is placed at the corner point of each photovoltaic module, the signal sending device is connected to the GPS positioning instruments at the four corners of the photovoltaic module, the signal sending device collects the position information of the GPS positioning instruments at the four corners of the photovoltaic module in real time and sends it to the signal receiving device, the drone receives the position information of the GPS positioning instruments at the four corners of the photovoltaic module from the signal receiving device in real time and judges the area above the photovoltaic module, and when the drone flies to the area above the photovoltaic module, the infrared camera is turned on to detect hot spots towards the photovoltaic module directly below.

6. The method for locating hot spots of photovoltaic panels based on drones and thermal imaging as claimed in claim 1, characterized in that: In step 3, the hot spots are detected in real time and processed to obtain the location information of the hot spots. The specific method is as follows: The infrared images collected by the infrared camera of the drone in real time are preprocessed using Gaussian filtering, adaptive binarization, and morphological filtering image algorithms. The preprocessed infrared images are then used with the improved Faster RCNN model for target detection and identification of hot spots. When a hot spot is detected, the current real-time accumulated column number is used as the column number of the photovoltaic panel where the hot spot is located in the photovoltaic module, and then the row number of the photovoltaic panel in the photovoltaic module is determined according to the positional relationship of the photovoltaic panel where the hot spot is located in the image, and then the position information of the four corner points of the photovoltaic panel where the hot spot is located is extracted. Finally, the position information of the hot spot is calculated based on the position information of the four corner points of the photovoltaic panel where the hot spot is located, the pixel position of the hot spot in the image and the pixel positions of the four corner points of the photovoltaic panel where the hot spot is located.

7. A method for locating hot spots of photovoltaic panels based on drones and thermal imaging as claimed in claim 6, characterized in that: The improved Faster RCNN model replaces the CNN feature extraction network in the Faster RCNN model with the VGG16 network, and replaces the last two fully connected layers in the VGG16 network with a fully connected layer with 1024 neurons. The improved VGG16 network is combined with the RPN network to extract candidate box information, and hot spot detection is implemented according to the candidate box information. The specific steps are as follows: (1) Input infrared image; (2) Input the infrared image into the improved VGG16 network for feature extraction; (3) The extracted features are input into the RPN network to generate candidate frame windows. Multiple candidate frame windows are generated for each infrared image. (4) Map the extracted candidate box window to the convolutional feature map finally output by the VGG16 network; (5) The convolution feature map is processed into a feature map of fixed size through the pooling layer; (6) On a feature map of a fixed size, Softmax Loss is used to detect the classification probability and Smooth L1 Loss is used to detect the bounding box regression. The classification probability and bounding box regression are jointly trained on the candidate box window, and the position of the detection box is continuously corrected. The detection box is a candidate box window that completely contains the hot spot, so as to achieve the recognition and detection of the hot spot area.

Images