A method for quickly finding and locating hot spots in infrared video based on drones

Abstract

The present invention discloses a method for quickly finding and locating hot spots based on infrared video of unmanned aerial vehicles. Infrared video is collected and preprocessed by a drone; the starting line, the ending line and the hot spot are identified in the infrared video, the relationship between the starting line, the ending line and the hot spot in the infrared video is obtained, and the location of the hot spot is found according to the relationship of the infrared video and the GPS information on the drone. The present invention uses a multi-rotor drone carrying a thermal imager to obtain infrared images of photovoltaic components in the air, and uses an onboard processor to process the infrared image to determine whether there is a hot spot fault, and obtain the hot spot coordinates; it can accurately count the size of the hot spot, the number of hot spots, and the GPS of the hot spot, thereby reducing the cost of resource use.

Description

Technical Field

[0001] The invention relates to a method for detecting hot spots of photovoltaic panels of an unmanned aerial vehicle, and to a method for quickly finding and locating hot spots through infrared video based on an unmanned aerial vehicle. Background Art

[0002] In recent years, the alarm bell of energy crisis has been sounded, and the development of safe and sustainable clean energy is imminent. As the main force of new clean energy, solar power generation is favored by clean energy developers in various countries for its rich resources and pollution-free characteristics.

[0003] Large photovoltaic power stations are generally built in deserts and Gobi, where there are vast areas, sparse populations, and sufficient light sources, but the working environment of the equipment is also relatively harsh. In addition to being exposed to sunlight, solar panels that are exposed to the outside for a long time also have to face problems such as flying sand and rocks and bird droppings. These external influences will shield the surface of the solar panel. Over time, the small part of the solar panel that is shielded will have a higher resistance and become a load. Such a load will continue to heat up until the solar panel is burned. This is the so-called hot spot phenomenon. It can be seen that the hot spot phenomenon is a direct and irreparable harm to solar panels.

[0004] my country has already promoted the development of new energy sources such as solar photovoltaic power generation to a strategic level. my country is currently a major photovoltaic power generation country, and the scale of photovoltaic power generation will continue to develop by leaps and bounds in the foreseeable future. However, the early detection technology and operation and maintenance management system for faults such as hot spots, which are extremely harmful, are still backward.

[0005] In response to the above problems, we combine the latest theoretical and technological achievements in the fields of multi-source sensing, drones, wireless communications and digital image processing to serve the improvement of automated operation and maintenance technology and the construction of advanced operation and maintenance systems in the photovoltaic power generation industry. Summary of the invention

[0006] In order to solve the problems in the background technology, the present invention proposes a method technology for quickly finding and locating hot spots of infrared video based on drones, so as to improve the power generation efficiency of photovoltaic power stations, reduce the operation and maintenance costs of photovoltaic power stations and realize the intelligent operation and maintenance of photovoltaic power stations.

[0007] The technical solution adopted by the present invention is as follows:

[0008] Step 1: Collect infrared video through drone and pre-process it;

[0009] Step 2: Identify the start line, end line and hot spot in the infrared video, obtain the relationship between the start line, end line and hot spot in the infrared video, and find the location of the hot spot based on the relationship in the infrared video and the GPS information on the drone.

[0010] The step 1 includes the following: first, the infrared video taken by the drone is sent to the ground workstation, the ground workstation reads the infrared video taken by the drone, extracts frames for the infrared video, performs H.246 format transcoding on the extracted video, and saves the transcoded video.

[0011] In step 1, an infrared camera is installed on the drone, and when the drone flies above the photovoltaic panel, the infrared camera is used to look down and shoot infrared video.

[0012] The step 2 is specifically:

[0013] In the infrared video, each frame of the image is detected in chronological order, and the frame number of the image where the start line, end line and hot spot first appear in the infrared video is identified through image processing and analysis. The position of the hot spot is found by interpolation based on the GPS information on the drone when the start line and end line correspond to the frame number in the infrared video.

[0014] In step 2, identifying the starting line and the ending line is specifically as follows:

[0015] Perform Hough transform line detection on each frame of the infrared video in chronological order:

[0016] When the first black straight line is just detected, the first black straight line is marked as the starting line, and the frame number of the image where the current image frame first appears as the starting line in the infrared video is recorded;

[0017] When the second black straight line is just detected, the second black straight line is marked as the end line, and the frame number of the image where the current image frame first appears as the end line in the infrared video is recorded.

[0018] In step 2, identifying hot spots is specifically as follows:

[0019] The image is histogram equalized in real time, and then sent to the hot spot detection model after background denoising, dilation, and corrosion processing. The processing output is obtained to obtain the prediction frame, which is then processed for grayscale change and the average pixel value M of the prediction frame is calculated. At this time, pixels with grayscale values ​​> M in the prediction frame are judged to belong to hot spots.

[0020] The hot spot detection model is the Faster R-CNN detection model;

[0021] The Faster R-CNN detection model originally included the ROIpooling part, the RPN part, the classification and regression part, and the feature extraction part. The feature extraction part first extracts features and then inputs them into the RPN part. The RPN part outputs them to the ROIpooling part. The output of the ROIpooling part is sent to the classification and regression part, which then outputs the feature results.

[0022] The ROIpooling part in the Faster R-CNN detection model is replaced by the ROIAlign part. The specific structure of the ROIAlign part includes the following operations: taking the output result of the RPN part as the prediction area, the prediction area is divided into X*Y parts for the first time to obtain each first unit, and then Z*H division is performed on each first unit after the first division to obtain each second unit. In the second unit, the central pixel is obtained as the pixel value of the second unit through bilinear interpolation. The pixel values ​​of all the second units are recombined to form a new feature map, and the maximum pooling operation is performed to obtain the prediction box.

[0023] In step 2, the position of the hot spot is determined by interpolation between the GPS information on the drone at the frame numbers corresponding to the start line and the end line in the infrared video, based on the proportional relationship between the frame numbers of the image when the hot spot first appears in the infrared video and the frame numbers of the image when the start line and the end line first appear in the infrared video.

[0024] The present invention uses a multi-rotor unmanned aerial vehicle carrying a thermal imager to obtain infrared images of photovoltaic components in the air, and uses an onboard processor to process the infrared images to determine whether a hot spot fault exists and obtain the hot spot coordinates.

[0025] Compared with the prior art, the method for quickly finding and locating infrared hot spots based on drones provided by the embodiments of the present invention has the following beneficial effects:

[0026] 1. The embodiment of the present invention uses traditional image processing methods to quickly find and locate hot spots in infrared videos and provide corresponding hot spot coordinates, wherein the following processing is performed on the saved infrared video: histogram equalization, expansion, corrosion, start line detection, ROI, end line detection, and hot spot determination.

[0027] 2. The embodiment of the present invention has a better effect in detecting and locating hot spots with a large number of hot spots and inconsistent rules, and can accurately count the size of the hot spots, the number of hot spots, and the GPS of the hot spots.

[0028] 3. The method adopted in the embodiment of the present invention is fast and simple, and does not use a deep learning algorithm, thereby reducing the time for collecting a large number of data sets and training data sets, and further reducing resource usage costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a process flow chart of the method of the present invention;

[0030] Figure 2 Judgment and flow chart for the start and end of video processing;

[0031] Figure 3 Infrared heat map. DETAILED DESCRIPTION

[0032] The present invention will be further described below in conjunction with the accompanying drawings and embodiments, so that the present invention can be more clearly understood, thereby making a clearer and definite definition of the protection scope of the present invention.

[0033] like Figure 1 As shown, the embodiments of the present invention are as follows:

[0034] Step 1: Collect infrared video through drone and pre-process it;

[0035] An infrared camera is installed on the drone, and when the drone flies over the photovoltaic panel, it looks down and shoots infrared video. The drone flies in a direction parallel to one edge of the photovoltaic panel.

[0036] First, the infrared video taken by the drone is sent to the ground workstation. The ground workstation reads the infrared video taken by the drone, extracts frames from the infrared video, transcodes the extracted video into H.246 format, regenerates the file name of the transcoded video and saves it.

[0037] Step 2: If Figure 2 As shown, the starting line, the ending line and the hot spot are identified in the infrared video, the relationship between the starting line, the ending line and the hot spot in the infrared video is obtained, and the position of the hot spot is found according to the relationship in the infrared video and the GPS information on the UAV.

[0038] In the infrared video, each frame of the image is detected in chronological order, and the frame number of the image where the start line, end line and hot spot first appear in the infrared video is identified through image processing and analysis. The position of the hot spot is found by interpolation based on the GPS information on the drone when the start line and end line correspond to the frame number in the infrared video.

[0039] Normally, if Figure 3 As shown in the figure, each photovoltaic panel has two interval lines perpendicular to the direction of the drone's travel, and the interval lines are a black area. On each photovoltaic panel, the first interval line upstream along the direction of the drone's travel is the starting line, and the second interval line downstream along the direction of the drone's travel is the ending line.

[0040] The drone will record and save the GPS information of the start and end lines of each photovoltaic panel read in real time.

[0041] Identify the start line and end line as follows:

[0042] Perform Hough transform line detection on each frame of the infrared video in chronological order:

[0043] When the first black straight line is just detected, the first black straight line is marked as the starting line, and the frame number of the image where the current image frame first appears as the starting line in the infrared video is recorded;

[0044] When the second black straight line is just detected, the second black straight line is marked as the end line, and the frame number of the image where the current image frame first appears as the end line in the infrared video is recorded.

[0045] Identify hot spots specifically as follows:

[0046] The image is histogram-equalized in real time, and then sent to the hot spot detection model after background denoising, dilation, and corrosion processing. The processing output is a prediction frame, and the prediction area represents the possible hot spot area. Then the grayscale change processing is performed to calculate the average pixel value M of the prediction frame. At this time, the pixels with grayscale values ​​> M in the prediction frame are judged to belong to hot spots. After the hot spots are obtained, a black border is drawn around the hot spots.

[0047] The hot spot detection model is the Faster R-CNN detection model; the ROIpooling part in the Faster R-CNN detection model is replaced by the ROIAlign part. The specific structure of the ROIAlign part includes the following operations: using the output result of the RPN part as the prediction area, the prediction area is divided into X*Y parts for the first time to obtain each first unit, and then Z*H division is performed on each first unit after the first division to obtain each second unit. In the second unit, the central pixel is obtained as the pixel value of the second unit through bilinear interpolation. The pixel values ​​of all the second units are recombined to form a new feature map, and the maximum pooling operation is performed to obtain the prediction box.

[0048] Finally, after obtaining the frame numbers of the image where the start line, end line and hot spot first appear in the infrared video, the position of the hot spot is determined by interpolation between the GPS information on the drone at the frame numbers corresponding to the start line and end line in the infrared video according to the proportional relationship between the frame numbers of the image where the hot spot first appears in the infrared video and the frame numbers of the image where the start line and end line first appear in the infrared video, thereby realizing the positioning of the hot spot.

[0049] In summary, the embodiment of the present invention uses a multi-rotor drone carrying a thermal imager to obtain infrared images of photovoltaic modules in the air, and uses an onboard processor to process the infrared images to determine whether there is a hot spot fault and give the corresponding hot spot coordinates. The following processing is performed on the saved infrared video: histogram equalization, expansion, corrosion, start line detection, ROI, end line detection, and hot spot determination. The entire process detection is fast, effective, and simple, without the need to use a large amount of data sets for training, reducing resource costs.

[0050] Finally, it should be pointed out that the above embodiments and the proposed control method are only representative examples of the present invention. Obviously, the technical solution of the present invention is not limited to the above embodiments and the proposed control method, and there may be many variations. A person skilled in the art can directly derive or make several improvements from the disclosure of the present invention, and these improvements should also be within the scope of protection of the claims of the present invention.

Claims

1. A method for quickly finding and locating hot spots in infrared video based on drones. It is characterized in that The method comprises the following steps: Step 1: Collect infrared video through drone and pre-process it; Step 2: Identify the start line, end line and hot spot in the infrared video, obtain the relationship between the start line, end line and hot spot in the infrared video, and find the location of the hot spot based on the relationship in the infrared video and the GPS information on the drone; The step 2 is specifically as follows: in the infrared video, each frame image is detected in time sequence, the frame number of the image where the start line, the end line and the hot spot first appear in the infrared video is identified by image processing and analysis means, and the position of the hot spot is found by interpolation method according to the GPS information on the drone when the start line and the end line respectively appear in the infrared video. In step 2, the identification of hot spots is specifically as follows: performing histogram equalization on the image in real time, and then sending it to the hot spot detection model after background denoising, dilation, and corrosion processing in sequence, processing the output to obtain a prediction frame, and then performing grayscale change processing to calculate the average pixel value M of the prediction frame. At this time, pixels with grayscale values ​​> M in the prediction frame are determined to belong to hot spots; In step 2, the position of the hot spot is determined by interpolation between the GPS information on the drone at the frame numbers corresponding to the start line and the end line in the infrared video, based on the proportional relationship between the frame numbers of the image when the hot spot first appears in the infrared video and the frame numbers of the image when the start line and the end line first appear in the infrared video.

2. The method for quickly finding and locating hot spots in infrared video based on a drone according to claim 1, Its characteristics are: The step 1 includes the following: first, the infrared video taken by the drone is sent to the ground workstation, the ground workstation reads the infrared video taken by the drone, extracts frames for the infrared video, performs H.246 format transcoding on the extracted video, and saves the transcoded video.

3. The method for quickly finding and locating hot spots using infrared video based on a drone according to claim 1, Its characteristics are: In step 1, an infrared camera is installed on the drone, and when the drone flies above the photovoltaic panel, the infrared camera is used to look down and shoot infrared video.

4. The method for quickly finding and locating hot spots using infrared video based on a drone according to claim 1, Its characteristics are: In step 2, identifying the starting line and the ending line is specifically as follows: Perform Hough transform line detection on each frame of the infrared video in chronological order: When the first black straight line is just detected, the first black straight line is marked as the starting line, and the frame number of the image where the current image frame first appears as the starting line in the infrared video is recorded; When the second black straight line is just detected, the second black straight line is marked as the end line, and the frame number of the image where the current image frame first appears as the end line in the infrared video is recorded.

5. The method for quickly finding and locating hot spots using infrared video based on a drone according to claim 1, Its characteristics are: The hot spot detection model is a Faster R-CNN detection model; the ROIpooling part in the Faster R-CNN detection model is replaced by the ROIAlign part, and the specific structure of the ROIAlign part includes the following operations: using the output result of the RPN part as the prediction area, performing X*Y division on the prediction area for the first time to obtain each first unit, performing Z*H division on each first unit after the first division to obtain each second unit, obtaining the central pixel as the pixel value of the second unit in the second unit through bilinear interpolation, recombining the pixel values ​​of all the second units into a new feature map, and performing a maximum pooling operation to obtain a prediction box.

Images