Forest fire detection method

Abstract

The invention provides a forest fire detection method, which comprises the following steps of: firstly, constructing a core algorithm of an unmanned aerial vehicle flight channel identification module by taking an electronic geographic information map of a to-be-inspected area acquired by an unmanned aerial vehicle as a basis and combining attitude data of the unmanned aerial vehicle and estimated time consumed in a flight period according to unmanned aerial vehicle path planning; planning and setting a corresponding inspection line; and then the unmanned aerial vehicle is used for aerial photography of forest work, the whole forest area is covered by unmanned aerial vehicle shooting, and the ground console accurately detects all pictures shot by an unmanned aerial vehicle camera by using a target detection algorithm, so that close fit of the unmanned aerial vehicle and the ground console is realized, and a complete flame real-time monitoring scheme is formed. According to the invention, no matter how the weather condition of the day is, the flame phenomenon at each position can be quickly and accurately found and accurately positioned, and accurate detection and alarm are realized. The research result not only has a wide application prospect in theory, but also has a practical value.

Description

Technical Field

[0001] The invention relates to the technical field of fire detection, in particular to a forest fire detection method. Background Art

[0002] As an important natural resource, forests are vital to our survival. The earth is rich in forest resources. As of 2023, the area is as high as more than 3.8 billion hectares, and the global average forest coverage rate is close to 31.7%. Unfortunately, tens of thousands of hectares of forests in my country and other countries around the world are destroyed by fires every year. The loss of resources caused by tree felling, forest fires and even natural disasters is difficult to estimate. Countries around the world have suffered heavy losses after the fire, so it is particularly important to develop a complete and reliable forest fire prevention and fire extinguishing system. With the continuous advancement of science and technology, drones have played an important role in forest fire prevention, but their intelligence level is low and their functions are relatively single, making it difficult to simultaneously realize the functions of inspection and fire extinguishing.

[0003] Prior art 1, CN201811573497.0 A patrol robot and its fire extinguishing equipment and fire extinguishing method, the fire extinguishing equipment includes: a mounting frame and a fire extinguisher and a telescopic mechanism all arranged on the mounting frame, the fire extinguisher is provided with a push switch at one end of the telescopic mechanism, the telescopic mechanism can extend toward the push switch and touch the push switch, and the fire extinguisher is turned on when the push switch is touched. Using the telescopic mechanism to open the fire extinguisher can enhance the forest as one of the most important natural resources on the earth, which is of great significance to human survival and development. According to statistics, as of 2023, the total amount of forest resources worldwide has exceeded an astonishing 3.8 billion hectares, and the global average forest coverage rate has been close to 31.7%. However, it is regrettable that every year, whether in my country or other countries in the world, a large number of forests are severely damaged by fires, and the resource losses caused by tree felling and forest fires are immeasurable. In order to meet this severe challenge, countries around the world have increased their investment and are committed to developing a complete and reliable forest fire prevention and fire extinguishing system. With the rapid development of science and technology, drones are playing an increasingly important role in the field of forest fire prevention, but their intelligence level still needs to be improved, and their functions are relatively single, making it difficult to take into account both inspection and firefighting tasks at the same time.

[0004] First, a patrol robot and its fire extinguishing equipment and fire extinguishing method disclosed in patent application No. CN201811573497.0. The fire extinguishing equipment is mainly composed of a mounting frame, a fire extinguisher and a telescopic mechanism respectively arranged on the mounting frame, wherein a push switch is provided at one end of the fire extinguisher facing the telescopic mechanism, and the telescopic mechanism can extend toward the push switch when necessary, thereby triggering the push switch, and then turning on the fire extinguisher. Using a telescopic mechanism to start the fire extinguisher will undoubtedly help to improve the mechanization level of the fire extinguishing equipment and further realize the automatic control of the fire extinguishing equipment. Nevertheless, after such fire extinguishing equipment is mounted on the patrol robot, although the patrol robot can take immediate action to extinguish the fire when a fire is found, effectively preventing the spread of the fire, it is not suitable for forest fire prevention scenarios. In addition, since the robot can only move on the ground, its inspection range is limited and the inspection efficiency is low.

[0005] Secondly, a forest fire rescue and fire extinguishing device disclosed in patent application No. CN115300828A. The device mainly includes a monitoring station, multiple rescue drones, satellites, a fire handling platform, a forest fire center and a data transmission module. There is also a charging station specially for charging rescue drones between two adjacent monitoring stations. When the rescue drone, fire handling platform or satellite detects a fire, the relevant information will be transmitted to the forest fire center through the data transmission module. Although the device uses a variety of means such as monitoring stations, remote inspections of rescue drones, and satellite monitoring to carry out forest fire prevention and rescue work, and through the improvement and innovation of the charging floor, the rescue drone can automatically complete the plug-in and power-off operations at the charging floor, the operation is simple and easy, and the unmanned operation of the whole process is realized, which is particularly suitable for forest areas and the cost is relatively low. However, the device only realizes the functions of inspection and fire detection, lacks fire extinguishing capabilities, and has relatively single functions, so the efficiency in forest fire prevention is still low.

[0006] In summary, the existing technical solutions 1 and 2 have the problems of single function and low efficiency in fire source detection in terms of forest fire prevention intelligent equipment, and cannot give full play to their due forest fire prevention role. Therefore, the present invention aims to provide a method for realizing high-precision inspection of unmanned aerial vehicles in forest fire prevention, and realize large-scale inspection of forests through unmanned aerial vehicles to minimize the possibility of fire. Summary of the invention

[0007] In order to solve the problems of the prior art, the present invention provides a forest fire detection method, which can quickly and accurately find and locate each flame phenomenon regardless of the weather conditions of the day (such as sunny day, cloudy day, rainy and snowy weather, etc.), and realize accurate detection and alarm. This research result not only has broad application prospects in theory, but also has practical value.

[0008] The present invention provides a forest fire detection method, comprising the following steps: 1) The drone path planning is based on the electronic geographic information map of the planned inspection area collected by the drone, combined with the drone's attitude data and the estimated time required during the flight, to build the core algorithm of the drone flight channel identification module, and plan and set the corresponding inspection route; The specific process of building the core algorithm of the UAV flight channel identification module is as follows: 1.1) Determine the current coordinate point of the aircraft, that is, the overall center point A of the image; 1.2) We need to find the center coordinates of the candidate box. According to the center coordinates cx and cy of the candidate box returned by the target detection algorithm, we pass these values ​​to point C. 1.3) Calculate the coordinates of the control points to obtain a set of points F. Connect these points with straight lines to obtain the flight trajectory of the drone to a candidate target area. 2) Flame detection: Use drones to take aerial photos of forest operations. The drones cover the entire forest area. The ground control console uses the target detection algorithm to accurately detect all the images taken by the drone camera, thereby achieving close cooperation between the drone and the ground control console to form a complete real-time flame monitoring solution. The specific process is as follows: 2.1) Solve and train the 3D image registration parameters based on the flight data to obtain a relatively stable set of geo-registration parameters; 2.2) Obtain the latitude and longitude information corresponding to the fire occurrence image through time difference and optimize it using georeferencing parameters; 2.3) Using the UAV flight position information and the geometric relationship between the shooting angle and the ground target area, coordinate calculation is performed to obtain the location and range of the forest fire image area; 2.4) Generate a point shp file through the ARCGIS component function and load it into the 3D scene to visually display the location and scope of the forest fire in the 3D scene, achieve accurate positioning of the forest fire, and report the information to relevant supervisors in a timely manner.

[0009] Further improvement, the specific process of calculating the control point coordinates in step 1.3) is as follows: 1.31) Determine whether the target frame is on the left or right side of the drone. If it is on the left side, the horizontal coordinate of the one-quarter point of the line AC connecting the center point of the image and the target candidate frame is used as the horizontal coordinate of the control point B, and the vertical coordinate of B is the vertical coordinate of the midpoint of the AC line; if it is on the right side, the horizontal coordinate of the one-third point of the line AC connecting the center point of the image and the target candidate frame is used as the horizontal coordinate of the control point B, and the vertical coordinate of B is the vertical coordinate of the midpoint of the AC line; 1.32) Find point D on the line from the aircraft coordinate point A to the control point according to a certain ratio, and then find another coordinate E on the line connecting the control point B and the center of the candidate box C according to the same ratio; find point F on the line connecting point D and point E according to the same fixed ratio and store it; 1.33) Repeat the above steps until the ratio becomes 1, and finally obtain a set of points F.

[0010] Further improvement, step 1.32) the ratio is adjusted according to the camera resolution, and the ratio range is 0-1. Further improvement: in step 1.3), since there may be multiple candidate boxes, it is necessary to consider the path decision problem and return a path with the smallest Euclidean distance between the current position and the target center point as the current optimal return value.

[0011] Further improvement, step 2) the flame detection includes a processing method for image sequences and a processing method for video streams.

[0012] Further improvement, in step 2), if the flame detection process identifies an area suspected of fireworks, the event is marked as 1, and the corresponding image is output, and the forest fire calibration algorithm is processed at the same time; if no area suspected of fireworks is identified, the event is marked as 0, and the Continue function is executed to continue subsequent detection work.

[0013] The beneficial effects of the present invention are: 1. Regardless of the weather conditions of the day (such as sunny days, cloudy days, rainy and snowy days, etc.), each flame phenomenon can be quickly and accurately discovered and precisely located, and accurate detection and alarm can be achieved. This research result not only has broad application prospects in theory, but also has practical value.

[0014] 2. The efficiency of the UAV group's dynamic path planning has been greatly improved, making the flight route of the UAV group extremely smooth. This improved algorithm can accurately and efficiently plan safe routes for UAVs to multiple static or dynamic targets, meeting the effectiveness requirements of the aircraft quickly reaching the vicinity of the fire point, while also ensuring the safety of UAVs in complex environments. DETAILED DESCRIPTION

[0015] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] This technological innovation mainly includes a unique monocular depth measurement system, an efficient and accurate flight channel detection system, and an advanced and applicable path optimization strategy. Based on the electronic geographic information map of the planned inspection area collected by the drone, combined with the estimated time required during the drone flight, the corresponding inspection route is planned and set to ensure the smooth progress of the inspection work. At the same time, real-time flame monitoring uses drones to conduct aerial photography of forest operations. The drone photography covers the entire forest area, and the ground control console uses the target detection algorithm to accurately detect all the images taken by the drone camera, thereby achieving close cooperation between the drone and the ground control console to form a complete real-time flame monitoring solution.

[0017] The design of the flame monitoring system covers two key links: drone path planning and flame detection algorithm. First, we need to calculate the projection range of the drone camera sensor's field of view on the ground, and then conduct in-depth theoretical analysis and simulation verification of the two widely used algorithms, compare their advantages and disadvantages, and finally select the most suitable full-coverage path planning algorithm for drone inspection. On this basis, we also need to use monitoring drones to conduct patrol path generation simulation experiments to further verify the effectiveness of the algorithm. In addition, we also need to create a data set to train the flame detection network model, and finally use this model to detect flame targets in offline and real-time videos, and conduct experimental analysis. When the video sent back by the monitoring drone is detected by the YOLOV5 algorithm, the ground control console can calculate the specific location where the fire point may exist.

[0018] According to the experimental requirements, we determined two processing methods for the forest fire recognition algorithm, one for the image sequence and the other for the video stream. The processing speed of these two processing methods is closely related to the number of pre-processed image frames. Therefore, if the amount of data is small, the processing speed will be significantly improved, and may even reach several times or higher. After completing the algorithm processing, if an area suspected of fireworks is identified, the event is marked as 1, and the corresponding image is output, and the forest fire calibration algorithm is processed at the same time. Conversely, if no area suspected of fireworks is identified, the event is marked as 0, and the Continue function is executed to continue the subsequent detection work.

[0019] In the process of preparing flight data information, we need to solve and train the three-dimensional image registration parameters based on the flight data to obtain a set of relatively stable geo-registration parameters. Then, we can obtain the latitude and longitude information corresponding to the fire image through the time difference and optimize it using the geo-registration parameters. Next, we can use the geometric relationship between the flight position information and shooting angle of the drone and the ground target area to perform coordinate calculations to obtain the regional location and range of the forest fire image. Finally, we can generate a point shp file through the ARCGIS component function and load it into the three-dimensional scene to intuitively display the location and range of the forest fire in the three-dimensional scene and realize the precise positioning of the forest fire. At the same time, we will also report these important results to the relevant supervisors in a timely manner.

[0020] When using drones to extinguish forest fires, we need to quickly plan the shortest path for the drone from the starting point to the target location of the fire. However, due to the complex distribution of the forest environment, especially for mountain forest crops, the height difference is large, making it difficult for us to construct complete three-dimensional map information. Therefore, during the flight of the drone, we must avoid static obstacles such as trees and dynamic obstacles such as birds in real time to ensure flight safety.

[0021] The proposed firefighting drone path planning solution uses a network technology based on single-stage end-to-end recognition to build the core algorithm of the drone flight channel recognition module. In the algorithm design process, we first preprocessed and fitted the model to evaluate the module's ability to finely identify the flight channel. Then, we conducted in-depth research and analysis on the rationality of the model's flight path planning in the world coordinate system. Under the premise of ensuring that the accuracy and recognition speed are not reduced, we try our best to ensure the operating efficiency of the entire system. Specifically, the flight channel recognition box output by the algorithm represents the area where the drone can safely avoid obstacles in the current depth map environment. Once the container box information is successfully obtained, the flight path can be accurately depicted. The flight channel recognition system mainly consists of three functional blocks: first, obtaining the drone's attitude data; second, identifying the flight channel; and finally, planning the walking route.

[0022] In terms of path optimization, we use Bezier curve fitting to depict the flight trajectory. First, we need to determine the current coordinate point of the aircraft, that is, the overall center point of the image. This point represents the actual position of the aircraft in the camera coordinate system, which we call point A. Next, we need to find the center coordinates of the candidate box. According to the coordinates cx, cy of the center point of the candidate box returned by the target detection algorithm, we pass these values ​​to point C. Next, we need to calculate the coordinates of the control point. The specific algorithm steps are as follows: First, we need to determine whether the target box is on the left or right side of the drone. If it is on the left, then we take the horizontal coordinate of the one-quarter point of the line AC connecting the center point of the image and the target candidate box as the horizontal coordinate of the control point B, and the vertical coordinate of B takes the vertical coordinate of the midpoint of the AC line. Conversely, if it is on the right, then we take the horizontal coordinate of the one-quarter point of the line AC connecting the center point of the image and the target candidate box as the horizontal coordinate of the control point B, and the vertical coordinate of B takes the vertical coordinate of the midpoint of the AC line. Then, we find point B D from the aircraft coordinate point (that is, the image center point A) to the control point line according to a certain ratio (the ratio varies from 0 to 1), which can be adjusted according to the camera resolution, and then find another coordinate E on the line connecting control point B and the center of the candidate box C according to the same ratio. Next, we still find point F on the line connecting point D and point E according to the same fixed ratio and store it. Repeat the above steps until the ratio becomes 1. In the end, we will get a set of points F. Connecting these points with straight lines can get the flight trajectory of the drone to a candidate target area. Since there may be multiple candidate boxes, we need to consider the problem of path decision. We will return a path with the smallest Euclidean distance between the current position and the target center point as the current optimal return value.

[0023] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, the above is only a preferred embodiment of the present invention. Since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited to this. Any technician familiar with the technical field is within the technical scope disclosed by the present invention. For ordinary technicians in the technical field, changes or replacements that can be easily thought of should be covered within the protection scope of the present invention without departing from the principle of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A forest fire detection method, characterized in that it comprises the following steps: 1) The drone path planning is based on the electronic geographic information map of the planned inspection area collected by the drone, combined with the drone's attitude data and the estimated time required during the flight, to build the core algorithm of the drone flight channel identification module, and plan and set the corresponding inspection route; The specific process of building the core algorithm of the UAV flight channel identification module is as follows: 1.1) Determine the current coordinate point of the aircraft, that is, the overall center point A of the image; 1.2) We need to find the center coordinates of the candidate box. According to the center coordinates cx and cy of the candidate box returned by the target detection algorithm, we pass these values ​​to point C. 1.3) Calculate the coordinates of the control points to obtain a set of points F. Connect these points with straight lines to obtain the flight trajectory of the drone to a candidate target area. 2) Flame detection: Use drones to take aerial photos of forest operations. The drones cover the entire forest area. The ground control console uses the target detection algorithm to accurately detect all the images taken by the drone camera, thereby achieving close cooperation between the drone and the ground control console to form a complete real-time flame monitoring solution. The specific process is as follows: 2.1) Solve and train the 3D image registration parameters based on the flight data to obtain a relatively stable set of geo-registration parameters; 2.2) Obtain the latitude and longitude information corresponding to the fire occurrence image through time difference and optimize it using georeferencing parameters; 2.3) Using the UAV flight position information and the geometric relationship between the shooting angle and the ground target area, coordinate calculation is performed to obtain the location and range of the forest fire image area; 2.4) Generate a point shp file through the ARCGIS component function and load it into the 3D scene to visually display the location and scope of the forest fire in the 3D scene, achieve accurate positioning of the forest fire, and report the information to relevant supervisors in a timely manner.

2. The forest fire detection method according to claim 1, characterized in that: The specific process of calculating the control point coordinates in step 1.3) is as follows: 1.31) Determine whether the target frame is on the left or right side of the drone. If it is on the left side, the horizontal coordinate of the one-quarter point of the line AC connecting the center point of the image and the target candidate frame is used as the horizontal coordinate of the control point B, and the vertical coordinate of B is the vertical coordinate of the midpoint of the AC line; if it is on the right side, the horizontal coordinate of the one-third point of the line AC connecting the center point of the image and the target candidate frame is used as the horizontal coordinate of the control point B, and the vertical coordinate of B is the vertical coordinate of the midpoint of the AC line; 1.32) Find point D on the line from the aircraft coordinate point A to the control point according to a certain ratio, and then find another coordinate E on the line connecting the control point B and the center of the candidate box C according to the same ratio; find point F on the line connecting point D and point E according to the same fixed ratio and store it; 1.33) Repeat the above steps until the ratio becomes 1, and finally obtain a set of points F.

3. The forest fire detection method according to claim 2, characterized in that: Step 1.32) The ratio is adjusted according to the camera's resolution, and the ratio ranges from 0 to 1.

4. The forest fire detection method according to claim 2, characterized in that: In step 1.3), since there may be multiple candidate boxes, it is necessary to consider the path decision problem and return a path with the smallest Euclidean distance between the current position and the target center point as the current optimal return value.

5. The forest fire detection method according to claim 1, characterized in that: Step 2) The flame detection includes a processing method for image sequences and a processing method for video streams.

6. The forest fire detection method according to claim 1 or 5, characterized in that: Step 2) If the flame detection process identifies an area suspected of fireworks, the event is marked as 1, and the corresponding image is output, and the forest fire calibration algorithm is processed at the same time; If no suspected fireworks area is identified, the event is marked as 0, and the Continue function is executed to continue subsequent detection work.