Forest fire prevention inspection and disposal method, device and system based on unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicle inspection, and provides an unmanned aerial vehicle-based forest fire prevention inspection and disposal method, device and system. When an inspection unmanned aerial vehicle equipped with a multi-modal sensor is deployed to cruise according to a preset route, a visible light camera of the inspection unmanned aerial vehicle continuously collects forest region videos and transmits the videos to edge computing equipment of an inspection airport; fire hazards are analyzed in real time through a target recognition algorithm, and when a fire source is detected, position information is reported to a background terminal immediately; the system automatically dispatches the nearest fire extinguishing airport to dispatch the fire extinguishing unmanned aerial vehicle, and secondary accurate positioning is performed on a fire source by using the airborne infrared vision module, so that accurate throwing of the fire extinguishing bomb is ensured. The whole system enhances the real-time analysis capability through the edge computing device, breaks through the single-machine computing power limitation, also realizes the full-process automation from fire behavior recognition to fire extinguishing treatment, and significantly improves the response efficiency and treatment safety of forest fire prevention.

Description

Technical Field

[0001] The present application relates to the field of drone inspection technology, and more specifically, to a method, device, and system for forest fire inspection and disposal based on drones. Background Art

[0002] The contents of this section merely provide background information related to this application and may not constitute prior art.

[0003] Forest resources are fundamental to maintaining ecological balance, regulating climate, and safeguarding biodiversity. They play a vital role in preventing wind and sandstorms, conserving water resources, and other areas. However, forest fires are sudden, spread rapidly, and are difficult to rescue, often causing severe ecological damage and economic losses. Traditional patrols in remote mountain forests are particularly difficult due to terrain and transportation constraints, making it difficult to detect hidden dangers in a timely manner. Manual inspections also pose problems such as high costs and slow response times.

[0004] Existing forest fire prevention technologies primarily focus on fire point identification and localized smoke detection, lacking a comprehensive, systematic, and coordinated patrol mechanism around the clock and often relying on manual operations. While some drones have incorporated edge computing to enable local data processing and improve early warning efficiency, they generally only have patrol and early warning functions and lack autonomous firefighting capabilities. Fire response still relies on manual dispatch, resulting in low efficiency and high safety risks. Single-unit drone deployments are limited by their endurance and processing power, making them difficult to sustain and efficiently handle tasks. Furthermore, the system lacks a closed-loop design for sensing, responding, and handling, preventing rapid coordinated response to fires. For example, Chinese patent publication No. CN119625955A discloses a forest fire prevention drone patrol and early warning system and method based on edge computing. While this forest early warning system, built on drone edge computing and utilizing multi-drone collaborative charging and task coordination, still requires manual intervention, making it difficult to achieve true unmanned operation in remote or complex forest areas. The continuity of monitoring and system reliability require further improvement. Summary of the Invention

[0005] To address the above technical issues, the purpose of this application is to provide a forest fire inspection and disposal method, device, and system based on drones. This method uses inspection drones to identify fire sources, and then uses firefighting drones to extinguish fires, reducing manual reliance. At the same time, a hangar equipped with edge computing equipment is deployed for inspection drones and firefighting drones, breaking away from the energy consumption limitations of single drones, significantly enhancing computing power, supporting more complex fire identification algorithms and real-time data analysis, and improving processing efficiency and response speed.

[0006] The purpose of this application is achieved through the following technical solutions:

[0007] In a first aspect, the present invention provides a forest fire inspection and disposal method based on drones, comprising a patrol drone with a multimodal sensor, a patrol airport wirelessly connected to the patrol drone, a fire-fighting drone, a fire-fighting airport wirelessly connected to the fire-fighting drone, and a background terminal wirelessly connected to the patrol airport and the fire-fighting airport, respectively. The fire-fighting drone carries fire-fighting bombs that can be automatically released; the patrol airport and the fire-fighting airport are both equipped with corresponding edge computing devices. The method comprises:

[0008] Start the inspection airport and conduct a power-on self-test; after receiving confirmation that the inspection airport has no problems, set the route according to the task requirements through the background terminal and generate and issue inspection instructions;

[0009] In response to the inspection command, the inspection drone takes off from the inspection airport according to the route and continuously collects video data of the forest area through the visible light camera and transmits it to the inspection airport;

[0010] The edge computing equipment at the inspection airport uses the YOLO algorithm to identify the fire source target for each frame of the forest area video data and calculates the first confidence level of the corresponding frame. If the first confidence level is greater than or equal to the threshold, the positioning device of the inspection drone obtains the location information of the fire source in the corresponding frame, generates reporting information based on the location data, image data, and time data of the fire source in the corresponding frame, and reports it to the background terminal. The inspection drone then returns.

[0011] In response to the reported information, the backend terminal selects available firefighting airports and calculates the distances between the fire source and different firefighting airports based on the location data. The nearest firefighting airport is selected as the target airport and the fire source location information and firefighting instructions are sent to the target airport.

[0012] In response to the fire-fighting command, the fire-fighting drone located at the target airport flies to the target area according to the location information of the fire source, activates the onboard infrared vision module installed vertically downward on the fire-fighting drone, and performs real-time identification and positioning detection of the fire source again based on the temperature and pixel coordinates of the thermal imaging, and determines whether the current target is the fire source and whether it is directly under the fire-extinguishing bomb; if the detection result is yes, the fire-fighting bomb is dropped; if the detection result shows a deviation, the position of the fire-fighting drone is calibrated until the fire source is directly under the fire-extinguishing bomb, and then the fire-extinguishing bomb is dropped.

[0013] Furthermore, after the fire source target is identified by the Yolo algorithm and the first confidence of the corresponding frame is calculated, the following is also included:

[0014] If the first confidence level is less than the threshold, the infrared camera on the inspection drone is activated;

[0015] A fire source detection model for infrared cameras is trained within edge computing devices at airport inspection sites using the lightweight YOLO framework. This model identifies high-temperature targets by analyzing areas of abnormal temperature and calculates their infrared confidence. The confidence levels of visible light and infrared images are then combined to produce a comprehensive confidence score.

[0016] If the comprehensive confidence is greater than the set value, the location information of the fire source in the corresponding frame is obtained through the positioning device of the inspection drone, and the location data, image data and time data of the fire source in the corresponding frame are generated and reported to the background terminal; otherwise, it is judged as a false alarm and the current frame is discarded.

[0017] Furthermore, the steps for the inspection drone to return to the base include:

[0018] After completing the current inspection mission, the inspection drone will return to the corresponding inspection hangar according to the original flight route; after receiving the signal that the inspection drone has docked, the inspection hangar will automatically charge the inspection drone until the power level exceeds the preset power threshold.

[0019] Furthermore, the step of obtaining the location information of the fire source corresponding to the frame specifically includes:

[0020] The visible light camera is fixedly mounted on the bottom of the drone via a three-axis gimbal. The coordinate system directions of the visible light camera, the three-axis gimbal, and the drone are consistent.

[0021] Based on the focal length of the visible light camera and the position of the fire source in the corresponding frame, the pixel coordinates corresponding to the fire source position are converted into a direction vector in the camera coordinate system;

[0022] Define the three-axis rotation matrix by the right-hand coordinate system, transform the direction vector in the camera coordinate system to the body coordinate system, and then combine the current attitude angle data of the drone to transform it to the world coordinate system based on the geographic orientation to obtain the direction vector in the world coordinate system;

[0023] Obtain the takeoff position and current position of the inspection drone, convert the takeoff position and current position into spatial coordinates in the Earth-centered Earth-fixed coordinate system and calculate the difference vector; construct a rotation matrix from the Earth-centered Earth-fixed coordinate system to the world coordinate system with the takeoff position as a reference; obtain the current coordinates of the inspection drone's current position in the world coordinate system based on the difference vector and the rotation matrix; construct a spatial ray from the current coordinate along the direction vector in the world coordinate system, solve the coordinates of the intersection of the ray and the ground plane, and obtain the fire source coordinates in the world coordinate system;

[0024] Convert the fire source coordinates to latitude and longitude.

[0025] Furthermore, the step of calibrating the position of the fire-fighting drone until the fire source is directly below the fire-fighting bomb specifically includes:

[0026] Determine the center position of the image captured by the firefighting drone's downward-looking camera. The center position is the first midpoint coordinate corresponding to the width and height of the image.

[0027] Extracting the second center coordinate of the detection frame of the fire source in the image;

[0028] Calculate the pixel offset between the first midpoint coordinate and the second center coordinate;

[0029] Based on the current flight altitude of the firefighting drone and the camera focal length parameters, the pixel offset is converted into a viewing angle offset. The horizontal and vertical ground distance deviations between the actual location of the fire source and the theoretical location of the fire bomb release are calculated based on the geometric relationship between the viewing angle offset and the flight altitude.

[0030] According to the ground distance deviation, the fire-fighting drone is controlled to move horizontally until the ground distance deviation is less than the preset deviation.

[0031] Furthermore, after controlling the firefighting drone to move horizontally until the ground distance deviation is less than the preset deviation, the method further includes:

[0032] The laser ranging module on the fire-fighting drone measures the vertical height between the drone and the fire source in real time;

[0033] The edge computing device connected to the firefighting drone compares the measured vertical height with the preset safe release height threshold for fire extinguishing bombs. If the safe release height threshold is not reached, the firefighting drone is controlled to continue climbing. If the safe release height threshold is reached or exceeded, the drone is controlled to hover and maintain alignment with the fire source.

[0034] After confirming that the fire source is aligned and the height meets the standard, the fire extinguishing bomb delivery mechanism is triggered to release the fire extinguishing bomb;

[0035] After the fire is extinguished, the scene result image is captured and sent back to the backend terminal.

[0036] Furthermore, after capturing the on-site result image and transmitting it back to the backend terminal, the following steps are also included:

[0037] The fire-fighting drone returns to the corresponding fire-fighting airport according to the departure route. After receiving the signal that the fire-fighting drone has docked, the fire-fighting hangar automatically charges the fire-fighting drone and replenishes fire-fighting ammunition until the power and fire-fighting ammunition levels are greater than the preset corresponding thresholds.

[0038] Furthermore, it also includes:

[0039] An interactive command interface is built based on a map engine. The command interface uses a preset resolution map slice of the forest area to be monitored as a spatial basemap, loads static layers of topography, inspection hangars, and fire hangar coordinates, and synchronizes the status information of inspection drones, inspection airports, firefighting drones, and firefighting airports in real time.

[0040] After receiving the report information from the inspection drone or firefighting drone, it automatically parses its image, longitude and latitude, and timestamp, and dynamically annotates it on the command interface through the map annotation API;

[0041] At the operational level, map zooming and layer management are achieved by loading different resolutions, and device status query and historical task retracing are performed by setting up query windows.

[0042] In the second aspect, the present invention provides a forest fire prevention inspection and disposal device based on a drone, including a patrol drone with a multimodal sensor, an inspection airport wirelessly connected to the patrol drone, a fire-fighting drone, a fire-fighting airport wirelessly connected to the fire-fighting drone, and a background terminal wirelessly connected to the patrol airport and the fire-fighting airport respectively. The fire-fighting drone carries fire-fighting bombs that can be automatically released; the patrol airport and the fire-fighting airport are both equipped with corresponding edge computing devices; computer programs are pre-stored on the edge computing device and the background terminal, and when the edge computing device and the background terminal execute the computer program, the steps corresponding to the method in the first aspect are implemented.

[0043] In a third aspect, the present invention provides a forest fire inspection and disposal system based on a drone, comprising:

[0044] The power-on self-test module is used to start the inspection airport and perform a power-on self-test. After receiving the confirmation that the inspection airport has no problems, the background terminal sets the route according to the task requirements and generates and issues inspection instructions.

[0045] The visible light image acquisition module is used for the inspection drone to take off from the inspection airport according to the route and continuously collect forest area video data through the visible light camera and transmit it to the inspection airport;

[0046] The fire source identification module is used by the edge computing equipment of the inspection airport to identify the fire source target for each frame of forest video data using the YOLO algorithm and calculate the first confidence level of the corresponding frame. If the first confidence level is greater than or equal to the threshold, the positioning device of the inspection drone obtains the location information of the fire source in the corresponding frame, generates reporting information based on the location data, image data, and time data of the fire source in the corresponding frame, and reports it to the background terminal.

[0047] The fire-fighting airport screening module is used by the backend terminal to screen out available fire-fighting airports, calculate the distance between the fire source location and different fire-fighting airports based on the location data, select the nearest fire-fighting airport as the target airport, and send the fire source location information and fire-fighting instructions to the target airport;

[0048] The fire extinguishing module is used for flying the fire extinguishing unmanned aerial vehicle located at the target airport to the target area according to the position information of the fire source, enabling the vertically downward installed on-board infrared vision module of the fire extinguishing unmanned aerial vehicle, and performing real-time identification and positioning detection on the fire source again according to the temperature and pixel coordinates of the thermal imaging, and judging whether the current target is the fire source and whether it is located directly below the fire extinguishing bomb; if the detection result is yes, the fire extinguishing bomb is launched; if the detection result shows that there is deviation, the position of the fire extinguishing unmanned aerial vehicle is calibrated until the fire source is located directly below the fire extinguishing bomb, and then the fire extinguishing bomb is launched.

[0049] In summary, the technical scheme of the embodiment of the application has at least the following advantages and beneficial effects:

[0050] By deploying the inspection unmanned aerial vehicle equipped with multi-modal sensors and the fire extinguishing unmanned aerial vehicle system carrying fire extinguishing bombs, and combining the supporting inspection airport and fire extinguishing airport, the intelligent prevention and control of forest fires is realized. When the inspection unmanned aerial vehicle cruises according to the preset route, the visible light camera continuously collects the forest area video and transmits it to the edge computing device of the inspection airport, and the target recognition algorithm is used to analyze the fire hazard in real time, and when the fire source is detected, the position information is immediately reported to the background terminal; the system automatically dispatches the nearest fire extinguishing airport to send the fire extinguishing unmanned aerial vehicle, and uses the on-board infrared vision module to accurately position the fire source again, to ensure accurate launching of the fire extinguishing bomb. The whole system enhances the real-time analysis capability through the edge computing device, which not only breaks through the single machine power limit, but also realizes the full-process automation from fire identification to fire disposal, significantly improving the response efficiency and disposal safety of forest fire prevention. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 A flowchart of a forest fire prevention inspection and disposal method based on an unmanned aerial vehicle is provided in the present application.

[0052] Figure 2 An architecture diagram of the unmanned aerial vehicle forest fire prevention inspection and fire extinguishing in the present application

[0053] Figure 3 A pixel-to-longitude and latitude mapping process diagram in the present application;

[0054] Figure 4 A fire extinguishing logic block diagram of the fire extinguishing unmanned aerial vehicle in the present application;

[0055] Figure 5 A structure schematic diagram of a forest fire prevention inspection and disposal system based on an unmanned aerial vehicle provided in the present application. DETAILED DESCRIPTION

[0056] To make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in a clear and complete manner with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0057] The method for forest fire prevention and inspection and disposal based on a UAV proposed in the embodiments of the present application comprises Figure 1 and 2 as shown, including an inspection UAV with a multi-modal sensor, an inspection airport wirelessly connected with the inspection UAV, a fire extinguishing UAV, a fire extinguishing airport wirelessly connected with the fire extinguishing UAV, and a background terminal wirelessly connected with the inspection airport and the fire extinguishing airport respectively, the fire extinguishing UAV carrying an automatically releasable fire extinguishing bomb; the inspection airport and the fire extinguishing airport are each provided with a corresponding edge computing device.

[0058] The inspection UAV is equipped with a dual-modal sensor of a visible light camera and an infrared thermal imager, and the multi-spectral collaborative detection improves the accuracy of fire identification. The visible light camera is used for identifying open fires and smoke, and the infrared camera is used for auxiliary judgment through temperature anomaly detection, and the confidence fusion of the two can reduce the false alarm rate. The airport (including the inspection airport and the fire extinguishing airport) serves as the take-off and landing and operation hub of the UAV, and is internally provided with an automatic charging, bomb reloading device and an edge computing device. The inspection airport receives the video stream returned by the UAV, the fire extinguishing airport dispatches the fire extinguishing UAV to perform tasks, and the two communicate with the background terminal in real time through the MQTT protocol. In addition, the deployment of the distributed airport and the edge computing device can support the rapid response and continuous operation of the UAV. The edge computing device is deployed beside the corresponding hangar using the NVIDIA Jetson Orin Nano, and runs a lightweight YOLO model to realize fire source identification. The background terminal serves as a smart hub system, integrating a situation awareness, intelligent decision-making, and task control module. Fire data is obtained by subscribing to an MQTT topic, and a dispatch instruction is dynamically generated.

[0059] The method specifically comprises:

[0060] S101, starting the inspection airport and performing a self-check; after receiving that the self-check of the inspection airport has no problem, setting a flight route and generating and issuing an inspection instruction through the background terminal according to task requirements.

[0061] Specifically, as the ground hub for the drone system, the inspection airport's power-on self-test is the primary step in ensuring the reliable operation of the entire system. This self-test includes testing core modules such as the hangar's communication link status, the energy storage device's power level, the automatic charging device's functionality, and the operating status of the edge computing device. The deployed hangar must verify parameters such as the MQTT connection status with the backend terminal and the GPU load rate of the edge computing device to ensure that subsequent task scheduling instructions can be received and executed normally. After passing the power-on self-test, the hangar enters standby mode, providing physical support and energy supply for drone takeoff and landing. Its automated nature significantly reduces the need for manual intervention.

[0062] S102, in response to the inspection instruction, the inspection drone takes off from the inspection airport according to the route, and continuously collects forest area video data through the visible light camera and transmits it to the inspection airport.

[0063] Specifically, inspection drones transmit video streams via dedicated communication channels to the inspection airport. The airport acts as a gateway, forwarding the data to edge computing devices on the same local area network. These edge computing devices utilize the NVIDIA Jetson Orin Nano platform and deploy a fire source identification model trained using the lightweight YOLO framework, enabling real-time processing of video frames. For example, during one inspection, a drone transmitted a video stream containing an early fire. The edge device identified the target and triggered an alert within 0.5 seconds. This distributed edge computing architecture overcomes the computing power limitations of traditional single machines, ensuring the stable operation of complex algorithms in field environments.

[0064] In addition, when the inspection drone flies along a preset route, it has the ability to fine-tune its path based on environmental information. Specifically, when a sudden airflow or obstacle appears on the flight path, the drone senses it in real time through onboard sensors (for example, radar detects obstacles, and detects abnormal posture by angular velocity changes to determine whether it has encountered a sudden airflow) and automatically corrects its track. For example, during an inspection in a mountainous area, the drone once deviated from its route by 5 meters due to a sudden strong wind. The system immediately activated a dynamic path planning algorithm to return it to a safe route. This autonomy ensures the continuity and safety of inspection tasks.

[0065] S103, the edge computing equipment of the inspection airport uses the Yolo algorithm to identify the fire source target for each frame of the forest area video data and calculates the first confidence of the corresponding frame; if the first confidence is greater than or equal to the threshold, the positioning device of the inspection drone obtains the location information of the fire source of the corresponding frame, and generates reporting information for the location data, image data and time data of the fire source of the corresponding frame, and reports it to the background terminal; the inspection drone returns.

[0066] Specifically, after the inspection drone continuously collects video streams through visible light cameras and transmits them to the inspection airport, the airport acts as a gateway to forward the data to the edge computing device in the same local area network. The device uses the NVIDIA Jetson Orin Nano platform and deploys a fire source recognition model trained based on the lightweight YOLO framework to process each frame of visible light images in real time and calculate the first confidence level. For example, when a frame of image detects a suspected fire source and the confidence level reaches a preset threshold (such as 80%), the system immediately triggers the positioning process: the latitude and longitude coordinates of the fire source are obtained through the positioning device carried by the inspection drone, and structured reporting information is generated by combining image data and timestamps, which is pushed to the background terminal via the MQTT protocol. This process reflects the real-time advantage of edge computing. For example, in the embodiment, the edge device can complete target recognition and trigger an early warning within 0.5 seconds, which is significantly faster than the traditional cloud processing mode.

[0067] If the first confidence level falls below the threshold (e.g., the actual value is only 60%), a multimodal collaborative detection mechanism is activated: the inspection drone's infrared camera is activated to collect thermal imaging data, and the edge computing device uses a YOLO model, optimized and trained specifically for infrared data, to analyze temperature anomalies and generate an infrared confidence level. The system then uses a weighted fusion algorithm to comprehensively evaluate the visible light and infrared confidence levels. If the combined confidence level exceeds the set value, the inspection drone's positioning device obtains the location of the fire source for the corresponding frame. This information, along with the fire source's location, image data, and time data, is then reported to the backend terminal. Otherwise, the fire is deemed a false alarm and the current frame is discarded. For example, during one inspection, the visible light confidence level was 65% (due to smoke interference) and the infrared confidence level reached 90% (detecting a high temperature of 300°C). The combined confidence level increased to 85%, triggering a fire report. This design effectively reduces the false alarm rate of a single sensor, as measured data shows.

[0068] After confirming the fire, the inspection drone returns to the inspection hangar according to the preset program. The inspection hangar automatically docks with the charging device until the power is restored to more than 90%, reserving energy for subsequent tasks. The entire process forms a closed loop of "data collection-edge processing-multimodal verification-decision reporting". Its technical advantages are: First, by binding the edge computing equipment to the hangar, it breaks through the computing power limitations of a single drone and supports the stable operation of complex algorithms in the field environment; second, the fusion of dual-modal data improves the accuracy of fire identification and avoids misjudgments caused by light changes or local heat sources; third, the automated return charging mechanism ensures the system's continuous operation capability and realizes unmanned operation and maintenance. A typical application scenario is that during an inspection in a mountainous area, the drone transmits a video stream containing early fire conditions. The edge device completes the location of the fire point and triggers the fire-fighting dispatch within 3 seconds through the above process, which is significantly faster than the response speed of traditional manual inspections.

[0069] Among them, the step of obtaining the location information of the fire source of the corresponding frame is as follows: Figure 3 As shown, specifically including:

[0070] The visible light camera is fixedly mounted on the bottom of the drone via a three-axis gimbal. The coordinate system directions of the visible light camera, the three-axis gimbal, and the drone body are consistent; their definitions are shown in Table 1:

[0071] Coordinate system X-axis Y-axis Z-axis Pixel coordinate system To the right down Camera coordinate system To the right down Forward (camera facing) Gimbal coordinate system To the right down forward Body coordinate system Forward (nose) Right (right arm) Down (ground) World Coordinate System north East land

[0072] Table 1

[0073] Based on the focal length of the visible light camera and the fire source position (u, v) in the corresponding frame, the pixel coordinates corresponding to the fire source position are converted into a direction vector in the camera coordinate system.

[0074] Among them, the camera internal parameters are:

[0075]

[0076] Where, f x 、f y is the focal length of the camera, u0 and v0 are the positions of the fire source.

[0077] Converted to camera coordinate direction vector:

[0078]

[0079] The three-axis rotation matrix, namely the roll matrix, pitch matrix, and yaw matrix, is defined in the right-hand coordinate system. The direction vector in the camera coordinate system is converted to the body coordinate system. Then, combined with the current attitude angle data of the drone, it is converted to the world coordinate system based on the geographic orientation to obtain the direction vector in the world coordinate system.

[0080] Among them, the rolling matrix R x for:

[0081]

[0082] Where r is the roll angle.

[0083] Among them, the pitch matrix R y for:

[0084]

[0085] Where p is the pitch angle.

[0086] Among them, the yaw matrix R z for:

[0087]

[0088] Where y is the yaw angle.

[0089] The gimbal gives two angles, pitch and yaw, to construct the rotation matrix R cam :

[0090] R cam =R z ·R y (6)

[0091] During the flight of the inspection drone, its attitude angles of roll, pitch, and yaw can be obtained to construct the rotation matrix R:

[0092] R=R z ·R y ·R x (7)

[0093] Direction vector in the world coordinate system for:

[0094]

[0095] Obtain the takeoff position and current position of the inspection drone, convert the takeoff position and current position into spatial coordinates in the Earth-centered Earth-fixed coordinate system, and calculate the difference vector.

[0096] Specifically, first the take-off position GPS (φ0, λ0, h0) and the current position GPS (φ d ,λ d ,h d ) into the following formula:

[0097]

[0098] X=(N+h)cosφcosλ (10)

[0099] Y=(N+h)cosφsinλ (11)

[0100] Z=(N(1-e 2 )+h)sinφ (12)

[0101] Get the Earth-centered Earth-fixed (ECEF) coordinates: The takeoff point is The current point is Where, a=6378137.0m, e 2 =6.69437999014×10 -3 .

[0102] Subtract and calculate the difference vector Δ ecef for:

[0103]

[0104] Taking the takeoff position as the reference, construct the rotation matrix from the Earth-centered Earth-fixed coordinate system to the world coordinate system:

[0105]

[0106] Based on the difference vector and rotation matrix, the current coordinates of the inspection drone in the world coordinate system are obtained:

[0107]

[0108] That is, in the world coordinate system, the current position of the inspection drone is P d (x d ,y d ,h d ).

[0109] Construct a space ray from the current coordinate along the direction vector in the world coordinate system, that is, from P0 to Construct space ray:

[0110]

[0111] The height of the ground is z = 0. Find the coordinates of the intersection of the ray and the ground plane:

[0112]

[0113] Get the fire source coordinates in the world coordinate system:

[0114]

[0115] Convert the fire source coordinates to longitude and latitude, that is, convert the world coordinate difference to longitude and latitude:

[0116] The current latitude and longitude of the inspection drone is obtained as (φ0,λ0), and the position of the fire source relative to the inspection drone is (Δx,Δy), where:

[0117] Δx=x t -x d (19)

[0118] Δy=y t -y d (20)

[0119] The latitude and longitude of the fire source are:

[0120]

[0121] S104, in response to the reported information, the background terminal selects idle fire-fighting airports, and calculates the distance between the fire source location and different fire-fighting airports based on the location data, selects the fire-fighting airport closest to the target airport, and sends the fire source location information and fire-fighting instructions to the target airport.

[0122] Specifically, when the backend terminal receives the fire information (including the latitude and longitude coordinates of the fire source, image data, and timestamp) reported by the edge computing device via the MQTT protocol, the intelligent decision-making module first initiates the equipment screening process. This module obtains the operating status data of each fire-fighting airport in real time by subscribing to the corresponding topic, including key indicators such as the hangar space location (latitude and longitude), communication link status (connection status with the hub), hangar idle status (whether it is charging or reloading), the number of fire-fighting bombs mounted on the fire-fighting drone, and the remaining power, thereby forming a dynamically updated list of dispatchable devices. For example, when a fire is reported, the system detects that Firefighting Airport No. 1 is in a charging state (battery level 30%), Firefighting Airport No. 2 is performing other tasks (status is "busy"), and Firefighting Airport No. 3 is on standby (battery level 95% and fully loaded with fire-fighting bombs). At this time, the system automatically excludes nodes No. 1 and 2 and includes Firefighting Airport No. 3 in the candidate set. The spatial distance between the fire source and each candidate firefighting airport is then calculated, with priority given to the node with the closest straight-line distance. For example, in this example, firefighting airport No. 3 is only 2.3 kilometers away from the fire point, which has a significant locational advantage over other candidate nodes (Airport No. 4 is 5.8 kilometers away), and is therefore identified as the target airport. After selection, fields such as the fire source coordinates (such as 112.85° East longitude and 28.18° North latitude), task priority (automatically classified as "emergency" based on the size of the fire), and fire bomb delivery requirements (preset to 2 fire bombs) are encapsulated into a structured task data packet and released to the target airport. This scheduling strategy, based on real-time status monitoring and spatial distance optimization, ensures that firefighting drones arrive at the fire scene in the shortest possible time. In addition, the lightweight communication architecture implemented through the MQTT protocol enables stable transmission of task instructions in weak network environments (minimum 2G network support), adapting to the complex communication conditions in remote forest areas. The entire scheduling process forms an automated closed loop of "state perception - distance calculation - resource matching - instruction generation", which not only avoids the delay error introduced by human intervention, but also ensures the accuracy and traceability of instruction delivery through digital task packages.

[0123] S105, in response to the fire-fighting command, the fire-fighting drone located at the target airport flies to the target area according to the location information of the fire source, activates the airborne infrared vision module installed vertically downward on the fire-fighting drone, and performs real-time identification and positioning detection on the fire source again according to the temperature and pixel coordinates of the thermal imaging, and determines whether the current target is the fire source and whether it is directly under the fire-extinguishing bomb; if the detection result is yes, the fire-fighting bomb is dropped; if the detection result shows that there is a deviation, the position of the fire-fighting drone is calibrated until the fire source is directly under the fire-extinguishing bomb, and then the fire-extinguishing bomb is dropped.

[0124] Specifically, after receiving fire source location information from a backend terminal, the firefighting drone first autonomously flies to the target area. At this point, the onboard infrared vision module (i.e., downward-looking infrared camera) mounted vertically downward initiates a secondary detection process. This module uses thermal imaging to analyze temperature distribution characteristics and, combined with pixel coordinate positioning technology, identifies and verifies the fire source in real time.

[0125] Specifically, if Figure 4 As shown, the system first calculates the coordinates of the image center as the reference point, extracts the actual position of the center of the fire source detection frame in the image, and determines the deviation direction between the fire source and the fire extinguishing bomb delivery center through the pixel offset (if the horizontal offset is positive, it needs to be moved to the right, and if it is negative, it needs to be moved to the left; if the vertical offset is negative, it needs to be moved forward, and if it is positive, it needs to be moved backward). The calculation process is as follows:

[0126] Determine the center position of the image captured by the firefighting drone's downward-looking camera. The center position is the first midpoint coordinate corresponding to the image width and height. If the image size is W×H, the image center is:

[0127]

[0128] Extract the second center coordinates (u, v) of the detection box of the fire source in the image and calculate the pixel offset:

[0129] Δu=uc x (twenty four)

[0130] Δv=vc y (25)

[0131] If Δu>0, it means that the fire-fighting drone moves to the right, otherwise it moves to the left; if Δv<0, it means that the fire-fighting drone moves forward, otherwise it moves backward.

[0132] Based on the current flight altitude of the firefighting drone and the camera focal length parameters, the pixel offset is converted into a viewing angle offset. The horizontal and vertical ground distance deviations between the actual location of the fire source and the theoretical location of the fire bomb release are calculated based on the geometric relationship between the viewing angle offset and the flight altitude:

[0133]

[0134] The fire-fighting drone performs position translation correction based on the calculation results, and iterates repeatedly until the offset between the center of the fire source and the center of the image (i.e., directly below the fire-fighting bomb) is less than the set threshold (e.g., 5 pixels).

[0135] To ensure the safety of the release, the system combines a laser ranging module to monitor the vertical distance between the drone and the ground in real time, that is, it controls the fire-fighting drone to move horizontally until the ground distance deviation is less than the preset deviation. It also includes: using the laser ranging module carried by the fire-fighting drone to measure the vertical height between the drone and the fire source in real time; using the edge computing device bound to the fire-fighting drone, the measured vertical height is compared with the preset fire-fighting bomb safe release height threshold: if the safe release height threshold is not reached, the fire-fighting drone is controlled to continue climbing; if the safe release height threshold is reached or exceeded, it hovers and maintains the state of aiming at the fire source; after confirming that the fire source is aimed and the height meets the standard, the fire-fighting bomb release mechanism is triggered to release the fire-fighting bomb; after the fire is extinguished, the scene result image is captured and transmitted back to the background terminal.

[0136] Specifically, when the preset "safety radius + release altitude" value is reached (for example, a safety radius of 10 meters + a release altitude of 20 meters), the drone hovers and confirms the fire source is located in the center of the image before releasing the fire bomb. If the secondary detection reveals that the target is not the fire source (such as a misidentified target such as a hot rock), the mission is terminated and a report is sent to the backend terminal. A typical example of this process is an actual firefighting operation. The firefighting drone's initial positioning error caused the fire source to deviate by 150 pixels from the image center (approximately 4.5 meters above the ground). After three position calibrations, the error was reduced to within 3 pixels. Laser ranging then confirmed the altitude at 30 meters, allowing the fire bomb to be accurately dropped and extinguished. This design utilizes a triple-security mechanism of "visual positioning - dynamic calibration - altitude verification," which benefits from three key aspects: first, infrared secondary detection effectively avoids ineffective deployments due to misjudgments during the inspection phase; second, pixel-level calibration ensures the fire bomb's landing accuracy is within 1 meter, significantly improving firefighting efficiency; and third, the integration of laser ranging and visual positioning avoids delivery errors caused by complex terrain.

[0137] The laser ranging principle measures the time difference t between the laser emission and the reflected return, and combines this with the speed of light c to calculate the vertical distance d between the drone and the ground fire point in real time:

[0138]

[0139] After capturing the on-site image and transmitting it back to the backend terminal, the firefighting drone returns to the corresponding firefighting airport along its departure route. Upon receiving the docking signal from the firefighting drone, the firefighting hangar automatically charges the drone and replenishes firefighting ammunition until the power level and firefighting ammunition levels exceed the preset thresholds.

[0140] Furthermore, the above method also includes: building an interactive command interface based on a map engine; the command interface uses a preset resolution map slice of the forest area to be monitored as a spatial base map, loads static layers of topography, inspection hangars, and fire-fighting hangar coordinates, and synchronizes the status information of inspection drones, inspection airports, fire-fighting drones, and fire-fighting airports in real time; after receiving the report information from the inspection drone or the fire-fighting drone, automatically parses its image, longitude and latitude and timestamp, and dynamically annotates it on the command interface through the map annotation API; at the operational level, map zooming and layer management are achieved by loading different resolutions, and equipment status query and historical task backtracking are performed by setting a query window.

[0141] Specifically, the system uses high-resolution map slices of the forest area to be monitored as the spatial basemap (such as satellite images with a resolution of 0.5 meters), superimposed with static layers such as terrain elevation data and hangar deployment coordinates (such as the inspection hangar is located at 112.78° east longitude and 28.15° north latitude, and the fire-fighting hangar is located at 112.82° east longitude and 28.12° north latitude) to form a basic spatial framework. At the dynamic data level, the system subscribes to and parses the status information of various devices under the corresponding topic in real time through the MQTT protocol (including the longitude and latitude of the drone, altitude, power and hangar operating status, etc.), and converts it into spatial markers and dynamically renders it to the map interface. For example, in a certain mission, the real-time position of the inspection drone (112.80° east longitude and 28.14° north latitude) and the remaining power (78%) information are displayed synchronously, and the status of the fire-fighting hangar is distinguished by color coding (green for standby, red for mission in progress).

[0142] When the system receives a fire report (e.g., a data packet containing fire coordinates, images, and timestamps transmitted via a corresponding topic), the command interface automatically triggers the annotation process: First, the longitude and latitude fields in the JSON-formatted data are parsed (e.g., 112.85° East, 28.18° North), and the map engine API is called to generate a fire source icon. Next, the image data is associated to generate a thumbnail pop-up window, which can be clicked to view a high-definition fire photo. Finally, a timestamp tag (e.g., "2025-08-06 14:30:22") is added to complete the annotation group. During an actual fire response, after receiving fire point data with a 92% confidence level from an edge computing device, the system dynamically annotated the command interface within 2 seconds and automatically focused on the fire point, assisting command personnel in rapid assessment and judgment.

[0143] In terms of interactive functionality, the system supports multi-level zooming (from a 1:5000 forest panorama to a 1:200 fire point detail view) and layer management (with the ability to individually toggle layers such as inspection routes, fire point history, and device status). Through a dedicated query window, operators can enter a device ID (such as "Drone_003") to retrieve the complete status log for a specific drone, or select a time range to review historical missions (e.g., viewing all firefighting drone dispatch records from August 5th). For example, during a mission review, a commander could select the time range "2025-08-05 09:00 to 12:00" to quickly retrieve the flight trajectory of Firefighting Drone No. 3, a before-and-after comparison of the fire point, and a task time statistics table.

[0144] Based on the same inventive concept, Figure 2 As shown, the present invention provides a forest fire inspection and disposal device based on a drone, including a patrol drone with a multimodal sensor, an inspection airport wirelessly connected to the patrol drone, a fire-fighting drone, a fire-fighting airport wirelessly connected to the fire-fighting drone, and a background terminal wirelessly connected to the patrol airport and the fire-fighting airport respectively. The fire-fighting drone carries fire-fighting bombs that can be automatically released; the patrol airport and the fire-fighting airport are both provided with corresponding edge computing devices; computer programs are pre-stored on the edge computing device and the background terminal, and when the edge computing device and the background terminal execute the computer program, a forest fire inspection and disposal method based on a drone is implemented.

[0145] Based on the same inventive concept, Figure 5 As shown, the present invention provides a forest fire inspection and disposal system based on drones, comprising:

[0146] The power-on self-test module is used to start the inspection airport and perform a power-on self-test. After receiving the confirmation that the inspection airport has no problems, the background terminal sets the route according to the task requirements and generates and issues inspection instructions.

[0147] The visible light image acquisition module is used for the inspection drone to take off from the inspection airport according to the route and continuously collect forest area video data through the visible light camera and transmit it to the inspection airport;

[0148] The fire source identification module is used by the edge computing equipment of the inspection airport to identify the fire source target for each frame of forest video data using the YOLO algorithm and calculate the first confidence level of the corresponding frame. If the first confidence level is greater than or equal to the threshold, the positioning device of the inspection drone obtains the location information of the fire source in the corresponding frame, generates reporting information based on the location data, image data, and time data of the fire source in the corresponding frame, and reports it to the background terminal.

[0149] The fire-fighting airport screening module is used by the backend terminal to screen out available fire-fighting airports, calculate the distance between the fire source location and different fire-fighting airports based on the location data, select the nearest fire-fighting airport as the target airport, and send the fire source location information and fire-fighting instructions to the target airport;

[0150] The fire-fighting module is used for the fire-fighting drone located at the target airport to fly to the target area according to the location information of the fire source, activate the airborne infrared vision module installed vertically downward on the fire-fighting drone, and perform real-time identification and positioning detection of the fire source again based on the temperature and pixel coordinates of the thermal imaging, and determine whether the current target is the fire source and whether it is directly under the fire-extinguishing bomb; if the detection result is yes, the fire-extinguishing bomb is dropped; if the detection result shows a deviation, the position of the fire-fighting drone is calibrated until the fire source is directly under the fire-extinguishing bomb, and then the fire-extinguishing bomb is dropped.

[0151] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A forest fire inspection and disposal method based on drones, characterized in that: The system comprises an inspection drone with a multimodal sensor, an inspection airport wirelessly connected to the inspection drone, a fire-fighting drone, a fire-fighting airport wirelessly connected to the fire-fighting drone, and backend terminals wirelessly connected to the inspection airport and the fire-fighting airport respectively. The fire-fighting drone carries a fire-fighting bomb that can be automatically released. The inspection airport and the fire fighting airport are both provided with corresponding edge computing devices, and the method includes: Start the inspection airport and perform a power-on self-test; after receiving a confirmation that the inspection airport has no problems after the power-on self-test, set the route according to the task requirements and generate and issue an inspection instruction through the background terminal; In response to the inspection instruction, the inspection drone takes off from the inspection airport according to the route, and continuously collects forest area video data through a visible light camera and transmits it to the inspection airport; The edge computing device of the inspection airport uses the Yolo algorithm to identify the fire source target for each frame of the forest area video data and calculates the first confidence level of the corresponding frame; if the first confidence level is greater than or equal to the threshold, the positioning device of the inspection drone obtains the location information of the fire source of the corresponding frame, generates reporting information based on the location data, image data and time data of the fire source of the corresponding frame, and reports it to the background terminal; the inspection drone returns; In response to the reported information, the backend terminal selects available fire-fighting airports, calculates the distances between the fire source location and different fire-fighting airports based on the location data, selects the fire-fighting airport with the closest distance as the target airport, and sends the fire source location information and fire-fighting instructions to the target airport; In response to the fire-fighting instruction, the fire-fighting drone located at the target airport flies to the target area according to the location information of the fire source, activates the airborne infrared vision module installed vertically downward on the fire-fighting drone, and performs real-time identification and positioning detection on the fire source again according to the temperature and pixel coordinates of the thermal imaging, and determines whether the current target is the fire source and whether it is directly under the fire-extinguishing bomb; if the detection result is yes, the fire-fighting bomb is dropped; if the detection result shows that there is a deviation, the position of the fire-fighting drone is calibrated until the fire source is directly under the fire-extinguishing bomb, and then the fire-extinguishing bomb is dropped.

2. The method for forest fire inspection and disposal based on drone according to claim 1, characterized in that: After the fire source target is identified by the Yolo algorithm and the first confidence of the corresponding frame is calculated, the method further includes: If the first confidence level is less than a threshold, the infrared camera on the inspection drone is activated; A fire source detection model for infrared cameras is trained within the edge computing device at the inspection airport using the lightweight YOLO framework. High-temperature targets are identified by analyzing temperature anomalies and their infrared confidence scores are calculated. The confidence scores of the visible light image and the infrared image are fused to produce a comprehensive confidence score. If the comprehensive confidence is greater than the set value, the location information of the fire source in the corresponding frame is obtained through the positioning device of the inspection drone, and the location data, image data and time data of the fire source in the corresponding frame are generated into reporting information and reported to the background terminal; otherwise, it is judged as a false alarm and the current frame is discarded.

3. The method for forest fire inspection and disposal based on drone according to claim 1, characterized in that: The steps of returning the inspection drone specifically include: After completing the current inspection mission, the inspection drone returns to the corresponding inspection hangar according to the original flight route; after receiving the signal that the inspection drone has docked, the inspection hangar automatically charges the inspection drone until the power level exceeds the preset power threshold.

4. The method for forest fire inspection and disposal based on drone according to claim 1, characterized in that: The step of obtaining the location information of the fire source of the corresponding frame specifically includes: The visible light camera is fixedly mounted on the bottom of the drone via a three-axis gimbal, and the coordinate system direction of the visible light camera, the coordinate system direction of the three-axis gimbal, and the coordinate system direction of the drone body are consistent; Based on the focal length of the visible light camera and the position of the fire source in the corresponding frame, converting the pixel coordinates corresponding to the fire source position into a direction vector in the camera coordinate system; Define the three-axis rotation matrix by the right-hand coordinate system, transform the direction vector in the camera coordinate system to the body coordinate system, and then combine the current attitude angle data of the drone to transform it to the world coordinate system based on the geographic orientation to obtain the direction vector in the world coordinate system; Obtain the takeoff position and current position of the inspection drone, convert the takeoff position and current position into spatial coordinates in the Earth-centered Earth-fixed coordinate system and calculate the difference vector; construct a rotation matrix from the Earth-centered Earth-fixed coordinate system to the world coordinate system with the takeoff position as a reference; obtain the current coordinates of the inspection drone's current position in the world coordinate system based on the difference vector and the rotation matrix; construct a spatial ray from the current coordinate along the direction vector in the world coordinate system, solve the coordinates of the intersection of the ray and the ground plane, and obtain the fire source coordinates of the fire source position in the world coordinate system; The fire source coordinates are converted into longitude and latitude.

5. The method for forest fire inspection and disposal based on drone according to claim 1 is characterized in that: The step of calibrating the position of the fire-fighting drone until the fire source is directly below the fire-fighting bomb specifically includes: Determine the center position of the image captured by the downward-looking camera of the fire-fighting drone, where the center position is the first midpoint coordinate corresponding to the width and height of the image; Extracting the second center coordinate of the detection frame of the fire source in the image; Calculate the pixel offset between the first midpoint coordinate and the second center coordinate; Based on the current flight altitude of the firefighting drone and the camera focal length parameters, the pixel offset is converted into a viewing angle offset. The horizontal and vertical ground distance deviations between the actual location of the fire source and the theoretical location of the fire bomb release are calculated based on the geometric relationship between the viewing angle offset and the flight altitude. According to the ground distance deviation, the fire-fighting drone is controlled to move horizontally until the ground distance deviation is less than a preset deviation.

6. The method for forest fire inspection and disposal based on drone according to claim 5, characterized in that: After the fire-fighting drone is controlled to move horizontally until the ground distance deviation is less than the preset deviation, the method further includes: The laser ranging module on the fire-fighting drone measures the vertical height between the drone and the fire source in real time; The edge computing device connected to the firefighting drone compares the measured vertical height with the preset safe release height threshold for fire extinguishing bombs. If the safe release height threshold is not reached, the firefighting drone is controlled to continue climbing. If the safe release height threshold is reached or exceeded, the drone is controlled to hover and maintain alignment with the fire source. After confirming that the fire source is aligned and the height meets the standard, the fire extinguishing bomb delivery mechanism is triggered to release the fire extinguishing bomb; After the fire is extinguished, the scene result image is captured and sent back to the backend terminal.

7. The method for forest fire inspection and disposal based on drone according to claim 6, characterized in that: After the scene result image is captured and transmitted back to the backend terminal, the following steps are also included: The fire-fighting drone returns to the corresponding fire-fighting airport according to the departure route. After receiving the signal that the fire-fighting drone has docked, the fire-fighting hangar automatically charges the fire-fighting drone and replenishes fire-fighting bombs until the power level and fire-fighting bombs are greater than the preset corresponding thresholds.

8. The method for forest fire inspection and disposal based on drone according to claim 1, characterized in that: Also includes: Build an interactive command interface based on the map engine; The command interface uses a preset resolution map slice of the forest area to be monitored as a spatial basemap, loads static layers of topography, inspection hangars, and fire-fighting hangar coordinates, and synchronizes the status information of inspection drones, inspection airports, fire-fighting drones, and fire-fighting airports in real time; After receiving the report information from the inspection drone or fire-fighting drone, it automatically parses its image, longitude and latitude, and timestamp, and dynamically annotates it on the command interface through the map annotation API; At the operational level, map zooming and layer management are achieved by loading different resolutions, and device status query and historical task retracing are performed by setting up query windows.

9. A forest fire inspection and disposal device based on drones, characterized in that: The system comprises an inspection drone with a multimodal sensor, an inspection airport wirelessly connected to the inspection drone, a fire-fighting drone, a fire-fighting airport wirelessly connected to the fire-fighting drone, and a background terminal wirelessly connected to the inspection airport and the fire-fighting airport respectively, wherein the fire-fighting drone carries fire-fighting bombs that can be automatically released; the inspection airport and the fire-fighting airport are both provided with corresponding edge computing devices; computer programs are pre-stored on the edge computing devices and the background terminals, and when the edge computing devices and the background terminals execute the computer programs, a forest fire inspection and disposal method based on drones as described in any one of claims 1 to 7 is implemented.

10. A forest fire inspection and disposal system based on drones, characterized by: include: Power-on self-test module, used to start the inspection airport and perform power-on self-test; After receiving the self-test results from the inspection airport, the flight route is set according to the mission requirements and the inspection instructions are generated and issued through the back-end terminal; A visible light image acquisition module is used for the inspection drone to take off from the inspection airport according to the route and continuously collect forest area video data through a visible light camera and transmit it to the inspection airport; The fire source judgment module is used for the edge computing device of the inspection airport to identify the fire source target for each frame of the forest area video data through the Yolo algorithm and calculate the first confidence level of the corresponding frame; if the first confidence level is greater than or equal to the threshold, the location information of the fire source of the corresponding frame is obtained through the positioning device of the inspection drone, and the location data, image data and time data of the fire source of the corresponding frame are generated into reporting information and reported to the background terminal; a fire extinguishing airport screening module, configured to screen out available fire extinguishing airports on the backend terminal, calculate the distances between the fire source location and different fire extinguishing airports based on the location data, select the fire extinguishing airport with the closest distance as the target airport, and send the fire source location information and fire extinguishing instructions to the target airport; The fire extinguishing module is used for a fire-fighting drone located at the target airport to fly to the target area based on the location information of the fire source, activate the airborne infrared vision module installed vertically downward on the fire-fighting drone, and perform real-time identification and positioning detection of the fire source again based on the temperature and pixel coordinates of the thermal imaging, and determine whether the current target is the fire source and whether it is directly under the fire extinguishing bomb; If the test result is yes, the fire extinguishing bomb will be dropped; if the test result shows a deviation, the position of the fire-fighting drone will be calibrated until the fire source is directly below the fire-fighting bomb, and then the fire-fighting bomb will be dropped.

Images