Multi-sensor forest fire real-time monitoring system based on unmanned aerial vehicle

Abstract

The embodiment of the invention discloses a forest fire real-time monitoring system based on multiple sensors of an unmanned aerial vehicle. The forest fire real-time monitoring system is characterized by comprising a four-rotor unmanned aerial vehicle, edge computing equipment, a carbon monoxide sensor and an infrared visible light double-camera, the quad-rotor unmanned aerial vehicle reciprocates and patrols the area for a long distance by relying on a flight task pre-input by edge computing equipment in a specified working range, and a charging hangar is configured at a take-off point of the quad-rotor unmanned aerial vehicle; the edge computing device is installed on the quad-rotor unmanned aerial vehicle, obtains video streams of visible light images and infrared images shot by the infrared and visible light double-camera in real time in the process of flying along a path, carries out real-time smoke alarm in combination with an electric signal input by the carbon monoxide sensor, and carries out multi-mode real-time monitoring by utilizing multiple sensors; when the edge computing device recognizes the forest fire, the forest fire is fed back to the quad-rotor unmanned aerial vehicle, then the quad-rotor unmanned aerial vehicle enters a return flight mode, and forest fire positioning is recorded in the device and fed back to the data center.

Description

Technical Field

[0001] The present invention relates to the field of forest fire monitoring technology, and specifically to a real-time forest fire monitoring system based on unmanned aerial vehicle (UAV) multi-sensor, which realizes real-time and accurate forest fire monitoring in a large forest area by carrying a variety of sensors and edge computing equipment on a quad-rotor UAV. Background Art

[0002] Forest fires pose a huge threat to the ecological environment and the safety of human life and property. Traditional forest fire monitoring methods, such as manual observation from watchtowers and ground patrols, have problems such as limited monitoring range, poor timeliness, and high labor costs. Although satellite remote sensing monitoring can cover a large area, it has a low resolution, making it difficult to detect small fires in the early stages, and has the defect of a long monitoring cycle. Patent ZL202010118233.7 has developed a large-scale forest fire patrol warning method and system based on drones, which mainly uses 5G base stations and mounted edge computing devices for forest fire monitoring. Its monitoring method is single and requires manual judgment and data processing, but there are no 5G base stations in the original forest area, so this method cannot achieve the monitoring of forest fires in the original forest area. In recent years, drone technology has been applied in the field of forest fire monitoring, but most of them rely on a single sensor, and the monitoring accuracy and reliability are insufficient, which cannot meet the needs of real-time and accurate monitoring of forest fires. Optical sensors are easily affected by environmental interference and have monitoring blind spots, while carbon monoxide sensors are feasible for forest fire monitoring. Carbon monoxide is inevitably produced when a forest fire occurs, which can be used as a fire indicator. Existing carbon monoxide sensors have high accuracy, small size, low power consumption, can quickly detect extremely low concentrations of carbon monoxide, and are suitable for being carried on drones.

[0003] Therefore, it is of great practical significance to develop an efficient and accurate real-time forest fire monitoring system based on drones. Summary of the invention

[0004] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0005] The present invention aims to provide a real-time forest fire monitoring system based on unmanned aerial vehicle (UAV) multi-sensor, which utilizes the maneuverability of a quad-rotor UAV and the collaborative work of multiple sensors to achieve long-distance patrol and protection of large forest areas and real-time monitoring, precise positioning and timely feedback of forest fires, effectively solving the shortcomings of existing forest fire monitoring technology.

[0006] The system is mainly composed of a quadrotor drone, edge computing equipment, a carbon monoxide sensor and an infrared and visible light dual-camera.

[0007] Step 1, mission pre-programming input: Pre-write the flight mission in the edge computing device, covering flight trajectory planning, flight altitude and time setting, mounting equipment setting, etc., and store computer programs for forest fire, smoke detection and carbon monoxide gas detection to provide instructions and data processing basis for system operation.

[0008] Step 2, UAV takeoff and cruise: The quadcopter takes off from a takeoff point equipped with a charging hangar within the specified working range according to the flight mission pre-input by the edge computing device, and begins a long-distance cruise to patrol the area, during which various sensors and equipment are activated and put into working state.

[0009] Step 3, image data processing: The edge computing device obtains the video stream of visible light images and infrared images taken in real time by the infrared and visible light dual cameras, extracts them, and performs real-time dynamic processing on the extracted images to compare the image features of smoke and forest fires, such as color, texture, shape features, etc. It can also perform local fusion analysis of multiple images.

[0010] Step 4, gas data monitoring: The carbon monoxide sensor detects the carbon monoxide concentration in real time and outputs a continuous voltage level to the edge computing device. The edge computing device uses the voltage level changes to determine the area where the forest fire may exist and assist in forest fire judgment.

[0011] Step 5, multi-sensor data judgment: The edge computing device combines the data from the infrared and visible light dual cameras and the carbon monoxide sensor to determine whether it is a false alarm. If it is a false alarm, no alarm is sent to the data center, and the drone continues to perform the flight mission; if it is not a false alarm, the next step is executed.

[0012] Step 6, alarm and positioning feedback: When a forest fire is detected, the edge computing device sends an alarm and its own location information to the data center so that the data center can grasp the forest fire situation in a timely manner and make response decisions, while recording the forest fire location information.

[0013] Step 7, drone return operation: The edge computing device feeds back forest fire information to the quadcopter drone, controls it to interrupt the current flight path, switches to automatic return mode, and guides the drone back to the charging hangar to ensure the safety of the equipment and prepare for the next mission.

[0014] Step 8, data upload and summary: After the quadcopter completes the flight mission and returns to the hangar, it uploads all the data sets taken during the flight mission to the data center. The data center summarizes and analyzes these data to provide data support for subsequent research on forest fire occurrence patterns and fire loss assessment.

[0015] One of the above embodiments of the present disclosure has the following beneficial effects:

[0016] Quad-rotor drones have significant advantages in improving monitoring efficiency. With their highly flexible maneuverability, they can accurately reach every corner of the forest area according to pre-planned flight trajectories, including remote areas that are difficult to reach with traditional monitoring methods. With their fast flight speed and large coverage, they can complete patrol tasks in a wider area within a unit of time. During the patrol process, quad-rotor drones can effectively identify potential forest fire hazards in remote areas and detect early fires in a timely manner with the professional equipment they carry. This feature greatly improves the comprehensiveness and timeliness of forest fire monitoring, and provides important technical support for early warning and efficient prevention and control of forest fires.

[0017] In this system, the infrared and visible light dual camera and the carbon monoxide sensor cooperate and work together. The infrared and visible light dual camera can capture visible light images and infrared images of the forest area in real time to obtain rich visual information; the carbon monoxide sensor focuses on real-time detection of changes in carbon monoxide concentrations in the forest area. As the data processing core, the edge computing device will conduct comprehensive analysis and processing of the multi-source data collected by these two sensors. This multi-sensor fusion and multi-dimensional data analysis method greatly improves the accuracy of forest fire monitoring, can effectively reduce false alarms caused by single factor judgment, and ensure that the judgment of forest fires is more accurate and reliable.

[0018] The entire drone-based multi-sensor forest fire real-time monitoring system has powerful real-time data processing capabilities. During the drone flight monitoring process, the edge computing device can analyze and detect multi-source data in real time. Once the signs of forest fire are identified through data analysis, it will respond quickly. On the one hand, the information is fed back to the quadcopter drone, so that it can adjust the flight mission in time and enter the return mode; on the other hand, the specific location and other forest fire information will be quickly collected and fed back to the data center. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the implementation of the present invention or the technical solution in the prior art, the following briefly introduces the drawings required for the implementation or the description of the prior art system. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0020] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this field. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0021] Figure 1 This is a flow chart of the real-time forest fire monitoring system based on unmanned aerial vehicle multi-sensor provided by the present invention;

[0022] Figure 2 This is a structural block diagram of the method for real-time forest fire monitoring based on multi-sensor drones provided by the present invention;

[0023] Figure 3 This is a structural block diagram of the real-time forest fire monitoring device based on unmanned aerial vehicle multi-sensor provided by the present invention. DETAILED DESCRIPTION

[0024] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. It should be understood that the embodiments of the present disclosure can be implemented by various technical means, and the embodiments shown in the accompanying drawings are only exemplary and are not intended to limit the scope of protection of the present invention. The purpose of providing these embodiments is to enable those skilled in the art to more thoroughly understand the technical details and implementation logic of the present solution.

[0025] In the specific implementation process, the forest fire monitoring process of this system is as follows: Figure 1 As shown, the main steps include: based on the geographic information data of the target forest area, the flight path, cruising altitude and sensor parameters of the drone are planned in the edge computing and fire judgment unit (300), a flight mission file in JSON format is generated, and it is uploaded to the drone platform control unit (100) through the ground station. After the drone takes off from the hangar equipped with an automatic charging function, it is controlled by the flight controller (102) to cruise along the preset path. The multi-sensor data acquisition unit (200) starts working synchronously. The visible light camera (201) dynamically adjusts the focal length to capture high-definition images and capture the smoke shape; the thermal infrared imager (202) scans the surface area and detects the temperature abnormality area; the carbon monoxide detector (203) monitors the gas concentration in real time and outputs a 0-3V analog signal. All data are transmitted to the edge computing device through a high-speed interface for real-time processing.

[0026] The edge computing and fire judgment unit (300) receives image data through the MIPI interface, performs texture analysis through the image preprocessing module (301), and simultaneously performs image processing on the infrared image and marks the abnormal high temperature area. If the spatial overlap area of ​​the two types of images reaches the set threshold, a preliminary alarm is triggered. The gas concentration analysis module (302) analyzes the concentration change trend through a sliding window algorithm, and when the continuous sampling value exceeds the safety threshold, it is determined that there are signs of combustion. The multi-source data fusion module (303) executes the judgment logic: if only the image is abnormal and the gas concentration is normal, it is determined to be optical interference; if only the gas is abnormal and there is no image support, the drone is controlled to return to the suspicious area for re-inspection; when both types of data meet the alarm conditions, it is confirmed that a fire has occurred, and the fire point coordinates, environmental parameters and image snapshots are immediately encrypted and transmitted to the data center.

[0027] After the fire is confirmed, the drone switches to emergency mode, adjusts the flight altitude according to real-time meteorological data, avoids the rising airflow of smoke, and outputs the core coordinates of the fire point. The communication and data feedback unit (400) encrypts and transmits back the fire point information and on-site images through the 5G transmission module (401), and the local SSD storage (402) caches the original data. The drone then terminates the cruise mission, starts dynamic path planning to return, avoids obstacles and lands in the charging hangar, completes data upload and automatic battery replacement. After receiving the alarm information, the data center generates an emergency response plan.

[0028] The hardware of this system includes a quad-rotor drone, an edge computing module, a multi-sensor unit and a charging hangar. The drone is equipped with a high-precision positioning module and an obstacle avoidance system, and the edge computing module has a built-in multi-source data fusion algorithm to support real-time image processing and gas concentration analysis. The charging hangar integrates solar power supply and visual guidance devices to ensure all-weather autonomous operation. The system also includes a server cluster deployed in the data center to store historical fire data, environmental parameters and fire simulation models.

[0029] The hardware devices of this system are Figure 3As shown, it includes a drone platform control unit (100), a multi-sensor data acquisition unit (200), an edge computing and fire situation judgment unit (300) and a communication and data feedback unit (400). The drone platform control unit (100) integrates an RTK positioning module (101) and a flight controller (102), and supports dual-frequency GNSS (GPS+Beidou) and binocular visual obstacle avoidance algorithms. The multi-sensor data acquisition unit (200) includes a visible light camera (201), a thermal infrared imager (202) and a carbon monoxide detector (203), which are respectively used for image capture, temperature monitoring and gas concentration detection. The edge computing and fire situation judgment unit (300) uses a multi-core ARM architecture processor (such as NVIDIA Jetson Nano) to accelerate the smoke recognition model (based on a lightweight network improved by YOLOv5). The communication and data feedback unit (400) realizes real-time data transmission and caching through a 5G transmission module (401) and a local SSD storage (402).

[0030] It should be noted that in this system, the edge computing device is the core of real-time data processing. Its hardware uses high-performance embedded processors (such as multi-core ARM architecture chips) and dedicated acceleration modules. The built-in storage unit integrates cache (such as LPDDR4 memory) and persistent storage media (such as eMMC flash memory) to temporarily save sensor raw data and intermediate processing results. The device receives sensor data carried by the drone through a multi-channel communication interface: visible light and infrared images are transmitted through the MIPI interface, carbon monoxide concentration signals are converted into digital inputs through the ADC module, and GPS positioning information is analyzed in real time by the UART serial port. At the software level, the device runs a lightweight operating system (such as the Linux RT kernel), deploys a smoke recognition model, a gas concentration dynamic analysis algorithm, and multi-source data fusion logic, and implements the following core functions: frame-level processing of image data (including color space conversion, hot spot detection, and feature registration), sliding window statistics and threshold judgment of gas concentration, and associates heterogeneous data into a unified event stream through a timestamp synchronization mechanism. The processed results are divided into two categories: real-time alarm information is directly uploaded to the cloud data center through the 4G / 5G module; the original data and intermediate calculation results are temporarily stored in the local storage unit and exported in batches through Gigabit Ethernet after the drone returns. The device supports remote firmware updates and algorithm iterations, and can load optimized detection models through OTA to adapt to the environmental characteristics of different forest areas. The computer storage medium of this system may include non-volatile memory (such as flash memory, solid-state hard disk) or a remote cloud storage platform, in which the stored program instructions are used to control the drone cruise, data processing and alarm response processes.

[0031] It should be understood that the hardware configuration, algorithm model and communication protocol in the above implementation can be adjusted according to the actual application scenario. For example, the carbon monoxide sensor can be replaced by other combustion characteristic gas detection devices, and the edge computing module can be deployed by local equipment or cloud collaborative processing. Any technical solution implemented by equivalent replacement or adaptive improvement on the basis of the technical solution of the present invention shall be deemed to fall within the scope of protection of the present invention.

Claims

1. A real-time forest fire monitoring system based on drone multi-sensor, the characteristics of which include: Quad-rotor drones, edge computing devices, carbon monoxide sensors, and infrared and visible light dual-cameras; The quadrotor UAV performs long-distance cruise and patrol within a specified working range based on the flight mission pre-input by the edge computing device, and a charging hangar is configured at the take-off point of the quadrotor UAV; the edge computing device is installed on the quadrotor UAV, and during the cruise, the video stream of visible light images and infrared images taken in real time by the infrared and visible light dual cameras is obtained, and multimodal data fusion analysis is performed in combination with the analog signal input by the carbon monoxide sensor to monitor forest fires in real time; when the edge computing device identifies a forest fire, the quadrotor UAV is controlled to switch to the return mode, and the fire location information is recorded and fed back to the data center.

2. The real-time forest fire monitoring system based on multi-sensor drone according to claim 1, wherein: The quad-rotor drone is connected to the edge computing device through an adapter expansion board, and the edge computing device pre-inputs the required flight tasks within the real-time monitoring range into the drone through a USB serial port; the flight tasks include flight trajectory planning, flight altitude and flight time setting, and mounting equipment setting.

3. The real-time forest fire monitoring system based on unmanned aerial vehicle multi-sensor according to claim 1, wherein: The quad-rotor drone itself is not equipped with a computing device, but an edge computing device is mounted on it via a special fixing frame, and the fixing frame is fixed to the drone body by screws.

4. The real-time forest fire monitoring system based on unmanned aerial vehicle multi-sensor according to claim 1, wherein: The charging station is equipped with a solar power supply module or a cable power supply interface.

5. The real-time forest fire monitoring system based on multi-sensor drone according to claim 1, wherein: The edge computing device performs real-time dynamic processing on visible light images and infrared images taken by the infrared and visible light dual cameras, including color space conversion, texture feature extraction, and multi-image local fusion analysis; in the processed images, smoke and forest fires are identified by comparing features such as color, texture, and shape.

6. The real-time forest fire monitoring system based on multi-sensor drone according to claim 1, wherein: The carbon monoxide sensor outputs a 0-3V analog signal to the edge computing device in real time; the edge computing device analyzes the trend of gas concentration changes through an algorithm to determine the forest fire area. According to the real-time forest fire monitoring system based on drone multi-sensor according to claim 1, when the quad-rotor drone performs long-distance cruising and patrolling within a specified working range, when the edge computing device detects a forest fire, the quad-rotor drone is controlled through the communication method according to claim 2 to interrupt the current flight path and switch to automatic return mode.

7. The real-time forest fire monitoring system based on unmanned aerial vehicle multi-sensor according to claim 1, wherein: The edge computing device simultaneously performs multi-sensor data fusion judgment, combining image features and gas concentration data to confirm forest fires.

8. The edge computing device according to claims 1-8, characterized in that: A portable embedded computer device weighing no more than 2kg, running a lightweight operating system and storing a forest fire detection algorithm program.

9. According to the UAV multi-sensor forest fire real-time monitoring system of claim 1, the system execution steps are as follows: Step 1: Pre-write flight missions and store forest fire, smoke detection, and carbon monoxide gas detection programs to edge computing devices; Step 2, the quadrotor drone performs long-distance cruise patrol based on the pre-input flight mission; Step 3: The edge computing device processes visible light images and infrared images in real time to extract smoke and forest fire features; Step 4: The edge computing device analyzes the analog signal of the carbon monoxide detector to determine the forest fire area and narrow the area where the smoke appears; Step 5: The edge computing device fuses the multi-sensor data to determine whether it is a false alarm: if it is a false alarm, it continues to cruise; otherwise, it sends an alarm and positioning information to the data center; Step 6: The edge computing device controls the quadcopter to return to the charging hangar; Step 7: The quadrotor drone completes the flight mission and uploads all data sets in the flight mission to the data center.

Images