A Smart Monitoring and Processing Method for Forest Fire Prevention Based on Unmanned Aerial Vehicles

Abstract

This application discloses a smart monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs), relating to the field of forest fire prevention technology. It collects multimodal data such as visible light images, infrared thermal imaging, smoke concentration, and environmental meteorological data, and after time synchronization, inputs this data into a multi-source information fusion fire identification model to achieve accurate fire identification and three-dimensional coordinate positioning of the fire source. Then, combining the fire source coordinates, meteorological data, three-dimensional information model, and a database of combustible combustion characteristics, it predicts the fire spread trend area within a future time window. Finally, based on the fire source location, fire trend, and fire extinguishing equipment information, a deep reinforcement learning decision model is used to generate target control commands including spray angle, pressure, and extinguishing agent type. After the commands are issued to the fire extinguishing equipment for execution, the model is updated through real-time feedback data, achieving adaptive optimization of the fire extinguishing strategy. This improves fire extinguishing efficiency, reduces resource consumption, ensures the safety of non-fire source targets, and is suitable for complex scenarios.

Description

Technical Field

[0001] This application relates to the field of forest fire prevention technology, specifically to a smart monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Forest fires are characterized by their suddenness, destructive power, and rapid spread. If they are not extinguished promptly and effectively, they will cause enormous losses to the ecological environment and people's lives and property. With the rapid development of drone technology, drones, due to their maneuverability and wide field of view, are widely used in forest fire monitoring. However, existing drone-based forest fire prevention solutions still have the following shortcomings: First, in terms of fire monitoring and identification, they usually rely solely on single infrared thermal imaging or visible light video, which is easily affected by environmental interference, leading to false alarms or missed alarms. Furthermore, multi-sensor data often lacks strict time synchronization, resulting in low accuracy in fusion identification. Second, in terms of fire prediction, they mostly use macroscopic empirical models, failing to combine specific three-dimensional forest topography, combustible material distribution characteristics, and real-time weather conditions for refined voxel-level spread prediction. Third, in terms of fire suppression control, existing drone fire suppression relies heavily on remote control by the flight controller or automatic spraying based on simple rules. They cannot dynamically and adaptively optimize the spray angle, pressure, and extinguishing agent type based on the fire source location, fire spread trend, weather conditions, and remaining extinguishing agent quantity, resulting in low fire suppression efficiency and a lack of closed-loop intelligent decision-making and learning capabilities based on real-time feedback. Summary of the Invention

[0003] The purpose of this application is to provide a smart monitoring and processing method for forest fire prevention based on drones, which solves the problems of low accuracy of drone forest fire monitoring, imprecise prediction of fire spread and / or lack of closed-loop adaptive optimization in fire extinguishing control in the prior art.

[0004] This application is achieved through the following technical solution: A smart monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) includes: The drone collects visible light image data, infrared thermal imaging data, smoke concentration data, and environmental meteorological data of the target forest area in real time and synchronizes them in time to obtain multimodal sensor data after time synchronization. The multimodal sensor data after time synchronization is input into a pre-trained multi-source information fusion fire identification model for analysis to obtain fire identification results. Based on the fire identification results, fire source localization analysis is triggered to obtain the three-dimensional coordinates of the fire source space. Based on the three-dimensional coordinates of the fire source space, environmental meteorological data, three-dimensional information model of the target forest area, and a pre-set database of combustible material distribution and combustion characteristics, the fire spread trend area in the next prediction time window is predicted. Based on the three-dimensional coordinates of the fire source space and the fire spread trend area, and combined with the fire extinguishing equipment information corresponding to the fire extinguishing equipment carried by the UAV, a deep reinforcement learning decision model is used to generate target fire extinguishing control commands; the target fire extinguishing control commands include the spray angle adjustment control command, the spray pressure adjustment control command, and the fire extinguishing agent type selection command of the fire extinguishing equipment nozzle. The target fire extinguishing control command is sent to the fire extinguishing equipment for fire extinguishing control, and fire extinguishing feedback data is acquired in real time. The deep reinforcement learning decision model is updated using the fire extinguishing feedback data to carry out the next fire extinguishing control.

[0005] In one possible implementation, visible light image data, infrared thermal imaging data, smoke concentration data, and environmental meteorological data of the target forest area are collected in real time by a drone and synchronized in time to obtain time-synchronized multimodal sensor data, including: During each data acquisition cycle, a rigidly connected binocular vision system and infrared thermal imaging device on the drone are used to collect visible light image data and infrared thermal imaging data. At the end of each data collection cycle, smoke concentration data is collected by the smoke concentration sensor on the drone and environmental meteorological data is collected by the environmental meteorological station deployed on the ground; the environmental meteorological data includes wind direction, wind speed, ambient temperature and ambient humidity. Visible light image data, infrared thermal imaging data, smoke concentration data, and environmental meteorological data collected at the same time are combined as multimodal sensor data after time synchronization.

[0006] In one possible implementation, the time-synchronized multimodal sensor data is input into a pre-trained multi-source information fusion fire identification model for analysis to obtain fire identification results. Based on the fire identification results, fire source localization analysis is triggered to obtain the three-dimensional spatial coordinates of the fire source, including: The time-synchronized multimodal sensor data is input into a pre-trained multi-source information fusion fire identification model for analysis to obtain the classification probability distribution; the pre-trained multi-source information fusion fire identification model adopts a hybrid structure of convolutional neural network and long short-term memory network; Based on the classification probability distribution, the category with the highest probability is determined as the fire identification result; wherein, the fire identification result includes whether a fire exists or not. If the fire identification result indicates that there is no fire, then the monitoring of multimodal sensor data after time synchronization for the next data acquisition cycle will continue. If the fire identification result indicates that a fire exists, then the fire source location analysis is triggered to obtain the three-dimensional spatial coordinates of the fire source.

[0007] In one possible implementation, fire source localization analysis is triggered to obtain the three-dimensional spatial coordinates of the fire source, including: Based on infrared thermal imaging data, the pixel coordinates of the first fire source center and the pixel coordinates of the second fire source center are determined from two visible light image data measured by a binocular vision system. Based on the pixel coordinates of the first fire source center and the pixel coordinates of the second fire source center, obtain the disparity data between the two visible light image data measured by the binocular vision system; The depth of the fire source in the camera coordinate system is obtained based on the parallax data between two visible light image data measured by the binocular vision system. Based on the pixel coordinates of the first or second fire source center, and combined with depth and camera extrinsic matrix, the pixel coordinates of the fire source center are transformed from the camera coordinate system to the world coordinate system to obtain the three-dimensional coordinates of the fire source space.

[0008] In one possible implementation, the pre-set combustible distribution and combustion characteristic database includes combustible types, calorific values, ignition points, and flame spread coefficients within the target forest area.

[0009] In one possible implementation, based on the three-dimensional coordinates of the fire source space, environmental meteorological data, a three-dimensional information model of the target forest area, and a pre-set database of combustible material distribution and combustion characteristics, the fire spread trend area within a future prediction time window is predicted, including: The target forest area is divided into voxels, and based on the pre-set database of combustible material distribution and combustion characteristics, the fire line spread rate between the voxel where the fire source is located and other adjacent voxels is determined as follows: ; in, This represents the fire line spread rate from the i-th voxel to the j-th voxel. The i-th voxel represents the voxel where the fire source is located, which is determined based on the three-dimensional coordinates of the fire source space. The j-th voxel is any other adjacent voxel that is adjacent to the voxel where the fire source is located. The baseline spread rate under windless conditions is determined based on the flame spread coefficient. After determining the type of combustible material in each voxel through a three-dimensional information model of the target forest area, the flame spread coefficient is obtained based on a pre-set database of combustible material distribution and combustion characteristics. This represents an exponential function with base e. Indicates the wind speed influence coefficient. This indicates the wind speed value. Indicates wind direction value. This represents the directional angle from the i-th voxel to the j-th voxel. Indicates reference wind speed. Rock mass barrier factor; Based on the fire spread rate, the predicted ignition time of other adjacent voxels adjacent to the voxel where the fire source is located is determined as follows: ; in, Let be the predicted ignition time from the i-th voxel to the j-th voxel. Let the voxel side length be , Let be the density of combustible material within the j-th genus. Let be the density of combustible material within the i-th voxel. Let J be the specific heat capacity of the combustible material in the j-th genus. Let be the ignition point of the combustible material within the j-th genus. For ambient temperature, Let be the calorific value of the combustible material in the j-th genus. For heat transfer efficiency; If the predicted ignition time is less than the preset prediction time window, then the j-th voxel is marked as the target flame spread voxel. Based on all target flame spread voxels, determine the fire spread trend area within a future forecast time window.

[0010] In one possible implementation, based on the three-dimensional coordinates of the fire source space and the fire spread trend area, and combined with the fire extinguishing equipment information corresponding to the fire extinguishing equipment carried by the UAV, a deep reinforcement learning decision model is used to generate target fire extinguishing control commands, including: Based on the three-dimensional coordinates of the fire source space and the fire spread trend area, the coordinate components of the fire source relative to the fire extinguishing equipment, the number of target flame spread voxels in the fire spread trend area and the average fire line spread rate of all target flame spread voxels are determined to obtain the first state data. The highest temperature in the voxel corresponding to the fire source, the current pitch angle of the fire extinguishing equipment nozzle, the current yaw angle of the fire extinguishing equipment nozzle, the current spray pressure of the fire extinguishing equipment nozzle, the current wind speed value, the current wind direction value, the remaining amount of extinguishing agent in the fire extinguishing equipment, and the rate of change of smoke concentration over time are obtained to obtain the second state data. The first state data and the second state data are used together as the target environment state, and the target environment state is used as the input of the deep reinforcement learning decision model to obtain the target fire extinguishing control action output by the deep reinforcement learning decision model; the target fire extinguishing control action includes the spray angle adjustment amount, spray pressure adjustment amount, and fire extinguishing agent type of the fire extinguishing equipment nozzle; The target fire extinguishing control action is converted into the instruction type specified by the fire extinguishing equipment to obtain the target fire extinguishing control instruction.

[0011] In one possible implementation, the target fire extinguishing control command is issued to the fire extinguishing equipment for fire extinguishing control, and fire extinguishing feedback data is acquired in real time. The deep reinforcement learning decision model is then updated using the fire extinguishing feedback data for the next fire extinguishing control operation, including: The target fire extinguishing control command is sent to the fire extinguishing equipment for fire extinguishing control, and fire extinguishing feedback data is acquired in real time; The execution reward is obtained based on the fire extinguishing feedback data, and the new target environmental status is also obtained at the same time; The target environmental state, target fire suppression control actions, execution rewards, and new target environmental states are all combined as historical data. The deep reinforcement learning decision model is periodically updated using historical data for the next fire suppression control.

[0012] In one possible implementation, the execution reward is obtained based on the fire extinguishing feedback data as follows: ; in, In order to implement the reward, As the first coefficient, As the second coefficient, The third coefficient, It is the fourth coefficient. It is the fifth coefficient. The average temperature drop rate of the fire source is obtained based on the fire extinguishing feedback data. The reduction rate of the target flame spread voxel. The Euclidean distance between the point of impact of the extinguishing agent sprayed by the fire extinguishing equipment and the center point of the voxel where the fire source is located. The rate of consumption of extinguishing agent. The event indicator function is set to +100 when the fire source is completely extinguished and -100 when the fire is out of control.

[0013] In one possible implementation, the deep reinforcement learning decision model is periodically updated using historical data, including: At each update cycle, the deep reinforcement learning decision model is updated using historical data to obtain the updated deep reinforcement learning decision model. Also includes: The updated deep reinforcement learning decision model is run in an offline simulator based on historical data. If its reward value is improved by more than 5% compared to the current deep reinforcement learning decision model, the updated deep reinforcement learning decision model replaces the current deep reinforcement learning decision model; otherwise, the current deep reinforcement learning decision model is retained.

[0014] Compared with the prior art, this application has the following advantages and beneficial effects: This application discloses a smart monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs). It collects multimodal data such as visible light images, infrared thermal imaging, smoke concentration, and environmental meteorological data, and after time synchronization, inputs this data into a multi-source information fusion fire identification model to achieve accurate fire identification and three-dimensional coordinate positioning of the fire source. Then, combining the fire source coordinates, meteorological data, three-dimensional information model, and a database of combustible combustion characteristics, it predicts the fire spread trend area within a future time window. Finally, based on the fire source location, fire trend, and fire extinguishing equipment information, a deep reinforcement learning decision model is used to generate target control commands including spray angle, pressure, and extinguishing agent type. After the commands are issued to the fire extinguishing equipment for execution, the model is updated through real-time feedback data, achieving adaptive optimization of the fire extinguishing strategy, improving fire extinguishing efficiency, and reducing resource consumption. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the exemplary embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart illustrating a smart forest fire monitoring and processing method based on unmanned aerial vehicles (UAVs) provided in this application embodiment. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this application are only for explaining this application and are not intended to limit this application.

[0017] like Figure 1 As shown in the figure, this application provides a method for intelligent monitoring and processing of forest fire prevention based on unmanned aerial vehicles (UAVs), including: S101. Real-time collection of visible light image data, infrared thermal imaging data, smoke concentration data and environmental meteorological data of the target forest area by drone and time synchronization to obtain multimodal sensor data after time synchronization. S102. Input the time-synchronized multimodal sensor data into the pre-trained multi-source information fusion fire identification model for analysis, obtain the fire identification result, and trigger the fire source localization analysis based on the fire identification result to obtain the three-dimensional coordinates of the fire source space. S103. Based on the three-dimensional coordinates of the fire source space, environmental meteorological data, three-dimensional information model of the target forest area, and a pre-set database of combustible material distribution and combustion characteristics, predict the fire spread trend area in a future prediction time window. S104. Based on the three-dimensional coordinates of the fire source space and the fire spread trend area, and combined with the fire extinguishing equipment information corresponding to the fire extinguishing equipment carried by the UAV, a deep reinforcement learning decision model is used to generate target fire extinguishing control instructions; the target fire extinguishing control instructions include the spray angle adjustment control instructions, the spray pressure adjustment control instructions, and the fire extinguishing agent type selection instructions of the fire extinguishing equipment nozzle. S105. The target fire extinguishing control command is sent to the fire extinguishing equipment for fire extinguishing control, and fire extinguishing feedback data is acquired in real time. The deep reinforcement learning decision model is updated using the fire extinguishing feedback data to carry out the next fire extinguishing control.

[0018] In one possible implementation, visible light image data, infrared thermal imaging data, smoke concentration data, and environmental meteorological data of the target forest area are collected in real time by a drone and synchronized in time to obtain time-synchronized multimodal sensor data, including: During each data acquisition cycle, a rigidly connected binocular vision system and infrared thermal imaging device on the drone are used to collect visible light image data and infrared thermal imaging data. At the end of each data collection cycle, smoke concentration data is collected by the smoke concentration sensor on the drone and environmental meteorological data is collected by the environmental meteorological station deployed on the ground; the environmental meteorological data includes wind direction, wind speed, ambient temperature and ambient humidity. Visible light image data, infrared thermal imaging data, smoke concentration data, and environmental meteorological data collected at the same time are combined as multimodal sensor data after time synchronization.

[0019] For example, you can first establish a world coordinate system. The origin of the coordinate system is set at the southwest corner of the target forest area or any other point on the ground. The positive direction of the axis points due east. The positive direction of the axis points due north. The axis is perpendicular to the ground and pointing upwards. A binocular vision system is deployed on the drone, and the drone's positioning and the world coordinates of the binocular vision system are calibrated. The relationship between the two cameras can be assumed to be that they share the same coordinates. The system consists of two visible light cameras (left camera...). With right camera It consists of a stereo camera and a long-wave infrared thermal imaging device; the three are rigidly connected and have completed joint calibration. The baseline distance of the stereo camera... ,focal length Smoke concentration sensors and environmental weather stations (for measuring wind speed with high accuracy) Wind direction accuracy ) world coordinates The relationships between the drone's positioning and the system's positioning are pre-defined and stored. The smoke concentration sensor can also default to having the same coordinates as the drone. Therefore, after positioning via the drone, the positioning of the binocular vision system and the smoke concentration sensor can be determined, while the environmental weather station is deployed on the ground and its positioning is fixed.

[0020] All sensor data are synchronized using a hardware method based on the IEEE 1588 precision time protocol, controlling the synchronization error of the system master clock within a specified range. Within this period. Establish a uniform data sampling period. That is, all sensor data are processed according to... The sampling frequency is packaged into a data frame, and each data frame is accompanied by a unique timestamp. (in (This is the frame number), which allows us to determine the data collected simultaneously.

[0021] It is worth noting that, in order to ensure effective data processing, the data involved in the various formulas in this application embodiment can also be dimensionless.

[0022] In one possible implementation, the time-synchronized multimodal sensor data is input into a pre-trained multi-source information fusion fire identification model for analysis to obtain fire identification results. Based on the fire identification results, fire source localization analysis is triggered to obtain the three-dimensional spatial coordinates of the fire source, including: The time-synchronized multimodal sensor data is input into a pre-trained multi-source information fusion fire identification model for analysis to obtain the classification probability distribution; the pre-trained multi-source information fusion fire identification model adopts a hybrid structure of convolutional neural network and long short-term memory network; Based on the classification probability distribution, the category with the highest probability is determined as the fire identification result; wherein, the fire identification result includes whether a fire exists or not. If the fire identification result indicates that there is no fire, then the monitoring of multimodal sensor data after time synchronization for the next data acquisition cycle will continue. If the fire identification result indicates that a fire exists, then the fire source location analysis is triggered to obtain the three-dimensional spatial coordinates of the fire source.

[0023] For example, a serial convolutional neural network and long short-term memory (LSTM) network architecture is employed. First, a ResNet-50 residual network is used as a spatial feature extractor to perform convolution operations on the stitched composite image (a composite image obtained by stitching visible light image data and infrared thermal imaging data to the same size, either by channel dimension or in two dimensions), outputting a spatial feature vector. Then, normalized multimodal sensor data is concatenated with this spatial feature vector and input into the LTM network for recognition, yielding a classification probability distribution. It is worth noting that the multi-source information fusion fire identification model can be trained beforehand using sample data.

[0024] Optionally, for any given moment, only the visible light image data and infrared thermal imaging data corresponding to that moment can be collected, while for other multimodal sensor data, data from the previous T moments can be collected to form spatiotemporal data, thereby further improving recognition accuracy.

[0025] This application embodiment improves the recognition accuracy by more than 30% through multimodal data fusion (visible light + infrared + smoke + meteorology) combined with a hybrid model of convolutional neural network and long short-term memory network; binocular vision and infrared thermal imaging are used for collaborative localization, and the three-dimensional coordinate error of the fire source is less than 0.5 meters, solving the problem of fuzzy localization in traditional methods.

[0026] In one possible implementation, fire source localization analysis is triggered to obtain the three-dimensional spatial coordinates of the fire source, including: Based on infrared thermal imaging data, the pixel coordinates of the first fire source center and the pixel coordinates of the second fire source center are determined from two visible light image data measured by a binocular vision system. Because the binocular vision system and the infrared thermal imaging device are fixed in position and rigidly connected, the mapping relationship between the pixel coordinates in the infrared thermal imaging data and the visible light image data can be pre-calibrated. Therefore, based on the obtained infrared thermal imaging data, the flame region (i.e., the region where the temperature is greater than the temperature threshold) can be determined. The center of the flame region, the temperature of the highest temperature point, or the average temperature of multiple highest temperature points (when there are multiple points with the same temperature) can be used as the basic fire source center pixel coordinates. Then, through the mapping relationship, the basic fire source center pixel coordinates are mapped to the first fire source center pixel coordinates and the second fire source center pixel coordinates, respectively.

[0027] Based on the pixel coordinates of the first fire source center and the pixel coordinates of the second fire source center, obtain the disparity data between the two visible light image data measured by the binocular vision system; The depth of the fire source in the camera coordinate system is obtained based on the parallax data between two visible light image data measured by the binocular vision system. Based on the pixel coordinates of the first or second fire source center (generally, the data from the left camera is used for calculation; if the pixel coordinates of the first fire source center are data from the left camera, then the first fire source center pixel coordinates are used for calculation; otherwise, the second fire source center pixel coordinates are used for calculation), combined with depth and camera extrinsic matrix, the pixel coordinates of the fire source center are transformed from the camera coordinate system to the world coordinate system to obtain the three-dimensional coordinates of the fire source space.

[0028] Binocular vision positioning technology is a relatively mature technology. It can locate fire sources by calibrating the camera in advance and combining depth data.

[0029] In one possible implementation, the pre-set combustible distribution and combustion characteristic database includes combustible types, calorific values, ignition points, and flame spread coefficients within the target forest area.

[0030] In one possible implementation, based on the three-dimensional coordinates of the fire source space, environmental meteorological data, a three-dimensional information model of the target forest area, and a pre-set database of combustible material distribution and combustion characteristics, the fire spread trend area within a future prediction time window is predicted, including: The target forest area is divided into voxels, and based on the pre-set database of combustible material distribution and combustion characteristics, the fire line spread rate between the voxel where the fire source is located and other adjacent voxels is determined as follows: ; in, This represents the fire line spread rate from the i-th voxel to the j-th voxel. The i-th voxel represents the voxel where the fire source is located, which is determined based on the three-dimensional coordinates of the fire source space. The j-th voxel is any other adjacent voxel that is adjacent to the voxel where the fire source is located. The baseline spread rate under windless conditions is determined based on the flame spread coefficient (for example, a basic reference spread rate can be preset, and different combustibles correspond to different flame spread coefficients. The flame spread coefficient of each combustible can be calibrated experimentally, and then the flame spread coefficient can be multiplied by the reference spread rate to obtain the baseline spread rate of the corresponding combustible). After determining the combustible type in each voxel using a 3D information model of the target forest area, the flame spread coefficient is obtained based on a pre-set database of combustible distribution and combustion characteristics. This represents an exponential function with base e. This represents the wind speed influence coefficient (dimensionless, can take a value of 0.3). This indicates the wind speed value. This indicates the wind direction value (which can be increased clockwise from 0 degrees due north). This represents the directional angle from the i-th voxel to the j-th voxel. Indicates reference wind speed. , which is the rock mass barrier factor (dimensionless, 0 for non-combustible rock mass, and 1 for the rest). Based on the fire spread rate, the predicted ignition time of other adjacent voxels adjacent to the voxel where the fire source is located is determined as follows: ; in, Let be the predicted ignition time from the i-th voxel to the j-th voxel. The side length of the voxel (for example, it can be set to 0.2 meters, 0.5 meters, 1 meter, etc.). The combustible density within the j-th genus (e.g., wood density ranges from 0.4 to 0.75 g / cm³). 3 The specific value is affected by the tree species, moisture content, and degree of dryness. For example, the density of wood in an air-dried state (moisture content 12%-15%) is often used as a practical indicator; high-density wood (such as hardwood) can reach 0.75 g / cm³. 3 Low-density wood (such as softwood) has a density of approximately 0.4 g / cm³. 3 It can be dynamically set according to the season and region, and the combustible density of herbaceous combustibles can also be set. Let be the density of combustible material within the i-th voxel. Let J be the specific heat capacity of the combustible material in the j-th genus. Let be the ignition point of the combustible material within the j-th genus. For ambient temperature, Let be the calorific value of the combustible material in the j-th voxel (which can be set to a value between 0.1 and 0.3, with a typical value of 0.2). For heat transfer efficiency.

[0031] If the predicted ignition time is less than the preset prediction time window, then the j-th voxel is marked as the target flame spread voxel. It is worth noting that there are often multiple fire sources at a fire scene; for each fire source, a corresponding target flame spread voxel can be obtained, thereby constructing a complete fire spread trend area. For forest fires, existing technologies can also be directly used to determine the fire spread trend.

[0032] Based on all target flame spread voxels, determine the fire spread trend area within a future forecast time window.

[0033] The embodiments of this application improve the accuracy of fire trend area prediction by using voxel division and physical model (spread rate / ignition time) prediction methods, providing advance notice for fire extinguishing decisions and avoiding fire from getting out of control.

[0034] In one possible implementation, based on the three-dimensional coordinates of the fire source space and the fire spread trend area, and combined with the fire extinguishing equipment information corresponding to the fire extinguishing equipment carried by the UAV, a deep reinforcement learning decision model is used to generate target fire extinguishing control commands, including: Based on the three-dimensional coordinates of the fire source space and the fire spread trend area, the coordinate components of the fire source relative to the fire extinguishing equipment (i.e., the difference between coordinates in each dimension; the total three-dimensional coordinates include the difference between the three-dimensional coordinates), the number of target flame spread voxels in the fire spread trend area, and the average fire line spread rate of all target flame spread voxels are determined to obtain the first state data. The highest temperature in the voxel corresponding to the fire source, the current pitch angle of the fire extinguishing equipment nozzle, the current yaw angle of the fire extinguishing equipment nozzle, the current spray pressure of the fire extinguishing equipment nozzle, the current wind speed value, the current wind direction value, the remaining amount of extinguishing agent in the fire extinguishing equipment, and the rate of change of smoke concentration over time are obtained to obtain the second state data. The first state data and the second state data are used together as the target environment state, and the target environment state is used as the input of the deep reinforcement learning decision model to obtain the target fire extinguishing control action output by the deep reinforcement learning decision model; the target fire extinguishing control action includes the spray angle adjustment amount, spray pressure adjustment amount, and fire extinguishing agent type of the fire extinguishing equipment nozzle; The target fire extinguishing control action is converted into the instruction type specified by the fire extinguishing equipment to obtain the target fire extinguishing control instruction.

[0035] Optionally, the deep reinforcement learning decision model can be set to existing models such as DQN (Deep Q-Network) and PPO (Proximal Policy Optimization). These models can be pre-trained, allowing them to be used directly.

[0036] In one possible implementation, the target fire extinguishing control command is issued to the fire extinguishing equipment for fire extinguishing control, and fire extinguishing feedback data is acquired in real time. The deep reinforcement learning decision model is then updated using the fire extinguishing feedback data for the next fire extinguishing control operation, including: The target fire extinguishing control command is sent to the fire extinguishing equipment for fire extinguishing control, and fire extinguishing feedback data is acquired in real time; The execution reward is obtained based on the fire extinguishing feedback data, and the new target environmental status is also obtained at the same time; The target environmental state, target fire suppression control actions, execution rewards, and new target environmental states are all combined as historical data. The deep reinforcement learning decision model is periodically updated using historical data for the next fire suppression control.

[0037] In one possible implementation, the execution reward is obtained based on the fire extinguishing feedback data as follows: ; in, In order to implement the reward, As the first coefficient, As the second coefficient, The third coefficient, It is the fourth coefficient. It is the fifth coefficient. The average temperature drop rate of the fire source is obtained based on the fire extinguishing feedback data. The reduction rate of the target flame spread voxel. The Euclidean distance between the point of impact of the extinguishing agent sprayed by the fire extinguishing equipment and the center point of the voxel where the fire source is located. The rate of consumption of extinguishing agent. The event indicator function is set to +100 when the fire source is completely extinguished and -100 when the fire is out of control.

[0038] For example, This represents the average rate of temperature decrease of the fire source obtained from the fire extinguishing feedback data. This is the weighting coefficient for this item. Let be the temperature of the fire source at time t. Let be the temperature of the fire source at time t-1; , representing the rate of reduction of the target flame spread voxel. This is the weighting coefficient for this item. Let be the number of target flame spread voxels at time t. The number of target flame spread voxels at time t-1; The Euclidean distance between the point of impact of the extinguishing agent sprayed by the fire extinguishing equipment and the center point of the voxel where the fire source is located. This is the weighting coefficient of the reciprocal term of the distance.

[0039] This indicates the rate of extinguishing agent consumption. This negative weighting coefficient is used to encourage the conservation of fire extinguishing agents. The remaining amount of extinguishing agent at time t. The remaining amount of extinguishing agent at time t-1.

[0040] To terminate the event indicator function, when the fire source is completely extinguished (the condition is...). Preset temperature threshold and The value is +100 when the preset smoke concentration threshold is reached, and +100 when the fire is out of control (the judgment condition is...). When the preset voxel count threshold is used, the value is -100, and λ5 = 1.0 is the weight coefficient (dimensionless) of the indicator function. This represents the highest temperature within the voxel containing the fire source. This refers to the smoke concentration.

[0041] Therefore, fire extinguishing feedback data can include various data for calculating performance rewards. To simplify calculations, the coefficients can be normalized and the parameters can be dedimensionalized to ensure accurate data processing.

[0042] The deep reinforcement learning model in this application dynamically adjusts the spray parameters based on the real-time environmental conditions (fire source location, spread rate, equipment status), reducing extinguishing agent consumption and shortening extinguishing time; the safety constraint indication function ensures that there is no water damage to non-fire source targets, and the termination event reward mechanism accelerates fire suppression.

[0043] In one possible implementation, the deep reinforcement learning decision model is periodically updated using historical data, including: At each update cycle, the deep reinforcement learning decision model is updated using historical data to obtain the updated deep reinforcement learning decision model. Also includes: The updated deep reinforcement learning decision model is run in an offline simulator based on historical data. If its reward value is improved by more than 5% compared to the current deep reinforcement learning decision model, the updated deep reinforcement learning decision model replaces the current deep reinforcement learning decision model; otherwise, the current deep reinforcement learning decision model is retained.

[0044] Based on a reward function derived from feedback data (temperature decrease, voxel reduction, landing accuracy, etc.) and an offline simulation verification mechanism, the model iteration efficiency is improved, continuously optimizing fire suppression strategies and adapting to complex environmental changes. The fully automated process from data acquisition to fire suppression execution reduces human intervention; real-time feedback and model updates ensure the system's adaptability, reducing the risk of fire losses in complex forest environments.

[0045] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0046] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0047] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0048] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0049] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0050] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A smart monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs), characterized in that, include: The drone collects visible light image data, infrared thermal imaging data, smoke concentration data, and environmental meteorological data of the target forest area in real time and synchronizes them in time to obtain multimodal sensor data after time synchronization. The multimodal sensor data after time synchronization is input into a pre-trained multi-source information fusion fire identification model for analysis to obtain fire identification results. Based on the fire identification results, fire source localization analysis is triggered to obtain the three-dimensional coordinates of the fire source space. Based on the three-dimensional coordinates of the fire source space, environmental meteorological data, three-dimensional information model of the target forest area, and a pre-set database of combustible material distribution and combustion characteristics, the fire spread trend area in the next prediction time window is predicted. Based on the three-dimensional coordinates of the fire source space and the fire spread trend area, and combined with the fire extinguishing equipment information corresponding to the fire extinguishing equipment carried by the UAV, a deep reinforcement learning decision model is used to generate target fire extinguishing control commands; the target fire extinguishing control commands include the spray angle adjustment control command, the spray pressure adjustment control command, and the fire extinguishing agent type selection command of the fire extinguishing equipment nozzle. The target fire extinguishing control command is sent to the fire extinguishing equipment for fire extinguishing control, and fire extinguishing feedback data is acquired in real time. The deep reinforcement learning decision model is updated using the fire extinguishing feedback data to carry out the next fire extinguishing control.

2. The intelligent monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, By using drones to collect visible light image data, infrared thermal imaging data, smoke concentration data, and environmental meteorological data of the target forest area in real time and synchronizing them with time, multimodal sensor data after time synchronization is obtained, including: During each data acquisition cycle, a rigidly connected binocular vision system and infrared thermal imaging device on the drone are used to collect visible light image data and infrared thermal imaging data. At the end of each data collection cycle, smoke concentration data is collected by the smoke concentration sensor on the drone and environmental meteorological data is collected by the environmental meteorological station deployed on the ground; the environmental meteorological data includes wind direction, wind speed, ambient temperature and ambient humidity. Visible light image data, infrared thermal imaging data, smoke concentration data, and environmental meteorological data collected at the same time are combined as multimodal sensor data after time synchronization.

3. The intelligent monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The time-synchronized multimodal sensor data is input into a pre-trained multi-source information fusion fire identification model for analysis to obtain fire identification results. Based on the fire identification results, fire source localization analysis is triggered to obtain the three-dimensional spatial coordinates of the fire source, including: The time-synchronized multimodal sensor data is input into a pre-trained multi-source information fusion fire identification model for analysis to obtain the classification probability distribution; the pre-trained multi-source information fusion fire identification model adopts a hybrid structure of convolutional neural network and long short-term memory network; Based on the classification probability distribution, the category with the highest probability is determined as the fire identification result; wherein, the fire identification result includes whether a fire exists or not. If the fire identification result indicates that there is no fire, then the monitoring of multimodal sensor data after time synchronization for the next data acquisition cycle will continue. If the fire identification result indicates that a fire exists, then the fire source location analysis is triggered to obtain the three-dimensional spatial coordinates of the fire source.

4. The intelligent monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) according to claim 2, characterized in that, Triggering fire source localization analysis yields the three-dimensional spatial coordinates of the fire source, including: Based on infrared thermal imaging data, the pixel coordinates of the first fire source center and the pixel coordinates of the second fire source center are determined from two visible light image data measured by a binocular vision system. Based on the pixel coordinates of the first fire source center and the pixel coordinates of the second fire source center, obtain the disparity data between the two visible light image data measured by the binocular vision system; The depth of the fire source in the camera coordinate system is obtained based on the parallax data between two visible light image data measured by the binocular vision system. Based on the pixel coordinates of the first or second fire source center, and combined with depth and camera extrinsic matrix, the pixel coordinates of the fire source center are transformed from the camera coordinate system to the world coordinate system to obtain the three-dimensional coordinates of the fire source space.

5. The intelligent monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The pre-set database of combustible distribution and combustion characteristics includes the types of combustibles, calorific value, ignition point, and flame spread coefficient within the target forest area.

6. The intelligent monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, Based on the three-dimensional coordinates of the fire source space, environmental meteorological data, a three-dimensional information model of the target forest area, and a pre-set database of combustible material distribution and combustion characteristics, the fire spread trend area within a future prediction time window is predicted, including: The target forest area is divided into voxels, and based on the pre-set database of combustible material distribution and combustion characteristics, the fire line spread rate between the voxel where the fire source is located and other adjacent voxels is determined as follows: ； in, This represents the fire line spread rate from the i-th voxel to the j-th voxel. The i-th voxel represents the voxel where the fire source is located, which is determined based on the three-dimensional coordinates of the fire source space. The j-th voxel is any other adjacent voxel that is adjacent to the voxel where the fire source is located. The baseline spread rate under windless conditions is determined based on the flame spread coefficient. After determining the type of combustible material in each voxel through a three-dimensional information model of the target forest area, the flame spread coefficient is obtained based on a pre-set database of combustible material distribution and combustion characteristics. This represents an exponential function with base e. Indicates the wind speed influence coefficient. This indicates the wind speed value. Indicates wind direction value. This represents the directional angle from the i-th voxel to the j-th voxel. Indicates reference wind speed. Rock mass barrier factor; Based on the fire spread rate, the predicted ignition time of other adjacent voxels adjacent to the voxel where the fire source is located is determined as follows: ； in, Let be the predicted ignition time from the i-th voxel to the j-th voxel. Let the voxel side length be , Let be the density of combustible material within the j-th genus. Let be the density of combustible material within the i-th voxel. Let J be the specific heat capacity of the combustible material in the j-th genus. Let be the ignition point of the combustible material within the j-th genus. For ambient temperature, Let be the calorific value of the combustible material in the j-th genus. For heat transfer efficiency; If the predicted ignition time is less than the preset prediction time window, then the j-th voxel is marked as the target flame spread voxel. Based on all target flame spread voxels, determine the fire spread trend area within a future forecast time window.

7. The intelligent monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, Based on the three-dimensional coordinates of the fire source space and the fire spread trend area, and combined with the fire extinguishing equipment information corresponding to the fire extinguishing equipment carried by the UAV, a deep reinforcement learning decision model is used to generate target fire extinguishing control commands, including: Based on the three-dimensional coordinates of the fire source space and the fire spread trend area, the coordinate components of the fire source relative to the fire extinguishing equipment, the number of target flame spread voxels in the fire spread trend area and the average fire line spread rate of all target flame spread voxels are determined to obtain the first state data. The highest temperature in the voxel corresponding to the fire source, the current pitch angle of the fire extinguishing equipment nozzle, the current yaw angle of the fire extinguishing equipment nozzle, the current spray pressure of the fire extinguishing equipment nozzle, the current wind speed value, the current wind direction value, the remaining amount of extinguishing agent in the fire extinguishing equipment, and the rate of change of smoke concentration over time are obtained to obtain the second state data. The first state data and the second state data are used together as the target environment state, and the target environment state is used as the input of the deep reinforcement learning decision model to obtain the target fire extinguishing control action output by the deep reinforcement learning decision model; the target fire extinguishing control action includes the spray angle adjustment amount, spray pressure adjustment amount, and fire extinguishing agent type of the fire extinguishing equipment nozzle; The target fire extinguishing control action is converted into the instruction type specified by the fire extinguishing equipment to obtain the target fire extinguishing control instruction.

8. The intelligent monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) according to claim 7, characterized in that, The target fire extinguishing control command is issued to the fire extinguishing equipment for fire extinguishing control, and fire extinguishing feedback data is acquired in real time. The deep reinforcement learning decision model is updated using the fire extinguishing feedback data for the next fire extinguishing control, including: The target fire extinguishing control command is sent to the fire extinguishing equipment for fire extinguishing control, and fire extinguishing feedback data is acquired in real time; The execution reward is obtained based on the fire extinguishing feedback data, and the new target environmental status is also obtained at the same time; The target environmental state, target fire suppression control actions, execution rewards, and new target environmental states are all combined as historical data. The deep reinforcement learning decision model is periodically updated using historical data for the next fire suppression control.

9. The intelligent monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) according to claim 8, characterized in that, The execution reward is obtained based on the aforementioned fire extinguishing feedback data: ； in, In order to implement the reward, As the first coefficient, As the second coefficient, The third coefficient, It is the fourth coefficient. It is the fifth coefficient. The average temperature drop rate of the fire source is obtained based on the fire extinguishing feedback data. The reduction rate of the target flame spread voxel. The Euclidean distance between the point of impact of the extinguishing agent sprayed by the fire extinguishing equipment and the center point of the voxel where the fire source is located. The rate of consumption of extinguishing agent. The event indicator function is set to +100 when the fire source is completely extinguished and -100 when the fire is out of control.

10. The intelligent monitoring and processing method for forest fire prevention based on unmanned aerial vehicles (UAVs) according to claim 8, characterized in that, The deep reinforcement learning decision model is periodically updated using historical data, including: At each update cycle, the deep reinforcement learning decision model is updated using historical data to obtain the updated deep reinforcement learning decision model. Also includes: The updated deep reinforcement learning decision model is run in an offline simulator based on historical data. If its reward value is improved by more than 5% compared to the current deep reinforcement learning decision model, the updated deep reinforcement learning decision model replaces the current deep reinforcement learning decision model; otherwise, the current deep reinforcement learning decision model is retained.

Images