A forest initial fire rapid identification method and system based on multi-sensor time sequence fusion

Abstract

The application discloses a forest initial fire rapid identification method and system based on multi-sensor time sequence fusion. The method synchronously collects thermal infrared sensor data and visible light sensor data based on a unified time reference, calculates a time sequence difference value between the thermal infrared signal and the visible light smoke signal, and adaptively reversely adjusts the weight distribution of the two sensors in the fusion decision according to the numerical range of the time sequence difference value. When the time sequence difference value is greater than a first preset threshold, the fusion weight of the thermal infrared sensor is increased. When the time sequence difference value is less than a second preset threshold, the fusion weight of the visible light sensor is increased. The application ensures that the multi-sensor fusion decision is always more sensitive to the current stage in different combustion stages, so that effective sensing in the whole combustion stage is realized.

Description

Technical Field

[0001] This invention relates to the field of forest fire monitoring and early warning technology, and in particular to a method and system for rapid identification of initial forest fires based on multi-sensor time-series fusion. Background Technology

[0002] Forest fires are characterized by their suddenness, destructiveness, and rapid spread. Rapid and accurate identification in the initial fire stage is crucial for early detection and suppression. With the continuous development of sensor technology, multi-sensor joint monitoring schemes based on space-based satellites, airborne drones, and ground-based video surveillance are gradually becoming the mainstream technical approach for forest fire detection. The basic idea of ​​multi-sensor fusion is to utilize the complementarity of different sensors in their detection dimensions (thermal infrared sensors are sensitive to high-temperature heat sources, and visible light sensors are sensitive to smoke characteristics) to improve the accuracy and robustness of detection through comprehensive judgment.

[0003] In existing multi-sensor fusion detection schemes, the weight allocation among sensors typically employs a fixed configuration strategy. For example, Chinese patent CN108537202A uses a unidirectional triggering logic of "smoke sensor triggering → infrared verification," while Chinese patent CN102693603B uses a fixed serial process of "visible light pre-detection → thermal imaging verification → visible light verification." These methods share the common characteristic that the weights of each sensor in the fusion decision are pre-set and fixed, and do not dynamically adjust with changes in the actual development stage of the fire.

[0004] However, forest fires undergo significant evolutionary stages from ignition to spread: the ignition stage is characterized by localized high temperatures and intense heat radiation; the stable combustion stage features both open flames and smoke; and the decay or smoldering stage is characterized by a sharp drop in temperature, little or no visible smoke. The detectability of thermal infrared signals and visible smoke signals differs fundamentally at different combustion stages. Using a fixed sensor weight configuration (e.g., always relying on thermal infrared signals as the primary criterion) may be effective during the ignition and stable combustion stages, but during the smoldering stage—when thermal infrared signals have significantly attenuated and temperatures are only 5°C to 50°C higher than the ambient background, while some pyrolysis smoke may still be captured by visible light sensors—a fixed weight configuration will severely reduce the system's ability to detect the smoldering stage, leading to missed detections. Summary of the Invention

[0005] In view of the above-mentioned shortcomings in the current field of forest fire monitoring and early warning technology, the present invention provides a method and system for rapid identification of initial forest fires based on multi-sensor time-series fusion. This method and system ensures that the sensor more sensitive to the current stage always has a dominant weight in the fusion decision under different combustion stages, thereby achieving effective perception of the entire combustion stage.

[0006] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0007] A rapid forest fire identification method based on multi-sensor temporal fusion includes:

[0008] Data from thermal infrared sensors and visible light sensors are collected synchronously using a unified time reference.

[0009] Calculate the time difference between the moment when the thermal infrared sensor first identifies a stable high-temperature anomaly and the moment when the visible light sensor first identifies a discernible smoke signal;

[0010] Based on the numerical range of the time difference, the weight allocation of the thermal infrared sensor data and the visible light sensor data in the fusion decision is adaptively adjusted in reverse; when the time difference is greater than a first preset threshold, the fusion weight of the thermal infrared sensor data is increased, and when the time difference is less than a second preset threshold, the fusion weight of the visible light sensor data is increased.

[0011] The thermal infrared sensor data and the visible light sensor data are fused and judged according to the adjusted weight allocation, and the fire identification result is output.

[0012] According to one aspect of the present invention, the moment when the thermal infrared sensor first identifies a stable high-temperature anomaly is determined by the following conditions: the temperature value is greater than a preset threshold, and the anomaly persists for N consecutive frames, where N ≥ 3, and interference from solar reflection and sensor noise is excluded; the moment when the visible light sensor first identifies a recognizable smoke signal is determined by the following conditions: the smoke plume area is greater than a preset number of pixels, and the smoke plume persists for more than M consecutive frames, where M ≥ 3.

[0013] According to one aspect of the present invention, the first preset threshold is a positive number and the second preset threshold is a negative number; when the timing difference is greater than the first preset threshold, it is determined to be the ignition period or the stable combustion period; when the timing difference is less than 0 seconds or the absolute value of the timing difference is less than 10 seconds, it is determined to be the decay period or the smoldering period.

[0014] According to one aspect of the present invention, during the ignition period or stable combustion period, the weight of the thermal infrared sensor is 0.5 to 0.7, and the weight of the visible light sensor is 0.3 to 0.5; during the decay period or smoldering period, the weight of the visible light sensor is 0.5 to 0.6, and the weight of the thermal infrared sensor is 0.4 to 0.5.

[0015] According to one aspect of the present invention, it further includes: when it is determined to be the ignition period or the stable combustion period, controlling the UAV to maintain a first flight altitude to perform high-altitude hovering monitoring; when it is determined to be the decay period or the smoldering period, controlling the UAV to switch to a second flight altitude to perform low-altitude grid scanning, wherein the low-altitude grid scanning is used to search for low-temperature thermal anomaly areas with temperatures 5°C to 50°C higher than the ambient background and lower than 500°C.

[0016] According to one aspect of the present invention, a three-level closed-loop verification process for smoldering point is also included:

[0017] The first level of verification involves the drone identifying suspected thermal anomalies in low-altitude scanning mode and recording their coordinates and timestamps.

[0018] The second level of verification involves replaying video surveillance footage near the coordinates, recordings within a preset time window before and after the timestamp, and determining the presence of smoke features based on the plume area and the existence of consecutive frames; if present, it is marked as a candidate for smoldering with smoke, and if absent, it is marked as a candidate for smoldering without smoke.

[0019] The third level of verification involves dispatching ground personnel with thermal imagers and flue gas detectors to the coordinates for on-site measurement, confirming the true smoldering point and recording the type of combustible material, and generating a smoldering point sample database with truth value labels.

[0020] According to one aspect of the present invention, the preset time window is 2 minutes before and after the timestamp; each record in the smoldering point sample database includes at least: smoldering point number, coordinates, surface temperature, carbon monoxide concentration, plume visibility label, and combustible type.

[0021] According to one aspect of the invention, it further includes:

[0022] Extended Kalman filtering is used for asynchronous incremental fusion of multiple sensors to construct a system state vector that includes fire probability, fire location, fire area and confidence level;

[0023] When any sensor data arrives asynchronously at its own sampling frequency, the extended Kalman filter is immediately executed for one-step prediction and measurement update to continuously output the optimal estimate of the fire status without waiting for all sensor data to arrive.

[0024] When the posterior fusion confidence exceeds a preset threshold, it is determined as a fire and an early warning is triggered.

[0025] According to one aspect of the invention, it further includes constructing a binary empirical model, the binary empirical model specifically comprising:

[0026] The fire line position is extracted frame by frame from the thermal infrared video obtained by the thermal infrared sensor. The fire line position is overlaid with the digital surface model to calculate the slope value corresponding to the fire head position, and the displacement of the fire head position at adjacent time points is calculated to obtain the instantaneous spread speed.

[0027] The nonlinear least squares method is used to fit the curve of the spread rate and the slope value to establish an empirical model of the spread rate as a function of the slope.

[0028] In addition, instantaneous wind speed data are collected simultaneously to establish a binary empirical model in which the spread speed varies with both slope and wind speed.

[0029] A rapid forest fire identification system based on multi-sensor time-series fusion includes:

[0030] The data synchronization acquisition module synchronously acquires data from the thermal infrared sensor and the visible light sensor using a unified time reference.

[0031] The timing difference calculation module calculates the timing difference between the moment when the thermal infrared sensor first identifies a stable high-temperature anomaly and the moment when the visible light sensor first identifies a recognizable smoke signal.

[0032] The weighted adaptive scheduling module adaptively and inversely adjusts the weight allocation of the thermal infrared sensor data and the visible light sensor data in the fusion decision based on the numerical range of the time difference.

[0033] The output results module performs a fusion decision on the thermal infrared sensor data and the visible light sensor data according to the adjusted weight allocation, and outputs the fire identification result.

[0034] Advantages of this invention: Through the above technical solution, this invention can adaptively and inversely adjust the weight allocation of each sensor in the fusion decision based on the time difference between thermal infrared and visible light signals. This ensures that the sensor more sensitive to the current stage always has a dominant weight in the multi-sensor fusion decision at different combustion stages, thereby achieving effective perception throughout the entire combustion process. Specifically:

[0035] 1. By quantifying the reversal pattern of timing differences in different combustion stages, a decision-making basis is provided for the effective perception of the entire combustion stage.

[0036] This invention statistically analyzes the temporal difference distribution characteristics between thermal infrared signals and visible smoke signals during the ignition, stable combustion, and smoldering phases, revealing a differentiated pattern in the order of appearance of thermal and smoke signals at different combustion stages—the thermal signal precedes the smoke signal during the ignition and stable combustion phases, while the smoke signal precedes the thermal signal during the smoldering phase. Based on this understanding, the system can accurately determine the current combustion stage according to the real-time temporal difference, thereby ensuring the selection of the correct sensitive sensor at different stages. This provides a physical basis for the core objective of "effective perception of all combustion stages."

[0037] 2. By using the timing difference as an active control signal to adjust the sensor weight allocation in reverse, the mechanism of "stage-sensitive sensors occupying the dominant weight" in the core effect is directly realized.

[0038] This invention abandons the passive mode of fixed sensor weight configuration or fixed cascading triggering in existing technologies. Instead, it dynamically adjusts the weight ratio of the thermal infrared sensor and the visible light sensor in the fusion decision based on the numerical range of the real-time time difference. The fusion weight of the thermal infrared sensor is increased during the ignition or stable combustion phase, while the fusion weight of the visible light sensor is increased during the smoldering phase. This mechanism ensures that the sensor more sensitive to the current combustion stage always dominates the fusion decision, directly achieving the methodological guarantee of "effective perception throughout the entire combustion stage" in the core effect.

[0039] 3. By establishing a drone flight mode switching mechanism based on combustion stage determination, the quality of core effects was enhanced from the data source.

[0040] After determining the current combustion stage, this invention adaptively controls the UAV to switch to the corresponding flight altitude—maintaining a higher flight altitude for high-altitude hovering monitoring during the ignition or stable combustion phase, and switching to a lower flight altitude for low-altitude grid scanning during the smoldering phase to search for areas of low-temperature thermal anomalies. This mechanism ensures that the data quality acquired by sensors more sensitive to the current stage is further improved due to the adaptive adjustment of the UAV's observation attitude, guaranteeing the reliability of the judgment that "stage-sensitive sensors have a dominant weight" in the core effect from the data source.

[0041] 4. Through a three-level closed-loop verification process for smoldering point, the effectiveness of the core effect in the most challenging smoldering stage was verified and reinforced.

[0042] This invention addresses the unique, subtle thermal anomaly characteristics of the smoldering stage by constructing a three-tiered closed-loop verification system: "UAV infrared detection—video playback—ground verification." This system uses UAV low-altitude scanning to identify suspected thermal anomalies, replays video monitoring to determine smoke characteristics, and ground equipment to measure true values ​​on-site, generating a database of smoldering point samples with true value labels. This process demonstrates, from the terminal output perspective, that the core effect of "effective perception throughout the entire combustion stage" is indeed achieved during the most challenging smoldering stage, and that the perception results are supported by high-precision true values.

[0043] 5. By extending the Kalman filter and using asynchronous incremental fusion, the response time of the core effects has been improved.

[0044] This invention employs an extended Kalman filter to construct a system state vector containing fire probability, fire location, fire area, and confidence level. When any sensor data arrives asynchronously at its own sampling frequency, a prediction and measurement update are performed immediately, without waiting for all sensor data to arrive. This streaming fusion architecture enables the rapid output of the determination result of "stage-sensitive sensors having dominant weight" in the core effect, ensuring the timeliness of effective perception throughout the entire combustion stage.

[0045] 6. By using the slope-spread speed response model, the application scope of the core effect is extended from fire identification to fire prediction.

[0046] This invention extracts the fire line position frame by frame from UAV thermal infrared video, overlays it with a digital terrain model to calculate the slope value corresponding to the fire front position, and establishes an empirical model of the spread speed changing with the slope. Furthermore, it introduces instantaneous wind speed to establish a binary empirical model. This mechanism enables the system to not only achieve the core effect of "effective perception throughout the entire combustion stage," but also to predict the fire spread trend. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating a method for rapid identification of initial forest fires based on multi-sensor temporal fusion, as described in Embodiment 1 of the present invention.

[0049] Figure 2 This is a flowchart illustrating a method for rapid identification of initial forest fires based on multi-sensor temporal fusion, as described in Embodiment 2 of the present invention.

[0050] Figure 3This is a flowchart illustrating a method for rapid identification of initial forest fires based on multi-sensor temporal fusion, as described in Embodiment 3 of the present invention.

[0051] Figure 4 This is a schematic diagram of the structure of a rapid forest fire identification system based on multi-sensor time-series fusion as described in Embodiment 4 of the present invention;

[0052] Figure 5 This is a schematic diagram of the complete structure of a rapid forest fire identification system based on multi-sensor time-series fusion as described in Embodiment 5 of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example 1

[0055] This embodiment uses a multi-source data fusion field planned burn test as a specific example to further illustrate the invention. The test observed a 0.43-hectare planned burn area from 11:45 to 16:29, employing a five-layer synchronous data acquisition system consisting of space-based satellites (Sentinel 2 multispectral, VIIRS / S-NPP thermal anomaly), airborne UAVs (a quadcopter UAV equipped with a thermal infrared camera, and a multirotor UAV equipped with a multispectral camera), and ground-based equipment (1080P video surveillance, D-RTK 2 precise positioning, FLIR T540 handheld thermal imager, and Kestrel 5500 portable weather station).

[0056] like Figure 1 As shown, a rapid identification method for initial forest fires based on multi-sensor temporal fusion includes:

[0057] Step S1: Synchronously acquire thermal infrared sensor data and visible light sensor data using a unified time reference.

[0058] In step S1 of this embodiment, the BeiDou Navigation Satellite System time is used as the unified time reference to synchronize and calibrate all sensors, ensuring that the timestamp deviation of each data source is less than 1 second. Space-based data is acquired via the VIIRS / S-NPP satellite receiving station, obtaining active fire point products with a spatial resolution of 375 meters, and simultaneously receiving Sentinel-2 multispectral data. Airborne data is simultaneously acquired via a thermal infrared camera (a specific implementation of the thermal infrared sensor, with a resolution of 640×512 pixels and a NETD better than 50mK) and a multispectral camera (used to collect visible light sensor data) mounted on a UAV. The UAV performs observation tasks over the fire site according to a preset flight path. Ground-based data is continuously acquired via a 1080P high-definition video monitoring device (frame rate 25 frames / second) installed at the observation points, and centimeter-level spatial coordinate references are provided for each observation point using a D-RTK 2 precision positioning device. Simultaneously, meteorological parameters such as ambient temperature and wind speed are recorded synchronously using a Kestrel 5500 portable weather station. All data is merged into the data storage unit after being identified by the BeiDou time as a unified timestamp, for use in subsequent steps.

[0059] In step S1 of this embodiment, all sensors complete time synchronization calibration before data acquisition to ensure that the timestamps of each data source have a unified comparison benchmark when calculating the time difference in step S2.

[0060] Step S2: Calculate the time difference between the moment when the thermal infrared sensor first identifies a stable high-temperature anomaly and the moment when the visible light sensor first identifies a recognizable smoke signal.

[0061] In this embodiment, the moment when the thermal infrared sensor first identifies a stable high-temperature anomaly is determined by the following conditions: the temperature value is greater than a preset threshold, and the anomaly persists for N consecutive frames (N≥3), excluding solar reflection and sensor noise interference. Specifically, temperature values ​​are extracted and high-temperature regions are segmented for each frame of thermal infrared image. When a temperature value greater than a preset threshold (150℃ in this embodiment, but other values ​​can be preset according to actual conditions) is detected, and the high-temperature region persists for N consecutive frames (generally N≥3, N is 3 in this embodiment), excluding solar reflection and sensor noise interference, this moment is recorded as the moment T_thermal when the thermal infrared sensor first identifies a stable high-temperature anomaly.

[0062] In step S2 of this embodiment, the multi-source synchronous data acquired in step S1 is analyzed frame by frame to monitor the detection trigger times of the thermal infrared channel and the visible light channel, respectively. The data sources of the thermal infrared channel include at least one of space-based VIIRS / S-NPP thermal anomaly data, airborne UAV thermal infrared images, and ground-based handheld thermal imager images. The detection logic of the thermal infrared channel is illustrated below using airborne UAV thermal infrared images as an example. This detection logic is also applicable to thermal infrared sensor data from space-based and ground-based platforms.

[0063] In this embodiment, the moment when the visible light sensor first identifies a identifiable smoke signal is determined by the following conditions: the smoke plume area is greater than a preset number of pixels, and it exists for more than M consecutive frames, where M ≥ 3. Specifically, smoke features are extracted for each frame of visible light image. When a smoke plume area is detected to be greater than a preset number of pixels (10 pixels in this embodiment) and the smoke plume region exists continuously in M ​​consecutive frames (M is 3 in this embodiment), this moment is recorded as the moment T_smoke when the visible light sensor first identifies a identifiable smoke signal.

[0064] In this embodiment, the system continuously monitors the data streams of both the thermal infrared and visible light channels after startup. During the experiment, the planned ignition time was used as the zero point for data analysis. The system was already in continuous monitoring before the ignition time, and continuously recorded the detection triggering status of each channel from the ignition time onwards. The detection logic of the thermal infrared channel is as follows: Temperature values ​​are extracted and high-temperature regions are segmented for each frame of thermal infrared image. When a temperature value greater than a preset threshold (150℃ in this embodiment, but other values ​​can be preset according to actual conditions) is detected, and the high-temperature region persists in N consecutive frames (generally N≥3, N is 3 in this embodiment), and solar reflection and sensor noise interference are excluded, this moment is recorded as the first time the thermal infrared sensor identifies a stable high-temperature anomaly point, T_thermal, and the coordinates of this point are marked. The detection logic of the visible light channel is as follows: smoke features are extracted for each frame of visible light image. When the detected smoke plume area is greater than the preset number of pixels (10 pixels in this embodiment) and the smoke plume area continues to exist in the continuous M frames (M is 3 in this embodiment) of the image, the moment is recorded as the moment T_smoke when the visible light sensor first recognizes the identifiable smoke signal, and the geometric center coordinates of the smoke plume area are marked.

[0065] By comparing the thermal infrared triggering time and the visible light triggering time near the same spatial location, the timing difference Delta-T = T_smoke - T_thermal is calculated between the two. In a typical detection sequence of this embodiment, T_thermal is first triggered approximately 92 seconds after ignition (detecting a local high temperature of approximately 180°C), and T_smoke is first triggered approximately 161 seconds after ignition (detecting a white, thin columnar plume). Based on this, the timing difference Delta-T of this detection sequence is calculated to be approximately 69 seconds.

[0066] Step S3: Based on the numerical range of the time difference, adaptively and inversely adjust the weight allocation of the thermal infrared sensor and the visible light sensor in the fusion decision; wherein, when the time difference is greater than a first preset threshold, increase the fusion weight of the thermal infrared sensor, and when the time difference is less than a second preset threshold, increase the fusion weight of the visible light sensor.

[0067] In step S3 of this embodiment, adaptive weight adjustment is performed based on the time difference Delta-T calculated in step S2.

[0068] The system internally incorporates decision logic based on a priori time difference database. In this embodiment, the first preset threshold is positive, and the second preset threshold is negative. When the calculated time difference value is greater than the first preset threshold—in this embodiment, the time difference values ​​are all positive and relatively large in multiple detection sequences of the ignition and stable combustion periods—it is determined that the system is currently in the ignition or stable combustion period, and a first weight allocation strategy is executed: the fusion weight of the thermal infrared sensor is increased to be higher than that of the visible light sensor, so that the thermal infrared sensor dominates the fusion decision. Specifically, in this embodiment, during the ignition or stable combustion period, the fusion weight of the thermal infrared sensor is set to a value in the range of 0.5 to 0.7, and the fusion weight of the visible light sensor is set to a value in the range of 0.3 to 0.5, with the sum of their weights being 1.

[0069] When the timing difference is detected to turn negative or its absolute value decreases to less than the third preset threshold—in this embodiment, the timing difference turns negative or approaches zero in multiple detection sequences during the decay and smoldering phases—it is determined that the current phase is either decay or smoldering, and a second weight allocation strategy is executed: the fusion weight of the visible light sensor is increased to a higher value than that of the thermal infrared sensor, making the visible light sensor dominant in the fusion decision. Specifically, in this embodiment, during the decay or smoldering phase, the fusion weight of the visible light sensor is set to a value between 0.5 and 0.6, and the fusion weight of the thermal infrared sensor is set to a value between 0.4 and 0.5, with the sum of their weights being 1. In this embodiment, based on the real-time timing difference continuously calculated in step S2, the above weight allocation strategy is dynamically executed in each combustion stage to ensure that the sensor more sensitive to the current stage always occupies a dominant weight in the fusion decision.

[0070] In this embodiment, the first preset threshold is a positive number, and the second preset threshold is a negative number. When the time difference is greater than 30 seconds, it is determined to be the ignition period or the stable combustion period; when the time difference is less than 0 seconds or the absolute value of the time difference is less than 10 seconds, it is determined to be the decay period or the smoldering period.

[0071] Step S4: Based on the adjusted weight allocation, perform a fusion decision on the thermal infrared sensor data and the visible light sensor data, and output the fire identification result.

[0072] In step S4 of this embodiment, based on the weight allocation adjusted in step S3, the thermal infrared sensor data and the visible light sensor data are fused and judged to output the fire identification result. Specifically, the system performs a weighted calculation on the detection confidence scores extracted from the two sensor data according to the adjusted weight allocation. When the overall confidence score after weighted fusion exceeds the preset alarm threshold, the system outputs a fire identification result that includes the fire probability value and the coordinates of the fire location.

[0073] Advantages of implementing this invention:

[0074] The feasibility of the method of the present invention in actual forest fire scenarios was verified through field planned burning experiments.

[0075] Using a 0.43-hectare planned burn site in Xinping County, Yuxi City, Yunnan Province, on April 18, 2026, as the experimental platform, and a five-layer synchronous data acquisition system (air, ground, and space) as the data foundation, the entire observation process from ignition and combustion to stable combustion and smoldering was fully reproduced from 11:45 to 16:29. In Example 1, the system successfully detected the typical characteristic of a positive time difference during the ignition and stable combustion periods, and successfully detected the stage transition signal where the time difference turned negative or approached zero during the smoldering period. Based on this, the system automatically triggered the UAV to switch from a hovering mode at an altitude of 150-200 meters to a low-altitude grid scanning mode at a height of no more than 50 meters within 2 minutes. This implementation process demonstrates that the method of the present invention can adaptively adjust the detection strategy between different combustion stages, and has engineering feasibility in real forest fire scenarios.

[0076] Example 2

[0077] This embodiment uses a multi-source data fusion field planned burn test as a specific example to further illustrate the invention. The test observed a 0.43-hectare planned burn area from 11:45 to 16:29, employing a five-layer synchronous data acquisition system consisting of space-based satellites (Sentinel 2 multispectral, VIIRS / S-NPP thermal anomaly), airborne UAVs (a quadcopter UAV equipped with a thermal infrared camera, and a multirotor UAV equipped with a multispectral camera), and ground-based equipment (1080P video surveillance, D-RTK 2 precise positioning, FLIR T540 handheld thermal imager, and Kestrel 5500 portable weather station).

[0078] The difference between this embodiment and Embodiment 1 is that, after step S4 in Embodiment 1, this embodiment further performs UAV flight altitude switching, three-level closed-loop verification of smoldering point, and extended Kalman filter incremental fusion to achieve refined confirmation of suspected fire points and adaptive perception of the entire combustion stage.

[0079] like Figure 2 As shown, a rapid identification method for initial forest fires based on multi-sensor temporal fusion includes:

[0080] Step S1: Synchronously acquire thermal infrared sensor data and visible light sensor data using a unified time reference.

[0081] In step S1 of this embodiment, the BeiDou Navigation Satellite System time is used as the unified time reference to synchronize and calibrate all sensors, ensuring that the timestamp deviation of each data source is less than 1 second. Space-based data is acquired via the VIIRS / S-NPP satellite receiving station, obtaining active fire point products with a spatial resolution of 375 meters, and simultaneously receiving Sentinel-2 multispectral data. Airborne data is simultaneously acquired via a thermal infrared camera (a specific implementation of the thermal infrared sensor, with a resolution of 640×512 pixels and a NETD better than 50mK) and a multispectral camera (used to collect visible light sensor data) mounted on a UAV. The UAV performs observation tasks over the fire site according to a preset flight path. Ground-based data is continuously acquired via a 1080P high-definition video monitoring device (frame rate 25 frames / second) installed at the observation points, and centimeter-level spatial coordinate references are provided for each observation point using a D-RTK 2 precision positioning device. Simultaneously, meteorological parameters such as ambient temperature and wind speed are recorded synchronously using a Kestrel 5500 portable weather station. All data is merged into the data storage unit after being identified by the BeiDou time as a unified timestamp, for use in subsequent steps. In practical applications, there can be various devices that can carry thermal infrared cameras and multispectral cameras, including but not limited to the drones used to carry thermal infrared cameras and multispectral cameras as described in this embodiment. Theoretically, manned aircraft can also carry thermal infrared cameras and multispectral cameras, but considering the cost, drones are generally used to carry thermal infrared cameras and multispectral cameras.

[0082] In step S1 of this embodiment, all sensors complete time synchronization calibration before data acquisition to ensure that the timestamps of each data source have a unified comparison benchmark when calculating the time difference in step S2.

[0083] Step S2: Calculate the time difference between the moment when the thermal infrared sensor first identifies a stable high-temperature anomaly and the moment when the visible light sensor first identifies a recognizable smoke signal.

[0084] In step S2 of this embodiment, the multi-source synchronous data acquired in step S1 is analyzed frame by frame to monitor the detection trigger times of the thermal infrared channel and the visible light channel, respectively. The data sources of the thermal infrared channel include at least one of space-based VIIRS / S-NPP thermal anomaly data, airborne UAV thermal infrared images, and ground-based handheld thermal imager images. The detection logic of the thermal infrared channel is illustrated below using airborne UAV thermal infrared images as an example. This detection logic is also applicable to thermal infrared sensor data from space-based and ground-based platforms.

[0085] In this embodiment, the system continuously monitors the data streams of both the thermal infrared and visible light channels after startup. During the experiment, the planned ignition time was used as the zero point for data analysis. The system was already in continuous monitoring before the ignition time, and continuously recorded the detection triggering status of each channel from the ignition time onwards. The detection logic of the thermal infrared channel is as follows: Temperature values ​​are extracted and high-temperature regions are segmented for each frame of thermal infrared image. When a temperature value greater than a preset threshold (150℃ in this embodiment, but other values ​​can be preset according to actual conditions) is detected, and the high-temperature region persists in N consecutive frames (generally N≥3, N is 3 in this embodiment), and solar reflection and sensor noise interference are excluded, this moment is recorded as the first time the thermal infrared sensor identifies a stable high-temperature anomaly point, T_thermal, and the coordinates of this point are marked. The detection logic of the visible light channel is as follows: Smoke features are extracted for each frame of visible light image. When the detected smoke plume area is greater than the preset number of pixels (the preset number of pixels is generally greater than or equal to 9, and 10 pixels in this embodiment) and the smoke plume area continues to exist in M ​​consecutive frames (M≥3 in this embodiment, and M is 3 in this embodiment), the moment is recorded as the moment T_smoke when the visible light sensor first recognizes the identifiable smoke signal, and the geometric center coordinates of the smoke plume area are marked.

[0086] By comparing the thermal infrared triggering time and the visible light triggering time near the same spatial location, the timing difference Delta-T = T_smoke - T_thermal is calculated between the two. In a typical detection sequence of this embodiment, T_thermal is first triggered approximately 92 seconds after ignition (detecting a local high temperature of approximately 180°C), and T_smoke is first triggered approximately 161 seconds after ignition (detecting a white, thin columnar plume). Based on this, the timing difference Delta-T of this detection sequence is calculated to be approximately 69 seconds.

[0087] Step S3: Based on the numerical range of the time difference, adaptively and inversely adjust the weight allocation of the thermal infrared sensor and the visible light sensor in the fusion decision; wherein, when the time difference is greater than a first preset threshold, increase the fusion weight of the thermal infrared sensor, and when the time difference is less than a second preset threshold, increase the fusion weight of the visible light sensor.

[0088] In step S3 of this embodiment, adaptive weight adjustment is performed based on the time difference Delta-T calculated in step S2.

[0089] In this embodiment, the first preset threshold is a positive number, and the second preset threshold is a negative number. Specifically, the first preset threshold is 30 seconds, the second preset threshold is 0 seconds, and the third preset threshold is 10 seconds. When the time difference is greater than the first preset threshold, it is determined to be the ignition period or the stable combustion period. When the time difference is less than the second preset threshold or the absolute value of the time difference is less than the third preset threshold, it is determined to be the decay period or the smoldering period. Internally configured with determination logic based on a priori time difference database. In this embodiment, the first preset threshold is a positive number, and the second preset threshold is a negative number. When the calculated time difference is greater than the first preset threshold—in the multiple detection sequences of the ignition period and the stable combustion period in this embodiment, the time difference is always positive and relatively large—it is determined that the current period is the ignition period or the stable combustion period, and a first weight allocation strategy is executed: the fusion weight of the thermal infrared sensor is increased to a higher level than that of the visible light sensor, so that the thermal infrared sensor dominates the fusion decision. Specifically, in this embodiment, during the ignition period or stable combustion period, the fusion weight of the thermal infrared sensor is set to a value in the range of 0.5 to 0.7, and the fusion weight of the visible light sensor is set to a value in the range of 0.3 to 0.5, with the sum of the two weights being 1.

[0090] When the timing difference is detected to turn negative or its absolute value decreases to less than the third preset threshold—in this embodiment, the timing difference turns negative or approaches zero in multiple detection sequences during the decay and smoldering phases—it is determined that the current phase is either decay or smoldering, and a second weight allocation strategy is executed: the fusion weight of the visible light sensor is increased to a higher value than that of the thermal infrared sensor, making the visible light sensor dominant in the fusion decision. Specifically, in this embodiment, during the decay or smoldering phase, the fusion weight of the visible light sensor is set to a value between 0.5 and 0.6, and the fusion weight of the thermal infrared sensor is set to a value between 0.4 and 0.5, with the sum of their weights being 1. In this embodiment, based on the real-time timing difference continuously calculated in step S2, the above weight allocation strategy is dynamically executed in each combustion stage to ensure that the sensor more sensitive to the current stage always occupies a dominant weight in the fusion decision.

[0091] In this embodiment, the first preset threshold is a positive number, and the second preset threshold is a negative number. When the time difference is greater than 30 seconds, it is determined to be the ignition period or the stable combustion period; when the time difference is less than 0 seconds or the absolute value of the time difference is less than 10 seconds, it is determined to be the decay period or the smoldering period.

[0092] Step S4: Based on the adjusted weight allocation, perform a fusion decision on the thermal infrared sensor data and the visible light sensor data, and output the fire identification result.

[0093] In step S4 of this embodiment, based on the weight allocation adjusted in step S3, the thermal infrared sensor data and the visible light sensor data are fused and judged to output the fire identification result. Specifically, the system performs a weighted calculation on the detection confidence scores extracted from the two sensor data according to the adjusted weight allocation. When the overall confidence score after weighted fusion exceeds the preset alarm threshold, the system outputs a fire identification result that includes the fire probability value and the coordinates of the fire location.

[0094] Step S5: Adaptively adjust the UAV's flight altitude based on the combustion stage determination results.

[0095] In step S5 of this embodiment, the method further includes: when the fire is determined to be in the ignition or stable combustion phase, controlling the UAV to maintain a first flight altitude for high-altitude hovering monitoring; when the fire is determined to be in the decay or smoldering phase, controlling the UAV to switch to a second flight altitude for low-altitude grid scanning. The low-altitude grid scanning is used to search for low-temperature thermal anomaly areas with temperatures 5°C to 50°C higher than the ambient background temperature and below 500°C. Specifically, the combustion stage determination result of step S3 is sent to the UAV ground control station in real time. When the fire is determined to be in the ignition or stable combustion phase, controlling the UAV to maintain a first flight altitude for high-altitude hovering monitoring to obtain macroscopic images of the fire scene; when the fire is determined to be in the decay or smoldering phase, controlling the UAV to automatically switch to a second flight altitude for low-altitude grid scanning. The second flight altitude is lower than the first flight altitude. The low-altitude grid scanning is used to search for low-temperature thermal anomaly areas with temperatures 5°C to 50°C higher than the ambient background temperature and below 500°C. In this embodiment, the first flight altitude is 150 meters to 200 meters, and the second flight altitude is no higher than 50 meters. After detecting the transition from the stable combustion period to the smoldering period (around 14:00), the drone automatically switches from high-altitude monitoring mode to low-altitude scanning mode within 2 minutes, and searches for low-temperature thermal anomaly areas with temperatures only slightly higher than the ambient background in low-altitude flight mode.

[0096] Step S6: Perform the three-level closed-loop verification process for smoldering point.

[0097] In step S6 of this embodiment, a three-level closed-loop verification process is sequentially executed for the suspected thermal anomalies discovered by the UAV in low-altitude scanning mode in step S5.

[0098] Level 1 verification: The UAV identifies suspected thermal anomalies in low-altitude scanning mode and records their coordinates and timestamp T1. In the low-altitude grid scan after burn-out in this embodiment, a total of 38 suspected thermal anomalies with temperatures 5°C to 50°C higher than the ambient background and lower than 500°C were identified.

[0099] Second-level verification: Play back the video surveillance footage near the coordinates within a preset time window before and after the timestamp to determine if smoke features are present. If present, it is marked as a candidate for smoldering with smoke; if not, it is marked as a candidate for smoldering without smoke. Specifically, in this embodiment, the preset time window is 2 minutes before and after the timestamp. Play back the video surveillance footage near each suspected point's coordinates for 2 minutes before and after the timestamp T1 to assist in manual judgment regarding the presence of smoke features. In this embodiment, the preset time window is 2 minutes before and after T1. Through video playback, weak smoke was observed at 21 suspected points, indicating the presence of weak smoke features, and these were marked as candidates for smoldering with smoke; no smoke was observed at 11 suspected points, indicating the absence of weak smoke features, and these were marked as candidates for smoldering without smoke; and 6 suspected points were determined to be false positives (later confirmed to be exposed rocks heated by the sun). In this embodiment, smokeless smoldering accounts for approximately 34.4% of all true smoldering points, indicating that the video retrospective step added to the three-level verification process plays an irreplaceable role in the identification of smokeless smoldering subtypes.

[0100] Level 3 verification: Dispatch ground personnel with thermal imagers and smoke detectors to the coordinates for on-site measurement, confirm the true smoldering point and record the type of combustible material, and generate a smoldering point sample database with true value labels. In this implementation, the specific details are as follows:

[0101] Ground personnel were dispatched with handheld thermal imagers (thermal sensitivity better than 30mK) and portable smoke detectors to conduct on-site measurements at the coordinates of each suspected smoldering point. The actual surface temperature and carbon monoxide concentration were measured to confirm the true smoldering points and record the type of combustible material. In this embodiment, after on-site verification, 32 points were confirmed as genuine smoldering points. The three-level closed-loop verification reduced the false alarm rate from approximately 16% to 0. All verification results were compiled to form a smoldering point sample database with true value labels (smoky smoldering or smokeless smoldering).

[0102] In this embodiment, each record in the smoldering point sample database includes at least: smoldering point number, coordinates, surface temperature, carbon monoxide concentration, plume visibility label, and combustible material type. All verification results are summarized to form a smoldering point sample database with truth value labels (smoldering with smoke or smoldering without smoke). Each record includes smoldering point number, WGS84 coordinates, surface temperature, carbon monoxide concentration, plume visibility label, and combustible material type (including at least one of leaf litter, humus layer, and decaying fallen wood).

[0103] Step S7: Perform asynchronous incremental fusion of multiple sensors using extended Kalman filtering.

[0104] In this embodiment, the method further includes: using extended Kalman filtering to perform asynchronous incremental fusion of multiple sensors to construct a system state vector containing fire probability, fire location, fire area, and confidence level; when any sensor data arrives asynchronously at its own sampling frequency, one-step prediction and measurement update of extended Kalman filtering are performed immediately to continuously output the optimal estimate of the fire status without waiting for all sensor data to arrive; when the posterior fusion confidence level exceeds a preset threshold, it is determined as fire confirmation and an early warning is triggered.

[0105] Step S7 may specifically include the following: constructing a state vector X = [fire probability, fire location x, fire location y, fire area, confidence level]^T, and using an extended Kalman filter as the fusion engine. When any sensor data arrives asynchronously at its own sampling frequency in step S1—for example, thermal infrared data is updated at the minute level, video AI detection results are updated at the frame level (25 frames / second), and satellite data arrives sparsely according to the transit cycle—there is no time alignment waiting for each data source. Instead, whenever new data arrives, one-step prediction and measurement update of the extended Kalman filter is performed immediately to continuously output the optimal estimate of the fire status.

[0106] In a typical fusion sequence of this embodiment (at 13:30 when the VIIRS satellite passes overhead), the fire status estimate is continuously updated using UAV thermal infrared data (latest data at 13:29:55) and video AI detection data (updated in real time to 13:29:59) before the satellite data arrives. When the satellite data arrives (13:30:00), the satellite observation is used as the new measurement information, and after gain calculation by extended Kalman filtering, it is fused with the predicted status, and the overall confidence level is updated from 0.82 to 0.91. During this process, no other sensor data is required; the update cycle for each fusion confirmation is the computation time of a single filtering iteration (approximately 2 seconds). When the posterior fusion confidence level exceeds a preset threshold (0.7 in this embodiment), a fire is immediately confirmed, and an early warning output is triggered.

[0107] Advantages of implementing this invention:

[0108] 1. The system execution of the three-level closed-loop verification process verified the effective detection capability of the present invention for weak thermal anomalies during the smoldering stage.

[0109] The drone identified 38 suspected thermal anomaly points with temperatures 5°C to 50°C above the background temperature but below 500°C in low-altitude grid scanning mode. Video review further differentiated these into 21 candidates with smoldering smoke and 11 without. Ground personnel then measured these points on-site using handheld thermal imagers and smoke detectors, ultimately confirming 32 true smoldering points, reducing the false alarm rate from approximately 16% to zero. This implementation demonstrates that even during the smoldering stage, when thermal infrared signals are significantly attenuated and smoke is minimal or nonexistent, the invention can still achieve systematic detection and accurate confirmation of weak thermal anomalies through a three-level verification process. This verifies that the core effect of "effective perception throughout the entire combustion stage" is indeed achieved during the most challenging smoldering period.

[0110] 2. The incremental fusion implementation of extended Kalman filtering verified the fast response capability of the present invention in multi-source asynchronous data scenarios.

[0111] The system uses the VIIRS satellite's passage at 13:30 as a typical fusion sequence. Before the satellite data arrives, the system continuously updates the state estimate using UAV thermal infrared data (13:29:55) and video AI detection data (updated in real time to 13:29:59). After the satellite data arrives, the confidence level is updated from 0.82 to 0.91 in a single filtering iteration (approximately 2 seconds). This implementation process demonstrates that the present invention can quickly output fire situation determination results through a streaming fusion architecture without waiting for all sensor data to arrive, verifying the timeliness of the core effect of "effective perception throughout the entire combustion phase".

[0112] Example 3

[0113] The difference between this embodiment and Embodiment 1 is that, based on Embodiment 1, this embodiment extracts the fire line position frame by frame from the thermal infrared video data collected by the thermal infrared sensor in step S1, overlays it with a high-precision digital surface model to obtain the slope value, and simultaneously introduces wind speed data. A binary empirical model of the spread speed changing with slope and wind speed is established using the nonlinear least squares method, thereby extending the technical effect of the present invention from current fire perception to fire spread trend prediction.

[0114] This embodiment uses a multi-source data fusion field planned burn test as a specific example to further illustrate the invention. The test observed a 0.43-hectare planned burn area from 11:45 to 16:29, employing a five-layer synchronous data acquisition system consisting of space-based satellites (Sentinel 2 multispectral, VIIRS / S-NPP thermal anomaly), airborne UAVs (a quadcopter UAV equipped with a thermal infrared camera, and a multirotor UAV equipped with a multispectral camera), and ground-based equipment (1080P video surveillance, D-RTK 2 precise positioning, FLIR T540 handheld thermal imager, and Kestrel 5500 portable weather station).

[0115] like Figure 3 As shown, a rapid identification method for initial forest fires based on multi-sensor temporal fusion includes:

[0116] Step S1: Synchronously acquire thermal infrared sensor data and visible light sensor data using a unified time reference.

[0117] In step S1 of this embodiment, the BeiDou Navigation Satellite System time is used as the unified time reference to synchronize and calibrate all sensors, ensuring that the timestamp deviation of each data source is less than 1 second. Space-based data is acquired via the VIIRS / S-NPP satellite receiving station, obtaining active fire point products with a spatial resolution of 375 meters, and simultaneously receiving Sentinel-2 multispectral data. Airborne data is simultaneously collected via a thermal infrared camera (640×512 pixels resolution, NETD better than 50mK) and a multispectral camera mounted on a UAV, which performs observation tasks over the fire site according to a preset flight path. Ground-based data is continuously collected via a 1080P high-definition video monitoring device (25 frames / second) installed at the observation points, and centimeter-level spatial coordinate references are provided for each observation point using a D-RTK 2 precision positioning device. Simultaneously, meteorological parameters such as ambient temperature and wind speed are recorded synchronously using a Kestrel 5500 portable weather station. All data is merged into the data storage unit after being identified by the BeiDou time as a unified timestamp, for use in subsequent steps.

[0118] In step S1 of this embodiment, all sensors complete time synchronization calibration before data acquisition to ensure that the timestamps of each data source have a unified comparison benchmark when calculating the time difference in step S2.

[0119] Step S2: Calculate the time difference between the moment when the thermal infrared sensor first identifies a stable high-temperature anomaly and the moment when the visible light sensor first identifies a recognizable smoke signal.

[0120] In step S2 of this embodiment, the multi-source synchronization data obtained in step S1 is analyzed frame by frame to monitor the detection trigger times of the thermal infrared channel and the visible light channel respectively.

[0121] In this embodiment, the data streams of both the thermal infrared and visible light channels are continuously monitored from the moment of ignition. The detection logic of the thermal infrared channel is as follows: temperature value extraction and high-temperature region segmentation are performed on each frame of thermal infrared image. When the detected temperature value is greater than a preset threshold (150℃ in this embodiment, but other preset values ​​can be used according to actual conditions) and the high-temperature region persists in N consecutive frames (generally N≥3, N is 3 in this embodiment), and solar reflection and sensor noise interference are excluded, this moment is recorded as the moment T_thermal when the thermal infrared sensor first identifies a stable high-temperature anomaly point, and the coordinates of this point are marked. The detection logic of the visible light channel is as follows: smoke features are extracted from each frame of visible light image. When the detected plume area is greater than a preset number of pixels (10 pixels in this embodiment) and the plume region persists in M ​​consecutive frames (M is 3 in this embodiment), this moment is recorded as the moment T_smoke when the visible light sensor first identifies a recognizable smoke signal, and the geometric center coordinates of the plume region are marked.

[0122] By comparing the thermal infrared triggering time and the visible light triggering time near the same spatial location, the timing difference Delta-T = T_smoke - T_thermal is calculated between the two. In a typical detection sequence of this embodiment, T_thermal is first triggered approximately 92 seconds after ignition (detecting a local high temperature of approximately 180°C), and T_smoke is first triggered approximately 161 seconds after ignition (detecting a white, thin columnar plume). Based on this, the timing difference Delta-T of this detection sequence is calculated to be approximately 69 seconds.

[0123] Step S3: Based on the numerical range of the time difference, adaptively and inversely adjust the weight allocation of the thermal infrared sensor and the visible light sensor in the fusion decision; wherein, when the time difference is greater than a first preset threshold, increase the fusion weight of the thermal infrared sensor, and when the time difference is less than a second preset threshold, increase the fusion weight of the visible light sensor.

[0124] In step S3 of this embodiment, adaptive weight adjustment is performed based on the time difference Delta-T calculated in step S2.

[0125] In this embodiment, the first preset threshold is a positive number, and the second preset threshold is a negative number. Specifically, the first preset threshold is 30 seconds, the second preset threshold is 0 seconds, and the third preset threshold is 10 seconds. When the time difference is greater than the first preset threshold, it is determined to be the ignition period or the stable combustion period. When the time difference is less than the second preset threshold or the absolute value of the time difference is less than the third preset threshold, it is determined to be the decay period or the smoldering period. Internally configured with determination logic based on a priori time difference database. In this embodiment, the first preset threshold is a positive number, and the second preset threshold is a negative number. When the calculated time difference is greater than the first preset threshold—in the multiple detection sequences of the ignition period and the stable combustion period in this embodiment, the time difference is always positive and relatively large—it is determined that the current period is the ignition period or the stable combustion period, and a first weight allocation strategy is executed: the fusion weight of the thermal infrared sensor is increased to be higher than the fusion weight of the visible light sensor, so that the thermal infrared sensor dominates the fusion decision. Specifically, in this embodiment, during the ignition period or stable combustion period, the fusion weight of the thermal infrared sensor is set to a value in the range of 0.5 to 0.7, and the fusion weight of the visible light sensor is set to a value in the range of 0.3 to 0.5, with the sum of the two weights being 1.

[0126] In practical applications, when the time difference is detected to turn negative or its absolute value shrinks to less than the third preset threshold—in the multiple detection sequences of the decay and smoldering phases in this embodiment, the time difference turns negative or approaches zero—it is determined that the current phase is either decay or smoldering, and a second weight allocation strategy is executed: the fusion weight of the visible light sensor is increased to be higher than that of the thermal infrared sensor, so that the visible light sensor dominates the fusion decision. Specifically, in this embodiment, during the decay or smoldering phase, the fusion weight of the visible light sensor is set to a value in the range of 0.5 to 0.6, and the fusion weight of the thermal infrared sensor is set to a value in the range of 0.4 to 0.5, with the sum of their weights being 1. In this embodiment, based on the real-time time difference continuously calculated in step S2, the above weight allocation strategy is dynamically executed in each combustion stage to ensure that the sensor more sensitive to the current stage always occupies the dominant weight in the fusion decision.

[0127] Step S4: Based on the adjusted weight allocation, perform a fusion decision on the thermal infrared sensor data and the visible light sensor data, and output the fire identification result.

[0128] In step S4 of this embodiment, based on the weight allocation adjusted in step S3, the thermal infrared sensor data and the visible light sensor data are fused and judged to output the fire identification result. Specifically, the system performs a weighted calculation on the detection confidence scores extracted from the two sensor data according to the adjusted weight allocation. When the overall confidence score after weighted fusion exceeds the preset alarm threshold, the system outputs a fire identification result that includes the fire probability value and the coordinates of the fire location.

[0129] Step S5: Adaptively adjust the UAV's flight altitude based on the combustion stage determination results.

[0130] In step S5 of this embodiment, the method further includes: when the fire is determined to be in the ignition or stable combustion phase, controlling the UAV to maintain a first flight altitude for high-altitude hovering monitoring; when the fire is determined to be in the decay or smoldering phase, controlling the UAV to switch to a second flight altitude for low-altitude grid scanning. The low-altitude grid scanning is used to search for low-temperature thermal anomaly areas with temperatures 5°C to 50°C higher than the ambient background temperature and below 500°C. Specifically, the combustion stage determination result of step S3 is sent to the UAV ground control station in real time. When the fire is determined to be in the ignition or stable combustion phase, controlling the UAV to maintain a first flight altitude for high-altitude hovering monitoring to obtain macroscopic images of the fire scene; when the fire is determined to be in the decay or smoldering phase, controlling the UAV to automatically switch to a second flight altitude for low-altitude grid scanning. The second flight altitude is lower than the first flight altitude. The low-altitude grid scanning is used to search for low-temperature thermal anomaly areas with temperatures 5°C to 50°C higher than the ambient background temperature and below 500°C. In this embodiment, the first flight altitude is 150 meters to 200 meters, and the second flight altitude is no higher than 50 meters. After detecting the transition from the stable combustion period to the smoldering period (around 14:00), the drone automatically switches from high-altitude monitoring mode to low-altitude scanning mode within 2 minutes, and searches for low-temperature thermal anomaly areas with temperatures only slightly higher than the ambient background in low-altitude flight mode.

[0131] Step S6: Perform the three-level closed-loop verification process for smoldering point.

[0132] In step S6 of this embodiment, a three-level closed-loop verification process is sequentially executed for the suspected thermal anomalies discovered by the UAV in low-altitude scanning mode in step S5.

[0133] Level 1 verification: The UAV identifies suspected thermal anomalies in low-altitude scanning mode and records their coordinates and timestamp T1. In the low-altitude grid scan after burn-out in this embodiment, a total of 38 suspected thermal anomalies with temperatures 5°C to 50°C higher than the ambient background and lower than 500°C were identified.

[0134] Second-level verification: Play back the video surveillance footage near the coordinates within a preset time window before and after the timestamp to determine if smoke features are present. If present, it is marked as a candidate for smoldering with smoke; if not, it is marked as a candidate for smoldering without smoke. Specifically, in this embodiment, the preset time window is 2 minutes before and after the timestamp. Play back the video surveillance footage near each suspected point's coordinates for 2 minutes before and after the timestamp T1 to assist in manual judgment regarding the presence of smoke features. In this embodiment, the preset time window is 2 minutes before and after T1. Through video playback, weak smoke was observed at 21 suspected points, indicating the presence of weak smoke features, and these were marked as candidates for smoldering with smoke; no smoke was observed at 11 suspected points, indicating the absence of weak smoke features, and these were marked as candidates for smoldering without smoke; and 6 suspected points were determined to be false positives (later confirmed to be exposed rocks heated by the sun). In this embodiment, smokeless smoldering accounts for approximately 34.4% of all true smoldering points, indicating that the video retrospective step added to the three-level verification process plays an irreplaceable role in the identification of smokeless smoldering subtypes.

[0135] Level 3 verification: Dispatch ground personnel with thermal imagers and smoke detectors to the coordinates for on-site measurement, confirm the true smoldering point and record the type of combustible material, and generate a smoldering point sample database with true value labels. In this implementation, the specific details are as follows:

[0136] Ground personnel were dispatched with handheld thermal imagers (thermal sensitivity better than 30mK) and portable smoke detectors to conduct on-site measurements at the coordinates of each suspected smoldering point. The actual surface temperature and carbon monoxide concentration were measured to confirm the true smoldering points and record the type of combustible material. In this embodiment, after on-site verification, 32 points were confirmed as genuine smoldering points. The three-level closed-loop verification reduced the false alarm rate from approximately 16% to 0. All verification results were compiled to form a smoldering point sample database with true value labels (smoky smoldering or smokeless smoldering).

[0137] In this embodiment, each record in the smoldering point sample database includes at least: smoldering point number, coordinates, surface temperature, carbon monoxide concentration, plume visibility label, and combustible material type. All verification results are summarized to form a smoldering point sample database with truth value labels (smoldering with smoke or smoldering without smoke). Each record includes smoldering point number, WGS84 coordinates, surface temperature, carbon monoxide concentration, plume visibility label, and combustible material type (including at least one of leaf litter, humus layer, and decaying fallen wood).

[0138] Step S7: Perform asynchronous incremental fusion of multiple sensors using extended Kalman filtering.

[0139] In this embodiment, the method further includes: using extended Kalman filtering to perform asynchronous incremental fusion of multiple sensors to construct a system state vector containing fire probability, fire location, fire area, and confidence level; when any sensor data arrives asynchronously at its own sampling frequency, one-step prediction and measurement update of extended Kalman filtering are performed immediately to continuously output the optimal estimate of the fire status without waiting for all sensor data to arrive; when the posterior fusion confidence level exceeds a preset threshold, it is determined as fire confirmation and an early warning is triggered.

[0140] Step S7 may specifically include the following: constructing a state vector X = [fire probability, fire location x, fire location y, fire area, confidence level]^T, and using an extended Kalman filter as the fusion engine. When any sensor data arrives asynchronously at its own sampling frequency in step S1—for example, thermal infrared data is updated at the minute level, video AI detection results are updated at the frame level (25 frames / second), and satellite data arrives sparsely according to the transit cycle—there is no time alignment waiting for each data source. Instead, whenever new data arrives, one-step prediction and measurement update of the extended Kalman filter is performed immediately to continuously output the optimal estimate of the fire status.

[0141] In a typical fusion sequence of this embodiment (at 13:30 when the VIIRS satellite passes overhead), the fire status estimate is continuously updated using UAV thermal infrared data (latest data at 13:29:55) and video AI detection data (updated in real time to 13:29:59) before the satellite data arrives. When the satellite data arrives (13:30:00), the satellite observation is used as the new measurement information, and after gain calculation by extended Kalman filtering, it is fused with the predicted status, and the overall confidence level is updated from 0.82 to 0.91. During this process, no other sensor data is required; the update cycle for each fusion confirmation is the computation time of a single filtering iteration (approximately 2 seconds). When the posterior fusion confidence level exceeds a preset threshold (0.7 in this embodiment), a fire is immediately confirmed, and an early warning output is triggered.

[0142] Step S8: Construct a binary empirical model, which specifically includes: extracting the fire line position frame by frame from the thermal infrared video obtained by the thermal infrared sensor; overlaying the fire line position with the digital surface model to calculate the slope value corresponding to the fire head position; and calculating the displacement of the fire head position at adjacent time points to obtain the instantaneous spread speed; using the nonlinear least squares method to perform curve fitting on the spread speed and the slope value to establish an empirical model of the spread speed changing with the slope; and simultaneously collecting instantaneous wind speed data to establish a binary empirical model of the spread speed changing with both slope and wind speed.

[0143] In step S8 of this embodiment, the thermal infrared video data collected by the thermal infrared sensor in step S1 is read, and the fire line position is extracted frame by frame from the video sequence. Specifically, each frame of the thermal infrared video is first preprocessed, including radiometric calibration and temperature pseudo-color mapping, converting pixel grayscale values ​​into temperature values; then, the image is binarized based on a preset temperature threshold, and continuous high-temperature regions above the threshold are extracted as fire line candidate regions; next, the Canny edge detection algorithm is used to extract the boundary contours of the fire line candidate regions, and the Hough transform algorithm is used to fit the boundary contours with straight lines and polylines to obtain smooth fire line geometric vectors; finally, the Kalman filter tracking algorithm is used to match and track the fire line contours between adjacent frames to ensure the continuity and consistency of the fire line extraction results in the time series.

[0144] In this embodiment, the fire line position is extracted frame by frame from the thermal infrared video acquired by the thermal infrared sensor. The fire line position data is continuously sampled at fixed time intervals (sampling interval of 5 seconds), resulting in approximately 1320 data points covering the entire upslope fire stage for about 110 minutes. The fire head position at each moment is spatially overlaid with the digital surface model of the test area (resolution better than 5 cm) to perform analysis. The corresponding slope value is calculated for each fire head position point through spatial analysis. At the same time, the wind speed data synchronously recorded by the portable weather station in step S1 is read to establish a spatiotemporal dataset containing timestamps, fire head position coordinates, instantaneous spread speed, slope, and wind speed.

[0145] In practical applications, step S8 calculates the spatial displacement of the fire head positions at adjacent time points, divides it by the time interval, and obtains the instantaneous spread rate at each moment. Based on the above dataset, with slope as the independent variable and spread rate as the dependent variable, a nonlinear least squares method (Levenberger-Marquardt algorithm) is used to fit the data points to a curve. The optimization objective is to minimize the sum of squared residuals. The model parameters are iteratively adjusted until convergence, establishing an empirical model of spread rate variation with slope. In this embodiment, the empirical model of upslope fire spread rate obtained by nonlinear least squares fitting is V_up=0.82×exp(0.089×S), with a coefficient of determination R²=0.83 (slope range from 5 degrees to 35 degrees); the empirical model of downslope fire spread rate is V_down=0.35×exp(0.042×|S|), with a coefficient of determination R²=0.68. Here, S represents the terrain slope corresponding to the fire's location, measured in degrees (°), with uphill slopes being positive and downhill slopes being negative. It characterizes the impact of terrain slope on the rate of forest fire spread. S is calculated by overlaying the fire's location extracted from thermal infrared video with a digital terrain model, and |S| is the absolute value of S. This model effectively reflects the exponential driving effect of slope on spread speed—for every 10-degree increase in slope, the fire spread rate increases by approximately 1.4 times.

[0146] Furthermore, the instantaneous wind speed W was introduced as a third variable to establish a binary empirical model in which the fire spread speed varies with both slope and wind speed. After nonlinear least squares fitting, the resulting binary empirical model was V = 0.58 × exp(0.076 × S + 0.21 × W), with a coefficient of determination R² = 0.87. Here, S represents the terrain slope corresponding to the fire front location, in degrees (°), calculated by overlaying the fire front location extracted from thermal infrared video with a digital surface model; W represents the instantaneous wind speed, in meters per second (m / s), collected synchronously from a portable weather station. The model's goodness of fit was further improved, indicating that wind speed is a significant factor affecting the fire spread speed—for every 1 m / s increase in wind speed, the fire spread speed increases by approximately 23%. The model's prediction accuracy is significantly better than existing models based on general North American combustible parameters (an improvement of approximately 15% to 25%). The model parameters are directly derived from localized measured data from the forest areas of Southwest China, providing a high-precision quantitative basis for fire spread trend prediction and emergency decision-making.

[0147] In practical applications, step S8 has no data dependency on steps S2, S3, S4, S6, and S7, and its execution timing can be flexibly selected according to the actual application scenario. Specifically, S8 can be executed immediately after step S1 is completed, or it can be executed in parallel during the execution of steps S2 to S4. Since step S5 switches the UAV to low-altitude mode to obtain higher resolution thermal infrared images, step S8 can optionally be executed after step S5 to obtain better fire line extraction accuracy. If the smoldering point sample database generated in step S6 is needed as a reference for fire line location extraction, step S8 can optionally be executed after step S6 is completed. That is, the only required inputs for step S8 are the thermal infrared video data, wind speed data, and the externally input digital surface model collected in step S1.

[0148] Advantages of implementing this invention:

[0149] 1. By extracting the fire line location frame by frame from UAV thermal infrared video and combining it with a high-precision digital surface model, a fully localized spread rate prediction model was established.

[0150] The system performs frame-by-frame radiometric calibration, temperature threshold binarization segmentation, Canny edge detection, Hough transform fitting, and Kalman filter tracking on thermal infrared video, successfully extracting approximately 1320 fireline location data points covering the entire upslope fire phase (approximately 110 minutes). After overlay analysis with a digital terrain model with a resolution better than 5 cm, the slope value corresponding to each firehead location is calculated. Empirical models for upslope and downslope fire spread rates are established through nonlinear least-squares fitting. This implementation demonstrates that the present invention can establish fire spread models based on localized measured data rather than general parameters, making the prediction results more consistent with the actual terrain characteristics of the forest regions in Southwest China.

[0151] 2. By introducing wind speed data collected synchronously from a portable weather station, the prediction accuracy of the spread rate model has been further improved.

[0152] The system incorporates instantaneous wind speed as a third-dimensional variable into the empirical model. A bivariate empirical model is established through nonlinear least-squares fitting, further improving its goodness of fit. This demonstrates that wind speed is a significant factor affecting the spread rate—for every 1 m / s increase in wind speed, the fire spread rate increases by approximately 23%. This implementation process proves that, based on fire identification, the present invention further possesses the ability to predict the fire spread trend, providing a more comprehensive quantitative basis for emergency decision-making.

[0153] Example 4

[0154] like Figure 4 As shown, this embodiment provides a rapid forest fire identification system based on multi-sensor time-series fusion, comprising:

[0155] The system consists of a data synchronization acquisition module M1, a time-series difference calculation module M2, a weighted adaptive scheduling module M3, and an output result module M4.

[0156] The data synchronization acquisition module M1 is used to synchronously acquire thermal infrared sensor data and visible light sensor data with a unified time reference. The data synchronization acquisition module M1 further includes a space-based satellite data receiving submodule M1-1, an airborne UAV thermal infrared and multispectral acquisition submodule M1-2, a ground-based video monitoring submodule M1-3, and a BeiDou time synchronization submodule M1-4. The space-based satellite data receiving submodule M1-1 is used to receive satellite thermal anomaly data; the airborne UAV thermal infrared and multispectral acquisition submodule M1-2 is used to acquire UAV thermal infrared and visible light multispectral images; the ground-based video monitoring submodule M1-3 is used to acquire ground video data with a resolution of no less than 1080P and a frame rate of no less than 25 frames per second; the BeiDou time synchronization submodule M1-4 is used to ensure that each data source uses the BeiDou satellite navigation system time as a unified time reference for timestamp identification, so that all data has a unified time reference before entering subsequent modules.

[0157] The timing difference calculation module M2 is used to calculate the timing difference between the moment when the thermal infrared sensor first detects a stable high-temperature anomaly and the moment when the visible light sensor first detects a discernible smoke signal. The input of the timing difference calculation module M2 is connected to the output of the data synchronization acquisition module M1, and receives synchronized multi-source data from the data synchronization acquisition module M1. The timing difference calculation module M2 monitors the detection trigger times of the thermal infrared channel and the visible light channel respectively, and outputs the calculated timing difference to the weighted adaptive scheduling module M3.

[0158] The weighted adaptive scheduling module M3 is used to adaptively and inversely adjust the weight allocation of the thermal infrared sensor data and the visible light sensor data in the fusion decision based on the numerical range of the time difference. The input of the weighted adaptive scheduling module M3 is connected to the output of the time difference calculation module M2. The weighted adaptive scheduling module M3 further includes a stage determination submodule M3-1 and a weight allocation submodule M3-2. The stage determination submodule M3-1 is used to determine the current combustion stage based on the numerical range of the time difference—when the time difference is greater than a first preset threshold, it is determined to be the ignition period or stable combustion period; when the time difference is less than a second preset threshold or the absolute value of the time difference is less than a third preset threshold, it is determined to be the decay period or smoldering period. The weight allocation submodule M3-2 is used to dynamically adjust the fusion weight of each sensor based on the determination result—increasing the fusion weight of the thermal infrared sensor during the ignition period or stable combustion period, and increasing the fusion weight of the visible light sensor during the decay period or smoldering period. The weight allocation signal output by the weighted adaptive scheduling module M3 is sent to the output result module M4.

[0159] The output result module M4 is used to fuse and determine the data from the thermal infrared sensor and the visible light sensor according to the adjusted weight allocation, and output the fire identification result. The input of the output result module M4 is connected to the output of the weight adaptive scheduling module M3, and receives the weight allocation signal output by the weight adaptive scheduling module M3 and the two-channel sensor data output by the data synchronization acquisition module M1. The output result module M4 performs weighted calculation on the detection confidence of each of the two-channel sensor data according to the adjusted weight allocation. When the comprehensive confidence after weighted fusion exceeds the preset alarm threshold, the output fire identification result includes the fire probability value and the coordinates of the fire point location.

[0160] The connection relationship and data flow between the above modules are as follows: the output of the data synchronization acquisition module M1 is connected to the input of the time difference calculation module M2 and the input of the output result module M4 respectively; the output of the time difference calculation module M2 is connected to the input of the weight adaptive scheduling module M3; the output of the weight adaptive scheduling module M3 is connected to the input of the output result module M4; the output result module M4 outputs the fire identification result to the outside world.

[0161] Example 5

[0162] like Figure 5 As shown, this embodiment provides a rapid forest fire initial fire identification system based on multi-sensor time-series fusion, including:

[0163] The module includes: data synchronization acquisition module M1, time difference calculation module M2, weight adaptive scheduling module M3, output result module M4, flight mode control module M5, extended Kalman filter fusion module M6, smoldering point three-level closed-loop verification module M7, and slope-spread speed modeling module M8.

[0164] The difference between this embodiment and embodiment four is that, based on embodiment four, this embodiment further adds a flight mode control module M5, an extended Kalman filter fusion module M6, a smoldering point three-level closed-loop verification module M7, and a slope-spreading speed modeling module M8 to realize extended functions such as adaptive switching of UAV flight altitude, accurate confirmation of smoldering point, and prediction of spread trend, forming a complete system configuration.

[0165] The data synchronization acquisition module M1 is used to synchronously acquire thermal infrared sensor data and visible light sensor data with a unified time reference. The data synchronization acquisition module M1 further includes a space-based satellite data receiving submodule M1-1, an airborne UAV thermal infrared and multispectral acquisition submodule M1-2, a ground-based video monitoring submodule M1-3, and a BeiDou time synchronization submodule M1-4. The space-based satellite data receiving submodule M1-1 is used to receive satellite thermal anomaly data; the airborne UAV thermal infrared and multispectral acquisition submodule M1-2 is used to acquire UAV thermal infrared and visible light multispectral images; the ground-based video monitoring submodule M1-3 is used to acquire ground video data with a resolution of no less than 1080P and a frame rate of no less than 25 frames per second; the BeiDou time synchronization submodule M1-4 is used to ensure that each data source uses the BeiDou satellite navigation system time as a unified time reference for timestamp identification, so that all data has a unified time reference before entering subsequent modules.

[0166] The timing difference calculation module M2 is used to calculate the timing difference between the moment when the thermal infrared sensor first detects a stable high-temperature anomaly and the moment when the visible light sensor first detects a discernible smoke signal. The input of the timing difference calculation module M2 is connected to the output of the data synchronization acquisition module M1, and receives synchronized multi-source data from the data synchronization acquisition module M1. The timing difference calculation module M2 monitors the detection trigger times of the thermal infrared channel and the visible light channel respectively, and outputs the calculated timing difference to the weighted adaptive scheduling module M3.

[0167] The weighted adaptive scheduling module M3 is used to adaptively and inversely adjust the weight allocation of the thermal infrared sensor data and the visible light sensor data in the fusion decision based on the numerical range of the time difference. The input of the weighted adaptive scheduling module M3 is connected to the output of the time difference calculation module M2. The weighted adaptive scheduling module M3 further includes a stage determination submodule M3-1 and a weight allocation submodule M3-2. The stage determination submodule M3-1 is used to determine the current combustion stage based on the numerical range of the time difference—when the time difference is greater than a first preset threshold, it is determined to be the ignition period or stable combustion period; when the time difference is less than a second preset threshold or the absolute value of the time difference is less than a third preset threshold, it is determined to be the decay period or smoldering period. The weight allocation submodule M3-2 is used to dynamically adjust the fusion weight of each sensor based on the determination result—increasing the fusion weight of the thermal infrared sensor during the ignition period or stable combustion period, and increasing the fusion weight of the visible light sensor during the decay period or smoldering period. The weight allocation signal output by the weight adaptive scheduling module M3 is sent to the output result module M4 and the extended Kalman filter fusion module M6, and the output stage determination result is sent to the flight mode control module M5.

[0168] The output result module M4 is used to fuse and decide the data from the thermal infrared sensor and the visible light sensor according to the adjusted weight allocation, and output the fire identification result. The input of the output result module M4 is connected to the output of the data synchronization acquisition module M1 and the output of the weight adaptive scheduling module M3, respectively, receiving the two-channel sensor data output by the data synchronization acquisition module M1 and the weight allocation signal output by the weight adaptive scheduling module M3; the output result module M4 performs weighted calculation on the detection confidence of each of the two-channel sensor data according to the adjusted weight allocation, and when the comprehensive confidence after weighted fusion exceeds the preset alarm threshold, it outputs the fire identification result including the fire probability value and the fire point location coordinates. When the confidence decision result of the extended Kalman filter fusion module M6 is received, the output result module M4 further outputs comprehensive early warning information including the fire point location coordinates, the fire probability value, and the spread trend prediction result.

[0169] The flight mode control module M5 controls the UAV to switch flight altitudes based on the combustion stage determination results. The input of the flight mode control module M5 is connected to the output of the weighted adaptive scheduling module M3. When the combustion stage is determined to be either ignition or stable combustion, the flight mode control module M5 controls the UAV to maintain a first flight altitude for high-altitude hovering monitoring; when the combustion stage is determined to be either decay or smoldering, the flight mode control module M5 controls the UAV to switch to a second flight altitude for low-altitude grid scanning, the second flight altitude being lower than the first flight altitude. The flight control commands output by the flight mode control module M5 are sent to the UAV ground control station.

[0170] The extended Kalman filter fusion module M6 is used for asynchronous incremental fusion of multiple sensors. The inputs of the extended Kalman filter fusion module M6 are connected to the outputs of the data synchronization acquisition module M1 and the weight adaptive scheduling module M3, respectively, receiving the raw data from each sensor output by the data synchronization acquisition module M1 and the weight allocation signal output by the weight adaptive scheduling module M3. The extended Kalman filter fusion module M6 constructs a system state vector containing fire probability, fire location, fire area, and confidence level. The extended Kalman filter fusion module M6 further includes a state prediction submodule M6-1, a measurement update submodule M6-2, and a confidence level decision submodule M6-3. The state prediction submodule M6-1 performs a one-step prediction based on the system state transition model; the measurement update submodule M6-2 performs Kalman gain calculation and state update when new sensor data arrives—when any sensor data arrives asynchronously at its own sampling frequency, a one-step prediction and measurement update are performed immediately without waiting for all sensor data to arrive; the confidence level decision submodule M6-3 triggers an early warning signal when the posterior fusion confidence level exceeds a preset threshold. The optimal fire status estimate and confidence level judgment results output by the extended Kalman filter fusion module M6 are sent to the smoldering point three-level closed-loop verification module M7 and the output result module M4.

[0171] The smoldering point three-level closed-loop verification module M7 is used to perform a three-level closed-loop verification process for suspected smoldering points. The input of the smoldering point three-level closed-loop verification module M7 is connected to the output of the extended Kalman filter fusion module M6. It is activated upon receiving a fire confirmation signal from the extended Kalman filter fusion module M6, or actively triggered by the weighted adaptive scheduling module M3 when it determines that the smoldering period has begun. The smoldering point three-level closed-loop verification module M7 further includes a UAV infrared detection submodule M7-1, a video backtracking submodule M7-2, and a ground verification submodule M7-3, which are executed sequentially in a closed loop. The UAV infrared detection submodule M7-1 is used to identify suspected thermal anomaly points with temperatures 5°C to 50°C higher than the ambient background temperature and below 500°C in the UAV's low-altitude scanning mode, and to record their coordinates and timestamps. The video playback submodule M7-2 is used to play back video surveillance recordings near the coordinates within a preset time window before and after the timestamp and to determine the characteristics of the smoke. The ground verification submodule M7-3 is used to receive the true value data recorded by ground personnel after on-site measurement with thermal imagers and smoke detectors, and to generate a smoldering point sample database with true value labels. The smoldering point sample data with true value labels output by the three-level closed-loop verification module M7 is sent to the slope-spread speed modeling module M8, and the output verification results are sent to the output result module M4.

[0172] The slope-spread rate modeling module M8 is used to construct a spread rate prediction model. The input terminals of the slope-spread rate modeling module M8 are connected to the output terminals of the data synchronization acquisition module M1 and the smoldering point three-level closed-loop verification module M7, respectively. It receives UAV thermal infrared video data and portable weather station wind speed data output from the data synchronization acquisition module M1, as well as fire point coordinate data with truth labels output from the smoldering point three-level closed-loop verification module M7. The slope-spread rate modeling module M8 further includes a fire line extraction submodule M8-1, a digital surface model overlay analysis submodule M8-2, and a curve fitting submodule M8-3. The fire line extraction submodule M8-1 is used to extract the fire line position frame by frame from the UAV thermal infrared video; the digital surface model overlay analysis submodule M8-2 is used to overlay the fire head position with the digital surface model to calculate the corresponding slope value; the curve fitting submodule M8-3 is used to establish an empirical model of spread rate variation with slope using the nonlinear least squares method, and to establish a binary empirical model by introducing wind speed data. The empirical model and its fitting parameters output by the slope-spread rate modeling module M8 are sent to the output results module M4 as a quantitative basis for predicting the fire spread trend.

[0173] In practical applications, the slope-spread rate modeling module M8 is used to construct a spread rate prediction model. Its function is to provide quantitative evidence for predicting fire spread trends, thus extending the fire identification function. If the actual application scenario only requires fire identification and not spread trend prediction, the slope-spread rate modeling module M8 can be omitted. In this case, the system can still complete the fire identification and early warning functions through the data synchronization acquisition module M1, the time-series difference calculation module M2, the weighted adaptive scheduling module M3, and the output result module M4. That is, M8 is an optional module of this system, and its deployment does not affect the implementation of the main fire identification function.

[0174] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A rapid identification method for initial forest fires based on multi-sensor temporal fusion, characterized in that, include: Data from thermal infrared sensors and visible light sensors are collected synchronously using a unified time reference. Calculate the time difference between the moment when the thermal infrared sensor first identifies a stable high-temperature anomaly and the moment when the visible light sensor first identifies a discernible smoke signal; Based on the numerical range of the time difference, the weight allocation of the thermal infrared sensor data and the visible light sensor data in the fusion decision is adaptively adjusted in reverse; when the time difference is greater than a first preset threshold, the fusion weight of the thermal infrared sensor data is increased, and when the time difference is less than a second preset threshold, the fusion weight of the visible light sensor data is increased. The thermal infrared sensor data and the visible light sensor data are fused and judged according to the adjusted weight allocation, and the fire identification result is output.

2. The rapid identification method for initial forest fires based on multi-sensor temporal fusion according to claim 1, characterized in that, The moment when the thermal infrared sensor first identifies a stable high-temperature anomaly is determined by the following conditions: the temperature value is greater than a preset threshold and it exists continuously for N frames, N≥3, excluding solar reflection and sensor noise interference; the moment when the visible light sensor first identifies a recognizable smoke signal is determined by the following conditions: the smoke plume area is greater than a preset number of pixels and it exists continuously for more than M frames, M≥3.

3. The rapid identification method for initial forest fires based on multi-sensor temporal fusion according to claim 1, characterized in that, The first preset threshold is a positive number, and the second preset threshold is a negative number; when the time difference is greater than the first preset threshold, it is determined to be the ignition period or the stable combustion period; when the time difference is less than 0 seconds or the absolute value of the time difference is less than 10 seconds, it is determined to be the decay period or the smoldering period.

4. The rapid identification method for initial forest fires based on multi-sensor time-series fusion according to claim 3, characterized in that, During the ignition or stable combustion period, the weight of the thermal infrared sensor is 0.5 to 0.7, and the weight of the visible light sensor is 0.3 to 0.5; during the decay or smoldering period, the weight of the visible light sensor is 0.5 to 0.6, and the weight of the thermal infrared sensor is 0.4 to 0.

5.

5. The rapid identification method for initial forest fires based on multi-sensor time-series fusion according to claim 4, characterized in that, Also includes: When the ignition or stable combustion phase is determined, the UAV is controlled to maintain the first flight altitude to perform high-altitude hovering monitoring; when the decay or smoldering phase is determined, the UAV is controlled to switch to the second flight altitude to perform low-altitude grid scanning. The low-altitude grid scanning is used to search for low-temperature thermal anomaly areas with temperatures 5°C to 50°C higher than the ambient background temperature and lower than 500°C.

6. The rapid identification method for initial forest fires based on multi-sensor temporal fusion according to claim 1, characterized in that, It also includes a three-level closed-loop verification process for smoldering point: The first level of verification involves the drone identifying suspected thermal anomalies in low-altitude scanning mode and recording their coordinates and timestamps. The second level of verification involves replaying video surveillance footage near the coordinates, recordings within a preset time window before and after the timestamp, and determining the presence of smoke features based on the plume area and the existence of consecutive frames. If it exists, mark it as a candidate for smoldering with smoke; if it does not exist, mark it as a candidate for smoldering without smoke. The third level of verification involves dispatching ground personnel with thermal imagers and flue gas detectors to the coordinates for on-site measurement, confirming the true smoldering point and recording the type of combustible material, and generating a smoldering point sample database with truth value labels.

7. The rapid identification method for initial forest fires based on multi-sensor temporal fusion according to claim 6, characterized in that, The preset time window is 2 minutes before and after the timestamp; each record in the smoldering point sample database contains at least: smoldering point number, coordinates, surface temperature, carbon monoxide concentration, plume visibility label, and combustible type.

8. The rapid identification method for initial forest fires based on multi-sensor temporal fusion according to claim 6, characterized in that, Also includes: Extended Kalman filtering is used for asynchronous incremental fusion of multiple sensors to construct a system state vector that includes fire probability, fire location, fire area and confidence level; When any sensor data arrives asynchronously at its own sampling frequency, the extended Kalman filter is immediately executed for one-step prediction and measurement update to continuously output the optimal estimate of the fire status without waiting for all sensor data to arrive. When the posterior fusion confidence exceeds a preset threshold, it is determined as a fire and an early warning is triggered.

9. The rapid forest fire identification method based on multi-sensor time-series fusion according to any one of claims 1 to 8, characterized in that, It also includes constructing a binary empirical model, which specifically includes: The fire line position is extracted frame by frame from the thermal infrared video obtained by the thermal infrared sensor. The fire line position is overlaid with the digital surface model to calculate the slope value corresponding to the fire head position, and the displacement of the fire head position at adjacent time points is calculated to obtain the instantaneous spread speed. The nonlinear least squares method is used to fit the curve of the spread rate and the slope value to establish an empirical model of the spread rate as a function of the slope. In addition, instantaneous wind speed data are collected simultaneously to establish a binary empirical model in which the spread speed varies with both slope and wind speed.

10. A rapid identification system for initial forest fires based on multi-sensor temporal fusion, characterized in that, The rapid forest fire identification method based on multi-sensor time-series fusion according to any one of claims 1 to 9 includes: The data synchronization acquisition module synchronously acquires data from the thermal infrared sensor and the visible light sensor using a unified time reference. The timing difference calculation module calculates the timing difference between the moment when the thermal infrared sensor first identifies a stable high-temperature anomaly and the moment when the visible light sensor first identifies a recognizable smoke signal. The weighted adaptive scheduling module adaptively and inversely adjusts the weight allocation of the thermal infrared sensor data and the visible light sensor data in the fusion decision based on the numerical range of the time difference. The output results module performs a fusion decision on the thermal infrared sensor data and the visible light sensor data according to the adjusted weight allocation, and outputs the fire identification result.

Images