A method and system for multimodal real-time monitoring and multidimensional fire data management

Abstract

This invention relates to the field of smart forestry technology and discloses a method and system for multimodal real-time monitoring and multidimensional fire data management. The system includes a multimodal sensing and acquisition module, a fire feature analysis module, a graded response scheduling module, a dynamic evolution tracking module, a closed-loop data management module, and a strategy adaptive optimization module. The method includes: collecting heterogeneous multimodal sensing data covering the entire forest fire cycle; performing spatiotemporal benchmark unification and quality verification on the data to construct a standardized sensing dataset; extracting static and dynamic features of the fire to generate multidimensional feature vectors; triggering graded response strategies based on fire severity levels and dynamically updating the feature vectors and strategies during fire evolution; and performing source tracing, archiving, and closed-loop management of the entire process data to generate an updated multimodal sensing deployment scheme and response strategy library. This invention reduces the prediction error of fire evolution and ensures the forward-looking and scientific nature of fire severity level determination.

Description

Technical Field

[0001] This invention relates to the field of smart forestry technology, specifically to a method and system for multimodal real-time monitoring and multidimensional fire data management. Background Technology

[0002] Smart forestry refers to a new model of forestry development that fully utilizes next-generation information technologies such as cloud computing, the Internet of Things, big data, and mobile internet, and forms a three-dimensional perception, efficient collaborative management, prominent ecological value, and integrated internal and external services through sensing, IoT, and intelligent means.

[0003] Currently, due to the large scale, strong dynamics, and complex terrain characteristics of the forest fire monitoring environment, when using multimodal data for fire situation assessment, there is a problem of inconsistent spatiotemporal references among the data sources collected by heterogeneous sensing devices. This leads to registration deviations when fusion of multi-source data. When spatiotemporal errors cannot be calibrated in real time, it will cause fire location drift and distortion in the prediction of spread trend.

[0004] Therefore, a multimodal real-time monitoring and multidimensional fire data management method and system are proposed to solve the above problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a multimodal real-time monitoring and multidimensional fire data management method and system, which solves the problems mentioned in the background art, such as the large-scale, highly dynamic, and complex terrain-covered characteristics of forest fire monitoring environment, and the problems caused by the inability to calibrate spatiotemporal errors in real time, which lead to fire location drift and distortion of spread trend prediction.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method and system for multimodal real-time monitoring and multidimensional fire data management, the method comprising the following steps: S1. Collect heterogeneous multimodal sensing data covering the entire forest fire cycle; S2. Perform spatiotemporal benchmark unification and quality verification on the heterogeneous multimodal sensing data to generate a standardized sensing dataset; S3. Construct a multi-dimensional feature vector of the fire situation based on the standardized perception dataset, and extract key parameters of the fire situation evolution; S4. Perform multi-dimensional matching between the fire situation multi-dimensional feature vector and the historical fire case database to generate a fire situation level determination result; S5. Trigger a graded response strategy based on the fire level determination result and coordinate the corresponding emergency resources. S6. Track the fire evolution process in real time and dynamically update the fire multidimensional feature vector and response strategy; S7. Trace and archive data from the entire fire response process to generate a closed-loop management log; S8. Optimize the multimodal perception deployment and response strategy library based on the closed-loop management log.

[0007] Preferably, the step S1 of unifying the spatiotemporal reference and verifying the quality of the heterogeneous multimodal sensing data to generate a standardized sensing dataset includes the following steps: S11. Collect large-scale surface temperature, vegetation cover and smoke diffusion time-series image data using geostationary satellite remote sensing equipment; S12. Collect mesoscale fire point thermal radiation intensity, flame morphology characteristics and spread direction vector data using a multispectral and thermal infrared camera carried by a medium-sized compound wing UAV. S13. Collect small-scale air temperature and humidity, wind speed and direction, soil moisture content and deep underground thermal anomaly data through ground IoT sensor nodes. S14. Collect personnel location, equipment status, and on-site audio and video stream data through the smart terminals worn by firefighters; The heterogeneous multimodal sensing data includes six types of data sources: optical, thermal infrared, synthetic aperture radar, acoustic, inertial navigation, and structured text logs.

[0008] Preferably, step S2 includes the following steps: S21. Using BeiDou-3 satellite timekeeping as a unified time reference and the National Geodetic Coordinate System as a unified spatial reference, perform spatiotemporal stamp interpolation and alignment on the heterogeneous multimodal sensing data. S22. Adaptive median filtering algorithm is used to remove impulse noise and abnormal jump values ​​in the heterogeneous multimodal sensing data, and Kriging space interpolation method is used to complete the missing data. S23. Based on geographic information metadata standards, label the filtered sensing data with data source, collection timestamp, spatial coordinates and confidence level labels to generate a standardized sensing dataset.

[0009] Preferably, the step S3, which involves constructing a multi-dimensional feature vector of the fire situation based on the standardized sensing dataset and extracting key parameters of the fire situation evolution, includes the following steps: S31. Extract static fire features from the standardized sensing dataset: including the latitude and longitude of the ignition point, the burned area, the vegetation type, the terrain slope, the aspect, and the humus thickness. S32. Extract fire dynamic features from the standardized sensing dataset: including average flame height, fire line advance speed, thermal radiation flux density, smoke extinction coefficient change rate, and underground fire temperature rise rate. S33. The static and dynamic features of the fire are concatenated in a 15-minute time window to construct a multi-dimensional feature vector of the fire containing 16 indicators. The 16 dimensions are as follows: longitude, latitude, altitude, vegetation cover, slope, aspect, humus thickness, flame height, spread rate, thermal radiation flux, smoke extinction coefficient, temperature rise rate, wind speed, wind direction, air humidity, and soil moisture content.

[0010] Preferably, step S4 involves performing multi-dimensional matching between the fire situation multi-dimensional feature vector and the historical fire case database to generate a fire situation level determination result, including the following steps: S41. Establish a historical fire case database, which includes typical forest fire cases that have been closed nationwide since 2015. Each case is marked with the fire level, response measures, resources invested and final losses. S42. Using an improved dynamic time warping algorithm, calculate the weighted Euclidean distance between the fire multidimensional feature vector and the feature vector of each case in the case library. The smaller the distance, the higher the similarity. S43. Select the top 3 cases with the highest similarity, and combine them with the current 72-hour meteorological forecast data for weighted fusion to generate a fire level determination result. The fire level is divided into four levels: Level 1 warning, Level 2 initial outbreak, Level 3 development, and Level 4 intense fire.

[0011] Preferably, in step S5, triggering a tiered response strategy based on the fire severity level determination result and coordinating the dispatch of corresponding emergency resources includes the following steps: S51, a predefined graded response strategy library: Level 1 fire triggers 2 drones for patrol and verification and 10 ground patrolmen for reinforcement; Level 2 fire triggers 30 firefighters to assemble and high-pressure water pumps to be deployed; Level 3 fire triggers 100 cross-regional reinforcements and road control around the fire site; Level 4 fire triggers evacuation of people within 3 kilometers and helicopter water bucket firefighting. S52. Based on the fire severity assessment result, call the GIS geographic information system to search for emergency material reserve points, fire fighting team camps and natural water sources within a 50km radius of the fire point; S53. Based on the shortest path algorithm of satellite A, plan emergency resource dispatch routes, generate a dispatch list including personnel, equipment and materials, and push it to the on-site command terminal and the rear command center simultaneously.

[0012] Preferably, the real-time tracking of the fire evolution process and the dynamic updating of the fire multidimensional feature vector and response strategy in step S6 includes the following steps: S61. Update the standardized sensing dataset every 15 minutes and recalculate the multidimensional feature vector of the fire situation; S62. Compare the multi-dimensional feature vectors of the fire situation at two adjacent time points. If the spread rate increases by more than 30% and the heat radiation intensity increases by more than 50%, the fire situation level will be automatically upgraded and a higher-level response strategy will be triggered. S63. Receive real-time feedback data from the firefighting team on-site. If the fire line advance speed does not reach the expected suppression target, dynamically adjust the number of water pump groups and the priority of firebreak excavation in the response strategy.

[0013] Preferably, the process of tracing and archiving the entire fire response data and generating a closed-loop management log in S7 includes the following steps: S71. Classify and store the data of the entire fire response process, including four categories: perception data, decision data, dispatch data, and on-site feedback data. Each category of data is indexed by timestamp. S72. Use the SM3 national cryptographic hash algorithm to generate a unique digital fingerprint for each type of data, and store the fingerprint and evidence on the blockchain to the consortium blockchain node to ensure that the data cannot be tampered with. S73. Based on the final fire loss assessment results, generate a closed-loop management log that includes the entire chain of perception, decision-making, handling, and assessment, and store it in a cloud-based distributed database for a retention period of more than 10 years.

[0014] Preferably, the optimization of the multimodal awareness deployment and response strategy library based on the closed-loop management log in step S8 includes the following steps: S81. Statistically analyze the false alarm rate and false alarm rate of multimodal sensing data in the past 6 months, and identify sensing blind spots including high mountain areas and densely vegetated areas. S82. Based on the disposal efficiency data in the closed-loop management log, evaluate the resource utilization rate and fire control timeliness of different response strategies; S83. Adjust the satellite transit frequency and sensor deployment density for sensing blind spots, optimize the resource scheduling weights in the strategy library for inefficient response strategies, and generate an updated multimodal sensing deployment scheme and response strategy library.

[0015] Preferably, the system includes: The multimodal sensing and acquisition module uses satellite remote sensing units, UAV-borne units, ground IoT units and mobile terminal units to collect heterogeneous sensing data, and outputs standardized sensing datasets through the spatiotemporal reference alignment unit. The fire feature analysis module receives the standardized sensing dataset, constructs a fire feature vector through a multi-dimensional feature extraction unit, and outputs the fire level result through a level determination unit. The graded response scheduling module receives the fire level result, calls the response strategy library through the strategy matching unit, and outputs a scheduling list through the resource retrieval unit and pushes it to the command terminal. The dynamic evolution tracking module receives the standardized perception dataset, updates the fire feature vector in real time through the feature update unit, and outputs the optimized response strategy through the strategy adjustment unit. The closed-loop data management module integrates data from the entire fire response process, generates closed-loop management logs through a blockchain-based evidence storage unit, and stores them in a cloud database. The strategy adaptive optimization module receives the closed-loop management log and generates a perception deployment optimization scheme and a response strategy library update instruction through the blind spot identification unit and the performance evaluation unit.

[0016] Compared with existing technologies, this invention provides a method and system for multimodal real-time monitoring and multidimensional fire data management, which has the following beneficial effects: 1. In this invention, when conducting full-cycle monitoring of forest fires, the spatiotemporal benchmark is unified and the quality is verified for heterogeneous multimodal sensing data. A multidimensional feature vector of fire situation is constructed using a standardized sensing dataset. This solves the problem of inaccurate fusion of multi-source data in the background technology, ensures the accuracy of fire location and situation assessment, and reduces the prediction error of fire evolution.

[0017] 2. In this invention, when determining the fire level, the multi-dimensional feature vector of the fire is matched with the historical fire case database in multiple dimensions, and the fire level determination result is dynamically generated in combination with meteorological forecast data. This can capture the hidden thermal anomalies caused by sudden changes in vegetation type and the burning of underground humus, and solve the problem that the determination result is lagging behind the actual evolution trend in the background technology, thus ensuring the foresight and scientific nature of the fire level determination.

[0018] 3. In this invention, when conducting emergency resource scheduling and strategy adjustment, a graded response strategy is triggered based on the fire level determination result and corresponding emergency resources are scheduled accordingly. At the same time, the fire multidimensional feature vector and response strategy are dynamically updated during the fire evolution process. This solves the problems of rescue forces being unable to reach the fire in complex terrain and the rigidity of strategies in the background technology, realizing the transformation from passive response to active defense, and further improving the timeliness and safety of firefighting operations. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the steps of a multimodal real-time monitoring and multidimensional fire data management method according to the present invention. Figure 2 This is a schematic diagram of the architecture of a multimodal real-time monitoring and fire multidimensional data management system according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figures 1-2 The specific implementation of a multimodal real-time monitoring and fire multidimensional data management method and system is as follows, and the method includes the following steps: S1. Collect heterogeneous multimodal sensing data covering the entire forest fire cycle; S2. Perform spatiotemporal benchmark unification and quality verification on heterogeneous multimodal sensing data to generate a standardized sensing dataset; S3. Construct a multi-dimensional feature vector of fire based on a standardized sensing dataset and extract key parameters of fire evolution; S4. Perform multi-dimensional matching between the fire situation multi-dimensional feature vector and the historical fire case database to generate fire situation level determination results; S5. Trigger the graded response strategy based on the fire severity level determination result and coordinate the dispatch of corresponding emergency resources; S6. Real-time tracking of the fire's evolution process and dynamic updates of the fire's multi-dimensional feature vector and response strategy; S7. Trace and archive data from the entire fire response process to generate a closed-loop management log; S8. Optimize the multimodal awareness deployment and response strategy library based on closed-loop management logs.

[0022] S1 performs spatiotemporal benchmark unification and quality verification on heterogeneous multimodal sensing data to generate a standardized sensing dataset, including the following steps: S11. Collect large-scale surface temperature, vegetation cover and smoke diffusion time-series image data using geostationary satellite remote sensing equipment; S12. Collect mesoscale fire point thermal radiation intensity, flame morphology characteristics and spread direction vector data using a multispectral and thermal infrared camera carried by a medium-sized compound wing UAV. S13. Collect small-scale air temperature and humidity, wind speed and direction, soil moisture content and deep underground thermal anomaly data through ground IoT sensor nodes. S14. Collect personnel location, equipment status, and on-site audio and video stream data through the smart terminals worn by firefighters; Heterogeneous multimodal sensing data includes six types of data sources: optical, thermal infrared, synthetic aperture radar, acoustic, inertial navigation, and structured text logs; Initial confidence weights were assigned to the six data sources, and the dynamic weights of each data source were calculated using the inverse variance weighting method, as shown in the formula: ; in, For the first Weights of data sources The standard deviation of the historical measurement data of this data source over the past 30 days is used for calculation. Outliers exceeding 3 times the standard deviation are removed during the calculation. The data is also updated every 24 hours based on newly added data. The weights are dynamically adjusted according to the data quality to ensure that high-confidence data dominates the subsequent analysis.

[0023] S2 includes the following steps: S21. Using BeiDou-3 satellite timekeeping as the unified time reference and the National Geodetic Coordinate System as the unified spatial reference, perform spatiotemporal stamp interpolation and alignment on heterogeneous multimodal sensing data. The formula for calculating the spatiotemporal alignment residual of heterogeneous multimodal sensing data is as follows: ; in, For spatiotemporal alignment residuals, For timestamps of satellite remote sensing equipment, For the timestamps of the ground sensors, , , Spatial coordinates collected by satellite, , , For the spatial coordinates of the ground sensor, the residual E must be less than the threshold of 10 meters + 10 nanoseconds; otherwise, re-interpolation and alignment must be performed. S22. An adaptive median filtering algorithm is used to remove impulse noise and abnormal jump values ​​in heterogeneous multimodal sensing data. The filtering window starts from 3×3. When the center pixel value is greater than the maximum value of the window and less than the minimum value of the window, it is judged as noise and median replacement is performed. Otherwise, the window is expanded to 5×5, with a maximum of less than 7×7. Kriging space interpolation is used to fill in missing data. A common Kriging model is selected, and a spherical model is used for the semi-variogram. The nugget value is 0.1, the sill value is 1.5, the range is 5km, and the search neighborhood is set to the 16 nearest neighbors within a radius of 10km. S23. Based on geographic information metadata standards, label the filtered sensing data with data source, collection timestamp, spatial coordinates and confidence level labels to generate a standardized sensing dataset.

[0024] The steps involved in constructing a multi-dimensional feature vector of fire based on a standardized sensing dataset in S3 and extracting key parameters of fire evolution are as follows: S31. Extract static fire features from the standardized sensing dataset: including the latitude and longitude of the ignition point, the burned area, the vegetation type, the terrain slope, the aspect, and the humus thickness. S32. Extract dynamic features of the fire situation from the standardized sensing dataset: including average flame height, fire line advance speed, thermal radiation flux density, smoke extinction coefficient change rate and underground fire temperature rise rate. S33. The static and dynamic features of the fire are concatenated in a 15-minute time window to construct a multi-dimensional feature vector of the fire containing 16 indicators. The 16-dimensional fire feature vector is normalized using the following formula: ; in, This is the original 16-dimensional feature vector. This represents the minimum value of each dimension in historical cases. The maximum value of each dimension in historical cases is used. After normalization, the feature value falls in the interval [0, 1], thus eliminating the difference in dimensions. The 16 indicators are as follows: longitude, latitude, altitude, vegetation cover, slope, aspect, humus thickness, flame height, spread rate, thermal radiation flux, smoke extinction coefficient, temperature rise rate, wind speed, wind direction, air humidity, and soil moisture content.

[0025] In S4, the multi-dimensional feature vector of the fire situation is matched with the historical fire case database in multiple dimensions to generate the fire situation level determination result, including the following steps: S41. Establish a historical fire case database, which includes typical forest fire cases that have been closed nationwide since 2015. Each case is marked with the fire level, response measures, resources invested and final losses. S42. An improved dynamic time warping algorithm is adopted. Based on the traditional dynamic time warping (DTW), a Sakoe-Chiba constraint band is introduced to limit the bending path. Only three key dynamic features—spread rate, thermal radiation flux, and temperature rise rate—are time-aligned. The weighted Euclidean distance between the fire's multidimensional feature vector and the feature vector of each case in the case library is calculated using the following formula: ; in, The weighted Euclidean distance between the current fire situation's multidimensional feature vector and the feature vectors of historical cases is used. For the first Weights of dimensional features The th feature vector of the current feature vector Dimensional value, The first feature vector of historical cases Dimension value: the smaller the distance, the higher the similarity. S43. Select the top 3 cases with the highest similarity, and combine them with the current 72-hour weather forecast data for weighted fusion to generate the fire level determination result. The fire level is divided into four levels: Level 1 warning, Level 2 initial outbreak, Level 3 developing, and Level 4 intense.

[0026] In S5, the tiered response strategy is triggered based on the fire severity level assessment, and the corresponding emergency resources are dispatched in conjunction with the following steps: S51, a predefined graded response strategy library: Level 1 fire triggers 2 drones for patrol and verification and 10 ground patrolmen for reinforcement; Level 2 fire triggers 30 firefighters to assemble and high-pressure water pumps to be deployed; Level 3 fire triggers 100 cross-regional reinforcements and road control around the fire site; Level 4 fire triggers evacuation of people within 3 kilometers and helicopter water bucket firefighting. S52. Based on the fire severity assessment, use the GIS geographic information system to search for emergency material storage points, fire fighting team locations, and natural water sources within a 50km radius of the fire origin. S53. Planning emergency resource scheduling routes based on the shortest path algorithm of satellite A, heuristic function. Using a weighted Euclidean distance that incorporates terrain resistance: ,in( , ) is a node coordinate,( , ( ) represents the coordinates of the target point. Let the terrain resistance coefficient be the cost function: ; in, For nodes The total cost, From emergency supplies storage points to key nodes The actual driving distance For nodes Prioritize the straight-line distance to the firefighting team's base. Minimal node expansion ensures optimal route; A dispatch list containing personnel, equipment, and materials is generated and simultaneously pushed to the on-site command terminal and the rear command center.

[0027] The real-time tracking of fire evolution in S6 and the dynamic updating of the fire's multi-dimensional feature vector and response strategy include the following steps: S61. Update the standardized sensing dataset every 15 minutes and recalculate the multidimensional feature vector of the fire situation. S62. Compare the multidimensional feature vectors of the fire situation at two adjacent time points, and calculate the rate of change of the feature vectors. The formula is: ; in, The rate of change of the eigenvector. The Euclidean distance norm is used, and the rate of change of the spread velocity dimension is separately weighted by a factor of 1.2. The feature vector for the current 15-minute window. This is the feature vector of the previous 15-minute window; The increase in the rate of spread exceeding 30% and the increase in thermal radiation intensity exceeding 50% refers to the rate of change of the eigenvector. The percentage changes in the spread rate and thermal radiation intensity dimensions will automatically raise the fire level and trigger a higher-level response strategy. S63. Receive real-time feedback data from the firefighting team on-site. If the fire line advance speed does not reach the expected suppression target, dynamically adjust the number of water pump groups and the priority of firebreak excavation in the response strategy.

[0028] In S7, the process of tracing and archiving data throughout the entire fire response process and generating closed-loop management logs includes the following steps: S71. Classify and store the data of the entire fire response process, including four categories: perception data, decision data, dispatch data, and on-site feedback data. Each category of data is indexed by timestamp. S72. Use the SM3 national cryptographic hash algorithm to generate a unique digital fingerprint for each type of data. The formula is as follows: ; in, A 128-bit hash value. This is a data block to be stored, containing the data source, timestamp, and spatial coordinates. The fingerprint and its evidence are stored on the consortium blockchain node to ensure that the data cannot be tampered with. S73. Based on the final fire loss assessment results, generate a closed-loop management log that includes the entire chain of perception, decision-making, handling, and assessment, and store it in a cloud-based distributed database for a retention period of more than 10 years.

[0029] The S8 library for optimizing multimodal awareness deployment and response strategies based on closed-loop management logs includes the following steps: S81. Calculate the false alarm rate and false alarm rate of multimodal sensing data in the past 6 months. Identify sensing blind spots, including high-altitude obstructed areas and densely vegetated areas. The formula for the false alarm rate is: ; in, The false negative rate, The number of fires that went undetected. The total number of actual fires, confirmed through both manual post-incident verification and official forestry department reports, excludes false fires caused by attempted arson or smoke alarms. The false alarm rate formula is: ; in, For false alarm rate, The number of false fire reports. The total number of alarms issued by the system is used to identify perception blind spots, including high-altitude areas and densely vegetated areas, based on the false alarm rate and the missed alarm rate. S82. Based on the response efficiency data in the closed-loop management log, evaluate the resource utilization rate and fire control timeliness of different response strategies. The response efficiency formula is: ; in, For efficiency, This refers to the time from the occurrence of a fire to its complete control, starting from the time when S1 first collects fire characteristic data. The time from fire severity assessment to the arrival of emergency resources on site is the starting point, which is the time when S4 generates the fire severity assessment result. The time base of both is consistent with the timestamp of the sensing data. The lower the efficiency, the more the response strategy needs to be optimized. S83. Adjust the satellite transit frequency and sensor deployment density for sensing blind spots, optimize the resource scheduling weights in the strategy library for inefficient response strategies, and generate an updated multimodal sensing deployment scheme and response strategy library.

[0030] The system includes: The multimodal sensing and acquisition module uses satellite remote sensing units, UAV-borne units, ground IoT units and mobile terminal units to collect heterogeneous sensing data, and outputs standardized sensing datasets through the spatiotemporal reference alignment unit. The fire feature analysis module receives a standardized sensing dataset, constructs a fire feature vector through a multi-dimensional feature extraction unit, and outputs the fire level result through a level determination unit. The graded response and dispatch module receives the fire level result, calls the response strategy library through the strategy matching unit, and outputs the dispatch list and pushes it to the command terminal using the resource retrieval unit. The dynamic evolution tracking module receives a standardized perception dataset, updates the fire feature vector in real time through the feature update unit, and outputs an optimized response strategy through the strategy adjustment unit. The closed-loop data management module integrates data from the entire fire response process, generates closed-loop management logs through a blockchain-based evidence storage unit, and stores them in a cloud database. The strategy adaptive optimization module receives closed-loop management logs and generates a perception deployment optimization plan and response strategy library update instructions through the blind spot identification unit and performance evaluation unit.

[0031] The operation steps of a multimodal real-time monitoring and fire multidimensional data management method and system are as follows: I. Multimodal Heterogeneous Sensing Data Acquisition: The system covers the entire lifecycle of forest fire monitoring through a multi-source collaborative sensing network. It uses geostationary satellite remote sensing equipment to collect large-scale surface temperature, vegetation coverage, and smoke diffusion time-series image data. It uses a medium-sized composite-wing UAV to collect mesoscale fire point thermal radiation intensity, flame morphology characteristics, and spread direction vector data through multispectral and thermal infrared cameras. It uses ground-based IoT sensor nodes to collect small-scale air temperature and humidity, wind speed and direction, soil moisture content, and underground fire depth thermal anomaly data. At the same time, it uses smart terminals worn by firefighters to collect personnel location, equipment status, and on-site audio and video stream data. The heterogeneous multimodal sensing data includes six types of data sources: optical, thermal infrared, synthetic aperture radar, acoustic, inertial navigation, and structured text logs, forming an integrated air-space-ground sensing system.

[0032] II. Unification and Standardization of Spatiotemporal References: Spatiotemporal consistency correction and quality control were performed on the collected heterogeneous multimodal sensing data: using BeiDou-3 satellite timekeeping as the unified time reference and the National Geodetic Coordinate System as the unified spatial reference, time difference and coordinate deviation of multi-source data were eliminated by spatiotemporal stamp interpolation alignment, and an adaptive median filtering algorithm was used to remove impulse noise and abnormal jump values. Kriging spatial interpolation was used to complete missing data. Based on geographic information metadata standards, the filtered sensing data was labeled with data source, collection timestamp, spatial coordinates and confidence level labels to generate a standardized sensing dataset, providing a reliable data foundation for subsequent analysis.

[0033] III. Construction of Multidimensional Feature Vectors of Fire Situation: Static and dynamic features of the fire situation were extracted from the standardized sensing dataset: static features included the latitude and longitude of the ignition point, burned area, vegetation type, terrain slope, aspect, and humus thickness; dynamic features included the average flame height, fire advance speed, thermal radiation flux density, smoke extinction coefficient change rate, and underground fire temperature rise rate. The two types of features were concatenated in a 15-minute time window to construct a multi-dimensional feature vector of the fire situation containing 16 indicators, specifically longitude, latitude, altitude, vegetation cover, slope, aspect, humus thickness, flame height, spread speed, thermal radiation flux, smoke extinction coefficient, temperature rise rate, wind speed, wind direction, air humidity, and soil moisture content, comprehensively representing the evolution of the fire situation.

[0034] IV. Fire Situation Level Determination and Case Matching: A historical fire case database was established. An improved dynamic time warping algorithm was used to calculate the weighted Euclidean distance between the multidimensional feature vector of the fire and the feature vector in the case database. The top three cases with the highest similarity were selected and combined with the current 72-hour meteorological forecast data for weighted fusion to generate the fire level judgment result, which is divided into four levels: Level 1 warning, Level 2 initial outbreak, Level 3 development, and Level 4 severe fire, so as to achieve accurate quantitative assessment of the fire situation.

[0035] V. Tiered Response Strategy and Resource Scheduling: A predefined tiered response strategy library: Level 1 fire triggers 2 drones for patrol and verification and 10 ground patrolmen for reinforcement; Level 2 fire triggers 30-person firefighting team assembly and high-pressure water pump forwarding; Level 3 fire triggers 100-person cross-regional reinforcement and road control around the fire site; Level 4 fire triggers evacuation of the masses within 3 kilometers and helicopter water-dropping firefighting. Based on the fire level determination results, the GIS geographic information system is called to retrieve emergency material reserve points, firefighting team camps and natural water sources within 50km of the fire point. Based on the shortest path algorithm of Satellite A, emergency resource dispatch routes are planned, and a dispatch list including personnel, equipment and materials is generated and simultaneously pushed to the on-site command terminal and the rear command center.

[0036] VI. Dynamic tracking of fire evolution and strategy adjustment: The standardized sensing dataset is updated every 15 minutes, and the multi-dimensional feature vector of the fire is recalculated. The changes in the feature vector at adjacent time points are compared: when the spread rate increases by more than 30% and the thermal radiation intensity increases by more than 50%, the fire level is automatically upgraded and a higher-level response strategy is triggered; real-time feedback data from the fire fighting teams is received. When the fire line advances at a speed that does not meet the expected suppression target, the number of water pump groups and the priority of firebreak excavation in the response strategy are dynamically adjusted to achieve closed-loop dynamic optimization of monitoring, assessment, and response.

[0037] VII. Full-process data traceability and closed-loop management: Data from the entire fire response process is categorized and stored into four types: perception data, decision-making data, dispatch data, and on-site feedback data. Each type of data is indexed by timestamp, and a unique digital fingerprint is generated for each type of data using the SM3 national cryptographic hash algorithm. The fingerprint is then stored on the blockchain to ensure immutability. The data is linked to the final fire loss assessment result to generate a closed-loop management log that includes perception, decision-making, response, and assessment. This log is stored in a cloud-based distributed database with a retention period of more than 10 years, providing data support for subsequent strategy optimization.

[0038] VIII. Adaptive Optimization of Sensing Deployment and Response Strategies: The false alarm rate and false alarm rate of multimodal sensing data over the past six months were statistically analyzed. Sensing blind spots were identified, including high-altitude obstructed areas and densely vegetated areas. Based on the disposal efficiency data in the closed-loop management log, the resource utilization rate and fire control timeliness of different response strategies were evaluated. Satellite transit frequency and sensor deployment density were adjusted for sensing blind spots. Resource scheduling weights in the strategy library were optimized for inefficient response strategies. An updated multimodal sensing deployment scheme and response strategy library were generated to achieve continuous evolution of system capabilities.

[0039] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0040] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-modal real-time monitoring and fire multi-dimensional data management method, characterized in that, The method includes the following steps: S1. Collect heterogeneous multimodal sensing data covering the entire forest fire cycle; S2. Perform spatiotemporal benchmark unification and quality verification on the heterogeneous multimodal sensing data to generate a standardized sensing dataset; S3. Construct a multi-dimensional feature vector of the fire situation based on the standardized perception dataset, and extract key parameters of the fire situation evolution; S4. Perform multi-dimensional matching between the fire situation multi-dimensional feature vector and the historical fire case database to generate a fire situation level determination result; S5. Trigger a graded response strategy based on the fire level determination result and coordinate the corresponding emergency resources. S6. Track the fire evolution process in real time and dynamically update the fire multidimensional feature vector and response strategy; S7. Trace and archive data from the entire fire response process to generate a closed-loop management log; S8. Optimize the multimodal perception deployment and response strategy library based on the closed-loop management log.

2. The method for multimodal real-time monitoring and multidimensional fire data management according to claim 1, characterized in that, The steps in S1 to unify the spatiotemporal reference and verify the quality of the heterogeneous multimodal sensing data, and to generate a standardized sensing dataset, include the following: S11. Collect large-scale surface temperature, vegetation cover and smoke diffusion time-series image data using geostationary satellite remote sensing equipment; S12. Collect mesoscale fire point thermal radiation intensity, flame morphology characteristics and spread direction vector data using a multispectral and thermal infrared camera carried by a medium-sized compound wing UAV. S13. Collect small-scale air temperature and humidity, wind speed and direction, soil moisture content and deep underground thermal anomaly data through ground IoT sensor nodes. S14. Collect personnel location, equipment status, and on-site audio and video stream data through the smart terminals worn by firefighters; The heterogeneous multimodal sensing data includes six types of data sources: optical, thermal infrared, synthetic aperture radar, acoustic, inertial navigation, and structured text logs.

3. The method for multimodal real-time monitoring and multidimensional fire data management according to claim 1, characterized in that, S2 includes the following steps: S21. Using BeiDou-3 satellite timekeeping as a unified time reference and the National Geodetic Coordinate System as a unified spatial reference, perform spatiotemporal stamp interpolation and alignment on the heterogeneous multimodal sensing data. S22. Adaptive median filtering algorithm is used to remove impulse noise and abnormal jump values ​​in the heterogeneous multimodal sensing data, and Kriging space interpolation method is used to complete the missing data. S23. Based on geographic information metadata standards, label the filtered sensing data with data source, collection timestamp, spatial coordinates and confidence level labels to generate a standardized sensing dataset.

4. The method for multimodal real-time monitoring and multidimensional fire data management according to claim 1, characterized in that, The steps in S3 to construct a multi-dimensional feature vector of the fire situation based on the standardized sensing dataset and extract key parameters of the fire situation evolution include the following: S31. Extract static fire features from the standardized sensing dataset: including the latitude and longitude of the ignition point, the burned area, the vegetation type, the terrain slope, the aspect, and the humus thickness. S32. Extract fire dynamic features from the standardized sensing dataset: including average flame height, fire line advance speed, thermal radiation flux density, smoke extinction coefficient change rate, and underground fire temperature rise rate. S33. The static and dynamic features of the fire are concatenated in a 15-minute time window to construct a multi-dimensional feature vector of the fire containing 16 indicators. The 16 dimensions are as follows: longitude, latitude, altitude, vegetation cover, slope, aspect, humus thickness, flame height, spread rate, thermal radiation flux, smoke extinction coefficient, temperature rise rate, wind speed, wind direction, air humidity, and soil moisture content.

5. The method for multimodal real-time monitoring and multidimensional fire data management according to claim 1, characterized in that, In step S4, the fire situation multi-dimensional feature vector is matched with the historical fire case database in multiple dimensions to generate a fire situation level determination result, which includes the following steps: S41. Establish a historical fire case database, which includes typical forest fire cases that have been closed nationwide since 2015. Each case is marked with the fire level, response measures, resources invested and final losses. S42. Using an improved dynamic time warping algorithm, calculate the weighted Euclidean distance between the fire multidimensional feature vector and the feature vector of each case in the case library. The smaller the distance, the higher the similarity. S43. Select the top 3 cases with the highest similarity, and combine them with the current 72-hour meteorological forecast data for weighted fusion to generate a fire level determination result. The fire level is divided into four levels: Level 1 warning, Level 2 initial outbreak, Level 3 development, and Level 4 intense fire.

6. The method for multimodal real-time monitoring and multidimensional fire data management according to claim 1, characterized in that, The steps in S5 to trigger a tiered response strategy based on the fire severity determination result and associate and dispatch corresponding emergency resources include: S51, a predefined graded response strategy library: Level 1 fire triggers 2 drones for patrol and verification and 10 ground patrolmen for reinforcement; Level 2 fire triggers 30 firefighters to assemble and high-pressure water pumps to be deployed; Level 3 fire triggers 100 cross-regional reinforcements and road control around the fire site; Level 4 fire triggers evacuation of people within 3 kilometers and helicopter water bucket firefighting. S52. Based on the fire severity assessment result, call the GIS geographic information system to search for emergency material reserve points, fire fighting team camps and natural water sources within a 50km radius of the fire point; S53. Based on the shortest path algorithm of satellite A, plan emergency resource dispatch routes, generate a dispatch list including personnel, equipment and materials, and push it to the on-site command terminal and the rear command center simultaneously.

7. The method for multimodal real-time monitoring and multidimensional fire data management according to claim 1, characterized in that, The real-time tracking of the fire evolution process and the dynamic updating of the fire multidimensional feature vector and response strategy in S6 include the following steps: S61. Update the standardized sensing dataset every 15 minutes and recalculate the multidimensional feature vector of the fire situation; S62. Compare the multi-dimensional feature vectors of the fire situation at two adjacent time points. If the spread rate increases by more than 30% and the heat radiation intensity increases by more than 50%, the fire situation level will be automatically upgraded and a higher-level response strategy will be triggered. S63. Receive real-time feedback data from the firefighting team on-site. If the fire line advance speed does not reach the expected suppression target, dynamically adjust the number of water pump groups and the priority of firebreak excavation in the response strategy.

8. The method for multimodal real-time monitoring and multidimensional fire data management according to claim 1, characterized in that, The steps involved in S7 to trace and archive data throughout the entire fire response process and generate a closed-loop management log are as follows: S71. Classify and store the data of the entire fire response process, including four categories: perception data, decision data, dispatch data, and on-site feedback data. Each category of data is indexed by timestamp. S72. Use the SM3 national cryptographic hash algorithm to generate a unique digital fingerprint for each type of data, and store the fingerprint and evidence on the blockchain to the consortium blockchain node to ensure that the data cannot be tampered with. S73. Based on the final fire loss assessment results, generate a closed-loop management log that includes the entire chain of perception, decision-making, handling, and assessment, and store it in a cloud-based distributed database for a retention period of more than 10 years.

9. The method for multimodal real-time monitoring and multidimensional fire data management according to claim 1, characterized in that, The optimization of the multimodal awareness deployment and response strategy library based on the closed-loop management log in S8 includes the following steps: S81. Statistically analyze the false alarm rate and false alarm rate of multimodal sensing data in the past 6 months, and identify sensing blind spots including high mountain areas and densely vegetated areas. S82. Based on the disposal efficiency data in the closed-loop management log, evaluate the resource utilization rate and fire control timeliness of different response strategies; S83. Adjust the satellite transit frequency and sensor deployment density for sensing blind spots, optimize the resource scheduling weights in the strategy library for inefficient response strategies, and generate an updated multimodal sensing deployment scheme and response strategy library.

10. A multimodal real-time monitoring and fire multidimensional data management system, used to implement the multimodal real-time monitoring and fire multidimensional data management method according to any one of claims 1 to 9, characterized in that the system include: The multimodal sensing and acquisition module uses satellite remote sensing units, UAV-borne units, ground IoT units and mobile terminal units to collect heterogeneous sensing data, and outputs standardized sensing datasets through the spatiotemporal reference alignment unit. The fire feature analysis module receives the standardized sensing dataset, constructs a fire feature vector through a multi-dimensional feature extraction unit, and outputs the fire level result through a level determination unit. The graded response scheduling module receives the fire level result, calls the response strategy library through the strategy matching unit, and outputs a scheduling list through the resource retrieval unit and pushes it to the command terminal. The dynamic evolution tracking module receives the standardized perception dataset, updates the fire feature vector in real time through the feature update unit, and outputs the optimized response strategy through the strategy adjustment unit. The closed-loop data management module integrates data from the entire fire response process, generates closed-loop management logs through a blockchain-based evidence storage unit, and stores them in a cloud database. The strategy adaptive optimization module receives the closed-loop management log and generates a perception deployment optimization scheme and a response strategy library update instruction through the blind spot identification unit and the performance evaluation unit.

Images