Natural disaster emergency rescue system based on multi-source perception information fusion

Abstract

The invention belongs to the technical field of emergency management, and discloses a natural disaster emergency rescue system based on multi-source sensing information fusion. The system is composed of a multi-source sensing data acquisition module, a data preprocessing and space-time registration module, a cross-modal feature extraction module, a multi-modal information fusion and conflict resolution module, a disaster type identification and grade discrimination module, a disaster influence range prediction and diffusion modeling module, and a dynamic emergency path planning and response plan generation module. A rescue scheduling and command control module; and an emergency feedback and closed loop dynamic correction module. Through multi-source sensing data fusion, cross-modal feature extraction and deep information fusion technologies, a full-space-time and full-process natural disaster emergency rescue system is constructed, comprehensive sensing, accurate recognition and dynamic plan generation of a disaster site are realized, the rescue response speed and decision scientificity are remarkably improved, and the intelligent level of emergency rescue is comprehensively improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of emergency management, and specifically relates to a natural disaster emergency rescue system based on multi-source sensing information fusion. BACKGROUND

[0002] Natural disasters are sudden, destructive and unpredictable, and once they occur, they will pose a serious threat to personnel safety, infrastructure and social operation. The existing natural disaster emergency rescue system relies on a single sensing means, such as video monitoring, weather observation or manual patrol, and has problems such as incomplete information collection, delayed response and inaccurate resource allocation, which makes it difficult to meet the needs of rapid and efficient emergency rescue at the disaster site.

[0003] The existing natural disaster emergency rescue system generally has the disadvantage of relying on a single sensor or limited monitoring means. The commonly used video monitoring, weather observation and manual on-site feedback methods cannot achieve comprehensive sensing at the disaster site, resulting in problems such as monitoring blind spots, delayed response and inaccurate allocation of rescue resources. In complex disaster environments, a single data source is easily invalidated, which cannot support high timeliness and high accuracy of emergency decision-making, affecting the overall rescue efficiency. At the same time, the existing system has limited integration capability for multi-source sensing data and lacks effective information fusion mechanism. Different monitoring data is often in a fragmented state, forming an information island, which makes it difficult to achieve dynamic tracking and intelligent prediction of the disaster site, and cannot meet the needs of collaborative management of the whole process and all factors of natural disaster emergency rescue. SUMMARY

[0004] The purpose of the present application is to provide a natural disaster emergency rescue system based on multi-source sensing information fusion to solve the problems raised in the background.

[0005] In order to achieve the above purpose, the present application provides the following technical scheme: a natural disaster emergency rescue system based on multi-source sensing information fusion, which is composed of a multi-source sensing data acquisition module, a data preprocessing and space-time registration module, a cross-modal feature extraction module, a multi-modal information fusion and conflict resolution module, a disaster type identification and grade discrimination module, a disaster influence range prediction and diffusion modeling module, a dynamic emergency path planning and response plan generation module, a rescue dispatching and command control module, and an emergency feedback and closed-loop dynamic correction module.

[0006] Multi-source perception data collection module, real-time collection of multi-source data of unmanned aerial vehicles, satellites, ground sensors, video monitoring and social media, to ensure comprehensive information, data preprocessing and spatio-temporal registration module for denoising, synchronization and standardization of multi-source data, to improve data quality, cross-modal feature extraction module, using deep learning to extract spatial, temporal and text multi-modal features, unified encoding as fusion input, multi-modal information fusion and conflict resolution module realizes deep fusion of heterogeneous data through Transformer and attention mechanism, removes redundancy, disaster type identification and grade determination module automatically identifies disaster type and grade based on fusion features, provides accurate classification, disaster impact range prediction and diffusion modeling module uses mathematical models to predict disaster spread path and potential impact area, dynamic emergency path planning and response plan generation module dynamically generates the optimal rescue path based on the prediction results using reinforcement learning algorithm, rescue dispatching and command control module realizes efficient dispatching and real-time monitoring of command center and front-line equipment, emergency feedback and closed-loop dynamic correction module adjusts the path and plan in real time through field feedback, and builds a continuous optimization closed loop.

[0007] Preferably, the multi-source perception data collection module comprises:

[0008] (1) Multi-source data collection unit composition: the system covers unmanned aerial vehicle laser radar unit, satellite remote sensing image acquisition unit, ground Internet of Things sensor group unit (including earthquake sensor, flood water level meter, gas detection sensor, temperature and humidity sensor), fixed video monitoring unit, social media data access unit, meteorological monitoring terminal unit, unmanned aerial vehicle laser radar and satellite remote sensing collect geographic information from the air and space; the ground Internet of Things sensor group monitors geological and environmental changes through multiple sensors such as earthquakes and water levels; fixed video monitoring records real-time pictures; the social media data access unit obtains social public opinion; the meteorological monitoring terminal collects meteorological information, and multiple units cooperate to collect data to build rich data dimensions;

[0009] The ground sensor shell is made of 316 stainless steel material with a protection level of IP68 (can work underwater for 10 meters); the unmanned aerial vehicle wind resistance level is greater than or equal to 12 levels, and it can be normally started in an environment of-20℃ to 60℃; in satellite signal weak areas (signal-to-noise ratio less than or equal to 15dB), the Beidou short message supplementary data is enabled to ensure transmission continuity;

[0010] (2) Wireless transmission and multi-modal data integration: the system uses 5G and NB-IoT wireless communication networks to transmit the original perception data collected by each unit in real time, ensuring data timeliness, and different types of data are integrated to form multi-modal basic data, which are correlated and cooperated with each other in the data collection formula, providing comprehensive and accurate data support for subsequent data processing, analysis and application;

[0011] After accessing social media data, automatic anonymization processing is performed (sensitive fields such as usernames and mobile phone numbers are removed), and the text content is transmitted through the AES-256 encryption protocol; the ground sensor data is pre-processed by edge computing, and only feature values (such as abnormal fluctuation identifiers) are uploaded instead of raw data, reducing the risk of privacy leakage;

[0012] The data collection expression formula is:

[0013]

[0014] In the formula, D total (t) is the total amount of multi-source data collected by the system at time t, D i (t) is the real-time data of the i-th sensing unit (including unmanned aerial vehicles, satellites, ground sensors, video surveillance, and social media) at time t, and n is the total number of sensing units.

[0015] The detection range of the unmanned aerial vehicle laser radar is 500-1000 meters, and the point cloud density is ≥200 points / ㎡; the sampling frequency of the ground seismic sensor is 100Hz, and the measurement error of the water level sensor is ≤±2cm; the spatial resolution of the satellite remote sensing image is 10 meters, and the revisit period is ≤24 hours; the frame rate of the fixed video surveillance is 25fps, and the resolution is ≥1080P.

[0016] Preferably, the data preprocessing and spatio-temporal registration module comprises:

[0017] (1) Basic data preprocessing operation: after the system receives the raw data of the collection module, a series of basic preprocessing operations are performed to remove interference information in the data through denoising and filtering, and to remove abnormal points to ensure data quality; time synchronization and spatial coordinate registration techniques are used to accurately align different source data in time and space dimensions, and a spatio-temporal registration formula is used to accurately position the data, laying a solid foundation for subsequent processing;

[0018] D norm (x,y,t)=f sync (D total (t),T offset ,S offset )

[0019] In the formula, D norm (x,y,t) is the standardized data set after time synchronization and spatial registration, f sync is a spatio-temporal registration function that handles time offset and spatial offset, T offset is the time offset of the sensing unit, and S offset is the spatial coordinate offset of the sensing unit.

[0020] (2) Deep processing of data by type: For different types of data, the system uses targeted processing methods. Image data uses de-distortion and image enhancement algorithms to restore the true picture and improve clarity; sensor data is processed through smoothing and anomaly detection to optimize data curves and detect abnormal fluctuations; social media data is text cleaned to remove irrelevant information, and finally outputs a standardized dataset that is spatio-temporally aligned and uniformly formatted, meeting the needs of subsequent analysis.

[0021] Preferably, the cross-modal feature extraction module comprises:

[0022] (1) Multimodal data feature extraction: After the standardized dataset enters the processing flow, a multi-channel deep convolutional neural network extracts spatial features from image data, capturing details and structures; a long short-term memory network is good at processing sensor data, accurately extracting time series features; natural language processing technology screens key information from social media text, and multiple technologies work together to comprehensively obtain core features of each modality data;

[0023] Social media text processing uses the XLM-R multilingual pre-training model, which supports 10 languages including English, Japanese, and Korean, with a translation accuracy BLEU value of ≥0.7; for mixed language text (such as Chinese and English), language detection algorithm (accuracy ≥95%) is used to split and process before feature fusion;

[0024] The CNN uses the "ResNet-18 architecture", the input image size is 224x224, uses a 3x3 convolution kernel and a pooling layer with a step of 2, and the output spatial feature dimension is 512;

[0025] (2) Feature mapping and fusion: The different modal features of space, time series, and text extracted are processed by a feature encoder, which maps different dimensional information and integrates them into a unified feature vector space, eliminating modal differences and achieving organic fusion between features, ultimately forming a structured and complete input feature stream, laying the foundation for subsequent analysis and prediction;

[0026] The feature encoder is a "3-layer fully connected network" with an input layer dimension of 1408 (512+128+768), which is processed by 2 hidden layers (768 dimensions, 256 dimensions, with ReLU as the activation function) and outputs a 128-dimensional unified feature vector;

[0027] The formula for multi-modal feature mapping is:

[0028] F multi = φ cnn (I) + φ lstm (S) + φ nlp (T)

[0029] In the formula, Fmulti : fused multimodal feature vector, F cnn (I): spatial feature extracted by CNN from input image data I, f lstm (S): time series feature extracted by LSTM from input time series data S, f nlp (T): semantic feature extracted by NLP from input text data T;

[0030] LSTM adopts a "2-layer hidden layer structure", each layer contains 128 neurons, and the dropout rate is set to 0.3, which is used to extract time series features;

[0031] NLP adopts a "BERT-base pre-training model" (fine-tuned by Chinese corpus), and the text feature dimension is 768.

[0032] Preferably, the multi-modal information fusion and conflict resolution module refers to a multi-source data interaction bridge built by the system through a Transformer fusion architecture based on cross-modal feature flow, combined with a modal attention mechanism focusing on key information, and a modal weighted fusion dynamically regulating the spatial details of each modal weight image, the time series changes of the sensor, and the semantic tendency of the text in this process. Real-time value is used to flexibly allocate weights, and redundant and conflicting data are synchronously eliminated. After this processing, different modal features break through the information barrier to achieve deep collaboration, which not only preserves the integrity of multi-dimensional data, but also strengthens the core features through weight optimization. The final generated dataset has high reliability and adaptability, accurately meets the needs of disaster identification for data consistency and effectiveness, and provides solid data support for subsequent disaster research and judgment;

[0033] The Transformer fusion architecture is a "6-layer encoder", each layer contains 8 self-attention mechanisms, the hidden layer dimension is 512, and the feedforward network dimension is 2048;

[0034] The modal weight is dynamically calculated based on the "signal-to-noise ratio (SNR)": the image data refers to the peak signal-to-noise ratio (PSNR), the sensor data refers to the signal variance ratio, and the text data refers to the keyword confidence. The sum of the weights of each modality is 1, and the real-time value is flexibly allocated;

[0035] The modal weighted fusion expression formula is:

[0036]

[0037] In the formula, F fusion : weighted fusion feature vector, F j : the jth modal feature (from CNN, LSTM, NLP), a j : modal weight coefficient dynamically calculated by Transformer and attention mechanism, and åa j= 1, m: total number of modalities.

[0038] Preferably, the disaster type recognition and grade discrimination module refers to a pre-trained multi-label classification deep learning model based on a fusion feature dataset. The system integrates the advantages of multi-modal features through a disaster classification fusion mechanism, combines the spatial distribution of image data, the time sequence evolution of sensor data, and the public opinion dynamics of text data, and combines the typical disaster feature patterns in the historical disaster knowledge base to identify disasters such as earthquakes and floods in real time, and determine the disaster grade according to the feature weight distribution. The identification result is automatically transmitted to the diffusion prediction module to provide clear classification and clear grade pre-data support for disaster space-time deduction.

[0039] The historical disaster knowledge base data source includes the annual disaster report of the National Emergency Management Department and the archived data of the local emergency office, which is updated monthly by 1 time; the IQR (interquartile range) method is used to remove abnormal data (data exceeding Q3+1.5IQR or lower than Q1-1.5IQR are marked as abnormal), to ensure the accuracy of the typical disaster feature pattern;

[0040] For fire disasters, the system strengthens the type determination through gas detection sensors (such as carbon monoxide concentration) and video monitoring flame recognition features (red pixel ratio ≥ 30%); in the scenario of mudslides, combined with soil moisture data (≥80%) of ground sensors and terrain change rate (≥0.5m / h) of satellite remote sensing, the recognition accuracy is optimized; in the scenario of typhoon, the wind speed data (≥10.8m / s) of meteorological monitoring terminal and the cloud layer movement features returned by unmanned aerial vehicle are linked to improve the accuracy of grade discrimination;

[0041] In the scenario of flood, the diffusion speed is calculated in combination with "water depth" and "terrain slope": the greater the slope and the deeper the water, the faster the diffusion speed (such as the empirical coefficient is set to 0.8, and the influence of both is integrated);

[0042] In the scenario of earthquake, the diffusion speed is based on the "seismic wave longitudinal wave propagation speed" (about 3.5km / s), and increases by 5% for each increase of 1 level of earthquake magnitude;

[0043] The disaster classification expression formula is:

[0044]

[0045] In the formula, C disaster : the identified disaster type, P c (F fusion ): the probability value of the fusion feature belonging to the category c, c: the disaster type label (such as earthquake, flood, fire, etc.).

[0046] Preferably, the disaster influence range prediction and diffusion modeling module includes:

[0047] (1) Disaster diffusion dynamic prediction mechanism: after receiving the disaster type and grade identification results, the system relies on the spatial framework of the GIS map, correlates the regular patterns in the historical diffusion database, starts the diffusion prediction process, and dynamically calculates the disaster impact range and spread path by integrating various data into the prediction logic through the adapted disaster diffusion mathematical model, such as the diffusion equation for flood and the wave propagation model for earthquake, to simultaneously predict the potential areas of secondary disasters;

[0048] The diffusion prediction expression formula is:

[0049]

[0050] In the formula, A(t): the impact area of the disaster at time t, A0: the initial disaster impact area, v(s): the disaster boundary diffusion speed (which can be calculated according to the on-site water flow speed and terrain slope), t0: the initial time of the disaster;

[0051] (2) Prediction result output and emergency support: the dynamic results obtained through diffusion prediction comprehensively present the disaster development trend, and these information is automatically output to the emergency path generation module to provide accurate spatial and timing basis for subsequent development of risk avoidance routes and allocation of rescue resources, so that emergency decision-making can closely match the real-time changes of disaster diffusion, and the pertinence and timeliness of response actions are improved.

[0052] Preferably, the dynamic emergency path planning and response plan generation module comprises:

[0053] (1) Emergency path intelligent generation mechanism: based on the disaster impact range prediction results, the system calls the reinforcement learning path optimization algorithm and graph search algorithm, and the path optimization reward function plays a role in this process, dynamically adjusts according to the efficiency and safety of the path, and drives the algorithm to generate the optimal solution, thereby obtaining various key paths such as emergency response path and personnel evacuation route;

[0054] The reinforcement learning adopts the "DDPG algorithm", and the state space includes the disaster impact range (binary matrix), the road traffic state (0-1 score), and the rescue resource location (coordinates);

[0055] Reward function parameters: time penalty coefficient is 0.3, safety priority coefficient is 0.7 (determined based on 1000 times of simulation experiment optimization), and priority is given to path safety;

[0056] The path optimization reward function expression formula is:

[0057]

[0058] In the formula, R total : total reward value of path planning, D k : moving distance of path node k, Tk : estimated time of path node k, S k : safety score of path node k (avoiding secondary disaster area), λ: time penalty coefficient, μ: safety priority coefficient, p: total number of path nodes;

[0059] (2) Multi-dimensional consideration of dynamic plan: During plan generation, real-time environmental changes, road traffic conditions, and safety coefficients are fully taken into account. Through algorithmic comprehensive weighing of these factors, the generated rescue priority and unmanned aerial vehicle inspection path are more in line with actual needs, ensuring efficient and safe emergency response and providing scientific path guidance for disaster response.

[0060] Preferably, the rescue dispatching and command control module comprises:

[0061] (1) Instruction issuance and execution state collection: After receiving the dynamic emergency path and response plan, the command center relies on multi-device dispatching instructions to accurately issue multi-level dispatching instructions to unmanned aerial vehicles, rescue vehicles, and emergency personnel terminals, while collecting the real-time execution state of each device and personnel, ensuring the accuracy of instruction transmission and the visibility of the execution process, and providing real-time basis for dispatch adjustment;

[0062] The multi-device dispatching instruction expression formula is:

[0063]

[0064] In the formula, Instruction set of the i-th rescue device at time t, f sched : dispatching function based on reward function, device location, and device state, Loc i (t): current location of the i-th device, State i (t): current working state (idle, busy, failure, etc.) of the i-th device;

[0065] (2) Multi-level collaboration and full-link monitoring: The system supports a multi-level control architecture, with efficient collaboration between front-line command and central command, forming a top-down command system. The dispatching process is monitored throughout the link, and each link from instruction generation, issuance to execution feedback can be traced back, ensuring the effective landing of dispatching instructions and improving the overall coordination and execution of emergency response;

[0066] The left side of the command center interface displays a dynamic heat map of disaster diffusion (red, orange, yellow, and blue color zones distinguish impact levels), and the right side displays the real-time distribution of rescue resources (unmanned aerial vehicles / vehicles / personnel icons mark locations). The bottom is equipped with 'one-key dispatching', 'path recalculation', and 'plan export' buttons, supporting single-device selection-instruction issuance shortcut operation, with a response delay of ≤1 second;

[0067] The ground Internet of Things sensor is powered by a solar panel (conversion efficiency ≥ 18%) + 12V backup lithium battery, with a continuous running time of ≥ 72 hours; the unmanned aerial vehicle is equipped with a 4500mAh lithium battery, with a single continuous running time of ≥ 40 minutes, supporting a hot spot battery replacement mode, and automatically returning to the nearest supply point when the power is low (remaining power ≤ 20%).

[0068] Preferably, the emergency feedback and closed-loop dynamic correction module comprises:

[0069] (1) Real-time feedback mechanism of field information: the system continuously collects field rescue progress, environmental changes, new sensing data and personnel positioning information, and timely returns them to the command center through the feedback channel. These dynamic information comprehensively reflects the actual situation of the rescue site, providing first-hand basis for subsequent decision adjustment, and ensuring that the command center can accurately grasp the rescue situation;

[0070] Real-time collection of actual field path, obtained by fusing GPS positioning and unmanned aerial vehicle inspection data;

[0071] When the error between the actual path and the planned path exceeds 50 meters, the A* algorithm is triggered to re-plan the path;

[0072] (2) Dynamic optimization under closed-loop control: based on the returned information, the system automatically calls the feedback correction algorithm, adjusts the logic of feedback error, dynamically corrects the path planning and emergency response plan, and forms a closed-loop control of data sensing-prediction-response-feedback, so that the whole rescue process is efficiently promoted in continuous optimization, and the emergency action always meets the actual demand;

[0073] The expression formula of feedback error adjustment is:

[0074] ΔP=γ(P real -P plan )

[0075] In the formula, ΔP: path adjustment correction amount, γ: feedback correction proportion coefficient, P real : actual path progress of the rescue site (real-time collection), P plan : original path planning result; the feedback correction proportion coefficient is set to 0.8 (determined based on 50 field tests), balancing the correction sensitivity and stability.

[0076] The preprocessed output is in JSON format (including time and space stamp), the feature extraction output is in CSV format, the fusion output is in Tensor format, and the final plan output is in XML format, ensuring the data transmission specification between modules.

[0077] The beneficial effects of the present application are as follows:

[0078] 1、The present application breaks through the problem of information island and monitoring blind area existing in the existing natural disaster emergency rescue system by designing a multi-source perception data fusion architecture, the system comprehensively accesses unmanned aerial vehicle laser radar, satellite remote sensing, ground Internet of Things sensing, video monitoring, weather monitoring and social media data, constructs a multi-level, wide coverage, full space-time disaster perception network, realizes real-time transmission and dynamic sharing of multi-source data through wireless communication technologies such as 5G and NB-IoT, greatly improves the spatial perception breadth and information acquisition timeliness of the disaster site, compared with the traditional system relying on single monitoring means, the present application can quickly, comprehensively and stereoscopically capture the complex changes of the disaster site, significantly improve the comprehensiveness and real-time response capability of disaster monitoring, and provide a solid data foundation for subsequent accurate rescue.

[0079] 2、The present application constructs a cross-modal feature extraction and multi-source information deep fusion mechanism, aiming at the problems of inconsistent data format, information redundancy and data conflict existing in multi-source heterogeneous perception data, proposes a multi-modal feature extraction method based on deep convolutional neural network, long short-term memory network and natural language processing technology, combines the Transformer fusion architecture and the modal attention mechanism, realizes adaptive weighted fusion of multi-source perception information, effectively eliminates redundancy and conflict, and significantly improves the accuracy of disaster type automatic identification and influence range dynamic prediction, which can provide high credibility and timeliness data support for emergency command and help scientific, reasonable and efficient emergency decision-making.

[0080] 3、The present application integrates intelligent emergency response decision and multi-level rescue dispatching function, designs dynamic path planning and reinforcement learning optimization algorithm, can generate optimal personnel evacuation route, rescue path and material scheduling plan based on real-time perception data and disaster prediction results, the system supports multi-level cooperation of command center and front-line rescue forces, adjusts rescue scheme and path strategy in real time through emergency feedback and closed-loop dynamic correction mechanism, realizes dynamic emergency command in whole process and whole link, compared with traditional static plan or manual command mode, the present application can significantly improve rescue response speed and execution efficiency, minimize personnel casualties and property losses, and improve the overall intelligentization and practical application ability of natural disaster emergency rescue. BRIEF DESCRIPTION OF DRAWINGS

[0081] Figure 1 The present application is a natural disaster emergency rescue system flowchart based on multi-source perception information fusion. DETAILED DESCRIPTION

[0082] Clearly and completely, the technical solutions in the embodiments of the present application will be described in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor belong to the scope of protection of the present application.

[0083] As shown in Figure 1 The embodiments of the present application provide a natural disaster emergency rescue system based on multi-source perception information fusion, which is composed of a multi-source perception data acquisition module, a data preprocessing and space-time registration module, a cross-modal feature extraction module, a multi-modal information fusion and conflict resolution module, a disaster type identification and grade discrimination module, a disaster influence range prediction and diffusion modeling module, a dynamic emergency path planning and response plan generation module, a rescue dispatching and command control module, and an emergency feedback and closed-loop dynamic correction module.

[0084] The multi-source perception data acquisition module collects multi-source data of unmanned aerial vehicles, satellites, ground sensors, video surveillance and social media in real time to ensure comprehensive information. The data preprocessing and space-time registration module denoises, synchronizes and standardizes the multi-source data to improve data quality. The cross-modal feature extraction module extracts spatial, temporal and text multi-modal features using deep learning and encodes them into fusion input. The multi-modal information fusion and conflict resolution module realizes deep fusion of heterogeneous data through Transformer and attention mechanism and removes redundancy. The disaster type identification and grade discrimination module automatically identifies disaster types and grades based on fusion features to provide accurate classification. The disaster influence range prediction and diffusion modeling module predicts disaster spread path and potential impact area using mathematical models. The dynamic emergency path planning and response plan generation module dynamically generates the optimal rescue path based on the prediction results using reinforcement learning algorithm. The rescue dispatching and command control module realizes efficient dispatching and real-time monitoring of command centers and front-line equipment. The emergency feedback and closed-loop dynamic correction module adjusts the path and plan in real time through field feedback to build a continuous optimization closed loop.

[0085] The present application further ensures real-time closed loop of multi-source perception, information fusion, disaster identification, path generation and rescue dispatching through high coupling design between modules. The system has continuous self-adjustment and self-optimization capabilities and can adapt to dynamic changes and sudden situations in disaster sites. In addition, the present application introduces an environmental perception and command dispatching bidirectional feedback channel to realize high-speed interaction of command center and field rescue data, ensuring that the rescue path and plan are always in the optimal state. The overall architecture has high reliability, scalability and rapid response capability, and is particularly suitable for complex, sudden and variable natural disaster rescue scenarios. It provides technical support for building an efficient, intelligent and comprehensive emergency rescue system and significantly improves the application value of the system in actual combat.

[0086] Example One: A sudden flood emergency rescue application in a certain river basin

[0087] 1. Multi-source perception data collection

[0088] Unmanned aerial vehicle laser radar unit: deploy 3 unmanned aerial vehicles to scan a 5km range along the coast of the river basin to obtain terrain elevation data (point cloud density 200 points / ㎡) and real-time water level change images;

[0089] Satellite remote sensing unit: call on Gaofen-7 satellite to obtain 10-meter resolution remote sensing images of the river basin to identify the boundaries of the flooded area;

[0090] Ground Internet of Things sensor group: 50 water level sensors (sampling frequency 10Hz, error ±2cm) and 20 soil moisture sensors (monitoring embankment stability) along the coast transmit data in real time;

[0091] Video monitoring unit: 30 coastal cameras (1080P, 25fps) capture embankment breach risk points and trapped personnel images;

[0092] Social media access unit: capture information on local microblog, TikTok and other platforms with keywords such as "flood" and "trapped" to filter out 23 effective rescue information (processed by XLM-R model multi-language processing, including 1 mixed English and Chinese information);

[0093] Weather monitoring terminal: real-time transmission of rainfall (50mm / h) and wind speed (8m / s) data.

[0094] 2. Data preprocessing and spatio-temporal registration

[0095] Basic preprocessing: wavelet threshold denoising (db4 wavelet) is used for unmanned aerial vehicle images to remove raindrop interference; synchronize device times through NTP protocol (deviation ≤10ms) and align satellite, ground and unmanned aerial vehicle data in space based on WGS84 coordinate system;

[0096] Type-specific processing: video images are de-distorted and enhanced (30% improvement in clarity), sensor data are smoothed (eliminate transient fluctuations), and social media text is cleaned (remove irrelevant comments and retain rescue coordinate information).

[0097] 3. Cross-modal feature extraction and fusion

[0098] Feature extraction:

[0099] Unmanned aerial vehicle / satellite images are extracted for spatial features (such as flooded area outline, embankment crack) using ResNet-18;

[0100] Water level / rainfall time series data is extracted by a 2-layer LSTM (128 neurons) to extract trend features (e.g., water level rising 0.4 meters per hour);

[0101] Social media text is extracted by the XLM-R model to extract semantic features (e.g., the location and needs of "XX village 3 people trapped on the roof");

[0102] Multi-modal fusion: dynamically weighted fusion of features through a 6-layer Transformer (8 attention heads), with water level sensor data weight 0.4 (high real-time), satellite image weight 0.3 (wide range), and social media weight 0.2 (precise rescue information), eliminating redundant and repetitive video features.

[0103] 4. Disaster identification and impact prediction

[0104] Type and level: the system calls a pre-trained model, combines the fusion features and historical flood knowledge base, and determines it as "sudden flood (level II)", with the core impact area being 3 km along the coast;

[0105] Diffusion prediction: based on GIS maps and a flood diffusion model (combined with water level rising speed 0.4m / h and terrain slope 15°), it is predicted that the flooded area will expand to 5 km after 1 hour 2 , threatening 2 villages downstream.

[0106] 5. Dynamic path planning and rescue dispatch

[0107] Path planning: using DDPG reinforcement learning algorithm, "safety first (coefficient 0.7) + efficiency (coefficient 0.3)" generates path:

[0108] Rescue vehicle route: avoid flooded sections, choose inside the embankment highlands, total distance 8.5 km, estimated time 22 minutes;

[0109] Personnel evacuation route: 5 evacuation channels are planned for 3 villages, prioritizing evacuation to temporary resettlement sites with an elevation of ≥50 meters;

[0110] Dispatch command: the command center issues instructions through the UI interface (left side heat map shows the flooded area, right side marks rescue resources):

[0111] Dispatch 5 rescue vehicles (with inflatable boats) to start according to the planned path, and 2 drones to scout the breach first;

[0112] Frontline personnel terminals receive "priority transfer of the elderly and children" hierarchical instructions and real-time position feedback (error ≤10 meters).

[0113] 6. Emergency feedback and closed-loop correction

[0114] On-site feedback: the UAV returns a new picture showing a secondary landslide on a certain section, with a deviation of 60 meters from the planned path (exceeding the 50-meter threshold);

[0115] Dynamic correction: the system automatically calls the A* algorithm to re-plan the route, bypassing the landslide area (adding 1.2 km of new distance, increasing the time by 5 minutes), synchronously updating the rescue vehicle instructions, and adjusting the resettlement point and resource allocation in real time at the command center.

[0116] Among them, the multi-source perception data collection module refers to the system covering unmanned aerial vehicle laser radar unit, satellite remote sensing image acquisition unit, ground Internet of Things sensor group unit (including earthquake sensor, flood water level meter, gas detection sensor, temperature and humidity sensor), fixed video monitoring unit, social media data access unit, meteorological monitoring terminal unit, unmanned aerial vehicle laser radar and satellite remote sensing collect geographic information from the air and space; the ground Internet of Things sensor group monitors geological and environmental changes through multiple sensors such as earthquakes and water levels; the social media data access unit obtains social public opinion; the meteorological monitoring terminal collects meteorological information, and multiple units work together to collect data from multiple dimensions.

[0117] The system uses 5G and NB-IoT wireless communication networks to transmit raw perception data collected by each unit in real time, ensuring data timeliness. After integrating different types of data, multi-modal basic data is formed. Through the correlation and synergistic effect of each element in the data collection formula, comprehensive and accurate data support is provided for subsequent data processing, analysis, and application.

[0118] Among them, the data preprocessing and spatio-temporal registration module refers to a series of basic preprocessing operations performed by the system after receiving the raw data from the collection module. Through denoising and filtering, interference information is removed from the data. Time synchronization and spatial coordinate registration techniques are used to accurately align different source data in time and space dimensions. The spatio-temporal registration formula is used to accurately locate the data, laying a solid foundation for subsequent processing.

[0119] Denoising uses the "wavelet threshold denoising algorithm" with db4 wavelet basis, which filters interference information through soft threshold processing.

[0120] Time synchronization uses the "NTP protocol" to ensure that the time deviation of each sensing unit is controlled within ±10ms.

[0121] Spatial registration is based on the "WGS84 coordinate system" and uses a coordinate conversion model to align different source data in space, such as converting satellite remote sensing data and ground sensor data to the same coordinate system.

[0122] Among them, the cross-modal feature extraction module refers to the standardized dataset entering the processing flow, and the multi-channel deep convolutional neural network excavates the spatial features of image data. The long short-term memory network is good at processing sensor data and accurately extracting time series features. Natural language processing technology screens key information from social media text. Multiple technologies work together to comprehensively obtain the core features of each modal data.

[0123] The extracted spatial, time series and text features of different modalities are processed by the feature encoder. Just like multi-modal feature mapping, the feature encoder maps these features to the same feature vector space, eliminating modal differences and realizing the organic integration of features. Finally, a structured and complete input feature stream is formed, laying the foundation for subsequent analysis and prediction.

[0124] Among them, the multi-modal information fusion and conflict resolution module refers to the system building a multi-source data interaction bridge through the Transformer fusion architecture, focusing on key information through the modal attention mechanism, and dynamically regulating the spatial details of each modal weight image, the timing changes of sensors, and the semantic tendency of text in the process of modal weighted fusion. Real-time value flexibly allocates weights, synchronously eliminates redundant and conflicting data, and through this processing, different modal features break through the information barrier to achieve deep collaboration, not only preserving the integrity of multi-dimensional data, but also strengthening core features through weight optimization. The final generated dataset has high reliability and adaptability, accurately meeting the needs of disaster identification for data consistency and effectiveness, providing solid data support for subsequent disaster research.

[0125] Among them, the disaster type identification and grade determination module refers to the system calling a pre-trained multi-label classification deep learning model based on the fused feature dataset, integrating the advantages of multi-modal features through disaster classification fusion mechanism, combining the spatial distribution of image data, the timing evolution of sensor data and the public opinion dynamics of text data, and combining the typical disaster feature patterns in the historical disaster knowledge base. Real-time identification of disaster types such as earthquakes and floods, and determination of disaster grades based on feature weight distribution, the identification results are automatically transmitted to the diffusion prediction module to provide clear classification and clear grade pre-data support for disaster spatio-temporal deduction.

[0126] Among them, the disaster impact range prediction and diffusion modeling module refers to the system receiving the disaster type and grade identification results, relying on the spatial framework of GIS maps, associating the regular patterns in the historical diffusion database, starting the diffusion prediction process, and through the adaptive disaster diffusion mathematical model such as the diffusion equation for floods and the wave propagation model for earthquakes, integrating various data into the prediction logic, dynamically calculating the disaster impact range and spread path, and simultaneously predicting potential secondary disaster areas.

[0127] The dynamic emergency path planning and response plan generation module refers to calling a reinforcement learning path optimization algorithm and a graph search algorithm based on disaster influence range prediction results, and a path optimization reward function plays a role in the process. The optimal solution is generated by dynamically adjusting the efficiency and safety of the path, driving the algorithm to generate the optimal solution, so as to obtain emergency response paths, personnel evacuation routes and other key paths. During plan generation, real-time environmental changes, road traffic conditions and safety factors are fully taken into account. Through the algorithm, the generated rescue priority and unmanned aerial vehicle inspection path are more in line with actual needs, ensuring efficient and safe emergency operations and providing scientific path guidance for disaster response.

[0128] The rescue dispatching and command control module refers to the command center relying on multi-device dispatching instructions to accurately issue multi-level dispatching instructions to unmanned aerial vehicles, rescue vehicles and emergency personnel terminals after the system receives dynamic emergency paths and response plans. At the same time, the system collects the real-time execution status of each device and personnel, ensuring the accuracy of instruction transmission and the visibility of the execution process, providing real-time basis for dispatching adjustment. The system supports a multi-level control architecture, with efficient cooperation between front-line command and central command, forming a command system with top-down linkage. The dispatching process realizes full-link monitoring, and each link from instruction generation, issuance to execution feedback can be traced back, ensuring the effective landing of dispatching instructions and improving the overall coordination and execution of emergency response.

[0129] The emergency feedback and closed-loop dynamic correction module refers to the system continuously collecting on-site rescue progress, environmental changes, new perception data and personnel positioning information, which are timely returned to the command center through the feedback channel. These dynamic information comprehensively reflects the actual situation of the rescue site, providing first-hand basis for subsequent decision adjustment, ensuring that the command center can accurately grasp the rescue situation.

[0130] Based on the returned information, the system automatically calls the feedback correction algorithm, adjusts the path planning and emergency response plan by means of feedback error adjustment logic, and forms a closed-loop control of data perception-prediction-response-feedback, so that the whole rescue process can be efficiently promoted in continuous optimization, ensuring that the emergency operations always meet the actual needs.

[0131] It should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device.

[0132] While embodiments of the application have been shown and described, it is to be understood that the application is not limited to the details of the embodiments described, since numerous changes, modifications, substitutions and variations can be made thereto without departing from the spirit and scope of the application as defined by the appended claims and their equivalents.

Claims

1. A natural disaster emergency rescue system based on multi-source sensing information fusion, characterized in that: The system consists of a multi-source sensing data acquisition module, a data preprocessing and spatiotemporal registration module, a cross-modal feature extraction module, a multi-modal information fusion and conflict resolution module, a disaster type identification and level determination module, a disaster impact range prediction and diffusion modeling module, a dynamic emergency path planning and response plan generation module, a rescue dispatch and command control module, and an emergency feedback and closed-loop dynamic correction module. The multi-source sensing data acquisition module collects multi-source data in real time from drones, satellites, ground sensors, video surveillance, and social media. The data preprocessing and spatiotemporal registration module denoises, synchronizes, and standardizes multi-source data; the cross-modal feature extraction module uses deep learning to extract spatial, temporal, and textual multimodal features, uniformly encoding them into fusion input; the multimodal information fusion and conflict resolution module achieves deep fusion of heterogeneous data through Transformer and attention mechanisms, removing redundancy; the disaster type identification and level judgment module automatically identifies disaster types and levels based on fusion features; the disaster impact range prediction and diffusion modeling module uses mathematical models to predict disaster spread paths and potential impact areas; the dynamic emergency path planning and response plan generation module dynamically generates optimal rescue paths based on prediction results using reinforcement learning algorithms; the rescue dispatch and command control module enables efficient dispatch and real-time monitoring of command centers and frontline equipment; and the emergency feedback and closed-loop dynamic correction module adjusts paths and plans in real time through on-site feedback, constructing a continuously optimized closed loop.

2. The natural disaster emergency rescue system based on multi-source sensing information fusion according to claim 1, characterized in that: The multi-source sensing data acquisition module includes: (1) Multi-source data acquisition unit composition: The system includes UAV lidar unit, satellite remote sensing image acquisition unit, ground IoT sensor group unit, fixed video surveillance unit, social media data access unit, and meteorological monitoring terminal unit. UAV lidar and satellite remote sensing collect geographic information from the air and space; the ground IoT sensor group monitors geological and environmental changes through earthquake and water level sensors; fixed video surveillance records real-time images; social media data access unit acquires public opinion; meteorological monitoring terminal collects meteorological information, and multiple units collect data collaboratively. (2) Wireless transmission and multimodal data integration: With the help of 5G and NB-IoT wireless communication networks, the raw sensing data collected by each unit is transmitted in real time. Different types of data are integrated to form multimodal basic data. The elements in the data acquisition formula are interconnected and work together. The formula for expressing data collection is: In the formula, D total (t): The total amount of multi-source data collected by the system at time t, D i (t): Real-time data of the i-th sensing unit at time t, n: Total number of sensing units.

3. A natural disaster emergency rescue system based on multi-source sensing information fusion according to claim 1, characterized in that: The data preprocessing and spatiotemporal registration module includes: (1) Basic data preprocessing operations: After receiving the raw data from the acquisition module, the system removes interference information and eliminates outliers by denoising and filtering; it uses time synchronization and spatial coordinate registration technology to make data from different sources accurately aligned in time and space, and uses the time-space registration formula to accurately locate the data; D norm (x,y,t)=f sync (D total (t),T offset ,S offset ) In the formula, D norm (x,y,t): A standardized dataset after time synchronization and spatial registration, f sync Spatiotemporal registration function, T offset : Time offset of sensing unit, S offset Spatial coordinate offset of the sensing unit; (2) In-depth processing of different types of data: For different types of data, the system adopts targeted processing methods. For image data, distortion correction and image enhancement algorithms are used to restore the real picture and improve the clarity. For sensor data, smoothing and anomaly detection are used to optimize the data curve and find abnormal fluctuations. For social media data, text cleaning is performed to remove irrelevant information and finally output a standardized dataset with spatiotemporal alignment and uniform format.

4. A natural disaster emergency rescue system based on multi-source sensing information fusion according to claim 1, characterized in that: The cross-modal feature extraction module includes: (1) Multimodal data feature extraction: After the standardized dataset enters the processing flow, the multi-channel deep convolutional neural network performs spatial feature mining on the image data to capture the details and structure of the picture; the long short-term memory network accurately extracts time series features; natural language processing technology filters key information from social media texts. Multiple technologies work together to comprehensively obtain the core features of each modality of data. (2) Feature mapping and fusion: The extracted spatial, temporal and textual modal features are processed by the feature encoder. Just as multimodal feature mapping integrates information from different dimensions, the feature encoder maps these features to the same feature vector space, eliminates modal differences, realizes the organic fusion between features, and finally forms a unified and complete input feature stream. The formula for expressing multimodal feature mapping is: F multi =φ cnn (I)+φ lstm (S)+φ nlp (T) In the formula, F multi : The fused multimodal feature vector, φ cnn (I): Spatial features extracted from input image data I via CNN, φ lstm (S): Time series features extracted from the input time series data S using LSTM, φ nlp (T): Semantic features extracted from input text data T by NLP.

5. A natural disaster emergency rescue system based on multi-source sensing information fusion according to claim 1, characterized in that: The multimodal information fusion and conflict resolution module refers to a system that builds a multi-source data interaction bridge based on cross-modal feature flow through the Transformer fusion architecture. It combines a modal attention mechanism to focus on key information. In this process, modal weighted fusion dynamically adjusts the spatial details of the image, the temporal changes of the sensor, and the semantic tendency of the text of each modality weight image. It flexibly allocates weights according to real-time value and simultaneously removes redundant and conflicting data.

6. A natural disaster emergency rescue system based on multi-source sensing information fusion according to claim 1, characterized in that: The disaster type identification and level determination module refers to a system that uses a fusion feature dataset as a basis, calls a pre-trained multi-label classification deep learning model, integrates the advantages of multimodal features through a disaster classification fusion mechanism, considers the spatial distribution of image data, the temporal evolution of sensor data and the public opinion dynamics of text data, and combines typical disaster feature patterns in the historical disaster knowledge base to identify earthquake and flood disaster types in real time, and determines the disaster level based on feature weight allocation. The identification results are automatically transmitted to the diffusion prediction module.

7. A natural disaster emergency rescue system based on multi-source sensing information fusion according to claim 1, characterized in that: The disaster impact range prediction and diffusion modeling module includes: (1) Disaster spread dynamic prediction mechanism: After receiving the disaster type and level identification results, the system relies on the spatial framework of the GIS map, associates the regular patterns in the historical spread database, starts the spread prediction process, integrates various data into the prediction logic through the adapted disaster spread mathematical model, dynamically calculates the disaster impact range and spread path, and simultaneously predicts the potential area of ​​secondary disasters. The diffusion prediction formula is: In the formula, A(t): the area affected by the disaster at time t, A0: the initial area affected by the disaster, v(s): the diffusion rate of the disaster boundary, and t0: the initial time of the disaster; (2) Prediction results output and emergency support: The dynamic results obtained from the diffusion prediction fully present the development trend of the disaster. This information is automatically output to the emergency path generation module, providing accurate spatial and temporal basis for subsequent formulation of evacuation routes and allocation of rescue resources.

8. A natural disaster emergency rescue system based on multi-source sensing information fusion according to claim 1, characterized in that: The dynamic emergency path planning and response plan generation module includes: (1) Emergency path intelligent generation mechanism: Based on the disaster impact range prediction results, the system calls the reinforcement learning path optimization algorithm and graph search algorithm. The path optimization reward function plays an implicit role in this process, and is dynamically adjusted according to factors such as path efficiency and safety, driving the algorithm to generate the optimal solution, thereby obtaining various key paths such as emergency response paths and personnel evacuation routes. The formula for the path optimization reward function is as follows: In the formula, R total : Total reward value for path planning, D k : The distance traveled by path node k, T k : Estimated time S for path node k k λ: security score of path node k, λ: time penalty coefficient, μ: security priority coefficient, p: total number of path nodes; (2) Multi-dimensional consideration of dynamic contingency plans: When generating contingency plans, real-time environmental changes, road traffic conditions and safety coefficient variables are fully incorporated. Through the comprehensive weighing of these factors by algorithms, the generated rescue priorities and drone inspection paths are more in line with actual needs.

9. A natural disaster emergency rescue system based on multi-source sensing information fusion according to claim 1, characterized in that: The rescue dispatch and command control module includes: (1) Command issuance and execution status collection: After receiving the dynamic emergency path and response plan, the command center relies on multi-device dispatch instructions to accurately issue multi-level dispatch instructions to drones, rescue vehicles and emergency personnel terminals, and collects the on-site execution status of each device and personnel in real time to ensure the accuracy of instruction transmission and the visibility of the execution process. The formula for expressing multi-device scheduling instructions is: CMD i (t)=f sched (R total ,Loc i (t),State i (t)) In the formula, CMD i (t): The instruction set of the i-th rescue device at time t, f sched : A scheduling function based on reward function, device location, and device status, Loc i (t): The current position of the i-th device, State i (t): The current working state of the i-th device; (2) Multi-level collaboration and full-link monitoring: The system supports a multi-level control architecture, enabling efficient collaboration between front-line command and central command to form a command system that links the upper and lower levels. The scheduling process is monitored throughout the entire link, and every step from instruction generation and issuance to execution feedback is traceable.

10. A natural disaster emergency rescue system based on multi-source sensing information fusion according to claim 1, characterized in that: The emergency feedback and closed-loop dynamic correction module includes: (1) Real-time on-site information feedback mechanism: The system continuously collects on-site rescue progress, environmental changes, new sensing data and personnel positioning information, and promptly transmits them back to the command center through the feedback channel. This dynamic information fully reflects the actual situation at the rescue site. (2) Dynamic optimization under closed-loop control: Based on the feedback information, the system automatically calls the feedback correction algorithm and uses the feedback error adjustment logic to dynamically correct the path planning and emergency response plan, thereby forming a closed-loop control of data perception-prediction-response-feedback. The formula for feedback error adjustment is: ΔP=γ(P real -P plan ) In the formula, ΔP: path adjustment correction amount, γ: feedback correction proportional coefficient, P real : Actual progress of the rescue route at the scene, P plan Original path planning results.

Images