Real-time intelligent disaster early warning method based on multi-modal data

Abstract

The invention discloses a real-time intelligent disaster early warning method based on multi-modal data, and belongs to the technical field of disaster early warning, and the method comprises the steps: 1, constructing a multi-modal data platform; 2, obtaining a first training model by using a graph neural network technology, and obtaining a second training model in combination with a federated learning technology; 3, integrating the first training model and the second training model to obtain a disaster propagation model; 4, deploying a response function on edge equipment close to the data source; and 5, continuously collecting data of actual disaster occurrence, comparing the data with a model prediction result, and carrying out early warning response. According to the invention, by integrating disaster data obtained by a plurality of data sources, more data sources can be obtained, the comprehensiveness and accuracy of early warning can be improved, and the generalization ability of the model can be significantly improved by constructing the disaster recognition model by using the graph neural network and federated learning, so that the system can cope with various complex and changeable disaster scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of disaster warning, and in particular relates to a real-time intelligent disaster warning method based on multimodal data. Background Art

[0002] As global climate change intensifies, the frequency and intensity of extreme weather events and natural disasters are increasing, posing a major threat to human society. Traditional disaster warning systems are often limited by a single data source, centralized processing, and limited response speed, making it difficult to meet the needs of fast and accurate warnings; multimodal data refers to a diverse collection of information from different sensors and data sources, including but not limited to remote sensing images, ground sensor data, social media information, weather forecasts, historical disaster records, etc. Therefore, in the disaster warning system, it is particularly important to have a real-time intelligent disaster warning method based on multimodal data to predict disasters more comprehensively and accurately. Summary of the invention

[0003] The purpose of the present invention is to provide a real-time intelligent disaster warning method based on multimodal data to solve the problems faced in the above-mentioned background technology.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] A real-time intelligent disaster warning method based on multimodal data, the method comprising:

[0006] Step 1: Obtain disaster-related data through multiple data sources such as remote sensing data, ground sensor data, crowdsourcing data, and historical disaster records, and build a multimodal data platform through data fusion technology;

[0007] Step 2: Construct a geospatial map, determine multiple nodes according to the disaster type, and extract feature information at each node corresponding to the disaster from the multimodal data platform. Use graph neural network technology to train the extracted feature information to obtain the first training model. At the same time, combine federated learning technology to perform collaborative training based on relevant model parameters uploaded by different institutions or regions to obtain the second training model.

[0008] Step 3: Integrate the first training model and the second training model to obtain a disaster propagation model of each node;

[0009] Step 4: Combine edge computing technology to deploy data preprocessing, preliminary analysis, and some early warning response functions on edge devices close to the data source;

[0010] Step 5: Establish a feedback loop, continuously collect data on actual disasters, compare the actual situation with the results predicted by the model, and respond to early warnings based on the differences.

[0011] Furthermore, the method for constructing a multimodal data platform in step 1 is:

[0012] S11. Screen and verify the reliability of the relevant data obtained from each data source to obtain cleaned reliable data;

[0013] S12, align the reliable data from each data source in time and space, extract features, and use data fusion engine technology for comprehensive processing to obtain fused data Q of each data item type. k ;

[0014] S13. Integrate the fused data from various data sources to build a multimodal data platform.

[0015] Furthermore, the fusion data Q in S12 k The acquisition method is:

[0016] By formula Get the fused data Q of each data item type k ;

[0017] Where Q k The fused data of the k-th data item type, is the parameter value collected under the i-th data source for the k-th data item type, is the weight coefficient of the kth data item type under the i-th data source, n is the number of data sources, and i∈(1,n).

[0018] Furthermore, the method for obtaining the first training model in step 2 is:

[0019] A geospatial map is constructed according to the disaster type, and multiple nodes are set on the geospatial map. The data item type related to the disaster type of each node is extracted to construct the first training model MO1. At the same time, the second training model MO2 is constructed according to the relevant model parameters uploaded by different institutions and regions.

[0020] Furthermore, the method for obtaining the disaster propagation model of each node in step 3 is:

[0021] By formula The disaster propagation model MO of each node is obtained, where α1 and α2 are the conversion coefficients of the preset first training model and the second training model respectively.

[0022] Furthermore, the method for performing the early warning response in step 5 is:

[0023] Collect parameter information related to the disaster type in real time and input it into the disaster propagation model of each node to obtain the disaster value B of each node, so as to obtain the disaster value B of each node through the formula F = BBth The deviation value F is obtained. When the deviation value F is greater than 0, an early warning response is performed;

[0024] Among them, B th is the disaster threshold predicted by the model.

[0025] Furthermore, the method for performing early warning response in step 5 further includes:

[0026] When no early warning response is performed, every detection period ΔT, according to the disaster value of each node, a curve B showing the change of disaster value over time for each node is drawn up. j (t);

[0027] By formula Obtain the early warning risk value Du of each node;

[0028] The obtained early warning risk value Du is compared with the early warning risk threshold Du set by the system th To compare:

[0029] When Du>Du th When an early warning response is carried out;

[0030] Among them, a1 and a2 are preset coefficients, The standard curve of the disaster value set for each node over time, t a is the start time of the detection cycle, t b is the end time of the detection cycle, m is the number of nodes, and j∈(1,m).

[0031] Beneficial effects of the present invention:

[0032] The present invention integrates disaster data obtained from data sources such as multimodal remote sensing data and crowdsourcing data to obtain more data sources, which can improve the comprehensiveness and accuracy of early warnings. It also uses graph neural networks and federated learning to build a disaster identification model, which can significantly improve the generalization ability of the model, enabling the system to cope with various complex and changeable disaster scenarios. At the same time, combined with the rapid response mechanism of edge computing, it can achieve accurate identification of disasters, real-time early warning and response, so as to timely understand the occurrence of disasters and provide strong technical support for disaster management and emergency response.

[0033] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0035] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] In one embodiment, a real-time intelligent disaster warning method based on multimodal data is disclosed. This embodiment is described by taking flood warning as an example. Figure 1 As shown, the early warning method mainly includes the following steps:

[0038] Step 1: Obtain disaster-related data through multiple data sources such as remote sensing data, ground sensor data, crowdsourcing data, and historical disaster records, and build a multimodal data platform through data fusion technology;

[0039] Step 2: Construct a geospatial map, determine multiple nodes according to the disaster type, and extract feature information at each node corresponding to the disaster from the multimodal data platform. Use graph neural network technology to train the extracted feature information to obtain the first training model. At the same time, combine federated learning technology to perform collaborative training based on relevant model parameters uploaded by different institutions or regions to obtain the second training model.

[0040] Step 3: Integrate the first training model and the second training model to obtain a disaster propagation model of each node;

[0041] Step 4: Combine edge computing technology to deploy data preprocessing, preliminary analysis, and some early warning response functions on edge devices close to the data source;

[0042] Step 5: Establish a feedback loop, continuously collect data on actual disasters, compare the actual situation with the results predicted by the model, and respond to early warnings based on the differences.

[0043] Through the above technical scheme, the present application first obtains disaster-related data through multiple data sources, such as remote sensing data, ground sensor data, social media information, and historical flood records, and integrates them into a unified data set to build a multimodal data platform. Then, a geographic spatial map is constructed according to the flood disaster, and multiple important nodes are determined on the spatial map, so as to extract the characteristic information at each node corresponding to the disaster from the multimodal data platform, and use the graph neural network technology to train the extracted characteristic information to obtain the first training model. At the same time, combined with the federated learning technology, collaborative training is performed according to the relevant model parameters uploaded by different institutions or regions to obtain the second training model. The first training model and the second training model are integrated to obtain the disaster propagation model of each node, and a feedback loop is established to continuously collect data on actual disaster occurrence, compare the actual situation with the results predicted by the model, and respond to the early warning according to the difference. Once a difference is found, a signal is immediately sent to the response device deployed on the edge device to respond to the alarm. In this way, disaster data obtained from data sources such as multimodal remote sensing data and crowdsourcing data can be integrated to obtain more data sources, which can improve the comprehensiveness and accuracy of early warnings. The disaster identification model is built using graph neural networks and federated learning, which can significantly improve the generalization ability of the model, enabling the system to cope with various complex and changeable disaster scenarios. At the same time, combined with the rapid response mechanism of edge computing, accurate identification, real-time early warning and response to disasters can be achieved, so that the occurrence of disasters can be understood in a timely manner, providing strong technical support for disaster management and emergency response.

[0044] As an implementation of the present invention, the method for constructing a multimodal data platform in step 1 is:

[0045] S11. Screen and verify the reliability of the relevant data obtained from each data source to obtain cleaned reliable data;

[0046] S12, align the reliable data from each data source in time and space, extract features, and use data fusion engine technology for comprehensive processing to obtain fused data Q of each data item type. k , through the formula Get the fused data Q of each data item type k , where Q k The fused data of the k-th data item type, is the parameter value collected under the i-th data source for the k-th data item type, is the weight coefficient of the kth data item type under the i-th data source, n is the number of data sources, and i∈(1,n);

[0047] S13. Integrate the fused data from various data sources to build a multimodal data platform.

[0048] Through the above technical solution, this embodiment provides a specific method for constructing a multimodal data platform. First, data related to rainfall, node water level, soil permeability, terrain (narrowness, steepness), etc. are obtained from various data sources including but not limited to remote sensing data, ground sensor data, social media information (crowdsourcing information), and historical flood disaster records. The data is cleaned, screened, and reliability verified to obtain cleaned reliable data. The reliable data of each data source is spatiotemporally aligned and features are extracted. The data fusion engine technology is used to obtain the reliable data through the formula Comprehensively obtain the fusion data Q of each data item type k For example, according to the river water level obtained from various data sources, a certain weight is assigned to each data source for comprehensive analysis to obtain the fused river water level value, and finally the fused data of each data source is integrated to build a multimodal data platform. In this way, multiple parameter information can be obtained from multiple data sources, which can greatly improve the data source, thereby improving the comprehensiveness and accuracy of the early warning.

[0049] As an implementation mode of the present invention, the method for obtaining the first training model in step 2 is: construct a geographic spatial map according to the disaster type, set multiple nodes on the geographic spatial map, extract the data item type related to the disaster type of each node, thereby constructing the first training model MO1, and at the same time constructing the second training model MO2 according to the relevant model parameters uploaded by different institutions and regions. The method for obtaining the disaster propagation model of each node in step 3 is: through the formula The disaster propagation model MO of each node is obtained, where α1 and α2 are the conversion coefficients of the preset first training model and the second training model respectively.

[0050] Through the above technical solution, this embodiment provides a specific method for obtaining the disaster propagation model of each node. Taking flood disaster as an example, a geographic space map is constructed according to the flood flow direction and terrain of the region. These nodes can be important nodes such as river intersections, reservoirs, lakes, urban areas, etc., and then the data item type parameters related to the node and the flood disaster are obtained. Through the formula The first training model MO1 is constructed, where W0 is the node water level, Q R is the precipitation, Q D is the measured node terrain narrowness, Q X is the measured node terrain steepness, Q Sis the soil permeability value of the measured node terrain, β, ε1, ε2, θ, δ are the respective preset influence coefficients; it can be seen from the formula that the greater the precipitation at the node, the higher the node water level, the greater the narrowness or steepness of the terrain, or the smaller the land infiltration rate, the greater the possibility of floods. Therefore, the greater the output value of the first training model, the greater the possibility of floods. Since the occurrence of floods is not only related to factors such as precipitation and node terrain, but also to the operation status of node reservoirs, the relevant model parameters uploaded by different node institutions and regions, including the dam pressure Q of the node reservoir, are used. P , Water leakage situation Q L , and the amount of water entering the reservoir over a period of time Q IN (t) and the outbound quantity Q OUT (t) changes, so through the formula A second training model MO2 is constructed, where: is the standard deviation of dam pressure, the formula It is expressed as the difference between the inflow and outflow of the reservoir from time t1 to time t2. The larger the value, the more the inflow of the reservoir exceeds the outflow of the reservoir, which may indicate that the reservoir has a higher flood risk when encountering similar rainfall. Similarly, when the water leakage situation Q L The larger the value, the greater the flood risk. The smaller the value, the more stable the dam pressure change and the smaller the flood risk. In any case, the larger the value, the greater the flood risk. Therefore, a comprehensive analysis is conducted based on the reservoir pressure, leakage, and the difference between the water inflow and outflow, and a second training model related to the reservoir operation status is constructed. It can be seen that the larger the value output by the second training model, the greater the possibility of floods. Therefore, in order to more accurately simulate the disaster propagation model MO of each node, the first training model MO1 and the second training model MO2 are integrated, and the formula is used. The disaster propagation model MO of each node is obtained. In this way, a comprehensive analysis can be conducted based on the operation of the node reservoir, the precipitation at the node, the node topography, and the soil infiltration conditions, and the flood propagation model of the node can be simulated more accurately, thereby improving the accuracy of the early warning.

[0051] It should be noted that, in the above technical solution, the conversion coefficients α1 and α2, and the respective preset influence coefficients β, ε1, ε2, δ can be determined based on relevant data and empirical data in big data, which will not be described here.

[0052] As an implementation of the present invention, the method for performing early warning response in step 5 is:

[0053] Collect parameter information related to the disaster type in real time and input it into the disaster propagation model of each node to obtain the disaster value B of each node, so as to obtain the disaster value B of each node through the formula F = BB th The deviation value F is obtained. When the deviation value F is greater than 0, an early warning response is performed;

[0054] Among them, B th is the disaster threshold under the model prediction;

[0055] The method for conducting an early warning response also includes: when no early warning response is conducted, a comprehensive analysis is conducted based on the disaster value of each node every detection cycle to obtain an early warning risk value, and when the early warning risk value exceeds a preset early warning risk threshold, an early warning response is conducted.

[0056] Through the above technical solution, this embodiment provides a method for the early warning response system to perform early warning response. First, parameter information related to the disaster type is collected in real time, such as parameter information related to flood disasters, and is sequentially introduced into the trained disaster propagation model to obtain an output value, that is, the disaster value B of each node. The obtained disaster value is compared with the disaster threshold B predicted by the model. th Compare, disaster threshold B th The size can be determined independently according to the actual situation of each node to obtain a deviation value. When the deviation value F is greater than 0, it means that the possibility of a flood is high at this time, and an early warning is issued, and the information is immediately sent to the response module on the edge device of the data source for an early warning response, so as to timely control the flood. When no early warning response is made, since the set disaster threshold is a fixed value set manually, an alarm response is only made when the disaster value exceeds the disaster threshold, otherwise no response is made. If the active disaster value is always near the disaster threshold for a period of time, even if no alarm response is made, it shows that there is a potential flood risk. Therefore, when no early warning response is made, every detection period ΔT, which can be determined manually, according to the disaster value of each node, a curve B of the disaster value change over time of each node is drawn up. j (t), and through the formula The warning risk value Du of each node is obtained, where t a is the start time of the detection cycle, t b is the end time of the detection cycle, The standard curve of the disaster value set for each node over time can be determined based on the historical data of the node operation combined with empirical data, through the formula It can determine the overall difference between the disaster value of a single node and all nodes in the historical time within the detection period, and the formula It means that when detecting the current time point t b The disaster value of a single node and the overall difference between all nodes, and the formula It represents the difference between the disaster value of a single node in the entire detection period and the preset standard disaster value, and then through the formula Comprehensive analysis is performed to obtain the early warning risk value. Since all nodes are in the same geographic space map, when the early warning risk value Du is greater than the preset early warning risk threshold Du th When the value of the disaster value is greater than that of the node, it indicates that there is a high potential flood risk. Therefore, in order to ensure safety, an early warning response is also carried out. In this way, the potential flood risk can be judged according to the change of the disaster value between each node, so as to make early predictions and reduce property losses.

[0057] It should be noted that in the above technical solution, the coefficients a1 and a2 set, the warning risk threshold Du set th They are all determined based on historical data and empirical data, so I will not elaborate on them here.

[0058] The above contents are merely examples and explanations of the concept of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A real-time intelligent disaster warning method based on multimodal data, characterized in that: The method comprises: Step 1: Obtain disaster-related data through multiple data sources such as remote sensing data, ground sensor data, crowdsourcing data, and historical disaster records, and build a multimodal data platform through data fusion technology; Step 2: Construct a geospatial map, determine multiple nodes according to the disaster type, and extract feature information at each node corresponding to the disaster from the multimodal data platform. Use graph neural network technology to train the extracted feature information to obtain the first training model. At the same time, combine federated learning technology to perform collaborative training based on relevant model parameters uploaded by different institutions or regions to obtain the second training model. Step 3: Integrate the first training model and the second training model to obtain a disaster propagation model of each node; Step 4: Combine edge computing technology to deploy data preprocessing, preliminary analysis, and some early warning response functions on edge devices close to the data source; Step 5: Establish a feedback loop, continuously collect data on actual disasters, compare the actual situation with the results predicted by the model, and respond to early warnings based on the differences.

2. A real-time intelligent disaster warning method based on multimodal data according to claim 1, characterized in that: The method for constructing a multimodal data platform in step 1 is: S11. Screen and verify the reliability of the relevant data obtained from each data source to obtain cleaned reliable data; S12, align the reliable data from each data source in time and space, extract features, and use data fusion engine technology for comprehensive processing to obtain fused data Q of each data item type. k ; S13. Integrate the fused data from various data sources to build a multimodal data platform.

3. A real-time intelligent disaster warning method based on multimodal data according to claim 2, characterized in that: The S12 fused data Q k The acquisition method is: By formula Get the fused data Q of each data item type k ; Where Q k The fused data of the k-th data item type, is the parameter value collected under the i-th data source for the k-th data item type, is the weight coefficient of the kth data item type under the i-th data source, n is the number of data sources, and i∈(1,n).

4. The real-time intelligent disaster warning method based on multimodal data according to claim 3 is characterized in that: The method for obtaining the first training model in step 2 is: A geospatial map is constructed according to the disaster type, and multiple nodes are set on the geospatial map. The data item type related to the disaster type of each node is extracted to construct the first training model MO1. At the same time, the second training model MO2 is constructed according to the relevant model parameters uploaded by different institutions and regions.

5. A real-time intelligent disaster warning method based on multimodal data according to claim 4, characterized in that: The method for obtaining the disaster propagation model of each node in step 3 is: By formula The disaster propagation model MO of each node is obtained, where α1 and α2 are the conversion coefficients of the preset first training model and the second training model respectively.

6. A real-time intelligent disaster warning method based on multimodal data according to claim 5, characterized in that: The method for performing early warning response in step 5 is: Collect parameter information related to the disaster type in real time and input it into the disaster propagation model of each node to obtain the disaster value B of each node, so as to obtain the disaster value B of each node through the formula F = BB th The deviation value F is obtained. When the deviation value F is greater than 0, an early warning response is performed; Among them, B th is the disaster threshold predicted by the model.

7. A real-time intelligent disaster warning method based on multimodal data according to claim 6, characterized in that: The method for performing early warning response in step 5 further includes: When no early warning response is performed, every detection period ΔT, according to the disaster value of each node, a curve B showing the change of disaster value over time for each node is drawn up. j (t); By formula Obtain the early warning risk value Du of each node; The obtained early warning risk value Du is compared with the early warning risk threshold Du set by the system th To compare: When Du>Du th When an early warning response is carried out; Among them, a1 and a2 are preset coefficients, The standard curve of the disaster value set for each node over time, t a is the start time of the detection cycle, t b is the end time of the detection cycle, m is the number of nodes, and j∈(1,m).

Images