Disaster situation rapid reporting method and device based on multi-source multi-modal data, storage medium and equipment
By constructing a causal relationship model and a dynamic fusion mechanism, the problem of spatiotemporal dynamic compatibility of multi-source and multimodal disaster data was solved, generating high-quality and highly accurate disaster reports and improving the efficiency and credibility of report generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING GLOBAL SAFETY TECH
- Filing Date
- 2025-12-27
- Publication Date
- 2026-05-29
AI Technical Summary
In the emergency response to disasters such as earthquakes and urban floods, how to optimize multimodal and multi-source data to generate high-quality and accurate disaster reports, especially how to ensure the spatiotemporal dynamic compatibility and logical consistency of multi-source data, so as to improve the physical credibility and generation efficiency of disaster reports.
By constructing a causal relationship model among disaster variables, combining physical laws and spatiotemporal transmission constraints, a systematic compatibility test is conducted on multi-source data. Using a causal-driven dynamic fusion mechanism, the fusion ratio of multi-source information is adaptively adjusted by comprehensively considering real-time evolution confidence and prior knowledge weights. Finally, a large model is used to generate a disaster bulletin.
It significantly improved the physical reliability and evolutionary consistency of disaster data, enhanced the system's ability to respond to sudden anomalies, and improved the efficiency and accuracy of disaster bulletin generation.
Smart Images
Figure CN122116559A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, storage medium and device for disaster rapid reporting of multi-source and multi-modal data. Background Technology
[0002] In emergency response to disasters such as earthquakes and urban flooding, real-time disaster reports integrating multimodal and multi-source disaster information are crucial supporting materials for event handling, rescue, and decision-making. High-quality disaster reports require high-quality integration of information from different modal data sources (such as text, video, and audio). The text information includes written reports and news updates from government or rescue organizations, typically containing information such as disaster type, affected areas, death toll, and damage assessment. The video information includes footage from disaster site monitoring, drone footage, and on-site videos from rescue teams, including details such as dynamic changes at the scene, the extent of damage, and the progress of post-disaster rescue efforts. The audio information includes voice reports from citizens via hotline, rescue personnel, and on-site command, including specific on-site feedback, disaster descriptions, and the locations of trapped individuals. Optimizing multimodal, multi-source data to generate high-quality and accurate disaster reports is of paramount importance. Summary of the Invention
[0003] This application provides a method, apparatus, storage medium, and device for disaster rapid reporting using multi-source, multi-modal data, to address the problem of how to optimize heterogeneous multi-source data and generate high-quality and high-accuracy disaster rapid reports. The technical solution is as follows: According to a first aspect of this application, a method for rapid disaster reporting using multi-source, multi-modal data is provided, the method comprising: Obtain multimodal disaster data sets from multiple data sources; For each disaster dataset corresponding to each data source, a causal relationship model describing the physical constraints of disaster evolution is constructed for each disaster variable in the disaster dataset. The local cost matrix constructed based on the causal relationship model is used to determine whether the disaster dataset meets the spatiotemporal dynamic compatibility. Based on the determination result, the real-time evolution confidence score of the disaster dataset is determined. For the same disaster variable in multiple disaster datasets, dynamic fusion weights are assigned based on the real-time evolution confidence scores and preset prior reliability scores corresponding to multiple data sources, and the fusion value of the disaster variable is calculated based on the dynamic fusion weights. A disaster report is generated by merging the values of various disaster variables using a large model.
[0004] In one possible implementation, determining whether the disaster dataset conforms to spatiotemporal dynamic compatibility using a local cost matrix constructed based on the causal relationship model includes: For each causal edge in the causal relationship model, the direction attribute of the causal edge is obtained, wherein positive causal relationship and negative causal relationship correspond to different direction attributes; Obtain the first observation sequence within a preset time window for the starting point of the dependent variable corresponding to the causal edge; Obtain the second observation sequence of the dependent variable endpoint corresponding to the causal edge within the time window; A local cost matrix under physical constraints is constructed based on the first observation sequence, the second observation sequence, and the direction attribute. The elements in the local cost matrix represent the matching distance between a sampling point in the first observation sequence and another observation point in the second observation sequence after considering physical polarity. The local cost matrix is calculated using a dynamic programming algorithm to calculate the cumulative minimum cost matrix, and the minimum cumulative cost and average lag steps are calculated from the minimum cost matrix. The average lag steps represent the average physical delay at which the dependent variable begins and ends. The feasibility of the disaster dataset is determined based on the minimum cumulative cost and the average lag step.
[0005] In one possible implementation, determining whether the disaster dataset conforms to spatiotemporal dynamics compatibility based on the minimum cumulative cost and the average lag step includes: The disaster type is determined based on the starting point and ending point of the dependent variable, and a reasonable physical transmission time interval for the disaster type is obtained. Determine whether the minimum cumulative cost is less than a preset fluctuation tolerance threshold, and whether the average lag step is within the physical transmission time interval; If so, then generate a judgment result showing that the disaster dataset conforms to spatiotemporal dynamic compatibility; Otherwise, the generated disaster dataset will not meet the criteria for spatiotemporal dynamic compatibility.
[0006] In one possible implementation, determining the real-time evolution confidence score of the disaster dataset based on the judgment result includes: When the judgment result indicates that the disaster dataset meets the spatiotemporal dynamic compatibility, a real-time evolution confidence score greater than the first threshold is generated; When the judgment result indicates that the disaster dataset does not conform to spatiotemporal dynamic compatibility, a real-time evolution confidence score less than a second threshold is generated, where the second threshold is less than the first threshold.
[0007] In one possible implementation, constructing a causal relationship model for each disaster variable in the disaster dataset includes: From the various disaster variables in the disaster dataset, filter the starting point and ending point of the dependent variable; For each set of dependent variable start and dependent variable end, the disaster type is determined based on the dependent variable start and dependent variable end. The large model is used to perform semantic reasoning on the disaster chain composed of the dependent variable start, dependent variable end and disaster type. The direction attribute of the causal edge between the dependent variable start and dependent variable end is determined based on the reasoning result. A causal relationship model is constructed based on the starting point, ending point, and directional attribute of the causal variable.
[0008] In one possible implementation, calculating the fusion value of the disaster variable based on the real-time evolution confidence scores corresponding to multiple data sources and the preset prior reliability scores includes: A causal-driven dynamic algorithm is used to assign dynamic fusion weights to the real-time evolution confidence score and prior reliability score of each data source; Calculate the weighted sum of the dynamic fusion weights of all data sources and the disaster variable, and divide the weighted sum by the sum of all dynamic fusion weights to obtain the fusion value of the disaster variable.
[0009] In one possible implementation, the causal-driven dynamic algorithm assigns dynamic fusion weights to the real-time evolutionary confidence score and prior reliability score of each data source, including: Obtain the causal-driven fusion coefficient; For each data source, the causal-driven fusion coefficient is multiplied by the real-time evolution confidence score corresponding to the data source to obtain the first value; Multiply the difference between 1 and the causal driving fusion coefficient by the prior reliability score corresponding to the data source to obtain the second value; The first value and the second value are added together to obtain the dynamic fusion weight.
[0010] According to a second aspect of this application, a disaster reporting device based on multi-source, multi-modal data is provided, the device comprising: The acquisition module is used to acquire multimodal disaster data sets provided by multiple data sources; The judgment module is used to construct a causal relationship model describing the physical constraints of disaster evolution for each disaster dataset corresponding to each data source, use the local cost matrix constructed based on the causal relationship model to judge whether the disaster dataset meets the spatiotemporal dynamic compatibility, and determine the real-time evolution confidence score of the disaster dataset based on the judgment result. The calculation module is used to assign dynamic fusion weights to the same disaster variable in multiple disaster datasets based on the real-time evolution confidence scores and preset prior reliability scores corresponding to multiple data sources, and to calculate the fusion value of the disaster variable based on the dynamic fusion weights. The generation module is used to generate disaster reports by using the fused values of various disaster variables from the large model.
[0011] According to a third aspect of this application, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement the disaster rapid reporting method for multi-source multimodal data as described above.
[0012] According to a fourth aspect of this application, a computer device is provided, the computer device including the above-mentioned disaster reporting device for multi-source multimodal data.
[0013] The beneficial effects of the technical solution provided in this application include at least the following: By constructing a causal relationship model among disaster variables and combining physical laws and spatiotemporal transmission constraints, a systematic compatibility test is conducted on multi-source data. This effectively eliminates noisy data with logical inconsistencies and temporal discrepancies, significantly improving the physical reliability and evolutionary consistency of disaster data. Through a causal-driven dynamic fusion mechanism, the fusion ratio of multi-source information is adaptively adjusted by comprehensively considering real-time evolution confidence and prior knowledge weights. This avoids misleading the overall situation due to biases from a single data source and enhances the system's response capability to sudden anomalies. Based on the fused high-confidence disaster variables, a large model is used to generate disaster reports with clear structure and logical coherence, improving report generation efficiency. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0015] Figure 1 This is a flowchart of a disaster reporting method using multi-source, multi-modal data provided in one embodiment of this application; Figure 2 This is a schematic diagram of a causal relationship model provided in one embodiment of this application; Figure 3 This is a flowchart of a disaster reporting method using multi-source, multi-modal data provided in one embodiment of this application; Figure 4 This is a structural block diagram of a disaster reporting device using multi-source, multi-modal data provided in one embodiment of this application. Detailed Implementation
[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.
[0017] like Figure 1 The diagram illustrates a flowchart of a disaster reporting method using multi-source, multi-modal data according to an embodiment of this application. This disaster reporting method using multi-source, multi-modal data can be applied to computer devices. The disaster reporting method using multi-source, multi-modal data may include: Step 101: Obtain multimodal disaster data sets from multiple data sources.
[0018] In this embodiment, multimodal data can be obtained from multiple data sources, and the data from each data source can be combined into a disaster data set. Data sources can be surveillance cameras, sensors, Weibo, telephones, WeChat groups, etc., and are not limited in this embodiment. Specifically, when the data source is a surveillance camera, the disaster data set is a video dataset; when the data source is a telephone, the disaster data set is an audio dataset; and when the data source is Weibo or WeChat groups, the disaster data set is a text dataset.
[0019] Specifically, a data aggregation platform can be used to preprocess and standardize the storage of text information, hotline voice data, and video data reported by departments at all levels, ensuring that data of each modality can be correctly parsed and represented. There are no explicit restrictions on the data collection and preprocessing methods during the data acquisition and preprocessing stage, as long as they can extract key information from text data, recognize content in video data, and convert speech to text.
[0020] Taking urban flooding incidents as an example, for text data, large language models are used to directly extract information such as the affected area, infrastructure damage, road damage, number of casualties, and rescue status from the text data; for video data, large video models are used to identify information such as the area of flooding and the extent of the disaster in the video data; for audio data, large audio models are used to identify the urgency and severity of the disaster in the audio data and extract key information such as trapped people and post-disaster needs.
[0021] Step 102: For each disaster dataset corresponding to each data source, construct a causal relationship model based on the physical constraints of disaster evolution for each disaster variable in the disaster dataset. Use the local cost matrix constructed based on the causal relationship model to determine whether the disaster dataset meets the spatiotemporal dynamic compatibility. Determine the real-time evolution confidence score of the disaster dataset based on the judgment result.
[0022] The causal relationship model includes nodes and edges. Each edge has two nodes: the starting point and the ending point of the dependent variable. The starting point is the initial triggering state of the dependent variable (an input node), and the ending point is the final state of the dependent variable (an output node). The edge represents the causal relationship between the starting and ending points of the dependent variable. In this embodiment, causal relationships are divided into positive and negative causal relationships. A positive causal relationship indicates that the change from the starting point to the ending point of the dependent variable conforms to a causal relationship, while a negative causal relationship indicates that the change from the starting point to the ending point does not conform to a causal relationship. Based on this characteristic, the edges in the causal relationship model can be called causal edges.
[0023] Taking urban flooding as an example, through expert knowledge, historical data analysis, and machine learning methods, the core variables affecting urban flooding are identified, including: inducing variables such as extreme rainfall, drainage capacity, low-lying terrain, pipe network blockage, and river water levels, which are the starting points of the dependent variables; direct consequence variables such as flooded area, flood depth, duration of flooding, traffic disruption, casualties, and facility damage; and secondary disaster variables such as power outages, secondary fires, and public health risks, which are the ending points of the dependent variables. Figure 2 As shown, based on the above variables, a directed causal graph is constructed, where nodes represent variables, directed edges represent causal directions, and edge weights represent the strength of causal influence and positive / negative relationships.
[0024] A node can serve as the starting point for the dependent variable, such as Figure 2 The "traffic control and diversion" in the context; or, a node can simply be the endpoint of the dependent variable, such as Figure 2 In the context of "public health risks"; or, a node can serve as both a starting point for a dependent variable, forming a combination with an adjacent ending point for a dependent variable and a causal edge, and an ending point for a dependent variable, forming a combination with an adjacent starting point for a dependent variable and another causal edge, such as... Figure 2 In this context, "extreme rainfall," as the endpoint of the dependent variable, forms a combination with "climate change," the starting point of the dependent variable, and one causal edge; conversely, "rainwater overflow," as the starting point of the dependent variable, forms a combination with "rainwater overflow," the endpoint of the dependent variable, and another causal edge. Traditional information fusion methods mostly focus on the consistency of data at the static or instantaneous level (such as numerical similarity and timestamp alignment), neglecting the inherent spatiotemporal dynamics of the evolution of disaster events as complex systems. The fused information may be numerically "averaged," but it may violate physical laws, causal logic, and spatiotemporal transmission constraints, resulting in reports or decisions generated based on it lacking practical guidance. However, high-quality disaster reports must be based on spatiotemporally and dynamically compatible multi-source information. To this end, this application innovatively uses a causal association model as a constraint framework for spatiotemporal dynamics, ensuring the spatiotemporal and dynamic compatibility of fused information through the following mechanisms: In the time dimension: a causal alignment algorithm based on physical constraints is proposed to examine whether the evolution of variable pairs in multi-source data conforms to the causal direction and reasonable time delay (time dynamics compatibility).
[0025] At the system level: the causal consistency score is used to assess the degree of conformity between each data source and the overall dynamic model, and this guides the dynamic fusion weights (system dynamic compatibility).
[0026] Final output: The self-consistent system state, after compatibility testing and optimization, is input into the large model to generate intelligent bulletins that conform to the logic of event development.
[0027] We first need to construct a local cost matrix based on the causal relationship model, then use the local cost matrix to determine whether the disaster dataset conforms to spatiotemporal dynamic compatibility, and finally determine the real-time evolution confidence score of the disaster dataset based on the judgment result. If the disaster dataset conforms to spatiotemporal dynamic compatibility, it means that the real-time evolution confidence score of the data source corresponding to the disaster dataset is high; if the disaster dataset does not conform to spatiotemporal dynamic compatibility, it means that the real-time evolution confidence score of the data source corresponding to the disaster dataset is low.
[0028] Step 103: For the same disaster variable in multiple disaster datasets, assign dynamic fusion weights based on the real-time evolution confidence scores of multiple data sources and the preset prior reliability scores of the data sources, and calculate the fusion value of the disaster variable based on the dynamic fusion weights.
[0029] The prior reliability score of a data source is an initial score based on the data source's observation accuracy, timeliness, and hardware link stability in historical similar disaster events. For example, the prior reliability score of human-operated hotline information is higher than that of Weibo data.
[0030] Specifically, the real-time evolution confidence score and the prior reliability score are weighted and calculated to obtain the fusion value of the disaster variables. This can take into account both information advantages and domain experience, reduce data bias, and improve the accuracy of variable data.
[0031] Step 104: Use the large model to generate a disaster report by fusing the values of various disaster variables.
[0032] Specifically, a general-purpose model can be used to intelligently generate content from the final fused values according to a fixed reporting template, thus obtaining a disaster alert.
[0033] In summary, the disaster reporting method based on multi-source and multi-modal data provided in this application constructs a causal relationship model among disaster variables and combines physical laws and spatiotemporal transmission constraints to conduct a systematic compatibility test on multi-source data. This effectively eliminates noisy data with logical inconsistencies and temporal discrepancies, significantly improving the physical reliability and evolutionary consistency of disaster data. Through a causal-driven dynamic fusion mechanism, the method comprehensively considers real-time evolution confidence and prior knowledge weights to adaptively adjust the fusion ratio of multi-source information. This avoids misleading the overall situation due to biases from a single data source and enhances the system's response capability to sudden abnormal information. Based on the fused high-confidence disaster variables, a large model is used to generate disaster reports with clear structure and logical coherence, improving report generation efficiency.
[0034] like Figure 3 The diagram illustrates a flowchart of a disaster reporting method using multi-source, multi-modal data according to an embodiment of this application. This method can be applied to computer devices. The disaster reporting method using multi-source, multi-modal data may include: Step 301: Obtain multimodal disaster data sets from multiple data sources.
[0035] In this embodiment, multimodal data can be obtained from multiple data sources, and the data from each data source can be combined into a disaster data set. Data sources can be surveillance cameras, sensors, Weibo, telephones, WeChat groups, etc., and are not limited in this embodiment. Specifically, when the data source is a surveillance camera, the disaster data set is a video dataset; when the data source is a telephone, the disaster data set is an audio dataset; and when the data source is Weibo or WeChat groups, the disaster data set is a text dataset.
[0036] Specifically, a data aggregation platform can be used to preprocess and standardize the storage of text information, hotline voice data, and video data reported by departments at all levels, ensuring that data of each modality can be correctly parsed and represented. There are no explicit restrictions on the data collection and preprocessing methods during the data acquisition and preprocessing stage, as long as they can extract key information from text data, recognize content in video data, and convert speech to text.
[0037] Taking urban flooding incidents as an example, for text data, large language models are used to directly extract information such as the affected area, infrastructure damage, road damage, number of casualties, and rescue status from the text data; for video data, large video models are used to identify information such as the area of flooding and the extent of the disaster in the video data; for audio data, large audio models are used to identify the urgency and severity of the disaster in the audio data and extract key information such as trapped people and post-disaster needs.
[0038] Step 302: For a disaster dataset corresponding to each data source, construct a causal relationship model describing the physical constraints of disaster evolution for each disaster variable in the disaster dataset. Use the local cost matrix constructed based on the causal relationship model to determine whether the disaster dataset meets the spatiotemporal dynamic compatibility. Determine the real-time evolution confidence score of the disaster dataset based on the judgment result.
[0039] (1) Constructing a causal relationship model Specifically, constructing a causal relationship model describing the physical constraints of disaster evolution for each disaster variable in the disaster dataset can include: selecting the starting point and ending point of the dependent variable from each disaster variable in the disaster dataset; for each set of dependent variable starting and ending points, determining the disaster type based on the starting and ending points; using a large model to perform semantic reasoning on the disaster chain composed of the dependent variable starting point, the dependent variable ending point, and the disaster type; determining the direction attribute of the causal edge between the dependent variable starting point and the dependent variable ending point based on the reasoning results, where positive causal relationships and negative causal relationships correspond to different direction attributes; and constructing a causal relationship model based on the causal variable starting point, causal variable ending point, and direction attribute.
[0040] Each data source's disaster dataset V = {v1, v2, ..., v...} n}, v i This represents a disaster-related variable. For example, in a flood scenario, v1 represents extreme rainfall, v2 represents drainage capacity, and v3 represents the flooded area, etc.
[0041] The set of causal edges E={e ij}, where e ij =(v i v j s ij ), v i Indicates the starting point of the dependent variable, v j Indicates the endpoint of the dependent variable, s ij The direction attribute is represented by +1 for positive causal relationships and -1 for negative causal relationships, and so on. Figure 2 The causal relationship model shown.
[0042] In this embodiment, a Large Language Model (LLM) can be used to perform semantic reasoning on the disaster chain to determine the causal edge e. ij Directional attribute s ij .
[0043] Specifically, a prompt function Prompt() can be defined, whose input is a disaster variable pair {v i v j} and disaster type Context disasterExamples include "urban flooding" and "forest fire"; the output is the directional attribute s. ij Its mathematical expression is: s ij =L LLM (Prompt(v i v j Context disaster )) (1) L LLM The logistic mapping function representing the large model transforms natural language reasoning into numerical constraints, if the dependent variable starts from v. i The increase in will inevitably lead to the endpoint v of the dependent variable in a logical and physical sense. j If s increases, then ij =+1; if the starting point of the dependent variable v i The increase in will inevitably lead to the endpoint v of the dependent variable in a logical and physical sense. j If the decrease is s, then ij =-1, and thus construct the physical logic constraint matrix S.
[0044] (2) Determine whether the disaster dataset conforms to spatiotemporal dynamic compatibility. Specifically, using a local cost matrix constructed based on a causal relationship model to determine whether a disaster dataset conforms to spatiotemporal dynamic compatibility can include: a) For each causal edge in the causal relationship model, obtain the direction attribute of the causal edge, where positive causal relationship and negative causal relationship correspond to different direction attributes.
[0045] For each causal edge e in the causal relationship model ij Obtain the directional attribute s obtained from large model inference. ij .
[0046] b) Obtain the first observation sequence of the dependent variable starting point corresponding to the causal edge within the preset time window.
[0047] Specifically, the first observation sequence of the dependent variable starting point vi in data source k within a preset time window Δt is extracted. .
[0048] c) Obtain the second observation sequence of the dependent variable endpoint corresponding to the causal edge within the time window.
[0049] Specifically, the second observation sequence of the dependent variable endpoint vj in data source k within a preset time window Δt is extracted. .
[0050] d) Construct a local cost matrix under physical constraints based on the first observation sequence, the second observation sequence, and the direction attribute. The elements in the local cost matrix represent the matching distance between a sampling point in the first observation sequence and another observation point in the second observation sequence after considering physical polarity.
[0051] Construct a local cost matrix C under physical constraints: C is an N*N matrix, where N represents the total number of sampling points within a preset time window Δt, and the elements c ab This represents the matching distance between the a-th sample point of sequence X and the b-th sample point of sequence Y, after considering physical polarity: (2) in, This is the modal reference adjustment constant, used to eliminate deviations caused by differences in the dimensions of different sensors (such as water level in centimeters versus rainfall in millimeters) or differences in initial values; The starting point v of the dependent variable in data source k i The observation value at the a-th time point within the current window; The starting point v of the dependent variable in data source k j The observation at the b-th time point within the current window.
[0052] When s ij When s = +1, the calculation result represents the positive evolutionary difference between the two variables; when s ij When =-1, the calculation result represents the physical overlap of the inverse evolution of two variables, thus allowing causal relationships of different polarities to be measured in the same cost space.
[0053] e) Calculate the cumulative minimum cost matrix for the local cost matrix using the dynamic programming algorithm, and calculate the minimum cumulative cost and average lag step for the minimum cost matrix. The average lag step represents the average physical delay between the start and end points of the dependent variable.
[0054] Using the dynamic programming algorithm, calculate the cumulative minimum cost matrix D based on the local cost matrix C: For any cell (a, b), its cumulative cost is defined as D(a, b): The initial datum is: D(1,1) = c 1,1 , Boundary recursion: For the first row or first column, the cumulative cost is the sum of the local cost and the cost of the preceding point. D(a,1)=D(a-1,1)+c a,1 Where a = 2, ..., N D(1,b)=D(1,b-1)+c 1,b Where b=2, ...,N For any non-boundary element (a, b) in the matrix, its cumulative cost D(a, b) is updated using the minimum cost principle: D(a, b) = c ab +min{ D(a-1,b),D(a,b-1),D(a-1,b-1)} (3) Extracting the optimal alignment path p*: By backtracking from D(N, N) to D(1, 1), a set of optimal alignment coordinate points p*={(a1, b1), (a2, b2), ..., (a... L b L )} Calculate the average lag step t: (4) Where L represents the total number of points in the optimal alignment path (N≤L≤2N-1), b l and a l This represents the index position of the l-th pair of matching points in the alignment path within the two observation sequences, and t represents the average physical delay from the start point to the end point of the dependent variable. If t > 0, it indicates that the end point v of the dependent variable... j Lagging behind the starting point v of the dependent variable i If t≈0, it means that the two occur almost simultaneously.
[0055] f) Determine whether the disaster dataset conforms to spatiotemporal dynamics compatibility based on the minimum cumulative cost and the average lag step.
[0056] Specifically, determining whether a disaster dataset conforms to spatiotemporal dynamics compatibility based on the minimum cumulative cost and average lag steps can include: determining the disaster type based on the starting and ending points of the dependent variable, obtaining a reasonable physical transmission time interval for the disaster type; determining whether the minimum cumulative cost is less than a preset fluctuation tolerance threshold and whether the average lag steps are within the physical transmission time interval; if so, generating a judgment result that the disaster dataset conforms to spatiotemporal dynamics compatibility; otherwise, generating a judgment result that the disaster dataset does not conform to spatiotemporal dynamics compatibility.
[0057] If the minimum cumulative cost D(N, N) < ( If the calculated t is within a reasonable physical propagation time range for the disaster type (assuming a preset fluctuation tolerance threshold), then the causal edge e is determined to be... ij This data source conforms to spatiotemporal dynamic compatibility, that is... =1; otherwise, it is determined to be inconsistent with spatiotemporal dynamics compatibility. =0.
[0058] (3) Generate real-time evolution confidence scores Specifically, determining the real-time evolution confidence score of the disaster dataset based on the judgment result can include: when the judgment result indicates that the disaster dataset meets the spatiotemporal dynamic compatibility, generating a real-time evolution confidence score greater than a first threshold; when the judgment result indicates that the disaster dataset does not meet the spatiotemporal dynamic compatibility, generating a real-time evolution confidence score less than a second threshold, wherein the second threshold is less than the first threshold.
[0059] Real-time evolution confidence score , among which, when At the time of its establishment, =1, otherwise =0, This represents the total number of causal edges. When... When k ≈ 1, it indicates that the data source k highly conforms to the causal relationship model, and a high real-time evolution confidence score can be obtained; when When the value is approximately 0, it indicates that the data source k is in serious conflict with the causal relationship model, and a low real-time evolution confidence score can be obtained.
[0060] In one example, a first threshold greater than 0.5 and a second threshold less than 0.5 can be set so that the real-time evolution confidence score greater than the first threshold is close to 1 and the real-time evolution confidence score less than the second threshold is close to 0.
[0061] Step 303: For the same disaster variable in multiple disaster datasets, a causal-driven dynamic algorithm is used to assign dynamic fusion weights to the real-time evolution confidence score and prior reliability score of each data source.
[0062] The fusion process of the same disaster variable is carried out in the unified disaster situation representation space. By extracting disaster feature vectors from multimodal data and performing spatiotemporal alignment, the consistency of text description, image recognition and sensor values in the physical dimension is ensured.
[0063] Specifically, calculating the fusion weight based on the real-time evolution confidence score and prior reliability score of each data source can include: obtaining the causal-driven fusion coefficient; for each data source, multiplying the causal-driven fusion coefficient by the real-time evolution confidence score corresponding to the data source to obtain a first value; multiplying the difference between 1 and the causal-driven fusion coefficient by the prior reliability score corresponding to the data source to obtain a second value; and adding the first value and the second value to obtain the dynamic fusion weight.
[0064] Dynamic fusion weights , where γ represents the causal-driven fusion coefficient, with a value range of [0, 1]. Considering the data source, its value range can be set to 0.5-0.8; This represents the prior reliability score of data source k, which can be set based on historical data. For example, the prior reliability score of manual hotline information should be higher than that of Weibo data.
[0065] Step 304: Calculate the weighted sum of the dynamic fusion weights of all data sources and the disaster variable, and divide the weighted sum by the sum of all dynamic fusion weights to obtain the fusion value of the disaster variable.
[0066] Fusion value ,in, Represents variable v i The final estimated value after fusion over time t, for example, the water depth is 40cm; This indicates that data source k corresponds to variable v. i For example, at time t, the video detected that "the water depth is 40cm" and the hotline described that "it is impassable". This indicates the fusion weight.
[0067] Taking a specific area and the same time period as an example, the variable of water depth is obtained from three data sources: video surveillance, hotline calls, and Weibo. The video surveillance identifies a water depth of 38cm, the hotline reports 35cm, and Weibo shows 25cm. Assuming the real-time evolution confidence scores of these three data sources are 0.95, 0.80, and 0.40, respectively, and the prior reliability scores are 0.9, 0.85, and 0.4, respectively, and γ is set to 0.7, then the final fusion result of the water depth is: .
[0068] Step 305: Use the large model to generate a disaster report by combining the values of various disaster variables.
[0069] Specifically, a general-purpose model can be used to intelligently generate content from the final fused values according to a fixed reporting template, thus obtaining a disaster alert.
[0070] In one implementation, the fused precise variable set V = {v1, v2, ..., v...} can be... n Input the large model, and then construct a prompt word structure based on causal chain: Prompt={Template, V, S}, where Template= Φ(S, R, V), R represents the preset quick report template, such as word limit, segmentation logic, etc.; the large model generates disaster quick reports based on the prompt word structure.
[0071] In summary, the disaster reporting method based on multi-source and multi-modal data provided in this application constructs a causal relationship model among disaster variables and combines physical laws and spatiotemporal transmission constraints to conduct a systematic compatibility test on multi-source data. This effectively eliminates noisy data with logical inconsistencies and temporal discrepancies, significantly improving the physical reliability and evolutionary consistency of disaster data. Through a causal-driven dynamic fusion mechanism, the method comprehensively considers real-time evolution confidence and prior knowledge weights to adaptively adjust the fusion ratio of multi-source information. This avoids misleading the overall situation due to biases from a single data source and enhances the system's response capability to sudden abnormal information. Based on the fused high-confidence disaster variables, a large model is used to generate disaster reports with clear structure and logical coherence, improving report generation efficiency.
[0072] like Figure 4 The diagram illustrates a structural block diagram of a disaster reporting device using multi-source, multi-modal data according to an embodiment of this application. This device can be applied to computer equipment and includes: The acquisition module 410 is used to acquire multimodal disaster data sets provided by multiple data sources; The judgment module 420 is used to construct a causal relationship model describing the physical constraints of disaster evolution for each disaster dataset corresponding to each data source, and use the local cost matrix constructed based on the causal relationship model to judge whether the disaster dataset meets the spatiotemporal dynamic compatibility, and determine the real-time evolution confidence score of the disaster dataset based on the judgment result. The calculation module 430 is used to assign dynamic fusion weights to the same disaster variable in multiple disaster datasets based on the real-time evolution confidence scores and preset prior reliability scores corresponding to multiple data sources, and to calculate the fusion value of the disaster variable based on the dynamic fusion weights. The generation module 440 is used to generate disaster reports by using the fusion values of various disaster variables from the large model.
[0073] In an optional embodiment, the determination module 420 is further configured to: For each causal edge in the causal relationship model, obtain the direction attribute of the causal edge, where positive causal relationship and negative causal relationship correspond to different direction attributes; Obtain the first observation sequence within a preset time window corresponding to the starting point of the dependent variable of the causal edge; Obtain the second observation sequence within the time window for the endpoint of the dependent variable corresponding to the causal edge; A local cost matrix under physical constraints is constructed based on the first observation sequence, the second observation sequence, and the direction attribute. The elements in the local cost matrix represent the matching distance between a sampling point in the first observation sequence and another observation point in the second observation sequence after considering physical polarity. The dynamic programming algorithm is used to calculate the cumulative minimum cost matrix of the local cost matrix, and the minimum cumulative cost and average lag steps are calculated from the minimum cost matrix. The average lag steps represent the average physical delay between the start and end points of the dependent variable. Determine whether the disaster dataset meets the spatiotemporal dynamics compatibility criteria based on the minimum cumulative cost and the average lag step.
[0074] In an optional embodiment, the determination module 420 is further configured to: The disaster type is determined based on the starting and ending points of the dependent variable, and a reasonable physical transmission time interval for the disaster type is obtained. Determine whether the minimum cumulative cost is less than the preset fluctuation tolerance threshold and whether the average lag step is within the physical transmission time interval; If so, then the disaster data set will be generated to determine whether it conforms to spatiotemporal dynamic compatibility. Otherwise, the generated disaster dataset will not meet the judgment result of spatiotemporal dynamic compatibility.
[0075] In an optional embodiment, the determination module 420 is further configured to: When the judgment result indicates that the disaster dataset meets the spatiotemporal dynamic compatibility, a real-time evolution confidence score greater than the first threshold is generated; When the judgment result indicates that the disaster dataset does not conform to spatiotemporal dynamic compatibility, a real-time evolution confidence score less than the second threshold is generated, and the second threshold is less than the first threshold.
[0076] In an optional embodiment, the determination module 420 is further configured to: Filter the starting point and ending point of the dependent variable from each disaster variable in the disaster dataset; For each set of dependent variable start and end points, the disaster type is determined based on the dependent variable start and end points. The large model is used to perform semantic reasoning on the disaster chain composed of the dependent variable start, the dependent variable end point and the disaster type. The direction attribute of the causal edge between the dependent variable start and the dependent variable end point is determined based on the reasoning results. A causal relationship model is constructed based on the starting point, ending point, and directional attributes of causal variables.
[0077] In an optional embodiment, the calculation module 430 is further configured to: A causal-driven dynamic algorithm is used to assign dynamic fusion weights to the real-time evolution confidence score and prior reliability score of each data source; Calculate the weighted sum of the dynamic fusion weights of all data sources and the disaster variable, and divide the weighted sum by the sum of all dynamic fusion weights to obtain the fusion value of the disaster variable.
[0078] In an optional embodiment, the calculation module 430 is further configured to: Obtain the causal-driven fusion coefficient; For each data source, the causal-driven fusion coefficient is multiplied by the real-time evolution confidence score corresponding to the data source to obtain the first value; Multiply the difference between 1 and the causal driving fusion coefficient by the prior reliability score corresponding to the data source to obtain the second value; The first and second values are added together to obtain the dynamic fusion weight.
[0079] In summary, the disaster reporting device based on multi-source, multi-modal data provided in this application constructs a causal relationship model among disaster variables and combines physical laws and spatiotemporal transmission constraints to conduct a systematic compatibility test on multi-source data. This effectively eliminates noisy data with logical inconsistencies and temporal discrepancies, significantly improving the physical reliability and evolutionary consistency of disaster data. Through a causal-driven dynamic fusion mechanism, it comprehensively considers real-time evolution confidence and prior knowledge weights to adaptively adjust the fusion ratio of multi-source information. This avoids misleading the overall situation due to biases from a single data source and enhances the system's response capability to sudden abnormal information. Based on the fused high-confidence disaster variables, it uses a large model to generate disaster reports with clear structure and logical coherence, improving report generation efficiency.
[0080] One embodiment of this application provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the disaster rapid reporting method for multi-source, multi-modal data as described above.
[0081] One embodiment of this application provides a computer device that includes the aforementioned disaster reporting device for arbitrary multi-source multimodal data.
[0082] It should be noted that the disaster reporting device for multi-source multimodal data provided in the above embodiments is only illustrated by the division of the above functional modules when performing disaster reporting of multi-source multimodal data. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the disaster reporting device for multi-source multimodal data can be divided into different functional modules to complete all or part of the functions described above. In addition, the disaster reporting device for multi-source multimodal data provided in the above embodiments and the disaster reporting method embodiments for multi-source multimodal data belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.
[0083] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0084] The above description is not intended to limit the embodiments of this application. Any adjustments, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of the embodiments of this application.
Claims
1. A method for rapid disaster reporting using multi-source, multi-modal data, characterized in that, The method includes: Obtain multimodal disaster data sets from multiple data sources; For each disaster dataset corresponding to each data source, a causal relationship model describing the physical constraints of disaster evolution is constructed for each disaster variable in the disaster dataset. The local cost matrix constructed based on the causal relationship model is used to determine whether the disaster dataset meets the spatiotemporal dynamic compatibility. Based on the determination result, the real-time evolution confidence score of the disaster dataset is determined. For the same disaster variable in multiple disaster datasets, dynamic fusion weights are assigned based on the real-time evolution confidence scores and preset prior reliability scores corresponding to multiple data sources, and the fusion value of the disaster variable is calculated based on the dynamic fusion weights. A disaster report is generated by merging the values of various disaster variables using a large model.
2. The disaster rapid reporting method using multi-source, multi-modal data according to claim 1, characterized in that, The step of using the local cost matrix constructed based on the causal relationship model to determine whether the disaster dataset conforms to spatiotemporal dynamic compatibility includes: For each causal edge in the causal relationship model, the direction attribute of the causal edge is obtained, wherein positive causal relationship and negative causal relationship correspond to different direction attributes; Obtain the first observation sequence within a preset time window for the starting point of the dependent variable corresponding to the causal edge; Obtain the second observation sequence of the dependent variable endpoint corresponding to the causal edge within the time window; A local cost matrix under physical constraints is constructed based on the first observation sequence, the second observation sequence, and the direction attribute. The elements in the local cost matrix represent the matching distance between a sampling point in the first observation sequence and another observation point in the second observation sequence after considering physical polarity. The local cost matrix is calculated using a dynamic programming algorithm to calculate the cumulative minimum cost matrix, and the minimum cumulative cost and average lag steps are calculated from the minimum cost matrix. The average lag steps represent the average physical delay at which the dependent variable begins and ends. The feasibility of the disaster dataset is determined based on the minimum cumulative cost and the average lag step.
3. The disaster rapid reporting method using multi-source, multi-modal data according to claim 2, characterized in that, The step of determining whether the disaster dataset conforms to spatiotemporal dynamics compatibility based on the minimum cumulative cost and the average lag step includes: The disaster type is determined based on the starting point and ending point of the dependent variable, and a reasonable physical transmission time interval for the disaster type is obtained. Determine whether the minimum cumulative cost is less than a preset fluctuation tolerance threshold, and whether the average lag step is within the physical transmission time interval; If so, then generate a judgment result showing that the disaster dataset conforms to spatiotemporal dynamic compatibility; Otherwise, the generated disaster dataset will not meet the criteria for spatiotemporal dynamic compatibility.
4. The disaster rapid reporting method using multi-source, multi-modal data according to claim 3, characterized in that, The step of determining the real-time evolution confidence score of the disaster dataset based on the judgment result includes: When the judgment result indicates that the disaster dataset meets the spatiotemporal dynamic compatibility, a real-time evolution confidence score greater than the first threshold is generated; When the judgment result indicates that the disaster dataset does not conform to spatiotemporal dynamic compatibility, a real-time evolution confidence score less than a second threshold is generated, where the second threshold is less than the first threshold.
5. The disaster rapid reporting method using multi-source, multi-modal data according to claim 1, characterized in that, The construction of a causal relationship model describing the physical constraints of disaster evolution for each disaster variable in the disaster dataset includes: From the various disaster variables in the disaster dataset, filter the starting point and ending point of the dependent variable; For each set of dependent variable start and dependent variable end, the disaster type is determined based on the dependent variable start and dependent variable end. The large model is used to perform semantic reasoning on the disaster chain composed of the dependent variable start, dependent variable end and disaster type. The direction attribute of the causal edge between the dependent variable start and dependent variable end is determined based on the reasoning result. A causal relationship model is constructed based on the starting point, ending point, and directional attribute of the causal variable.
6. The disaster rapid reporting method using multi-source, multi-modal data according to claim 1, characterized in that, The step of calculating the fusion value of the disaster variable based on the real-time evolution confidence scores corresponding to multiple data sources and the preset prior reliability scores includes: A causal-driven dynamic algorithm is used to assign dynamic fusion weights to the real-time evolution confidence score and prior reliability score of each data source; Calculate the weighted sum of the dynamic fusion weights of all data sources and the disaster variable, and divide the weighted sum by the sum of all dynamic fusion weights to obtain the fusion value of the disaster variable.
7. The disaster rapid reporting method based on multi-source, multi-modal data according to claim 6, characterized in that, The aforementioned causal-driven dynamic algorithm assigns dynamic fusion weights to the real-time evolutionary confidence score and prior reliability score of each data source, including: Obtain the causal-driven fusion coefficient; For each data source, the causal-driven fusion coefficient is multiplied by the real-time evolution confidence score corresponding to the data source to obtain the first value; Multiply the difference between 1 and the causal driving fusion coefficient by the prior reliability score corresponding to the data source to obtain the second value; The first value and the second value are added together to obtain the dynamic fusion weight.
8. A disaster reporting device using multi-source, multi-modal data, characterized in that, The device includes: The acquisition module is used to acquire multimodal disaster data sets provided by multiple data sources; The judgment module is used to construct a causal relationship model describing the physical constraints of disaster evolution for each disaster dataset corresponding to each data source, use the local cost matrix constructed based on the causal relationship model to judge whether the disaster dataset meets the spatiotemporal dynamic compatibility, and determine the real-time evolution confidence score of the disaster dataset based on the judgment result. The calculation module is used to assign dynamic fusion weights to the same disaster variable in multiple disaster datasets based on the real-time evolution confidence scores and preset prior reliability scores corresponding to multiple data sources, and to calculate the fusion value of the disaster variable based on the dynamic fusion weights. The generation module is used to generate disaster reports by using the fused values of various disaster variables from the large model.
9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, which is loaded and executed by a processor to implement the disaster reporting method for multi-source, multi-modal data as described in any one of claims 1 to 7.
10. A computer device, characterized in that, The computer device includes: the disaster reporting device for multi-source, multi-modal data as described in claim 8.