Disaster early warning method and device and electronic equipment

Abstract

The invention provides a disaster early warning method and device and electronic equipment, and the method comprises the steps: obtaining disaster monitoring data of a target monitoring point, carrying out the semantic extraction of the disaster monitoring data, and obtaining a text semantic vector; determining spatio-temporal topological characteristics according to the sensor data and the engineering topological graph; determining sensor waveform data corresponding to the sensor data, and performing cross-modal alignment processing on the sensor waveform data, the text semantic vector and the time-space topological features to obtain multi-modal features; inputting the multi-modal features into a variational threshold determination model, and determining a target alarm threshold interval by using the variational threshold determination model; the multi-modal features and the target alarm threshold interval are input into a disaster early warning model obtained through pre-training, processing is carried out through the disaster early warning model, a target early warning strategy is output, and the target early warning strategy comprises the predicted disaster occurrence time of the target monitoring point, the disaster type of the occurrence of the disaster and measures for avoiding the occurrence of the disaster. According to the invention, accurate early warning of disasters is realized.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular, to a disaster warning method, apparatus, and electronic device. Background Art

[0002] In the field of modern deep engineering safety monitoring, the spatio-temporal correlation modeling of multi-source heterogeneous data and the deduction of disaster chain propagation have always been the technical bottlenecks of the disaster warning system. Traditional warning systems mostly adopt single-modal data analysis and are difficult to meet the requirements of dynamic association and cross-modal collaboration in complex scenarios.

[0003] Geological text parsing relies on rule-based keyword matching and is difficult to resolve the context ambiguity of professional terms such as "rock burst tendency" and "seepage mutation area". Although the sensor network can collect time-series data such as stress and displacement, traditional spatio-temporal graph models (such as GCN and GAT) rely on static adjacency matrices and cannot represent the dynamic changes of neighborhood relationships caused by the redistribution of mining stress in real time, resulting in serious distortion of the disaster evolution path deduction. Summary of the Invention

[0004] In view of this, the purpose of the present disclosure is to propose a disaster warning method, apparatus, and electronic device to solve or partially solve the above problems.

[0005] Based on the above purpose, the first aspect of the present disclosure provides a disaster warning method, the method comprising: Obtaining disaster monitoring data of a target monitoring point, performing semantic extraction on the disaster monitoring data to obtain a text semantic vector; Obtaining sensor data and an engineering topology map of the target monitoring point, and determining spatio-temporal topology features according to the sensor data and the engineering topology map; Determining sensor waveform data corresponding to the sensor data, performing cross-modal alignment processing on the sensor waveform data, the text semantic vector, and the spatio-temporal topology features to obtain multi-modal features; Inputting the multi-modal features into a variational threshold determination model, and using the variational threshold determination model to determine a target alarm threshold interval; Inputting the multi-modal features and the target alarm threshold interval into a pre-trained disaster warning model, and outputting a target warning strategy through processing by the disaster warning model, where the target warning strategy includes the time when a disaster is expected to occur at the target monitoring point, the type of disaster that occurs, and measures to avoid the occurrence of the disaster.

[0006] Based on the same inventive concept, the second aspect of the present disclosure proposes a disaster warning apparatus, the apparatus comprising: A data acquisition module, configured to acquire disaster monitoring data of a target monitoring point, perform semantic extraction on the disaster monitoring data, and obtain a text semantic vector; A spatio-temporal topological feature determination module, configured to acquire sensor data and an engineering topological map of a target monitoring point, and determine spatio-temporal topological features according to the sensor data and the engineering topological map; A multi-modal feature determination module, configured to determine sensor waveform data corresponding to the sensor data, perform cross-modal alignment processing on the sensor waveform data, the text semantic vector, and the spatio-temporal topological features, and obtain multi-modal features; A target alarm threshold interval determination module, configured to input the multi-modal features into a variational threshold determination model, and use the variational threshold determination model to determine a target alarm threshold interval; A target early warning strategy determination module, configured to input the multi-modal features and the target alarm threshold interval into a pre-trained disaster early warning model, and output a target early warning strategy after being processed by the disaster early warning model, where the target early warning strategy includes the time when a disaster is expected to occur at the target monitoring point, the type of disaster that occurs, and measures to avoid the occurrence of the disaster.

[0007] Based on the same inventive concept, a third aspect of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, the disaster early warning method described above is implemented.

[0008] Based on the same inventive concept, a fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the disaster early warning method described above.

[0009] As can be seen from the above, the present disclosure proposes a disaster warning method, device and electronic device, which obtain disaster monitoring data of a target monitoring point, perform semantic extraction on the disaster monitoring data to obtain a text semantic vector, obtain sensor data and an engineering topology map of the target monitoring point, determine spatio-temporal topology features according to the sensor data and the engineering topology map, determine sensor waveform data corresponding to the sensor data, perform cross-modal alignment processing on the sensor waveform data, the text semantic vector and the spatio-temporal topology features to obtain multi-modal features. By performing cross-modal alignment on text data, sensor data and an engineering topology map, subsequent disaster warning can be performed based on the determined multi-modal features. When performing disaster warning, more modal data is considered, thus making the disaster warning more accurate. At the same time, due to cross-modal alignment and fusion, compared with directly using data of different modalities, the data volume is reduced and the time for disaster warning is shortened. Input the multi-modal features into a variational threshold determination model, use the variational threshold determination model to determine a target alarm threshold interval, input the multi-modal features and the target alarm threshold interval into a pre-trained disaster warning model, and after processing by the disaster warning model, output a target warning strategy, where the target warning strategy includes the time when a disaster is expected to occur at the target monitoring point, the type of disaster that occurs, and measures to avoid the occurrence of the disaster. Since the disaster warning model is a model that has been trained using a large amount of data, when the multi-modal features and the target alarm threshold interval are input into the disaster warning model, the output target warning strategy is more accurate, reducing the occurrence of disasters and improving the safety of production work for front-line engineering personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following will briefly introduce the drawings required for use in the embodiments or related technology descriptions. Obviously, the drawings described below are only embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0011] Figure 1 It is a flowchart of the disaster warning method according to an embodiment of the present disclosure; Figure 2 It is a structural block diagram of the disaster warning device according to an embodiment of the present disclosure; Figure 3 It is a schematic structural diagram of the electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] In order to make the objectives, technical solutions and advantages of the present disclosure clearer and more understandable, the following further elaborates on the present disclosure in detail with reference to specific embodiments and the accompanying drawings.

[0013] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the ordinary meanings understood by those of ordinary skill in the art to which the present disclosure pertains. The terms "first", "second" and similar terms used in the embodiments of the present disclosure do not denote any order, quantity or importance, but are only used to distinguish different components. Words such as "including" or "comprising" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. Words such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Words such as "upper", "lower", "left", "right" are only used to indicate relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0014] The following are the explanations of the terms related to the present disclosure: DeBERTa-v3: DeBERTa (Decoding-enhanced BERT with Disentangled Attention) is an improved BERT model proposed by Microsoft in 2021. It significantly improves the pre-training efficiency of the model and the performance of downstream tasks by introducing two new technologies and a new fine-tuning method. DeBERTa-v3 is the third version of DeBERTa.

[0015] CVAE: The conditional variational autoencoder (CVAE) model can generate the desired data by specifying its label when generating data.

[0016] Geohash: The GeoHash algorithm is a method for converting longitude and latitude into comparable strings. By different numbers of digits and encoding methods, the accuracy and range of the encoding can be controlled.

[0017] Time2Vec: Time2Vec is a method for vectorizing time information, aiming to capture the periodic and aperiodic patterns of time and can be applied to different models.

[0018] GRU: The gated recurrent unit (GRU) is a variant of the recurrent neural network (RNN), aiming to solve the problem of gradient disappearance in traditional RNNs when dealing with long sequences, and it has a more concise structure compared to the long short-term memory (LSTM).

[0019] KL divergence: Relative entropy, also known as Kullback-Leibler divergence or information divergence, is an asymmetric measure of the difference between two probability distributions.

[0020] In the field of modern deep engineering safety monitoring, the spatio-temporal correlation modeling of multi-source heterogeneous data and the deduction of disaster chain propagation have always been the technical bottlenecks of the disaster warning system. The traditional warning system mostly uses single-modal data analysis and is difficult to meet the requirements of dynamic correlation and cross-modal collaboration in complex scenarios.

[0021] Geological text parsing relies on rule-based keyword matching and is difficult to resolve the context ambiguity of professional terms such as "rock burst tendency" and "seepage mutation area". Although the sensor network can collect time-series data such as stress and displacement, traditional spatio-temporal graph models (such as GCN and GAT) rely on static adjacency matrices and cannot represent the dynamic changes of neighborhood relationships caused by the redistribution of mining-induced stress in real time, resulting in serious distortion of the disaster evolution path deduction. The setting of fixed alarm thresholds is difficult to adapt to the non-linear changes in the deep engineering environment. Traditional methods lack a data-driven dynamic optimization mechanism, and the problems of missed alarms and false alarms are prominent. Isolated warning strategies cannot capture the cross-regional correlation of disaster chain propagation. Existing reinforcement learning methods are difficult to achieve collaborative responses at multiple monitoring points due to sparse rewards and state fragmentation problems. The causal relationship between text descriptions, sensor waveforms, and atlas topologies is broken. Early fusion only simply stitches features and lacks a cross-modal contrast learning and dynamic weight allocation mechanism, resulting in insufficient evidence chain collaborative judgment ability. The cloud centralized computing architecture is difficult to meet the sub-second warning requirements. Traditional models have a large number of parameters and low inference efficiency, and cannot support lightweight deployment at the edge and collaborative deduction in the cloud.

[0022] Based on the above description, this embodiment proposes a disaster warning method, as Figure 1 shown, the method includes: Step 101, obtain the disaster monitoring data of the target monitoring point, perform semantic extraction on the disaster monitoring data, and obtain a text semantic vector; Step 102, obtain the sensor data and engineering topology map of the target monitoring point, and determine spatio-temporal topology features according to the sensor data and the engineering topology map; Step 103, determine the sensor waveform data corresponding to the sensor data, perform cross-modal alignment processing on the sensor waveform data, the text semantic vector, and the spatio-temporal topology features, and obtain multi-modal features; Step 104: Input the multi-modal features into the variational threshold determination model, and use the variational threshold determination model to determine the target alarm threshold range; Step 105: Input the multi-modal features and the target alarm threshold range into a pre-trained disaster warning model, and after being processed by the disaster warning model, output a target warning strategy, where the target warning strategy includes the time when a disaster is expected to occur at the target monitoring point, the type of disaster that occurs, and measures to avoid the occurrence of the disaster.

[0023] Specifically in implementation, obtain the disaster monitoring data of the target monitoring point, where the disaster monitoring data is deep engineering disaster monitoring data. Perform semantic extraction on the disaster monitoring data to obtain text semantic vectors.

[0024] In this embodiment, the specific extraction method for semantic extraction of disaster monitoring data is extraction by a trained text encoder. The specific training process of the text encoder is as follows: Integrate the mining log text, and construct a deep engineering disaster field corpus according to the mining log text. Among them, the deep engineering disaster field corpus includes geological reports, monitoring logs, expert-annotated texts, etc.

[0025] Obtain a deep disaster distribution map, and determine the coordinate descriptions in the deep disaster distribution map. Obtain an initial text encoder, and perform incremental pre-training on the initial text encoder using the coordinate descriptions in the deep disaster distribution map and the deep engineering disaster field corpus, and adopt an adversarial training strategy to enhance the semantic robustness of deep engineering professional terms, where the deep engineering professional terms are professional nouns involved in the deep engineering field. Exemplarily, the deep engineering professional terms are rock burst tendency, permeability mutation area, etc.

[0026] In this embodiment, before inputting the text data in the deep engineering disaster field corpus into the initial text encoder, first perform word segmentation on the text data in the deep engineering disaster field corpus, and screen key context fragments through a dynamic sparse attention mechanism to reduce redundant semantic interference.

[0027] In this embodiment, the architecture of the initial text encoder is a teacher-student model architecture. The teacher model is a 24-layer DeBERTa-v3 structure, and the student model is a compressed 12-layer lightweight DeBERTa-v3 structure. Through knowledge distillation technology, the parameter scale is first reduced by 50%, while retaining the domain semantic features. Apply the compressed 12-layer model structure to expert mining suggestions to generate text semantic vectors for subsequent alignment processing with other modalities to obtain multi-modal features, ensuring that the knowledge transfer accuracy loss is less than or equal to 0.8%.

[0028] In this embodiment, the knowledge distillation technology is the attention matrix alignment technology. Through the attention matrix alignment technology, that is, the mean squared error loss function is used to achieve knowledge transfer, and at the same time, the parameters of the first six layers of the student model are frozen to retain the basic semantic features.

[0029] In response to the mean squared error loss function converging to a preset convergence threshold, it is determined that the training of the initial text encoder is completed, and a lightweight text encoder is obtained.

[0030] Obtain the sensor data of the target monitoring point and the engineering topology map, and determine the spatio-temporal topology features according to the sensor data and the engineering topology map. Among them, the sensor data is sensor network data, which is the data of the monitoring point collected and sent by the sensor. The sensor is a sensor required for deep engineering monitoring, such as a stress sensor, a displacement sensor, etc. Further, the sensor data includes stress, displacement, etc.

[0031] Determine the sensor waveform data corresponding to the sensor data, and perform cross-modal alignment processing on the sensor waveform data, the text semantic vector, and the spatio-temporal topology features to obtain multi-modal features.

[0032] In this embodiment, performing cross-modal alignment processing on the sensor waveform data, the text semantic vector, and the spatio-temporal topology features means performing alignment processing on any two of the sensor waveform data, the text semantic vector, and the spatio-temporal topology features respectively to achieve the purpose of aligning the sensor waveform data, the text semantic vector, and the spatio-temporal topology features.

[0033] Input the multi-modal features into the variational threshold determination model, and use the variational threshold determination model to determine the target alarm threshold interval. Among them, the variational threshold determination model is a trained model, and the training process of the variational threshold determination model specifically includes: Obtain an initial variational threshold determination model and a training data set, where the training data set includes all the disaster data events in the preset deep disaster data report. Construct a CVAE input sequence according to the training data set, where the CVAE input sequence includes energy, three-dimensional coordinates, and timestamps.

[0034] Input the CVAE input sequence into the initial variational threshold determination model. The initial variational threshold determination model includes a CVAE encoder and a CVAE decoder. The CVAE encoder and the CVAE decoder are introduced as follows: The CVAE encoder uses a bidirectional GRU to extract temporal features, and the CVAE decoder generates a threshold interval , and set the initial according to the sensor range .

[0035] The CVAE encoder is represented by the formula:

[0036] where, is the CVAE input sequence, is the hidden state.

[0037] The CVAE decoder is represented by the formula:

[0038] where, is the latent variable, c is the conditional information, and y is the output alarm threshold interval.

[0039] In this embodiment, the input of the initial variational threshold determination model is the disaster data feature corresponding to the disaster data event, and the output is the alarm threshold interval.

[0040] Input the multi-modal feature and the target alarm threshold interval into a pre-trained disaster warning model, and after being processed by the disaster warning model, output a target warning strategy, where the target warning strategy includes the time when a disaster is expected to occur at the target monitoring point, the type of disaster that occurs, and the measures to avoid the occurrence of the disaster.

[0041] Through the above solution, obtain the disaster monitoring data of the target monitoring point, perform semantic extraction on the disaster monitoring data to obtain a text semantic vector, obtain the sensor data and engineering topology map of the target monitoring point, determine the spatio-temporal topology features according to the sensor data and the engineering topology map, determine the sensor waveform data corresponding to the sensor data, and perform cross-modal alignment processing on the sensor waveform data, the text semantic vector and the spatio-temporal topology features to obtain multi-modal features. By performing cross-modal alignment on text data, sensor data and engineering topology maps, subsequent disaster warnings can be made based on the determined multi-modal features. When making disaster warnings, more modal data is considered, thus making the disaster warnings more accurate. At the same time, due to cross-modal alignment and fusion, compared with directly using data of different modalities, the data volume is reduced and the time for disaster warnings is shortened. Input the multi-modal features into the variational threshold determination model, use the variational threshold determination model to determine the target alarm threshold interval, input the multi-modal features and the target alarm threshold interval into the pre-trained disaster warning model, and after being processed by the disaster warning model, output the target warning strategy, where the target warning strategy includes the time when a disaster is expected to occur at the target monitoring point, the type of disaster that occurs, and the measures to avoid the occurrence of the disaster. Since the disaster warning model is a model that has been trained using a large amount of data, when the multi-modal features and the target alarm threshold interval are input into the disaster warning model, the output target warning strategy is more accurate, reducing the occurrence of disasters and improving the safety of production work for front-line engineering personnel.

[0042] In some embodiments, after determining the target warning strategy, the target warning strategy and multi-modal features can be input into the edge computing unit, and after being processed by the edge computing unit, a cloud deduction result is output.

[0043] Specifically, construct a computing offloading decision tree and calculate the local inference latency. In response to the local inference latency exceeding the preset maximum latency, segment the spatio-temporal graph corresponding to the multi-modal features to obtain a plurality of node sub-graphs, and input the plurality of node sub-graphs into the cloud for parallel deduction.

[0044] In this embodiment, the preset maximum latency can be a preset latency or can be dynamically adjusted according to the hardware performance of the edge side. In this embodiment, the preset maximum latency is at least 0.8 seconds.

[0045] In this embodiment, when segmenting the spatio-temporal graph corresponding to the multi-modal features, segmentation processing can be performed according to a preset quantity. Exemplarily, the spatio-temporal graph can be segmented into 512 node sub-graphs. Due to the memory capacity limitation of the edge side, a greedy algorithm is used to select 512 nodes with the closest physical distance to construct the sub-graph.

[0046] In this embodiment, the edge device executes alarm decisions in real time (such as acoustic and optical warnings, emergency stops of equipment, etc.), and the cloud is responsible for global disaster chain deduction and strategy optimization.

[0047] In this embodiment, to further reduce the model inference loss, 8-bit fixed-point quantization technology is used to compress the parameters of each module. Combining with the weight sharing strategy, the overall model parameter quantity is controlled. A lightweight inference engine is deployed, which supports INT8 quantization inference and sub-second response at INT8 precision. After quantization, the model parameter quantity is compressed by 50%, and the model inference accuracy loss is only 0.8%.

[0048] In some embodiments, in step 102, the sensor data of the target monitoring point and the engineering topology map are obtained, and the spatio-temporal topology features are determined according to the sensor data and the engineering topology map, specifically including: Step 1021, obtain the location information of the target monitoring point, the sensor data of the target monitoring point and the engineering topology map, and determine the spatio-temporal fusion coding corresponding to the target monitoring point according to the location information and the sensor data; Step 1022, construct an initial adjacency matrix corresponding to the target monitoring point according to the sensor data and the engineering topology map; Step 1023, obtain a preset dynamic graph propagation operator, and use the dynamic graph propagation operator to perform update processing on the initial adjacency matrix to obtain a target adjacency matrix; Step 1024, combine the spatio-temporal fusion coding with the target adjacency matrix to obtain spatio-temporal topology features.

[0049] Specifically in implementation, obtain the location information of the target monitoring point, the sensor data of the target monitoring point and the engineering topology map, and determine the spatio-temporal fusion coding corresponding to the target monitoring point according to the location information and the sensor data.

[0050] Construct an initial adjacency matrix corresponding to the target monitoring point according to the sensor data and the engineering topology map, where the initial adjacency matrix includes nodes and edges, the nodes represent the monitoring points in the engineering topology map, and the weight value of the edge represents the physical connection strength formed between the monitoring points based on the actual engineering environment (geological structure association, stress / displacement correlation, engineering activity influence, etc.) In this embodiment, according to the deep disaster high-incidence areas in the preset deep disaster data report, the coverage range of the dynamic graph nodes is set, and the number of nodes is expanded to 512.

[0051] Obtain a preset dynamic graph propagation operator, use the dynamic graph propagation operator to perform update processing on the initial adjacency matrix, and update the edge weights in real time according to the timestamp embedding, i.e., Time2Vec coding, to obtain a target adjacency matrix.

[0052] In this embodiment, the update period is 500 ms, and by default, it is continuously updated according to the update period until the monitoring task terminates. The termination of the monitoring task means that the early warning is completed or the monitoring system is shut down.

[0053] In this embodiment, the preset dynamic graph propagation operator is specifically: Assume the adjacency matrix is , the number of nodes is N = 512, and the edge weight update rule is expressed by the formula:

[0054] Where, is the hidden state of node i at time t, is the hidden state of node j at time t, which is generated by encoding sensor data such as stress data and displacement data, is the Sigmoid activation function, is the trainable weight matrix, is the timestamp embedding function, and the timestamp embedding function adopts Time2Vec encoding. The timestamp embedding function is specifically expressed by the formula:

[0055] Where, is the learnable parameter used to capture the time periodicity feature.

[0056] Through the multi-head attention mechanism, the global spatio-temporal dependence relationship is captured. Then, according to the global spatio-temporal dependence relationship, the spatio-temporal fusion encoding is combined with the target adjacency matrix to obtain the spatio-temporal topological feature. Exemplarily, the global spatio-temporal dependence relationship is the mining stress propagation path and the cross-regional disaster chain effect.

[0057] In some embodiments, in step 1021, determining the spatio-temporal fusion encoding corresponding to the target monitoring point according to the position information and the sensor data specifically includes: Step 10211, perform mapping processing on the position information to obtain the geographical coordinate encoding; Step 10212, determine the acquisition time corresponding to the sensor data, and perform frequency domain feature extraction on the acquisition time to obtain the time series encoding; Step 10213, determine the preset feature dimension, and splice the geographical coordinate encoding and the time series encoding in the preset feature dimension to obtain the spatio-temporal fusion encoding corresponding to the target monitoring point.

[0058] In specific implementation, the location information of the obtained target monitoring point is subjected to mapping processing to obtain a geographical coordinate code. In this embodiment, the specific manner of the mapping processing is to use the Geohash algorithm for mapping. Specifically, the monitoring point coordinates are mapped into a 12-bit hash string by using the Geohash algorithm, with an error ≤ ±5m, meeting the accuracy requirements. And the Geohash string is mapped into a high-dimensional vector through an embedding layer, retaining the spatial topological features, to obtain the geographical coordinate code, where the high-dimensional vector is a 128-dimensional vector.

[0059] Determine the acquisition time corresponding to the sensor data, obtain the interpolation data in the sensor failure period in the preset deep disaster data processing report, perform embedding splicing on the interpolation data and the acquisition time, and use wavelet transform to extract frequency domain features to obtain a time series code. By using wavelet transform to extract frequency domain features, the periodic fluctuations in the sensor data can be identified, improving the robustness of the model to data loss.

[0060] Determine a preset feature dimension, and splice the geographical coordinate code and the time series code in the preset feature dimension to obtain a spatio-temporal fusion code corresponding to the target monitoring point.

[0061] In some embodiments, in step 103, cross-modal alignment processing is performed on the sensor waveform data, the text semantic vector, and the spatio-temporal topological features to obtain multi-modal features, which specifically includes: Step 1031, use a contrast loss function to perform alignment processing on the text semantic vector and the spatio-temporal topological features, use a cosine similarity loss function to perform alignment processing on the spatio-temporal topological features and the sensor waveform data, and use a triplet loss function to perform alignment processing on the text semantic vector and the sensor waveform data; Step 1032, respectively use the sensor waveform data, the text semantic vector, and the spatio-temporal topological features as target modal data. For each target modal data: determine the target entropy value of the target modal data, and determine the target weight value of the target modal data according to the target entropy value, where the target weight value is represented by the formula:

[0062] Among them, is the target weight value, m is the target modal data, is the target entropy value, and M is the number of target modal data; Step 1033, combine the sensor waveform data, the text semantic vector, and the spatio-temporal topological features according to the target weight value corresponding to each target modal data to obtain multi-modal features.

[0063] In specific implementation, a loss function is used to perform alignment processing on any two of the sensor waveform data, the text semantic vector, and the spatio-temporal topological features. Specifically, a contrastive loss function is used to align the text semantic vector and the spatio-temporal topological features, a cosine similarity loss function is used to align the spatio-temporal topological features and the sensor waveform data, and a triplet loss function is used to align the text semantic vector and the sensor waveform data.

[0064] For any one of the sensor waveform data, the text semantic vector, and the spatio-temporal topological features, it is used as the target modal data. Determine the target entropy value of the target modal data, and the determination method of the target entropy value is expressed by the formula:

[0065] where, is the probability distribution of the elements in the feature vector, is the element in the feature vector, and n is the feature dimension.

[0066] Determine the target weight value of the target modal data according to the target entropy value, where the target weight value is expressed by the formula:

[0067] where, is the target weight value, m is the target modal data, is the target entropy value, and M is the number of target modal data. In this embodiment, M = 3, which respectively represent the sensor waveform data, the text semantic vector, and the spatio-temporal topological features.

[0068] In this embodiment, the lower the target entropy value, the higher the confidence level, and thus the larger the target weight value. The target weight value represents the reliability of different target modal data, and the reliability is the confidence level. According to the target weight value, the contribution ratio of different target modal data in the multi-modal features can be dynamically adjusted.

[0069] Combine the sensor waveform data, the text semantic vector, and the spatio-temporal topological features based on the target weight value corresponding to each target modal data to obtain multi-modal features.

[0070] In some embodiments, step 1031 specifically includes: Compare and learn the text semantic vector and the spatio-temporal topological features in a timely manner to maximize the similarity of the first positive sample pair, where the first positive sample represents the text semantic vector and the spatio-temporal topological features of the same disaster event. The similarity of the first positive sample pair is calculated by cosine similarity and maximized through optimizing the contrast loss function. Exemplarily, align the text description of "accelerate the filling of 16P" with the topological features of the 16P stope in the dynamic graph, and the positioning error is reduced from ±15m to ±5m.

[0071] Specifically, determine the similarity value between the text semantic vector and the spatio-temporal topological features, where the similarity value is represented by the formula:

[0072] Where, is the similarity value, is the text semantic vector, is the spatio-temporal topological feature, represents the dot product of vectors, is the L2 norm between the text semantic vector and the spatio-temporal topological feature.

[0073] Determine the contrast loss function according to the similarity value until the contrast loss function converges to the first preset convergence threshold, and determine that the text semantic vector is aligned with the spatio-temporal topological feature. Where, the contrast loss function is represented by the formula:

[0074] Where, is the contrast loss function, is the first positive sample pair, and the first positive sample represents the text semantic vector and the spatio-temporal topological features of the same disaster event, is the first negative sample pair, is the temperature parameter, and 𝑁 is the number of the first negative sample pairs.

[0075] Map the spatio-temporal topological features and the sensor waveform data to a shared space through fully connected layers respectively, and adopt the cosine similarity loss function to maximize the similarity between the spatio-temporal topological features and the sensor waveform data of the same monitoring point, and minimize the similarity between the spatio-temporal topological features and the sensor waveform data of different monitoring points.

[0076] Specifically, project the spatio-temporal topological features and the sensor waveform data into the same preset space to obtain the target spatio-temporal topological vector and the target sensor waveform vector, where the target spatio-temporal topological vector and the target sensor waveform vector are represented by the formula:

[0077] Where, is the target spatio-temporal topological vector, is the preset first training weight matrix, is the space-time topological feature, is the target sensor waveform vector, is the preset second training weight matrix, is the sensor waveform data; A cosine similarity loss function is determined according to the target spatiotemporal topological vector and the target sensor waveform vector, until the cosine similarity loss function converges to a second preset convergence threshold, and the spatiotemporal topological feature and the sensor waveform data are aligned, wherein the cosine similarity loss function is expressed by the formula:

[0078] in, is the cosine similarity loss function.

[0079] In this embodiment, the fusion weight is dynamically adjusted according to the confidence of the sensor data (such as the signal noise level or the degree of missing). For example, if the waveform is abnormal due to interference from a scraper, that is, when the confidence of the sensor waveform data is less than 0.7, the weight of the spatiotemporal topological features is increased to 70%, that is, the topological features are prioritized. If the confidence of the sensor waveform data is greater than 0.9, it is fused according to the balanced weight, that is, 50% of the spatiotemporal topological features and 50% of the sensor waveform data are fused.

[0080] The text semantic vector and sensor waveform data are fused through the cross-modal attention mechanism, and then the triplet loss function is used to bring the second positive sample pair closer and push the second negative sample pair further away.

[0081] Specifically, the text semantic vector is used as a query vector, the sensor waveform data is used as a key-value pair, and the attention weight is used to determine the association strength between the text semantic vector and the sensor waveform data, wherein the association strength is expressed by the formula:

[0082]

[0083] in, is the association strength value; The association strength and the sensor waveform data are spliced ​​to obtain a fusion vector, wherein the fusion vector is expressed by the formula:

[0084] in, is the fusion vector; Determine a triplet loss function based on the fusion vector until the triplet loss function converges to a third preset convergence threshold, and determine that the text semantic vector and the sensor waveform data are aligned. The triplet loss function is expressed by the formula:

[0085] Wherein, is the triplet loss function, is the second positive sample, and the second positive sample represents the text semantic vector and the sensor waveform data of the same disaster event, is the second negative sample, is the distance between the fusion vector and the second positive sample pair, is the distance between the fusion vector and the second negative sample pair, is the preset margin parameter.

[0086] In some embodiments, in step 104, inputting the multimodal features into a variational threshold determination model, and using the variational threshold determination model to determine a target alarm threshold interval specifically includes: Step 1041, input the multimodal features into the variational threshold determination model, and use the variational threshold determination model to output an initial alarm threshold interval; Step 1042, determine the historical alarm threshold interval corresponding to the multimodal features, and determine the relative entropy value between the initial alarm threshold interval and the historical alarm threshold interval; Step 1043, in response to the relative entropy value being greater than a preset threshold, adjust the model parameters of the variational threshold determination model to obtain a new variational threshold determination model, and input the multimodal features into the new variational threshold determination model until the obtained relative entropy value is less than or equal to the preset threshold; Step 1044, in response to the relative entropy value being less than or equal to the preset threshold, use the initial alarm threshold interval as the target alarm threshold interval.

[0087] Specifically in implementation, obtain a pre-trained variational threshold determination model, input the multimodal features into the variational threshold determination model, and after being processed by the variational threshold determination model, output an initial alarm threshold interval.

[0088] Search the database based on the multimodal features, determine the historical alarm threshold interval corresponding to the multimodal features, calculate the relative entropy value between the initial alarm threshold interval and the historical alarm threshold interval, and the relative entropy value is the KL divergence. The calculation method of the relative entropy value is expressed by the formula:

[0089] Wherein, is the relative entropy value, is the initial alarm threshold interval, is the historical alarm threshold interval.

[0090] Compare the relative entropy value with a preset threshold. In response to the relative entropy value being greater than the preset threshold, adjust the model parameters of the variational threshold determination model to obtain a new variational threshold determination model. Input the multi-modal features into the new variational threshold determination model until the obtained relative entropy value is less than or equal to the preset threshold.

[0091] In this embodiment, the preset threshold can be obtained by statistically analyzing the KL divergence distribution of historical data in a sliding window, and taking the 95th percentile as the preset threshold.

[0092] In this embodiment, when the relative entropy value is greater than the preset threshold, start an active learning process, adjust the model parameters of the variational threshold determination model, collect unlabeled data from the edge device, and incrementally update the CVAE parameters after expert verification to achieve dynamic adjustment of the threshold boundary.

[0093] In some embodiments, the training process of the disaster warning model involved in step 105 specifically includes: Step 10A, obtain a training data set and an initial disaster warning model, where the training data set includes a plurality of training data, and each training data includes training multi-modal features, a training alarm threshold interval, and an actual warning strategy; Step 10B, respectively use each training data as a first target training data. For each first target training data: input the first target training data into the initial disaster warning model, process it through the initial disaster warning model, output a first training warning strategy, determine the target actual warning strategy corresponding to the first target training data, and determine the first accuracy rate corresponding to the initial disaster warning model according to the first training warning strategy and the target actual warning strategy; Step 10C, train the initial disaster warning model based on the first accuracy rate until the first accuracy rate is greater than a preset accuracy rate threshold, determine that the training of the initial disaster warning model is completed, and obtain a first disaster warning model; Step 10D, respectively select a preset number of training data from the training data set as target training data sets. For each target training data set: input all the training data in the target training data set into the first disaster warning model at the same time, process it through the first disaster warning model, output a set of training warning strategies, determine the set of actual warning strategies corresponding to the target training data set, and determine a target reward function according to the set of training warning strategies and the set of actual warning strategies; Step 10E: Train the first disaster warning model based on the target reward function until the target reward function converges to a preset reward convergence threshold, determine that the training of the first disaster warning model is completed, and obtain the disaster warning model.

[0094] During specific implementation, obtain a training data set, which contains multiple training data. Each training data includes training multi-modal features, a training alarm threshold range, and an actual warning strategy.

[0095] Obtain an initial disaster warning model and first perform the first-stage training process. The first-stage training is single-region training, that is, each training data is separately input into the initial disaster warning model for training.

[0096] Specifically, for each training data, use the training data as the first target training data, input the first target training data into the initial disaster warning model, and after being processed by the initial disaster warning model, output the first training warning strategy.

[0097] Determine the target actual warning strategy corresponding to the first target training data, and determine the first accuracy rate corresponding to the initial disaster warning model according to the first training warning strategy and the target actual warning strategy. The reward function for the first-stage training is 𝑅1 = log(1 + warning lead time).

[0098] When the first accuracy rate corresponding to the initial disaster warning model is greater than the preset accuracy threshold, and / or the number of training rounds is greater than the preset number, the first-stage training ends, the first disaster warning model is obtained, and the second-stage training is entered.

[0099] Exemplarily, the preset accuracy threshold is 90%, and the preset number is 1000 times.

[0100] The second-stage training is cross-region collaborative training, that is, a preset number of training data are separately selected from the training data set as the target training data set, and then the first disaster warning model is trained using the target training data set.

[0101] Specifically, for each target training data set: input all the training data in the target training data set into the first disaster warning model at the same time, and after being processed by the first disaster warning model, output a set of training warning strategies.

[0102] Determine the set of actual warning strategies corresponding to the target training data set, and determine the target reward function according to the set of training warning strategies and the set of actual warning strategies. The target reward function for the second-stage training is 𝑅2 = ∑(risk decay rate × collaboration coefficient).

[0103] Train the first disaster warning model based on the target reward function until the target reward function converges to a preset reward convergence threshold, determine that the training of the first disaster warning model is completed, and obtain the disaster warning model.

[0104] In this embodiment, a virtual target generation strategy is adopted to reconstruct failed experiences into successful trajectories, solving the sparse reward problem. That is, reconstructing missed alarm events into successful trajectories to solve the sparse reward problem and improve the policy generalization ability. The Actor network of each agent outputs actions (such as starting emergency support, evacuation instructions), and the Critic network evaluates the cross-region cooperation utility.

[0105] Specifically, after obtaining the disaster warning model, the disaster warning model can be further updated according to the failed experiences. Specifically, adjust the target reward function for the training in the second stage, specifically: In response to the difference between the training warning strategy set and the actual warning strategy set, determine the actual sensor data corresponding to the disaster occurrence moment when the training warning strategy set is executed. Determine new training multimodal features according to the actual sensor data, input the new training multimodal features into the variational threshold determination model, and use the variational threshold determination model to determine a new training alarm threshold interval.

[0106] Input the new training multimodal features and the new training alarm threshold interval into the disaster warning model to obtain a new training warning strategy set. Determine a new target reward function according to the new training warning strategy set and the actual warning strategy set until the new target reward function converges to a preset reward convergence threshold to obtain a new disaster warning model.

[0107] Exemplarily, if the training warning strategy set is to trigger an alarm when the displacement ≥ 0.4 mm / s, and the determined actual sensor data is a successful alarm when the displacement reaches 0.5 mm / s. Then determine a new target reward function, and the new target reward function is 𝑅new = log(1 + the early warning lead time of the new target).

[0108] If, based on the actual sensor data, the actions of the agent meet the conditions (such as an early warning 10 seconds in advance), a positive reward is given, otherwise the penalty is maintained.

[0109] For the following same set of deep disaster data, the disaster warning method proposed in this disclosure and the traditional GCN + fixed threshold warning system are respectively used for disaster warning, and comparisons are made from four indicators: disaster detection accuracy, positioning accuracy (average error), warning delay, and threshold adaptability.

[0110] Using the traditional GCN + fixed threshold warning system, the disaster detection accuracy rate has 3 missed alarms, the positioning accuracy depends on the static map, specifically ±15m, the warning delay is 3 - 5 seconds, and due to the use of a fixed threshold, there are 2 false alarms in area 2 in January.

[0111] Using the disaster warning method proposed in this disclosure, there are 0 missed alarms, and the disaster detection accuracy rate is 100%. The positioning accuracy is ±5m. The warning delay is 0.8 seconds, and the target alarm threshold interval is dynamically adjusted, with 0 false alarms.

[0112] In summary, using the disaster warning method proposed in this disclosure, the positioning error is reduced by 66.7%, and the missed alarm / false alarm rate is zero; the response speed is increased by 75%, and edge computing supports sub - second decision - making. The dynamic threshold and the graph model adapt to the mining stress change, and the false alarm rate is reduced by 100%. The number of model parameters is compressed by 50%, which supports efficient deployment at the edge, meets the safety requirements of deep engineering, increases the possibility of safe production for front - line engineering staff, promotes the realization of engineering safety and intelligence, ensures the intrinsic safety of production, improves production efficiency, and promotes the intelligent transformation of engineering to achieve high - quality development.

[0113] It should be noted that the method of the embodiments of this disclosure can be executed by a single device, such as a computer or a server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In such a distributed scenario, one of the multiple devices can only execute one or more steps of the method of the embodiments of this disclosure, and these multiple devices will interact with each other to complete the described method.

[0114] It should be noted that some embodiments of this disclosure have been described above. Additionally, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0115] Based on the same inventive concept, corresponding to the method of any of the above embodiments, this disclosure also provides a disaster warning device.

[0116] Refer to Figure 2 , Figure 2 For the disaster warning device of the embodiment, it specifically includes: A data acquisition module 201, configured to acquire disaster monitoring data of a target monitoring point, perform semantic extraction on the disaster monitoring data, and obtain a text semantic vector; A spatio - temporal topology feature determination module 202, configured to acquire sensor data of a target monitoring point and an engineering topology map, and determine spatio - temporal topology features according to the sensor data and the engineering topology map; The multimodal feature determination module 203 is configured to determine the sensor waveform data corresponding to the sensor data, perform cross-modal alignment processing on the sensor waveform data, the text semantic vector, and the spatio-temporal topological feature to obtain multimodal features; The target alarm threshold interval determination module 204 is configured to input the multimodal features into a variational threshold determination model, and use the variational threshold determination model to determine the target alarm threshold interval; The target early warning strategy determination module 205 is configured to input the multimodal features and the target alarm threshold interval into a pre-trained disaster early warning model, and output a target early warning strategy through processing by the disaster early warning model, where the target early warning strategy includes the time when a disaster is expected to occur at the target monitoring point, the type of disaster that occurs, and measures to avoid the occurrence of the disaster.

[0117] In some embodiments, the spatio-temporal topological feature determination module 202 is specifically configured to: Obtain the location information of the target monitoring point, the sensor data of the target monitoring point, and the engineering topology map, and determine the spatio-temporal fusion encoding corresponding to the target monitoring point according to the location information and the sensor data; Construct an initial adjacency matrix corresponding to the target monitoring point according to the sensor data and the engineering topology map; Obtain a preset dynamic graph propagation operator, and use the dynamic graph propagation operator to update the initial adjacency matrix to obtain a target adjacency matrix; Combine the spatio-temporal fusion encoding with the target adjacency matrix to obtain a spatio-temporal topological feature.

[0118] In some embodiments, the spatio-temporal topological feature determination module 202 is specifically further configured to: Perform mapping processing on the location information to obtain a geographic coordinate encoding; Determine the acquisition time corresponding to the sensor data, and perform frequency domain feature extraction on the acquisition time to obtain a time series encoding; Determine a preset feature dimension, and splice the geographic coordinate encoding and the time series encoding in the preset feature dimension to obtain the spatio-temporal fusion encoding corresponding to the target monitoring point.

[0119] In some embodiments, the multimodal feature determination module 203 is specifically configured to: Use a contrast loss function to align the text semantic vector with the spatio-temporal topological feature, use a cosine similarity loss function to align the spatio-temporal topological feature and the sensor waveform data, and use a triplet loss function to align the text semantic vector and the sensor waveform data; Respectively take the sensor waveform data, the text semantic vector, and the spatio-temporal topological features as the target modal data. For each target modal data: determine the target entropy value of the target modal data, and determine the target weight value of the target modal data according to the target entropy value, where the target weight value is expressed by the formula:

[0120] where, is the target weight value, m is the target modal data, is the target entropy value, and M is the number of target modal data; Combine the sensor waveform data, the text semantic vector, and the spatio-temporal topological features according to the target weight value corresponding to each target modal data to obtain multi-modal features.

[0121] In some embodiments, the multi-modal feature determination module 203 is further specifically configured to: Determine the similarity value between the text semantic vector and the spatio-temporal topological features, where the similarity value is expressed by the formula:

[0122] where, is the similarity value, is the text semantic vector, is the spatio-temporal topological feature, represents the vector dot product, is the L2 norm between the text semantic vector and the spatio-temporal topological feature; Determine the contrast loss function according to the similarity value until the contrast loss function converges to a first preset convergence threshold, and determine that the text semantic vector is aligned with the spatio-temporal topological feature. Among them, the contrast loss function is expressed by the formula:

[0123] where, is the contrast loss function, is the first positive sample pair, and the first positive sample represents the text semantic vector and the spatio-temporal topological feature of the same disaster event, is the first negative sample pair, is the temperature parameter, and 𝑁 is the number of first negative sample pairs; Project the spatio-temporal topological feature and the sensor waveform data into the same preset space to obtain the target spatio-temporal topological vector and the target sensor waveform vector, where the target spatio-temporal topological vector and the target sensor waveform vector are expressed by the formula:

[0124] where, is the target spatio-temporal topological vector, is the preset first training weight matrix, is the spatio-temporal topological feature, is the target sensor waveform vector, is the preset second training weight matrix, is the sensor waveform data; Determine the cosine similarity loss function according to the target spatio-temporal topological vector and the target sensor waveform vector, until the cosine similarity loss function converges to a second preset convergence threshold, and determine that the spatio-temporal topological feature and the sensor waveform data are aligned. Among them, the cosine similarity loss function is expressed by the formula:

[0125] Among them, is the cosine similarity loss function; Use the text semantic vector as the query vector, use the sensor waveform data as the key-value pair, and determine the association strength between the text semantic vector and the sensor waveform data by using the attention weight. Among them, the association strength is expressed by the formula:

[0126]

[0127] Among them, is the association strength value; Perform splicing processing on the association strength and the sensor waveform data to obtain a fusion vector. Among them, the fusion vector is expressed by the formula:

[0128] Among them, is the fusion vector; Determine the triplet loss function according to the fusion vector, until the triplet loss function converges to a third preset convergence threshold, and determine that the text semantic vector and the sensor waveform data are aligned. The triplet loss function is expressed by the formula:

[0129] Among them, is the triplet loss function, is the second positive sample, and the second positive sample represents the text semantic vector and the sensor waveform data of the same disaster event, is the second negative sample, is the distance between the fusion vector and the second positive sample pair, is the distance between the fusion vector and the second negative sample pair, is a preset margin parameter.

[0130] In some embodiments, the target alarm threshold range determination module 204 is specifically configured to: Input the multi-modal features into a variational threshold determination model, and use the variational threshold determination model to output an initial alarm threshold range; Determine the historical alarm threshold range corresponding to the multi-modal features, and determine the relative entropy value between the initial alarm threshold range and the historical alarm threshold range; In response to the relative entropy value being greater than a preset threshold, adjust the model parameters of the variational threshold determination model to obtain a new variational threshold determination model, and input the multi-modal features into the new variational threshold determination model until the obtained relative entropy value is less than or equal to the preset threshold; In response to the relative entropy value being less than or equal to the preset threshold, use the initial alarm threshold range as the target alarm threshold range.

[0131] In some embodiments, the device further includes a training module, and the training module is specifically configured to: Obtain a training data set and an initial disaster warning model, where the training data set includes a plurality of training data, and each training data includes training multi-modal features, a training alarm threshold range, and an actual warning strategy; Respectively take each training data as a first target training data. For each first target training data: input the first target training data into the initial disaster warning model, process it through the initial disaster warning model, output a first training warning strategy, determine the target actual warning strategy corresponding to the first target training data, and determine the first accuracy rate corresponding to the initial disaster warning model according to the first training warning strategy and the target actual warning strategy; Train the initial disaster warning model based on the first accuracy rate until the first accuracy rate is greater than a preset accuracy rate threshold, determine that the training of the initial disaster warning model is completed, and obtain a first disaster warning model; Respectively select a preset number of training data from the training data set as target training data sets. For each target training data set: input all the training data in the target training data set into the first disaster warning model at the same time, process it through the first disaster warning model, output a set of training warning strategies, determine the set of actual warning strategies corresponding to the target training data set, and determine a target reward function according to the set of training warning strategies and the set of actual warning strategies; Train the first disaster warning model based on the target reward function until the target reward function converges to a preset reward convergence threshold, determine that the training of the first disaster warning model is completed, and obtain a disaster warning model.

[0132] In some embodiments, the training module is further specifically configured to: In response to the difference between the training warning policy set and the actual warning policy set, when determining to execute the training warning policy set, determine the actual sensor data corresponding to the disaster occurrence moment; Determine new training multimodal features according to the actual sensor data, input the new training multimodal features into the variational threshold determination model, and use the variational threshold determination model to determine a new training alarm threshold interval; Input the new training multimodal features and the new training alarm threshold interval into the disaster warning model to obtain a new training warning policy set; Determine a new target reward function according to the new training warning policy set and the actual warning policy set, and until the new target reward function converges to a preset reward convergence threshold, obtain a new disaster warning model.

[0133] For the convenience of description, when describing the above device, various modules are described separately according to their functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0134] The device in the above embodiment is used to implement the corresponding disaster warning method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0135] Based on the same inventive concept, corresponding to the method in any of the above embodiments, the present disclosure further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the disaster warning method described in any of the above embodiments.

[0136] Figure 3 FIG. shows a more specific schematic diagram of the hardware structure of the electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. Among them, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other inside the device through the bus 1050.

[0137] The processor 1010 may be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0138] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020 and called and executed by the processor 1010.

[0139] The input / output interface 1030 is used to connect to the input / output module to implement information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Among them, the input devices can include keyboards, mice, touchscreens, microphones, various sensors, etc., and the output devices can include displays, speakers, vibrators, indicator lights, etc.

[0140] The communication interface 1040 is used to connect to a communication module (not shown in the figure) to implement communication interaction between this device and other devices. Among them, the communication module can implement communication through wired means (such as USB, network cable, etc.) or through wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0141] The bus 1050 includes a path for transmitting information between various components of the device (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0142] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device may also only include the components necessary to implement the solutions of the embodiments of this specification, and do not have to include all the components shown in the figure.

[0143] The electronic device in the above embodiment is used to implement the corresponding disaster warning method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0144] Based on the same inventive concept, corresponding to the method in any of the above embodiments, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the disaster warning method as described in any of the foregoing embodiments.

[0145] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0146] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the disaster warning method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0147] It can be understood that before using the technical solutions of the various embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0148] For example, in response to receiving an active request from the user, a prompt message is sent to the user to clearly prompt the user that the operation requested by them will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application program, server, or storage medium that performs the operations of the technical solutions of the present disclosure based on the prompt message.

[0149] As an optional but non-limiting implementation manner, the way of sending a prompt message to the user in response to receiving an active request from the user can be, for example, in the form of a pop-up window, and the prompt message can be presented in text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0150] It can be understood that the above process of notifying and obtaining the user's authorization is only illustrative and does not constitute a limitation on the implementation manner of the present disclosure. Other ways that comply with relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

Claims

1. A disaster warning method, characterized in that, Including: Obtain disaster monitoring data of a target monitoring point, perform semantic extraction on the disaster monitoring data to obtain a text semantic vector; Obtain sensor data and an engineering topology map of the target monitoring point, and determine spatio-temporal topology features according to the sensor data and the engineering topology map; Determine sensor waveform data corresponding to the sensor data, and perform cross-modal alignment processing on the sensor waveform data, the text semantic vector, and the spatio-temporal topology features to obtain multi-modal features; Input the multi-modal features into a variational threshold determination model, and use the variational threshold determination model to determine a target alarm threshold interval; Input the multi-modal features and the target alarm threshold interval into a pre-trained disaster warning model, and through processing by the disaster warning model, output a target warning strategy, where the target warning strategy includes the time when a disaster is expected to occur at the target monitoring point, the type of disaster that occurs, and measures to avoid the occurrence of the disaster.

2. The method according to claim 1, wherein The obtaining of sensor data and an engineering topology map of the target monitoring point, and determining spatio-temporal topology features according to the sensor data and the engineering topology map includes: Obtain the position information of the target monitoring point, as well as the sensor data and the engineering topology map of the target monitoring point, and determine a spatio-temporal fusion encoding corresponding to the target monitoring point according to the position information and the sensor data; Construct an initial adjacency matrix corresponding to the target monitoring point according to the sensor data and the engineering topology map; Obtain a preset dynamic graph propagation operator, and use the dynamic graph propagation operator to update the initial adjacency matrix to obtain a target adjacency matrix; Combine the spatio-temporal fusion encoding with the target adjacency matrix to obtain spatio-temporal topology features.

3. The method according to claim 2, wherein The determining of the spatio-temporal fusion encoding corresponding to the target monitoring point according to the position information and the sensor data includes: Perform a mapping process on the position information to obtain a geographic coordinate encoding; Determine the acquisition time corresponding to the sensor data, and perform frequency domain feature extraction on the acquisition time to obtain a time series encoding; Determine a preset feature dimension, and splice the geographic coordinate encoding and the time series encoding in the preset feature dimension to obtain a spatio-temporal fusion encoding corresponding to the target monitoring point.

4. The method according to claim 1, wherein The performing of cross-modal alignment processing on the sensor waveform data, the text semantic vector, and the spatio-temporal topology features to obtain multi-modal features includes: Use a contrast loss function to perform alignment processing on the text semantic vector and the spatio-temporal topology features, use a cosine similarity loss function to perform alignment processing on the spatio-temporal topology features and the sensor waveform data, and use a triplet loss function to perform alignment processing on the text semantic vector and the sensor waveform data; Respectively use the sensor waveform data, the text semantic vector, and the spatio-temporal topology features as target modal data. For each target modal data: determine the target entropy value of the target modal data, and determine the target weight value of the target modal data according to the target entropy value, where the target weight value is represented by the formula: Among them, is the target weight value, m is the target modal data, is the target entropy value, and M is the number of target modal data; Combining the sensor waveform data, the text semantic vector, and the spatio-temporal topological features according to the target weight value corresponding to each target modality data to obtain multi-modal features.

5. The method according to claim 4, characterized in that, The alignment process of the text semantic vector and the spatio-temporal topological features using the contrast loss function, the alignment process of the spatio-temporal topological features and the sensor waveform data using the cosine similarity loss function, and the alignment process of the text semantic vector and the sensor waveform data using the triplet loss function includes: Determining the similarity value between the text semantic vector and the spatio-temporal topological features, where the similarity value is represented by a formula as: wherein, is the similarity value, is the text semantic vector, is the spatio-temporal topological feature, represents the vector dot product, is the L2 norm between the text semantic vector and the spatio-temporal topological feature; Determining the contrast loss function according to the similarity value until the contrast loss function converges to a first preset convergence threshold, and determining that the text semantic vector is aligned with the spatio-temporal topological features. Among them, the contrast loss function is represented by a formula as: Among them, is the contrastive loss function, is the first positive sample pair, and the first positive sample represents the text semantic vector and spatio-temporal topological features of the same disaster event, is the first negative sample pair, is the temperature parameter, and 𝑁 is the number of the first negative sample pairs; Projecting the spatio-temporal topological features and the sensor waveform data into the same preset space to obtain a target spatio-temporal topological vector and a target sensor waveform vector, where the target spatio-temporal topological vector and the target sensor waveform vector are represented by formulas as: Among them, is the target spatio-temporal topological vector, is the preset first training weight matrix, is the spatio-temporal topological feature, is the target sensor waveform vector, is the preset second training weight matrix, is the sensor waveform data; Determining the cosine similarity loss function according to the target spatio-temporal topological vector and the target sensor waveform vector until the cosine similarity loss function converges to a second preset convergence threshold, and determining that the spatio-temporal topological features and the sensor waveform data are aligned. Among them, the cosine similarity loss function is represented by a formula as: Among them, is the cosine similarity loss function; Using the text semantic vector as the query vector and the sensor waveform data as the key-value pair, and determining the association strength between the text semantic vector and the sensor waveform data using the attention weight. Among them, the association strength is represented by a formula as: Among them, is the association strength value; Performing splicing processing on the association strength and the sensor waveform data to obtain a fusion vector, where the fusion vector is represented by a formula as: Among them, is the fusion vector; Determining the triplet loss function according to the fusion vector until the triplet loss function converges to a third preset convergence threshold, and determining that the text semantic vector and the sensor waveform data are aligned. The triplet loss function is represented by a formula as: Among them, is the triplet loss function, is the second positive sample, and the second positive sample represents the text semantic vector and sensor waveform data of the same disaster event, is the second negative sample, is the distance between the fusion vector and the second positive sample pair, is the distance between the fusion vector and the second negative sample pair, is the preset margin parameter.

6. The method according to claim 1, characterized in that Inputting the multi-modal features into the variational threshold determination model, and using the variational threshold determination model to determine the target alarm threshold interval, including: Inputting the multi-modal features into the variational threshold determination model, and using the variational threshold determination model to output an initial alarm threshold interval; Determining the historical alarm threshold interval corresponding to the multi-modal features, and determining the relative entropy value between the initial alarm threshold interval and the historical alarm threshold interval; In response to the relative entropy value being greater than the preset threshold, adjusting the model parameters of the variational threshold determination model to obtain a new variational threshold determination model, and inputting the multi-modal features into the new variational threshold determination model until the obtained relative entropy value is less than or equal to the preset threshold; In response to the relative entropy value being less than or equal to the preset threshold, using the initial alarm threshold interval as the target alarm threshold interval.

7. The method according to claim 1, characterized in that The training process of the disaster warning model includes: Obtain a training dataset and an initial disaster warning model, where the training dataset includes multiple training data, and each training data includes training multi-modal features, a training alarm threshold range, and an actual warning strategy; Respectively take each training data as the first target training data. For each first target training data: input the first target training data into the initial disaster warning model, process it through the initial disaster warning model, output the first training warning strategy, determine the target actual warning strategy corresponding to the first target training data, and determine the first accuracy rate corresponding to the initial disaster warning model according to the first training warning strategy and the target actual warning strategy; Train the initial disaster warning model based on the first accuracy rate until the first accuracy rate is greater than the preset accuracy threshold, determine that the training of the initial disaster warning model is completed, and obtain the first disaster warning model; Respectively select a preset number of training data from the training dataset as the target training dataset. For each target training dataset: input all the training data in the target training dataset into the first disaster warning model at the same time, process it through the first disaster warning model, output a set of training warning strategies, determine the corresponding set of actual warning strategies for the target training dataset, and determine the target reward function according to the set of training warning strategies and the set of actual warning strategies; Train the first disaster warning model based on the target reward function until the target reward function converges to the preset reward convergence threshold, determine that the training of the first disaster warning model is completed, and obtain the disaster warning model.

8. The method according to claim 7, wherein After obtaining the disaster warning model, it further includes: In response to the difference between the set of training warning strategies and the set of actual warning strategies, determine the actual sensor data corresponding to the moment of disaster occurrence when the set of training warning strategies is executed; Determine new training multi-modal features according to the actual sensor data, input the new training multi-modal features into the variational threshold determination model, and use the variational threshold determination model to determine a new training alarm threshold range; Input the new training multi-modal features and the new training alarm threshold range into the disaster warning model to obtain a new set of training warning strategies; Determine a new target reward function according to the new set of training warning strategies and the set of actual warning strategies until the new target reward function converges to the preset reward convergence threshold to obtain a new disaster warning model.

9. A disaster warning device, characterized in that, It includes: A data acquisition module configured to acquire disaster monitoring data of a target monitoring point, perform semantic extraction on the disaster monitoring data, and obtain a text semantic vector; A spatio-temporal topology feature determination module configured to acquire sensor data and an engineering topology map of a target monitoring point, and determine spatio-temporal topology features according to the sensor data and the engineering topology map; A multi-modal feature determination module configured to determine sensor waveform data corresponding to the sensor data, perform cross-modal alignment processing on the sensor waveform data, the text semantic vector, and the spatio-temporal topology features to obtain multi-modal features; A target alarm threshold range determination module, configured to input the multi-modal features into a variational threshold determination model, and use the variational threshold determination model to determine a target alarm threshold range; A target early warning strategy determination module, configured to input the multi-modal features and the target alarm threshold range into a pre-trained disaster early warning model, and output a target early warning strategy after being processed by the disaster early warning model, wherein the target early warning strategy includes the time when a disaster is expected to occur at a target monitoring point, the type of disaster that occurs, and measures to avoid the occurrence of the disaster.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in any one of claims 1 to 8 is implemented.

Images