Mountain torrent multimode disaster-causing factor dangerous case research and judgment method, equipment and medium

Abstract

The invention discloses a mountain torrent multimode disaster-causing factor dangerous case research and judgment method and device and a medium, and relates to the technical field of mountain torrent disaster dangerous case prediction, and the method comprises the steps: obtaining a fusion feature matrix of all monitoring points of a to-be-detected region through employing a feature extraction network and multimode monitoring data of a plurality of monitoring points of the to-be-detected region; constructing a drainage basin graph based on the topological graph of the monitoring points of the to-be-monitored area and the fusion feature matrix of each monitoring point; and inputting the drainage basin graph into the space-time convolutional network to obtain a mountain torrent disaster dangerous case research and judgment result. According to the method, the mountain torrent multimode disaster-causing factor dangerous case research and judgment precision is improved by constructing the feature extraction network and the space-time convolution network.

Description

Technical Field

[0001] The present application relates to the technical field of flash flood disaster risk prediction, and in particular to a flash flood multi-mode disaster-causing factor risk assessment method, equipment, and medium. Background Art

[0002] Against the backdrop of intensifying global climate change and frequent extreme weather events, flash floods in small mountainous watersheds are characterized by sudden onset and complex causal chains. Traditional flash flood warning systems generally suffer from three major technical bottlenecks: First, single-element monitoring (such as water level) struggles to capture the precursors of disasters coupled with multiple physical fields; second, there is a modeling gap in the spatiotemporal alignment and feature fusion of heterogeneous data (video, audio, hydrology); and third, the complex terrain of mountainous areas makes it difficult to quantify the spatiotemporal correlations in the dynamic characteristics of flood propagation. Especially in densely populated areas such as tourist development zones, the average warning response time of existing systems is as long as 15-30 minutes, which cannot meet the timeliness requirements of disaster prevention and mitigation. Summary of the Invention

[0003] The purpose of this application is to provide a method, equipment and medium for assessing the hazard of multi-mode disaster-causing factors of flash floods, which can improve the accuracy of assessing the hazard of multi-mode disaster-causing factors of flash floods.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In the first aspect, the present application provides a method for assessing the risk of multi-mode flash flood disaster factors, including:

[0006] Acquire multimodal monitoring data from multiple monitoring points in the area to be measured; the multimodal monitoring data includes: infrared video stream, visible light video stream, audio signal and water level meter data;

[0007] Determine any monitoring point as the current monitoring point;

[0008] Input the water level gauge data of the monitoring point into the water level feature extraction module to obtain the water level composite feature matrix;

[0009] Input the infrared video stream and visible light video stream of the monitoring point into the water level feature extraction module to obtain the turbidity composite feature matrix;

[0010] The audio signal of the monitoring point is input into the water level feature extraction module to obtain the sound composite feature matrix;

[0011] The water level composite feature matrix, turbidity composite feature matrix and sound composite feature matrix are input into the feature fusion module for multimodal feature synchronization to obtain the fusion feature matrix of the current monitoring point;

[0012] Traverse all monitoring points to obtain the fusion feature matrix of all monitoring points in the area to be measured;

[0013] Construct a watershed map based on the topological map of the monitoring points in the area to be measured and the fusion feature matrix of each monitoring point;

[0014] The watershed map is input into the spatiotemporal convolutional network to obtain a flash flood disaster risk assessment result; the flash flood disaster risk assessment result includes the probability and arrival time of flash floods; the water level feature extraction module, the turbidity feature extraction module, the sound feature extraction module, the feature fusion module and the spatiotemporal convolutional network are obtained by using a plurality of multimodal historical monitoring data marked with flash flood data in different test areas, and using a loss function to train the initial water level feature extraction module, the initial turbidity feature extraction module, the initial sound feature extraction module, the initial feature fusion module and the initial spatiotemporal convolutional network.

[0015] Optionally, after obtaining multimodal monitoring data of multiple monitoring points in the area to be measured, the method includes:

[0016] Build a timeline through the GPS clock module;

[0017] The water level gauge data were aligned to the time axis using cubic spline interpolation;

[0018] Downsample the infrared video stream and the visible light video stream to the minute level through average pooling;

[0019] The audio signal is mapped to the time axis using the linear projection method to complete the time synchronization processing of multimodal monitoring data.

[0020] Optionally, the working principle of the water level feature extraction module is:

[0021] The time series of water level gauge data is decomposed by STL, and the acceleration characteristics in the trend term after STL decomposition are extracted by second-order difference.

[0022] A 5-dimensional composite feature vector is constructed based on the STL decomposition result and the acceleration feature to obtain a 5-dimensional composite feature vector.

[0023] Optionally, the turbidity feature extraction module includes:

[0024] The illumination compensation module, dual-video feature fusion module, illumination sub-network and multi-stage ResNet architecture are connected in sequence;

[0025] The illumination compensation module is used to perform illumination compensation on the infrared video stream and the visible light video stream using the MSRCR algorithm;

[0026] The dual video feature fusion module is used to fuse the infrared video stream and the visible light video stream to obtain a merged video;

[0027] The illumination sub-network is used to identify the shooting time type of the merged video; the shooting time type is day or night;

[0028] The multi-stage ResNet architecture is used to determine a turbidity composite feature matrix based on the merged video and the shooting time type of the merged video.

[0029] Optionally, the sound feature extraction module includes: a noise reduction module, a multidimensional feature extraction module and a BiLSTM-Attention network connected in sequence;

[0030] The noise reduction module is used to perform noise reduction processing on the audio signal using an improved spectral subtraction method;

[0031] The multidimensional feature extraction module is used to extract multidimensional features from the audio signal after noise reduction processing; the multidimensional features include MFCC, spectral centroid and low-frequency energy ratio;

[0032] The BiLSTM-Attention network is used to determine the sound composite feature matrix based on multi-dimensional features.

[0033] Optionally, the improved spectral subtraction method is:

[0034]

[0035] in, is the estimated clean signal power spectrum; |Y(k)| 2 is the power spectrum of the observed noisy speech signal; E[|N(k)| 2 ]Statistical estimate of the noise power spectrum; α is the over-subtraction factor.

[0036] Optionally, the initial spatiotemporal convolutional network includes: an input layer, a graph convolution layer, a first dilated causal convolution layer, a graph attention layer, a second dilated causal convolution layer, a spatiotemporal interaction layer, a third dilated causal convolution layer and an output layer, which are connected in sequence.

[0037] Optionally, the loss function is:

[0038]

[0039] Where L is the loss function; BCE(P,Y) is the binary cross entropy loss, which measures the difference between the flash flood probability prediction result P and the true label Y; is the mean absolute percentage error, which is used to measure the predicted value of the arrival time of flash floods The percentage error between the actual time τ and the actual time τ. In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned method for assessing the hazard of multi-mode flash flood disaster-causing factors.

[0040] On the third aspect, the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the above-mentioned method for assessing the risk of multi-mode disaster-causing factors of mountain torrents.

[0041] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0042] The present application provides a method, device and medium for assessing the hazard of multi-modal disaster-causing factors of flash floods. In turbidity recognition, the robustness under complex lighting conditions is significantly improved through multi-modal illumination compensation and dynamic weight fusion. Traditional methods are prone to feature deviations when day and night alternate or illumination changes suddenly. This solution is optimized through the following technical points: a three-scale Gaussian kernel (σ=15,80,250) is used to perform multi-scale Retinex color restoration (MSRCR) on infrared and visible light videos respectively, and by dynamically adjusting the balance factor α(x,y) of the RGB channel, local illumination interference such as shadows and reflections is effectively eliminated, so that the turbidity feature remains stable under different illumination intensities. A day and night classification model is constructed based on a lightweight CNN (including a 7×7 convolutional layer and global average pooling) to output the day and night probability, P d ,P n The lighting state is embedded as a meta-feature in the subsequent fusion process. This design enables the model to adaptively distinguish different lighting scenes and avoid the limitations of manual threshold setting. illum =[P d ,P n ] to channel attention mechanism (MLP m Network), dynamically generate the weight distribution coefficient g∈[0,1] of infrared and visible light modes. fuse =g·W ir ⊙F ir +(1-g)·W vi ⊙F vi , realizing the optimized fusion of illumination sensitivity of dual-modal features, the feature fusion error is reduced by 23.6% compared with the traditional weighted average method. Through the synergistic effect of MSRCR enhancement (eliminating local illumination interference), illumination sub-network (global illumination state modeling) and channel attention mechanism (dynamic modal weight allocation), through the combination of physical enhancement and deep learning, the dependence of turbidity recognition on illumination conditions is fundamentally solved. Through multimodal spatiotemporal joint modeling and deep balanced fusion algorithm, the spatiotemporal prediction accuracy of flash flood warning is improved to minute level. The main technical support includes: using GPS clock module to establish a unified time axis τ∈R m , through cubic spline interpolation to align water level data, video turbidity features (average pooling downsampling) and audio features (linear projection mapping), the time drift error of multimodal data is eliminated, providing highly consistent input for subsequent fusion. Through the implicit balance equation Z = f θ ([Hlevel ∥H turb ∥H voice ]+α·ATT(Z)) and Anderson acceleration solver, the water level, turbidity, and voiceprint trimodal features are nonlinearly interacted in a 512-dimensional latent space. The model converges to an equilibrium state after 5 iterations. The spatiotemporal graph convolutional network constructs a spatiotemporal propagation model based on the monitoring point topology graph G = (V, E, A), where the edge weight A ij =exp(-d ij / 100) encodes the attenuation pattern of river channel distances, combining three layers of graph convolution with dilated time series convolution (dilation factors d = 1, 2, and 4) to simultaneously capture spatial flood propagation dynamics and temporal evolution patterns. Inspired by the "three observations and one listening" approach to flash flood disaster assessment, detection equipment collects video, sound, and water level data to detect multiple disaster-causing factors, fusion-based assessment of flash flood hazards, and real-time information collected by the equipment to reduce average warning response time. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 This is a schematic diagram of a feature extraction network in an embodiment of the present application;

[0045] Figure 2 This is a schematic diagram of a spatiotemporal convolutional network in one embodiment of the present application;

[0046] Figure 3 This is a flow chart of a method for assessing the risk of flood multi-mode disaster-causing factors in one embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0049] In an exemplary embodiment, Figure 1-Figure 3As shown in FIG, a method for assessing the hazard of multi-mode flash flood disaster factors is provided, including:

[0050] Step 101: Acquire multimodal monitoring data from multiple monitoring points in the area to be measured. The multimodal monitoring data includes: infrared video stream, visible light video stream, audio signal and water level meter data.

[0051] Step 102: Determine any monitoring point as the current monitoring point.

[0052] Step 103: Input the water level gauge data of the monitoring point into the water level feature extraction module to obtain a water level composite feature matrix.

[0053] Step 104: Input the infrared video stream and visible light video stream of the monitoring point into the water level feature extraction module to obtain a turbidity composite feature matrix.

[0054] Step 105: Input the audio signal of the monitoring point into the water level feature extraction module to obtain a sound composite feature matrix.

[0055] Step 106: Input the water level composite feature matrix, the turbidity composite feature matrix, and the sound composite feature matrix into the feature fusion module for multimodal feature synchronization to obtain the fusion feature matrix of the current monitoring point.

[0056] Step 107: Traverse all monitoring points to obtain a fusion feature matrix of all monitoring points in the area to be measured.

[0057] Step 108: Construct a watershed map based on the topological map of the monitoring points in the area to be measured and the fusion feature matrix of each monitoring point.

[0058] Step 109: Input the watershed map into the spatiotemporal convolutional network to obtain flash flood disaster risk assessment results. These include flash flood probability and arrival time. The water level feature extraction module, turbidity feature extraction module, sound feature extraction module, feature fusion module, and spatiotemporal convolutional network are trained using a loss function using multimodal historical monitoring data from multiple areas labeled with flash flood data.

[0059] Prepare a training data set. For the target small watershed, obtain the data of the existing monitoring stations, including: video history data of water surface changes recorded by the monitoring points, sound data recorded by the monitoring points, water level data recorded by the monitoring points, and the relative position relationship between the monitoring points. Establish a prediction model. The algorithm for assessing the risk of multi-modal disaster factors of flash floods mainly consists of a single-site multi-modal fusion module (SMMF) and a spatio-temporal graph convolutional network (STGCN). The SMMF module completes the following tasks: (1) Water level feature extraction module: obtain the water level composite feature matrix V level ∈R t×5 ; (2) Turbidity feature extraction module: obtain turbidity composite feature matrix V turb ∈R t×512 ; (3) Sound feature extraction module: obtain the sound composite feature matrix V voice ∈R t×3 (4) Feature fusion module: Use cubic spline interpolation to align the time series of each modality and obtain the three aligned feature matrices H level 、H turb 、H voice . And establish the implicit equilibrium equation Z=f θ ([H level ||H turb ||H voice ]+α·ATT(Z)). Through the SMMF module, the mixed features of node i can be obtained By combining the mixed features of each node and the upstream and downstream relationships of each node, a spatiotemporal map of the watershed area can be constructed. The obtained spatiotemporal map is input into the STGCN module to determine the probability of flash floods at a specified node. The training data (video data, sound data, and water level data) is input into the flash flood multimodal disaster risk assessment algorithm, and the model parameters are adjusted using the loss function value. The real-time video data, sound data, and water level data acquired every minute are transferred to the trained model to identify flash flood disaster risks and provide flash flood warnings at designated detection points.

[0060] After step 101, the following steps are included:

[0061] Step 1010: Construct a time axis through the GPS clock module.

[0062] Step 1011: Use the cubic spline interpolation method to align the water level gauge data to the time axis.

[0063] Step 1012: Downsample the infrared video stream and the visible light video stream to minute level by average pooling.

[0064] Step 1013: Use the linear projection method to map the audio signal to the time axis to complete the time synchronization processing of the multimodal monitoring data.

[0065] The water level feature extraction module works by performing STL decomposition on the time series of water level gauge data and extracting the acceleration features from the trend terms after the STL decomposition using second-order differences. A 5-dimensional composite feature vector is constructed based on the STL decomposition results and the acceleration features.

[0066] The turbidity feature extraction module consists of a sequentially connected illumination compensation module, a dual-video feature fusion module, an illumination sub-network, and a multi-stage ResNet architecture. The illumination compensation module uses the MSRCR algorithm to perform illumination compensation on the infrared and visible light video streams. The dual-video feature fusion module fuses the infrared and visible light video streams to generate a combined video. The illumination sub-network identifies the time of day and night for the combined video. The multi-stage ResNet architecture determines the turbidity composite feature matrix based on the combined video and its time of day.

[0067] The turbidity feature extraction module includes:

[0068] (1) The MSRCR algorithm performs illumination compensation, and its color restoration factor is:

[0069] (2) Dual video feature fusion module to achieve the synthesis of infrared video and visible light video.

[0070] (3) Construction of the illumination sub-network to achieve the discrimination between day and night and reduce the impact of illumination on water turbidity identification.

[0071] (4) The five-stage ResNet architecture extracts spatiotemporal features, and the output dimensions are compressed and aligned through the TCN network.

[0072] The sound feature extraction module includes a noise reduction module, a multidimensional feature extraction module, and a BiLSTM-Attention network, all connected in sequence. The noise reduction module uses an improved spectral subtraction method to reduce noise on the audio signal. The multidimensional feature extraction module extracts multidimensional features from the noise-reduced audio signal. Multidimensional features include MFCC, spectral centroid, and low-frequency energy ratio. The BiLSTM-Attention network determines the sound composite feature matrix based on the multidimensional features. The improved spectral subtraction method is: in, is the estimated clean signal power spectrum; |Y(k)| 2 is the power spectrum of the observed noisy speech signal; E[|N(k)| 2]Statistical estimate of the noise power spectrum; α is the over-subtraction factor.

[0073] The sound feature extraction module includes:

[0074] (1) Improved spectral subtraction noise reduction: The oversubtraction factor α is 1.5.

[0075] (2) 17-dimensional acoustic feature extraction, including MFCC, spectral centroid, and low-frequency energy ratio.

[0076] (3) BiLSTM-Attention network output instantaneous probability p t and trend characteristics Δp t .

[0077] The feature fusion module includes:

[0078] (1) Learnable linear transformation H i =W i V′ i +b i Unified feature dimensions.

[0079] (2) The multi-head attention mechanism calculates cross-modal associations, and the number of heads is set to 8.

[0080] (3) The implicit differential equation solver uses the Broyden iteration method.

[0081] The initial spatiotemporal convolutional network consists of: an input layer, a graph convolutional layer, a first dilated causal convolutional layer, a graph attention layer, a second dilated causal convolutional layer, a spatiotemporal interaction layer, a third dilated causal convolutional layer, and an output layer, which are connected in sequence.

[0082] Spatiotemporal graph convolution includes:

[0083] (1) Spatial convolution kernel parameter W S ∈R d×d Learn hydrological relationships between nodes.

[0084] (2) The temporal convolution expansion coefficient is configured as a five-layer structure of [1, 2, 4, 8, 16] [1, 2, 4, 8, 16].

[0085] (3) Spatiotemporal attention weight Dynamically focus on key nodes.

[0086] Multi-task learning includes:

[0087] (1) The probability prediction branch uses the sigmoid activation function.

[0088] (2) The time prediction branch uses ReLU to constrain non-negative output.

[0089] (3) Weight coefficients λ1 = 0.7, λ2 = 0.3 balance the loss terms.

[0090] The loss function is: Where L is the loss function; BCE(P,Y) is the binary cross entropy loss, which measures the difference between the flash flood probability prediction result P and the true label Y; is the mean absolute percentage error, which is used to measure the predicted value of the arrival time of flash floods The percentage error from the true time τ.

[0091] The technical solution of the present invention achieves minute-level flash flood warning through multimodal data fusion and spatiotemporal graph convolutional network. The specific implementation steps are described in detail below with reference to the accompanying drawings:

[0092] Step 1: Multimodal data acquisition and synchronous processing.

[0093] The three-in-one sensor deployed at the monitoring point collects 1080P video stream (25fps), 16kHz audio signal and water level meter data in real time. The GPS clock module is used to synchronize the three modal data and establish a unified time axis τ∈R m Use cubic spline interpolation to convert the water level feature vector V level ∈R n×5 Aligned to the τ axis, the video turbidity feature V turb The audio feature V is downsampled to minute level by average pooling. voice By linearly projecting data onto a common time axis, the time synchronization of heterogeneous data from multiple sources can be achieved, facilitating subsequent fusion.

[0094] Step 2: Water level feature extraction.

[0095] Perform STL decomposition on the water level time series to obtain Y t =T t +S t +R t , where the trend term $T_t$ uses second-order difference to extract acceleration features:

[0096]

[0097] Construct a 5-dimensional composite feature vector V t =[T t ,ΔT t ,Δ 2 T t ,S t ,R t ] T By capturing the acceleration characteristics of water level changes, the design can identify abnormal surge / dip patterns in advance.

[0098] Step 3: Turbidity feature extraction.

[0099] For video turbidity analysis, MSRCR enhancement is performed on both infrared and visible light videos to balance the lighting conditions. Furthermore, an illumination subnetwork is constructed to implement day / night classification and incorporated into the features to further mitigate the influence of illumination on turbidity recognition.

[0100] 1.MSRCR enhancement: A three-scale Gaussian kernel σ=15, 80, 250 is used for illumination compensation, and the color restoration factor α(x, y) dynamically adjusts the RGB channel balance.

[0101] 2. Construct a light perception subnetwork to implement day and night classification: the lightweight CNN network contains a 7×7 convolution layer and global average pooling, and outputs the day and night probability P d ,P n Guide feature fusion. The illumination perception sub-network is shown in Table 1.

[0102] Table 1 Light perception sub-network structure

[0103]

[0104] Dual video fusion: infrared and visible light features are weighted by channel attention, and when channel attention weighting is performed on each modal feature, the illumination state information is fused. m For a two-layer fully connected network, lighting condition information is introduced into the attention weight generation process.

[0105] s illum =[P d ,P n ];

[0106] z m =GAP(F m )m∈{ir,vi};

[0107] w m =σ(MLP m (Concat(z m ,s illum ))); Generate inter-modal weight distribution coefficients based on illumination state:

[0108] g=sigmoid(FC(s illum )).

[0109] Where g∈[0,1] is the infrared feature weight, and 1-g represents the visible light weight.

[0110] The final fusion formula is as follows:

[0111]

[0112] in represents channel-by-channel multiplication,

[0113] 4. ResNet for Turbidity Recognition: After reducing the influence of illumination, ResNet was used to identify the turbidity of the river in the video. The network structure is shown in Table 2.

[0114] Table 2 ResNet network structure

[0115]

[0116] Step 4: Mountain sound pattern recognition.

[0117] Build a mixed signal processing-deep learning architecture.

[0118] 1. Preprocessing: 25ms Hamming window framing, improved spectral subtraction to retain low-frequency features α = 1.5.

[0119] 2. Feature engineering: Extract 13-dimensional MFCC + short-time energy + zero-crossing rate + spectral centroid to form a 17-dimensional feature vector.

[0120] 3. BiLSTM-ATT network: 128-unit bidirectional LSTM captures temporal dependencies, and the 8-head self-attention mechanism focuses on key frames.

[0121] Output constructs a three-dimensional probability matrix Effectively suppress instantaneous noise interference.

[0122] Step 5: Multimodal feature fusion.

[0123] Design a deep equalization fusion model:

[0124] 1. Through the learnable matrix Unify the trimodal features into a 512-dimensional latent space.

[0125] 2. Establish the implicit equilibrium equation Z = f θ ([H level ∥H turb ∥H voice ]+α·ATT(Z)).

[0126] 3. Use Anderson acceleration method to solve Z * =root_find(Zf θ (...)), the number of iterations is ≤ 5 times.

[0127] 4. After multimodal feature fusion, the fusion feature matrix can be obtained for detection site i

[0128] Step 6: Watershed graph structure modeling.

[0129] Define the monitoring point topology G = (V, E, A):

[0130] Nodes: v1,...,v k represents k monitoring points.

[0131] Edge weight: A ij =exp(-d ij / 100) reflects the upstream and downstream spatial association.

[0132] Node feature matrix: For each detection site i, the fusion feature matrix can be obtained

[0133] Spatiotemporal convolution: 3 layers of graph convolution + dilated temporal convolution (d = 1, 2, 4) to capture the dynamic characteristics of flood propagation. The spatiotemporal convolutional network design is shown in Table 3.

[0134] Table 3. Design of spatiotemporal convolutional network

[0135]

[0136] Step 7: Early warning and prediction module.

[0137] Building multi-task output

[0138] 1. Flash flood probability: Integrate current status with historical features.

[0139] 2. Arrival time: The rectified linear unit is used to constrain the output non-negativity.

[0140] Loss Function Balance classification and regression tasks.

[0141] With respect to the flash flood multi-mode disaster-causing factor hazard assessment algorithm of the present invention, the following alternative technical solutions may exist under the core goal of achieving minute-level early warning:

[0142] 1. Alternatives based on multimodal temporal transformers: A temporal attention mechanism can be used to replace the spatiotemporal graph convolution module, and early warning can be achieved by building global spatiotemporal dependencies. Specifically: (1) In the feature fusion stage, a multi-head self-attention mechanism is used to replace the implicit balance equation, and the spatial topological relationship between monitoring points is preserved through position encoding; (2) The upstream and downstream relationships of monitoring points are encoded as an attention mask matrix to constrain the physical laws of flood propagation; (3) A layered transformer architecture is used, with the bottom layer processing single-site multimodal features and the top layer modeling cross-site spatiotemporal correlations. This solution avoids the need to construct an explicit adjacency matrix for graph convolution, but may increase the computational complexity of long-term temporal series modeling.

[0143] 2. Hybrid architecture driven by physical models: An alternative approach can be constructed by combining hydrodynamic equations with data-driven models: (1) Using the Saint-Venant equations to establish physical constraints on river flood propagation, a numerical simulator based on partial differential equations is constructed; (2) Neural networks are embedded in the physical model as residual correction terms, and real-time data from monitoring stations is integrated through data assimilation techniques; (3) Using a transfer learning strategy, the parameters of the hydrological model trained in a large watershed are transferred to a small watershed for fine-tuning. This approach enhances the interpretability of the model, but requires accurate river topography data.

[0144] 3. Federated learning-driven distributed computing solution: To address the need for data privacy protection at multiple monitoring points, this solution can be restructured into a distributed learning framework: (1) Each monitoring site trains a single-modal feature extractor locally and only uploads the encrypted feature vector to the central server; (2) The central server completes multi-site feature fusion through a secure aggregation algorithm; (3) Spatiotemporal modeling uses a differentially private graph neural network, adding Gaussian noise during parameter updates. This solution sacrifices some real-time performance in exchange for data security and is suitable for coordinated watershed early warning across administrative regions.

[0145] 4. Lightweight edge computing solution: To meet the deployment requirements of low-power devices, a model compression alternative can be designed: (1) Replacing the ResNet backbone network with a depthwise separable convolution reduces the computational complexity of the turbidity feature extraction module by more than 60%; (2) Using knowledge distillation technology, the spatiotemporal graph convolution module is compressed into a cascade structure of temporal causal convolution and spatial attention mechanism; (3) Constructing a multimodal feature early fusion strategy to complete partial feature alignment and dimensionality reduction at the sensor end. This solution is suitable for remote mountainous areas with limited communication conditions, but may affect model accuracy.

[0146] All of the above alternatives achieve the core goals of multimodal data fusion and minute-level early warning, but they differ in model performance, computational efficiency, deployment cost, and interpretability. The core innovation of this invention lies in the realization of multimodal dynamic coupling through implicit equilibrium equations and the convolutional modeling of spatiotemporal graphs based on monitoring point topology. This approach offers unique advantages over alternatives in terms of real-time performance and prediction accuracy.

[0147] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for assessing the hazard of multi-mode disaster-causing factors of flash floods is implemented.

[0148] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0149] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0150] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0151] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0152] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0153] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for assessing the risk of multi-mode flash flood disaster factors, characterized by: include: Obtain multimodal monitoring data from multiple monitoring points in the area to be measured; The multimodal monitoring data includes: infrared video stream, visible light video stream, audio signal and water level meter data; Determine any monitoring point as the current monitoring point; Input the water level gauge data of the monitoring point into the water level feature extraction module to obtain the water level composite feature matrix; Input the infrared video stream and visible light video stream of the monitoring point into the water level feature extraction module to obtain the turbidity composite feature matrix; The audio signal of the monitoring point is input into the water level feature extraction module to obtain the sound composite feature matrix; The water level composite feature matrix, turbidity composite feature matrix and sound composite feature matrix are input into the feature fusion module for multimodal feature synchronization to obtain the fusion feature matrix of the current monitoring point; Traverse all monitoring points to obtain the fusion feature matrix of all monitoring points in the area to be measured; Construct a watershed map based on the topological map of the monitoring points in the area to be measured and the fusion feature matrix of each monitoring point; The watershed map is input into the spatiotemporal convolutional network to obtain a flash flood disaster risk assessment result; the flash flood disaster risk assessment result includes the probability and arrival time of flash floods; the water level feature extraction module, the turbidity feature extraction module, the sound feature extraction module, the feature fusion module and the spatiotemporal convolutional network are obtained by using a plurality of multimodal historical monitoring data marked with flash flood data in different test areas, and using a loss function to train the initial water level feature extraction module, the initial turbidity feature extraction module, the initial sound feature extraction module, the initial feature fusion module and the initial spatiotemporal convolutional network.

2. The method for assessing the risk of multi-mode flash flood disaster factors according to claim 1 is characterized in that: After obtaining multimodal monitoring data from multiple monitoring points in the area to be measured, including: Build a timeline through the GPS clock module; The water level gauge data were aligned to the time axis using cubic spline interpolation; Downsample the infrared video stream and the visible light video stream to the minute level through average pooling; The audio signal is mapped to the time axis using the linear projection method to complete the time synchronization processing of multimodal monitoring data.

3. The method for assessing the risk of multi-mode flash flood disaster factors according to claim 2 is characterized in that: The working principle of the water level feature extraction module is: The time series of water level gauge data is decomposed by STL, and the acceleration characteristics in the trend term after STL decomposition are extracted by second-order difference. A 5-dimensional composite feature vector is constructed based on the STL decomposition result and the acceleration feature to obtain a 5-dimensional composite feature vector.

4. The method for assessing the risk of multi-mode flash flood disaster factors according to claim 3 is characterized in that: The turbidity feature extraction module includes: The illumination compensation module, dual-video feature fusion module, illumination sub-network and multi-stage ResNet architecture are connected in sequence; The illumination compensation module is used to perform illumination compensation on the infrared video stream and the visible light video stream using the MSRCR algorithm; The dual video feature fusion module is used to fuse the infrared video stream and the visible light video stream to obtain a merged video; The illumination sub-network is used to identify the shooting time type of the merged video; the shooting time type is day or night; The multi-stage ResNet architecture is used to determine a turbidity composite feature matrix based on the merged video and the shooting time type of the merged video.

5. The method for assessing the risk of multi-mode flash flood disaster factors according to claim 4 is characterized in that: The sound feature extraction module includes: a noise reduction module, a multidimensional feature extraction module and a BiLSTM-Attention network connected in sequence; The noise reduction module is used to perform noise reduction processing on the audio signal using an improved spectral subtraction method; The multidimensional feature extraction module is used to extract multidimensional features from the audio signal after noise reduction processing; the multidimensional features include MFCC, spectral centroid and low-frequency energy ratio; The BiLSTM-Attention network is used to determine the sound composite feature matrix based on multi-dimensional features.

6. The method for assessing the risk of multi-mode flash flood disaster factors according to claim 5 is characterized in that: The improved spectral subtraction method is: in, is the estimated clean signal power spectrum; |Y(k)| 2 is the power spectrum of the observed noisy speech signal; E[|N(k)| 2 ]Statistical estimate of the noise power spectrum; α is the over-subtraction factor.

7. The method for assessing the risk of multi-mode flash flood disaster factors according to claim 5 is characterized in that: The initial spatiotemporal convolutional network includes: an input layer, a graph convolution layer, a first dilated causal convolution layer, a graph attention layer, a second dilated causal convolution layer, a spatiotemporal interaction layer, a third dilated causal convolution layer and an output layer, which are connected in sequence.

8. The method for assessing the risk of multi-mode flash flood disaster factors according to claim 7 is characterized in that: The loss function is: Where L is the loss function; BCE(P,Y) is the binary cross entropy loss, which measures the difference between the flash flood probability prediction result P and the true label Y; is the mean absolute percentage error, which is used to measure the predicted value of the arrival time of flash floods The percentage error from the true time τ.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and runnable on the processor, characterized in that the processor executes the computer program to implement the method for assessing the hazard of multi-mode disaster-causing factors of mountain torrents according to any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for assessing the hazard of multi-mode disaster-causing factors of mountain torrents according to any one of claims 1 to 8 is implemented.

Images