A flash flood multi-mode disaster-causing element risk judgment method, device and medium

Abstract

The application discloses a mountain torrent multi-mode disaster-causing element risk judgment method, equipment and medium, relates to the mountain torrent disaster risk prediction technical field, and the method comprises the steps of: obtaining the fusion feature matrix of all monitoring points in the to-be-detected area by using a feature extraction network and multi-modal monitoring data of a plurality of monitoring points in the to-be-detected area; constructing a river basin graph based on the monitoring point topology graph of the to-be-detected area and the fusion feature matrix of each monitoring point; inputting the river basin graph into a space-time convolution network to obtain a mountain torrent disaster risk judgment result. The application improves the mountain torrent multi-mode disaster-causing element risk judgment precision by constructing the feature extraction network and the space-time convolution network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mountain torrent disaster risk prediction, in particular to a mountain torrent multi-mode disaster-causing element risk judgment method, device and medium. BACKGROUND

[0002] Under the background of global climate change intensifying and extreme weather occurring frequently, mountain torrent disasters in small mountainous watersheds show the characteristics of strong suddenness and complex disaster-causing chain. There are three major technical bottlenecks in the traditional mountain torrent early warning system: first, single-element monitoring (such as water level) is difficult to capture the disaster precursor of multi-physical field coupling; second, there is a modeling gap in the spatio-temporal alignment and feature fusion of heterogeneous data (video, audio, hydrology); third, the spatio-temporal correlation of flood propagation dynamics characteristics is difficult to quantify due to the complex terrain in mountainous areas. In particular, in densely populated areas such as tourist development zones, the average early warning response time of existing systems is as long as 15-30 minutes, which cannot meet the timeliness requirements of disaster prevention and reduction. SUMMARY

[0003] The purpose of the present application is to provide a mountain torrent multi-mode disaster-causing element risk judgment method, device and medium, which can improve the risk judgment accuracy of mountain torrent multi-mode disaster-causing elements.

[0004] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0005] In a first aspect, the present application provides a mountain torrent multi-mode disaster-causing element risk judgment method, comprising:

[0006] Obtaining multi-modal monitoring data of a plurality of monitoring points in a to-be-measured area; the multi-modal monitoring data includes infrared video stream, visible light video stream, audio signal and water level meter data;

[0007] Determining any monitoring point as a current monitoring point;

[0008] Inputting the water level meter data of the monitoring point into a water level feature extraction module to obtain a water level composite feature matrix;

[0009] Inputting the infrared video stream and the visible light video stream of the monitoring point into the water level feature extraction module to obtain a turbidity composite feature matrix;

[0010] Inputting the audio signal of the monitoring point into the water level feature extraction module to obtain a sound composite feature matrix;

[0011] Inputting the water level composite feature matrix, the turbidity composite feature matrix and the sound composite feature matrix into a feature fusion module for multi-modal feature synchronization to obtain a fusion feature matrix of the current monitoring point;

[0012] Traversing all monitoring points to obtain fusion feature matrices of all monitoring points in the to-be-measured area;

[0013] constructing a river basin graph based on a topological graph of monitoring points in the to-be-tested region and a fusion feature matrix of each monitoring point;

[0014] inputting the river basin graph into a spatio-temporal convolution network to obtain a flash flood danger judgment result; the flash flood danger judgment result includes a flash flood probability and an arrival time; the water level feature extraction module, the turbidity feature extraction module, the sound feature extraction module, the feature fusion module, and the spatio-temporal convolution network are obtained by training an initial water level feature extraction module, an initial turbidity feature extraction module, an initial sound feature extraction module, an initial feature fusion module, and an initial spatio-temporal convolution network using a plurality of multi-modal historical monitoring data labeled with flash flood data of different to-be-tested regions and using a loss function.

[0015] Optionally, after obtaining the multi-modal monitoring data of the plurality of monitoring points in the to-be-tested region, the method comprises:

[0016] constructing a time axis by a GPS clock module;

[0017] aligning the water level gauge data to the time axis by using a cubic spline interpolation method;

[0018] averaging and pooling the infrared video stream and the visible light video stream to a minute level;

[0019] mapping the audio signal to the time axis by using a linear projection method to complete time synchronization processing of the multi-modal monitoring data.

[0020] Optionally, the working principle of the water level feature extraction module is as follows:

[0021] performing STL decomposition on the time series of the water level gauge data, and extracting an acceleration feature in a trend item after STL decomposition by using second-order difference;

[0022] constructing a 5-dimensional composite feature vector based on the STL decomposition result and the acceleration feature.

[0023] Optionally, the turbidity feature extraction module comprises:

[0024] a light compensation module, a dual-video feature fusion module, a light sub-network, and a multi-stage ResNet architecture connected in sequence;

[0025] the light compensation module is configured to compensate the infrared video stream and the visible light video stream by using an MSRCR algorithm;

[0026] the dual-video feature fusion module is configured to fuse the infrared video stream and the visible light video stream to obtain a combined video;

[0027] the light sub-network is configured to identify a shooting time type of the combined video; the shooting time type is daytime or nighttime.

[0028] The multi-stage ResNet architecture is used to determine the turbidity composite feature matrix based on the confluence video and the shooting time type of the confluence video.

[0029] Optionally, the sound feature extraction module comprises: a noise reduction module, a multi-dimensional feature extraction module and a BiLSTM-Attention network connected in sequence.

[0030] The noise reduction module is used to perform noise reduction processing on the audio signal by using an improved spectral subtraction method.

[0031] The multi-dimensional feature extraction module is used to extract multi-dimensional features from the noise-reduced audio signal; the multi-dimensional features include MFCC, spectral centroid and low-frequency energy ratio.

[0032] The BiLSTM-Attention network is used to determine the sound composite feature matrix based on the multi-dimensional features.

[0033] Optionally, the improved spectral subtraction method is:

[0034]

[0035] wherein, is the estimated clean signal power spectrum; |Y(k)| 2 is the observed noisy speech signal power spectrum; E[|N(k) 2 ] is the statistical estimate of the noise-free power spectrum; and a is the over-subtraction factor.

[0036] Optionally, the initial spatio-temporal convolutional network comprises: an input layer, a graph convolutional layer, a first dilated causal convolutional layer, a graph attention layer, a second dilated causal convolutional layer, a spatio-temporal interaction layer, a third dilated causal convolutional layer and an output layer connected in sequence.

[0037] Optionally, the loss function is:

[0038]

[0039] wherein, L is the loss function; BCE(P, Y) is the binary cross-entropy loss, which measures the difference between the flood probability prediction result P and the true label Y; is the mean absolute percentage error, which is used to measure the percentage error between the flood arrival time prediction value and the true time τ. In a second aspect, the present application provides a computer device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned flood multi-modal disaster element hazard judgment method.

[0040] In a third aspect, the present application provides a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the above flood multi-modal disaster-causing element danger judgment method.

[0041] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0042] The present application provides a flood multi-modal disaster-causing element danger judgment method, device and medium, which significantly improves the robustness under complex lighting conditions through multi-modal lighting compensation and dynamic weight fusion in turbidity recognition. Traditional methods are prone to feature deviation when day and night change or lighting changes suddenly, while the present solution optimizes through the following technical points: a three-scale Gaussian kernel (σ = 15, 80, 250) is used to perform multi-scale Retinex color restoration (MSRCR) on infrared and visible light videos respectively, and by dynamically adjusting the balance factor α(x, y) of the RGB channel, local lighting interference such as shadows and reflections is effectively eliminated, so that the turbidity feature remains stable under different lighting intensities. A day-night classification model is constructed based on a lightweight CNN (containing a 7x7 convolution layer and a global average pooling), which outputs a day-night probability, P d , n The lighting state is embedded as a meta-feature in the subsequent fusion process. This design enables the model to adaptively distinguish different lighting scenarios, avoiding the limitations of manual threshold setting. The lighting state information s illum = [P d , n ] is introduced to the channel attention mechanism (MLP m network), which dynamically generates the weight distribution coefficient g ∈ [0, 1] of the infrared and visible light modalities. Through the formula F fuse = g·W ir ⊙F ir + (1-g)·W vi ⊙F vi , the lighting sensitivity optimization fusion of the dual-modal features is realized, and compared with the traditional weighted average method, the feature fusion error is reduced by 23.6%. Through the synergistic effect of MSRCR enhancement (elimination of local lighting interference), lighting subnetwork (global lighting state modeling) and channel attention mechanism (dynamic modal weight distribution), the dependence of turbidity recognition on lighting conditions is fundamentally solved by combining physical enhancement with deep learning. Through multi-modal spatio-temporal joint modeling and deep balanced fusion algorithm, the spatio-temporal prediction accuracy of flood warning is improved to the minute level, and the main technical support includes: a unified time axis τ ∈ R m is established using a GPS clock module, and the water level data, video turbidity features (average pooling down-sampling) and audio features (linear projection mapping) are aligned through cubic spline interpolation, eliminating the time drift error of multi-modal data and providing high consistency input for subsequent fusion. Through the implicit balance equation Z = f θ ([Hlevel ∥H turb ∥H voice The model, using α·ATT(Z) and an Anderson accelerated solver, achieves nonlinear interaction of the three modal features of water level, turbidity, and acoustic signature in a 512-dimensional latent space. The model converges to an equilibrium state after 5 iterations. A spatiotemporal graph convolutional network constructs a spatiotemporal propagation model based on the monitoring point topology graph G=(V,E,A), where the edge weight A... ij =exp(-d ij The / 100) encoding method encodes the river channel distance attenuation pattern, and combines 3-layer graph convolution and dilated temporal convolution (dilation factor d = 1, 2, 4) to simultaneously capture the spatial flood propagation dynamics and temporal evolution patterns. Inspired by the "three observations and one listening" method for judging flash flood disasters, the system collects video, sound, and water level data through detection equipment to achieve multi-hazard element detection, multi-modal fusion judgment of flash flood disaster risks, and reduces the average early warning response time by collecting real-time information through the equipment. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of a feature extraction network in one embodiment of this application;

[0045] Figure 2 This is a schematic diagram of a spatiotemporal convolutional network in one embodiment of this application;

[0046] Figure 3 This is a flowchart of a flood multi-mode disaster risk assessment method in one embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] In one exemplary embodiment, such as Figures 1-3As shown, a flash flood multi-modal disaster element risk judgment method is provided, comprising:

[0050] Step 101: Obtain multi-modal monitoring data of a plurality of monitoring points in the to-be-tested area. The multi-modal monitoring data includes infrared video stream, visible light video stream, audio signal and water level meter data.

[0051] Step 102: Determine any monitoring point as a current monitoring point.

[0052] Step 103: Input the water level meter data of the monitoring point into a water level feature extraction module to obtain a water level composite feature matrix.

[0053] Step 104: Input the infrared video stream and visible light video stream of the monitoring point into the water level feature extraction module to obtain a turbidity composite feature matrix.

[0054] Step 105: Input the audio signal of the monitoring point into the water level feature extraction module to obtain a sound composite feature matrix.

[0055] Step 106: Input the water level composite feature matrix, turbidity composite feature matrix and sound composite feature matrix into a feature fusion module for multi-modal feature synchronization to obtain a fusion feature matrix of the current monitoring point.

[0056] Step 107: Traverse all monitoring points to obtain fusion feature matrices of all monitoring points in the to-be-tested area.

[0057] Step 108: Construct a watershed graph based on the to-be-tested area monitoring point topology graph and the fusion feature matrix of each monitoring point.

[0058] Step 109: Input the watershed graph into a space-time convolution network to obtain a flash flood disaster risk judgment result. The flash flood disaster risk judgment result includes a flash flood probability and an arrival time. The water level feature extraction module, turbidity feature extraction module, sound feature extraction module, feature fusion module and space-time convolution network are obtained by training an initial water level feature extraction module, initial turbidity feature extraction module, initial sound feature extraction module, initial feature fusion module and initial space-time convolution network using a plurality of multi-modal historical monitoring data labeled with flash flood data of different to-be-tested areas using a loss function.

[0059] Prepare the training data set. For the target small watershed, obtain the existing monitoring station data, including: monitoring point record water surface change video historical data, monitoring point record sound data, monitoring point record water level data, and relative position relationship between monitoring points. Establish a prediction model. The mountain torrent multi-modal disaster element danger judgment algorithm mainly includes a single-site feature fusion module (Single-site Multi-modal Fusion, SMMF) and a spatio-temporal graph convolutional network module (Spatio-Temporal Graph Convolutional Networks, STGCN). The SMMF module completes the following work: (1) water level feature extraction module: obtain the water level composite feature matrix V level ∈R t×5 ; (2) turbidity feature extraction module: obtain the turbidity composite feature matrix V turb ∈R t×512 ; (3) sound feature extraction module: obtain the sound composite feature matrix V voice ∈R t×3 ; (4) feature fusion module: align the time series of each modality using cubic spline interpolation, and obtain the three aligned feature matrices H level , H turb , H voice . An implicit balance equation Z = f θ ([H level ||H turb ||H voice ]+α·ATT(Z)) is established. Through the SMMF module, the mixed features of node i can be obtained. Through the mixed features of each node and the upstream and downstream relationships of each node, the spatio-temporal graph of the watershed area can be constructed. Using the obtained spatio-temporal graph, the STGCN module can be input to obtain the mountain torrent probability judgment of the specified node. The training data (video data, sound data, water level data) are input into the mountain torrent multi-modal disaster element danger judgment algorithm, and the model parameters are adjusted through the loss function value. The real-time obtained video data, sound data, and water level data every minute are transmitted to the trained model to realize the discrimination of the specified detection point mountain torrent disaster danger and the mountain torrent early warning.

[0060] After step 101, the following steps are included:

[0061] Step 1010: Construct a time axis through a GPS clock module.

[0062] Step 1011: Align the water level gauge data to the time axis using the cubic spline interpolation method.

[0063] Step 1012: Average pool and downsample the infrared video stream and visible light video stream to the minute level.

[0064] Step 1013: mapping the audio signal to a time axis by using a linear projection method, to complete the time synchronization processing of the multi-modal monitoring data.

[0065] The working principle of the water level feature extraction module is: performing STL decomposition on the time sequence of the water level gauge data, and extracting the acceleration feature in the trend item after the STL decomposition by using second-order difference. Based on the STL decomposition result and the acceleration feature, a 5-dimensional composite feature vector is constructed as the 5-dimensional composite feature vector.

[0066] The turbidity feature extraction module includes: a light compensation module, a dual video feature fusion module, a light sub-network and a multi-stage ResNet architecture connected in sequence. The light compensation module is used to compensate the infrared video stream and the visible light video stream by using the MSRCR algorithm. The dual video feature fusion module is used to fuse the infrared video stream and the visible light video stream to obtain a combined video. The light sub-network is used to identify the shooting time type of the combined video. The shooting time type is daytime or night. The multi-stage ResNet architecture is used to determine the turbidity composite feature matrix based on the combined video and the shooting time type of the combined video.

[0067] The turbidity feature extraction module includes:

[0068] (1) The MSRCR algorithm is used for light compensation, and the color restoration factor is:

[0069] (2) The dual video feature fusion module is used to realize the synthesis of the infrared video and the visible light video.

[0070] (3) The light sub-network is constructed to realize the discrimination between daytime and night, and reduce the influence of light on the identification of water flow turbidity.

[0071] (4) The five-stage ResNet architecture extracts the spatiotemporal features, and the output dimension is compressed and aligned by the TCN network.

[0072] The sound feature extraction module includes: a noise reduction module, a multi-dimensional feature extraction module and a BiLSTM-Attention network connected in sequence. The noise reduction module is used to perform noise reduction processing on the audio signal by using an improved spectral subtraction method. The multi-dimensional feature extraction module is used to extract multi-dimensional features from the noise reduction processed audio signal. The multi-dimensional features include MFCC, spectral centroid and low frequency energy ratio. The BiLSTM-Attention network is used to determine the sound composite feature matrix based on the multi-dimensional features. The improved spectral subtraction method is: wherein, is the estimated clean signal power spectrum; |Y(k)| 2 is the power spectrum of the observed noisy speech signal; E[|N(k)| 2Statistical estimation of noise-free power spectrum; α is the over-reduction factor.

[0073] The sound feature extraction module comprises:

[0074] (1) Improved spectral subtraction denoising: Where the over-reduction factor α = 1.5.

[0075] (2) 17-dimensional acoustic feature extraction, including MFCC, spectral centroid, and low-frequency energy ratio.

[0076] (3) BiLSTM-Attention network outputs instantaneous probability p t and trend feature Δp t .

[0077] The feature fusion module comprises:

[0078] (1) Learnable linear transformation H i = W i V′ i +b i Uniform feature dimension.

[0079] (2) Multi-head attention mechanism calculates cross-modal correlation, with 8 heads.

[0080] (3) The implicit differential equation solver uses the Broyden iterative method.

[0081] The initial spatio-temporal convolutional network comprises: an input layer, a graph convolutional layer, a first dilated causal convolutional layer, a graph attentional layer, a second dilated causal convolutional layer, a spatio-temporal interactional layer, a third dilated causal convolutional layer, and an output layer connected in sequence.

[0082] The spatio-temporal graph convolution comprises:

[0083] (1) Spatial convolution kernel parameters W S ∈ R d×d Learn the hydrological relationship between nodes.

[0084] (2) The time convolution dilated coefficient is configured as a five-layer structure of [1, 2, 4, 8, 16][1, 2, 4, 8, 16].

[0085] (3) Spatio-temporal attention weight Dynamic focusing key nodes.

[0086] The multi-task learning comprises:

[0087] (1) The probability prediction branch adopts a sigmoid activation function.

[0088] (2) The time prediction branch adopts a ReLU constraint non-negative output.

[0089] (3) Weight coefficients λ1=0.7, λ2=0.3 balance loss term.

[0090] The loss function is: Wherein, L is a loss function; BCE(P, Y) is a binary cross-entropy loss, which measures the difference between the flood probability prediction result P and the true label Y; MAPE is the mean absolute percentage error, which is used to measure the percentage error between the predicted value of the flood arrival time and the true time τ.

[0091] The technical scheme of the application realizes minute-level flood warning through multi-modal data fusion and spatio-temporal graph convolution network. The specific implementation steps are described in detail below in conjunction with the drawings:

[0092] Step 1: Multi-modal data acquisition and synchronous processing.

[0093] The three-in-one sensor deployed at the monitoring point collects 1080P video stream (25fps), 16kHz audio signal and water level meter data in real time. Through the GPS clock module, the time synchronization of the three modal data is realized, and a unified time axis τ∈R m is established. level The water level feature vector V n×5 is aligned to the τ axis, the video turbidity feature V turb is down-sampled to the minute level through average pooling, and the audio feature V voice is mapped to the common time axis through linear projection, so as to realize the time synchronization of the multi-source heterogeneous data here, which is convenient for subsequent fusion

[0094] Step 2: Water level feature extraction.

[0095] The STL decomposition is performed on the water level time series to obtain Y t = T t + S t + R t , wherein the trend item $T_t$ adopts second-order difference extraction to extract acceleration features:

[0096]

[0097] A 5-dimensional composite feature vector V t = [T t , ΔT t , Δ 2 T t , S t , R t ] T is constructed. This design can identify abnormal rapid rise / rapid drop patterns in advance by capturing the acceleration features of water level changes.

[0098] Step 3: Turbidity feature extraction.

[0099] In the video turbidity analysis, the infrared and visible light videos are respectively enhanced by MSRCR to balance the light conditions in the video. In addition, a light perception subnetwork is constructed to realize day-night classification and join the features to further eliminate the influence of light on turbidity recognition.

[0100] 1. MSRCR enhancement: 3-scale Gaussian kernel σ = 15, 80, 250 is used for light compensation, and the color restoration factor α(x, y) is dynamically adjusted to balance the RGB channels.

[0101] 2. Constructing a light perception subnetwork to realize day-night classification: the lightweight CNN network contains a 7 × 7 convolutional layer and a global average pooling, and outputs the day-night probability P d , n guiding feature fusion. The light perception subnetwork is shown in Table 1.

[0102] Table 1 Structure table of light perception subnetwork

[0103]

[0104] Dual video fusion: the infrared and visible light features are weighted by channel attention, and the light state information is fused when each modality feature is weighted by channel attention. The MLP m is a two-layer fully connected network that introduces light condition information into the attention weight generation process.

[0105] s illum = [P d , P n ];

[0106] z m = GAP(F m )m∈{ir, vi};

[0107] w m = σ(MLP m (Concat(z m , s illum ))) The inter-modal weight distribution coefficient is generated by the light state:

[0108] g = sigmoid(FC(s illum )).

[0109] where g ∈ [0, 1] is the infrared feature weight, and 1-g represents the visible light weight.

[0110] The final fusion formula is shown in the figure:

[0111]

[0112] where denotes a per-channel multiplication,

[0113] 4. ResNet for turbidity recognition: After the influence of light is weakened, ResNet is used to recognize the turbidity of the river in the video, and the network structure is shown in Table 2.

[0114] Table 2 ResNet network structure table

[0115]

[0116] Step 4: Mountain soundprint recognition.

[0117] Construct a hybrid signal processing-deep learning architecture.

[0118] 1. Preprocessing: 25ms Hamming window framing, improved spectrum subtraction to retain low frequency features α = 1.5.

[0119] 2. Feature engineering: Extract 13-dimensional MFCC + short-time energy + zero-crossing rate + spectral centroid to form a 17-dimensional feature vector.

[0120] 3. BiLSTM-ATT network: 128-unit bidirectional LSTM to capture temporal dependencies, 8-head self-attention mechanism to focus on key frames.

[0121] Output a three-dimensional probability matrix Effectively suppresses transient noise interference.

[0122] Step 5: Multimodal feature fusion.

[0123] Design a deep balanced fusion model:

[0124] 1. Through a learnable matrix Unify the three modalities to a 512-dimensional hidden space.

[0125] 2. Establish an implicit balance equation Z = f θ ([H level ∥H turb ∥H voice ]+α·ATT(Z))).

[0126] 3. Use Anderson's acceleration method to solve Z * = root_find(Z-f θ (...)), the number of iterations ≤ 5 times.

[0127] 4. After multimodal feature fusion, for detection site i, the fused feature matrix

[0128] Step 6: Modeling of basin graph structure.

[0129] Definition of monitoring point topology graph G=(V,E,A):

[0130] Node: v1,...,v k Indicates k monitoring points.

[0131] Edge weight: A ij =exp(-d ij / 100) reflects the upstream and downstream spatial correlation.

[0132] Node feature matrix: for each detection site i, the fusion feature matrix

[0133] Spatio-temporal convolution: 3-layer graph convolution + dilated temporal convolution (d=1,2,4), capturing the dynamic characteristics of flood propagation, here the spatio-temporal convolution network design table 3.

[0134] Table 3 Spatio-temporal convolution network design table

[0135]

[0136] Step 7: Early warning prediction module.

[0137] Constructing multi-task output

[0138] 1. Torrential flood probability: Fusion of current state and historical features.

[0139] 2. Arrival time: Adopting a modified linear unit to constrain the output non-negativity.

[0140] Loss function Balancing classification and regression tasks.

[0141] For the torrential flood multi-modal disaster-causing element risk judgment algorithm of the application, in the core goal of realizing minute-level early warning, the following alternative technical solutions may exist:

[0142] 1. Alternative scheme based on multi-modal time series Transformer: time series attention mechanism can be used instead of spatio-temporal graph convolution module to realize early warning by constructing global spatio-temporal dependency. Specifically: (1) In the feature fusion stage, use multi-head self-attention mechanism instead of implicit balance equation, and retain the spatial topology relationship between monitoring points through position encoding; (2) Encode the upstream and downstream relationship of monitoring points into an attention mask matrix, and constrain the physical law of flood propagation; (3) Use a hierarchical Transformer architecture, the bottom layer processes single-site multi-modal features, and the high layer models the cross-site spatio-temporal association. This scheme avoids the need for explicit adjacency matrix construction of graph convolution, but may increase the computational complexity of long-time series modeling.

[0143] 2. Hybrid architecture based on physical model driving: A solution can be constructed by combining hydrodynamic equations and data-driven models: (1) Use the Saint-Venant equation set to establish the physical constraints of river flood propagation and construct a numerical simulator based on partial differential equations; (2) Embed a neural network as a residual correction term into the physical model and integrate real-time data from monitoring stations through data assimilation technology; (3) Use transfer learning strategy to transfer the hydrological model parameters trained in large basins to small basins for fine-tuning. This solution enhances the interpretability of the model, but requires accurate river terrain data support.

[0144] 3. Federated learning driven distributed computing solution: For multi-monitoring point data privacy protection requirements, it can be reconstructed into a distributed learning framework: (1) Each monitoring station locally trains a single-modal feature extractor and only uploads encrypted feature vectors to the central server; (2) The central server completes multi-station feature fusion through a secure aggregation algorithm (Secure Aggregation); (3) The spatio-temporal modeling uses a differentially private protected graph neural network that adds Gaussian noise when updating parameters. This solution sacrifices some real-time performance for data security and is suitable for cross-administrative area basin collaborative early warning.

[0145] 4. Lightweight edge computing solution: To adapt to the deployment needs of low-power devices, a model compression alternative solution can be designed: (1) Replace the ResNet backbone network with a depth separable convolution to reduce the turbidity feature extraction module computation by more than 60%; (2) Use knowledge distillation technology to compress the spatio-temporal graph convolution module into a cascaded structure of time series causal convolution and spatial attention mechanism; (3) Build a multi-modal feature early fusion strategy to complete part of the feature alignment and dimensionality reduction at the sensor end. This solution is suitable for remote mountainous areas with limited communication conditions, but may affect model accuracy.

[0146] The above alternative solutions can achieve the core goal of multi-modal data fusion and minute-level early warning, but differ in model performance, computational efficiency, deployment cost, and interpretability. The core innovation of the invention is to achieve multi-modal dynamic coupling through implicit balance equations and spatio-temporal graph convolution modeling based on monitoring point topology, which has unique advantages in real-time performance and prediction accuracy compared to alternative solutions.

[0147] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The I / O interfaces of the computer device are used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for assessing the risk of flash floods caused by multiple disaster factors.

[0148] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0149] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0150] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0151] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0152] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0153] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for assessing the risk of flash floods caused by multiple disaster factors, characterized in that, include: Acquire multimodal monitoring data from multiple monitoring points in the area to be tested; The multimodal monitoring data includes: infrared video stream, visible light video stream, audio signal, and water level gauge data; A time axis is constructed using a GPS clock module; cubic spline interpolation is used to align the water level gauge data to the time axis; infrared and visible light video streams are averaged and downsampled to the minute level using pooling; and audio signals are mapped to the time axis using linear projection to complete the time synchronization processing of multimodal monitoring data. Select any monitoring point as the current monitoring point; Input the water level gauge data from the monitoring points into the water level feature extraction module to obtain the water level composite feature matrix; The infrared video stream and visible light video stream from the monitoring point are input into the turbidity feature extraction module to obtain the turbidity composite feature matrix; The turbidity feature extraction module includes: The lighting compensation module, dual video feature fusion module, lighting sub-network, and multi-stage ResNet architecture are connected in sequence. The illumination compensation module is used to perform illumination compensation on infrared video streams and visible light video streams using the MSRCR algorithm. The MSRCR algorithm uses a three-scale Gaussian kernel to perform multi-scale Retinex color restoration on infrared and visible light video streams respectively, and dynamically adjusts the balance factor of the RGB channels. The dual-video feature fusion module is used to fuse infrared video streams and visible light video streams to obtain a merged video. The dual-video feature fusion module performs channel attention weighting on infrared and visible light features. When performing channel attention weighting on each modal feature, it fuses illumination state information. The illumination sub-network is used to identify the shooting time type of the merged video; the shooting time type is daytime or nighttime; the illumination sub-network is a day / night classification model built on a lightweight CNN; the lightweight CNN includes 7×7 convolutional layers and global average pooling; The multi-stage ResNet architecture is used to determine the turbidity composite feature matrix based on the merged video and the shooting time type of the merged video. The audio signal from the monitoring point is input into the sound feature extraction module to obtain the composite sound feature matrix; The composite feature matrix of water level, composite feature matrix of turbidity, and composite feature matrix of sound are input into the feature fusion module for multimodal feature synchronization to obtain the fused feature matrix of the current monitoring point; By traversing all monitoring points, the fusion feature matrix of all monitoring points in the area to be measured is obtained; A watershed map is constructed based on the topology map of monitoring points in the area to be measured and the fusion feature matrix of each monitoring point; The watershed map is input into a spatiotemporal convolutional network to obtain the flash flood hazard assessment results. The flash flood hazard assessment results include the flash flood probability and arrival time. The water level feature extraction module, the turbidity feature extraction module, the sound feature extraction module, the feature fusion module, and the spatiotemporal convolutional network are obtained by training the initial water level feature extraction module, the initial turbidity feature extraction module, the initial sound feature extraction module, the initial feature fusion module, and the initial spatiotemporal convolutional network using a loss function after using multiple multimodal historical monitoring data labeled with flash flood data from different areas to be measured.

2. The method for assessing the multi-mode disaster-causing factors of flash floods according to claim 1, characterized in that, The working principle of the water level feature extraction module is as follows: STL decomposition was performed on the time series of water level gauge data, and the acceleration feature in the trend term after STL decomposition was extracted using second-order difference. A 5-dimensional composite feature vector is constructed based on the STL decomposition results and the acceleration features.

3. The method for assessing the multi-mode disaster-causing factors of flash floods according to claim 2, characterized in that, The sound feature extraction module includes: a noise reduction module, a multi-dimensional feature extraction module, and a BiLSTM-Attention network connected in sequence; The noise reduction module is used to perform noise reduction processing on audio signals using an improved spectral subtraction method; The multidimensional feature extraction module is used to extract multidimensional features from the noise-reduced audio signal; the multidimensional features include MFCC, spectral centroid, and low-frequency energy ratio; BiLSTM-Attention network is used to determine the composite feature matrix of sound based on multidimensional features.

4. The method for assessing the multi-mode disaster-causing factors of flash floods according to claim 3, characterized in that, The improved spectral subtraction method is as follows: ； in, The estimated clean signal power spectrum; The power spectrum of the observed noisy speech signal; Statistical estimation of the power spectrum without noise; It is an over-subtraction factor.

5. The method for assessing the multi-mode disaster-causing factors of flash floods according to claim 3, characterized in that, The initial spatiotemporal convolutional network comprises: an input layer, a graph convolutional layer, a first dilated causal convolutional layer, a graph attention layer, a second dilated causal convolutional layer, a spatiotemporal interaction layer, a third dilated causal convolutional layer, and an output layer, connected in sequence.

6. The method for assessing the multi-mode disaster-causing factors of flash floods according to claim 5, characterized in that, The loss function is: ； in, The loss function; Binary cross-entropy loss is used to measure the probability prediction results of flash floods. With real labels The differences between them; The mean absolute percentage error is used to measure the predicted time of arrival of flash floods. With real time The percentage error between them.

7. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method for assessing the multi-mode disaster-causing factors of flash floods as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for assessing the multi-mode disaster-causing factors of flash floods as described in any one of claims 1-6.

Images