Multi-mode coupling flood forecasting method and system based on gated cross attention mechanism

Abstract

The invention provides a multi-mode coupling flood forecasting method and system based on a gated cross attention mechanism, and relates to the technical field of flood forecasting. The method comprises the following steps: acquiring high-precision rainfall monitoring information, numerical forecast rainfall information and hydrological station session flood excerpt information of a predetermined drainage basin; constructing training, verification and test samples; a spatial-temporal feature coding module based on a convolutional neural network and Transform is constructed, and rainfall information is fully mined; a time sequence feature coding module based on a time sequence convolutional network is constructed, and runoff information is fully mined; constructing a multi-modal feature fusion module based on a gated cross attention mechanism, and fusing different modal features mined by the time sequence and spatial-temporal feature coding modules; and constructing a flood prediction module based on a self-attention mechanism to perform flood prediction. According to the invention, the accuracy and reliability of flood forecasting are improved based on the multi-modal feature fusion technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of flood forecasting, in particular to a multi-modal coupled flood forecasting method and system based on a gated cross-attention mechanism. BACKGROUND

[0002] Flood forecasting, as a key means of disaster prevention and mitigation, is of great significance to improving emergency response efficiency and disaster risk management and control capacity.

[0003] Traditional flood forecasting is mostly based on physical mechanisms or conceptual hydrological models, which have numerous parameters, complex calibration, and rely on a large amount of prior knowledge, making it difficult to meet real-time and regional generalization requirements. Currently, artificial intelligence technologies represented by deep learning are developing rapidly, and due to their strong non-linear mapping capabilities and advantages in automatically extracting features from massive data, they have been widely applied in the field of flood forecasting and have shown great potential in capturing complex hydrological time series dependencies. Typical models such as LSTM, CNN, and Transformer have shown good performance in rainfall-runoff modeling and water level prediction. Flood processes are essentially complex systems driven by multiple factors. High-precision flood forecasting not only depends on historical runoff one-dimensional time series data, but also depends on three-dimensional spatio-temporal data such as precipitation. With the development of remote sensing, meteorological, and hydrological observation technologies, more rich multi-modal data information is provided, bringing great opportunities to improve flood forecasting accuracy. However, different modal data has great differences in dimension, scale, and physical meaning, and direct fusion can easily lead to unstable model performance and affect prediction accuracy.

[0004] In summary, designing an effective architecture to extract and deeply fuse these multi-modal features is an important technical means to improve flood forecasting accuracy. SUMMARY

[0005] The purpose of the present application is to propose a multi-modal coupled flood forecasting method based on a gated cross-attention mechanism, and further propose a system for implementing the above method, which adaptively controls information flow by introducing a gated cross-attention mechanism, balances feature information from cross-attention, realizes more fine and efficient coupling between multi-modal information, improves the model's ability to identify key factors that cause floods, to solve the above problems existing in the prior art, and improve the accuracy of flood forecasting.

[0006] To achieve the above purpose, the first aspect of the present application proposes a multi-modal coupled flood forecasting method based on a gated cross-attention mechanism, comprising the following steps:

[0007] Obtain high-precision precipitation monitoring information, numerical prediction precipitation information, and hydrological station field flood runoff information of a predetermined watershed to form multi-source information;

[0008] The multi-source information is sample-paired according to a preset sliding window size, and training samples, verification samples and test samples including input information and target information are constructed;

[0009] A multi-modal coupled flood forecasting model is constructed, which comprises a space-time feature encoding module, a time series feature encoding module, a multi-modal feature fusion module and a flood forecasting module.

[0010] The space-time feature encoding module is constructed based on a convolutional neural network and a Transformer encoder, and is used to mine precipitation space-time features from the high-precision precipitation monitoring information and the numerical prediction precipitation information.

[0011] The time series feature encoding module is constructed based on a time series convolution network, and is used to mine flood runoff time series features from the hydrological station site flood runoff information.

[0012] The multi-modal feature fusion module is constructed based on a gated cross-attention mechanism, and is used to fully fuse the mined precipitation space-time features and flood runoff time series features to obtain multi-modal features.

[0013] The flood forecasting module is constructed based on a self-attention mechanism, and is used to input the fused multi-modal features into the flood forecasting module for time series runoff prediction to obtain the final flood forecasting result.

[0014] As a preferred scheme of the first aspect, the obtained high-precision precipitation monitoring information, numerical prediction precipitation information and hydrological station site flood runoff information need to be accumulated or interpolated to achieve a target time scale.

[0015] The spatial scales of the high-precision precipitation monitoring information and the numerical prediction precipitation information are unified to LxV, where L is the number of data horizontal direction grids and V is the number of data vertical direction grids.

[0016] As a preferred scheme of the first aspect, before sample pairing the multi-source information according to a preset sliding window size, the following needs to be performed:

[0017] The target flood forecasting period T1, the required past precipitation period T2 and past runoff period T3 of the multi-modal coupled flood forecasting model and the future forecast precipitation period T4 are determined.

[0018] The sliding window size T5 is determined, T5 = T1+ T3, and T5 is used to determine the time 1 in the site flood runoff that can be used for training of the multi-modal coupled flood forecasting model, and for the case that the terminal flood runoff is less than T5 periods but greater than T5-T3 periods, zero value is allowed to be filled to T5 periods.

[0019] As a preferred embodiment of the first aspect, the multi-source information is sample-paired according to a preset sliding window size to construct training samples, validation samples, and test samples, including input information and target information, specifically including:

[0020] Based on time 1, high-precision precipitation monitoring information and hydrological station flood runoff information are paired to construct training samples and validation samples. Based on time 1, high-precision precipitation monitoring information before time T2×6 h and after time T4×6 h are paired with hydrological station flood runoff information, and then 70% are randomly selected as training samples and 30% as validation samples.

[0021] Based on time 1, determine time 2 for conducting the test experiment and construct test samples; determine the daily rolling forecast time 3 for numerical precipitation, and select the intersection of time 1 and time 3 as time 2; based on time 2, pair the precipitation monitoring information and runoff information T2×6 h before time 2, and the runoff information T4×6 h after time 4 with the precipitation forecast information for different forecast periods T4×6 h after time 4, and construct test samples for different forecast periods.

[0022] As a preferred embodiment of the first aspect, the construction process of the spatiotemporal feature encoding module specifically includes:

[0023] A convolutional neural network (CNN) is used to extract local features and compress dimensionality from high-precision precipitation monitoring information. Then, a Transformer encoder is used to establish the dependencies between local features and capture global temporal features. The input to the high-precision precipitation monitoring information is B×(T2+T4)×L×V, and the spatiotemporal features of precipitation are F. img The output is B×(T2+T4)×128, where B is the batch sample size.

[0024] As a preferred embodiment of the first aspect, the flood runoff information input of the time-series feature encoding module is B×T3, and the flood runoff time-series feature F ts The output is B×128.

[0025] As a preferred embodiment of the first aspect, the multimodal feature fusion module is constructed based on a gated cross-attention mechanism, used to fully fuse the mined spatiotemporal features of precipitation and the temporal features of flood runoff, and to integrate the flood runoff temporal features F ts The B×128 extension is the same as the described spatiotemporal precipitation feature F. img With the same dimension B×(T2+T4)×128, the output multimodal fusion feature F is obtained. fused Its shape is B×128.

[0026] As a preferred embodiment of the first aspect, the flood prediction module based on the self-attention mechanism consists of a self-attention mechanism layer, a fully connected layer, an activation function layer, a Dropout layer, and another fully connected layer, respectively. The self-attention mechanism enhances the key information of the fused features. The input to this flood prediction module is the multimodal fused feature F. fused The final flood forecast result is output, which is in the shape of B×T1.

[0027] A second aspect of the present invention provides a multimodal coupled flood forecasting system, the system comprising:

[0028] The data acquisition unit is used to acquire high-precision precipitation monitoring information, numerical forecast precipitation information, and hydrological station flood runoff information for the predetermined watershed, forming multi-source information.

[0029] The sample construction unit is used to perform sample pairing on the multi-source information according to a preset sliding window size, and construct training samples, verification samples and test samples including input information and target information.

[0030] The spatiotemporal feature encoding module, built on a convolutional neural network and a Transformer encoder, is used to extract spatiotemporal features of precipitation from the high-precision precipitation monitoring information and numerical precipitation forecast information.

[0031] The temporal feature encoding module, constructed based on a temporal convolutional network, is used to extract temporal features of flood runoff from the flood runoff information of the hydrological stations.

[0032] The multimodal feature fusion module, built on a gated cross-attention mechanism, is used to fully fuse the spatiotemporal features of precipitation and the temporal features of flood runoff obtained from mining, to obtain multimodal features;

[0033] The flood forecasting module is built based on a self-attention mechanism. The multimodal features obtained by fusion are input into the flood forecasting module to perform time-series runoff prediction and obtain the final flood forecast result.

[0034] When the system is running, it can automatically execute the entire process of the multimodal coupled flood forecasting method based on the gated cross-attention mechanism disclosed in the first aspect.

[0035] A third aspect of the invention provides a computer-readable storage medium storing at least one executable instruction that, when executed on an electronic device, causes the electronic device to perform the multimodal coupled flood forecasting method based on a gated cross-attention mechanism disclosed in the first aspect.

[0036] Beneficial Effects: This invention effectively mines feature information from one-dimensional historical runoff and three-dimensional precipitation using dedicated temporal (TCN) and spatiotemporal (CNN-Transformer) feature encoders. Based on a gated cross-attention mechanism, it adaptively balances the information flow of different modalities and prevents the loss of key features, achieving efficient coupling of multimodal feature information. This method can deeply mine the historical dependence of runoff and the spatiotemporal evolution of precipitation, improve the model's ability to identify key flood-causing factors, and enhance flood forecast accuracy. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0038] Figure 2 This is a flowchart detailing the method of the present invention.

[0039] Figure 3 This is an architecture diagram of the system of the present invention.

[0040] Figure 4 The box plot shows the accuracy indices of flood forecasting models composed of different multimodal fusion methods in the example at the Xinhezhuang station on the training and validation sets.

[0041] Figure 5 The scatter density plot of the multimodal coupled flood forecasting model proposed in this invention at the Xinhezhuang station is shown in the embodiment.

[0042] Figure 6 The figure shows the process of the mean of the flood ensemble forecast and the 90% confidence interval of a typical flood event at Xinhezhuang Station, obtained from observed precipitation and three numerical precipitation forecasts driving the TCTGC model.

[0043] Figure 7 In this example, the flood forecast accuracy index of Xinhezhuang Station under different lead times was obtained by using observed precipitation and three numerical forecast precipitation-driven TCTGC models. Detailed Implementation

[0044] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0045] Example 1:

[0046] With the development and improvement of observation technologies such as remote sensing, meteorology, and hydrology, abundant multimodal data information has been provided for data-driven flood forecasting models, bringing tremendous opportunities to improve flood forecasting accuracy. However, data from different modalities differ in dimension, scale, and physical meaning. Direct fusion can easily lead to unstable model performance, affecting forecast accuracy and restricting the effective application of multimodal data information in the field of flood forecasting.

[0047] Therefore, this embodiment discloses a multimodal coupled flood forecasting method based on a gated cross-attention mechanism. By introducing a gated cross-attention mechanism, deep coupling between multimodal information is achieved, improving the model's ability to identify key flood-causing factors and thus enhancing flood forecasting accuracy. The method flowchart is as follows: Figure 1 As shown, the specific steps are as follows:

[0048] Step 1: Obtain high-precision precipitation monitoring information, numerical forecast precipitation information, and hydrological station flood data for the predetermined watershed; perform spatiotemporal scale registration for all types of information;

[0049] Step 2: Preprocess the high-precision precipitation monitoring information, numerical precipitation forecast information and flood extract information, and pair the multi-source information according to the preset sliding window size to construct training samples, verification samples and test samples including input information and target information.

[0050] Step 3: Construct a spatiotemporal feature encoding module based on convolutional neural networks and Transformers to fully mine the spatiotemporal features of high-precision precipitation monitoring information and numerical forecast precipitation information;

[0051] Step 4: Construct a temporal feature encoding model based on a temporal convolutional network to fully mine the temporal features of floods at hydrological stations;

[0052] Step 5: Construct a multimodal feature fusion module based on a gated cross-attention mechanism to fully integrate the spatiotemporal features of precipitation mined in Step 3 with the temporal features of flood runoff mined in Step 4.

[0053] Step 6: Construct a flood prediction module based on a self-attention mechanism. Input the multimodal features fused in Step 5 based on a gated cross-attention mechanism into this module to perform time-series runoff prediction and obtain the final flood forecast results.

[0054] Example 2:

[0055] Building upon Example 1, this example discloses a refined process for a multimodal coupled flood forecasting method based on a gated cross-attention mechanism, combined with... Figure 1 and Figure 2It should be noted that the detailed process disclosed in this embodiment is only a feasible solution for a multimodal coupled flood forecasting method based on a gated cross-attention mechanism. In practical applications, it can be reduced or supplemented according to objective factors such as environment and equipment.

[0056] Step 1: Obtain high-precision precipitation monitoring information, numerical forecast precipitation information, and hydrological station flood data for the predetermined watershed; perform spatiotemporal scale registration for all types of information.

[0057] Step 11: Obtain spatiotemporal data of high-precision monitored precipitation and numerically predicted precipitation in the predetermined watershed, as well as time series data of flood runoff from hydrological stations, and clarify the spatiotemporal resolution of various types of raw information;

[0058] Step 12: Determine the target time resolution for spatiotemporal and time-series data. The target time resolution for spatiotemporal data is 6 hours, and the target time resolution for time-series data is 3 hours. Accumulate or interpolate high-precision precipitation monitoring information, numerical precipitation forecast information, and hydrological station flood runoff information to achieve the target time resolution for each type of information.

[0059] Step 13: Determine the target spatial resolution of the spatiotemporal data. The target spatial resolution is 0.1°×0.1°. Unify the spatial scale of precipitation monitoring and forecast information of spatiotemporal data type to L×V, where L is the number of grids in the horizontal direction of the data and V is the number of grids in the vertical direction of the data.

[0060] Step 2: Preprocess the high-precision precipitation monitoring information, numerical precipitation forecast information and flood extract information, and pair the multi-source information according to the preset sliding window size to construct training samples, verification samples and test samples including input information and target information.

[0061] Step 21: Determine the target flood forecast period length T1, the past precipitation period length T2 and past runoff period length T3 required by the model, and the future forecast precipitation period length T4;

[0062] Step 22: Determine the sliding window size T5, based on the flood runoff information of each flood event, to determine the time available for model training. T5 is the sum of T1 and T3. The time available for model training is determined by sliding T5 through the flood runoff of each flood event. For cases where the terminal flood runoff is less than T5 time periods but greater than T5-T3 time periods, zero values ​​can be used to fill in T5 time periods.

[0063] Step 23: Based on time 1, pair precipitation information with runoff information to construct training and validation samples. To ensure the accuracy and reliability of the model, the training and validation sets used for model training only contain high-precision precipitation monitoring information and flood runoff information. Based on time 1, pair precipitation monitoring information and flood runoff information from T2×6 hours before and T4×6 hours after that time, and then randomly select 70% as training samples and 30% as validation samples.

[0064] Step 24: Based on the stated time 1, determine the time 2 for conducting the test experiment and construct a test sample set. Determine the daily rolling forecast time 3 for numerical precipitation, and select the intersection of time 1 and time 3 as time 2. Based on time 2, pair the precipitation monitoring information and runoff information T2×6 hours before this time, and the runoff information T4×6 hours after this time, with the precipitation forecast information for different forecast periods T4×6 hours after this time to construct test samples for different forecast periods.

[0065] Step 3: Construct a spatiotemporal feature encoding module based on convolutional neural networks (CNN) and Transformers to fully mine the spatiotemporal features of high-precision precipitation monitoring information and numerical precipitation forecast information.

[0066] The construction process of the spatiotemporal feature coding module includes:

[0067] The core components are a CNN and a standard Transformer encoder. First, the CNN extracts local features and compresses the dimensionality of the precipitation information. Then, the Transformer establishes the dependencies between local features to capture global temporal features. The input precipitation information for this module is B×(T2+T4)×L×V, and the output of the spatiotemporal precipitation features is B×(T2+T4)×128, where B is the batch sample size, and F is denoted as the spatiotemporal precipitation feature. img .

[0068] The CNN consists of Block 1 and Block 2, which are composed of two-dimensional convolution (Conv2D), batch normalization (Batch Norm), and activation function (ReLU); Block 3, which is composed of two-dimensional convolution (Conv2D) and pooling (Pool); and Block 4, which is composed of flattening and fully connected layers (Linear).

[0069] The Transformer encoder consists of multiple identical sub-modules stacked together. Each sub-module includes two sub-layers: a multi-head self-attention network and a position-based feedforward network. Each sub-layer also features residual connections and layer normalization. To achieve a lightweight design, a Transformer encoder with four stacked sub-modules is used, resulting in a target feature dimension of 128, 8 multi-head attention layers, and 512 hidden layers in the feedforward network.

[0070] Step 4: Construct a temporal feature encoding model based on a temporal convolutional network to fully mine the temporal features of floods at hydrological stations.

[0071] The construction process of the temporal feature encoding module includes:

[0072] The core component is an improved temporal convolutional neural network (TCN) used to fully extract past runoff temporal features. The input to this module is B×T3 flood runoff information, and the output of the flood runoff temporal features is B×128, denoted as F. ts .

[0073] The improved TCN removes the restriction of consistent input and output channels imposed by the conventional TCN residual connection mechanism by introducing one-dimensional convolution (Conv1D). It adds a batch normalization layer and an activation function layer after the convolutional layer in the conventional TCN residual block to enhance the non-linear expressive power of the residual block. The improved TCN consists of a dimension expansion layer (Unsqueeze), an initial convolutional block (Conv1D, Batch Norm, ReLU), two TCN residual blocks with d values ​​of 2 and 4 respectively (TCN_ResBlock), a feature aggregation layer, and a fully connected layer.

[0074] Step 5: Construct a multimodal feature fusion module based on a gated cross-attention mechanism to fully integrate the spatiotemporal features of precipitation mined in Step 3 with the temporal features of flood runoff mined in Step 4.

[0075] The construction process of the multimodal feature coupling module includes:

[0076] Building upon cross-attention, a gating mechanism is introduced. By concatenating image and key features and passing them through a fully connected layer and a sigmoid activation function, an adaptive balance between the original features and the attention input is achieved, preserving original feature information and mitigating gradient vanishing. To ensure successful fusion of multimodal features, F... ts Expanded from B×128 to the same F img The same dimension is B×(T2+T4)×128. The input of this module is F. img and F ts The multimodal fusion feature output by this module is denoted as F. fused Its shape is B×128.

[0077] The basic cross-attention mechanism is to extend F ts As the key and value, F img As a query, the attention weight A is then calculated, which can be expressed by the following formula:

[0078]

[0079] In the formula: F img and F ts These represent the spatiotemporal characteristics of precipitation and the temporal series characteristics of runoff, respectively; W q W k and W v The attention weights are Q, K, and V, respectively; softmax(·) is the activation function; d k K is the length of K; T is the matrix transpose.

[0080] The gating mechanism and the final multimodal feature fusion output calculation formula are as follows:

[0081]

[0082] In the formula: || represents feature concatenation, sigmoid(·) is the activation function that generates the gate value in the interval [0,1], and W g is the weight matrix of the linear layer; ⊙ represents the matrix dot product.

[0083] Step 6: Construct a flood prediction module based on a self-attention mechanism. Input the multimodal features fused in Step 5 based on a gated cross-attention mechanism into this module to perform time-series runoff prediction and obtain the final flood forecast results.

[0084] The construction process of the flood prediction module includes:

[0085] The flood prediction module based on the self-attention mechanism consists of a self-attention mechanism layer, a fully connected layer, an activation function layer, a Dropout layer, and another fully connected layer. The self-attention mechanism enhances the key information of the fused features. The input to this module is F. fused The model output is the final flood prediction result, with a shape of B×T1. fused After entering this module, it first goes through the Unsqueeze layer, expanding from B×128 to B×1×128. Then it is simultaneously used as a query, key, and value to enter the self-attention mechanism layer, followed by the Squeeze layer, which transforms it back from B×1×128 to B×128. After that, it enters the fully connected layer, activation function layer, and Dropout layer for flood prediction.

[0086] Example 3:

[0087] This embodiment discloses a multimodal coupled flood forecasting system based on a gated cross-attention mechanism, such as... Figure 3 As shown, the system includes a data acquisition unit, a sample construction unit, a spatiotemporal feature coding unit, a time-series feature coding unit, a feature fusion unit, and a flood prediction unit.

[0088] The data acquisition unit is used to acquire high-precision precipitation monitoring information, numerical forecast precipitation information, and hydrological station flood data for the predetermined watershed; and to perform spatiotemporal scale registration on various types of information.

[0089] The sample construction unit is used to preprocess the high-precision precipitation monitoring information, numerical precipitation forecast information and flood extract information, and to pair the multi-source information according to the preset sliding window size to construct training samples, verification samples and test samples including input information and target information.

[0090] The spatiotemporal feature coding unit is used to construct a spatiotemporal feature coding module based on convolutional neural networks and Transformers to fully mine the spatiotemporal features of high-precision precipitation monitoring information and numerical precipitation forecast information;

[0091] The temporal feature coding unit is used to construct a temporal feature coding model based on a temporal convolutional network to fully mine the temporal features of floods at hydrological stations.

[0092] The feature fusion unit is used to construct a multimodal feature fusion module based on a gated cross-attention mechanism, which fully integrates the spatiotemporal features of precipitation mined by the spatiotemporal feature coding unit with the temporal features of flood runoff mined by the temporal feature coding unit.

[0093] The flood prediction unit is used to construct a flood prediction module based on a self-attention mechanism. The multimodal features coupled by the feature fusion unit are input into this module to perform time-series runoff prediction and obtain the final flood forecast result.

[0094] The system can be used to execute a multimodal flood forecasting method based on a gated cross-attention mechanism disclosed in Example 1 or Example 2. The specific execution process will not be described here.

[0095] Example 4:

[0096] This embodiment takes the lower reaches of the Yangtze River as the research area and proposes a multimodal coupled flood forecasting method based on a gated cross-attention mechanism. The lower reaches of the Yangtze River have a distinct monsoon climate and are also influenced by maritime climate, with an average annual precipitation exceeding 1000 mm. Precipitation is unevenly distributed throughout the year, with the flood season (May-September) accounting for 70% of the annual precipitation. This precipitation characteristic, combined with the low-lying terrain of the lower reaches of the Yangtze River, easily exacerbates the risk of flooding. This embodiment constructs a multimodal coupled flood forecasting scenario based on a gated cross-attention mechanism in the lower reaches of the Yangtze River, using the Xinhezhuang hydrological station in the lower reaches of the Yangtze River as the main site, and conducts several flood experiments. Specifically:

[0097] S1): Multi-source information collection and spatiotemporal scale registration: High-precision precipitation monitoring information from May to September 2018 to 2022 was acquired from ground observations, GSMAPGauge V8, IMERG Final V06, and ERA5 fusion, with a temporal resolution of 1 h, a spatial resolution of 0.1°×0.1°, and a cumulative temporal resolution of 6 h. Three numerical precipitation forecasts from May to September 2018 to 2022 were also acquired from CMA (GRAPESGFS), NCEP (NCEP GEFS v12), and ECMWF (ECMWF EPS 33r2) through multi-model fusion, deep learning statistical post-processing, and joint probabilistic statistical post-processing, denoted as CNE, U-Net, and BGG, respectively, with a temporal resolution of 6 h, a spatial resolution of 0.1°×0.1°, and a forecast period of 72 h. Nine flood events were collected from Xinhezhuang Station between May and September 2018 and 2022. These events were then processed using temporal interpolation to obtain flood runoff information with a temporal resolution of 3 hours. The spatial scale of the precipitation raster data, obtained from a spatial resolution of 0.1°×0.1°, was standardized to 46×66, where 44 represents the number of horizontal raster cells and 66 represents the number of vertical raster cells.

[0098] S2): Training, Testing, and Validation Sample Construction: The target flood forecast period T1 is defined as 24, meaning a 72-hour flood forecast is targeted, with a step size of 3 hours, totaling 24 periods. The required past precipitation period T2 is defined as 4, meaning 24 hours of precipitation information is needed, with a step size of 6 hours, totaling 4 periods. The required past runoff period T3 is defined as 8, meaning 24 hours of runoff information is needed, with a step size of 3 hours, totaling 8 periods. The required future precipitation period T4 is defined as 12, meaning 72 hours of precipitation information is needed, with a step size of 6 hours, totaling 12 periods. The sliding window size T5 is defined as the sum of T1 and T3, equal to 32. A sliding window of size 32 is used to slide through 9 flood runoff data points from the Xinhezhuang hydrological station to determine the first time point suitable for model training, with a cumulative total of 1807 time points. For each determined time 1, precipitation monitoring information and flood runoff information from the 24 hours before and 72 hours after that time are paired, and then 70% are randomly selected as training samples and 30% as validation samples. Time 2 suitable for conducting test experiments is determined. First, time 3 is determined, consisting of the daily rolling forecast times of the three numerical weather prediction systems from May to September 2018 to 2022. Time 2 is the intersection of time 1 and time 3, and after filtering, the number of time 2 is 127. For each time 2, precipitation monitoring information and runoff information from the 24 hours before that time, and runoff information from the 72 hours after that time, are paired sequentially with precipitation forecast information from the three numerical weather prediction systems for different lead times in the 72 hours after that time, constructing test samples for different lead times under the three different numerical weather prediction systems to test the model's prediction accuracy.

[0099] S3): Construction of a multimodal coupled flood forecasting model based on a gated cross-attention mechanism, including a spatiotemporal feature encoding module based on convolutional neural networks and Transformers, a temporal feature encoding module based on temporal convolutional networks, a multimodal feature fusion module based on a gated cross-attention mechanism, and a flood prediction module based on a self-attention mechanism. Details are as follows:

[0100] The core components of the spatiotemporal feature encoding module based on convolutional neural networks and Transformers are a CNN and a standard Transformer encoder. First, the CNN extracts local features from precipitation information and compresses the dimensionality. Then, the Transformer establishes the dependencies between local features to capture global temporal features. The precipitation information input to this module is B×16×46×66, and the spatiotemporal feature output is B×16×128, where B is the batch sample size, and the spatiotemporal feature is denoted as F. img .

[0101] A CNN consists of Block 1 and Block 2, which are composed of 2D convolutions (Conv2D), batch normalization (Batch Normalization), and ReLU activation functions; Block 3, which is composed of 2D convolutions (Conv2D) and pooling; and Block 4, which is composed of flattening and fully connected layers (Linear). The function f of a CNN... cnn It can be summarized as follows:

[0102]

[0103] In the formula: y t The feature output extracted by the CNN; x t This is the input precipitation information.

[0104] The Transformer encoder consists of multiple identical sub-modules stacked together. Each sub-module includes two sub-layers: a multi-head self-attention network and a position-based feedforward network. Each sub-layer also features residual connections and layer normalization. To achieve a lightweight design, a Transformer encoder with four stacked sub-modules is used, resulting in a target feature dimension of 128, 8 multi-head attention layers, and 512 hidden layers in the feedforward network.

[0105] The core component of the temporal feature encoding module based on a temporal convolutional neural network (TCN) is an improved version of the TCN, used to fully extract the temporal features of past runoff. The input of flood runoff information for this module is B×8, and the output of flood runoff temporal features is B×128, denoted as F. ts .

[0106] Conventional TCNs (Transient Transformer Networks) gradually increase the receptive field of convolutional kernels by introducing a dilation factor (d) to capture long-term dependencies in sequences. Residual connections effectively alleviate the gradient vanishing problem in deep networks. The improved TCN introduces one-dimensional convolution (Conv1D) to remove the restriction of consistent input and output channels in the conventional TCN residual connection mechanism. It adds a batch normalization layer and an activation function layer after the convolutional layer in the conventional TCN residual block to enhance the non-linear expressive power of the residual block. The improved TCN consists of a dimension expansion layer (Unsqueeze), an initial convolutional block (Conv1D, BatchNorm, ReLU), two TCN residual blocks (TCN_ResBlock) with d values ​​of 2 and 4 respectively, a feature aggregation layer, and a fully connected layer, which can be represented by the following function:

[0107]

[0108] In the formula: X in and Y outThese are the flood runoff information input and flood runoff time-series feature output for the time-series feature encoding module, respectively; F TCN (·) represents the improved TCN function expression; Unsqueeze(·) is a dimension expansion layer that expands the B×8 input to B×1×8; f init (·) represents the initial convolutional layer consisting of Conv1D, Batch Norm, and ReLU; f ResBlock,d=k (·) represents a TCN residual block with a specific expansion factor (d=k); f downsample (·) represents the downsampling path of the residual connection; Mean(·) is the feature aggregation layer, which aggregates the shape of H2 from B×C×8 to B×C in the last dimension; f Linear (·) represents a fully connected layer. This is the transpose of the weight matrix of this layer. The bias term maps the aggregated features to the final output dimension B×128.

[0109] The multimodal feature fusion module based on gated cross-attention introduces a gating mechanism on top of cross-attention. By concatenating image and key features and passing them through a fully connected layer and a sigmoid activation function, it adaptively balances the original features with the attention input, preserving original feature information and mitigating gradient vanishing. To ensure successful multimodal feature fusion, F... ts Expanded from B×128 to the same F img Same dimensions B×16×128. The input to this module is F. img and F ts The multimodal fusion feature output by this module is denoted as F. fused Its shape is B×128.

[0110] The basic cross-attention mechanism is to extend F ts As the key and value, F img As a query, the attention weight A is then calculated, which can be expressed by the following formula:

[0111]

[0112] In the formula: F img and F ts These represent the spatiotemporal characteristics of precipitation and the temporal series characteristics of runoff, respectively; W q W k and W v The attention weights are Q, K, and V, respectively; softmax(·) is the activation function; d k K is the length of K; T is the matrix transpose.

[0113] The gating mechanism and the final multimodal feature fusion output calculation formula are as follows:

[0114]

[0115] In the formula: || represents feature concatenation, sigmoid(·) is the activation function that generates the gate value in the interval [0,1], and W g is the weight matrix of the linear layer; ⊙ represents the matrix dot product.

[0116] The flood prediction module based on the self-attention mechanism consists of a self-attention mechanism layer, a fully connected layer, an activation function layer, a Dropout layer, and another fully connected layer. The self-attention mechanism enhances the key information of the fused features. The input to this module is F. fused The model output is the final flood prediction result, which has a shape of B×24.

[0117] F fused After entering this module, it first passes through the Unsqueeze layer, expanding from B×128 to B×1×128. Then, it is simultaneously used as a query, key, and value in the self-attention mechanism layer, followed by the Squeeze layer, where it is transformed back from B×1×128 to B×128. Finally, it enters the fully connected layer, activation function layer, and Dropout layer for flood prediction, which can be described by the following formula:

[0118]

[0119] In the formula: Y out For the final flood forecast result; f SelfAttn (·) represents the self-attention mechanism layer; Unsqueeze(·) represents the dimension expansion layer; MHSA(·) represents the multi-head self-attention layer, set to 8 heads; Squeeze(·) represents the dimension compression layer; f ForecastHead (·) represents the final prediction layer; f Dropout (·) represents the Dropout layer, which randomly discards a portion of the information and is set to 0.3; W5 and b5 are the weight matrices and bias terms of the first fully connected layer; W6 and b6 are the weight matrices and bias terms of the second fully connected layer.

[0120] S4): Training and hyperparameter setting of a multimodal coupled flood forecasting model based on a gated cross-attention mechanism. Based on the training and validation samples constructed in S2, the optimal parameters θ of the model are obtained. * The hyperparameters Epochs are determined by the accuracy evaluation results on the validation set. The model parameters θ are the set of all trainable parameters (weight matrix and bias terms). The specific hyperparameter settings for the model are as follows: the learning rate is 10. -4The batch size is 16; the number of training epochs is set to [1100, 1200, 1300, 1400, 1500]. The model training process is based on defining an objective function (loss function) and selecting an optimization algorithm (trainer) to adjust the model's θ to minimize the value of the objective function. The loss function is set to mean squared error (MSE), and the trainer is set to Adam. The training formulas for both are as follows:

[0121]

[0122] In the formula: L(θ) is the MSE loss function; N is the batch sample size; T is the length of the predicted sequence; Let be the true observed runoff of the i-th sample at time t; Let F be the predicted runoff value of the i-th sample at time t; θ is the set of all trainable parameters (weights and biases) of the model; θ The function representing the entire deep learning model; X (i) The complete input for the i-th sample includes temporal and spatial data of runoff and precipitation; argmin θ The Adam learner iteratively updates the parameter θ through gradient descent and adaptive moment estimation to minimize L(θ).

[0123] S5): Accuracy Evaluation. For flood forecast results, the correlation coefficient (CC), root mean square error (RMSE), Nash efficiency coefficient (NSE), and improved Kling-Gupta efficiency coefficient (KGE') are used as evaluation indicators. The formulas for each indicator are as follows:

[0124]

[0125] In the formula: N is the number of moments in the flood process, and S and O represent the runoff predicted by the model and observed by the hydrological station, respectively. and The score represents the average value, CV S and CV O Q represents the coefficients of variation for S and O, respectively. obs (i) represents the measured flow rate at time i; Q pre (i) represents the predicted flow rate at time i; This is the average of the measured flow rates, with the unit being m³. 3 / s.

[0126] Figure 4Box plots of accuracy indices for flood forecasting models composed of different multimodal fusion methods at the Xinhezhuang station are presented. The multimodal coupling method based on a gated cross-attention mechanism used in this invention is denoted as TCTGC. The multimodal fusion methods compared are cross-attention fusion, hierarchical fusion, and residual fusion, denoted as TCTC, TCTH, and TCTR, respectively. As shown in the figure, TCTC performs the worst among the four fusion methods, with all its indices being the worst at different epochs. Secondly, TCTH and TCTR show improvements compared to TCTC, with TCTR performing slightly better. Overall, although TCTGC's accuracy indices is lower than TCTR in the early epochs (1100, 1200), when the epoch exceeds 1200, the median and 25%–75% quantile distribution of various indices of TCTGC are superior to the other three fusion methods. Therefore, the multimodal coupling method based on a gated cross-attention mechanism is more advantageous in exploring the interdependence between the spatiotemporal characteristics of precipitation and the temporal characteristics of runoff, and is more conducive to improving the accuracy of flood forecasting. When the epoch is 1500, the median values ​​of CC, RMSE, NSE, and KGE' for the four models on the training and validation sets are greater than 0.950 and less than 18m, respectively. 3 The results were generally good, with values ​​of 0.820 and 0.840, so the hyperparameter Epoch was set to 1500.

[0127] Figure 5 The scatter plots of the proposed multimodal coupled flood forecasting model (TCTGC) based on a gated cross-attention mechanism at Xinhezhuang station are presented at Epoch 1500 for both the training and validation sets. As shown in the figure, the TCTGC model at Xinhezhuang station effectively predicts the flood conditions for the next 72 hours in both the training and validation sets. The pairs of measured and predicted values ​​are evenly distributed along the 45° diagonal, and CC, NSE, and KGE' are all above 0.970, indicating a high consistency between the simulation results of the TCTGC model and the measured flood. Therefore, the multimodal coupled flood forecasting model based on a gated cross-attention mechanism constructed in this invention can effectively mine the characteristic information of one-dimensional historical runoff and three-dimensional precipitation, and achieve efficient coupling of multimodal characteristic information through a multimodal fusion mechanism, deeply exploring the spatiotemporal dependency between the two.

[0128] Figure 6The paper presents process diagrams of the mean and 90% confidence interval of typical flood events at Xinhezhuang Station in June 2022, driven by observed precipitation and three numerical precipitation forecasts (CNE, BGG, and U-Net) using the TCTGC model. Overall, the flood process diagrams for typical events driven by different numerical precipitation forecasts capture the rise and fall of the observed flood events, indicating that the multimodal flood forecasting method based on the gated cross-attention mechanism proposed in this invention can effectively combine precipitation monitoring and numerical precipitation forecasting information to conduct flood forecasting experiments, demonstrating significant application value. Furthermore, the process diagrams show some differences in flood forecast results driven by different precipitation information. CNE and U-Net generally perform well in predicting the flood peak time, but both show deviations in peak prediction at some rolling forecast times. BGG, on the other hand, failed to accurately predict the peak time and exhibited significant overestimation during the receding phase. This demonstrates that the quality of numerical precipitation forecasting information has a significant impact on the flood forecasting model proposed in this invention.

[0129] Figure 7 The accuracy indices of flood forecasts for the Xinhezhuang station under different lead times are presented, driven by observed precipitation and three numerical precipitation forecasts (CNE, BGG, and U-Net) to drive the TCTGC model. Overall, the multimodal coupled flood forecasting method based on a gated cross-attention mechanism proposed in this invention can be effectively applied to practical flood forecasting operations. The correlation coefficient (CC) is greater than 0.65 under different forecast information, and greater than 0.75 for most lead times. The constructed TCTGC model exhibits performance differences under different lead times, with the overall performance in the first half of the lead time being better than that in the second half. The overall performance is consistent with... Figure 6 Consistently, the quality of numerical precipitation forecast information determines the actual application effect of the model. It can be seen that the accuracy of numerical precipitation forecasts (BGG and U-Net) after statistical post-processing is better than that of CNE in most cases. The efficiency coefficient NSE improves from negative to about 0.500 in the last 36-hour forecast period, with the highest improvement reaching 3.170. The correlation coefficient CC is improved in all forecast periods, and CC exceeds 0.850 in more than half of the forecast periods.

[0130] The technical process of the flood forecasting method disclosed in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or other arbitrary combinations. When implemented using software, the above embodiments can be implemented in whole or in part as a computer program product. A computer program product includes one or more computer instructions or computer programs.

[0131] When computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives (SSDs).

[0132] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0133] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A multimodal coupled flood forecasting method based on a gated cross-attention mechanism, characterized in that, Includes the following steps: Acquire high-precision precipitation monitoring information, numerical forecast precipitation information, and hydrological station flood runoff information for the predetermined watershed to form multi-source information; The multi-source information is sample-paired according to a preset sliding window size to construct training samples, verification samples, and test samples, including input information and target information. A multimodal coupled flood forecasting model is constructed, consisting of a spatiotemporal feature encoding module, a time-series feature encoding module, a multimodal feature fusion module, and a flood prediction module, wherein: The spatiotemporal feature encoding module is built based on a convolutional neural network and a Transformer encoder, and is used to extract spatiotemporal features of precipitation from the high-precision precipitation monitoring information and numerical precipitation forecast information. The temporal feature encoding module is constructed based on a temporal convolutional network and is used to extract temporal features of flood runoff from the flood runoff information of the hydrological station. The multimodal feature fusion module is built based on a gated cross-attention mechanism, which is used to fully fuse the spatiotemporal features of precipitation and the temporal features of flood runoff obtained from mining to obtain multimodal features; The flood prediction module is built based on a self-attention mechanism and is used to input the fused multimodal features into the flood prediction module to perform time-series runoff prediction and obtain the final flood forecast result.

2. The multimodal coupled flood forecasting method based on gated cross-attention mechanism according to claim 1, characterized in that, The high-precision precipitation monitoring information, numerical forecast precipitation information, and hydrological station flood runoff information are accumulated or interpolated to achieve the target time scale. The spatial scale of the high-precision precipitation monitoring information and numerical precipitation forecast information is unified as L×V, where L is the number of grids in the horizontal direction of the data and V is the number of grids in the vertical direction of the data.

3. The multimodal coupled flood forecasting method based on gated cross-attention mechanism according to claim 1, characterized in that, Before performing sample pairing on the multi-source information according to a preset sliding window size, the process also includes: Define the target flood forecast period length T1, the past precipitation period length T2 and past runoff period length T3 required for the multimodal coupled flood forecast model, and the future forecast precipitation period length T4; Define the sliding window size T5, where T5 = T1 + T3. Use T5 to slide through the flood runoff of each flood event to determine the time 1 that can be used for training the multimodal coupled flood forecasting model. For cases where the end flood runoff is less than T5 time periods but greater than T5-T3 time periods, it is allowed to fill the T5 time periods with zero values.

4. The multimodal coupled flood forecasting method based on gated cross-attention mechanism according to claim 3, characterized in that, The multi-source information is sample-paired according to a preset sliding window size to construct training samples, validation samples, and test samples, including input information and target information, specifically including: Based on time 1, high-precision precipitation monitoring information and hydrological station flood runoff information are paired to construct training samples and validation samples. Based on time 1, high-precision precipitation monitoring information before time T2×6 h and after time T4×6 h are paired with hydrological station flood runoff information, and then 70% are randomly selected as training samples and 30% as validation samples. Based on time 1, determine time 2 for conducting the test experiment and construct test samples; determine the daily rolling forecast time 3 for numerical precipitation, and select the intersection of time 1 and time 3 as time 2; based on time 2, pair the precipitation monitoring information and runoff information T2×6 h before time 2, and the runoff information T4×6 h after time 4 with the precipitation forecast information for different forecast periods T4×6 h after time 4, and construct test samples for different forecast periods.

5. The multimodal coupled flood forecasting method based on gated cross-attention mechanism according to claim 3, characterized in that, The construction process of the spatiotemporal feature encoding module specifically includes: A convolutional neural network (CNN) is used to extract local features and compress dimensionality from high-precision precipitation monitoring information. Then, a Transformer encoder is used to establish the dependencies between local features and capture global temporal features. The input to the high-precision precipitation monitoring information is B×(T2+T4)×L×V, and the spatiotemporal features of precipitation are F. img The output is B×(T2+T4)×128, where B is the batch sample size.

6. The multimodal coupled flood forecasting method based on gated cross-attention mechanism according to claim 5, characterized in that, The flood runoff information input for the time-series feature encoding module is B×T3, and the flood runoff time-series feature F ts The output is B×128.

7. The multimodal coupled flood forecasting method based on gated cross-attention mechanism according to claim 6, characterized in that, The multimodal feature fusion module is built based on a gated cross-attention mechanism to fully fuse the mined spatiotemporal features of precipitation and the temporal features of flood runoff, and to integrate the flood runoff temporal features F ts The B×128 extension is the same as the described spatiotemporal precipitation feature F. img With the same dimension B×(T2+T4)×128, the output multimodal fusion feature F is obtained. fused Its shape is B×128.

8. The multimodal coupled flood forecasting method based on gated cross-attention mechanism according to claim 7, characterized in that, The flood prediction module based on the self-attention mechanism consists of a self-attention mechanism layer, a fully connected layer, an activation function layer, a Dropout layer, and another fully connected layer. The self-attention mechanism enhances key information from the fused features. The input to this flood prediction module is the multimodal fused feature F. fused The final flood forecast result is output, which is in the shape of B×T1.

9. A multimodal coupled flood forecasting system, used to execute the multimodal coupled flood forecasting method based on a gated cross-attention mechanism as described in any one of claims 1 to 8, characterized in that, The system includes: The data acquisition unit is used to acquire high-precision precipitation monitoring information, numerical forecast precipitation information, and hydrological station flood runoff information for the predetermined watershed, forming multi-source information. The sample construction unit is used to perform sample pairing on the multi-source information according to a preset sliding window size, and construct training samples, verification samples and test samples including input information and target information. The spatiotemporal feature encoding module, built on a convolutional neural network and a Transformer encoder, is used to extract spatiotemporal features of precipitation from the high-precision precipitation monitoring information and numerical precipitation forecast information. The temporal feature encoding module, constructed based on a temporal convolutional network, is used to extract temporal features of flood runoff from the flood runoff information of the hydrological stations. The multimodal feature fusion module, built on a gated cross-attention mechanism, is used to fully fuse the spatiotemporal features of precipitation and the temporal features of flood runoff obtained from mining, to obtain multimodal features; The flood forecasting module is built based on a self-attention mechanism. The multimodal features obtained by fusion are input into the flood forecasting module to perform time-series runoff prediction and obtain the final flood forecast result.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on an electronic device, causes the electronic device to perform the multimodal coupled flood forecasting method based on a gated cross-attention mechanism as described in any one of claims 1 to 8.

Images