Drought disaster prediction method based on ABST-Net space-time prediction model

Abstract

The invention provides a drought disaster prediction method based on an ABST-Net space-time prediction model, and belongs to the technical field of deep learning. According to the method, a module used for enhancing the space-time feature extraction capability of a model is designed, and according to the characteristics of space-time prediction, the module extracts space-time features from three different dimensions of time, channel and space by using time attention, channel attention and space attention mechanisms; the spatial-temporal features can be captured more comprehensively and accurately by adopting a multi-dimensional attention mechanism, and the accuracy of feature extraction is enhanced; meanwhile, an ABST-Net model suitable for spatio-temporal prediction is designed based on the module, so that spatio-temporal prediction of the model on drought disasters is more accurate.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and specifically relates to a method for drought disaster prediction based on an ABST-Net spatiotemporal prediction model. Background Art

[0002] Drought disasters are a common challenge faced by countries around the world. Drought disasters have the characteristics of long time span and slow change trend, so it is quite difficult to carry out detailed long-term monitoring and accurate future prediction. It is even more difficult to predict drought disasters in a large area. It is necessary not only to predict the development trend of drought disasters in time, but also to predict the spatial distribution characteristics of drought disasters in space. Soil moisture can reflect the severity of drought disasters, and meteorological data such as surface temperature and rainfall are the main factors leading to drought disasters. Therefore, predicting soil moisture and meteorological data in arid areas can effectively reflect the development trend of drought disasters.

[0003] Most existing technologies use spatiotemporal prediction models to predict drought disasters. A spatiotemporal prediction model is a model used to predict data with spatiotemporal characteristics. The model can understand and analyze the complex relationship between data in the time and space dimensions. Spatiotemporal prediction models are widely used in various fields, such as meteorology, environmental science, traffic flow management, public safety, economic forecasting, etc., aiming to predict future events or states by analyzing past and current spatiotemporal data. Spatiotemporal prediction models based on deep learning refer to the use of deep learning methods to predict future states by learning the spatial and temporal dependencies of historical data. It has been a popular research field in recent years.

[0004] In recent years, the main spatiotemporal prediction methods are divided into four categories, namely RNN-RNN-RNN, CNN-RNN-CNN, CNN-ViT-CNN and CNN-CNN-CNN. Among them, CNN-CNN-CNN has attracted widespread attention due to its advantages such as easy optimization of structure, flexible model design, and one-time prediction. Liu et al. combined the convolutional GRU model and the spatial-temporal neural network (STNN) of the 3D convolutional neural network, and used the variational Bayesian method to predict the wind speed probabilistically, demonstrating the ability of the model to make effective predictions at the temporal and spatial levels. However, the model combines the convolutional GRU and the 3D convolutional neural network, which increases the complexity of the model. The 3D convolutional neural network has a large amount of computation. When processing long time series and large-scale spatial data, the computational cost of training and inference is high. The DVF model learns voxel flow through the CNN autoencoder and reconstructs future frames by borrowing voxels from nearby video frames, but this may cause errors to accumulate in multiple time steps, especially in long-term predictions. Gao et al. proposed a simple spatiotemporal prediction model called SimVP, which is built entirely on convolutional networks, does not use recursive architecture, and can achieve excellent performance on various benchmark datasets without introducing any additional techniques and strategies. Although convolutional networks are good at capturing local features, they perform poorly when dealing with long-term dependencies and global features. The ST-ResNet model captures the flow of people within a city by stacking multiple residual networks. The model uses CNN to extract spatial features and uses a multi-layer structure to capture temporal dependencies, but the performance of the model is too dependent on the quality and spatial resolution of the input data. Low-quality data may lead to inaccurate predictions.

[0005] To sum up, how to optimize the CNN-CNN-CNN spatiotemporal prediction model so that it can be used to predict drought disasters has become the key point of the research. Summary of the invention

[0006] In view of the problems existing in the background technology, the purpose of the present invention is to provide a method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model. This method designs a module for enhancing the spatiotemporal feature extraction capability of the model. According to the characteristics of spatiotemporal prediction, the module uses time attention, channel attention and space attention mechanisms to extract spatiotemporal features from three different dimensions of time, channel and space respectively. The use of a multi-dimensional attention mechanism can capture spatiotemporal features more comprehensively and accurately, and enhance the accuracy of feature extraction; at the same time, based on this module, a deep learning model Attention-Based Spatio-temporal Prediction Network (ABST-Net) suitable for spatiotemporal prediction is designed, so that the model can make more accurate spatiotemporal predictions of drought disasters.

[0007] To achieve the above object, the technical solution of the present invention is as follows:

[0008] A method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model comprises the following steps:

[0009] Step 1. Obtain meteorological data and soil moisture data of a certain area in a certain period of time based on remote sensing satellites, and preprocess the data:

[0010] Step 1.1. Based on the correlation coefficient calculation formula, calculate the average value of the correlation coefficient between each variable of the meteorological data and soil moisture during the period, arrange the average values ​​from large to small, and select the meteorological data variables corresponding to the first five average values, that is, the selected meteorological data variables have high sensitivity to soil moisture;

[0011] Step 1.2. Merge the five meteorological data variables selected in step 1.1 with the soil moisture data to obtain a multi-band image;

[0012] Step 2. Convert the data format. The specific process is as follows:

[0013] The multi-band image obtained in step 1 is converted into format data, and the longitude and latitude information in the format data is removed, only the pixel value information is retained, and the spatiotemporal data X with dimensions of (T, C, H, W) is constructed in chronological order, where T is the time dimension, that is, the moment to which the current data belongs; C is the channel dimension, that is, which attribute the current data is, and the attribute in this data is soil moisture or a certain type of meteorological data variable; H represents the height of the data, and W represents the width of the data, both of which are determined by the spatial resolution of the data, indicating the spatial location of the current data;

[0014] Step 3. Construct an attention-based spatio-temporal prediction network model (Attention-Based Spatio-temporal Prediction Network, ABST-Net), wherein the attention-based spatio-temporal prediction network model includes an encoder, a spatio-temporal attention module, and a decoder;

[0015] The encoder is used to convert high-dimensional spatiotemporal data X into low-dimensional hidden features with richer semantic information; the spatiotemporal attention module takes the low-dimensional hidden features as input and uses multiple attention mechanisms to extract spatiotemporal features from three different dimensions: time, channel, and space; the decoder is used to map the spatiotemporal features extracted from different dimensions back to the high-dimensional space, thereby obtaining a prediction result;

[0016] Step 4. Divide the spatiotemporal data X obtained in step 2 into a training set and a test set, use the data in the training set as input, and use the data in the test set as labels to train the model of the attention spatiotemporal prediction network;

[0017] Step 5. Use the trained spatiotemporal prediction network model to predict drought disasters.

[0018] Furthermore, the meteorological data variables in step 1 include air temperature at a certain height, total precipitation, ground pressure, wind speed U component, and wind speed V component.

[0019] Furthermore, in step 1, the calculation formula of the correlation coefficient is as follows:

[0020]

[0021] Among them, ρ P,Q is the correlation coefficient between variables P and Q, Cov(P,Q) is the covariance of P and Q, σ P and σ Q are the standard deviations of P and Q respectively.

[0022] Furthermore, in step 2, the height and width of the data are both determined by the spatial resolution of the data, indicating the spatial position of the current data.

[0023] Further, in step 3, the encoder includes 3 2D convolutional layers, 1 deformable convolutional layer, a normalized layer and a SiLU activation function layer; wherein the first 2D convolutional layer, the second 2D convolutional layer, the third 2D convolutional layer and the deformable convolutional layer are cascaded, the output of the first 2D convolutional layer is simultaneously used as the input of the second 2D convolutional layer and the first normalized layer, the output of the second 2D convolutional layer is simultaneously used as the input of the third 2D convolutional layer and the first normalized layer, the output of the third 2D convolutional layer is used as the input of the first normalized layer, the output of the first normalized layer is used as the input of the first SiLU activation function layer, and the first SiLU activation function layer outputs a feature map;

[0024] The Deformable Convolution Networks (DCNs) are obtained by adding offsets to the grid sampling positions in the standard convolution;

[0025] The formula of the first SiLU activation function is:

[0026]

[0027] Furthermore, for a certain position P0 of the input feature map x, a convolution kernel of size 3×3 is used, and the sampling range is R∈{(-1,-1),(-1,0),...,(0,1),(1,1)}, where (-1,-1) represents the lower left corner of the position P0 and (1,1) represents the upper right corner of the position P0. The output of the deformable convolution at this position is shown in formula (3).

[0028]

[0029] Among them, P n is the offset of the nth position of the input feature map, R is the adjustment parameter, ΔP n is the offset of a certain position; w(P n ) represents the convolution weight coefficient of the position, indicating the importance of this position on the feature map. This coefficient will be updated through back propagation during training; Δm k It is an adjustment parameter with a value of [0,1], which is used to distinguish whether the current area is an area of ​​interest. If it is an area of ​​interest, it is set to 1, otherwise it is set to 0.

[0030] Further, in step 3, the spatiotemporal attention module includes a temporal attention unit, a channel attention unit and a spatial attention unit;

[0031] The temporal attention unit includes a deformable convolution layer and a depthwise separable convolution layer, wherein the depthwise separable convolution layer is composed of a depthwise convolution (DW Conv) and a pointwise convolution; the depthwise separable convolution layer applies an independent convolution kernel to each time channel, thereby realizing independent convolution between channels, capturing local spatiotemporal patterns in time series data, and helping the model learn the time series correlation in input data; the pointwise convolution uses a 1×1 convolution kernel to linearly combine the output of the depthwise convolution, adjust the depth and dimension of the feature map, and obtain the output of the depthwise separable convolution;

[0032] The formula of temporal attention TA is shown in formula (4):

[0033] TA=Conv 1×1 (DW Conv(DCNs(X))) (4)

[0034] The channel attention unit includes a global average pooling, a global maximum pooling layer, a multilayer perceptron (MLP) and a Sigmoid function layer; the feature maps output from the encoder are respectively input into the global average pooling layer and the global maximum pooling layer to capture the channel-level information of the input feature map; the multilayer perceptron with shared weights calculates the complex nonlinear dependencies between channels based on the output results of the global average pooling and the global maximum pooling layers, and then adds the output results to achieve feature information fusion, and inputs the fusion results into the Sigmoid function layer, and the Sigmoid function layer is activated to generate a channel attention map;

[0035] The calculation formula of channel attention CA is shown in formula (5).

[0036] CA=σ(MLP(AvgPool(X))+MLP(MaxPool(X))) (5)

[0037] Among them, AvgPool and MaxPool represent average pooling and maximum pooling respectively, MLP represents shared fully connected layer (multi-layer perceptron with shared weights), and σ represents Sigmoid activation function;

[0038] The spatial attention unit includes a Flatten layer, a Transpose layer, a second normalization layer (Norm) and a Transformer Encoder layer cascaded in sequence; the Transformer Encoder layer is composed of a Multi-Head Self-Attention Mechanism (MSA), a Feed-Forward Neural Network (FFNN), and a Multi-Layer Perceptron Head (MLP Head);

[0039] The Flatten layer is used to flatten the input data according to width and height, and the data dimension changes from (B, T, C, H, W) to (B, T, C, H×W). The Transpose layer exchanges the flattened spatial dimension (H×W) with the channel dimension C, and the data dimension becomes (B, T, H×W, C). The Norm layer is used to implement the image block (Patches). At the same time, the Patches are arranged in sequence, and multiple channel information is still retained at each spatial position, so that each spatial position can perform self-attention calculation independently. MSA is used for the encoder to learn the spatial position between patches, thereby capturing long-distance feature associations. FFNN maps the input data to the output space through a series of nonlinear transformations, thereby realizing complex pattern recognition and feature extraction of the input data, and is used to further process the feature representation of each position after the self-attention mechanism. The spatial attention is used as the output layer of the neural network model to map the model from the intermediate representation space to the output space, so MLP is used as the classification head to obtain the final output.

[0040] The temporal attention map is multiplied with the original input to achieve adaptive weighting of feature channels, enhancing the model's ability to capture dynamic information between frames. The output results of temporal attention, channel attention, and spatial attention are fused as the output of the spatiotemporal attention module to fuse features at different levels.

[0041] The fusion formula is shown in formula (6).

[0042] Y=(TA(X)⊙CA(X))⊙X+SA(X) (6)

[0043] Where SA represents spatial attention, ⊙ represents Hadamard product, and Y is the final output feature map.

[0044] Furthermore, the loss function based on the ABST-Net spatiotemporal prediction model uses the spatially normalized mean square error (MSE). The spatially normalized MSE takes the spatial scale into account in the error calculation process to reduce the impact of the spatial scale change in the image on the model training. The calculation formula of the spatially normalized MSE is shown in formula (7):

[0045]

[0046] Among them, pred is the predicted value, true is the true value, channel, height, and width are the number of channels, height, and width of the spatiotemporal data X respectively.

[0047] Furthermore, the structure of the decoder is similar to that of the encoder, except that a pixel rearrangement layer is added before the first standard normalization layer. The pixel rearrangement layer is used to perform pixel shuffle on the convolution operation results of each layer to achieve upsampling of hidden features and gradually reconstruct the model output.

[0048] Furthermore, during training in step 4, the initial learning rate (lr) can be set between 0.001 and 0.1, the learning rate decay factor (lr_k_decay) can be set between 0.1 and 1.0, the learning rate in the warm-up stage (warmup_lr) can be set to 1 / 10 or 1 / 100 of the initial learning rate, and the learning rate scheduling strategy uses the cosine annealing learning rate scheduler.

[0049] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0050] 1. The present invention proposes a deep learning model Attention-Based Spatio-temporal Prediction Network (ABST-Net) suitable for spatiotemporal prediction, wherein Attention-Based means that the model uses three attention mechanisms, namely temporal attention, channel attention and spatial attention, to enhance the model's ability to extract different spatiotemporal features in the data; at the same time, the temporal attention of the Spatio-temporal Attention Module is implemented using deformable convolution and depthwise separable convolution, so that the model can better capture the relationship between temporal features and channel features and the detailed information in the feature map.

[0051] 2. The present invention selects the Inner Mongolia Autonomous Region as the research area. Based on the long-term, high-spatial-resolution, and high-precision soil moisture data of the Inner Mongolia Autonomous Region and the ERA 5 meteorological data of the same time using GEE (Google Earth Engine), the single variable (soil moisture) prediction and multivariate (soil moisture + 5 meteorological variables) prediction experiments are carried out respectively, and similar spatiotemporal prediction models (such as ConvLSTM, SimVP, PredRNN, etc.) are selected for comparison. The results of the comparative experiments also prove that the ABST-Net spatiotemporal prediction model proposed in the present invention has excellent performance in spatiotemporal prediction tasks and can well capture the development and change trends of drought disasters. In the field of drought disaster prevention and control, the method of the present invention can provide researchers with more powerful means in predicting the degree of drought in future regions and long-term monitoring of drought areas, and plays an important role in subsequent research. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is the overall structure diagram of the ABST-Net spatiotemporal prediction model of the present invention.

[0053] Figure 2 It is a structural diagram of the spatio-temporal attention module (Spatio-temporalAttention Module) in the ABST-Net spatio-temporal prediction model of the present invention.

[0054] Figure 3 This is a visualization result diagram of soil moisture for single variable spatiotemporal prediction in Example 1 of the present invention.

[0055] Figure 4 This is a visualization result diagram of soil moisture for multivariate spatiotemporal prediction in Example 2 of the present invention.

[0056] Figure 5 This is a visualization result diagram of the surface temperature of Example 2 of the present invention and the multivariate spatiotemporal prediction of the prior art. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the implementation modes and the accompanying drawings.

[0058] A method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model comprises the following steps:

[0059] Step 1. Based on remote sensing satellites, the SMAP L4 soil moisture data and ERA5 meteorological data of Inner Mongolia Autonomous Region in 2018 are obtained and preprocessed. The meteorological data variables include air temperature at a certain height, total precipitation, ground pressure, wind speed U component and wind speed V component:

[0060] Step 1.1. Calculate the average value of the correlation coefficient between each variable of meteorological data and soil moisture during this period, arrange the average values ​​from large to small, and select the meteorological data variables corresponding to the first 5 average values, that is, the selected meteorological data variables are more sensitive to soil moisture. The formula for calculating the correlation coefficient is as follows:

[0061]

[0062] Among them, ρ P,Q is the correlation coefficient between variables P and Q, Cov(P,Q) is the covariance of P and Q, σ P and σ Q are the standard deviations of P and Q respectively;

[0063] The correlations between each candidate variable (downscaling factor) and the SMAP L4 soil moisture product are shown in Tables 1 and 2.

[0064] Table 1 Correlation between candidate variables and SMAP L4 soil moisture products (first half of 2018)

[0065]

[0066] Table 2 Correlation between candidate variables and SMAP L4 soil moisture products (second half of 2018)

[0067]

[0068] The average values ​​of the correlation coefficients of various variables with soil moisture throughout the year were calculated. The average values ​​of VV (vertical polarization), VH (horizontal polarization), incident angle, LST (land surface temperature), DEM (digital elevation model), NDVI (normalized difference vegetation index), EVI (enhanced vegetation index), NDWI (normalized water index), SAVI (soil adjusted vegetation index), MSAVI (modified soil adjusted vegetation index), and LAI (leaf area index) were 0.4509, 0.5003, 0.0003, -0.7701, -0.3845, 0.3706, 0.5570, -0.1442, 0.3298, 0.3208, and 0.3598, respectively. Under the condition of ensuring strong correlation and stability, VV band, VH band, LST, DEM, and EVI were selected as variables with high correlation.

[0069] Step 1.2. Use GEE to merge the five meteorological data variables selected in step 1.1 with the soil moisture data to obtain a multi-band image with soil moisture, air temperature at 2 meters, total precipitation, ground pressure, 10-meter wind speed U component, and 10-meter wind speed V component;

[0070] Step 2. Convert the data format. The specific process is as follows:

[0071] Use the xarray library to convert the multi-band tif image obtained in step 1 into the NetCDF-4 (Network Common Data Form version 4) format suitable for deep learning research, remove the longitude and latitude information, retain only the pixel value information, and construct the spatiotemporal data X with dimensions (T, C, H, W) in chronological order, where T is the time dimension, that is, the moment to which the current data belongs; C is the channel dimension, that is, which attribute the current data is, and the attribute in this data is soil moisture or a certain type of meteorological data variable; H represents the height of the data, and W represents the width of the data, both of which are determined by the spatial resolution of the data, indicating the spatial location of the current data;

[0072] Step 3. Construct an attention-based spatio-temporal prediction network model (ABST-Net). The attention-based spatio-temporal prediction network model includes an encoder, a spatio-temporal attention module and a decoder. The structure diagram of the model is as follows: Figure 1 As shown;

[0073] The encoder is used to convert high-dimensional spatiotemporal data into low-dimensional hidden features with richer semantic information. The specific principle is as follows: Assume that the input data Where B is the batch size. In the Encoder-Decoder structure, the present invention reshapes the temporal input data B×T×C×H×W into (B×T)×C×H×W so that it only considers the spatial correlation in the data. In the spatio-temporal attention module, the input data B×T×C×H×W is reshaped into B×(T×C)×H×W so that the time frames are arranged in the order of the channel dimension.

[0074] The encoder includes three cascaded 2D convolutional layers and one deformable convolutional layer, as well as a normalization layer and a first SiLU activation function layer; in each convolutional layer, the output of the convolution operation will be normalized and SiLU activated; by cascading multiple convolutional layers, the model can extract the features of the input data layer by layer and gradually expand the receptive field of the convolution kernel; different convolutional layers can capture details in different aspects, thereby gradually building complex hidden features; by standardizing the output of the convolution, it helps to speed up the convergence speed and ensure the stability of each layer in the network; the first SiLU activation function has an adaptive gating mechanism and a smooth gradient characteristic, and the adaptive gating mechanism allows the network to automatically adjust the activation level according to the input, thereby effectively capturing complex patterns in the data; the smooth gradient characteristic helps to avoid the gradient vanishing problem and improve the training stability and efficiency of the network, so using the SiLU activation function after normalization helps the network learn more complex nonlinear features in the data;

[0075] The formula of the first SiLU activation function is:

[0076]

[0077] In the spatiotemporal prediction task of the present invention, high-resolution soil moisture data and meteorological data have rich detailed information, in which the characteristics of some areas have complex deformations, and 2D convolution is limited by the size and shape of its convolution kernel and cannot accurately capture the deformation characteristics. Therefore, the present invention additionally introduces a deformable convolution layer (Deformable Convolution Networks, DCNs) in the Encoder; DCNs introduces the concept of offset, and adds the offset to the grid sampling position in the standard convolution, so that the sampling position of the standard convolution is deformed and more concentrated in the area of ​​interest, so that the sampling points of the convolution kernel will change their positions according to the shape of the input image, forming the ability of adaptive changes, and having better geometric invariance;

[0078] For a certain position P0 of the input feature map x, a convolution kernel of size 3×3 is used, and the sampling range is R∈{(-1,-1),(-1,0),...,(0,1),(1,1)}, where (-1,-1) represents the lower left corner of the position P0 and (1,1) represents the upper right corner of the position P0. The output of the deformable convolution at this position is shown in formula (3).

[0079]

[0080] Among them, P n is the offset of the nth position of the input feature map, R is the adjustment parameter, ΔP n is the offset of a certain position; w(Pn ) represents the convolution weight coefficient of the position, indicating the importance of this position on the feature map. This coefficient will be updated through back propagation during training; Δm k It is an adjustment parameter with a value of [0,1], which is used to distinguish whether the current area is an area of ​​interest. If it is an area of ​​interest, it is set to 1, otherwise it is set to 0;

[0081] The spatiotemporal attention module takes low-dimensional hidden features as input and uses a variety of attention mechanisms to extract spatiotemporal features from three different dimensions: time, channel, and space. Its structural diagram is shown in Figure 2 As shown, the spatiotemporal attention module includes a temporal attention unit, a channel attention unit and a spatial attention unit;

[0082] The temporal attention unit includes a deformable convolution layer and a depthwise separable convolution layer (DepthwiseSeparable Conv olutions), so that the model can better capture the relationship between temporal features and channel features and the detail information in the feature map; the deformable convolution is consistent with that in the encoder and is used to extract complex detail features within the frame; the depthwise separable convolution layer consists of a depthwise convolution (DW Conv) and a pointwise convolution; DW Conv applies an independent convolution kernel to each time channel to achieve independent convolution between channels, capture local spatiotemporal patterns in time series data, and help the model learn the time series correlation in the input data; the pointwise convolution uses a 1×1 convolution kernel to linearly combine the output of the depthwise convolution, adjust the depth and dimension of the feature map, and obtain the output of the depthwise separable convolution; assuming X∈R B×(T×C)×H×W is the feature map output by the Encoder, then the formula of temporal attention TA is shown in formula (4):

[0083] TA=Conv 1×1 (DW Conv(DCNs(X))) (4)

[0084] Because the present invention uses multiple variables involved in spatiotemporal data, it is necessary to deepen the model's ability to extract features from different channels; the channel attention unit includes global average pooling, global maximum pooling layer, multilayer perceptron (Multilayer Per ceptron, MLP) and Sigmoid function layer; the feature map output from the encoder is respectively input into the global average pooling layer and the global maximum pooling layer to capture the channel-level information of the input feature map, and then the output results of the two pools are sent to a multilayer perceptron (Multilayer Perceptron, MLP) with shared weights to learn the complex nonlinear dependencies between channels; then the output results of the MLP are added to achieve feature information fusion; the channel attention map is generated by activating the Sigmoid function to represent the weights of each channel; finally, the attention map is multiplied by the original input to achieve adaptive weighting of the feature channel, thereby enhancing the model's ability to capture dynamic information between frames;

[0085] The calculation formula of channel attention CA is shown in formula (5);

[0086] CA=σ(MLP(AvgPool(X))+MLP(MaxPool(X))) (5)

[0087] Among them, AvgPool and MaxPool represent average pooling and maximum pooling respectively, MLP represents shared fully connected layer, and σ represents Sigmoid activation function;

[0088] The spatial attention unit includes a Flatten layer, a Transpose layer, a second normalization layer (Norm) and a Transformer Encoder layer cascaded in sequence; the Transformer Encoder layer is composed of a Multi-Head Self-Attention mechanism (MSA), a Feed-Forward Neural Network (FFNN), a residual connection and a normalization layer;

[0089] Spatial attention first uses the Flatten operation to flatten the data according to width and height, and the data dimension changes from (B, T, C, H, W) to (B, T, C, H×W); then Transpose is used to exchange the flattened spatial dimension (H×W) with the channel dimension C, and the data dimension becomes (B, T, H×W, C); the Norm layer is used to implement the image block (Patches), and the Patches are arranged in sequence. Multiple channel information is still retained at each spatial position, so that each spatial position can perform self-attention calculation independently; then, Patches are sent to the Transformer Encoder, which consists of a multi-head self-attention mechanism (Multi-Head Self-Attention, MSA), a feed-forward neural network (Feed-Forwa rd Neural Network, etc.). The proposed method is composed of a neural network model, a residual connection, and a normalization layer. MSA is used for the encoder to learn the spatial position between patches, thereby capturing long-distance feature associations. FFNN maps the input data to the output space through a series of nonlinear transformations, thereby realizing complex pattern recognition and feature extraction of the input data, and is used to further process the feature representation of each position after the self-attention mechanism. The spatial attention is used as the output layer of the neural network model to map the model from the intermediate representation space to the output space. Therefore, MLP is used as the classification head to obtain the final output.

[0090] The temporal attention map is multiplied with the original input to achieve adaptive weighting of feature channels, enhancing the model's ability to capture dynamic information between frames. The output results of temporal attention, channel attention, and spatial attention are fused as the output of the spatiotemporal attention module to fuse features at different levels.

[0091] The fusion formula is shown in formula (6):

[0092] Y=(TA(X)⊙CA(X))⊙X+SA(X) (6)

[0093] Among them, SA represents spatial attention, ⊙ represents Hadamard product, and Y is the final output feature map;

[0094] The decoder is used to map the spatiotemporal features extracted from different dimensions back to the high-dimensional space to obtain the prediction results. The decoder has a similar structure to the encoder, except that a pixel rearrangement layer is added before the first standard normalization layer. The pixel rearrangement layer is used to perform pixel shuffle on the convolution operation results of each layer to achieve upsampling of hidden features and gradually reconstruct the model output.

[0095] The loss function of the entire model uses the spatially normalized mean square error (MSE). The spatially normalized MSE takes the spatial scale into account in the error calculation process to reduce the impact of spatial scale changes in the image on model training. The calculation formula of the spatially normalized MSE is shown in formula (7):

[0096]

[0097] Where pred is the predicted value, true is the true value, channel, height, and width are the number of channels, height, and width of the data respectively;

[0098] Step 4. Divide the spatiotemporal data X obtained in step 2 into a training set and a test set, use the data in the training set as input, and use the data in the test set as labels to train the model of the attention spatiotemporal prediction network;

[0099] Step 5. Use the trained spatiotemporal prediction network model to predict drought disasters.

[0100] Example 1

[0101] The method of the present invention is used to predict a single variable and to evaluate the prediction result.

[0102] Divide the data set. In order to ensure the accuracy of model training, the present invention selects 48 months of data from January 2017 to December 2020 as the training data set, selects 9 months of data from January 2021 to September 2021 as the verification data set, and selects 10 months of data from March 2021 to December 2021 as the test data set. Choose to use previous_length=6, after_length=2, step=1, that is, use 6 months of historical data to predict the next 2 months of data. Due to the limitations of data size and server video memory, the spatial resolution of the data used in the present invention is 5000m. For the Inner Mongolia Autonomous Region, the longitude direction occupies 644 pixels and the latitude direction occupies 356 pixels.

[0103] The present invention uses a 40GB NVIDIA A100 graphics card to complete the entire training and testing, the number of training rounds (Epoch) is set to 200, the batch size is set to 2, the initial learning rate (lr) is set to 0.001, the attenuation factor of the learning rate (lr_k_de cay) is set to 1.0, the learning rate (warmup_lr) of the warm-up stage is set to 0.0001, and the learning rate scheduling strategy uses the cosine annealing learning rate scheduler. Regarding the internal parameters of ABST-Net, the convolution kernel size of both ordinary convolution and deformable convolution is set to 3; the number of convolution layers in Encoder and Decoder is set to 4 (3 ordinary convolution layers + 1 deformable convolution layer), and the feature dimension of the hidden layer in the convolution layer is set to 32; the number of Spatio-temporal Attention Modules is set to 8, and the feature dimension of the hidden layer in the module is set to 256. The loss function uses the spatially normalized MSE.

[0104] Based on the above parameters, spatiotemporal prediction experiments based on ABST-Net were carried out, and ConvLSTM, PredRNN, PredRNN++, MIM, PhyDNet, MAU, SimVP and other popular spatiotemporal prediction models were selected for control experiments. The test data set was divided into three subsets. One subset used data from March to August 2021 to predict data from September to October 2021; one subset used data from April to September 2021 to predict data from October to November 2021; and the last subset used data from May to October 2021 to predict data from November to December 2021.

[0105] Select model parameters. Select soil moisture as the target of univariate prediction, so the number of channels of the data is 1. According to the above settings, the dimension of the training data set is (48,1,356,644) and the data set length is 41; the dimension of the validation data set is (9,1,356,644) and the data set length is 2; the dimension of the test data set is also (10,1,356,644) and the data set length is 3.

[0106] Select evaluation indicators. Using MSE and MAE as evaluation indicators, the evaluation indicators of the three data subsets are averaged as the evaluation results of the test data set. The specific evaluation results are shown in Table 3.

[0107] Table 3

[0108]

[0109]

[0110] From the analysis of the above table, it can be seen that in the univariate prediction of soil moisture content, the MSE and MAE of ABST-Net are the lowest values ​​among the above models, among which the MSE and MAE are 20.21% and 2.65% lower than those of SimVP, which has equally excellent results. This shows that the ABST-Net of the present invention is more accurate in predicting soil moisture in the univariate spatiotemporal prediction task, and is superior to similar spatiotemporal prediction models to a certain extent.

[0111] In order to more vividly judge the results of spatiotemporal prediction, a test data subset was selected to visualize the spatiotemporal prediction results of ABST-Net, and the results of SimVP and PredRNN++ were selected for comparison. Part of the Inner Mongolia Autonomous Region was selected for visualization, and the results are shown in Figure 2. Figure 3 As shown. It can be seen that the three models can all achieve correct predictions of future soil moisture content and can accurately grasp the characteristics of low soil moisture content in the study area. Among them, the prediction results of ABS T-Net and SimVP of the present invention are clearer and have more details, while the prediction results of PredRNN++ are not very precise and the image is relatively smooth, which can also confirm that the MSE and MAE of PredRNN++ are higher than those of ABST-Net and SimVP. In the future real data in September 2021, the soil moisture content of some plots in the southern part of the study area is relatively high (green area). The prediction data of ABST-Net of the present invention accurately reflects this feature, while SimVP loses the corresponding information. Therefore, for univariate spatiotemporal prediction tasks, ABST-Net of the present invention has better performance.

[0112] Example 2

[0113] The method of the present invention is used to predict multiple variables and evaluate the prediction results.

[0114] Divide the data set. Similarly, the dimension of the training data set is (48, 6, 356, 644), and the data set length is 41; the dimension of the validation data set is (9, 6, 356, 644), and the data set length is 2; the dimension of the test data set is also (10, 6, 356, 644), and the data set length is 3. The training parameters, model hyperparameters, loss function, and GPU model used in the experiment are consistent with the single variable prediction in Example 1.

[0115] Select model parameters. In this step, select soil moisture, air temperature at 2 meters, total precipitation, ground pressure, wind speed U component at 10 meters, and wind speed V component at 10 meters for simultaneous spatiotemporal prediction. The basic information of the above six variables is shown in Table 4.

[0116] Table 4

[0117]

[0118] Select evaluation indicators. MSE and MAE are used as evaluation indicators, the same as in Example 1.

[0119] The results of the evaluation experiments are shown in Table 5.

[0120] Table 5

[0121]

[0122]

[0123] The bold data in the above table represent the minimum value of the prediction evaluation index of a certain variable.

[0124] By comparing the performance of the ABST-Net of the present invention with other spatiotemporal prediction models in terms of soil moisture, temperature, rainfall, atmospheric pressure, wind speed and other indicators, it can be seen from the data in the table that the average MSE of ABST-Net is the lowest among the six models, 60.7% lower than the second lowest SimVP, and the average MAE of ABST-Net is the third lowest among all models, but only 5.5% higher than the lowest PredRNN; ABST-Net has the lowest MSE in the prediction of soil moisture, rainfall and atmospheric pressure, and shows lower MSE and MAE in the prediction of other variables, which indirectly reflects that ABST-Net has higher accuracy and stability in multiple spatiotemporal prediction models.

[0125] A subset of the test data set is selected to visualize the spatiotemporal prediction results of ABST-Net, and two models with better performance, SimV P and PredRNN, are selected for comparison. The variables to be visualized are soil moisture and rainfall, and the results are as follows: Figure 4As shown. As shown in the figure, soil moisture data from March 2021 to August 2021 are needed to predict future data from September to October 2021. In September 2021, all three models can successfully predict the changing trend of soil moisture, but the image contours predicted by SimVP and PredRNN are not as clear as those of ABST-Net. The plots with lower soil moisture content in the central part of the study area (red area) are reflected in the three prediction results. The shape of the red area predicted by ABST-Net of the present invention is closest to the true value, followed by SimVP. However, SimVP failed to capture the low soil moisture characteristics in the northwest of the study area well, and the prediction result is slightly higher than the true value (bluish-yellow). For the feature that the soil moisture in the southernmost region is slightly higher than that in other regions, ABST-Net of the present invention can also capture it well, while the predicted values ​​of SimVP and PredRNN are one lower and one higher. At the same time, a little grid effect can be vaguely seen in the background of PredRNN's prediction results, which is caused by the excessive fitting of the model during the training process. For October 2021, ABST-Net still has the most accurate prediction for the central red area, and the geometric outline is most similar to the true value. However, the model's prediction results for the northwest region are lower, while SimVP and PredRNN's predictions for the northwest region are higher.

[0126] Figure 5 It is the result of spatiotemporal prediction of rainfall in the same time period. It can be seen that ABST-Net of the present invention still performs best among the three models. In the future true value, only the southeastern region of the study area has higher rainfall (the blue color is darker), and the rainfall in other regions is slightly lower. Only the prediction result of ABST-Net is closest to the above characteristics. The color of the southern region is darker than other parts, but the prediction of the southeastern region is low, and the phenomenon of slightly higher rainfall is not reflected; in the prediction results of SimVP, the areas with high rainfall occupy most of the entire study area, with obvious prediction errors; in the prediction results of PredRNN, the areas with higher rainfall are concentrated in the southwest of the study area, with obvious position deviations from the true value. The visualization effects of the spatiotemporal prediction of rainfall by the three models are not as good as those of the spatiotemporal prediction of soil moisture. This may be because the rainfall data comes from the ERA5 public data set, and its spatial resolution is not as high as the downscaled soil moisture data. Therefore, the data itself does not have too many details and is relatively smooth. At the same time, the value of rainfall itself is small and the fluctuation is weak, which affects the learning of the temporal and spatial features in the data by the spatiotemporal prediction model.

[0127] In summary, for multivariate spatiotemporal prediction tasks, the prediction results of the ABST-Net of the present invention are closest to the true values ​​in both overall and detailed aspects. Although the prediction accuracy is slightly weaker than that of the single-variable spatiotemporal prediction, this is mainly because the time dimension and channel dimension of the data are mixed together, and the model does not treat the two channels separately during training, which has a certain negative impact on the prediction results. However, in general, the performance of the ABST-Net spatiotemporal prediction model is better than other similar models.

[0128] The above description is only a specific implementation mode of the present invention. Any feature disclosed in this specification, unless otherwise stated, can be replaced by other alternative features that are equivalent or have similar purposes; all the disclosed features, or all the steps in the methods or processes, except for mutually exclusive features and / or steps, can be combined in any way.

Claims

1. A method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model, characterized in that: The following steps are involved: Step 1. Obtain meteorological data and soil moisture data of a certain area in a certain period of time based on remote sensing satellites, and preprocess the data: Step 1.

1. Based on the correlation coefficient calculation formula, calculate the average value of the correlation coefficient between each variable of the meteorological data and soil moisture during the period, arrange the average values ​​from large to small, and select the meteorological data variables corresponding to the top 5 average values; Step 1.

2. Merge the five meteorological data variables selected in step 1.1 with the soil moisture data to obtain a multi-band image; Step 2. Convert the data format. The specific process is as follows: The multi-band image obtained in step 1 is converted into format data, the longitude and latitude information in the format data is removed, only the pixel value information is retained, and the spatiotemporal data X with dimensions (T, C, H, W) is constructed in chronological order, where T is the time dimension, C is the channel dimension, H represents the height of the data, and W represents the width of the data; Step 3. construct a model of an attention-based spatiotemporal prediction network, wherein the model of the attention-based spatiotemporal prediction network includes an encoder, a spatiotemporal attention module, and a decoder; The encoder is used to convert high-dimensional spatiotemporal data X into low-dimensional hidden features; the spatiotemporal attention module takes the low-dimensional hidden features as input and uses multiple attention mechanisms to extract spatiotemporal features from three different dimensions: time, channel, and space; the decoder is used to map the spatiotemporal features extracted from different dimensions back to the high-dimensional space, thereby obtaining a prediction result; Step 4. Divide the spatiotemporal data X obtained in step 2 into a training set and a test set, use the data in the training set as input, and use the data in the test set as labels to train the model of the attention spatiotemporal prediction network; Step 5. Use the trained spatiotemporal prediction network model to predict drought disasters.

2. The method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model according to claim 1, characterized in that: The meteorological data variables in step 1 include air temperature at a certain height, total precipitation, ground pressure, wind speed U component and wind speed V component.

3. The method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model according to claim 1, characterized in that: In step 1, the calculation formula of the correlation coefficient is as follows: Among them, ρ P,Q is the correlation coefficient between variables P and Q, Cov(P,Q) is the covariance of P and Q, σ P and σ Q are the standard deviations of P and Q respectively.

4. The method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model according to claim 1, characterized in that: In step 2, the height and width of the data are determined by the spatial resolution of the data, indicating the spatial position of the current data.

5. The method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model according to claim 1, characterized in that: In step 3, the encoder includes 3 2D convolutional layers, 1 deformable convolutional layer, a normalized layer and a SiLU activation function layer; wherein the first 2D convolutional layer, the second 2D convolutional layer, the third 2D convolutional layer and the deformable convolutional layer are cascaded, the output of the first 2D convolutional layer is simultaneously used as the input of the second 2D convolutional layer and the first normalized layer, the output of the second 2D convolutional layer is simultaneously used as the input of the third 2D convolutional layer and the first normalized layer, the output of the third 2D convolutional layer is used as the input of the first normalized layer, the output of the first normalized layer is used as the input of the first SiLU activation function layer, and the first SiLU activation function layer outputs a feature map; The deformable convolution layer is obtained by adding an offset to the grid sampling positions in the standard convolution.

6. The method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model according to claim 5, characterized in that: For a certain position P0 of the input feature map x, the output of the deformable convolution layer after the deformable convolution is shown below: Among them, P n is the offset of the nth position of the input feature map, R is the adjustment parameter, ΔP n is the offset of a certain position; w(P n ) represents the convolution weight coefficient at that position, Δm k is a tuning parameter.

7. The method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model according to claim 1, characterized in that: In step 3, the spatiotemporal attention module includes a temporal attention unit, a channel attention unit and a spatial attention unit; The temporal attention unit includes a deformable convolution layer and a depthwise separable convolution layer, wherein the depthwise separable convolution layer is composed of depthwise convolution and pointwise convolution; the depthwise separable convolution layer applies an independent convolution kernel to each time channel, thereby realizing independent convolution between channels, capturing local spatiotemporal patterns in time series data, and helping the model learn the time series correlation in input data; the pointwise convolution uses a 1×1 convolution kernel to linearly combine the output of the depthwise convolution, adjust the depth and dimension of the feature map, and obtain the output of the depthwise separable convolution; The channel attention unit includes a global average pooling layer, a global maximum pooling layer, a multi-layer perceptron and a Sigmoid function layer; the feature maps output from the encoder are respectively input into the global average pooling layer and the global maximum pooling layer to capture the channel-level information of the input feature map; The weight-sharing multi-layer perceptron calculates the complex nonlinear dependencies between channels based on the output results of the global average pooling and global maximum pooling layers, then adds the output results to achieve feature information fusion, and inputs the fusion results to the Sigmoid function layer, which activates the channel attention map. The spatial attention unit includes a Flatten layer, a Transpose layer, a second normalization layer and a Transformer encoding layer which are cascaded in sequence; the Transformer encoding layer is composed of a multi-head self-attention mechanism, a feedforward neural network and a multi-layer perception head; The Flatten layer is used to flatten the input data according to width and height, and the data dimension changes from (B, T, C, H, W) to (B, T, C, H×W). The Transpose layer exchanges the flattened spatial dimension (H×W) with the channel dimension C, and the data dimension becomes (B, T, H×W, C). The second normalization layer is used to realize the image segmentation. At the same time, the segmented images are arranged in sequence, and multiple channel information is still retained at each spatial position, so that each spatial position can perform self-attention calculation independently. The multi-head self-attention mechanism is used for the encoder to learn the spatial position between the segmented images, so as to capture the long-distance feature association. The feedforward neural network maps the input data to the output space through a series of nonlinear transformations, so as to realize the complex pattern recognition and feature extraction of the input data, and is used to further process the feature representation of each position after the self-attention mechanism. The multi-layer perception head is used as the classification head to obtain the final output. The temporal attention map is multiplied with the original input to achieve adaptive weighting of feature channels, enhancing the model's ability to capture dynamic information between frames; the output results of temporal attention, channel attention and spatial attention are fused as the output of the spatiotemporal attention module to fuse features at different levels.

8. The method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model according to claim 1, characterized in that: The loss function based on the ABST-Net spatiotemporal prediction model uses the spatially normalized mean square error MSE. The calculation formula of the spatially normalized MSE is shown as follows: Among them, pred is the predicted value, true is the true value, channel, height, and width are the number of channels, height, and width of the spatiotemporal data X respectively.

9. The method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model according to claim 1, characterized in that: The structure of the decoder is similar to that of the encoder, except that a pixel rearrangement layer is added before the first standard normalization layer. The pixel rearrangement layer is used to rearrange the rows of pixels of the convolution operation results of each layer to achieve upsampling of hidden features and gradually reconstruct the model output.

10. The method for drought disaster prediction based on the ABST-Net spatiotemporal prediction model according to claim 1, characterized in that: During step 4 training, the initial learning rate can be set between 0.001 and 0.1, the learning rate attenuation factor is set between 0.1 and 1.0, the learning rate in the warm-up phase is set to 1 / 10 or 1 / 100 of the initial learning rate, and the learning rate scheduling strategy uses the cosine annealing learning rate scheduler.

Images