Typhoon cloud picture prediction method based on self-attention memory mechanism
By introducing a self-attention memory mechanism into the typhoon cloud map prediction method, the global and local spatial characteristics and long-term dependency characteristics were extracted, and the existing methods had low prediction accuracy and poor generalization were solved, and a more efficient typhoon cloud map was achieved. predict.
Patent Information
- Application Number
- CN202411408888.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2025-02-07
AI Technical Summary
The existing typhoon cloud diagram prediction methods rely on a large amount of prior knowledge and artificial parameter settings, resulting in low prediction accuracy, short timeliness and poor generalization.
Using the Typhoon cloud map prediction method based on the self-attention memory mechanism, the global and local spatial characteristics of image frames are extracted through the self-attention module, and the memory mechanism is added to the self-attention module to extract inter-frames with long-term and spatial dependencies. feature.
It improves the accuracy and timeliness of typhoon cloud map prediction, enhances the generalization ability of the model, and can learn cloud movement trends more accurately.
Smart Images
Figure CN119399644A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of remote sensing image processing, and in particular to a typhoon cloud image prediction method based on a self-attention memory mechanism. Background Art
[0002] Typhoons are often accompanied by strong winds and heavy rains, and the disasters they cause pose a huge threat to people's daily travel and the safety of life and property. Therefore, how to scientifically and effectively provide typhoon warning services is of great significance to my country's construction of a meteorological disaster prevention system. At present, the commonly used methods in typhoon forecasting business include numerical forecasting, statistical forecasting, and statistical dynamic forecasting. All three methods rely on a large amount of prior knowledge and artificial parameter settings, which are prone to problems such as low prediction accuracy, short timeliness, and poor generalization. In recent years, with the widespread deployment of meteorological satellites and the rapid development of remote sensing technology, global satellite cloud image big data has exploded. More and more scholars are trying to apply deep learning technology to the processing and analysis of satellite cloud images, opening up a new field of meteorological visualization forecasting research. However, most deep learning models now rely on stacked convolutional layers to obtain local spatial features. This leads to the problem of inaccurate prediction of typhoon cloud images. In order to cope with the complex changes in cloud position and shape in the cloud image, the prediction module of the model needs to be able to extract global and local spatial features from the cloud image. In addition, for irregular cloud motion, more attention should be paid to the spatiotemporal sequence features between input cloud image frames in the time series prediction module, and the extraction of temporal features with long-term dependencies should be considered so that the spatiotemporal sequence prediction network can learn the cloud motion trend more accurately. Summary of the invention
[0003] In view of the shortcomings of the above-mentioned prior art, the present invention provides a typhoon cloud image prediction method based on a self-attention memory mechanism, wherein the self-attention module of the model is designed to extract spatial features of global and local dependencies of internal image frames. In addition, a memory mechanism is added to the self-attention module to extract inter-frame features with long-term and spatial dependencies to solve the problem of inaccurate typhoon cloud image prediction.
[0004] A typhoon cloud image prediction method based on a self-attention memory mechanism comprises the following steps:
[0005] Step (1) Obtain typhoon cloud images with the ".nc" format suffix from Sunflower-8
[0006] Step (2) preprocesses the typhoon cloud image with the ".nc" format suffix obtained in step (1).
[0007] Eliminate the incomplete and incomplete data in the acquired cloud map data, select the cloud map data with band "8", and use python code for batch processing to convert the original ".nc" format data into ".png" or ".jpg" format. And normalize the image pixel values. The normalization formula is as follows:
[0008]
[0009] Step (3) divides the preprocessed data set from step (2) into a training set and a test set in a ratio of 8:2.
[0010] Step (4) constructing a SAM-Net network; the SAM-Net network includes a self-attention mechanism module, a self-attention memory mechanism module, and a SAM-Net network unit;
[0011] Step (5), construct the self-attention module in step (4). The formula of the self-attention mechanism module is as follows:
[0012]
[0013]
[0014] Among them, Q h , V h , V h Represents query, key and value respectively, H t represents hidden features, W q , W k , W v is the weight of the 1×1 convolution, represents the set of real numbers, C and is the number of channels, N = H × W,
[0015] α i,j represents the similarity between the i-th and j-th points, Z i Represents the calculation of the position and features of the i-th position by weighting all positions, W v H t；j Yes V h The value of column J;
[0016] Step (6), construct the self-attention memory mechanism module in step (4), the self-attention memory mechanism module design formula
[0017]
[0018] Z=W z [Z h ; Z m ]
[0019] i t =σ(W m；zi *Z+W m；hi *H t +b m；i )
[0020] g t =tanh(W m；zg *Z+W m；hg *H t +b m；g )
[0021]
[0022] o t =σ(W m；zo *Z+W m；ho *H t +b m；o )
[0023]
[0024] * indicates convolution operation, represents the Hadamard product, e m Represents the query Q h and key K m Perform matrix multiplication to compute the similarity score between the input and memory, α m ；i,j represents the weight value of the aggregated feature calculated along the row direction, σ represents the SoftMax function, and Z m；i Represents calculation Z m The pixel value at the i-th position, M t-1；j M stands for memory cell t-1 The Jth column, i t , g t , o t They represent input gate, modulation gate, and output gate respectively. Represents the updated space-time memory cells, Represents the hidden state after the update;
[0025] Step (7), construct the SAM-Net network unit in step (4). The SAM-Net network unit formula is as follows:
[0026] g′ t , g″ t represents the modulation gate at the current moment, i′ t , i″ t , represents the input gate at the current moment, f′ t , f″ tThe output gate at the current moment, o′ t Represents the output gate, W xg , W hg , W xi , W hi , W xf , W hf , W′ xg , W mg , W′ xi , W mi , W′ xf , W mf , W xo , W ho , W mo represents the weight, X t Represents the input features at the current moment, Represents the memory state, Represents the hidden state, stands for spatiotemporal memory unit, SAM stands for self-attention memory mechanism;
[0027] Step (8) The SAM-Net network units constructed in step (7) are stacked into 4 layers to form a SAM-Net network. The trained cloud map is input into the network structure to obtain the corresponding memory state. Represents hidden state Space-time memory unit The ADAM optimizer and L2 loss function are used during training. The formula of L2 loss function is as follows:
[0028]
[0029] f(x i ) represents the xth i The pixel prediction value of the point, y i Represents the xth i The true value of a point. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 This is a flowchart of the typhoon cloud image prediction method based on the self-attention memory mechanism.
[0031] Figure 2 Schematic diagram of the self-attention mechanism and the self-attention memory mechanism.
[0032] Figure 3 Schematic diagram of SAM-Net network unit. DETAILED DESCRIPTION
[0033] The present invention provides a typhoon cloud image prediction method based on a self-attention memory mechanism. The self-attention module of the model is designed to extract the spatial features of global and local dependencies of internal image frames. In addition, a memory mechanism is added to the self-attention module to extract inter-frame features with long-term and spatial dependencies to solve the problem of inaccurate typhoon cloud image prediction. It includes the following steps:
[0034] (1) Obtain the cloud image data of Typhoon Sunflower No. 8 for data preprocessing
[0035] (1-1) Obtain a sample of historical typhoon cloud image sequences, remove missing and incomplete data from the ".nc" format files obtained by meteorological satellites, and convert them into ".jpg" format images. Select images with a band of "8", reshape the images to a size of 128*128, and normalize the pixel values to between [0, 1].
[0036] Combined model training, and finally the data is converted into normalized grayscale data. The normalization formula is:
[0037]
[0038] (1-2) Dataset division: Set total_length to 20, that is, every 20 data are divided into a sequence, of which the first 10 data are input sequences and the last 10 are validation sequences. The data set is divided into the training sequence sample set train_data and the test sequence sample set test_data in a ratio of 8:2.
[0039] (2) Construct and train a neural network model to obtain a SAM-Net neural network model. Input the divided training sequence sample set train_data into the constructed convolutional spatiotemporal prediction neural network model and train it through the multi-layer network of the neural network model.
[0040] (2-1) Initialization of training parameters. Set the input image height height = 128, width width = 128 and channel channel = 3, convolution kernel size filter_size = 5, convolution step stride = 1, prediction unit stacking layer num_layers = 4, hidden layer number num_hidden = 128, number of samples input each time in the training phase batch_size = 16, maximum number of training rounds max_epoch = 10000, learning rate λ = 0.0001, input sequence length input_length = 10 and prediction sequence length output_lengt = 10, use ADAM as the optimizer, L2 as the loss function, etc. The L2 loss function formula is as follows:
[0041]
[0042] f(x i ) represents the xth i The pixel prediction value of the point, y i Represents the xth i The true value of a point.
[0043] (2-2) Initialize the parameters of the self-attention memory module, set the number of hidden layers num_hidden = 128, the self-attention hidden layer dimension attention_hidden_dims = 128, construct the 4-layer SAM-Net network unit parameters, set the convolution kernel of the 1-3 layers to 5, the step size to 1, padding = 2, the convolution kernel of the fourth convolution layer to 1, the step size to 1, padding = 0, the bias of all layers to bias = 0, and the schematic diagram of the self-attention memory module is as follows Figure 2 (below) shown.
[0044] (2-3) Training sample reading. During each training, batch_size=16 sequence samples are read from the training sample set as the input of the SAM-Net network. The SAM-Net network is as follows: Figure 3 As shown. The formula of SAM-Net network is as follows:
[0045]
[0046]
[0047] g′ t , g″ t represents the modulation gate at the current moment, i′ t , i″ t , represents the input gate at the current moment, f′ t , f″ t The output gate at the current moment, o′ t Represents the output gate, W xg , W hg , W xi , W hi , W xf , W hf , W′ xg , W mg , W′ xi , W mi , W′ xf , W mf , W xo , W ho , W mo represents the weight, X t Represents the input features at the current moment, and SAM represents the self-attention memory mechanism.
[0048] (2-4) After the input training cloud image passes through the SAM-Net network, the predicted image O will be output t = {O 11 ,O 12 ,...,O 20}.
[0049] (2-5) Calculate the loss function value. The predicted sequence and the predicted reference sequence obtained by forward propagation from step (2-1) to step (2-4) are used as the L2 loss function. The network parameter gradient is calculated based on the value obtained by the loss function, and the network parameters are updated to complete the back propagation.
[0050] (2-6) All data in the training set are calculated once, which is called one round. Steps (2-3) to (2-5) are repeated until the maximum number of training rounds is completed or the convergence condition is reached. The training is completed and the SAM-Net neural network model is obtained.
[0051] (3) Use the SAM-Net neural network model to predict typhoon cloud images. Use the SAM-Net neural network model trained in step 2 and the test sequence sample set divided in step 1 to make predictions. During prediction, read 20 sequence sample data from the test sequence sample set test_data each time, input the sample data into the trained SAM-Net network, and obtain a predicted typhoon cloud image sequence with a length of output_length = 10. The overall flow chart of the SAM-Net network predicting typhoon cloud images is shown in the following figure. Figure 3 shown.
[0052] (3-1) The mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM) and image similarity (LPIPS) are used, which are widely used in image evaluation. The difference between these indicators is that MSE is used to calculate the absolute pixel error value. PSNR is an indicator used to measure the similarity of two images, SSIM is used to measure the similarity of structural information in a spatial neighborhood, and LPIPS is an evaluation indicator based on deep features that are more in line with human perception. The larger the values of SSIM and PSNR, or the smaller the values of MSE and LPIPS, the better the prediction effect of the model.
[0053] The method in this application can fill the gaps in the existing cloud map prediction problems such as low accuracy, short timeliness and poor generalization, and meet subsequent meteorological model and prediction needs.
Claims
1. A typhoon cloud image prediction method based on self-attention memory mechanism, characterized in that: include: Get the cloud chart data of the typhoon Sunflower No. 8; Preprocess the acquired cloud image data to obtain ".png" or ".jpg" format; Divide the preprocessed data set into training set and test set; The typhoon cloud image sequence samples are input into the SAM-Net neural network model to obtain the predicted typhoon cloud image sequence; wherein the SAM-Net neural network training process includes: building a SAM-Net model; wherein the SAM-Net model includes a self-attention mechanism module, a self-attention memory mechanism module, and a SAM-Net network unit.
2. The method according to claim 1, characterized in that Get the Sunflower-8 wind and cloud chart data, including: Get typhoon cloud chart data in the format of ".nc" suffix.
3. The method according to claim 1, characterized in that The acquired cloud image data is preprocessed, including: Eliminate the incomplete and incomplete data in the acquired cloud map data, select the cloud map data with band "8", use python code for batch processing, and convert the original ".nc" format data into ".png" or ".jpg" format. And normalize the image pixel values. The normalization formula is:
4. The method according to claim 1, characterized in that The preprocessed data set is divided into a training set and a test set, including: dividing the preprocessed data set into a training set and a test set in a ratio of 8:
2.
5. The method according to claim 1, characterized in that Build a self-attention mechanism module, including: the self-attention module obtains the hidden features H of the input cloud map t , and passed to the SAM-Net network unit of the next layer. The design formula of the self-attention mechanism module is: Among them, Q h , V h , V h Represents query, key and value respectively, H t represents hidden features, W q , W k , W v is the weight of the 1×1 convolution, represents the set of real numbers, C and is the number of channels, N = H × W, α i,j represents the similarity between the i-th and j-th points, Z i Represents the calculation of the position and features of the i-th position by weighting all positions, W v H t；j Yes V h The value of column J.
6. The method according to claim 1, wherein the self-attention memory mechanism module is constructed, comprising: The self-attention module obtains the hidden features H of the input cloud image t and spatiotemporal characteristics And it is passed to the SAM-Net network unit of the next layer. The design formula of the self-attention memory mechanism module is: W=W z [WITH h ;WITH m ] i t =σ(W m；zi *Z+W m；hi *H t +b m；i ) g t =tanh(W m；zg *Z+W m；hg *H t +b m；g ) about t =σ(W m；zo *Z+W m；ho *H t +b m；o ) * indicates convolution operation, represents the Hadamard product, e m Represents the query Q n and key K m Perform matrix multiplication to compute the similarity score between the input and memory, α m；i,j represents the weight value of the aggregated feature calculated along the row direction, σ represents the SoftMax function, and Z m；i Represents calculation Z m The pixel value at the i-th position, M t-1；j M stands for memory cell t-1 The Jth column, i t , g t , o t They represent input gate, modulation gate, and output gate respectively. Represents the updated space-time memory cells, Represents the hidden state after the update.
7. The method according to claim 1, characterized in that Build a SAM-Net network unit, including: the SAM-Net network unit is used to store the memory state of the same layer network output at the previous moment Hidden State and the spatiotemporal memory unit of the previous layer at the same time Input into the SAM-Net network unit of the current layer l, and after forward propagation, pass through the self-attention memory module to obtain the current time memory state Space-time memory unit and hidden state The SAM-Net network unit design formula is: g′ t , g″ t represents the modulation gate at the current moment, i′ t , i″ t , represents the input gate at the current moment, f′ t , f″ t The output gate at the current moment, o′ t Represents the output gate, W xg , W hg , W xi , W hi , W xf , W hf , W′ xg , W mg , W′ xi , W mi , W′ xf , W mf , W xo , W ho , W mo represents the weight, X t Represents the input features at the current moment, and SAM represents the self-attention memory mechanism.
8. The method according to claim 1, characterized in that The SAM-Net neural network training process includes: building a 4-layer SAM-Net network unit. During the training process, the ADAM optimizer and L2 loss function are used. The L2 loss function formula is: f(x i ) represents the xth i The pixel prediction value of the point, y i Represents the xth i The true value of a point.