Parallel spatiotemporal temperature prediction method based on d3at-lstm model

Abstract

The application discloses a spatiotemporal temperature parallel prediction method based on a D3AT-LSTM model, and uses a historical meteorological observation sequence X treated by a sliding window as the input of the network. The meteorological data is input into a 3DCNN module combined with a parallel attention branch, is processed through a three-dimensional convolution module, plays a role in the short sequence feature abstraction capability, introduces a channel attention mechanism to process the importance difference between the short sequence features, automatically learns the feature weight according to the loss loss for all input features, makes it pay attention to a specific time channel, realizes the extraction of high-order spatiotemporal significant features, and sequentially inputs the data of each time node into an LSTM recurrent neural unit to capture the coarse-grained dependency relationship under the long time sequence of each node. Finally, the high-dimensional vector fused with different dimensional features is transmitted into a full connection layer, and finally, the numerical value of the predicted temperature is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of air temperature prediction, in particular to a spatiotemporal air temperature parallel prediction method based on a D3AT-LSTM model. BACKGROUND

[0002] The earliest short-term air temperature prediction mainly adopts empirical prediction method and statistical method. The empirical prediction method is based on historical weather conditions and the personal experience of forecasters to predict future air temperature. This method depends on the understanding ability of forecasters on past weather events, while the statistical method analyzes and selects meteorological factors related to air temperature to construct a statistical model to predict the trend of air temperature in the future. However, both methods lack a deep understanding of atmospheric physical processes, which limits the accuracy of predicting air temperature changes.

[0003] With the development of technology, numerical weather prediction has become the mainstream method of air temperature prediction. This method is based on atmospheric physics and dynamics, and simulates the evolution of atmospheric state through high-performance computers to predict the weather in the future. Although numerical prediction has made significant progress in improving the accuracy of prediction, due to the delay in model initialization and data assimilation process, numerical prediction still faces challenges in meeting the high-precision air temperature prediction, especially in the prediction of volatile air temperature sequences.

[0004] With the rapid development of observation technologies such as remote sensing satellites and space radars, new possibilities have emerged for short-term air temperature prediction. Satellites can provide large-scale, high-frequency observations of surface and atmospheric temperature data, making available meteorological data more massive and complex, which to some extent improves the accuracy of weather prediction, but also brings certain redundancy and interference information to weather monitoring. Due to the influence of various spatiotemporal inducing factors on the evolution of meteorological sequences, various meteorological variables and air temperature are accompanied by complex interaction relationships, and the overall trend of meteorological elements presents nonlinear volatility. How to fully utilize massive meteorological data and fully exploit potential information to further improve the accuracy of air temperature prediction is a problem that needs to be solved in current prediction methods.

[0005] With the advent of the big data era, deep learning technology has developed rapidly, bringing new solutions to the field of meteorological prediction. The network architecture based on deep learning simulates the structure of human brain neurons to simulate the process of signal transmission from different synapses, thereby adaptively learning the implicit rules. Through training a large amount of historical meteorological data, machine learning models can identify complex patterns of air temperature changes and effectively predict future air temperature changes, providing a new solution to the problems existing in traditional air temperature prediction methods.

[0006] Therefore, the high-dimensional massive meteorological data is utilized to carry out temperature prediction, and compared with the traditional method, the method has greater advantages and is widely applied. However, the deep learning method still has problems of weak generalization and low prediction accuracy in temperature prediction, and still has room for improvement.

[0007] The existing research in the field of meteorological prediction shows that although the existing method has made progress in improving the temperature prediction accuracy, due to the limitation of the structure of the method itself, there are still some deficiencies. The deep learning model inevitably faces the problem of overfitting, and the numerous parameters lead to huge consumption of computing resources, and the "black box" property limits the performance of the model on the explanation of the inherent characteristics of the time series. In long-term sequence prediction, the generalization ability is weak.

[0008] Therefore, it is a problem worthy of study to provide a spatiotemporal temperature parallel prediction method based on a D3AT-LSTM model which is improved on the basis of 3DCNN and LSTM model, and models and analyzes the spatiotemporal elements affecting the temperature sequence to comprehensively improve the accuracy of temperature prediction. SUMMARY

[0009] The purpose of the present application is to provide a spatiotemporal temperature parallel prediction method based on a D3AT-LSTM model which is improved on the basis of 3DCNN and LSTM model, and models and analyzes the spatiotemporal elements affecting the temperature sequence to comprehensively improve the accuracy of temperature prediction.

[0010] The purpose of the present application is achieved as follows:

[0011] The parallel prediction method for spatio-temporal temperature based on a D3AT-LSTM model, wherein the network structure of the D3AT-LSTM model comprises an input layer, a three-dimensional convolution layer combined with a parallel attention mechanism and an LSTM recurrent neural unit; the result of a previous layer is taken as the input of a subsequent layer from the input layer, the three-dimensional convolution layer combined with the parallel attention mechanism, the LSTM recurrent neural unit to the final temperature prediction; the input layer is responsible for inputting the original data required by the network, wherein the original data is a historical meteorological observation sequence processed by a sliding window, and the original data is first subjected to cleaning and normalization processing so as to provide stable and suitable input for the neural network; the meteorological data processed by the input layer is input into the three-dimensional convolution layer combined with the parallel attention mechanism, is processed by a three-dimensional convolution module and is subjected to channel attention mechanism to extract high-order spatio-temporal significant features; more spatial information is extracted and utilized by fusing the data of multiple adjacent stations, and the channel attention mechanism is introduced to identify important features on different time channels, so that the network can focus more on the features having a greater impact on the prediction result; the LSTM recurrent neural unit receives the output from the convolution layer and adjusts its internal state according to the output to better adapt to the internal features of the meteorological sequence; the LSTM recurrent neural unit captures the dependency relationship in a long time span in the sequence by introducing a gating mechanism, so as to more accurately predict the time series data such as temperature; the neural unit is responsible for processing long-time dependency relationship; the three cooperate together to improve the processing capability and prediction accuracy of the meteorological data. The meteorological data processed by the input layer is input into the three-dimensional convolution layer combined with the parallel attention mechanism, is processed by the three-dimensional convolution module and is subjected to channel attention mechanism to extract high-order spatio-temporal significant features. More spatial information is extracted and utilized by fusing the data of multiple adjacent stations, and the channel attention mechanism is introduced to identify important features on different time channels, so that the network can focus more on the features having a greater impact on the prediction result. The method comprises the following steps:

[0012] Step 1: processing of the input layer: using a historical meteorological observation sequence “X” processed by a sliding window as the input of the network; the input layer is responsible for inputting the original data required by the network, and here, the historical meteorological observation sequence processed by the sliding window is used, and the data is first subjected to cleaning and normalization processing so as to provide stable and suitable input for the neural network. In order to enable the model to fully consider the complex influence features in time and space, the input layer combines the meteorological sequences of the adjacent coordinate points of the target position to construct a hyper voxel structure fusing time, space and multiple features, so as to take the structure as the input of the subsequent network, and enable the subsequent model to more fully focus on the complex spatio-temporal elements acting on the temperature.

[0013] Step 2: Processing of the three-dimensional convolution layer combined with parallel attention mechanism: the processed historical meteorological observation sequence "X" of step 1 is input into the 3DCNN module combined with parallel attention branch, processed through the three-dimensional convolution module, and the channel attention mechanism is introduced to realize the extraction of high-order spatio-temporal significant features; the processed meteorological data of the input layer are input into the three-dimensional convolution layer combined with parallel attention mechanism, processed through the three-dimensional convolution module, and the channel attention mechanism is introduced to realize the extraction of high-order spatio-temporal significant features.

[0014] Step 3: Application of LSTM recurrent neural unit: the data of each time node are sequentially input into the LSTM recurrent neural unit to capture the coarse-grained dependency relationship under the long time sequence of each node; the LSTM recurrent neural unit receives the output from the convolution layer and adjusts its internal state according to the output to better adapt to the internal features of the meteorological sequence. The convolution network combined with the attention mechanism extracts the local spatio-temporal features of the meteorological sequence, and it is important to mine the global long-time dependency relationship of the meteorological sequence for temperature prediction; the recurrent neural network obtains the long-time dependency of the input features by modeling the context information; the LSTM prevents memory loss and gradient dispersion problems caused by too long step length by introducing the gating mechanism, and effectively processes the dependency relationship of long sequence data.

[0015] Step 4: Temperature prediction: the high-dimensional vector fused with different dimensional features processed by the LSTM recurrent neural unit is transmitted into the full connection layer, and finally the numerical value of the predicted temperature is obtained.

[0016] The specific operation of step 1 is as follows: first, the meteorological data is cleaned, and the missing data is filled with the data of the previous time; since the meteorological features have different numerical ranges, in order to accelerate the convergence of network weight parameters and obtain the optimal solution, the data is normalized using the linear function Min-Max-Scaling before being used as the input of the neural network;

[0017] The reanalysis meteorological data used is time series data with specific spatio-temporal resolution, before inputting into the network, for the prediction of the temperature value of a certain station, the eight adjacent points in space are combined to fuse the meteorological features in the spatio-temporal range as the subsequent input data; in order to meet the data input format of 3DCNN, first expand the channel C=1 in the second dimension, the original meteorological data is represented as a multi-dimensional tensor of C*W*H in each time slice, W and H represent the number of stations and the number of features respectively, then use the sliding window technology with window size S=72 to divide into multiple time-overlapping feature sequences, the input features are finally represented as a four-dimensional tensor X MFTS (S*C*W*H), S is the length of the input historical time series.

[0018] The specific operation of step 2 is as follows: for the meteorological feature super voxel reconstruction X MFTS , X MFTS is the input layer processed super voxel structure fused with time-space-multiple meteorological variables, which is used as the input of the subsequent network. X MFTS There are feature matrices of W*H*C pixels in the image format on all T Tot time slices, X MFTS fuses spatial correlation and time correlation information; a 3DCNN module is used to perform spatio-temporal convolution operation on the "cube" tensor X MFTS stacked by multiple continuous feature matrices, so as to capture deep changes in time and space dimensions at the same time. The output of the 3DCNN module each time extracts local continuous spatio-temporal information in the previous layer, and obtains high-order spatio-temporal features of finer granularity; a parallel channel attention branch SENET is introduced in the 3DCNN module to evaluate the importance of the time channel; at the same time, in order to ensure that the attention branch can more comprehensively analyze the importance difference between different time points in the sequence, the sub-domain range input to the attention module is expanded, that is, on the basis of the convolution module input sequence length l c , the length of l c is expanded by 1 / 2 on the left and right to obtain l a , and l a =2l c , so that it can make a comprehensive decision on the importance degree of different time channels;

[0019] The channel attention branch SENET completes the significance evaluation on the time channel through the compression process Squeeze and the scaling process Excitation. For the input super voxel information, F Squeeze function performs global average pooling in the spatial dimension, and compresses the features in each time slice into a real number, F Excitation function uses a gating mechanism to adaptively learn the importance of each time channel; F Squeeze function and F Excitation function are existing function formulas in the SENET network structure;

[0020]

[0021] Wherein, u is a two-dimensional matrix, H is high, W is wide, and z is the result of performing global average pooling on the features in the spatial dimension;

[0022] s=F Excitation (z,W)=σ(W2δ(W1z))

[0023] Wherein, v is a sigmoid function, delta is a ReLU function, W1 is a dimension reduction layer parameter, W2 is a dimension increase layer parameter, and s is a result of adaptive learning of importance of each time channel by using a gating mechanism.

[0024] The specific operation of the step 3 is as follows: the high-dimensional features of multiple time steps obtained by the convolutional subnetwork are flattened into one-dimensional vectors at each time step, and are respectively taken as the input of each time step of the LSTM recurrent neural unit; the LSTM recurrent neural unit determines the degree of preservation of the previous sequence state through the forgetting gate, and determines the level of the received new feature information added to the memory unit through the input gate, so as to constantly update the hidden state; in this way, the LSTM recurrent neural unit constantly adjusts its internal structure, better adapts to the internal features of the meteorological sequence, and further improves the prediction effect by mining long-time dependence.

[0025] The prediction image of the present application contains more detailed information and is more sensitive to local temperature value fluctuations. The present application can effectively predict future temperature distribution, has good long-time prediction ability, and can flatten the high-dimensional features of multiple time steps obtained by the convolutional subnetwork into one-dimensional vectors at each time step, and take them as the input of each time step of the LSTM recurrent neural unit. The LSTM recurrent neural unit determines the degree of preservation of the previous sequence state through the forgetting gate, and determines the level of the received new feature information added to the memory unit through the input gate, so as to constantly update the hidden state. In this way, the LSTM recurrent neural unit constantly adjusts its internal structure, better adapts to the internal features of the meteorological sequence, and further improves the prediction effect by mining long-time dependence. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 It is a D3AT-LSTM network structure diagram of the present application;

[0027] Figure 2 It is a meteorological feature processing and fusion flowchart of the present application;

[0028] Figure 3 It is a structure diagram of the 3DCNN module of the present application;

[0029] Figure 4 It is a structure diagram of the convolutional module combined with parallel attention branches of the present application;

[0030] Figure 5 It is a structure diagram of the LSTM recurrent neural unit of the present application;

[0031] Figure 6 It is a thermal distribution diagram of the predicted temperature of each model of the present application. DETAILED DESCRIPTION

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0033] A parallel prediction method for spatiotemporal air temperature based on the D3AT-LSTM model, such as Figure 1 As shown, the network structure of the D3AT-LSTM model includes an input layer, a 3D convolutional layer with parallel attention mechanism, and an LSTM recurrent neural unit. From the input layer, through the 3D convolutional layer with parallel attention mechanism, and the LSTM recurrent neural unit to the final temperature prediction, the result of the previous layer serves as the input for the next layer. The input layer is responsible for inputting the raw data required by the network. Here, historical meteorological observation sequences processed by a sliding window are used. The data is first cleaned and normalized to provide stable and suitable input for the neural network. The meteorological data processed by the input layer is input into the 3D convolutional layer with parallel attention mechanism. It is processed by the 3D convolutional module, and a channel attention mechanism is introduced to extract high-order spatiotemporal saliency features. By fusing data from multiple adjacent stations, more spatial information is extracted and utilized. At the same time, by introducing the channel attention mechanism, important features in different time channels can be identified, allowing the network to focus more on features that have a greater impact on the prediction results. The LSTM recurrent neural unit receives the output from the convolutional layer and adjusts its internal state according to these outputs to better adapt to the internal features of the meteorological sequence. By introducing a gating mechanism, the LSTM recurrent neural unit can capture dependencies over long time spans in the sequence, thus more accurately predicting time series data such as temperature. The neural unit is responsible for handling long-term dependencies. The three work together to improve the processing power and prediction accuracy of meteorological data. The meteorological data processed by the input layer is fed into a 3D convolutional layer incorporating a parallel attention mechanism. The 3D convolutional module processes the data and introduces a channel attention mechanism to extract high-order spatiotemporal saliency features. By fusing data from multiple adjacent stations, more spatial information is extracted and utilized. Simultaneously, the introduction of a channel attention mechanism identifies important features in different time channels, allowing the network to focus more on features that significantly impact the prediction results. The process includes the following steps:

[0034] Step 1: Input Layer Processing: The historical meteorological observation sequence "X" processed by a sliding window is used as the network input. The input layer is responsible for providing the network with the raw data required by the network. Here, a historical meteorological observation sequence processed by a sliding window is used. The data is first cleaned and normalized to provide a stable and suitable input for the neural network. To enable the model to fully consider the complex temporal and spatial influences, the input layer combines the meteorological sequences of adjacent coordinate points of the target location to construct a supervoxel structure that integrates time, space, and multiple features. This supervoxel is used as the input for subsequent networks, allowing the subsequent model to more fully consider the complex internal and external spatiotemporal factors affecting temperature. The specific operations of step 1 are as follows: First, the meteorological data is cleaned, and missing data is filled in with data from the previous time step. Since meteorological features have different numerical ranges, to accelerate the convergence of network weight parameters and obtain the optimal solution, each feature element of the meteorological dataset is normalized using the linear function Min-Max-Scaling before being used as input to the neural network. The reanalysis meteorological data used has time-series data with specific spatiotemporal resolution. Before inputting it into the network, the prediction of the temperature value of a certain station will be combined with its eight spatially adjacent points to fuse meteorological features in the spatiotemporal range, which will then be used as subsequent input data. To conform to the data input format of 3DCNN, the second dimension is first expanded with channel C=1. The original meteorological data is represented as a multidimensional tensor C*W*H for each time slice, where W and H represent the number of stations and the number of features, respectively. Then, using a sliding window technique with a window size S=72, it is divided into multiple temporally overlapping feature sequences. The input features are finally represented as a four-dimensional tensor X that fuses time, space, and multiple features. MFTS (S*C*W*H), where S is the length of the input historical time series, such as Figure 2 As shown.

[0035] Step 2: Processing with a 3D convolutional layer incorporating parallel attention: The processed historical meteorological observation sequence "X" from Step 1 is input into a 3DCNN module incorporating a parallel attention branch. It is processed through a 3D convolutional module, and a channel attention mechanism is introduced to extract high-order spatiotemporal saliency features. The meteorological data processed by the input layer is input into a 3D convolutional layer incorporating a parallel attention mechanism. It is processed through a 3D convolutional module, and a channel attention mechanism is introduced to extract high-order spatiotemporal saliency features. The specific operation of Step 2 is as follows: For X obtained from meteorological feature supervoxel reconstruction... MFTS X MFTS The supervoxel structure, which incorporates time, space, and multiple meteorological variables and has been processed by the input layer, is used as the input to subsequent networks. MFTS In all T Tot Each time slice contains a W*H*C pixel feature matrix in an image-like format, X MFTSThe spatial correlation and the temporal correlation information are fused.

[0036] The model can fully exploit the inherent spatiotemporal characteristics, and is a key to improve the prediction accuracy of air temperature. part part Tot The 3DCNN module can capture the features in the space and depth (time) dimensions by sliding convolution operation in three dimensions, which overcomes the shortcomings of the 2DCNN module.

[0037] The 3DCNN module is used to perform spatiotemporal convolution operation on the "cube" tensor X MFTS , which is stacked by multiple continuous feature matrices, to capture the deep-level change law in the time and space dimensions, as shown in Figure 3 The output of the 3DCNN module each time extracts the local continuous spatiotemporal information in the previous layer to obtain high-order spatiotemporal features with finer granularity.The SENET parallel channel attention branch is introduced into the 3DCNN module to evaluate the importance of the time channel.In order to ensure that the attention branch can comprehensively analyze the importance difference between different time points in the sequence, the range of the sub-domain input to the attention module is expanded, that is, the length of the convolution module input sequence l c is extended by 1 / 2 l c on the left and right to obtain l a , and l a = 2l c , as shown in Figure 4 , so that the importance of different time channels can be comprehensively decided.

[0038] The SENET channel attention branch completes the significance evaluation on the time channel through the compression process Squeeze and the scaling process Excitation. The F Squeeze function performs global average pooling in the spatial dimension to compress the features in each time slice into a real number. The F Excitation function uses a gating mechanism to adaptively learn the importance of each time channel.

[0039] The F Squeeze function and the F Excitation function are existing function formulas in the SENET network structure.

[0040]

[0041] Wherein, u is a two-dimensional matrix, H is high, W is wide, and z is the result of performing global average pooling on the features in the spatial dimension.

[0042] s = F Excitation (z, W) = σ(W2δ(W1z))

[0043] Wherein, sigma is a sigmoid function, delta is a ReLU function, W1 is a dimension reduction layer parameter, W2 is a dimension increase layer parameter, and s is the result of adaptively learning the importance of each time channel using a gating mechanism.

[0044] Step 3: Application of LSTM recurrent neural unit: sequentially input the data of each time node into the LSTM recurrent neural unit to capture the coarse-grained dependency relationship under long time sequence of each node; the LSTM recurrent neural unit receives the output from the convolutional layer and adjusts its internal state according to the output to better adapt to the internal features of the meteorological sequence. The convolutional network combined with the attention mechanism extracts the local spatiotemporal features of the meteorological sequence, and it is important to mine the global long-time dependency relationship of the meteorological sequence for temperature prediction; the recurrent neural network models the context information to obtain the long-time dependency of the input features; the LSTM prevents memory loss and gradient dispersion caused by too long step length by introducing a gating mechanism, effectively processing the dependency relationship of long sequence data; the specific operation of step 3 is as follows: the high-dimensional features of multiple time steps obtained by the convolutional subnet are flattened into one-dimensional vectors at each time step and are input into the LSTM recurrent neural unit at each time step respectively; the LSTM recurrent neural unit determines the degree of preservation of the previous sequence state through the forget gate and determines the level of new feature information accepted into the memory cell through the input gate, so as to continuously update the hidden state; in this way, the LSTM recurrent neural unit continuously adjusts its internal structure to better adapt to the internal features of the meteorological sequence, so as to mine long-time dependency relationship and further improve the prediction effect, as shown in Figure 5 .

[0045] Step 4: Temperature prediction: input the high-dimensional vector fused with different dimensional features into the fully connected layer to finally obtain the numerical value of the predicted temperature. That is, input the high-dimensional vector fused with different dimensional features processed by the LSTM recurrent neural unit into the fully connected layer to finally obtain the numerical value of the predicted temperature.

[0046] In summary, the application first uses the historical meteorological observation sequence "X" processed by the sliding window as the input of the network. Then, the meteorological data is input into the 3DCNN module combined with the parallel attention branch, processed by the three-dimensional convolution module, aiming to exert its short sequence feature abstraction capability, considering that in the continuous time sequence, the characteristics of abnormal fluctuations and the importance of features in the adjacent time of the prediction stage far exceed the stationary stage, and the standard 3DCNN module is difficult to depict this feature, therefore, the channel attention mechanism is introduced to process the importance difference between short sequence features, all input features are automatically learned according to the loss loss Feature weights are multiplied with the output of the CNN module element by element, so as to pay attention to specific time channels, realize the extraction of high-order spatio-temporal saliency features, on the basis of reducing information redundancy, convert the abstracted fine-grained data into a feature vector of multiple time steps. The data of each time node is input into the LSTM recurrent neural unit in turn to capture the coarse-grained dependency relationship under each node long time sequence. Finally, the high-dimensional vector fused with different dimensional features is transmitted into the full connection layer, and the numerical value of the predicted air temperature is obtained.

[0047] The specific implementation is as follows: using the ERA5 dataset, the time coverage is from 1979 to now, the spatial resolution is 0.25°*0.25°, the time resolution is 1h, the Yellow River Basin (95°E-112°E, 32°N-42°N) is selected as the research area, and the data set from 2018 to 2023-5 under the standard pressure layer of 1000hPa is selected. The meteorological data from 2018 to 2022 is used as the training set, and the meteorological data of 2023 is used as the test set, the past 144 hours (six days) of meteorological sequence is input, and the air temperature in the future 24h is predicted.

[0048] To increase the persuasiveness of the experimental results and the prediction effect difference of different positions in the observation space, five coordinate points representing different positions in the study are selected, which are sequentially recorded as Station1 (34.5 °N, 103 °E), Station2 (34.5 °N, 111 °E), Station3 (37 °N, 107 °E), Station4 (39.5 °N, 103 °E), and Station5 (39.5 °N, 111 °E) to roughly depict the prediction of the model in the entire study area. To verify the effectiveness of each module in the D3AT-LSTM model, the following ablation experiments are performed by controlling a single variable: 1) the 1D CNN-LSTM network inputs a single temperature variable of the target point at each time; 2) the 1D CNN-LSTM inputs the historical meteorological sequence of the target point and the adjacent eight coordinate points to verify the effectiveness of the hyper voxel unit in improving the prediction results; 3) the 1D CNN-LSTM verifies the effectiveness of the local convolution in the time dimension; and the D3AT-LSTM verifies the effectiveness of the parallel attention module. The 24-hour average accuracy obtained by the ablation experiment is shown in Table 1.

[0049] Table 1: 24-hour average accuracy of ablation experiment

[0050]

[0051] It can be seen from the observation of the experimental results that the prediction results of the five coordinate points do not change much, and have good applicability in different positions, which is also the result of modeling in the entire spatial range. Station3 is located in the middle of the study area, and the temperature also remains at an intermediate level, so the trained model has strong generalization in this position, which also shows that the addition of a large number of adjacent coordinate points can improve the prediction accuracy.

[0052] To verify the effectiveness of the D3AT-LSTM model, the present application is compared with the latest and most classic prediction methods in the temperature prediction field, including Random Forest, LSTM-AE, Transforme, ConvLSTM, and TCN-LSTM and D3AT-LSTM. RF represents a case of traditional machine learning algorithms, LSTM-AE, Transformer represents a single deep learning network, ConvLSTM and TCN-LSTM belong to hybrid networks, which also helps to compare the performance of different methods from multiple angles.

[0053] The spatial distribution of the prediction value of the present application for the next 24 hours and the real temperature value is visualized by a heat map, as shown in Figure 6As shown in the figure, it can be seen that the predicted image of the application contains more detailed information and is more sensitive to local temperature value fluctuations compared with other prediction methods. It can be seen that the application can effectively predict the future temperature distribution and has good long-term prediction ability.

Claims

1. A spatiotemporal air temperature parallel prediction method based on a D3AT-LSTM model, characterized in that: The network structure of the D3AT-LSTM model comprises an input layer, a three-dimensional convolution layer combined with a parallel attention mechanism and an LSTM recurrent neural unit; the meteorological data processed by the input layer is input to the three-dimensional convolution layer combined with the parallel attention mechanism, is processed through the three-dimensional convolution module, and high-order spatiotemporal significant feature extraction is realized by introducing the channel attention mechanism; more spatial information is extracted and utilized by fusing the data of multiple adjacent stations, and important features on different time channels are identified by introducing the channel attention mechanism, so that the network focuses on the features that have a great influence on the prediction result; the method comprises the following steps: Step 1: processing of the input layer: using the historical meteorological observation sequence "X" processed by a sliding window as the input of the network; the input layer is responsible for inputting the original data required by the network, and the original data is the historical meteorological observation sequence processed by a sliding window; the data is first cleaned and normalized to provide stable and suitable input for the neural network; in order to make the model consider the complex influence features in time and space, the input layer combines the meteorological sequence of the adjacent coordinate points of the target position to construct a hyper voxel structure integrating time, space and multiple features, which is used as the input of the subsequent network, so that the subsequent model pays attention to the complex spatiotemporal elements of the internal and external environment affecting the air temperature; Step 2: processing of the three-dimensional convolution layer combined with the parallel attention mechanism: the historical meteorological observation sequence "X" processed in step 1 is input to the 3DCNN module combined with the parallel attention branch, processed through the three-dimensional convolution module, and high-order spatiotemporal significant feature extraction is realized by introducing the channel attention mechanism; the meteorological data processed by the input layer is input to the three-dimensional convolution layer combined with the parallel attention mechanism, processed through the three-dimensional convolution module, and high-order spatiotemporal significant feature extraction is realized by introducing the channel attention mechanism; Step 3: application of the LSTM recurrent neural unit: the data of each time node is sequentially input to the LSTM recurrent neural unit to capture the coarse-grained dependency relationship under the long time sequence of each node; the LSTM recurrent neural unit receives the output from the three-dimensional convolution layer in step 2 and adjusts its internal state according to the output to adapt to the internal features of the meteorological sequence; the three-dimensional convolution module combined with the attention mechanism in step 2 extracts the local spatiotemporal features of the meteorological sequence, and the global long-time dependency relationship of the meteorological sequence is important for air temperature prediction; the recurrent neural network models the context information to obtain the long-time dependency of the input features; the LSTM recurrent neural unit prevents memory loss and gradient dispersion caused by too long step length by introducing the gating mechanism, and effectively processes the dependency relationship of long sequence data; Step 4: air temperature prediction: the high-dimensional vector fused with different dimensional features processed by the LSTM recurrent neural unit in step 3 is transmitted to the full connection layer, and finally the numerical value of the predicted air temperature is obtained.

2. The spatiotemporal temperature parallel prediction method based on the D3AT-LSTM model according to claim 1, characterized in that: The specific operation of step 1 is as follows: first, the weather data is cleaned, and the missing data is filled with the data of the previous time; since the weather features have different numerical ranges, in order to accelerate the convergence of network weight parameters and obtain the optimal solution, the data is normalized as the input of the neural network before each feature element of the weather data set is normalized using the linear function Min-Max-Scaling; The reanalysis meteorological data used is time series data with specific spatial and temporal resolution. Before inputting into the network, the prediction of the temperature value of a certain station will be combined with the eight adjacent points in space to fuse the meteorological characteristics in the space-time range as the subsequent input data. In order to conform to the data input format of the 3DCNN module, first, the channel C=1 is expanded in the second dimension, and the original meteorological data is represented as a multi-dimensional tensor of C*W*H in each time slice, W and H representing the number of stations and the number of features respectively. Then, using the sliding window technique with a window size S=72, it is divided into multiple overlapping feature sequences in time, and the input feature is finally represented as a four-dimensional tensor X that fuses time, space and multi-features MFTS (S*C*W*H), S is the length of the input historical time series.

3. The D3AT-LSTM model-based spatiotemporal air temperature parallel prediction method according to claim 1, characterized in that: The specific operation of step 2 is as follows: For X obtained by supervoxel reconstruction of meteorological features... MFTS X MFTS The supervoxel structure, processed by the input layer and incorporating time-space-multiple meteorological variables, is used as the input to subsequent networks; X MFTS In all T Tot Each time slice contains a W*H*C pixel feature matrix in an image-like format, X MFTS It integrates spatial correlation and temporal correlation information; it uses a 3DCNN module to process the "cubic" tensor X, which is composed of multiple consecutive feature matrices stacked together. MFTS Spatiotemporal convolution operations are performed to simultaneously capture deep-seated patterns of change in both temporal and spatial dimensions. Each output of the 3DCNN module extracts local, continuous spatiotemporal information from the previous layer, resulting in finer-grained high-order spatiotemporal features. A parallel channel attention branch, SENET, is introduced into the 3DCNN module to evaluate the importance of each temporal channel. Furthermore, to ensure the attention branch can more comprehensively analyze the differences in importance across different time points in the sequence, the subdomain of the attention module's input is expanded; that is, the input sequence length l of the convolution module is increased. c Expand by 1 / 2 l on each side based on the existing structure. c The length of l is obtained a , let l a =2l c This enables them to make comprehensive decisions regarding the importance of different time channels; The channel attention branch of SENET performs temporal saliency evaluation through a compression process (Squeeze) and a scaling process (Excitation). For the input hypervoxel information, F... Squeeze The function performs global average pooling across the spatial dimension, compressing the features at each time slice into a single real number, F. Excitation The function uses a gating mechanism to adaptively learn the importance of each time channel; F Squeeze Functions and F Excitation The function is a function formula in the SENET network structure; Wherein, u is a two-dimensional matrix, H is high, W is wide, and z is the result of performing global average pooling on the spatial dimension of the feature. s = F Excitation (z,W) = σ(W2δ(W1z)) Wherein, sigma is a sigmoid function, delta is a ReLU function, W1 is a dimension reduction layer parameter, W2 is a dimension increase layer parameter, and s is the result of adaptively learning the importance of each time channel using a gating mechanism.

4. The spatiotemporal temperature parallel prediction method based on the D3AT-LSTM model according to claim 1, characterized in that: The specific operation of step 3 is as follows: the high-dimensional features of multiple time steps obtained by the convolutional subnet are flattened into one-dimensional vectors at each time step, which are respectively input into each time step of the LSTM recurrent neural unit; the LSTM recurrent neural unit determines the degree of preservation of the previous sequence state through the forget gate, and determines the level of new feature information received into the memory unit through the input gate, so as to continuously update the hidden state; in this way, the LSTM recurrent neural unit continuously adjusts its internal structure to better adapt to the internal features of the weather sequence, so as to further improve the prediction effect by mining long-time dependence relationship.

Images