Implementation method of radar echo extrapolation model based on deep spatio-temporal fusion network

Abstract

The application discloses an implementation method of a radar echo extrapolation model based on a deep space-time fusion network, and comprises the following steps: S1, radar image data is processed to obtain radar sequence data; S2, a space-time fusion neural network model is constructed by integrating a time difference network and a space-time trajectory network, difference operation in the time difference network is used to extract difference features of adjacent two radar images, and the difference features are transmitted to a higher layer, are abstracted layer by layer, and certainty in an uncertain space-time change process is extracted; meanwhile, the space-time trajectory network is used to learn a transient change part; S3, the space-time fusion neural network model is trained by using the radar sequence data, and the training is stopped until a condition is met, and the model is saved; and S4, a future movement trend of a cloud picture is predicted by using the final model obtained in the step S3. The space-time fusion neural network model of the application fuses the movement trend and the transient change, and long-short time correlation memory is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to a radar echo extrapolation implementation method, and particularly relates to a radar echo extrapolation implementation method based on a space-time fusion network. BACKGROUND

[0002] Short-term precipitation forecast has great uncertainty, chaos and highly nonlinear complex characteristics, and has become a challenging worldwide problem. Current short-term precipitation forecast is mainly realized through radar echo extrapolation. Traditional radar echo extrapolation methods include centroid tracking method, cross-correlation method and optical flow method. Among them, the centroid tracking method is only suitable for tracking strong convective thunderstorm cells; the cross-correlation method has the defect that the extrapolated radar echo is distorted as the extrapolation time increases; the application premise of the optical flow method needs to be based on three assumptions of constant brightness, time continuity and spatial consistency, and the three assumptions are not true in the field of radar echo extrapolation.

[0003] With the development of big data and artificial intelligence, deep learning based methods have the ability to model highly nonlinear systems. Through learning the weather evolution law from massive weather data, more accurate prediction can be achieved. In 2015, Shi et al. proposed Convolutional LSTM (ConvLSTM) by optimizing the fully connected operation of the gate in the traditional Long Short-Term Memory (LSTM) to convolution operation. ConvLSTM inherits the advantages of LSTM time series learning and the ability of convolution structure to extract spatial features, and outperforms traditional radar echo extrapolation methods in accuracy. Since then, researchers have used ConvLSTM based methods to solve the problem of short-term precipitation prediction. However, the locally invariant convolution structure in ConvLSTM is not suitable for processing radar images with local variation characteristics. To improve the problem that ConvLSTM is not suitable for radar echo extrapolation, Shi et al. further proposed Trajectory Gated Recurrent Unit (TrajGRU). This model proposes a dynamic connection form of convolution GRU with time-varying convolution structure in convolution recurrent network, which can capture the non-rigid motion characteristics of radar echo sequence. TrajGRU is more clear in details of short-term prediction than ConvLSTM. In addition, since the spatial dimension of the input information in ConvLSTM is encoded layer by layer, and the time dimension is independent of each other, the lowest layer of recurrent unit will ignore the highest layer of time information in the previous time step. To overcome the defect that the memory state in ConvLSTM is limited in the same layer LSTM, Wang et al. proposed Predictive Recurrent Neural Network (PredRNN). PredRNN passes the top layer information of the previous time step to the bottom layer of the current time step in a zigzag shape, so that the same memory unit learns spatial representation and temporal variation at the same time, and considers the interaction between high-level abstract information and low-level detailed information. PredRNN further improves the accuracy of prediction.

[0004] Although TrajGRU improves ConvLSTM, TrajGRU has a more obvious echo disappearance problem. TrajGRU points out that the radar echo has obvious non-rigid motion characteristics, and proposes a model for capturing instantaneous information, but ignores the introduction of "new information" at each time step caused by non-rigid motion characteristics, so that the statistical characteristics of the radar echo are constantly changing over time, causing great uncertainty in the long-term future image, that is, high-order non-stationarity. Therefore, in order to achieve more stable prediction and prolong the prediction time, it is necessary to model this high-order non-stationarity. However, the current model basically does not have the ability to model this high-order non-stationarity information, resulting in serious degradation of the extrapolation result. SUMMARY

[0005] The purpose of the present application is to provide a radar echo extrapolation method based on a deep spatio-temporal fusion neural network (STUNNER). In order to alleviate the problems of short extrapolation time and extrapolation ambiguity in the current method, the prediction problem is divided into predicting long-term trends and instantaneous changes. Two sub-networks extract long-term trend information and short-term instantaneous information, respectively, and realize stable extrapolation through two-way fusion to improve the accuracy of extrapolation.

[0006] Technical scheme: The implementation method of the radar echo extrapolation model of the present application comprises the following steps:

[0007] S1, processing the radar image data to obtain radar sequence data;

[0008] S2, constructing a spatio-temporal fusion neural network model by integrating a time difference network TDN and a spatio-temporal trajectory network STN, using difference operation in the time difference network to extract difference features of adjacent two radar images, and passing the difference features to higher layers for layer-by-layer abstraction to extract the deterministic part in the uncertain spatio-temporal change process; and using the spatial trajectory network to learn the transient change part;

[0009] S3, using the radar sequence data to train the spatio-temporal fusion neural network model until the condition is met, then stopping the training and saving the model;

[0010] S4, using the final model obtained in step S3 to predict the future motion trend of the cloud image.

[0011] Further, the specific implementation steps of step S2 are as follows:

[0012] S21, constructing a spatio-temporal fusion neural network model by cross-fusing a time difference network TDN and a spatio-temporal trajectory network STN;

[0013] S22, learning long-term motion trends of the image sequence by constructing a time difference network TDN;

[0014] S23, learning long-term trend information of the image sequence by constructing a recurrent unit of the time difference network;

[0015] S24, learning short-term transient changes of the image sequence by constructing a space-time trajectory network STN;

[0016] S25, memorizing short-term trajectory information of the image sequence by constructing a recurrent unit of the space-time trajectory network.

[0017] Further, in step S21, the structure of the space-time fusion neural network model is: the bottom layer uses an ST-LSTM recurrent unit, and other layers are STUNNER-blocks formed by stacking a time difference TDiff-LSTM and a space-time trajectory STraj-LSTM; the input of the ST-LSTM recurrent unit is the original radar image, and the abstract features are extracted and the space-time sequence is encoded into the network, and the expression is as follows:

[0018]

[0019] wherein, is the input radar image, is the hidden state, is the time sequence state, is the space-time memory state; the superscript l represents the number of layers, and the total number of layers is represented by L; the subscript t represents the time step;

[0020] The STUNNER-block is used to extract and integrate motion trends and trajectory changes, and the expression is as follows:

[0021]

[0022] wherein, is the difference hidden state, is the difference time sequence state.

[0023] Further, in step S22, the bottom layer of the time difference network TDN is an ST-LSTM recurrent unit, and the recurrent unit of other layers is a time difference TDiff-LSTM;

[0024] The stacked time difference TDiff-LSTM of the second layer of the time difference network TDN takes the difference of the space-time features extracted by the bottom layer ST-LSTM recurrent unit as input.

[0025] Further, in step S23, the time difference TDiff-LSTM extracts difference features and The stationary trend information in the radar image sequence is learned, and the expression is as follows:

[0026]

[0027] The expression of the stacked time difference differential TDiff-LSTM is as follows:

[0028]

[0029]

[0030]

[0031]

[0032]

[0033]

[0034]

[0035]

[0036] Wherein, the concat(.,.) function represents concatenation in the channel dimension, the symbol sigma is a sigmoid function; the symbols and represent Hadamard product and convolution operation respectively; g t , i t , f t and o t are update gate, input gate, forget gate and output gate respectively; W is a convolution with a convolution kernel size of 5*5, b is a bias term, and different combinations of subscripts of W and b represent different parameters; W 1×1 is a convolution with a convolution kernel size of 1*1;

[0037] All gates g t , i t , f t and o t of the time difference differential TDiff-LSTM of the second layer of the time difference network TDN are updated through the difference between two continuous hidden states at the bottom layer and the difference hidden state at the same layer;

[0038] All gates of the last two layers of the time difference network TDN are updated by the difference state of the time difference differential TDiff-LSTM of the last layer and the difference hidden state at the same layer.

[0039] Further, in step S24, the space-time trajectory network STN has four layers, and the bottom layer shares the same ST-LSTM recurrent unit with the time difference network TDN.

[0040] Further, in step S25, the space-time trajectory STraj-LSTM is a recurrent unit of the space-time trajectory network STN, and the stacked space-time trajectory STraj-LSTM uses locally connected varying convolution to capture transient changes and fuse with the trend information extracted by the time difference TDiff-LSTM ;

[0041] The space-time trajectory STraj-LSTM uses the difference hidden state of the time difference TDiff-LSTM in the same STUNNER-block the hidden state of the previous layer the space-time memory state and the hidden state of the same layer the time sequence state As input, the expression is as follows:

[0042]

[0043] Further, in step S25, the locally invariant convolution in the ST-LSTM recurrent unit is optimized to a position-varying connection structure using the "warp" operation, and the expression of the space-time trajectory STraj-LSTM is as follows:

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] Wherein [.,.] represents splicing in the channel dimension; * and represent convolution and Hadamard operation respectively; W is a convolution with a convolution kernel size of 5*5, b is a bias term, and different combinations of subscripts of W and b represent different parameters;

[0055] g' t , i' t , f' t , o t is the selection gate, the input gate, the forget gate and the output gate of the spatiotemporal memory state 1 1×1 is a convolution with a convolution kernel size of 1*1.

[0056] Compared with the prior art, the present application has the following remarkable effects:

[0057] 1. The present application uses the difference operation in the time difference subnetwork of STUNNER to extract the difference features of adjacent two images, and passes the difference features to the high layer, abstracts layer by layer, learns the deterministic trend part in the uncertain spatiotemporal change process, so that the future motion trend can be predicted; through this way of extracting features layer by layer, the high-order non-stationary information in the sequence is modeled to learn the long-term motion trend of the sequence;

[0058] 2. The present application applies the idea of dynamic convolution in the image to the convolution recurrent network in the spatiotemporal trajectory subnetwork, because the fixed rectangular parameter structure and the sequential sampling position of the ordinary convolution in the convolution recurrent network are not suitable for processing the non-rigid motion of the radar echo, so the idea of dynamic convolution is used to improve the ordinary convolution in the convolution recurrent network, so that the parameter sampling position of the convolution at each time step can be dynamically changed according to the characteristics of the radar echo, so that the convolution sampling position conforms to the semantic characteristics of the radar image; different sampling positions at different time steps constitute a kind of dynamic connection in time, and through this way, the instantaneous change of the non-rigid motion of the radar image is captured;

[0059] 3. The present application realizes the fusion of long-term trend and transient change by cross-connecting the time difference network and the spatiotemporal trajectory network, and the two complement each other, realizing stable trend prediction while increasing the details of each time step by using the captured instantaneous change; this two-way fusion structure of STUNNER, on the one hand, reduces the burden of the model learning long-term spatiotemporal sequence, and on the other hand, makes the two parts focus on the learning of motion trend and transient change information respectively, improves the extrapolation efficiency, alleviates the serious degradation problem of extrapolation result, prolongs the extrapolation time, and improves the extrapolation accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is the total flowchart of the present application;

[0061] Figure 2A structural diagram of a deep deep-space fusion network model of the present application;

[0062] Figure 3 A structural diagram of a time difference network (TDN);

[0063] Figure 4 A structural diagram of a TDiff-LSTM;

[0064] Figure 5 A structural diagram of a spatio-temporal trajectory network (STN);

[0065] Figure 6 A structural diagram of an STraj-LSTM;

[0066] Figure 7 A structural diagram of a Diff-M in the STraj-LSTM. DETAILED DESCRIPTION

[0067] The present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0068] In the radar echo extrapolation task, the time resolution of the radar image is 6 minutes. Since short-term precipitation prediction needs to extrapolate the echo distribution of the next 2 hours based on the previous 1 hour, it is necessary to extrapolate the next 20 radar images based on the previous 10 radar images. The extrapolation model currently studied basically does not have the ability to model the high-order non-stationary information of the radar image sequence, resulting in the problems of short extrapolation time and blurred extrapolation results. The high-order non-stationarity of the radar image sequence is due to the accumulation, deformation or dissipation of radar echoes, which makes the overall transformation trend more complex, and it is extremely challenging to predict the future for a long time. In view of the serious problem of echo degradation in the extrapolation process, the present application proposes a deep deep-space fusion network model (STUNNER) for radar echo extrapolation, which can model the spatio-temporal correlation and high-order non-stationarity of data. The STUNNER adopts a two-way spatio-temporal fusion strategy to reduce the burden of learning high-dimensional information while extracting and fusing spatio-temporal signals. Specifically, the radar echo extrapolation model of the present application cross-integrates a temporal differencing network (TDN) and a spatio-temporal trajectory network (STN), wherein the idea of the temporal differencing network comes from the Kramer Decomposition theorem: any non-stationary process can be decomposed into a deterministic time-varying polynomial plus a zero-mean random term. Therefore, the temporal differencing network extracts the deterministic part of the radar sequence by introducing the idea of difference to learn the motion trend of the radar echo, and realizes the modeling of the high-order non-stationarity of the radar data; the spatio-temporal trajectory network optimizes the ordinary convolution in the spatio-temporal long-short term memory network (ST-LSTM) into dynamic convolution, so that the convolution in the ST-LSTM is a dynamic connection structure that changes with time, to extract the transient changes in the radar image. The STUNNER fuses the motion trend and the transient change, and realizes the long-short time correlation memory.

[0069] Since the long-term change of the radar image is accumulated by the change of each time step, the extrapolation problem can be divided into two parts of predicting long-term trend and transient change. According to the Kramer decomposition theorem, the difference operation is introduced, and the time-varying polynomial is converted into a constant based on the difference operation, so that the deterministic trend part is predictable. For the transient change part, the idea of dynamic convolution is introduced into the convolution recurrent neural network to realize the capture of the trajectory. At the same time, the network architecture of two-way fusion can fuse the two parts, realize the modeling of the high-order non-stationarity of the radar image, make the model more stable and accurate to predict the future for a long time, and alleviate the problem of serious echo degradation.

[0070] As Figure 1 The overall flowchart of the present application is shown, and the implementation steps of the deep deep space fusion network model of the present application are as follows:

[0071] Step one, construct a two-way fusion spatio-temporal sequence prediction network structure.

[0072] The two-way fusion spatio-temporal sequence prediction network structure (i.e. deep deep space fusion network model (STUNENR)) realizes the capture, fusion and transmission of long-term trend and transient change by cross-fusing the temporal differencing network (TDN) and the spatio-temporal trajectory network (STN), and the overall structure of the spatio-temporal sequence prediction network is as follows: Figure 2The bottom layer of the spatio-temporal sequence prediction network uses an ST-LSTM recurrent unit proposed in PredRNN, and other layers are formed by stacking TDiff-LSTM and STraj-LSTM to form an STUNNER-block (as shown in the dashed box in FIG. 1). Figure 2 The TDiff-LSTM and STraj-LSTM are the recurrent units of TDN and STN, respectively. The input of the bottom layer ST-LSTM is the original radar image, and the ST-LSTM is responsible for extracting abstract features and encoding the spatio-temporal sequence into the network, as shown in equation (1):

[0073]

[0074] wherein, is the input radar image, is the hidden state, is the time series state, is the spatio-temporal memory state, the superscript l represents the layer number, the total number of layers is represented by L, and the subscript t represents the time step. When the layer number is l = 1, the spatio-temporal memory state of the top layer is input to the bottom layer of the next time step, that is,

[0075] The recurrent unit of other layers of STUNNER is STUNNER-block, which is used to extract and integrate motion trends and trajectory changes, as shown in equation (2):

[0076]

[0077] wherein, is the difference hidden state, is the difference time series state, the superscript l represents the layer number, and the subscript t represents the time step.

[0078] The input of the second layer STUNNER-block of STUNENR is the spatio-temporal features extracted by the ST-LSTM and the difference of the spatio-temporal features extracted by the ST-LSTM wherein is the input of the STraj-LSTM, is the input of the TDiff-LSTM. Since there is no difference of the features extracted by the ST-LSTM at the first time, the recurrent unit of the first time step of the second, third, and fourth layers of STUNNER is only the STraj-LSTM, as shown in Figure 2 ​

[0079] Step two, constructing a temporal difference network (TDN) to learn long-term motion trends of image sequences

[0080] TDN introduces a difference operation into spatio-temporal sequence prediction, making the deterministic part predictable, and the TDN structure is shown in Figure 3 . The TDiff-LSTM of the second layer of TDN takes the difference of the spatio-temporal features extracted by the bottom ST-LSTM as input . According to the Kramer decomposition theorem, the difference is used to extract the predictable stationary information in non-stationary processes. The difference operation is used in the first layer of TDN to learn the long-term stationary trends in the sequence by abstracting the difference features layer by layer. The existence of independent TDN not only ensures that STUNNER focuses on learning the motion trends of radar echoes from the extraction of stationary information by difference operation, but also to some extent relieves the pressure of the model to learn high-dimensional information of spatio-temporal sequences. The bottom layer of TDN is ST-LSTM, and the other layers of recurrent units are TDiff-LSTM.

[0081] Step three, designing the TDiff-LSTM recurrent unit of TDN to learn long-term trend information

[0082] TDiff-LSTM is the recurrent unit of TDN, and the stacked TDiff-LSTM abstracts the difference features layer by layer and to learn the stationary trend information in the radar image sequence, as shown in equation (3).

[0083]

[0084] The structure of TDiff-LSTM is shown in Figure 4 , and the formula is shown in equation (4).

[0085]

[0086]

[0087]

[0088]

[0089] where the concat(.,.) function represents concatenation in the channel dimension, and the symbol σ is the sigmoid function; the symbols ⊙ and * represent Hadamard product and convolution operation, respectively; g t , i t , f t and o tThese are the update gate, input gate, forget gate, and output gate, respectively. W represents a 5×5 convolution kernel, and b is the bias term. Different combinations of the indices of W and b represent different parameters, such as W... hg b represents the convolutional network parameters applied to the hidden state h in the update gate g. g To update the bias of gate g, all gates g of the TDiff-LSTM in layer 2 of TDN are... t i t f t and o t By the difference between two consecutive hidden states at the bottom layer Differential hidden states at the same level Update. All gates in the last two layers of TDN are derived from the difference states of the previous layer's TDiff-LSTM. Differential hidden states at the same level Update. TDN utilizes temporal continuity to extract long-term time-difference features and abstracts layer by layer to learn long-term trends in spatiotemporal sequences. From Figure 3 It can be seen that the differential cellular state of TDN It is independent within TDN and is only responsible for learning long-term trends.

[0090] Step 4: Build a spatiotemporal trajectory network (STN) to learn short-term transient changes in image sequences.

[0091] The network structure of STN is as follows: Figure 5 As shown, the network has four layers. The bottom layer of STN shares the same ST-LSTM as TDN. STN is responsible for capturing instantaneous changes in the spatiotemporal sequence and fusing it with the trend information learned by TDN at each time step. Based on the deterministic long-term trend, it adds supplementary or corrective details of instantaneous changes to the predicted echo, achieving stable and accurate long-term prediction.

[0092] Step 5: Design the STN cyclic unit Straj-LSTM to memorize short-term trajectory information.

[0093] Straj-LSTM is a recurrent unit of STN. Stacked Straj-LSTMs use convolutions with varying local connectivity to capture transient changes and combine them with trend information extracted by TDiff-LSTM. Fusion. Straj-LSTM uses the differential hidden states of TDiff-LSTM within the same STUNNER-block. Hidden state of the previous layer Spatiotemporal memory state and the hidden state of the same layer Timing State As input, as shown in formula (5).

[0094]

[0095] The structure of STraj-LSTM is shown in Figure 6

[0096] STraj-LSTM designs a virtual complex "forget gate", which is replaced by a Differencing Memory (Diff-M) module shown in Figure 7 to prevent the premature saturation of the forget gate in the recurrent network, which will cause the recurrent network to no longer consider the subsequent input, making the entire inference a linear inference process that only considers the first few time step inputs. The formula of the Diff-M module is shown in (6):

[0097]

[0098] wherein the symbol sigma is a sigmoid function, the symbols and represent Hadamard product and convolution operation respectively, g t , i t , f t and o t are update gate, input gate, forget gate and output gate respectively, W is a convolution with a kernel size of 5x5, b is a bias term, and different combinations of subscripts of W and b represent different parameters. represents the horizontally transmitted memory unit in the Diff-M module, and the motion trend T is memorized through the complex virtual "forget gate".

[0099] In order to realize the extraction of instantaneous changes, it is necessary to capture the non-rigid motion trajectory of the radar echo. The invention uses the "warp" operation to optimize the local invariant convolution in ST-LSTM into a position change connection structure, which is achieved by migrating the dynamic convolution operation into ST-LSTM. The formulas of STraj-LSTM are shown in formulas (7)-(16).

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107] ​

[0108]

[0109]

[0110] where [.,.] denotes concatenation in channel dimension; *, ⊙ represent convolution and Hadamard operation respectively; W is a convolution with kernel size 5x5, b is the bias term, and different combinations of the subscripts of W and b represent different parameters. The warp operation can dynamically change the sampling position according to the radar image features. Then the motion trend T is fused with i t ⊙g t to obtain the time sequence memory as shown in equation (10). g' t , i' t , f' t are the selection gate, input gate and forget gate of the spatio-temporal memory state respectively. The three gates g' t , i' t , f' t are updated through the spatio-temporal memory state and the hidden state as shown in equations (11-13). The STraj-LSTM unit fuses the two types of memory and through the shared output gate o t to realize seamless memory fusion as shown in equation (15). Finally, the final output is obtained by the output gate o t and where W 1×1 is a convolution with kernel size 1x1. The deep space fusion network of the application can effectively simulate the motion trend and transient change in the spatio-temporal sequence through the two-way fusion structure of the two networks.

[0111] The implementation process of the warp operation is introduced as follows:

[0112] The conventional convolution in the ST-LSTM recurrent unit performs inner product mapping on the rectangular neighborhood of the input feature , where

[0113]

[0114] Equation (17) defines a normal convolution with kernel size 3x3, dilation 1, assuming all inputs are 0 and all reset gates are 1. y1(p0) is the convolution output value of point p0, p n is the enumeration of .

[0115]

[0116] The position transformation convolution kernel in STraj-LSTM is... This is achieved through the warp function. The coordinates of the currently required inner product are offset to the semantically correct position p0+p. n +Δp. The offset Δp is obtained through a convolutional neural network. The result is learned and therefore is a dynamic output, as shown in equation (18):

[0117]

[0118] A convolutional neural network consists of three 3×3 convolutional layers, each operating on a different frequency. The concatenation of the outputs from the first two convolutional layers is used as input. Due to the biased position p0+p n +Δp is not necessarily an integer value, so bilinear interpolation is needed to obtain the position p0+p after resetting the bias. n +Δp The value is shown in formula (19):

[0119]

[0120] Where p0+p n +Δp can be divided into an integer part and a fractional part, where q represents the value of the integer part and p represents the value of the fractional part; q x q y represent the projections of q onto the x-axis and y-axis in the Cartesian coordinate system, respectively; p x p y Let and represent the projections of p onto the x-axis and y-axis in the Cartesian coordinate system, respectively. Using the formula for bilinear interpolation, we can obtain... The value of .

[0121] Finally, Ordinary convolution is applied to perform convolution at positions that conform to the semantics of the image. Equation (20) is the output y2(p0) obtained by applying ordinary convolution W after warping, where Δp is the offset and y2 is the sampling position p after the offset. n The value of +Δp The result of applying convolution.

[0122]

[0123] The warp operation is applied to the hidden states of the recurrent network, as shown in equations (7), (8), and (15). As shown, with the dynamic change of sampling position at each time step, a dynamically connected structure is formed in the time-continuous process, realizing the capture of instantaneous changes of radar echoes.

Claims

1. An implementation method of a radar echo extrapolation model based on a deep spatio-temporal fusion network, characterized in that, The steps comprise the following: S1, processing radar image data to obtain radar sequence data; S2, constructing a space-time fusion neural network model by integrating a time difference network TDN and a space-time trajectory network STN, using difference operation in the time difference network TDN to extract difference features of adjacent two radar images, and passing the difference features to a higher layer, abstracting layer by layer, and extracting deterministic part in uncertain space-time change process; meanwhile, using the space-time trajectory network to learn the transient change part; the specific implementation steps are as follows: S21, constructing a space-time fusion neural network model by cross-fusing a time difference network TDN and a space-time trajectory network STN; S22, learning long-term motion trend of image sequence by constructing a time difference network TDN; S23, learning long-term trend information of image sequence by constructing a recurrent unit of the time difference network; S24, learning short-term transient change of image sequence by constructing a space-time trajectory network STN; S25, memorizing short-term trajectory information of image sequence by constructing a recurrent unit of the space-time trajectory network; In step S21, the structure of the space-time fusion neural network model is as follows: the bottom layer uses an ST-LSTM recurrent unit, and other layers are STUNNER-blocks formed by stacking a time difference TDiff-LSTM and a space-time trajectory STraj-LSTM; the input of the ST-LSTM recurrent unit is the original radar image, which is used to extract abstract features and encode space-time sequence into the network, and the expression is as follows: wherein, is an input radar image, is a hidden state, is a temporal state, is a spatio-temporal memory state; the superscript l denotes the number of layers, the total number of layers is denoted by L; the subscript t denotes the time step; The STUNNER-block is used to extract and integrate motion trend and trajectory change, and the expression is as follows: wherein, is a differential hidden state, is a differential timing state; S3, training the space-time fusion neural network model using radar sequence data, stopping training and saving the model when the condition is met; S4, predicting future motion trend of the cloud image using the final model obtained in step S3.

2. The implementation method of the radar echo extrapolation model based on the deep spatio-temporal fusion network according to claim 1, characterized in that, In step S22, the bottom layer of the time difference network TDN is an ST-LSTM recurrent unit, and other layers are time difference TDiff-LSTM recurrent units. A stack temporal difference (TDiff-LSTM) of a temporal difference network (TDN) layer 2 takes the difference of the spatio-temporal features extracted by the lower layer ST-LSTM recurrent unit as input. as input.

3. The implementation method of the radar echo extrapolation model based on the deep spatio-temporal fusion network according to claim 1, characterized in that, In step S23, the time-difference DifflSTM abstracts the difference features layer by layer and Learning the stationary trend information in the radar image sequence, expressed as follows: The expression of the stacked time difference TDiff-LSTM is as follows: wherein the concat(.,.) function represents concatenation in the channel dimension, the symbol σ is a sigmoid function; the symbols ⊙ and * represent Hadamard product and convolution operation respectively; g t , i t , f t and o t are update gate, input gate, forget gate and output gate respectively; W is a convolution with a kernel size of 5x5, b is a bias term, and different combinations of subscripts of W and b represent different parameters; W 1×1 is a convolution with a kernel size of 1x1. All gates g of the time-difference TDiff-LSTM of the time-difference network TDN layer 2 t i t f t and o t by the difference of two consecutive hidden states of the underlying layer and the same layer difference hidden state update; All gates of the last two layers of the time-difference network TDN are updated by the difference states of the previous layer of time-difference TDiff-LSTM and the same layer difference hidden states TDiff-LSTM 4. The implementation method of the radar echo extrapolation model based on the deep spatio-temporal fusion network according to claim 1, characterized in that, In step S24, the space-time trajectory network STN has four layers, and the bottom layer shares the same ST-LSTM recurrent unit with the time difference network TDN.

5. The implementation method of the radar echo extrapolation model based on the deep spatio-temporal fusion network according to claim 1, characterized in that, In step S25, the spatio-temporal trajectory STraj-LSTM is a recurrent unit of the spatio-temporal trajectory network STN, and the stacked spatio-temporal trajectory STraj-LSTM uses locally connected varying convolutions to capture transient changes and trends information extracted by the time difference DDiff-LSTM fusion; spatial-temporal trajectory STraj-LSTM with the differential hidden state of the time-difference TDiff-LSTM in the same STUNNER-block hidden state of the previous layer spatial-temporal memory state and hidden state of the same layer time-series state As input, the expression is as follows:

6. The implementation method of the radar echo extrapolation model based on the deep spatio-temporal fusion network according to claim 1, characterized in that, In step S25, the local invariant convolution in the ST-LSTM recurrent unit is optimized to a position change connection structure by using the "warp" operation, and the expression of the space-time trajectory STraj-LSTM is as follows: wherein [.,.] represents concatenation in the channel dimension; * and ⊙ represent convolution and Hadamard operation respectively; W is a convolution with a convolution kernel size of 5*5, b is a bias term, and different combinations of subscripts of W and b represent different parameters; g ′ t , ′ t , ′ t , t is a selection gate, an input gate, a forget gate and an output gate of a spatiotemporal memory state ; W 1×1 is a convolution with a convolution kernel size of 1*1.

Images