TG-LSTM short-term rainfall forecasting method and system in combination with attention mechanism

Abstract

The invention discloses a TG-LSTM short-term rainfall forecasting method and system combined with an attention mechanism, and the method comprises the following steps: collecting hour-level live meteorological data, and obtaining the required meteorological element data; counting meteorological element characteristics of the meso-scale system at different heights in each meteorological station; through analysis of historical meteorological data, meteorological features closely related to precipitation are selected, and a multi-dimensional feature time sequence vector is constructed; a TG-LSTM model based on an encoder-decoder architecture is constructed, and model training is completed on a data set; inputting the processed data into the trained TG-LSTM model for reasoning, and generating a short-term precipitation forecast result; and carrying out post-processing, anti-normalization and smoothing processing on the forecast result, outputting forecast data, and pushing the forecast data to a corresponding service system for application. The method has strong robustness and generalization ability, can stably operate under different meteorological conditions, and has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the field of precipitation forecasting, and in particular to a TG-LSTM short-term precipitation forecasting method and system combined with an attention mechanism. Background Art

[0002] In the field of weather forecasting, precipitation prediction is a complex task, which is affected by a variety of meteorological factors, such as temperature, humidity, air pressure, wind speed, etc. The fluctuations of these factors over time will form complex time series data. Changes in meteorological factors often manifest as sudden and drastic fluctuations, and these mutation information is crucial for accurate precipitation forecasting. Traditional precipitation forecasting methods and existing deep learning models are difficult to effectively capture these mutation information, thus affecting the accuracy of forecast results.

[0003] In recent years, with the rapid development of artificial intelligence and deep learning technology, weather forecasting methods based on big data have gradually become a research hotspot. However, how to simultaneously consider the complex interactions between multiple meteorological elements is still a difficult point in model design. When dealing with multivariate interactions, existing models often fail to fully consider the importance of each meteorological element, resulting in inaccurate prediction results.

[0004] Therefore, how to accurately capture the mutation information of meteorological elements and their importance in affecting precipitation and improve the accuracy of short-term precipitation forecasts has become a technical problem that needs to be solved urgently in the field of meteorological forecasting. Summary of the invention

[0005] The purpose of the present invention is to provide a TG-LSTM short-term precipitation forecasting method and system combined with an attention mechanism, so as to solve the aforementioned problems existing in the prior art.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A TG-LSTM short-term precipitation forecasting method combined with an attention mechanism includes the following steps:

[0008] S1. Collect hourly real-time meteorological data to obtain required meteorological element data;

[0009] S2. Count the meteorological element characteristics of small-scale systems at different altitudes at each meteorological station; select meteorological features closely related to precipitation through the analysis of historical meteorological data, and construct a multi-dimensional feature time series vector;

[0010] S3, construct multidimensional time series data according to a fixed time length, and divide the training set and the validation set into a preset ratio;

[0011] S4. Build a TG-LSTM model based on the encoder-decoder architecture and complete model training on the dataset;

[0012] S5. Obtain hourly atmospheric forecast data and real-time data of the meteorological stations to be forecast in real time, and combine the real-time data every 3 hours in the past 3 days to organize them into a data format that meets the model input requirements;

[0013] S6. Input the processed data into the trained TG-LSTM model for inference to generate short-term precipitation forecast results;

[0014] S7. Post-process the forecast results, perform inverse normalization and smoothing, and then output the forecast data, which is pushed to the corresponding business system for application.

[0015] Further, the formula definition of the TG-LSTM model is as follows:

[0016] i t = σ(W i x t + U i h t-1 + b i ) (1)

[0017] f t = σ(W f x t + U f h t-1 + b f ) (2)

[0018] o t = σ(W o x t + U o h t-1 + b o ) (3)

[0019]

[0020] T t = 1 - tanh(f t ) (5)

[0021]

[0022] Among them, x, i, f, o, c, h, T, t represent input features, input gate, forget gate, output gate, cell state, hidden state, transformation gate, and current time respectively; σ and tanh are the sigma function and the hyperbolic tangent function; represents vector multiplication; W, U, b are the parameters that the model needs to learn.

[0023] Furthermore, an influence attention mechanism is added to the decoder structure to obtain the influence information of each non-predicted time series on the target sequence at the same moment.

[0024] Furthermore, the implementation formula of the influence attention mechanism is as follows:

[0025]

[0026]

[0027]

[0028] Among them, [h t-1 ; s t-1 represents the concatenation operation of the hidden state h and the cell state s at the previous moment; is the newly generated state information; x i is the i-th input feature vector; is the attention score, which is used to obtain the attention weight; is the parameter that the model needs to learn; T represents the transpose of a vector or matrix; the final attention score represents the importance of the influence of each non-predicted time series on the target sequence; is the output vector weighted by the attention score at the same moment t; taking the hidden state h t-1 of the previous moment and the new weighted feature as the new input of the encoder to generate the hidden state h t at time t; f en represents a TG-LSTM network unit.

[0029] Furthermore, a temporal attention mechanism is used to capture periodic patterns, thereby indirectly obtaining mutation information; specifically: calculating the attention vector of each hidden state of the decoder at each output time t, and the calculation formula is as follows:

[0030]

[0031]

[0032]

[0033] Among them [d t-1 ; s t-1 is the concatenation operation of the hidden state and the cell state at time t-1 in the decoder; h k is the hidden state from the encoder at time k; is the temporal attention score; which is used to obtain the attention weight; is the temporal attention weight; c′ tIt is the context vector obtained by time attention weighting of the hidden states of the encoder at all times; c′ t and the predicted output of the previous moment After concatenation, it is combined with d t-1 and input into the LSTM network together to obtain the updated encoder hidden state d t ; f de represents an LSTM cell;

[0034] Finally, the predicted value of the target sequence at time t is calculated by the following formula:

[0035]

[0036] where, [d t ; c′ t is the concatenation operation of the hidden state and context vector of the encoder; W y , b w , b y are the parameters that the model needs to learn; the prediction result is obtained by calculating through the above linear function

[0037] A TG-LSTM short-term precipitation forecasting system combined with an attention mechanism based on the same concept, for this method,

[0038] This system includes: a data acquisition module, a feature extraction module, a model forecasting module, a result output module, and a forecasting visualization module;

[0039] The data acquisition module is used to obtain hourly atmospheric forecast data and real-time data of the meteorological station to be forecasted in real time;

[0040] The feature extraction module is used to extract the physical quantities required by the model from the grid atmospheric forecast data and process them into a multi-dimensional time series feature data format that meets the model input requirements together with the real-time precipitation data of the meteorological station;

[0041] The model forecasting module is used to input the processed data into the trained TG-LSTM model for inference to generate short-term precipitation forecasting results;

[0042] The result output module is used to post-process the forecasting results and output the forecasting data after anti-normalization and smoothing processing;

[0043] The forecasting visualization module is used to present the short-term precipitation forecasting results to users in a visual manner in the weather forecasting business system.

[0044] The beneficial effects of the present invention are: The present invention discloses a TG-LSTM short-term precipitation forecasting method and system combined with an attention mechanism, having the following beneficial effects:

[0045] (1) The present invention establishes a TG-LSTM model introducing an attention mechanism and a temporal attention mechanism. The model can focus on important meteorological elements and critical moments, especially the mutation parts of meteorological elements, significantly improving the accuracy of short-term precipitation forecasting.

[0046] (2) The present invention can effectively process multi-dimensional meteorological data and capture the complex spatio-temporal influence relationships of meteorological elements on precipitation factors, providing comprehensive and accurate information support for precipitation forecasting.

[0047] (3) The present invention can process the input meteorological data in real time. Combining with fewer model parameters, it can quickly provide precipitation prediction results to meet the needs of real-time meteorological forecasting and early warning.

[0048] (4) Through reasonable training and optimization strategies, the present invention can enhance the robustness of the model under different meteorological conditions and improve its generalization ability in various weather environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic flowchart of a TG-LSTM short-term precipitation forecasting method combining an attention mechanism proposed in an embodiment of the present invention;

[0050] Figure 2 It is a schematic flowchart of a feature extraction module in an embodiment of the present invention;

[0051] Figure 3 It is a schematic structural diagram of a TG-LSTM unit in an embodiment of the present invention;

[0052] Figure 4 It is a schematic structural diagram of a TG-LSTM model combining an attention mechanism in an embodiment of the present invention;

[0053] Figure 5 It is a comparison chart of forecasting errors of a TG-LSTM short-term precipitation forecasting method combining an attention mechanism provided in an embodiment of the present invention;

[0054] Figure 6 It is a schematic framework diagram of a TG-LSTM short-term precipitation forecasting system combining an attention mechanism provided in an embodiment of the present invention.

[0055] In the drawings, 1, data acquisition module; 2, feature extraction module; 3, model forecasting module; 4, result output module; 5, forecasting visualization module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] To make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0057] Referring to Figure 1 as shown, this embodiment provides a TG-LSTM short-term precipitation forecasting method combined with an attention mechanism, including the following implementation steps:

[0058] S1. Data collection: Collect hourly atmospheric reanalysis data and hourly actual meteorological data of national meteorological stations to obtain the required meteorological element data. In this embodiment, physical quantities related to different geopotential heights and precipitation heights are selected, including specific humidity (q) and humidity field (r) at 500 hPa, specific humidity and humidity field at 700 hPa, height field (z) at 850 hPa, and U-wind field (u) and V-wind field (v) from 500 hPa to 925 hPa.

[0059] Step S2. Feature extraction: Statistically analyze the meteorological element characteristics of small-scale systems at different heights of each meteorological station. By analyzing historical meteorological data, meteorological characteristics closely related to precipitation are selected to construct a multi-dimensional feature time series vector.

[0060] In this embodiment, in order not to increase the complexity of the model, referring to the method of using convolution and pooling operations in a convolutional neural network to extract spatial features, the spatial features are constructed according to the steps Figure 2 as shown:

[0061] S21. Map the meteorological station to the corresponding grid in the atmospheric reanalysis data according to its longitude and latitude, and the data is statistically analyzed according to the grid with a spatial resolution of 0.25°×0.25°.

[0062] S22. Statistically analyze the values of the physical quantities of the grid where the meteorological station is located at this time point, and calculate the wind speeds at 500 hPa and 700 hPa, and the wind directions at 850 hPa and 925 hPa.

[0063] S23. Statistically analyze the maximum value, minimum value, mean value, and standard deviation of each physical quantity within the range of two grids (0.5-degree radius) around the grid where the meteorological station is located.

[0064] S24. Statistically analyze the maximum value, minimum value, mean value, and standard deviation of each physical quantity within the range of four grids (1-degree radius) around the grid where the meteorological station is located.

[0065] S25. Extract effective features from all the generated features for feature dimensionality reduction and perform normalization operations. In this embodiment, 30 effective features are selected, which together with the precipitation of the weather station constitute the multi-dimensional time series feature data at the 3-hour level. The specific feature descriptions are as follows in the table:

[0066]

[0067]

[0068]

[0069] Step S3. Dataset division: After the normalization operation, construct multi-dimensional time series data according to a fixed time length, and divide the training set and the validation set according to a preset ratio.

[0070] In this embodiment, the first 80% of the data are selected in chronological order to form the training set, and the remaining 20% is used as the validation set. Each data sample consists of 48 time instants. The first 24 time instants (the first 3 days) are used as input data, and the last 24 time instants (the last 3 days) are used as the predicted output data.

[0071] Step S4. Model construction: Construct a TG-LSTM model based on the encoder-decoder architecture and complete the model training on the dataset.

[0072] From the perspective of time series, a certain factor will fluctuate greatly over time, which is called mutation information. This mutation state may be caused by the mutation of a certain related factor or by the conditional mutation caused by the interaction of multiple factors. If the mutation information can be effectively captured, the accuracy of factor prediction can be improved. The TG-LSTM unit in this embodiment can effectively capture the mutation information. It is an improved long short-term memory network (LSTM) structure, and there are as many TG-LSTM units as there are past time instants. In this embodiment, there are 24 TG-LSTM units in total, which is optional.

[0073] As Figure 3 shown, the main improvement of TG-LSTM is to modify the output of the forget gate to allow as much information as possible to pass through the forget gate. The standard LSTM forget gate passes through the sigmod activation function, and the output value of the sigmod activation function is [0, 1]. In this way, when the data stream is very small, it will be filtered. If the standard LSTM is used, after passing through multiple LSTMs, the data stream is very likely to be filtered out. TG-LSTM mainly modifies the output of the forget gate. After passing through the transformation gate, the output value of the activation function is inversely mapped to [0.25, 1], which can effectively reduce information loss. The formula definition of TG-LSTM is as follows:

[0074] i t =σ(W i xt +U i h t-1 +b i ) (1)

[0075] f t =σ(W f x t +U f h t-1 +b f ) (2)

[0076] o t =σ(W o x t +U o h t-1 +b o ) (3)

[0077]

[0078] T t =1 - tanh(f t ) (5)

[0079]

[0080] Among them, x, i, f, o, c, h, T, and t respectively represent input features, input gate, forget gate, output gate, cell state, hidden state, transformation gate, and current time; σ and tanh are the sigma function and the hyperbolic tangent function; represents vector multiplication; W, U, and b are parameters that the model needs to learn.

[0081] The output of the traditional forget gate is reversely mapped to the interval [0.25, 1] through formula (5) after passing through the transformation gate, effectively capturing the mutation information discarded by the LSTM.

[0082] In practical applications, the target sequence is usually affected to varying degrees by non-predictive time series, resulting in extremely complex influence information. To learn the influence information of the mutation information of multiple non-predictive sequence features on the target sequence to varying degrees, the present invention adds an influence attention mechanism to the encoder structure to obtain the influence information of each non-predictive time series on the target sequence at the same moment.

[0083] As Figure 4 shown, the present invention introduces an influence attention mechanism in the encoder in the same time stage to adaptively capture the different influence relationships between the target sequence and the features of each time series. The formula for implementing the influence attention mechanism is as follows:

[0084]

[0085]

[0086]

[0087] Among them, [h t-1 ; s t-1 represents the concatenation operation of the previous hidden state h and cell state s; is the newly generated state information; x i is the i-th input feature vector; is the attention score, which is used to obtain the attention weight; is the parameter that the model needs to learn; T represents the transpose of a vector or matrix; the final attention score represents the importance of the influence of each non-predicted time series on the target sequence; is the output vector weighted by the attention score at the same moment t; the previous hidden state h t-1 and the new weighted feature are used as the new inputs of the encoder to generate the hidden state h t ; f en represents a TG-LSTM network unit.

[0088] There are certain periodic change rules in time series data, and mutation information also appears along with the periodic rules. Therefore, the present invention uses a time attention mechanism to capture periodic patterns, thereby indirectly obtaining mutation information. As Figure 4 shown, the time attention mechanism adaptively selects relevant hidden states from the encoder and obtains more hidden state information. In this way, the model can capture the dynamic temporal correlation of the target sequence. Specifically, at each output moment t, the attention vector of each hidden state of the decoder is calculated according to the following formula:

[0089]

[0090]

[0091]

[0092]

[0093] Among them [d t-1 ; s t-1 is the concatenation operation of the hidden state and cell state at time t-1 in the decoder; h k is the hidden state from the encoder at time k; is the time attention score; which is used to obtain the attention weight; is the time attention weight; c′ tIt is the context vector obtained by time attention weighting of the hidden states of the encoder at all times; c' t and the predicted output at the previous time After concatenation, it is combined with d t-1 and input into the LSTM network together to obtain the updated encoder hidden state d t ; f de represents an LSTM cell.

[0094] Finally, the predicted value of the target sequence at time t is calculated by the following formula:

[0095]

[0096] where, [d t ; c' t is the concatenation operation of the hidden state and context vector of the encoder; W y , b w , b y are the parameters that the model needs to learn; the prediction result is calculated through the above linear function

[0097] In this embodiment, the TG-LSTM model is trained using the training set and the validation set. As Figure 5 shown, compared with the traditional numerical weather prediction model (NWP), TG-LSTM shows smaller prediction errors, indicating that the present invention can effectively improve the accuracy of precipitation prediction.

[0098] Step S5, Forecast data processing: Real-time obtain hourly atmospheric forecast data and the actual data of the meteorological stations to be forecasted, and combine the actual data every 3 hours in the past 3 days to organize them into a data format that meets the model input requirements.

[0099] In this embodiment, due to the lag of the atmospheric reanalysis data, the required forecast input data is not available. Therefore, this method extracts the physical quantities required by the model from the hourly atmospheric forecast data and processes them into a data format that meets the model input requirements according to the data processing method described in step S2.

[0100] Step S6, Precipitation forecast: Input the processed data into the trained TG-LSTM model for inference to generate short-term precipitation forecast results.

[0101] In this embodiment, the TG-LSTM model is used to output the precipitation forecast values for each station every 3 hours in the next 3 days.

[0102] Step S7, Post-processing and output: Post-process the forecast results, and output the forecast data after anti-normalization and smoothing, and push it to the corresponding business system for application.

[0103] As Figure 6 shown, in another embodiment, a TG-LSTM short-term precipitation forecasting system incorporating an attention mechanism includes: a data acquisition module 1, a feature extraction module 2, a model forecasting module 3, a result output module 4, and a forecasting visualization module 5.

[0104] The data acquisition module 1 is used to obtain hourly atmospheric forecast data and hourly actual data of national meteorological stations to be forecast in real time.

[0105] The feature extraction module 2 is used to extract physical quantities required by the model from grid atmospheric forecast data and process them together with precipitation actual data into a multi-dimensional time series feature data format that meets the model input requirements.

[0106] The model forecasting module 3 is used to input the processed data into the trained TG-LSTM model for inference and generate short-term precipitation forecasting results.

[0107] The result output module 4 is used to post-process the forecasting results, and output the forecasting data after anti-normalization and smoothing.

[0108] The forecasting visualization module 5 is used to present the short-term precipitation forecasting results to users in a visual manner in the weather forecasting business system.

[0109] By adopting the above technical solutions disclosed in the present invention, the following beneficial effects are obtained:

[0110] (1) The present invention establishes a TG-LSTM model incorporating influence attention and time attention mechanisms. The model can focus on important meteorological elements and critical moments, especially the mutation parts of meteorological elements, significantly improving the accuracy of short-term precipitation forecasting.

[0111] (2) The present invention can effectively process multi-dimensional meteorological data and capture the complex spatio-temporal influence relationships of meteorological elements on precipitation factors, providing comprehensive and accurate information support for precipitation forecasting.

[0112] (3) The present invention can process input meteorological data in real time, combine fewer model parameters, and quickly provide precipitation prediction results to meet the needs of real-time meteorological forecasting and early warning.

[0113] (4) Through reasonable training and optimization strategies, the present invention can enhance the robustness of the model under different meteorological conditions and improve its generalization ability in various weather environments.

[0114] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A TG-LSTM short-term precipitation forecasting method combined with an attention mechanism, characterized in that: The following steps are involved: S1. Collect hourly real-time meteorological data to obtain required meteorological element data; S2. Count the meteorological element characteristics of small-scale systems at different altitudes at each meteorological station; select meteorological features closely related to precipitation through the analysis of historical meteorological data, and construct a multi-dimensional feature time series vector; S3, construct multidimensional time series data according to a fixed time length, and divide the training set and the validation set into a preset ratio; S4. Build a TG-LSTM model based on the encoder-decoder architecture and complete model training on the dataset; S5. Obtain hourly atmospheric forecast data and the actual data of the meteorological stations required for forecast in real time, combine the actual data of 3 hours every 3 days in the past, and organize them into a data format that meets the model input requirements; S6, inputting the processed data into the trained TG-LSTM model for inference to generate a short-term precipitation forecast result; S7. Post-process the forecast results, output the forecast data after denormalization and smoothing, and push it to the corresponding business system for application.

2. The TG-LSTM short-term precipitation forecasting method combined with the attention mechanism according to claim 1 is characterized in that: The formula of the TG-LSTM model is defined as follows: i t =σ(W i x t +U i h t-1 +b i ) (1) f t =σ(W f x t +U f h t-1 +b f ) (2) the t =σ(W o x t +U o h t-1 +b o ) (3) T t =1-tanh(f t ) (5) Among them, x, i, f, o, c, h, T, t represent input features, input gate, forget gate, output gate, cell state, hidden state, transformation gate and current time respectively; σ and tanh are sigma function and hyperbolic tangent function; Represents vector multiplication; W, U, b are the parameters that the model needs to learn.

3. The TG-LSTM short-term precipitation forecasting method combined with the attention mechanism according to claim 2 is characterized in that: An influence attention mechanism is added to the decoder structure to obtain the influence information of each non-predicted time series on the target sequence at the same moment.

4. The TG-LSTM short-term precipitation forecasting method combined with the attention mechanism according to claim 3 is characterized in that: The implementation formula of the attention mechanism is as follows: Among them, [h t-1 ;s t-1 ] represents the concatenation operation of the hidden state h and the cell state s at the previous moment; is the newly generated state information; x i is the i-th input feature vector; is the attention score, used to obtain the attention weight; is the parameter that the model needs to learn; T represents the vector or matrix transpose; the final attention score Indicates the importance of each non-predicted time series’ impact on the target series; is the output vector weighted by the attention score at the same time t; the hidden state h at the previous time t-1 and the new weighted features As the new input of the encoder, the hidden state h at time t is generated t ;f en Represents a TG-LSTM network unit.

5. The TG-LSTM short-term precipitation forecasting method combined with the attention mechanism according to claim 4 is characterized in that: The temporal attention mechanism is used to capture periodic patterns and thus indirectly obtain mutation information. Specifically, the attention vector of each hidden state of the decoder is calculated at each output time t. The calculation formula is as follows: Where [d t-1 ;s t-1 ] is the concatenation operation of the hidden state and the cell state at time t-1 in the decoder; h k is the hidden state from the encoder at time k; is the temporal attention score; Used to obtain attention weights; is the temporal attention weight; c′ t is the context vector of the encoder’s hidden state at all times weighted by time attention; c′ t The predicted output of the previous moment After splicing with d t-1 Input them into the LSTM network together to obtain the updated encoder hidden state d t ;f de Represents an LSTM unit; Finally, the target sequence prediction value at time t is calculated by the following formula: Among them, [d t ; c′ t ] is the concatenation operation of the encoder’s hidden state and the context vector; W y ,b w ,b y is the parameter that the model needs to learn; the prediction result is obtained by calculating the above linear function 6. A TG-LSTM short-term precipitation forecast system combined with an attention mechanism, used in the method described in any one of claims 1 to 5, characterized in that: The system includes: a data acquisition module, a feature extraction module, a model prediction module, a result output module, and a prediction visualization module; The data acquisition module is used to obtain hourly atmospheric forecast data and the actual data of the meteorological stations required for forecast in real time; The feature extraction module is used to extract the physical quantities required by the model from the grid atmospheric forecast data, and process them together with the actual precipitation data of the meteorological station into a multi-dimensional time series feature data format that meets the model input requirements; The model forecasting module is used to input the processed data into the trained TG-LSTM model for inference to generate a short-term precipitation forecast result; The result output module is used to post-process the forecast results, and output the forecast data after denormalization and smoothing; The forecast visualization module is used to present the short-term precipitation forecast results to users in a visual manner in the weather forecast service system.

Images