Multitask learning meteorological element prediction method based on LSTM-GAN

Abstract

The invention discloses a multi-task learning meteorological element prediction method based on LSTM-GAN, and relates to the field of meteorological element prediction and deep learning. The problem that a traditional LSTM-GAN prediction model based on deep learning is low in prediction precision is solved. According to the method, time features are added into input data, a model is helped to capture seasonal and periodic changes in a weather sequence, and an LSTM-GAN model constructed based on a three-layer stacked LSTM, a multi-head attention mechanism and a multi-task learning mechanism captures a long-distance dependency relationship in a time sequence and performs subsequent multi-task learning. And the prediction effect of each climate variable is improved. The method is mainly used for simultaneously predicting various meteorological elements.

Description

Technical Field

[0001] The present invention relates to the field of meteorological element prediction and deep learning. Background Art

[0002] Weather forecasting plays an extremely important role in our daily production and life, and accurately predicting future weather conditions has always been the focus and difficulty of weather forecasting. By timely predicting weather conditions, farmers can reasonably arrange planting, fertilization and harvesting, maximize yields and reduce losses, or help manage water resources, predict reservoir water storage and irrigation needs. In addition, long-term weather data can be used to monitor climate change trends and promote the formulation of sustainable development policies. Therefore, if the meteorological conditions in the future can be accurately predicted, it will have a profound impact on all aspects of social life.

[0003] Commonly used prediction methods include: numerical weather prediction (NWP), statistical methods, machine learning methods, etc. In recent years, deep learning methods have shown strong performance in time series data analysis, especially neural networks and long short-term memory networks, which provide new ideas for the generation and prediction of complex meteorological data. However, the disadvantage of neural networks is that they are prone to gradient vanishing and gradient explosion problems. When the output of LSTM is multivariable, it may lead to instability in the training process, especially when there are complex dependencies between variables. The prediction accuracy of the model is significantly lower than that of the model with a single variable output. In addition, the existing LSTM-GAN prediction model is mainly used to predict a single variable when performing meteorological prediction, and the prediction accuracy of multiple variables is low. Therefore, the above problems need to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of low prediction accuracy of the traditional LSTM-GAN prediction model based on deep learning; the present invention provides a multi-task learning meteorological element prediction method based on LSTM-GAN.

[0005] A multi-task learning meteorological element prediction method based on LSTM-GAN, the method comprises the following steps:

[0006] S1. Obtain historical meteorological element data to form a training set, and preprocess the data in the training set; wherein each sample in the training set includes historical meteorological element data of a historical date and real meteorological element data corresponding to a predicted date; there are multiple meteorological elements;

[0007] S2. Adding time-related features to the processed data; wherein the time-related features include quarters and months;

[0008] S3. Build an LSTM-GAN model, which includes a generator and a discriminator. The discriminator is used to judge the prediction results of the generator, and the generator is implemented based on a three-layer stacked LSTM, a multi-head attention mechanism, and a multi-task learning mechanism;

[0009] S4. Train the LSTM-GAN model:

[0010] The historical meteorological element data of the historical date in the sample is used as the input data of the generator, and its input data is predicted, and the prediction results are sent to the discriminator;

[0011] At the same time, the discriminator judges the prediction results of the generator according to the real meteorological element data corresponding to the predicted date in the sample, and updates the parameters of the generator and the discriminator according to the judgment results to obtain the trained LSTM-GAN model;

[0012] S5. Predict meteorological element data based on the trained LSTM-GAN model.

[0013] Preferably, the meteorological elements include air temperature, dew point temperature, sea level pressure, visibility and wind speed.

[0014] Preferably, the implementation method of preprocessing the data in the training set is:

[0015] When preprocessing the data, first delete the duplicate data, then use interpolation to fill in the missing data, and finally use Min-Max normalization to normalize all data to the range of [0,1]. The normalization formula is as follows:

[0016]

[0017] In the formula, X′ is the normalized value, X is the original value, and X min is the minimum value in the training set, X max is the maximum value in the training set.

[0018] Preferably, the generator includes a shared layer and multiple specific task layers. The shared layer consists of three stacked LSTM layers, which perform preliminary feature extraction on the historical meteorological element data of historical dates in the input samples, perform weighted combination of the extracted features through a multi-head attention mechanism, and then capture the temporal feature dependencies of the corresponding meteorological elements through the corresponding specific task layers, and then output the predicted meteorological element data corresponding to the predicted date; wherein the predicted meteorological element data corresponding to the predicted date serves as the prediction result of the generator, and a task learning mechanism is embedded in each specific task layer.

[0019] Preferably, the discriminator performs convolution operations on the input data step by step through three one-dimensional convolution layers arranged step by step, and then outputs the data through a fully connected layer.

[0020] Preferably, the total loss L of the generator during the training phase is gen The expression is:

[0021] L gen =α·L adv +β·L tasks ;

[0022]

[0023] Among them, L adv represents the adversarial loss of the generator, L tasks represents the weighted sum of the losses of all tasks, α and β are the first and second weight coefficients respectively, z j represents the prediction result of the three-layer stacked LSTM output for the jth sample, G(z j ) indicates that the generator is for z j The generated prediction result, D(G(z j )) represents the j )’s judgment result, represents the loss function of the mth task, N represents the total number of samples, M represents the total number of tasks, and λ m represents the weight loss corresponding to the mth task, y j,i is the true value of the i-th feature in the j-th sample, is the predicted value of the i-th feature in the j-th sample, and δ represents the smoothness of the control loss function.

[0024] Preferably, the loss L of the discriminator during the training phase is disc The expression is:

[0025]

[0026] Where N represents the total number of samples, D(y j ) represents the judgment result of the discriminator for the real data in the jth sample, z j represents the prediction result of the three-layer stacked LSTM output for the jth sample, G(z j ) indicates that the generator is for z j The generated prediction result, D(G(z j )) represents the j ), N represents the total number of samples.

[0027] A multi-task learning meteorological element prediction device based on LSTM-GAN includes a storage device, a processor, and a computer program stored in the storage device and executable on the processor. The processor executes the computer program to implement the multi-task learning meteorological element prediction method based on LSTM-GAN.

[0028] The beneficial effects brought by the present invention are:

[0029] The present invention uses three-layer stacked LSTM as the generator of GAN and adds time features to the input data to help the model capture seasonal and periodic changes in weather series, making the generated data more reasonable; the multi-head attention mechanism captures long-distance dependencies in time series and performs subsequent multi-task learning, which not only improves the prediction effect of each climate variable, but also reduces the interference between tasks, so that the model can better adapt to different climate indicators;

[0030] The LSTM-GAN model adopts the generative adversarial network model architecture as a whole, which not only has high prediction accuracy, but also performs well in the task of distinguishing real data from generated data. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of the multi-task learning meteorological element prediction method based on LSTM-GAN of the present invention;

[0032] Figure 2 This is a schematic diagram of the structure of the LSTM-GAN model;

[0033] Figure 3 This is a schematic diagram of the internal principle of LSTM;

[0034] Figure 4 This is a schematic diagram of the principle of the discriminator in the LSTM-GAN model;

[0035] Figure 5 This is a comparison chart of the prediction effects of different models under the same configuration conditions. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0038] Specific implementation method 1. Figure 1 The present embodiment describes a multi-task learning meteorological element prediction method based on LSTM-GAN, which comprises the following steps:

[0039] S1. Obtain historical meteorological element data to form a training set, and preprocess the data in the training set; wherein each sample in the training set includes historical meteorological element data of a historical date and real meteorological element data corresponding to a predicted date; there are multiple meteorological elements; specifically, meteorological elements include air temperature, dew point temperature, sea level pressure, visibility and wind speed;

[0040] S2. Adding time-related features to the processed data; wherein the time-related features include quarters and months;

[0041] S3. Build an LSTM-GAN model, which includes a generator and a discriminator. The discriminator is used to judge the prediction results of the generator, and the generator is implemented based on a three-layer stacked LSTM, a multi-head attention mechanism, and a multi-task learning mechanism;

[0042] S4. Train the LSTM-GAN model:

[0043] The historical meteorological element data of the historical date in the sample is used as the input data of the generator, and its input data is predicted, and the prediction results are sent to the discriminator;

[0044] At the same time, the discriminator judges the prediction results of the generator according to the real meteorological element data corresponding to the predicted date in the sample, and updates the parameters of the generator and the discriminator according to the judgment results to obtain the trained LSTM-GAN model;

[0045] S5. Predict meteorological element data based on the trained LSTM-GAN model.

[0046] In this implementation, by using three layers of stacked LSTM as the generator of the generative adversarial network (GAN), and adding time features to the input data, the model is helped to capture seasonal and periodic changes in the weather series, making the generated data more reasonable; the multi-head attention mechanism captures the long-distance dependencies in the time series and performs subsequent multi-task learning, which not only improves the prediction effect of each climate variable, but also reduces the interference between tasks, so that the model can better adapt to different climate indicators; the LSTM-GAN model adopts the generative adversarial network model architecture as a whole, which not only has high prediction accuracy, but also performs well in the task of distinguishing real data from generated data. And when the meteorological elements include air temperature, dew point temperature, sea level pressure, visibility and wind speed, five meteorological element indicators are predicted at the same time, so that information between different tasks can be shared.

[0047] See also Figure 3 LSTM is an existing technology, and its LSTM includes a forget gate f t , input gate i t , output gate o t , cell state Ct and the hidden state h t , the calculation formula is as follows:

[0048] f t =σ(W f x t +U f h t-1 +b f );

[0049] i t =σ(W i x t +U i h t-1 +b i );

[0050] o t =σ(W o x t +U o h t-1 +b o );

[0051] C t =f t ☉C t-1 +i t ☉tanh(W c x t +U c h t-1 +b c );

[0052] h t =o t ☉tanh(C t );

[0053] Among them, W f , W i , W o , W C are the weights of the first to fourth neurons, respectively, and b f 、b i 、b o 、b C Corresponding to neurons W f , W i , W o , W C The bias, U f , U i , U o , U C are respectively related to the previous hidden state h t-1 The first to fourth weight matrices associated with t-1 is the cell state at the previous time step, σ is the sigmoid activation function, xt is the input of the current time step, h t-1 is the hidden state at the previous time step.

[0054] Specifically, the implementation method of preprocessing the data in the training set in step S1 is:

[0055] When preprocessing the data, first delete the duplicate data, then use interpolation to fill in the missing data, and finally use Min-Max normalization to normalize all data to the range of [0,1]. The normalization formula is as follows:

[0056]

[0057] In the formula, X′ is the normalized value, X is the original value, and X min is the minimum value in the training set, X max is the maximum value in the training set.

[0058] See also Figure 2 The generator includes a shared layer and multiple specific task layers. The shared layer consists of three stacked LSTM layers. After preliminary feature extraction of the historical meteorological element data of the historical date in the input sample, the extracted features are weighted combined through the multi-head attention mechanism, and the corresponding specific task layer captures the temporal feature dependencies of the corresponding meteorological elements. Then, the predicted meteorological element data corresponding to the predicted date is output; the predicted meteorological element data corresponding to the predicted date is used as the prediction result of the generator, and a task learning mechanism is embedded in each specific task layer.

[0059] In this implementation, the output of LSTM is weighted through a multi-head attention mechanism to capture long-distance dependencies in the time series, improve the prediction effect of each climate variable, and reduce interference between tasks. In specific applications, when using the trained LSTM-GAN model to predict meteorological element data, only the historical meteorological element data needs to be used as the input of the generator to output the predicted meteorological element data corresponding to the predicted date.

[0060] Furthermore, in the three-layer stacked LSTM, the first LSTM layer contains 256 hidden units, the second LSTM layer contains 128 hidden units, and the third LSTM layer contains 64 hidden units.

[0061] See also Figure 4 , the discriminator performs convolution operations on the input data step by step through three one-dimensional convolutional layers set step by step, and then outputs it through the fully connected layer. In specific applications, the convolution kernel sizes of the first to third one-dimensional convolutional layers are 64, 128, and 256 respectively. The dimension of the final input to the fully connected layer is 256, and the output dimension is 1, which is the judgment result. Each layer uses the Sigmoid activation function.

[0062] Furthermore, the total loss L of the generator during the training phase gen The expression is:

[0063] L gen =α·L adv +β·L tasks ;

[0064]

[0065] Among them, L adv represents the adversarial loss of the generator, L tasks represents the weighted sum of the losses of all tasks, α and β are the first and second weight coefficients respectively, z j represents the prediction result of the three-layer stacked LSTM output for the jth sample, G(z j ) indicates that the generator is for z j The generated prediction result, D(G(z j )) represents the j )’s judgment result, represents the loss function of the mth task, N represents the total number of samples, M represents the total number of tasks, and λ m represents the weight loss corresponding to the mth task, y j,i is the true value of the i-th feature in the j-th sample, is the predicted value of the i-th feature in the j-th sample, and δ represents the smoothness of the control loss function.

[0066] In this embodiment, the total loss L of the constructed generator is gen The advantage of the expression is that it comprehensively considers adversarial learning and multi-task learning. By adjusting the weight coefficients α and β, the balance between adversarial and multi-task of the generator can be flexibly controlled, and the generation results and prediction performance can be optimized according to specific needs. This method not only promotes the diversity of generated samples, but also improves the overall prediction accuracy and convergence of the model.

[0067] Furthermore, the loss L of the discriminator during the training phase disc The expression is:

[0068]

[0069] Where N represents the total number of samples, D(y j ) represents the judgment result of the discriminator for the real data in the jth sample, z j represents the prediction result of the three-layer stacked LSTM output for the jth sample, G(z j ) indicates that the generator is for z j The generated prediction result, D(G(z j)) represents the j ), N represents the total number of samples.

[0070] In this preferred embodiment, the loss L of the constructed discriminator is disc The advantages of the expression are that it improves the stability of model training and avoids the problems of gradient disappearance and mode collapse. The gradient penalty mechanism provides more reliable gradient information, enabling the generator to update parameters more effectively and generate higher quality prediction results.

[0071] Specific implementation method 2: The multi-task learning meteorological element prediction device based on LSTM-GAN described in this implementation method includes a storage device, a processor, and a computer program stored in the storage device and executable on the processor. The processor executes the computer program to implement the multi-task learning meteorological element prediction method based on LSTM-GAN as described in specific implementation method 1.

[0072] Verification test:

[0073] The technical effects of the present invention are illustrated by the following verification tests, specifically:

[0074] Figure 5 This is a comparison chart of the prediction effects of different models under the same configuration conditions. The present invention selects multiple deep learning models for comparison, including RNN, LSTM, CNN, and CNN-LSTM models;

[0075] Quantitative indicators include mean square error (MSE), mean absolute error (MAE), and coefficient of determination (R2). MSE measures the average of the squared difference between the predicted value and the true value, MAE provides the average absolute value of the prediction error, and the coefficient of determination R2 measures the model's ability to explain data variability. Its value ranges from -∞ to 1, and the calculation formulas are:

[0076]

[0077] Among them, Y i is the actual value, is the predicted value, is the mean of Y, and n is the number of samples.

[0078] First, calculate the MSE, MAE, and R for each model 2 , the comprehensive MSE, MAE and R2 of each model, and R 2 is the average of the prediction errors on the five climate indicators, from Figure 5 We can conclude that the MSE and MAE of the multi-task learning model based on LSTM-GAN are much lower than those of other models. 2It reached 0.89. By combining multi-task learning, LSTM's time series modeling capabilities and GAN's generative adversarial structure, it effectively improved the prediction accuracy, especially the performance on multiple weather indicators was more stable and accurate.

[0079] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. A multi-task learning meteorological element prediction method based on LSTM-GAN, characterized in that: The method comprises the following steps: S1. Obtain historical meteorological element data to form a training set, and preprocess the data in the training set; wherein each sample in the training set includes historical meteorological element data of a historical date and real meteorological element data corresponding to a predicted date; there are multiple meteorological elements; S2. Adding time-related features to the processed data; wherein the time-related features include quarters and months; S3. Build an LSTM-GAN model, which includes a generator and a discriminator. The discriminator is used to judge the prediction results of the generator, and the generator is implemented based on a three-layer stacked LSTM, a multi-head attention mechanism, and a multi-task learning mechanism; S4. Train the LSTM-GAN model: The historical meteorological element data of the historical date in the sample is used as the input data of the generator, and its input data is predicted, and the prediction results are sent to the discriminator; At the same time, the discriminator judges the prediction results of the generator according to the real meteorological element data corresponding to the predicted date in the sample, and updates the parameters of the generator and the discriminator according to the judgment results to obtain the trained LSTM-GAN model; S5. Predict meteorological element data based on the trained LSTM-GAN model.

2. The multi-task learning meteorological element prediction method based on LSTM-GAN according to claim 1 is characterized in that: Meteorological elements include air temperature, dew point temperature, sea level pressure, visibility and wind speed.

3. The multi-task learning meteorological element prediction method based on LSTM-GAN according to claim 1 is characterized in that: The implementation method for preprocessing the data in the training set is: When preprocessing the data, first delete the duplicate data, then use interpolation to fill in the missing data, and finally use Min-Max normalization to normalize all data to the range of [0,1]. The normalization formula is as follows: In the formula, X′ is the normalized value, X is the original value, and X min is the minimum value in the training set, X max is the maximum value in the training set.

4. The multi-task learning meteorological element prediction method based on LSTM-GAN according to claim 1 is characterized in that: The generator includes a shared layer and multiple specific task layers. The shared layer consists of three stacked LSTM layers. After preliminary feature extraction of historical meteorological element data of historical dates in the input samples, the extracted features are weighted combined through a multi-head attention mechanism. Then, the corresponding specific task layer captures the temporal feature dependencies of the corresponding meteorological elements, and outputs the predicted meteorological element data corresponding to the predicted date. The predicted meteorological element data corresponding to the predicted date serves as the prediction result of the generator, and a task learning mechanism is embedded in each specific task layer.

5. The multi-task learning meteorological element prediction method based on LSTM-GAN according to claim 1 is characterized in that: The discriminator performs convolution operations on the input data step by step through three one-dimensional convolutional layers set up step by step, and then outputs it through a fully connected layer.

6. The multi-task learning meteorological element prediction method based on LSTM-GAN according to claim 1 is characterized in that: The total loss L of the generator during the training phase gen The expression is: L gen =α·L adv +β·L tasks ; Among them, L adv represents the adversarial loss of the generator, L tasks represents the weighted sum of the losses of all tasks, α and β are the first and second weight coefficients respectively, z j represents the prediction result of the three-layer stacked LSTM output for the jth sample, G(z j ) indicates that the generator is for z j The generated prediction result, D(G(z j )) represents the j )’s judgment result, represents the loss function of the mth task, N represents the total number of samples, M represents the total number of tasks, and λ m represents the weight loss corresponding to the mth task, y j,i is the true value of the i-th feature in the j-th sample, is the predicted value of the i-th feature in the j-th sample, and δ represents the smoothness of the control loss function.

7. The multi-task learning meteorological element prediction method based on LSTM-GAN according to claim 1 is characterized in that: The loss L of the discriminator during the training phase disc The expression is: Where N represents the total number of samples, D(y j ) represents the judgment result of the discriminator for the real data in the jth sample, z j represents the prediction result of the three-layer stacked LSTM output for the jth sample, G(z j ) indicates that the generator is for z j The generated prediction result, D(G(z j )) represents the j ), N represents the total number of samples.

8. A multi-task learning meteorological element prediction device based on LSTM-GAN, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that: The processor executes a computer program to implement the multi-task learning meteorological element prediction method based on LSTM-GAN as described in any one of claims 1 to 7.

Images