A long time series prediction method based on standardized flow

Abstract

The present invention discloses a long time series prediction method based on standardized flow, comprising the following steps: data standardization; the data is a large amount of long time series data of multiple categories and non-uniform units; extracting multivariate and multi-scale information of the long time series data; based on the multivariate and multi-scale information, respectively extracting long-term trend information, seasonal trend information and discreteness information; fusing different feature data; and performing data prediction based on standardized flow according to the features obtained by fusion. The present invention can improve the prediction accuracy of long time series data, and is particularly suitable for long-term meteorological prediction.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and application technology, relates to big data analysis and prediction, and particularly to a long time series prediction method based on standardized flow. Background Art

[0002] Long time series refers to data series with a long time span, generally in years. It has typical application significance in meteorological forecasting, especially in the forecasting of disastrous weather. For example, disastrous weather such as drought, low temperature, heavy rain, flood, hail, etc. has a great impact on agriculture and animal husbandry. Accurate prediction of future rainfall, temperature and other data can provide reference for management departments, more accurately guide spring planting and autumn harvest, reserve livestock for winter grain, and take preventive measures in time to prevent waterlogging, low temperature damage, early frost damage and other disastrous effects. However, most of the current mainstream forecasting methods use LSTM and Transformer-based forecasting methods, which have high time complexity, do not fully consider the information of each dimension of the time series, and the feature extraction is not comprehensive. Summary of the invention

[0003] In order to overcome the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a long time series prediction method based on standardized flow, so as to improve the prediction accuracy of long time series data and use it for long-term meteorological forecasting.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is:

[0005] A long time series prediction method based on standardized flow includes the following steps:

[0006] Step 1: Data standardization; the data is a large amount of long-term series data of various types and in non-uniform units;

[0007] Step 2: Extract the multivariate and multiscale information X of the long time series data in ;

[0008] Step 3: Based on the multivariate multi-scale information X in , respectively extract the long-term trend information F t , seasonal trend information F s and dispersion information F d ; The extraction of long-term trend information F t , is to obtain the overall trend of the sequence through a linear model containing a moving average kernel; extract seasonal trend information F s , is to map the obtained features to the frequency domain space through fast discrete Fourier transform to obtain the periodic change law of the sequence; extract the discreteness information F d , is by including trend information F t And seasonal information Fs The formula is obtained;

[0009] Step 4, merging the different feature data obtained in step 3;

[0010] Step 5: According to the fused features, perform data prediction based on the standardized flow.

[0011] Compared with the prior art, the present invention predicts long time series, uses a multivariate multi-scale feature extraction method to obtain the features of input data, and based on this feature, uses a linear trend extraction method to extract time series trend information, uses discrete Fourier transform to extract seasonal trend information and discrete degree information, and uses the features containing the three types of information as input to predict the changes in the time series data of the region in the future. The present invention can improve the prediction accuracy of the prediction model for long time series data.

[0012] The present invention can be typically applied to the prediction of long-term time series data such as meteorology, as well as the prediction of other time series data similar to meteorological data. Experimental data show that in terms of the prediction of long-term meteorological data, compared with the existing LSTNet model and Informer model, the prediction accuracy of the present invention is significantly improved, and more accurate prediction data can be provided for agricultural and animal husbandry management departments and scheduling departments. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the process of the present invention.

[0014] Figure 2 It is a fully connected network FCN μ and FCN δ Structure diagram of .

[0015] Figure 3 These are the monthly observed rainfall data for station 54218 from 2012 to 2022.

[0016] Figure 4 Input and forecast data for rainfall at station 54218.

[0017] Figure 5 Input and predicted data for temperature at site 54117.

[0018] Figure 6 The following is a display of the average temperature forecast results for some of the 18 leagues and cities.

[0019] Figure 7 The interannual changes in the average annual temperature of 18 leagues and cities are predicted from 2022 to 2031.

[0020] Figure 8 The following is a display of rainfall forecast results for some of the 18 leagues and cities.

[0021] Fig. 9 The interannual changes in annual rainfall in 18 leagues and cities are predicted from 2022 to 2031. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail by specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0023] To facilitate understanding of the technical solution of the present invention, some terms in the present invention are explained as follows:

[0024] Normalized flow: Use simple probability distributions to construct more complex and more expressive probability distributions. This process mainly uses a series of transformations to gradually become complex and more expressive probability distributions.

[0025] Long cycle: A time series with a relatively long cycle, generally in years, and which includes specific time data such as year, month, day, hour, and minute. In the embodiment of the present invention, the data used in the prediction is a total of twenty years from 2002 to 2021.

[0026] Multivariate and multi-scale information: scale refers to different time lengths, such as years, months, days, hours, and minutes; variables refer to different input parameters, such as temperature, air pressure, wind speed, humidity, evaporation, sunshine, rainfall, and air quality.

[0027] Long-term trend information: Trend refers to the overall change trend of the time series as time increases. It can be linear or nonlinear.

[0028] Seasonal trend information: Seasonality refers to cyclical changes that occur repeatedly within the same time period.

[0029] Discreteness information: represents the changes of time series elements in the above period.

[0030] Combined with the above definition, the long time series prediction method based on standardized flow of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0031] Step 1: Data standardization.

[0032] The data designed by the present invention are massive long-term series data of various types and non-uniform units, such as Figure 1 The meteorological historical series data shown. For subsequent calculations, these data need to be standardized to unify their dimensions. For example, the present invention uses the Z-score standardization method to unify the data to the same dimension, and the method is as follows:

[0033] Calculate the mean μ and standard deviation δ of the original data, and use the following formulas for different input data x i0 To standardize:

[0034]

[0035] In the formula, x i is x i0 After normalization, x i0 ∈R dx , dx≥1.

[0036] Step 2: Extract multivariate and multiscale information X from long time series data in .

[0037] After data standardization in step 1, the multivariate and multiscale information X of the long time series data of the present invention can be extracted. in , specifically, the extraction method is as follows:

[0038] Step 21: Extract the correlation information between different data, and increase the weight of variables with high prediction importance to obtain multivariate correlation information.

[0039] For massive practical sequence data, different data often have certain correlations. The purpose of this step is to extract the correlation information and give higher weights to variables with high prediction importance.

[0040] Among them, the formula for extracting the correlation information between different data is as follows:

[0041]

[0042] In the formula, Represents the relationship between different data variables, f and f -1 Represents fast Fourier transform and its inverse transform, x i and x j Represent different data variables respectively;

[0043] The present invention uses the Softmax function to highlight the relationship values ​​with high prediction importance, that is, the higher the importance, the greater the weight set. The weight W Cr It is expressed as follows:

[0044]

[0045] Step 22: Extract multi-scale information.

[0046] Multi-scale information is related information of various time scales. The present invention extracts multi-scale information using the following formula:

[0047]

[0048]

[0049] In the formula, Δ S represents multi-scale time variable information, W S and b S are the optimal weights and biases trained in the network, express The transpose of (b S )′ represents b S The transpose of It represents the values ​​of different data variables at different time scales. In the embodiment of the present invention, the scale S includes seconds, minutes, hours, days, months, and years.

[0050] Step 23: Fuse the multivariate association information obtained in step 21 with the multiscale information obtained in step 22 to obtain the multivariate multiscale information X in a unified space. in The fusion formula is as follows:

[0051] X in =(W v (W Cr x i )+b v )+Δ S

[0052] Where W v and b v Represents the optimal weights and biases obtained after network training.

[0053] Step 3: Based on the multivariate and multi-scale information X obtained in step 2 in , respectively extract long-term trend information F t , seasonal trend information F s and dispersion information F d .

[0054] In the present invention, long-term trend information F is extracted t , is to obtain the overall trend of the sequence through a linear model containing a moving average kernel. The present invention specifically extracts long-term trend information through a linear layer. The linear layer is a layer structure commonly used in deep learning, and it plays a very important role in neural networks. The linear layer performs a weighted summation and bias operation on the input to achieve a linear transformation of the input data. Specifically, the linear layer is used to extract long-term trend information F t The formula can be expressed as:

[0055] F t=Linear(X in )=W t X in +b t

[0056] Where W t , b t They are the weight and bias terms in the process of trend information extraction. Linear() represents the linear layer operation, which realizes the linear transformation of the input data by performing weighted summation and biasing operations on the input.

[0057] In the present invention, seasonal trend information F is extracted s , is to map the obtained features to the frequency domain space through fast discrete Fourier transform to obtain the periodic change law of the sequence. Specifically, fast Fourier transform is used to extract seasonal trend information F s The formula is as follows:

[0058]

[0059] in, Yes X in The sth scale variable in , s represents any item between 0 and N-1, N is X in The number of variables in

[0060] In the present invention, the dispersion information F is extracted d , is by including trend information F t And seasonal information F s Specifically, the discrete information F is extracted d The formula is as follows:

[0061]

[0062] Step 4: data fusion.

[0063] In this step, the feature data containing different information obtained in step 3 are fused as follows:

[0064] F=W sea F s +W tre F t +W dis F d

[0065] Where W sea Represents F s The learnable weight parameter, W tre Represents F t The learnable weight parameter, Wdis Represents F d The learnable weight parameters.

[0066] Step 5: According to the fused features, perform data prediction based on the standardized flow.

[0067] The present invention uses a data enhancement method to enhance the accuracy of data prediction by learning the distribution of the latent state of the feature, as follows:

[0068] Step 51: Draw a random variable ε∽N(0,I) from the Gaussian distribution. The initial state of the standardized flow is expressed as:

[0069] z0=FCN μ (F)+FCN δ (F)·ε

[0070] in and are two fully connected networks, μ and σ represent different ResNet layer structures, the specific structures are as follows Figure 2 shown.

[0071] Step 52: The continuous iteration of the overall normalization flow is expressed as:

[0072] z t =FCN μ (F,z t-1 )+FCN δ (F,z t-1 )·z t-1

[0073] z t It outputs the continuously optimized final prediction value, t=1,....T, T is a certain moment in the future time series to be predicted, and ultimately improves the reliability of the predicted value by learning the distribution of latent variables.

[0074] In this step of data enhancement method, the loss function is set as follows:

[0075] L = MSE(z t ,Y)

[0076] Calculate the predicted value z t The error between the true value Y and the training model parameters is optimized until the model converges.

[0077] The corresponding system of the present invention comprises:

[0078] Multivariate and multiscale information extraction module, used to extract multivariate correlation features and multiscale time features of input data;

[0079] Time series trend information extraction module, used to learn channel-independent long-term stable trend information in time series;

[0080] Seasonal trend information extraction module, used to extract seasonal expression information in the frequency domain;

[0081] The information enhancement module based on the standardized flow is used to learn the distribution of the latent variable state space and improve the reliability of information prediction;

[0082] The present invention fully extracts information from different angles through three information extraction modules and improves information utilization through an information enhancement module, thereby being able to effectively predict future data changes in long-term time series.

[0083] In a specific embodiment of the present invention, the hardware is a computer, and the configuration includes the hardware environment: CPU: 2 Intel Xeon 6130 processors (2.1GHz / 16c) / 2666MHz / 10.4GT; GPU: 6 16G_TESLA-P100_4096b_P_CAC; Memory: 16 32G ECC Registered DDR42666; Software environment: Operating system: Ubantu 16.04; Deep learning framework: Tensorflow, pandas, scikit-learn; Language and development environment: Python 3.6, Anaconda3, pycharm2020.

[0084] The method of the present invention is used to perform weather forecasting, taking at least two of the data of temperature, air pressure, wind speed, humidity, evaporation, sunshine, rainfall, air quality information, etc. of a certain area as input, and can be used to predict changes in future rainfall and temperature information in the area.

[0085] Specifically, the embodiment of the present invention adopts the China Meteorological Elements-18 Leagues and Cities in Eastern Inner Mongolia from 2002 to 2021. The site observation data set of different element variables, and the feature extraction of multivariate and multi-scale information is performed according to the above steps, and the features of the two parts are integrated as input to extract the long-term time trend and the seasonal trend, and then the data enhancement of the standardized flow is performed, and then the prediction model is trained and tested to obtain the optimal prediction model. The following Table 1 is a part of the multivariate data information of different stations in Inner Mongolia in June 2018. The table shows the evapotranspiration, ground temperature, daily rainfall, average air pressure, average humidity, sunshine hours, average temperature, average 2-minute wind speed, maximum wind speed and wind direction of different stations on June 11.

[0086] Table 1

[0087]

[0088] Taking the prediction of future temperature changes as an example, the embodiment of the present invention can be a single variable input, that is, inputting the temperature values ​​of each station at different times, or multiple variables such as rainfall, air pressure, wind speed, etc. can be added to predict future temperature changes.

[0089] Figure 3 Monthly observed rainfall data for station 54218 from 2012 to 2022. Figure 4 The input data and forecast data for the rainfall at station 54218 are as follows. The input data are all the data for the station in Table 1, namely, evapotranspiration, ground temperature, daily rainfall, average air pressure, average humidity, sunshine hours, average temperature, average 2-minute wind speed, maximum wind speed and wind direction. The forecast data are the predicted future rainfall data.

[0090] Figure 5 The temperature input data and forecast data for station 54117. The input data are all the data of the station in Table 1, namely, evapotranspiration, ground temperature, daily rainfall, average air pressure, average humidity, sunshine hours, average temperature, average 2-minute wind speed, maximum wind speed and wind direction. The forecast data are the predicted future temperature data.

[0091] Figure 4 and Figure 5 All of them reflect examples of using multivariate data as input to predict future data of a single variable. When training the model, data from 2002 to 2011 can be selected as training data, and rainfall (or temperature, or other single variables) from 2012 to 2021 can be selected as the true value to be output. In the LOSS function, it is compared with the predicted value (predicted using data from 2002 to 2011 as input) to update the model parameters. The prediction accuracy can be improved through multivariate input, which is due to the extraction of multivariate features and features between variables. For a trained model, data from 2012 to 2021 can be used as input to predict rainfall or temperature and other single variables from 2022 to 2031. The prediction results are as follows: Figure 4 and Figure 5 shown.

[0092] Similarly, taking all the data of each station in Table 1 as input, we can obtain the corresponding average temperature forecast results and some rainfall forecast maps for 18 leagues and cities, as shown in Figure 6 As shown, the interannual variation results of the annual average temperature and the interannual variation results of the annual precipitation are further obtained, as shown in Figure 7 and Fig. 9 shown.

[0093] In order to verify the prediction accuracy of this algorithm, different algorithms are applied to predict the monthly rainfall data information in 2021, and the MAE and MSE values ​​of the algorithms are calculated. The specific results are compared in the prediction accuracy in Table 2.

[0094] Table 2

[0095]

[0096] Although the present invention has been described through preferred embodiments, the present invention is not limited to the embodiments described herein but includes various changes and modifications that may be made without departing from the scope of the present invention.

Claims

1. A long time series prediction method based on standardized flow, which is used for long-term meteorological prediction, characterized in that: The steps include: Step 1, data standardization; the data is a large amount of long-term series data of multiple types and non-uniform units; the data includes at least two of the following data: temperature, air pressure, wind speed, humidity, evaporation, sunshine, rainfall and air quality information of a certain area; Step 2: Extract the multivariate and multiscale information X of the long time series data in , the scale includes seconds, minutes, hours, days, months, and years; the extraction method is as follows: Step 21: Extract the correlation information between different data, and increase the weight of variables with high prediction importance to obtain multivariate correlation information; Step 22: Extract multi-scale information; Step 23: Fuse the multivariate association information with the multiscale information to obtain the multivariate multiscale information X in a unified space in ; Step 3: Based on the multivariate multi-scale information X in , respectively extract long-term trend information F t , seasonal trend information F s and dispersion information F d ; The extraction of long-term trend information F t , is to obtain the overall trend of the sequence through a linear model containing a moving average kernel; extract seasonal trend information F s , is to map the obtained features to the frequency domain space through fast discrete Fourier transform to obtain the periodic change law of the sequence; extract the discreteness information F d , is by including trend information F t And seasonal information F s The formula is as follows: Step 4, merging the different feature data obtained in step 3; Step 5: According to the fused features, data prediction based on the standardized flow is performed to predict the changes in future rainfall and temperature information in the certain area.

2. The long time series prediction method based on standardized flow according to claim 1 is characterized in that: In step 1, the Z-score standardization method is used to unify the data into the same dimension, and the method is as follows: Calculate the mean μ and standard deviation δ of the original data, and use the following formulas for different input data x i0 To standardize: In the formula, x i is x i0 After normalization, x i0 ∈R dx , dx≥1.

3. The long time series prediction method based on standardized flow according to claim 1 is characterized in that: In step 21, the correlation information between different data is extracted, and the formula is as follows: in Represents the relationship between different data variables, f and f -1 Represents fast Fourier transform and its inverse transform, x i and x j Represent different data variables respectively; Then, the Softmax function is used to highlight the relationship values ​​that are important for prediction. The higher the importance, the greater the weight set. The weight W Cr It is expressed as follows: The step 22 extracts multi-scale information, and the formula is as follows: Where Δ S represents multi-scale time variable information, W S and b S are the optimal weights and biases trained in the network, express The transpose of (b S )′ represents b S The transpose of Represents the values ​​of different data variables at different time scales. The scale S includes seconds, minutes, hours, days, months, and years. In step 23, the fusion formula is as follows: X in =(W v (W Cr x i )+b v )+Δ S Where W v and b v Represents the optimal weights and biases obtained after network training.

4. The long time series prediction method based on standardized flow according to claim 1 is characterized in that: Step 3, extracting long-term trend information F t The formula is as follows: F t =Linear(X in )=W t X in +b t Where W t , b t They are the weight and bias terms in the process of trend information extraction. Linear() means that the linear transformation of the input data is achieved by performing weighted summation and biasing operations on the input.

5. The long time series prediction method based on standardized flow according to claim 1 is characterized in that: In step 3, the seasonal trend information F is extracted by fast Fourier transform. s The formula is as follows: in, Yes X in The sth scale variable in , s represents any item between 0 and N-1, N is X in The number of variables in 6. The long time series prediction method based on standardized flow according to claim 1 is characterized in that: In step 4, the method for fusing feature data containing different information is as follows: F=W sea F s +W tre F t +W dis F d Among them, W sea Represents F s The learnable weight parameter, W tre Represents F t The learnable weight parameter, W dis Represents F d The learnable weight parameters.

7. The long time series prediction method based on standardized flow according to claim 6 is characterized in that: In step 5, the accuracy of data prediction is enhanced by learning the distribution of latent states of features using a data enhancement method, as follows: Step 51: Draw a random variable ε∽N(0,I) from the Gaussian distribution. The initial state of the standardized flow is expressed as: z0=FCN μ (F)+FCN δ (F)·e Among them, FCN μ and FCN δ are two fully connected networks, μ and σ represent different ResNet layer structures; Step 52: The continuous iteration of the overall normalization flow is expressed as: with t =FCN μ (F,z t-1 )+FCN δ (F,z t-1 ) with t-1 z t It outputs the continuously optimized final prediction value, t=1,....T, T is a certain moment in the future time series to be predicted, and ultimately improves the reliability of the predicted value by learning the distribution of latent variables.

8. The long time series prediction method based on standardized flow according to claim 7 is characterized in that: In the data enhancement method, the loss function is set as follows: L=MSE(z t ,Y) Calculate the predicted value z t The error between the true value Y and the training model parameters is optimized until the model converges.

Images