Intelligent prediction method for surface turbulence flux based on iTransform

Abstract

The invention belongs to the technical field of deep learning technology and atmospheric science, and particularly relates to an iTransform-based surface turbulence flux intelligent prediction method, which comprises the following steps of: obtaining original flux meteorological data; flux meteorological data processing; constructing a prediction model; model training; and evaluating the model. According to the method, an iTransform model is adopted, firstly, an embedded coding mode is changed, time sequences are transposed, and sequences of each variable at all time points are independently embedded into a token, so that correlation among the variables in flux meteorological data is more effectively captured; secondly, the multivariable attention mechanism and the feedforward neural network responsibility are inverted, the inverted multivariable attention mechanism responsibility is the correlation between capture variables or time sequences, and the feedforward neural network responsibility is to encode and decode each time sequence to extract internal characteristics of the time sequence, so that better prediction performance is achieved.

Description

Technical Field

[0001] The present invention belongs to the fields of deep learning technology and atmospheric science technology, and specifically relates to an intelligent prediction method for surface turbulent flux based on iTransformer. Background Art

[0002] Friction velocity, kinetic sensible heat flux and kinetic latent heat flux are key parameters that together describe the complex interaction and energy exchange between the atmosphere and the surface. In the atmospheric boundary layer, friction velocity reflects turbulent friction and affects the vertical distribution of wind speed and the development of the boundary layer. Kinetic sensible heat flux is the heat exchange in the form of turbulence between the atmosphere and the surface, which plays a role in regulating surface temperature and local climate. Kinematic latent heat flux represents the heat flux released or absorbed by water vapor during phase change, which is very important for climate analysis, resource management and environmental planning. These three interact with each other and jointly affect the atmospheric system, reflecting the complex energy exchange mechanism between the atmosphere and the surface. The traditional Morning-Obukhov similarity theory (MOST) is a commonly used theoretical method to describe the relationship between turbulence and meteorological elements such as temperature, humidity, and wind speed in the near-surface atmosphere. However, due to the large limitations of the MOST method, its prediction performance is poor. Therefore, when studying surface turbulent flux, it is necessary to combine other methods and means for comprehensive analysis and prediction.

[0003] In recent years, the rapid progress of deep learning (DL) and its innovative applications in meteorology have provided a promising approach. Compared with traditional MOST methods, DL models are able to handle highly complex and nonlinear problems. In addition, through a large amount of training data, DL models can perform multi-layer nonlinear operations on the data, learn the complex nonlinear relationships of the data, and more accurately capture the inherent laws and patterns of the data. Therefore, DL models can provide a new approach for the prediction of surface turbulent fluxes. Among them, the Transformer model is widely used in the prediction of time series because it can effectively process sequence data and capture long-distance dependencies. However, the Transformer model cannot fully reveal the complex relationships between multiple variables, and therefore cannot efficiently handle the correlation between multivariate variables. Summary of the invention

[0004] In order to solve the technical problem that the above-mentioned Transformer model cannot fully reveal the complex relationship between multiple variables, the present invention provides an intelligent prediction method for surface turbulent flux based on iTransformer.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] An intelligent prediction method for surface turbulent flux based on iTransformer comprises the following steps:

[0007] S1, acquisition of original flux meteorological data;

[0008] S2, flux meteorological data processing: remove the outliers and missing values ​​in the original flux meteorological data obtained in S1, and use physical formulas to calculate the input data, which includes wind speed, potential temperature gradient, Richardson number water vapor mixing ratio and surface turbulent flux observations. The input data is divided into training set, validation set and test set;

[0009] S3. Construct prediction model: Establish the iTransformer surface turbulent flux intelligent prediction model based on deep learning method; the basic architecture of the model includes the following layers: input layer, embedding coding layer, iTransformer layer, projection layer and output layer, among which the iTransformer layer includes multivariate attention module, feedforward neural network module and layer normalization;

[0010] S4, model training: using the mean square error (MSE) between the predicted value and the true value of the surface turbulent flux as the loss function of the model, the training set and validation set obtained in S2 are input into the prediction model constructed in S3 for training until the model converges;

[0011] S5. Model evaluation: Use the test set of S2 to verify and evaluate the prediction model after S4 convergence. Calculate the root mean square error RMSE, mean absolute error MAE and Pearson correlation coefficient R between the predicted value and the true value of the surface turbulent flux as evaluation indicators, and compare the results with the traditional MOST method and the existing prediction model artificial neural network ANN.

[0012] The original flux meteorological data in S1 are: air density, wind speed, temperature, pressure, upward long-wave radiation, relative humidity, water content, friction speed, sensible heat flux, and latent heat flux; the original flux meteorological data provided by the Beijing Meteorological Tower is obtained in S1, and the sampling time of the Meteorological Tower is the whole year of 2023, and the sampling time frequency is 30 minutes.

[0013] The method for processing the collected original flux meteorological data in S2 is: removing missing values ​​and invalid values, obtaining wind speed, potential temperature gradient, Richardson number and water vapor mixing ratio changes as model input variables according to physical formulas in meteorology; obtaining surface turbulent flux model labels according to physical formulas for calculating surface turbulent flux in meteorology: friction velocity, kinetic sensible heat flux, kinetic latent heat flux; the processed input data includes wind speed, potential temperature gradient, Richardson number water vapor mixing ratio and surface turbulent flux observations, and constructing a training set, a validation set and a test set for the obtained input data in a ratio of 7:2:1.

[0014] The prediction model constructed in S3 includes:

[0015] The input layer passes data such as wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and surface turbulent flux label to the embedded coding layer;

[0016] The embedding layer transposes the input data and embeds the sequence of each variable at all time points into a token.

[0017] The iTransformer layer predicts the surface turbulent flux. First, the token obtained by embedding the encoding layer is passed to the multivariate attention module with residual connection. Secondly, the multivariate attention module can effectively capture the relationship between the input variables wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and the predicted label surface turbulent flux, and obtain Q, K, V∈R through linear projection. N×dk , use Q and K point multiplication to calculate the score between wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and surface turbulent flux, and obtain the score matrix A of the correlation between surface turbulent flux and input variables; then, use the normalization strategy to reduce the error of the input data and pass it to the feedforward neural network module for calculation; and perform layer normalization and projection operations on the calculation results, and finally pass the prediction results to the output layer to obtain the predicted value of the surface turbulent flux.

[0018] The calculation formula of the score matrix A is:

[0019]

[0020] where d k is the projection dimension, q i is the query vector and key of the token corresponding to the i-th input variable, k j is the query vector and key of the token corresponding to the jth input variable; the obtained score matrix A∈R N×N represents the correlation between the i-th and j-th input variables.

[0021] The model training method in S4 is: calculate the mean square error MSE between the predicted value and the true value of the surface turbulent flux as the loss function of the model, and use the training set and verification set divided by S2 to train and verify the model constructed in S3 until the model converges. The mean square error MSE formula is as follows:

[0022]

[0023] Where y_pred is the predicted value of friction velocity, kinematic sensible heat flux, and kinematic latent heat flux output by the model, y_true is the true observed value of friction velocity, kinematic sensible heat flux, and kinematic latent heat flux, and n is the number of samples.

[0024] The specific execution process of the model is as follows:

[0025] S4.1, the processed flux meteorological data include wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio and surface turbulent flux label, and the input layer passes these data to the embedding coding layer;

[0026] S4.2, the embedding coding layer transposes the flux meteorological data passed by the input layer and encodes the sequence of each variable at all time points into a token;

[0027] S4.3. Perform a multivariate attention mechanism on the encoded tokens to calculate the correlation score matrix between the input variables wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and the surface turbulent flux label;

[0028] S4.4, performing layer normalization and feed-forward neural network layer operations according to the correlation score matrix, and using residual connections for calculation;

[0029] S4.5. Perform layer normalization and projection on the calculation results and transfer them to the output layer to obtain the final surface turbulent flux prediction results.

[0030] The model evaluation method in S5 is: using the root mean square error RMSE, mean absolute error MAE and Pearson correlation coefficient R between the predicted value and the true value of the surface turbulent flux as evaluation indicators, using the test set divided by S2 to test the converged model after S4 is completed, and comparing the results with the traditional MOST method and the existing prediction model artificial neural network (ANN). The evaluation index related formula is as follows:

[0031]

[0032] Where y_pred is the predicted value of friction velocity, kinematic heat flux, and kinematic latent heat flux output by the model, y_true is the true observed value of friction velocity, kinematic heat flux, and kinematic latent heat flux, and n is the number of samples. cov(y_pred,y_true) represents the covariance between the predicted value and the true value, σ y_pred represents the standard deviation of the predicted value, σ y_true Indicates the standard deviation of the true value.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The surface turbulent flux intelligent prediction method based on iTransformer proposed in the present invention has higher correlation and smaller error, and better prediction performance; and the iTransformer model is adopted in the method of the present invention. First, the embedding coding method is changed. By transposing the time series, the sequence of each variable at all time points is independently embedded into a token, so as to more effectively capture the correlation between variables in the flux meteorological data; secondly, the multivariate attention mechanism and the feedforward neural network responsibilities are inverted. The inverted multivariate attention mechanism is responsible for capturing the correlation between variables or time series, and the feedforward neural network is responsible for encoding and decoding each time series to extract its internal features, so as to achieve better prediction performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0036] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0037] Figure 1 is a flow chart of the steps of the present invention;

[0038] Figure 2 This is a model diagram of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. These descriptions are only to further illustrate the features and advantages of the present invention, rather than to limit the claims of the present invention. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0040] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0041] The specific implementation process of the present invention is as follows Figure 1 As shown, the specific implementation steps are as follows:

[0042] 1. Collect raw flux meteorological data at different time points

[0043] Air density, wind speed, temperature, pressure, upward long-wave radiation, relative humidity, water content, friction velocity, sensible heat flux, latent heat flux. The data dimension collected is 17520×10, where 17520 means the collection time is the whole year of 2023, the sampling time interval is 30 minutes, and 10 represents the number of variables collected.

[0044] 2. Process the collected raw flux meteorological data

[0045] First, the missing and invalid values ​​in the collected original flux meteorological data are removed;

[0046] Secondly, the following physical formulas are used to obtain the input variables: wind speed (V), potential temperature gradient (ΔQ surface_high ), Richardson number (R i ), water vapor mixing ratio change (Δq); wind speed can be collected directly, and the calculation formulas for the remaining variables are as follows:

[0047] The formula for calculating the potential temperature gradient is:

[0048]

[0049] ΔQ surface_high is the potential temperature gradient, T is Kelvin degrees Celsius, P0 = 1000hPa represents the standard atmospheric pressure, P is the corresponding air pressure at height h, L wout is the upward long-wave radiation, ε = 1 is the emissivity, σ =

[0050] 5.673067×10 -8 is the Stefan-Boltzmann constant.

[0051] The formula for calculating the Richardson number is:

[0052]

[0053] R i is the Richardson number, g = 9.81 is the gravitational acceleration, Q is the potential temperature, and u and v are the velocity components.

[0054] The formula for calculating the change of water vapor mixing ratio is:

[0055]

[0056] Δq is the change in water vapor mixing ratio, M avail is soil moisture availability, RH is relative humidity, e s0

[0057] =6.11hPa is the saturated water vapor pressure at standard temperature (T0 = 273.15K), L = 2.5*10 6 J / kg (0℃, 1 standard atmosphere) is the latent heat of vaporization of water, ε=0.622 is the molecular weight ratio of water vapor to dry air, and P is the atmospheric pressure (unit: hpa).

[0058] M avail Soil water availability is calculated using the following formula:

[0059]

[0060] q fc =0.41 is the maximum amount of water that the soil can hold;

[0061] The observed values ​​of surface turbulent flux include friction velocity, kinetic sensible heat flux and kinetic latent heat flux. The kinetic sensible heat flux and kinetic latent heat flux are obtained by dividing the sensible heat flux and latent heat flux by the air density ρ and the air specific heat capacity cp=1004. The input data are divided to construct the training set, validation set and test set in a ratio of 7:2:1.

[0062] 3. Build the model

[0063] Establish an intelligent prediction model for surface turbulent flux based on deep learning technology iTransformer. Figure 2 As shown in the figure, the basic architecture of the model includes the following layers: input layer, embedding coding layer, iTransformer layer, projection layer and output layer, among which the iTransformer layer contains a multivariate attention module, a feedforward neural network module and layer normalization.

[0064] The input layer passes data such as wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and surface turbulent flux label to the embedded coding layer.

[0065] The embedding coding layer transposes the input data and embeds the sequence of each variable at all time points as a token. The dimension of the flux meteorological data is converted from (32, 48, 5) to (32, 5, 48), where 32 is the BatchSize, 48 is the length of the time series, and 5 is the feature dimension.

[0066] The iTransformer layer predicts the surface turbulent flux. First, the token obtained by embedding the encoding layer is passed to the multivariate attention module with residual connection. Secondly, the multivariate attention module can effectively capture the relationship between the input variables wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and the predicted label surface turbulent flux, and obtain Q, K, The scores between wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and surface turbulent flux are calculated by point multiplication of Q and K, and the score matrix A of the correlation between surface turbulent flux and input variables is obtained; then, the normalization strategy is used to reduce the error of the input data and then passed to the feedforward neural network module for calculation; the calculation results are layer normalized and projected, and finally the prediction results are passed to the output layer to obtain the predicted value of the surface turbulent flux.

[0067] The specific calculation formula of the score matrix A is as follows:

[0068]

[0069] where d k is the projection dimension, q i is the query vector and key of the token corresponding to the i-th input variable, k j is the query vector and key of the token corresponding to the jth input variable; the obtained relevance score matrix A∈R N×N Shows the correlation between the i-th and j-th input variables.

[0070] 4. Model Training

[0071] The mean square error (MSE) between the predicted value and the true value of the surface turbulent flux is calculated as the loss function of the model. The model constructed in S3 is trained and verified using the training set and validation set divided by S2 until the model converges. The mean square error (MSE) formula is as follows:

[0072]

[0073] Where y_pred is the predicted value of friction velocity, kinematic sensible heat flux, and kinematic latent heat flux output by the model, y_true is the true observed value of friction velocity, kinematic sensible heat flux, and kinematic latent heat flux, and n is the number of samples.

[0074] The specific execution process of the model is as follows:

[0075] 1) The processed flux meteorological data include wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio and surface turbulent flux label. The input layer passes these data to the embedded coding layer.

[0076] 2) The embedding coding layer transposes the flux meteorological data passed by the input layer and encodes the sequence of each variable at all time points into a token.

[0077] 3) Perform a multivariate attention mechanism on the encoded tokens to calculate the correlation score matrix between the input variables wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and the surface turbulent flux label.

[0078] 4) Perform layer normalization and feed-forward neural network layer operations based on the relevance score matrix, using residual connections for calculations.

[0079] 5) The calculation results are normalized, projected and passed to the output layer to obtain the final surface turbulent flux prediction results.

[0080] 5. Model Evaluation

[0081] The root mean square error RMSE, mean absolute error MAE and Pearson correlation coefficient R between the predicted value and the true value of the surface turbulent flux are used as evaluation indicators. The converged model is tested using a test set, and the results are compared with the traditional MOST method and the existing prediction model artificial neural network (ANN). The relevant formula is as follows:

[0082]

[0083] Where y_pred is the predicted value of friction velocity, kinematic heat flux, and kinematic latent heat flux output by the model, y_true is the true observed value of friction velocity, kinematic heat flux, and kinematic latent heat flux, and n is the number of samples. cov(y_pred,y_true) represents the covariance between the predicted value and the true value, σ y_pred represents the standard deviation of the predicted value, σ y_true The standard deviation of the true value is shown in Table 1. The experimental results of iTransformer and the traditional MOST method are compared. The experimental results of iTransformer and artificial neural network (ANN) are compared. The present invention uses the deep learning method of iTransformer to capture the correlation between wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and surface turbulent flux, so as to achieve better prediction effect of surface turbulent flux.

[0084] Table 1. Comparison of experimental results between the Transformer model and the MOST method

[0085]

[0086] Table 2 Comparison of experimental results between Transformer model and ANN model

[0087]

[0088] Only the preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention, and various changes should be included in the protection scope of the present invention.

Claims

1. An intelligent prediction method for surface turbulent flux based on iTransformer, characterized in that: The following steps are involved: S1, acquisition of original flux meteorological data; S2, flux meteorological data processing: remove the outliers and missing values ​​in the original flux meteorological data obtained in S1, and use physical formulas to calculate the input data, which includes wind speed, potential temperature gradient, Richardson number water vapor mixing ratio and surface turbulent flux observations. The input data is divided into training set, validation set and test set; S3. Construct prediction model: Establish the iTransformer surface turbulent flux intelligent prediction model based on deep learning method; the basic architecture of the model includes the following layers: input layer, embedding coding layer, iTransformer layer, projection layer and output layer, among which the iTransformer layer includes multivariate attention module, feedforward neural network module and layer normalization; S4, model training: using the mean square error (MSE) between the predicted value and the true value of the surface turbulent flux as the loss function of the model, the training set and validation set obtained in S2 are input into the prediction model constructed in S3 for training until the model converges; S5. Model evaluation: Use the test set of S2 to verify and evaluate the prediction model after S4 convergence. Calculate the root mean square error RMSE, mean absolute error MAE and Pearson correlation coefficient R between the predicted value and the true value of the surface turbulent flux as evaluation indicators, and compare the results with the traditional MOST method and the existing prediction model artificial neural network ANN.

2. According to claim 1, a surface turbulent flux intelligent prediction method based on iTransformer is characterized in that: The original flux meteorological data in S1 are: air density, wind speed, temperature, pressure, upward long-wave radiation, relative humidity, water content, friction speed, sensible heat flux, and latent heat flux; the original flux meteorological data provided by the Beijing Meteorological Tower is obtained in S1, and the sampling time of the Meteorological Tower is the whole year of 2023, and the sampling time frequency is 30 minutes.

3. The method for intelligent prediction of surface turbulent flux based on iTransformer according to claim 1, characterized in that: The method for processing the collected original flux meteorological data in S2 is: removing missing values ​​and invalid values, obtaining wind speed, potential temperature gradient, Richardson number and water vapor mixing ratio changes as model input variables according to physical formulas in meteorology; obtaining surface turbulent flux model labels according to physical formulas for calculating surface turbulent flux in meteorology: friction velocity, kinetic sensible heat flux, kinetic latent heat flux; the processed input data includes wind speed, potential temperature gradient, Richardson number water vapor mixing ratio and surface turbulent flux observations, and constructing a training set, a validation set and a test set for the obtained input data in a ratio of 7:2:

1.

4. The method for intelligent prediction of surface turbulent flux based on iTransformer according to claim 1, characterized in that: The prediction model constructed in S3 includes: The input layer passes data such as wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and surface turbulent flux label to the embedded coding layer; The embedding layer transposes the input data and embeds the sequence of each variable at all time points into a token. The iTransformer layer predicts the surface turbulent flux. First, the token obtained by embedding the encoding layer is passed to the multivariate attention module with residual connection. Secondly, the multivariate attention module can effectively capture the relationship between the input variables wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and the predicted label surface turbulent flux, and obtain Q, K, V∈R through linear projection. N×dk , use Q and K point multiplication to calculate the score between wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and surface turbulent flux, and obtain the score matrix A of the correlation between surface turbulent flux and input variables; then, use the normalization strategy to reduce the error of the input data and pass it to the feedforward neural network module for calculation; and perform layer normalization and projection operations on the calculation results, and finally pass the prediction results to the output layer to obtain the predicted value of the surface turbulent flux.

5. The method for intelligent prediction of surface turbulent flux based on iTransformer according to claim 4, characterized in that: The calculation formula of the score matrix A is: where d k is the projection dimension, q i is the query vector and key of the token corresponding to the i-th input variable, k j is the query vector and key of the token corresponding to the jth input variable; the obtained score matrix A∈R N×N represents the correlation between the i-th and j-th input variables.

6. The method for intelligent prediction of surface turbulent flux based on iTransformer according to claim 1, characterized in that: The model training method in S4 is: calculate the mean square error MSE between the predicted value and the true value of the surface turbulent flux as the loss function of the model, and use the training set and verification set divided by S2 to train and verify the model constructed in S3 until the model converges. The mean square error MSE formula is as follows: Where y_pred is the predicted value of friction velocity, kinematic sensible heat flux, and kinematic latent heat flux output by the model, y_true is the true observed value of friction velocity, kinematic sensible heat flux, and kinematic latent heat flux, and n is the number of samples.

7. The method for intelligent prediction of surface turbulent flux based on iTransformer according to claim 6, characterized in that: The specific execution process of the model is as follows: S4.1, the processed flux meteorological data include wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio and surface turbulent flux label, and the input layer passes these data to the embedding coding layer; S4.2, the embedding coding layer transposes the flux meteorological data passed by the input layer and encodes the sequence of each variable at all time points into a token; S4.

3. Perform a multivariate attention mechanism on the encoded tokens to calculate the correlation score matrix between the input variables wind speed, potential temperature gradient, Richardson number, water vapor mixing ratio change and the surface turbulent flux label; S4.4, performing layer normalization and feed-forward neural network layer operations according to the correlation score matrix, and using residual connections for calculation; S4.

5. Perform layer normalization and projection on the calculation results and transfer them to the output layer to obtain the final surface turbulent flux prediction results.

8. The method for intelligent prediction of surface turbulent flux based on iTransformer according to claim 1, characterized in that: The model evaluation method in S5 is: using the root mean square error RMSE, mean absolute error MAE and Pearson correlation coefficient R between the predicted value and the true value of the surface turbulent flux as evaluation indicators, using the test set divided by S2 to test the converged model after S4 is completed, and comparing the results with the traditional MOST method and the existing prediction model artificial neural network (ANN). The evaluation index related formula is as follows: Where y_pred is the predicted value of friction velocity, kinematic heat flux, and kinematic latent heat flux output by the model, y_true is the true observed value of friction velocity, kinematic heat flux, and kinematic latent heat flux, and n is the number of samples. cov(y_pred,y_true) represents the covariance between the predicted value and the true value, σ y_pred represents the standard deviation of the predicted value, σ y_true Indicates the standard deviation of the true value.

Images