Meteorological feature prediction method based on U-Net and PatchMixer modules

Abstract

The invention provides a meteorological feature prediction method based on a U-Net module and a PatchMixer module, and aims to improve the prediction precision. The method comprises the following steps: firstly, normalizing meteorological sequence data, dynamically adjusting a feature matrix through an adaptive module, and enhancing local correlation of the data; and then, filling and blocking the adjusted feature matrix, and embedding the feature matrix into a high-dimensional space through linear mapping to provide support for subsequent feature extraction. According to the model, a plurality of PatchMixer blocks are used for building a U-Net, and the model comprises an encoder, a bottom feature aggregation layer and a decoder. The encoder extracts meteorological features of different levels layer by layer, the bottom layer further integrates deep features, and the decoder fuses encoding features through jump connection and residual connection to generate a prediction result. And finally, optimizing the model by using a mean square error (MSE) loss function. Experimental results show that the meteorological prediction precision is remarkably improved, the model complexity is reduced, and high efficiency and low errors are achieved.

Description

Technical Field

[0001] The present invention is a meteorological feature prediction method based on U-Net and PatchMixer modules, which involves the intersection of artificial intelligence and meteorological science and is suitable for high-precision prediction of meteorological features. The method models meteorological data and can play an important role in meteorological scenarios that require accurate prediction, such as agricultural production, environmental monitoring, and meteorological early warning, through adaptive feature adjustment and multi-level feature extraction, and can effectively improve the timeliness and accuracy of meteorological forecasts. Background Art

[0002] Weather forecasting relies on accurately extracting key features from time series data. Traditional methods often fail to capture local features and global trends in weather data, thus limiting the accuracy of forecasting. In recent years, the development of deep learning, especially the network structure of hierarchical feature extraction and adaptive dynamic modeling, has provided a new research direction for weather forecasting.

[0003] This paper proposes an improved method based on the combination of U-Net and PatchMixer modules. U-Net can not only realize multi-scale feature extraction, but also use jump connections to retain key feature information, thereby enhancing the integrity of feature transfer. The PatchMixer module efficiently extracts local and global features through a combination of deep separable convolution and point convolution, further improving the computational efficiency and prediction accuracy of the model. This method provides a new solution for efficient and accurate meteorological feature prediction. Summary of the invention

[0004] The present invention provides a meteorological feature prediction method based on U-Net and PatchMixer modules, which has good performance in processing complex time dependencies and improving prediction accuracy. The technical solution is as follows:

[0005] Step 1: Assume that the input meteorological data sequence contains L time steps, each time step has M variables, represented as an L×M matrix Where M means that there are M meteorological related features in each time step, including temperature, humidity, air pressure and other meteorological features;

[0006] Step 2: Normalize the input data to obtain the normalized feature matrix x norm , the specific formula is as follows, where μ is the mean of the input data x and σ is the standard deviation of the input data x:

[0007]

[0008] Step 3: Through matrix multiplication and nonlinear activation function, the features are dynamically adjusted to adapt to local changes in the data, and the feature matrix x is adaptively adjusted by the adaptive module.norm Perform local adaptive adjustment and use nonlinear activation function transformation to enhance the local correlation of features. The process is as follows:

[0009] x adapt =ReLU(W adapt ·x norm )

[0010] Where W adapt is the adaptive weight matrix, ReLU is the activation function, x adapt is the output matrix;

[0011] Step 4: Transform the matrix x adapt Expand through the padding layer and then decompose the matrix x with stride S adapt Get the feature matrix x after M blocks patch ;

[0012] Step 5: After the block feature matrix x patch Perform linear mapping to map it to the specified high-dimensional space so that subsequent modules can extract its features. The linear mapping formula is as follows, where W P is the mapping matrix used to increase the feature dimension, x proj For a higher dimensional feature matrix:

[0013] x proj =W P ·x patch

[0014] Step 6: Use U-Net to train the feature matrix x proj Perform multi-level processing. This module consists of three parts:

[0015] Step 6.1: Use multiple layers of PatchMixer blocks to build the encoder part. Each layer contains several PatchMixer blocks, which extract features at different levels layer by layer to extract and aggregate input features.

[0016] Step 6.2: The bottom layer consists of multiple PatchMixer blocks. The bottom layer is located between the encoder and decoder and is used to perform the deepest aggregation of time series features.

[0017] Step 6.3: Use multiple layers of PatchMixer blocks to build the decoder part. Each layer contains several PatchMixer blocks. It traces back to the high-level output by layer-by-layer weighted feature fusion. Each layer fuses the decoded features with the encoded features through residual connections and nonlinear activation functions.

[0018] Step 7: Map the output of the decoder to the target output space through the prediction head, and use the residual connection to combine the decoded output with the initial features of the encoder to generate the prediction results for the next T time steps:

[0019]

[0020] Step 8: Use the mean squared error (MSE) loss function to calculate the loss using the following formula:

[0021]

[0022] in, represents the predicted value of the i-th time series, represents the true value of the i-th time series, M represents the number of features of the meteorological data series, It means to average the loss of each feature channel, summarize the losses of M channels, average the summarized losses over the entire time series, get the overall target loss, and evaluate the prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flow chart of a meteorological feature prediction method based on U-Net and PatchMixer modules provided in an embodiment of this patent.

[0025] Figure 2 A schematic diagram of a model of a meteorological feature prediction method based on U-Net and PatchMixer modules provided in an embodiment of this patent.

[0026] Figure 3 This is the combined structure diagram of U-Net and PatchMixer in this patent.

[0027] Figure 4 The experimental comparison results of the patented model U-PatchMixer and the current mainstream architecture of weather forecasting. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0029] like Figure 2As shown, the embodiment of the present invention provides a meteorological feature prediction method based on U-Net and PatchMixer modules, including:

[0030] Step 1: Select the public weather dataset, which is meteorological data recorded every ten minutes in 2020. Each set of meteorological data contains 21 features, including temperature, humidity, wind speed, etc. Set the lookback window length of the time series to 96, then the input sequence can be represented as a 96×21 matrix Set the output step size to 96. In the subsequent steps, the meteorological data of the 96 time steps of the input are used to predict the meteorological conditions of the next 96 time steps. The 96 input data and the 96 predicted data are adjacent but not overlapping. Each feature contains multiple blocks of data in units of 96.

[0031] Step 2: Normalize the input data to obtain the normalized feature matrix x norm ;

[0032] Step 3: Dynamically adjust the features to adapt to local changes in the data through matrix multiplication combined with nonlinear activation functions. First, use the adaptive module to adjust the feature matrix x norm Perform local adjustments. Next, apply a nonlinear activation function to transform the features to enhance their local relevance. The specific process is as follows:

[0033] x adapt =ReLU(W adapt ·x norm )

[0034] Where W adapt is the adaptive weight matrix, ReLU is the activation function, and d_modle is 512, so, is the output matrix;

[0035] Step 4: Transform the matrix x adapt Expand through the padding layer and then decompose the matrix x with a stride of 8 and a block length of 16 adapt , and get the feature matrix after 12 blocks

[0036] Step 5: After the block feature matrix x patch Perform linear mapping to map it to a specified high-dimensional space so that subsequent modules can extract its features. The linear mapping formula is as follows, where is a mapping matrix used to increase the feature dimension. For a higher dimensional feature matrix:

[0037] x proj =W P ·xpatch

[0038] Step 6: Use U-Net to train the feature matrix x proj To carry out multi-level processing, the architecture consists of three parts, such as Figure 3 As shown:

[0039] Step 6.1: The encoder is composed of 5 PatchMixer blocks. The encoder contains 3 layers. The first layer contains 1 PatchMixer block, the second layer contains 2 PatchMixer blocks, and the third layer contains 2 PatchMixer blocks. The encoder extracts features at different levels layer by layer to extract and aggregate input features.

[0040] Step 6.2: The bottom layer consists of 2 PatchMixer blocks, located between the encoder and decoder, for the deepest aggregation of time series features;

[0041] Step 6.3: The decoder consists of 5 PatchMixer blocks. The decoder also contains 3 layers. The first layer contains 1 PatchMixer block, the second layer contains 2 PatchMixer blocks, and the third layer contains 2 PatchMixer blocks. The decoder traces back to the high-level output by layer-by-layer weighted feature fusion. The decoder uses residual connections and nonlinear activation functions to fuse the encoded features at each layer.

[0042] Step 7: Map the output of the decoder to the target output space through the prediction head, and use the residual connection to combine the decoded output with the initial features of the encoder to generate the prediction results for the next 96 time steps:

[0043]

[0044] Step 8: Use the mean squared error (MSE) loss function to calculate the loss using the following formula:

[0045]

[0046] in, represents the predicted value of the i-th time series, represents the true value of the i-th time series, 21 means that the meteorological data contains 21 features, It means to average the losses in each feature channel, summarize the losses of each channel, average the summarized losses over the entire time series, get the overall target loss, and evaluate the prediction results. Figure 4The experimental comparison results of the patented model U-PatchMixer and the current mainstream weather forecasting architecture are shown, where DataName represents the data set, InputLength represents the input length, OutputLength represents the output length, Model_name represents the model name, mse represents the mean square error, and mae represents the mean absolute error. The results of mes and mae show that the patented model has improved the average performance of the prediction results of 21 meteorological characteristics in the weather data set, and the model has a good prediction effect in terms of temperature, humidity, air pressure, etc. The model not only improves the prediction accuracy, but also maintains good stability and response speed in the environment of real-time meteorological data updates, and adapts to rapidly changing meteorological conditions. This is particularly important for short-term meteorological forecasts in practical applications.

Claims

1. A meteorological feature prediction method based on U-Net and PatchMixer modules, characterized in that The following steps are involved: Step 1: Assume that the input meteorological data sequence contains L time steps, each time step has M variables, represented as an L×M matrix Where M means that there are M meteorological related features in each time step, including temperature, humidity, air pressure and other meteorological features; Step 2: Normalize the input data to obtain the normalized feature matrix x norm , the specific formula is as follows, where μ is the mean of the input data x and σ is the standard deviation of the input data x: Step 3: Through matrix multiplication and nonlinear activation function, the features are dynamically adjusted to adapt to local changes in the data, and the feature matrix x is adaptively adjusted by the adaptive module. norm Perform local adaptive adjustment and use nonlinear activation function transformation to enhance the local correlation of features. The process is as follows: x adapt =ReLU(W adapt ·x norm ) Where W adapt is the adaptive weight matrix, ReLU is the activation function, x adapt is the output matrix; Step 4: Transform the matrix x adapt Expanded by padding layer, then decomposed matrix x with stride S adapt Get the feature matrix x after M blocks patch ; Step 5: After the block feature matrix x patch Perform linear mapping to map it to the specified high-dimensional space so that subsequent modules can extract its features. The linear mapping formula is as follows, where W P is the mapping matrix used to increase the feature dimension, x proj For a higher dimensional feature matrix: x proj =W P ·x patch Step 6: Use U-Net to train the feature matrix x proj Perform multi-level processing. This module consists of three parts: Step 6.1: Use multiple layers of PatchMixer blocks to build the encoder part. Each layer contains several PatchMixer blocks, which extract features at different levels layer by layer to extract and aggregate input features. Step 6.2: The bottom layer consists of multiple PatchMixer blocks. The bottom layer is located between the encoder and decoder and is used to perform the deepest aggregation of time series features. Step 6.3: Use multiple layers of PatchMixer blocks to build the decoder part. Each layer contains several PatchMixer blocks. It traces back to the high-level output by layer-by-layer weighted feature fusion. Each layer fuses the decoded features with the encoded features through residual connections and nonlinear activation functions. Step 7: Map the output of the decoder to the target output space through the prediction head, and use the residual connection to combine the decoded output with the initial features of the encoder to generate the prediction results for the next T time steps: Step 8: Use the mean squared error (MSE) loss function to calculate the loss using the following formula: in, represents the predicted value of the i-th time series, represents the true value of the i-th time series, M represents the number of features of the meteorological data series, It means to average the loss of each feature channel, summarize the losses of M channels, average the summarized losses over the entire time series, get the overall target loss, and evaluate the prediction results.

Images