Extended-range forecasting system and method for intraseasonal oscillation components of meteorological elements based on Transformer

By improving the input layer, attention mechanism, and multilayer perceptron of the Transformer model, and by increasing the number of input times, the problems of physical interpretability and insufficient global attention in the forecast of intraseasonal oscillation components of meteorological elements were solved, thereby improving the accuracy of extended-range forecasts.

CN118569315BActive Publication Date: 2024-11-22WUXI UNIV +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411046032.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-22
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

Existing meteorological forecasting models based on Transformer models suffer from insufficient physical interpretability and global attention in extended-range forecasts of meteorological elements, and lack effective forecasting methods for the intraseasonal oscillation components of meteorological elements.

Method used

By improving the Transformer model, incorporating Cross-scale Embedding, AFNO3D blocks, SwishMLP blocks, and combining convolution operations with pixel shuffle layers, optimizing the input layer, attention mechanism, and multilayer perceptron, and increasing the number of input times, extended-range forecast models for the intraseasonal oscillation components of meteorological elements are constructed and trained.

Benefits of technology

It improves the forecast accuracy of the intra-seasonal oscillation component of meteorological elements and enhances the extended-range forecast performance, showing a significant improvement over the forecast results of the ordinary Transformer model and the CFS numerical model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118569315B_ABST
    Figure CN118569315B_ABST
Patent Text Reader

Abstract

The present invention discloses an extended-term forecast system and method for seasonal oscillation components of meteorological elements based on Transformer, which belongs to the field of meteorological forecasting. The system includes: a data collection module, a model optimization module, a model training module and a forecasting module; the data collection module is used to collect historical meteorological data, and use a filtering method to extract the seasonal oscillation components of meteorological elements in the historical meteorological data; the model optimization module is used to build and optimize the Transformer model to obtain an optimized model; the model training module trains and evaluates the optimized model based on the seasonal oscillation components of meteorological elements to obtain an extended-term forecast model; the forecasting module is used to use the extended-term forecast model for meteorological forecasting. The present invention improves the traditional neural network forecasting method, and at the same time, the forecast accuracy of the seasonal oscillation components of extended-term meteorological elements is improved by increasing the number of input times to the optimal value, thereby improving the effect of the extended-term forecast.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of meteorological forecasting, and in particular relates to a Transformer-based extended-range forecasting system and method for seasonal oscillation components of meteorological elements. Background Art

[0002] Forecasts with a validity period of 10-30 days are called extended-range forecasts in the meteorological community. Their forecast validity is between conventional weather forecasts and short-term climate forecasts. Since they just cover the upper limit of the predictable time of conventional weather forecasts and the lower limit of the time when external forcing begins to take effect, there are many difficulties and challenges in practice. Their theoretical research and forecast technology development have always been hot issues at home and abroad.

[0003] In 2023, the Huawei team developed the Pangu model and successfully carried out weather forecasts, providing a new solution for the meteorological community to solve various weather forecasting problems. However, most of the existing weather forecast models based on the Transformer model focus on weather-scale forecasts, and there are few models for extended-term forecasts of meteorological elements. At the same time, ordinary Transformer models have certain defects in physical interpretability and global attention considerations. Therefore, this method is improved based on the ordinary Transformer model, adding modules to make up for the defects of physical interpretability, global attention mechanism, etc., while increasing the number of input times, and using the improved model to perform extended-term forecasts of seasonal oscillation components of meteorological elements, thereby promoting the development of extended-term forecasts. Summary of the invention

[0004] The present invention aims to solve the deficiencies of the prior art and provides the following solutions:

[0005] The extended-range forecast system of seasonal oscillation components of meteorological elements based on Transformer includes: data collection module, model optimization module, model training module and forecast module;

[0006] The data collection module is used to collect historical meteorological data and extract the seasonal oscillation component of the meteorological elements of the historical meteorological data by using a filtering method;

[0007] The model optimization module is used to construct and optimize the Transformer model to obtain an optimized model;

[0008] The model training module performs training and evaluation on the optimized model based on the intra-seasonal oscillation component of the meteorological element to obtain an extended-range forecast model;

[0009] The forecast module is used to perform weather forecasting using the extended-range forecast model.

[0010] Preferably, the historical meteorological data includes: 4 high-altitude variables, 5 ground elements, 3 constants and 5 target variables;

[0011] The four upper air variables include: geopotential, temperature, radial wind and zonal wind;

[0012] The five ground elements include: temperature at 2m, mean sea level pressure, surface pressure, radial wind at 10m and zonal wind at 10m;

[0013] The three constants include: topography, land and sea mask, and latitude cosine value;

[0014] The five target variables include: apparent altitude at 500hPa, temperature field at 850hPa, radial wind field, latitudinal wind field and temperature field at 2m.

[0015] Preferably, the model optimization module includes: an input layer optimization unit, an attention mechanism optimization unit, a multi-layer perceptron optimization unit and a deconvolution optimization unit;

[0016] The input layer optimization unit is used to replace the Patch Embedding part in the original Transformer model with Cross-scale Embedding;

[0017] The attention mechanism optimization unit is used to replace the attention mechanism calculation part in the original Transformer model with the AFNO3D block;

[0018] The multi-layer perceptron optimization unit is used to replace the MLP block part in the original Transformer model with the SwishMLP block;

[0019] The deconvolution optimization unit is used to replace the deconvolution operation of the Patch Recovery part in the original Transformer model with a combination of a convolution operation and a Pixel Shuffle layer.

[0020] Preferably, the model training module includes: a first model training unit, an optimal time confirmation unit, a second model training unit and a model evaluation unit;

[0021] The model training unit uses time configurations with different numbers of input time, and uses the intra-seasonal oscillation components of the meteorological elements as training sets to train the optimized model, thereby obtaining models with different input time with a forecast validity of 10 days;

[0022] The optimal time confirmation unit uses the models of the different input times to forecast the five target variables, and takes the input time corresponding to the model with the smallest root mean square error value of the forecast result as the optimal input time;

[0023] The second model training unit uses the time configuration of the optimal input time number and takes the seasonal oscillation component of the meteorological element as a training set to train the optimized model to obtain a number of initial forecast models with a forecast time validity of 11-30 days;

[0024] The model evaluation unit uses several of the initial forecast models to forecast the five target variables, and calculates the root mean square error and the abnormal correlation coefficient of the forecast results. Then, the root mean square error and the abnormal correlation coefficient are used to evaluate the forecasting effects of several of the initial forecast models, and the one with the best effect is selected as the extended period forecast model.

[0025] The present invention also provides a Transformer-based extended-range forecasting method for seasonal oscillation components of meteorological elements, which is applied to any of the above-mentioned systems and comprises the following steps:

[0026] S1. Collect historical meteorological data and use filtering methods to extract seasonal oscillation components of meteorological elements of the historical meteorological data;

[0027] S2. Build and optimize the Transformer model to obtain the optimized model;

[0028] S3. Based on the seasonal oscillation component of the meteorological element, the optimized model is trained and evaluated to obtain an extended-range forecast model;

[0029] S4. Using the extended-range forecast model to perform weather forecasting.

[0030] Preferably, the historical meteorological data includes: 4 high-altitude variables, 5 ground elements, 3 constants and 5 target variables;

[0031] The four upper air variables include: geopotential, temperature, radial wind and zonal wind;

[0032] The five ground elements include: temperature at 2m, mean sea level pressure, surface pressure, radial wind at 10m and zonal wind at 10m;

[0033] The three constants include: topography, land and sea mask, and latitude cosine value;

[0034] The five target variables include: apparent altitude at 500hPa, temperature field at 850hPa, radial wind field, latitudinal wind field and temperature field at 2m.

[0035] Preferably, S2 includes:

[0036] Replace the Patch Embedding part in the original Transformer model with Cross-scale Embedding;

[0037] Replace the attention mechanism calculation part in the original Transformer model with the AFNO3D block;

[0038] Replace the MLP block in the original Transformer model with the SwishMLP block;

[0039] The deconvolution operation in the Patch Recovery part of the original Transformer model is replaced by a combination of convolution operation and Pixel Shuffle layer.

[0040] Preferably, S3 includes:

[0041] Using time configurations with different numbers of input times and taking the intra-seasonal oscillation components of the meteorological elements as training sets, the optimized model is trained to obtain models with several different input times with a forecast validity of 10 days;

[0042] Using the models with different input times to forecast the five target variables, and taking the input time corresponding to the model with the smallest root mean square error value of the forecast result as the optimal input time;

[0043] Using the time configuration of the optimal input time number and taking the seasonal oscillation component of the meteorological element as a training set, the optimized model is trained to obtain several initial forecast models with a forecast time validity of 11-30 days;

[0044] The five target variables are predicted using several of the initial prediction models, and the root mean square error and the abnormal correlation coefficient of the prediction results are calculated. The root mean square error and the abnormal correlation coefficient are then used to evaluate the prediction effects of the several initial prediction models, and the one with the best effect is selected as the extended period prediction model.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] Compared with the common Transformer model, the present invention has been optimized in terms of physical interpretability and global attention, improving the problems of poor physical interpretability and more partial attention of the traditional neural network forecasting method; at the same time, the forecast accuracy of the seasonal oscillation component of the extended-period meteorological elements is improved by increasing the number of input times to the optimal value, and the forecast effect of the extended-period forecast is improved;

[0047] The 10-30 day forecast results of the extended-period intraseasonal oscillation component forecasting method proposed in the present invention were compared with the forecast results of the ordinary Transformer model and the forecast results of the CFS numerical model, which improved the accuracy to a certain extent and achieved better results. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A schematic diagram of the system structure of an embodiment of the present invention;

[0050] Figure 2 It is a schematic diagram of an improved Transformer model obtained by modifying the traditional Transformer model in an embodiment of the present invention;

[0051] Figure 3 The figure is a schematic diagram of a method flow of an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] Embodiment 1

[0055] In this embodiment, if Figure 1 As shown, the extended-range forecast system of the intraseasonal oscillation component of meteorological elements based on Transformer includes: a data collection module, a model optimization module, a model training module and a forecast module.

[0056] The data collection module is used to collect historical meteorological data and use filtering methods to extract the seasonal oscillation components of meteorological elements in historical meteorological data.

[0057] The historical meteorological data include: 4 high-altitude variables, 5 ground elements, 3 constants and 5 target variables; the 4 high-altitude variables include: potential, temperature, radial wind and zonal wind; the 5 ground elements include: temperature at 2m, mean sea level pressure, surface pressure, radial wind at 10m and zonal wind at 10m; the 3 constants include: topography, land and sea mask and latitude cosine value; the 5 target variables include: apparent altitude at 500hPa, temperature field at 850hPa, radial wind field, zonal wind field and temperature field at 2m.

[0058] In this embodiment, 4 high-altitude variable fields (potential, temperature, radial wind and zonal wind) at 7 levels in the ERA5 reanalysis data set from 2010 to 2022, 5 ground element fields (temperature at 2m, mean sea level pressure, surface pressure, radial wind at 10m and zonal wind at 10m) and 2 constant fields (topography, sea and land mask and latitude cosine value) are downloaded, where the latitude cosine value of the corresponding grid point is calculated for the topography and sea and land mask at the same time. The forecast results of the CFS forecast system for the five target variables (500hPa apparent height, 850hPa temperature field, radial wind field, zonal wind field and 2m temperature field) for the whole year of 2022 are downloaded when the forecast time is 10-30 days. Afterwards, the seasonal oscillation component of the meteorological elements of the meteorological data for 10-30 days is extracted by filtering method. The filtering method is to use the original meteorological data minus the 90-day low-pass filter component and the first 15-day sliding average, and then perform a 5-day sliding average to finally complete the extraction of the seasonal oscillation component of the meteorological elements.

[0059] The model optimization module is used to build and optimize the Transformer model to obtain the optimized model.

[0060] The model optimization module includes: input layer optimization unit, attention mechanism optimization unit, multi-layer perceptron optimization unit and deconvolution optimization unit. The input layer optimization unit is used to replace the Patch Embedding part in the original Transformer model with Cross-scale Embedding; the attention mechanism optimization unit is used to replace the attention mechanism calculation part in the original Transformer model with the AFNO3D block; the multi-layer perceptron optimization unit is used to replace the MLP block part in the original Transformer model with the SwishMLP block; the deconvolution optimization unit is used to replace the deconvolution operation of the Patch Recovery part in the original Transformer model with a combination of convolution operation and Pixel Shuffle layer.

[0061] In this embodiment, the input layer optimization unit replaces the Patch Embedding module in the original Transformer with a Cross-scale Embedding module. The Cross-scale Embedding module processes the input data by using convolution layers with convolution kernels of different sizes, thereby improving the Transformer model's grasp of global attention while retaining local attention. The attention mechanism optimization unit changes the attention mechanism calculation part in the original Transformer to the AFNO3D block. The AFNO3D block introduces Fourier transform into the neural network calculation and attempts to learn the physical constraints behind the meteorological data. The multi-layer perceptron optimization unit changes the MLP block part in the original Transformer to the SwishMLP block. The SwishMLP block can better learn the nonlinear relationship between data by adding the Swish activation function to the original MLP block. The deconvolution optimization unit changes the deconvolution operation of the Patch Recovery part in the original Transformer to a combination of convolution and Pixel Shuffle layers. The combined layer avoids the checkerboard noise problem existing in the original deconvolution layer by learning the mapping relationship between low-resolution images and high-resolution images. Through the above optimization, the optimized model is obtained, such as Figure 2 shown.

[0062] The model training module trains and evaluates the optimized model based on the intraseasonal oscillation components of meteorological elements to obtain an extended-range forecast model.

[0063] The model training module includes: a first model training unit, an optimal time confirmation unit, a second model training unit and a model evaluation unit. The model training unit uses time configurations with different input time numbers, and uses the seasonal oscillation components of meteorological elements as training sets to train the optimized model, and obtains several models with different input time numbers with a forecast time limit of 10 days; the optimal time confirmation unit uses several models with different input time numbers to forecast 5 target variables, and uses the input time number corresponding to the model with the smallest root mean square error value of the forecast result as the optimal input time number; the second model training unit uses the time configuration of the optimal input time number, and uses the seasonal oscillation components of meteorological elements as training sets to train the optimized model, and obtains several initial forecast models with a forecast time limit of 11-30 days; the model evaluation unit uses several initial forecast models to forecast 5 target variables, and calculates the root mean square error and abnormal correlation coefficient of the forecast results, and then uses the root mean square error and abnormal correlation coefficient to evaluate the forecast effect of several initial forecast models, and selects the model with the best effect as the extended period forecast model.

[0064] In this embodiment, taking the model with a forecast time of 10 days as an example, the optimal number of input times is determined. The first model training unit uses a configuration with an input time of 3, 4, 5, 6, 7, 8, and 9 times, and uses the data from 2010 to 2019 in the seasonal oscillation component data of meteorological elements as the training set, and the data from 2020 to 2021 as the verification set to input the optimized model for training, and the forecast time is 10 days. In the process of training the optimized model, the Adam optimizer is used for training, and the model with the smallest loss function value is selected and then trained using the SGD optimizer. The model with the smallest loss function value is the final training result; the training parameters of the optimized model include: the learning rate is 1×10 -4 , the number of early stopping rounds is 2, and the minimum improvement is 1×10 -5 The optimal time confirmation unit uses models with different input time configurations to forecast the five target variables (500hPa apparent altitude, 850hPa temperature field, radial wind field, zonal wind field and 2m temperature field) for the whole year of 2022, and selects the input time corresponding to the model with the smallest RMSE value of the forecast result as the optimal input time. Among them, RMSE is defined as:

[0065] , (1)

[0066] in, f Represents the model prediction result, t is the ERA5 reanalysis data of the corresponding time, N forecasts Indicates the number of forecast times, N lat represents the number of latitudinal grid points, N lon represents the number of meridional grid points, L ( j ) indicates the latitude is j The weight factor when i Indicates the time. j Indicates latitude, k Indicates longitude. The second model training unit uses the selected optimal input time to train a model with a forecast time of 11-30 days. When training the 11-30 day model, the specific training method of the optimized model is that each model inherits the final model of the previous forecast time, and directly uses the SGD optimizer for tuning to obtain several initial forecast models with a forecast time of 11-30 days; the training parameters of the optimized model include: the learning rate is 1×10 -4 , the number of early stopping rounds is 2, and the minimum improvement is 1×10 -5The model evaluation unit uses 21 initial forecast models with a trained forecast validity of 10-30 days to forecast five target variables (apparent altitude at 500hPa, temperature field at 850hPa, radial wind field, zonal wind field and temperature field at 2m) for the whole year of 2022. The two evaluation indicators of the root mean square error RMSE and the anomaly correlation coefficient ACC of the forecast results are calculated, and the forecast effect of this model, CFS forecast and climate value forecast is evaluated. The definition of RMSE is as shown in formula (1), and the definition of ACC is:

[0067] , (2)

[0068] , (3)

[0069] in, f Represents the model prediction result, t is the ERA5 reanalysis data of the corresponding time, symbol ’ represents the centered variable, L ( j ) indicates the latitude is j The weight factor when lat ( j ) indicates taking the latitude, N lat represents the number of latitudinal grid points, i Indicates the time. j Indicates latitude, k Represents longitude. Then, the root mean square error and anomaly correlation coefficient are used to evaluate the forecasting effects of several initial forecast models, and the one with the best effect is selected as the extended-term forecast model.

[0070] The forecast module is used to make weather forecasts using the extended-range forecast model.

[0071] Embodiment 2

[0072] In this embodiment, if Figure 3 As shown, the extended-range forecasting method of the intra-seasonal oscillation component of meteorological elements based on Transformer includes the following steps:

[0073] S1. Collect historical meteorological data and use filtering methods to extract the seasonal oscillation components of meteorological elements in historical meteorological data.

[0074] The historical meteorological data include: 4 high-altitude variables, 5 ground elements, 3 constants and 5 target variables; the 4 high-altitude variables include: potential, temperature, radial wind and zonal wind; the 5 ground elements include: temperature at 2m, mean sea level pressure, surface pressure, radial wind at 10m and zonal wind at 10m; the 3 constants include: topography, sea and land mask and latitude cosine value; the 5 target variables include: apparent height at 500hPa, temperature field at 850hPa, radial wind field, zonal wind field and temperature field at 2m. The filtering method is to use the original meteorological data minus the 90-day low-pass filter component and the first 15-day sliding average, and then perform a 5-day sliding average to finally complete the extraction of the seasonal oscillation component of the meteorological elements.

[0075] S2. Build and optimize the Transformer model to obtain the optimized model.

[0076] S2 includes: replacing the Patch Embedding part in the original Transformer model with Cross-scaleEmbedding; replacing the attention mechanism calculation part in the original Transformer model with the AFNO3D block; replacing the MLP block part in the original Transformer model with the SwishMLP block; replacing the deconvolution operation of the PatchRecovery part in the original Transformer model with a combination of convolution operation and Pixel Shuffle layer.

[0077] S3. Based on the intra-seasonal oscillation components of meteorological elements, the optimized model is trained and evaluated to obtain an extended-range forecast model.

[0078] S3 includes: using time configurations with different numbers of input times, and taking the seasonal oscillation components of meteorological elements as training sets, training the optimized model, and obtaining several models with different input times with a forecast time limit of 10 days; using several models with different input times to forecast 5 target variables, and taking the number of input times corresponding to the model with the smallest root mean square error value of the forecast result as the optimal input time; using the time configuration with the optimal number of input times, and taking the seasonal oscillation components of meteorological elements as training sets, training the optimized model, and obtaining several initial forecast models with a forecast time limit of 11-30 days; using several initial forecast models to forecast 5 target variables, and calculating the root mean square error and anomaly correlation coefficient of the forecast results, and then using the root mean square error and anomaly correlation coefficient to evaluate the forecast effect of several initial forecast models, and selecting the model with the best effect as the extended period forecast model.

[0079] S4. Use the extended-range forecast model to make weather forecasts.

[0080] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. The extended-range forecast system of seasonal oscillation components of meteorological elements based on Transformer is characterized by: include: Data collection module, model optimization module, model training module and forecasting module; The data collection module is used to collect historical meteorological data and extract the seasonal oscillation component of the meteorological elements of the historical meteorological data by using a filtering method; The model optimization module is used to construct and optimize the Transformer model to obtain an optimized model; The model training module performs training and evaluation on the optimized model based on the intra-seasonal oscillation component of the meteorological element to obtain an extended-range forecast model; The forecast module is used to perform weather forecast using the extended-range forecast model; The model optimization module includes: an input layer optimization unit, an attention mechanism optimization unit, a multi-layer perceptron optimization unit and a deconvolution optimization unit; The input layer optimization unit is used to replace the Patch Embedding part in the original Transformer model with Cross-scale Embedding; The attention mechanism optimization unit is used to replace the attention mechanism calculation part in the original Transformer model with the AFNO3D block; The multi-layer perceptron optimization unit is used to replace the MLP block part in the original Transformer model with the SwishMLP block; The deconvolution optimization unit is used to replace the deconvolution operation of the Patch Recovery part in the original Transformer model with a combination of a convolution operation and a Pixel Shuffle layer; The historical meteorological data include: 4 high-altitude variables, 5 ground elements, 3 constants and 5 target variables; The four upper air variables include: geopotential, temperature, radial wind and zonal wind; The five ground elements include: temperature at 2m, mean sea level pressure, surface pressure, radial wind at 10m and zonal wind at 10m; The three constants include: topography, land and sea mask, and latitude cosine value; The five target variables include: apparent height at 500 hPa, temperature field at 850 hPa, radial wind field, latitudinal wind field and temperature field at 2 m; The model training module includes: a first model training unit, an optimal time confirmation unit, a second model training unit and a model evaluation unit; The model training unit uses time configurations with different numbers of input time, and uses the intra-seasonal oscillation components of the meteorological elements as training sets to train the optimized model, thereby obtaining models with different input time with a forecast validity of 10 days; The optimal time confirmation unit uses the models of the different input times to forecast the five target variables, and takes the input time corresponding to the model with the smallest root mean square error value of the forecast result as the optimal input time; The second model training unit uses the time configuration of the optimal input time number and takes the seasonal oscillation component of the meteorological element as a training set to train the optimized model to obtain a number of initial forecast models with a forecast time validity of 11-30 days; The model evaluation unit uses several of the initial forecast models to forecast the five target variables, and calculates the root mean square error and the abnormal correlation coefficient of the forecast results. Then, the root mean square error and the abnormal correlation coefficient are used to evaluate the forecasting effects of several of the initial forecast models, and the one with the best effect is selected as the extended period forecast model.

2. A Transformer-based extended-range forecasting method for seasonal oscillation components of meteorological elements, the method being applied to the system of claim 1, characterized in that: The following steps are involved: S1. Collect historical meteorological data and use filtering methods to extract seasonal oscillation components of meteorological elements of the historical meteorological data; S2. Build and optimize the Transformer model to obtain the optimized model; S3. Based on the seasonal oscillation component of the meteorological element, the optimized model is trained and evaluated to obtain an extended-range forecast model; S4. Using the extended-range forecast model to perform weather forecasting; The S2 includes: Replace the Patch Embedding part in the original Transformer model with Cross-scale Embedding; Replace the attention mechanism calculation part in the original Transformer model with the AFNO3D block; Replace the MLP block in the original Transformer model with the SwishMLP block; The deconvolution operation in the Patch Recovery part of the original Transformer model is replaced by a combination of convolution operation and Pixel Shuffle layer; The historical meteorological data include: 4 high-altitude variables, 5 ground elements, 3 constants and 5 target variables; The four upper air variables include: geopotential, temperature, radial wind and zonal wind; The five ground elements include: temperature at 2m, mean sea level pressure, surface pressure, radial wind at 10m and zonal wind at 10m; The three constants include: topography, land and sea mask, and latitude cosine value; The five target variables include: apparent height at 500 hPa, temperature field at 850 hPa, radial wind field, latitudinal wind field and temperature field at 2 m; The S3 includes: Using time configurations with different numbers of input times and taking the intra-seasonal oscillation components of the meteorological elements as training sets, the optimized model is trained to obtain models with several different input times with a forecast validity of 10 days; Using the models with different input times to forecast the five target variables, and taking the input time corresponding to the model with the smallest root mean square error value of the forecast result as the optimal input time; Using the time configuration of the optimal input time number and taking the seasonal oscillation component of the meteorological element as a training set, the optimized model is trained to obtain several initial forecast models with a forecast time validity of 11-30 days; The five target variables are predicted using several of the initial prediction models, and the root mean square error and the abnormal correlation coefficient of the prediction results are calculated. The root mean square error and the abnormal correlation coefficient are then used to evaluate the prediction effects of the several initial prediction models, and the one with the best effect is selected as the extended period prediction model.