Medium and long term weather prediction system and method based on physical law correction deep learning technology

Abstract

The invention discloses a medium and long term weather prediction system based on a physical law correction deep learning technology, and the system comprises a high-resolution data input module, a multi-resolution network construction module, a sub-graph processing module based on a graph neural network, and a linear attention processing module. The system comprises an advanced attention processing module used for reducing secondary calculation complexity to linear complexity, a feature fusion and adaptive fusion module, a multi-level decoding and physical corresponding area optimization correction model, a rolling forecasting module and a multi-step training fine tuning module based on DORA. The medium and long term weather prediction system based on the physical law correction deep learning technology can process multi-resolution input data, and realizes high-precision and long-time weather prediction. Furthermore, global weather forecast with the resolution ratio of 16 days and 13 kilometers can be achieved, and meanwhile refined forecast with the higher resolution ratio is achieved in China.

Description

Technical Field

[0001] The present invention belongs to the technical field of meteorological prediction, and particularly relates to a high-precision and long-term meteorological prediction system and method using deep learning and graph neural network technologies. Background Art

[0002] Traditional meteorological prediction models face challenges in processing multi-scale and long-time series meteorological data and are difficult to effectively capture complex atmospheric dynamics processes. Existing technologies usually use input data with a fixed resolution and are difficult to make full use of multi-source and multi-resolution meteorological observations and reanalysis data. In addition, there is still room for improvement in the long-term forecasting ability and computational efficiency of existing models. Summary of the Invention

[0003] In order to solve the above problems existing in the prior art, the present invention provides a medium- and long-term meteorological prediction system and method based on physical law correction deep learning technology. The technical problems to be solved by the present invention are realized through the following technical solutions:

[0004] A medium- and long-term meteorological prediction system based on physical law correction deep learning technology, comprising:

[0005] A high-resolution data input module, configured to obtain original meteorological data from a preset port and process the original meteorological data to obtain multi-resolution meteorological data;

[0006] A multi-resolution network construction module, configured to perform data extraction and resolution transformation on the multi-resolution meteorological data to obtain integrated meteorological data;

[0007] A subgraph processing module based on a graph neural network, configured to perform spherical geometry processing on the integrated meteorological data to obtain spherical geometry meteorological data, and input the spherical geometry meteorological data into a preset heterogeneous graph neural network model to obtain a prediction result;

[0008] A linear attention processing module, configured to reduce the quadratic computational complexity to linear complexity;

[0009] An advanced attention processing module, configured to set different attention heads according to the integrated meteorological data at different spatial and temporal scales, and cross-fuse the different attention heads and output, wherein the attention heads include a local weather head, a global climate head, and a seasonal head;

[0010] A feature fusion and adaptive fusion module, configured to dynamically adjust the fusion weights according to the features, geographical location, and forecast time of the input integrated meteorological data;

[0011] A multi-level decoding and physical corresponding area optimization and correction model is used to restore the features of the prediction results output by a preset heterogeneous graph neural network model into specific meteorological variables and then correct them with historical error statistical data to obtain the final prediction results;

[0012] A rolling forecasting module is used to iteratively forecast the final prediction results;

[0013] A multi-step training and fine-tuning module based on DORA is used to dynamically adjust the optimizer parameters to meet the requirements of different training stages.

[0014] In a specific embodiment, the multi-resolution network construction module includes a multi-scale feature extraction unit, a resolution transformation unit, and a feature fusion unit;

[0015] The multi-scale feature extraction unit is used to design independent feature extraction networks for each resolution level, where the feature extraction network is a convolutional neural network; the convolutional neural network is expressed as:

[0016] F_i = CNN_i(X_i), where X_i is the input data with resolution i, CNN_i is the corresponding convolutional network, and F_i is the extracted feature;

[0017] The resolution transformation unit is used to map the original meteorological data with different resolutions to a unified resolution through upsampling or downsampling operations to obtain feature data;

[0018] The feature fusion unit is used to fuse the feature data through weighted summation or attention mechanism to obtain integrated meteorological data;

[0019] The formula for weighted summation is: F_final = w_1 * F_13_orig + w_2 * F_13_from_6 + w_3 * F_13_from_1, where w_1, w_2, and w_3 are learnable weights.

[0020] The formula for the attention mechanism is: A = Softmax(MLP([F_13_orig; F_13_from_6; F_13_from_1])), F_final = A_1 * F_13_orig + A_2 * F_13_from_6 + A_3 * F_13_from_1, where MLP is a multi-layer perceptron and A is the attention weight.

[0021] In a specific embodiment, the sub-graph processing module based on the graph neural network includes: a spherical graph convolutional network processing unit, a local encrypted graph structure processing unit;

[0022] The spherical graph convolutional network processing unit includes a spherical grid construction subunit and a spherical graph convolutional layer design subunit;

[0023] The spherical grid construction subunit is used to map the integrated meteorological data onto a spherical grid, where the spherical grid discretizes the Earth's surface using equal-area hexagonal grids;

[0024] The spherical graph convolutional layer design subunit is used to perform spherical geometry design so as to obtain spherical geometric meteorological data through the integrated meteorological data;

[0025] The local encrypted graph structure processing unit includes a multi-resolution graph construction subunit and a heterogeneous graph neural network design subunit;

[0026] The multi-resolution graph construction subunit is used to split the nodes formed by the spherical geometric meteorological data to obtain a number of multi-resolution nodes;

[0027] The heterogeneous graph neural network design subunit is used to assign unique type encodings to a number of multi-resolution nodes and perform heterogeneous graph convolution on the multi-resolution nodes to obtain a prediction result, where the type encodings include same resolution, upsampling, and downsampling.

[0028] In a specific embodiment, the advanced attention processing module includes a multi-head self-attention unit, a cross-attention unit, and an attention fusion unit;

[0029] The multi-head self-attention unit is used to set different attention heads at different spatial and temporal scales according to the integrated meteorological data;

[0030] The cross-attention unit is used to integrate information from different height levels and different physical quantities and establish interactions between different physical quantities;

[0031] The attention fusion unit is used to capture spatial dependencies using a graph attention network and cross-fuse different attention heads and then output.

[0032] In a specific embodiment, the feature fusion and adaptive fusion module includes an early feature fusion unit, a late feature fusion unit, and an adaptive weight learning unit;

[0033] The early feature fusion unit is used to fuse features from different sources in the underlying network;

[0034] The late feature fusion unit is used to fuse features from different sources in the high-level network;

[0035] The adaptive weight learning unit dynamically adjusts the fusion weights according to the features of the integrated meteorological data, geographical location, and forecast time in the underlying network that fuses features from different sources and the features in the high-level network that fuses features from different sources.

[0036] In a specific embodiment, the multi-level decoding and physical corresponding area optimization and correction model includes a multi-scale decoding unit, a physical constraint unit, and a post-processing correction unit;

[0037] The multi-scale decoding unit is used to restore the abstract features of the neural network to specific meteorological variables;

[0038] The physical constraint unit is used to ensure that the prediction results conform to the laws of atmospheric dynamics according to the constraint rules, and the constraint rule is the first law of atmospheric thermodynamics;

[0039] The post-processing correction unit is used to correct the historical error statistical information by using a preset correction function to obtain a corrected prediction value.

[0040] In a specific embodiment, the decoding formula of the multi-scale decoding unit is: X_{i + 1}=f(U(X_i)+R_i+A(X_i)), where A(·) represents the attention mechanism, f(·) represents the non-linear activation function, X_i represents the feature map of the i-th layer, U(·) represents the upsampling operation, and R_i represents the residual connection.

[0041] The present invention also provides a medium- and long-term meteorological prediction method based on physical law correction deep learning technology, including:

[0042] Obtaining original meteorological data from a preset port and processing the original meteorological data to obtain multi-resolution meteorological data;

[0043] Performing data extraction and resolution transformation on the multi-resolution meteorological data to obtain integrated meteorological data;

[0044] Performing spherical geometry processing on the integrated meteorological data to obtain spherical geometry meteorological data, and inputting the spherical geometry meteorological data into a preset heterogeneous graph neural network model to obtain a prediction result;

[0045] Reducing the quadratic computational complexity to linear complexity;

[0046] Setting different attention heads according to the integrated meteorological data at different spatial and temporal scales, and cross-fusing the different attention heads and outputting, wherein the attention heads include a local weather head, a global climate head, and a seasonal head;

[0047] Dynamically adjusting the fusion weights according to the features, geographical location, and forecast time of the input integrated meteorological data;

[0048] The predicted results output by the preset heterogeneous graph neural network model are restored to specific meteorological variables in terms of features and then corrected with historical error statistical data to obtain the final predicted results;

[0049] Perform iterative prediction on the final predicted results;

[0050] Dynamically adjust the optimizer parameters to meet the requirements of different training stages.

[0051] Advantages of the present invention:

[0052] The medium- and long-term meteorological prediction system based on the physical law correction deep learning technology of the present invention can process multi-resolution input data and achieve high-precision and long-term meteorological prediction. Further, it can achieve global weather forecasting with a resolution of 16 days and 13 kilometers, and at the same time achieve more refined forecasting with a higher resolution in the Chinese region.

[0053] The following will further elaborate on the present invention in conjunction with the accompanying drawings and embodiments. Description of the Drawings

[0054] Figure 1 It is a block diagram of a medium- and long-term meteorological prediction system based on the physical law correction deep learning technology provided by an embodiment of the present invention. Specific Embodiments

[0055] The following will further describe the present invention in detail with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0056] Embodiment 1

[0057] Please refer to Figure 1 , Figure 1 It is a block diagram of a medium- and long-term meteorological prediction system based on the physical law correction deep learning technology provided by an embodiment of the present invention, including:

[0058] A high-resolution data input module, configured to obtain original meteorological data from a preset port and process the original meteorological data to obtain multi-resolution meteorological data; specifically, CRA-V1.5 global reanalysis data with a resolution of 13 kilometers, integrate CLDAS2.0 Chinese regional data with a resolution of 6 kilometers, and reserve an interface for future integration of CLDAS 3.0 data with a resolution of 1 kilometer can be used.

[0059] A multi-resolution network construction module, configured to perform data extraction and resolution transformation on the multi-resolution meteorological data to obtain integrated meteorological data;

[0060] In a specific embodiment, the multi-resolution network construction module includes a multi-scale feature extraction unit, a resolution transformation unit, and a feature fusion unit;

[0061] The multi-scale feature extraction unit is used to design an independent feature extraction network for each resolution level, where the feature extraction network is a convolutional neural network; the convolutional neural network is expressed as:

[0062] F_i = CNN_i(X_i), where X_i is the input data with resolution i, CNN_i is the corresponding convolutional network, and F_i is the extracted feature;

[0063] The resolution transformation unit is used to map the original meteorological data with different resolutions to a unified resolution through upsampling or downsampling operations to obtain feature data;

[0064] The feature fusion unit is used to fuse the feature data through weighted summation or attention mechanism to obtain integrated meteorological data;

[0065] The formula for weighted summation is: F_final = w_1 * F_13_orig + w_2 * F_13_from_6 + w_3 * F_13_from_1, where w_1, w_2, and w_3 are learnable weights.

[0066] The formula for the attention mechanism is: A = Softmax(MLP([F_13_orig; F_13_from_6; F_13_from_1])), F_final = A_1 * F_13_orig + A_2 * F_13_from_6 + A_3 * F_13_from_1, where MLP is a multi-layer perceptron and A is the attention weight.

[0067] Specifically, the main objective of the multi-resolution network construction module is to process input data with different resolutions and map them to a unified resolution (13km in this embodiment). This process can be divided into key steps (multi-scale feature extraction, resolution transformation, feature fusion):

[0068] 1) Multi-scale feature extraction

[0069] For each resolution level (13km, 6km, 1km), we design an independent feature extraction network. This can be achieved using a convolutional neural network (CNN).

[0070] The formula is expressed as:

[0071] F_i = CNN_i(X_i)

[0072] where X_i is the input data with resolution i, CNN_i is the corresponding convolutional network, and F_i is the extracted feature.

[0073] 2) Resolution transformation

[0074] To map features of different resolutions to a unified 13-km resolution, we need to design upsampling and downsampling operations.

[0075] For data with 6-km and 1-km resolutions, we use downsampling operations:

[0076] F_13 = Downsample(F_6),

[0077] F_13 = Downsample(F_1),

[0078] For data with 13-km resolution, we keep it as it is.

[0079] The downsampling operation can be implemented using convolution with a stride of 2 or average pooling.

[0080] 3) Feature fusion

[0081] Finally, the 13-km resolution features from different sources need to be fused, which can be achieved through weighted summation or an attention mechanism.

[0082] Formula for weighted summation:

[0083] F_final = w_1 * F_13_orig + w_2 * F_13_from_6 + w_3 * F_13_from_1,

[0084] where w_1, w_2, and w_3 are learnable weights.

[0085] Formula for the attention mechanism:

[0086] A = Softmax(MLP([F_13_orig; F_13_from_6; F_13_from_1])),

[0087] F_final = A_1 * F_13_orig + A_2 * F_13_from_6 + A_3 * F_13_from_1,

[0088] where MLP is a multi-layer perceptron and A is the attention weight.

[0089] A subgraph processing module based on a graph neural network is used to perform spherical geometric processing on integrated meteorological data to obtain spherical geometric meteorological data, and input the spherical geometric meteorological data into a preset heterogeneous graph neural network model to obtain a prediction result;

[0090] In a specific embodiment, the subgraph processing module based on the graph neural network includes: a spherical graph convolutional network processing unit, a local encrypted graph structure processing unit;

[0091] The spherical graph convolutional network processing unit includes a spherical grid construction subunit and a spherical graph convolutional layer design subunit;

[0092] The spherical grid construction subunit is used to map the integrated meteorological data onto a spherical grid, where the spherical grid discretizes the Earth's surface using an equal-area hexagonal grid;

[0093] The spherical graph convolutional layer design subunit is used to perform spherical geometry design so as to obtain spherical geometric meteorological data through the integrated meteorological data;

[0094] The local encrypted graph structure processing unit includes a multi-resolution graph construction subunit and a heterogeneous graph neural network design subunit;

[0095] The multi-resolution graph construction subunit is used to split the nodes formed by the spherical geometric meteorological data to obtain a number of multi-resolution nodes;

[0096] The heterogeneous graph neural network design subunit is used to assign a unique type code to a number of multi-resolution nodes and perform heterogeneous graph convolution on the multi-resolution nodes to obtain a prediction result, where the type code includes same resolution, upsampling, and downsampling.

[0097] Specifically, the subgraph processing based on the graph neural network includes:

[0098] Using the spherical graph convolutional network to process global data with a resolution of 13 km,

[0099] Designing a locally encrypted graph structure to process high-resolution data (such as 6 km or even 1 km) in the Chinese region.

[0100] Specific solution:

[0101] The spherical graph convolutional network processes global data with a resolution of 13 km.

[0102] To process meteorological data at the global scale, we designed a spherical graph convolutional network (SphericalGraph Convolutional Network, SGCN). This network can effectively process data on the spherical geometry and is suitable for global meteorological modeling.

[0103] Spherical grid construction.

[0104] First, we need to map the global data with a resolution of 13 km onto a spherical grid. We use the Hexagonal Equal Area grid (HEALPix) to discretize the Earth's surface.

[0105] 1) HEALPix parameter settings:

[0106] The basic resolution parameter Nside = 512 (which can be fine-tuned according to the actual 13 km resolution).

[0107] The total number of pixels Npix = 12 * Nside^2 ≈ 3,145,728.

[0108] 2) Grid point coordinate calculation:

[0109] For each pixel i, its spherical coordinates (θ, φ) can be calculated through the HEALPix library function:

[0110] theta, phi = hp.pix2ang(Nside, i),

[0111] Spherical graph convolutional layer design.

[0112] We define the spherical graph convolution operation as follows:

[0113] H^(l + 1) = σ(D^(-1 / 2)AD^(-1 / 2)H^(l)W^(l)),

[0114] Where:

[0115] H^(l) is the feature matrix of the l-th layer,

[0116] A is the adjacency matrix,

[0117] D is the degree matrix,

[0118] W^(l) is the learnable weight matrix,

[0119] σ is the non-linear activation function.

[0120] To adapt to the spherical geometry, we make a special design for the adjacency matrix A:

[0121] 1) Distance calculation: For any two points i and j, calculate their great circle distance on the sphere:

[0122] Copyd_ij = arccos(sin(θ_i)sin(θ_j) + cos(θ_i)cos(θ_j)cos(φ_i - φ_j))

[0123] 2) Adjacency matrix construction:

[0124] CopyA_ij = exp(-d_ij^2 / σ^2) if d_ij ≤ r, else 0

[0125] Where r is the predefined neighborhood radius and σ is the adjustable Gaussian kernel parameter.

[0126] Multi-scale feature extraction. To capture atmospheric dynamics processes at different scales, we designed a multi-scale SGCN structure.

[0127] Parallel SGCN layers:

[0128] Design multiple parallel SGCN layers, each using a different neighborhood radius r and Gaussian kernel parameter σ.

[0129] Feature fusion:

[0130] Use the attention mechanism to fuse features at different scales:

[0131] CopyH_fused = Σ_kα_k * H_k

[0132] where α_k is the attention weight calculated by the softmax function.

[0133] The locally encrypted graph structure processes high-resolution data in the Chinese region.

[0134] To process high-resolution data (6 km or 1 km) in the Chinese region, we designed a locally encrypted graph structure.

[0135] Multi-resolution graph construction

[0136] 1) Basic graph structure:

[0137] Keep the SGCN structure with a global resolution of 13 km as the basis.

[0138] 2) Local encryption: Locally encrypt the Chinese region to increase the graph node density:

[0139] 6 km resolution: Each 13 km node is split into 4 sub-nodes,

[0140] 1 km resolution: Each 13 km node is split into 169 sub-nodes.

[0141] 3) Edge connection strategy:

[0142] Between nodes of the same resolution: Use the same connection strategy as the global SGCN,

[0143] Between nodes of different resolutions: Design special upsampling and downsampling edges,

[0144] E_up(i,j) = 1 if node_i is a parent of node_j,

[0145] E_down(j,i) = 1 if node_j is a child of node_i.

[0146] Heterogeneous graph neural network design

[0147] To process the multi-resolution heterogeneous graph structure, we designed a Heterogeneous Graph Neural Network (HGNN).

[0148] 1) Node type encoding: Assign a unique type encoding to nodes of each resolution.

[0149] 2) Heterogeneous graph convolution operation:

[0150] h_i^(l+1) = σ(Σ_rΣ_j∈N_r(i)1 / c_ij * W_r^(l) * h_j^(l))

[0151] Where:

[0152] r represents the relationship type (same resolution, upsampling, downsampling)

[0153] N_r(i) is the neighborhood of node i under relationship r,

[0154] c_ij is the normalization constant,

[0155] W_r^(l) is the learnable weight matrix specific to relationship r.

[0156] Resolution adaptive layer: Design an adaptive layer to dynamically adjust the weights of features at different resolutions.

[0157] h_final = Σ_kβ_k * MLP_k(h_k),

[0158] Where β_k is the weight learned through the gating mechanism, and MLP_k is the multi-layer perceptron specific to resolution k.

[0159] With this design, our graph neural network can process both global 13km resolution data and high-resolution data in the Chinese region, achieving multi-scale and multi-resolution atmospheric dynamics modeling. This method not only improves the accuracy of global weather forecasting but also provides more detailed high-resolution forecasts for the Chinese region.

[0160] Linear attention processing module, used to reduce the quadratic computational complexity to linear complexity;

[0161] Specifically, the linear attention processing module implements an attention mechanism with linear complexity to process longer time series and designs special time encoding to capture long-term dependencies within the 16-day forecast period.

[0162] Specific implementation of this part:

[0163] Attention mechanism with linear complexity

[0164] The computational complexity of the traditional self-attention mechanism is O(n^2), where n is the sequence length. For weather forecasts spanning up to 16 days (64 time steps), this quadratic complexity can lead to inefficient computations. Therefore, we designed an attention mechanism with linear complexity.

[0165] 1) Core idea of linear attention

[0166] The core of linear attention is to transform dot-product attention into a decomposable form. Suppose we have a query matrix Q, a key matrix K, and a value matrix V. The traditional attention calculation is as follows:

[0167] Attention(Q, K, V) = softmax(QK^T)V

[0168] We introduce a kernel function φ(·) to transform the above formula into:

[0169] LinearAttention(Q, K, V) = φ(Q)(φ(K)^TV) / (φ(Q)(φ(K)^T1))

[0170] where 1 is a vector of all ones. This form can achieve a computational complexity of O(n) through the associative law of matrix multiplication.

[0171] 2) Kernel function selection

[0172] ELU (Exponential Linear Unit) is selected as the kernel function:

[0173] φ(x) = ELU(x) + 1 = max(0, x) + min(0, α(exp(x) - 1)) + 1

[0174] This choice not only maintains non-linearity but also avoids the numerical instability that the exponential function may bring.

[0175] 3) Algorithm implementation

[0176] The following is the pseudo-code implementation of linear attention:

[0177] def linear_attention(Q, K, V):

[0178] Q = ELU(Q) + 1

[0179] K = ELU(K) + 1

[0180] KV = K.transpose() @ V

[0181] Z = 1 / (Q @ K.sum(axis = 1))

[0182] return(Q@KV)*Z.unsqueeze(-1)

[0183] An advanced attention processing module is used to set different attention heads at different spatial and temporal scales according to the integrated meteorological data, and cross-fuse different attention heads and then output. Among them, the attention heads include a local weather head, a global climate head, and a seasonal head;

[0184] In a specific embodiment, the advanced attention processing module includes a multi-head self-attention unit, a cross-attention unit, and an attention fusion unit;

[0185] The multi-head self-attention unit is used to set different attention heads at different spatial and temporal scales according to the integrated meteorological data;

[0186] The cross-attention unit is used to integrate information of different altitude levels and different physical quantities, and establish interactions between different physical quantities;

[0187] The attention fusion unit is used to capture spatial dependence relationships using a graph attention network and cross-fuse different attention heads and then output.

[0188] The advanced attention processing module is used to implement the multi-head self-attention and cross-attention mechanisms and design a special spatio-temporal attention layer to capture atmospheric teleconnection phenomena.

[0189] Specific implementation:

[0190] Head self-attention mechanism.

[0191] The multi-head self-attention mechanism allows the model to simultaneously focus on different representation subspaces, enhancing the model's expressive ability.

[0192] Given the input sequence X, the calculation of multi-head self-attention is as follows:

[0193] MultiHead(X)=Concat(head_1,...,head_h)W^O

[0194] where head_i=Attention(XW^Q_i,XW^K_i,XW^V_i)

[0195] Among them:

[0196] W^Q_i,W^K_i,W^V_i are the linear transformation matrices of the i-th head

[0197] W^O is the output linear transformation matrix

[0198] The Attention function uses the linear attention mechanism defined previously.

[0199] Weather-specific optimization.

[0200] To adapt to the characteristics of weather forecasting, we have made the following optimizations to multi-head attention:

[0201] 1) Dynamic head number adjustment

[0202] class DynamicMultiHeadAttention(nn.Module):

[0203] def __init__(self, d_model, max_heads):

[0204] super().__init__()

[0205] self.max_heads = max_heads

[0206] self.head_selector = nn.Linear(d_model, max_heads)

[0207] def forward(self, X):

[0208] head_weights = torch.sigmoid(self.head_selector(X.mean(dim = 1)))

[0209] active_heads = (head_weights > 0.5).sum(dim = 1)

[0210] return MultiHeadAttention(X, num_heads = active_heads)

[0211] This allows the model to dynamically adjust the number of heads used according to the complexity of the input data.

[0212] 2) Scale-specific heads:

[0213] We design specialized attention heads for different spatial and temporal scales, such as:

[0214] Local weather head: Focus on short-term changes in a small area

[0215] Global climate head: Capture long-term trends at a large scale

[0216] Seasonal head: Focus on capturing seasonal patterns

[0217] Cross-attention mechanism

[0218] The cross-attention mechanism is used to integrate information from different sources, such as the interactions between different altitude levels and different physical quantities.

[0219] Formula definition

[0220] Given two input sequences X and Y, the calculation of cross-attention is as follows:

[0221] CrossAttention(X,Y) = Attention(XW^Q,YW^K,YW^V)

[0222] Meteorological applications

[0223] In weather forecasting, we use cross-attention to:

[0224] Integrate information from different altitude levels:

[0225] class VerticalCrossAttention(nn.Module):

[0226] def __init__(self, num_layers):

[0227]

[0228] Model the interactions between different physical quantities

[0229]

[0230] Spatio-temporal attention layer

[0231] To capture atmospheric teleconnection phenomena, we design a special spatio-temporal attention layer. The spatial attention uses a graph attention network (GAT) to capture spatial dependencies:

[0232] super().__init__()

[0233] self.gat = GATConv(d_model, d_model / / num_heads, heads = num_heads)

[0234] def forward(self, x, edge_index):

[0235] return self.gat(x, edge_index)

[0236] Temporal attention uses the linear attention mechanism defined above:

[0237] class TemporalAttention(nn.Module):

[0238] def __init__(self, d_model):

[0239] super().__init__()

[0240] self.linear_attention = LinearAttention(d_model)

[0241] def forward(self, x):

[0242] return self.linear_attention(x, x, x)

[0243] To capture the teleconnection phenomenon, we designed a cross-scale fusion mechanism:

[0244] class CrossScaleFusion(nn.Module):

[0245] def __init__(self, d_model, num_scales):

[0246] super().__init__()

[0247] self.scale_attention = nn.MultiheadAttention(d_model, num_heads = num_scales)

[0248] def forward(self, x):

[0249] scales = [F.avg_pool2d(x, 2 ** i) for i in range(num_scales)]

[0250] fused = self.scale_attention(x, torch.cat(scales, dim = 1), torch.cat(scales, dim = 1))

[0251] return x + fused

[0252] The complete spatio-temporal attention layer

[0253] class SpatioTemporalAttention(nn.Module):

[0254] def __init__(self, d_model, num_heads, num_scales):

[0255] super().__init__()

[0256] self.spatial_attention = SpatialAttention(d_model, num_heads)

[0257] self.temporal_attention = TemporalAttention(d_model)

[0258] self.cross_scale_fusion = CrossScaleFusion(d_model, num_scales)

[0259] def forward(self, x, edge_index):

[0260] x = self.spatial_attention(x, edge_index)

[0261] x = self.temporal_attention(x)

[0262] x = self.cross_scale_fusion(x)

[0263] return x

[0264] This design allows the model to capture complex spatio-temporal dependencies and atmospheric teleconnection phenomena, making it very suitable for high-precision long-term weather forecasting tasks.

[0265] A feature fusion and adaptive fusion module for dynamically adjusting the fusion weights according to the features, geographical locations, and forecast times of the integrated meteorological data input;

[0266] In a specific embodiment, the feature fusion and adaptive fusion module includes an early feature fusion unit, a late feature fusion unit, and an adaptive weight learning unit;

[0267] The early feature fusion unit is used to fuse features from different sources in the underlying network;

[0268] The late feature fusion unit is used to fuse features from different sources in the high-level network;

[0269] The adaptive weight learning unit dynamically adjusts the fusion weights for the features, geographical location, and forecast time of the input integrated meteorological data based on the features from different sources fused in the low-level network and the features from different sources fused in the high-level network.

[0270] Early fusion is performed in the shallow layer of the network, which can integrate information from different sources at the early stage of feature extraction. Late fusion is performed in the deep layer of the network, which can integrate higher-level abstract features. We combine these two strategies to make full use of their advantages.

[0271] To adapt to the characteristics of different regions and different forecast periods, we designed a dynamic weight adjustment mechanism. This mechanism can dynamically adjust the fusion weights according to the features, geographical location, and forecast time of the input data.

[0272] Specific implementation of each component:

[0273] import torch

[0274] import torch.nn as nn

[0275] import torch.nn.functional as F

[0276] class EarlyFusion(nn.Module):

[0277] def __init__(self, input_channels, output_channels):

[0278] super(EarlyFusion, self).__init__()

[0279] self.conv = nn.Conv2d(input_channels, output_channels, kernel_size = 1)

[0280] def forward(self, x1, x2, x3):

[0281] #x1,x2,x3 are features from different sources or resolutions

[0282] x = torch.cat([x1, x2, x3], dim = 1)

[0283] return self.conv(x)

[0284] class LateFusion(nn.Module):

[0285] def __init__(self, input_channels, output_channels):

[0286] super(LateFusion, self).__init__()

[0287] self.conv1 = nn.Conv2d(input_channels, output_channels, kernel_size = 1)

[0288]

[0289]

[0290] Explanation of the key parts of the implementation:

[0291] 1) EarlyFusion class

[0292] Implements the early fusion strategy, using a 1x1 convolutional layer to connect and fuse feature maps from different sources.

[0293] 2) LateFusion class

[0294] Implements the late fusion strategy, applying independent 1x1 convolutions to each input feature map, and then connecting and fusing the results through another 1x1 convolution.

[0295] 3) Dynamic Weight Adjustment class

[0296] Implements the dynamic weight adjustment mechanism

[0297] Contains three components: spatial attention, temporal attention, and regional embedding. Spatial attention captures the importance of different positions, temporal attention processes the weights for different prediction time periods, and regional embedding allows the model to learn specific weights for different geographical regions. 4) FeatureFusionAdaptiveFusion class integrates early fusion, deep feature extraction, late fusion, and dynamic weight adjustment, providing an end-to-end feature fusion and adaptive fusion solution.

[0298] The main advantages of the design of this embodiment include:

[0299] Flexibility: By combining early and late fusion, the model can integrate information at different levels.

[0300] Self - adaptability: The dynamic weight adjustment mechanism enables the model to dynamically adjust the fusion strategy according to spatial, temporal, and geographical information. Interpretability: The spatial and temporal attention weights can be visualized to provide insights into the model's decision - making.

[0301] Scalability: The design can be easily extended to handle more input sources or more complex fusion strategies.

[0302] This system can effectively integrate multi - source data and dynamically adjust its prediction strategy according to different geographical locations and forecast periods, thereby improving the accuracy and adaptability of the prediction.

[0303] The multi - level decoding and physical corresponding area optimization correction model is used to restore the features of the prediction results output by the preset heterogeneous graph neural network model into specific meteorological variables and then correct them with historical error statistical data to obtain the final prediction results.

[0304] In a specific embodiment, the multi - level decoding and physical corresponding area optimization correction model includes a multi - scale decoding unit, a physical constraint unit, and a post - processing correction unit.

[0305] The multi - scale decoding unit is used to restore the abstract features of the neural network into specific meteorological variables.

[0306] The physical constraint unit is used to ensure that the prediction results conform to the laws of atmospheric dynamics according to the constraint rules, and the constraint rule is the first law of atmospheric thermodynamics.

[0307] The post - processing correction unit is used to correct the historical error statistical information using a preset correction function to obtain a corrected prediction value.

[0308] In a specific embodiment, the decoding formula of the multi - scale decoding unit is: \(X_{i + 1}=f(U(X_i)+R_i+A(X_i))\), where \(A(\cdot)\) represents the attention mechanism, \(f(\cdot)\) represents the non - linear activation function, \(X_i\) represents the feature map of the \(i\) - th layer, \(U(\cdot)\) represents the up - sampling operation, and \(R_i\) represents the residual connection.

[0309] The multi - level decoding and physical optimization correction module is a key component of the meteorological prediction system in this embodiment, aiming to achieve high - precision restoration of meteorological variables and physical consistency of prediction results.

[0310] The main objective of the multi - scale decoding unit is to restore the abstract features of the neural network into specific meteorological variables with a 13 - kilometer resolution. It adopts a hierarchical design, including the following key components:

[0311] 1) Feature up - sampling layer

[0312] 2) Residual connection

[0313] 3) Attention mechanism

[0314] 4) Multi-scale fusion

[0315] 2.7.1.3 Calculation process

[0316] The core operation of the multi-scale decoding unit can be expressed as:

[0317] $$

[0318] X_{i + 1}=f(U(X_i)+R_i+A(X_i))

[0319] $$

[0320] Where:

[0321] $X_i$ is the feature map of the $i$-th layer

[0322] $U(·)$ is the upsampling operation

[0323] $R_i$ is the residual connection

[0324] $A(·)$ is the attention mechanism

[0325] $f(·)$ is the non-linear activation function

[0326] The specific implementation is as follows:

[0327] class MultiScaleDecoder(nn.Module):

[0328] def __init__(self, in_channels, out_channels):

[0329] super().__init__()

[0330] self.upsample = nn.Upsample(scale_factor = 2, mode = 'bilinear')

[0331] self.conv1 = nn.Conv2d(in_channels, out_channels, 3, padding = 1)

[0332] self.attention = SelfAttention(out_channels)

[0333] self.conv2 = nn.Conv2d(out_channels, out_channels, 3, padding = 1)

[0334] def forward(self, x, skip):

[0335] x = self.upsample(x)

[0336] x = self.conv1(x)

[0337] x = x + skip # Residual connection

[0338] x = self.attention(x)

[0339] x = self.conv2(x)

[0340] return x

[0341] The goal of the physical constraint unit is to ensure that the prediction results conform to the laws of atmospheric dynamics and improve the physical consistency of the prediction. It includes: soft constraint, taking physical laws as part of the loss function; hard constraint: directly embedding physical equations into the network structure.

[0342] The calculation formula is

[0343] Taking the first law of atmospheric thermodynamics as an example, we can introduce the following constraints:

[0344] $$\frac{\partial T}{\partial t}+\mathbf{v}\cdot\nabla T+\omega\frac{\partial T}{\partial p} = \frac{R T\omega}{c_p p}+\frac{Q}{c_p}$$

[0345] where:

[0346] $T$ is temperature

[0347] $\mathbf{v}$ is the horizontal wind speed

[0348] $\omega$ is the vertical velocity

[0349] $p$ is pressure

[0350] $R$ is the gas constant

[0351] $c_p$ is the specific heat capacity at constant pressure

[0352] $Q$ is the heating rate

[0353] The specific code implementation is as follows:

[0354] class PhysicalConstraintLayer(nn.Module):

[0355] def __init__(self):

[0356] super().__init__()

[0357] def forward(self, T, v, omega, p, Q):

[0358] # Calculate the left side term

[0359] dT_dt = torch.gradient(T, dim = 1)[0]

[0360] dT_dx = torch.gradient(T, dim = 2)[0]

[0361] dT_dy = torch.gradient(T, dim = 3)[0]

[0362] dT_dp = torch.gradient(T, dim = 4)[0]

[0363] left_side = dT_dt+(v[:, :, 0]*dT_dx + v[:, :, 1]*dT_dy)+omega*dT_dp

[0364] # Calculate the right side term

[0365] R = 287.05 # J / (kg·K)

[0366] cp = 1004.0 # J / (kg·K)

[0367] right_side = (R*T*omega) / (cp*p)+Q / cp

[0368] # Return the constraint error

[0369] return torch.mean((left_side - right_side)**2) The post - processing correction unit aims to further improve the prediction accuracy by using historical error statistical information. The corrected predicted value can be expressed as:

[0370] $$

[0371] \hat{y}_\text{corrected}=\hat{y}+f(\hat{y},x,t,l)

[0372] $$

[0373] Wherein:

[0374] $\hat{y}$ is the original predicted value

[0375] $x$ is the input feature

[0376] $t$ is the prediction time

[0377] $l$ is the location information

[0378] $f(·)$ is the learned correction function

[0379] The specific implementation code is as follows:

[0380]

[0381] A rolling prediction module for iteratively predicting the final prediction result;

[0382] The mathematical representation of the rolling prediction module is: representing the weather state at time slot i as a high-dimensional tensor:

[0383] Wherein:

[0384] C = 189 (number of atmospheric variables considered)

[0385] W = 3072 (width)

[0386] H = 1536 (height)

[0387] This representation corresponds to mapping continuous atmospheric fluid to a 2D spatial plane with a 13-kilometer resolution.

[0388] The objective of this embodiment is to generate a 16-day forecast:

[0389] {X^(i+1), X^(i+2), ..., X^(i+64)} = LongShan(X^i) (1)

[0390] Where X^(i+τ) is the weather state prediction at time slot i+τ, with a time interval of 6 hours.

[0391] Due to the extremely large scale of global atmospheric data, it is difficult to directly learn the function of equation (1). Therefore, this embodiment adopts an autoregressive method to generate multi-step predictions:

[0392] X^(i + 1) = LongShan(X^i) X^(i + 2) = LongShan(X^(i + 1)) ... X^(i + 64) = LongShan(X^(i + 63))(2)

[0393] The specific steps of the rolling prediction mechanism are as follows:

[0394] a) Initial prediction: Use the current observation data X^i for the first prediction: X^(i + 1) = LongShan(X^i)

[0395] b) Iterative prediction: for τ = 1 to 63: X^(i + τ + 1) = LongShan(X^(i + τ))

[0396] c) Error accumulation control: Introduce the error correction term δ^τ: X^(i + τ + 1) = LongShan(X^(i + τ)) + δ^τ

[0397] where δ^τ is calculated based on historical forecast error statistics and dynamic correction.

[0398] d) Regular update: Update the initial field X^i with the latest observation data every 6 hours and restart the 64-step prediction.

[0399] To improve the prediction accuracy, fuse the prediction results with different spatial resolutions:

[0400] X^(i + τ) = α_1X^(i + τ)_high + α_2X^(i + τ)_medium + α_3X^(i + τ)_low

[0401] where α_1, α_2, α_3 are the weight coefficients for different resolutions.

[0402] Through this rolling prediction mechanism, the system of this embodiment can, while controlling the error accumulation, extend the effective prediction period to 11.75 days, significantly exceeding the industry average. This technological breakthrough provides new possibilities for medium- and long-term weather forecasting and has important practical significance for agricultural planning, disaster prevention and mitigation, and economic decision-making.

[0403] The multi-step training fine-tuning module based on DORA is used to dynamically adjust the optimizer parameters to meet the requirements of different training stages.

[0404] The core idea of the multi-step training fine-tuning module based on DORA is to dynamically adjust the optimization rate according to the performance of the model at different forecast time steps. This helps to balance the accuracy of short-term and long-term forecasts.

[0405] The key formula is:

[0406] η_t = η_0 * exp(-λ * t / T)

[0407] Among them:

[0408] η_t: Learning rate at time step t

[0409] η_0: Initial learning rate

[0410] λ: Decay coefficient

[0411] t: Current time step

[0412] T: Total number of time steps (64 in this example)

[0413] 2) Progressive training process

[0414] To optimize the 16-day (64 six-hour time steps) forecast, we designed a progressive training process:

[0415] Phase 1 (Short-term forecast): 1 - 2 days (1 - 8 time steps)

[0416] Phase 2 (Medium-term forecast): 3 - 7 days (9 - 28 time steps)

[0417] Phase 3 (Long-term forecast): 8 - 16 days (29 - 64 time steps)

[0418] a) Loss function design:

[0419] L = Σ(w_t * L_t), t = 1 to 64

[0420] Among them:

[0421] L_t: Forecast loss at the t-th time step

[0422] w_t: Weight at the t-th time step, decreasing with the increase of time steps

[0423] w_t = exp(-β * t / T)

[0424] β is the weight decay coefficient

[0425] b) Gradient accumulation:

[0426] To handle the gradients of different time steps, we use the gradient accumulation technique:

[0427] g_acc = Σ(g_t * w_t), t = 1 to 64

[0428] where g_t is the gradient at the t-th time step

[0429] c) DORA optimizer:

[0430] Combining the DORA mechanism and the Adam optimizer, update the parameter θ:

[0431] $m_t = \beta_1 \cdot m_{t - 1}+(1 - \beta_1) \cdot g_{acc}$

[0432] $v_t = \beta_2 \cdot v_{t - 1}+(1 - \beta_2) \cdot g_{acc}^2$

[0433] $\theta_t=\theta_{t - 1}-\frac{\eta_t \cdot m_t}{\sqrt{v_t+\varepsilon}}$

[0434] Design a progressive training process for 16 - day forecasts

[0435] 1) Progressive training algorithm

[0436]

[0437] Through this DORA - based multi - step training and fine - tuning method, the Longshan - 1 system can optimize its forecasting performance at different time scales, effectively balancing the accuracy of short - term and long - term forecasts. This method allows the model to gradually improve its long - term forecasting ability while maintaining the accuracy of short - term forecasts, thus achieving a significant improvement in the overall forecasting performance.

[0438] The present invention also provides a medium - and long - term meteorological prediction method based on physical - law - corrected deep - learning technology, including:

[0439] Obtain original meteorological data from a preset port and process the original meteorological data to obtain multi - resolution meteorological data;

[0440] Perform data extraction and resolution transformation on the multi - resolution meteorological data to obtain integrated meteorological data;

[0441] Perform spherical geometric processing on the integrated meteorological data to obtain spherical - geometric meteorological data, and input the spherical - geometric meteorological data into a preset heterogeneous graph neural network model to obtain a prediction result;

[0442] Reduce the quadratic computational complexity to linear complexity;

[0443] Set different attention heads according to the integrated meteorological data at different spatial and temporal scales, and cross - fuse the different attention heads and output, where the attention heads include local weather heads, global climate heads, and seasonal heads;

[0444] Dynamically adjust the fusion weights according to the characteristics, geographical location, and forecast time of the input integrated meteorological data;

[0445] Restore the features of the prediction result output by the preset heterogeneous graph neural network model to specific meteorological variables and correct them with historical error statistical data to obtain the final prediction result;

[0446] Perform iterative prediction on the final prediction result;

[0447] Dynamically adjust the optimizer parameters to meet the requirements of different training stages.

[0448] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should all be regarded as belonging to the protection scope of the present invention.

Claims

1. A medium- and long-term weather forecast system based on physical law correction deep learning technology, characterized in that: include: A high-resolution data input module, used for acquiring raw meteorological data from a preset port and processing the raw meteorological data to obtain multi-resolution meteorological data; A multi-resolution network construction module, used for performing data extraction and resolution transformation on the multi-resolution meteorological data to obtain integrated meteorological data; A subgraph processing module based on a graph neural network is used to perform spherical geometry processing on the integrated meteorological data to obtain spherical geometry meteorological data, and input the spherical geometry meteorological data into a preset heterogeneous graph neural network model to obtain a prediction result; Linear attention processing module, used to reduce quadratic computational complexity to linear complexity; An advanced attention processing module is used to set different attention heads at different spatial and temporal scales according to the integrated meteorological data, and output the different attention heads after cross-fusion, wherein the attention heads include local weather head, global climate head and seasonal head; Feature fusion and adaptive fusion modules are used to dynamically adjust the fusion weights according to the characteristics, geographic location and forecast time of the input integrated meteorological data; The multi-level decoding and physical corresponding area optimization correction model is used to restore the prediction results output by the preset heterogeneous graph neural network model to specific meteorological variables and then correct them with historical error statistics to obtain the final prediction results; A rolling forecast module, used for iteratively forecasting the final forecast result; A multi-step training fine-tuning module based on DORA is used to dynamically adjust optimizer parameters to meet the needs of different training stages.

2. The medium- and long-term weather forecast system based on physical law correction deep learning technology according to claim 1 is characterized in that: The multi-resolution network construction module includes a multi-scale feature extraction unit, a resolution transformation unit and a feature fusion unit; The multi-scale feature extraction unit is used to design an independent feature extraction network for each resolution level, wherein the feature extraction network is a convolutional neural network; the convolutional neural network is expressed as: F_i=CNN_i(X_i), where X_i is the input data with resolution i, CNN_i is the corresponding convolutional network, and F_i is the extracted feature; The resolution conversion unit is used to map the original meteorological data of different resolutions to a uniform resolution to obtain feature data through upsampling or downsampling operations; The feature fusion unit is used to fuse the feature data by weighted summation or attention mechanism to obtain integrated meteorological data; The formula of the weighted summation is: F_final=w_1*F_13_orig+w_2*F_13_from_6+w_3*F_13_from_1, wherein w_1, w_2, w_3 are learnable weights. The formula of the attention mechanism is: A=Softmax(MLP([F_13_orig;F_13_from_6;F_13_from_1])), F_final=A_1*F_13_orig+A_2*F_13_from_6+A_3*F_13_from_1, where MLP is a multilayer perceptron and A is the attention weight.

3. The medium- and long-term weather forecast system based on physical law correction deep learning technology according to claim 1 is characterized in that: The subgraph processing module based on graph neural network includes: a spherical graph convolution network processing unit and a local encrypted graph structure processing unit; The spherical graph convolution network processing unit includes a spherical grid construction subunit and a spherical graph convolution layer design subunit; The spherical grid construction subunit is used to map the integrated meteorological data onto a spherical grid, wherein the spherical grid discretizes the earth's surface using an equal-area hexagonal grid; The spherical graph convolutional layer design subunit is used to perform spherical geometry design so as to obtain spherical geometry meteorological data by integrating meteorological data; The local encrypted graph structure processing unit includes a multi-resolution graph construction subunit and a heterogeneous graph neural network design subunit; The multi-resolution graph construction subunit is used to split the nodes formed by the spherical geometric meteorological data to obtain a plurality of multi-resolution nodes; The heterogeneous graph neural network design subunit is used to assign unique type codes to several multi-resolution nodes, and perform heterogeneous graph convolution on the multi-resolution nodes to obtain prediction results, wherein the type codes include same resolution, upsampling, and downsampling.

4. The medium- and long-term weather forecast system based on physical law correction deep learning technology according to claim 1 is characterized in that: The advanced attention processing module includes a multi-head self-attention unit, a cross-attention unit and an attention fusion unit; The multi-head self-attention unit is used to set different attention heads at different spatial and temporal scales according to the integrated meteorological data; The cross attention unit is used to integrate information of different altitude layers and different physical quantities, and establish the interaction between different physical quantities; The attention fusion unit is used to capture spatial dependencies using a graph attention network and output cross-fusion of different attention heads.

5. The medium- and long-term weather forecast system based on physical law correction deep learning technology according to claim 1 is characterized in that: The feature fusion and adaptive fusion module includes an early feature fusion unit, a late feature fusion unit, and an adaptive weight learning unit; The early feature fusion unit is used to fuse features from different sources in the underlying network; The late feature fusion unit is used to fuse features from different sources in a high-level network; The adaptive weight learning unit dynamically adjusts the fusion weight according to the features of different sources fused in the bottom network and the features of different sources fused in the high-level network, the features of the input integrated meteorological data, the geographical location and the forecast time.

6. The medium- and long-term weather forecasting system based on physical law correction deep learning technology according to claim 1 is characterized in that: The multi-level decoding and physical corresponding area optimization correction model includes a multi-scale decoding unit, a physical constraint unit, and a post-processing correction unit; The multi-scale decoding unit is used to restore the abstract features of the neural network to specific meteorological variables; The physical constraint unit is used to ensure that the prediction result complies with the law of atmospheric dynamics according to the constraint rule, and the constraint rule is the first law of atmospheric thermodynamics; The post-processing correction unit is used to correct the historical error statistical information using a preset correction function to obtain a corrected prediction value.

7. The medium- and long-term weather forecasting system based on physical law correction deep learning technology according to claim 6 is characterized in that: The decoding formula of the multi-scale decoding unit is: X_{i+1}=f(U(X_i)+R_i+A(X_i)), where A(·) represents the attention mechanism, f(·) represents the nonlinear activation function, X_i represents the feature map of the i-th layer, U(·) represents the upsampling operation, and R_i represents the residual connection.

8. A medium- and long-term weather forecasting method based on physical law correction deep learning technology, characterized in that: include: Acquire raw meteorological data from a preset port and process the raw meteorological data to obtain multi-resolution meteorological data; Performing data extraction and resolution transformation on the multi-resolution meteorological data to obtain integrated meteorological data; Performing spherical geometry processing on the integrated meteorological data to obtain spherical geometry meteorological data, and inputting the spherical geometry meteorological data into a preset heterogeneous graph neural network model to obtain a prediction result; Reduce the quadratic computational complexity to linear complexity; According to the integrated meteorological data, different attention heads are set at different spatial and temporal scales, and the different attention heads are cross-fused and outputted, wherein the attention heads include local weather head, global climate head and seasonal head; Dynamically adjust the fusion weights according to the characteristics, geographic location and forecast time of the input integrated meteorological data; The prediction results output by the preset heterogeneous graph neural network model are restored to specific meteorological variables and corrected with historical error statistics to obtain the final prediction results. Performing iterative prediction on the final prediction result; Dynamically adjust optimizer parameters to meet the needs of different training stages.

Images