Intelligent post-processing method and device for coupling spatiotemporal errors of multiple numerical weather predictions

Abstract

The application discloses a kind of spatio-temporal error intelligent post-processing methods and devices coupled multiple numerical weather forecasts, first obtain hydro-meteorological measured data and precipitation forecast data;Then coupling clustering enhanced graph convolution network model CEGCN and long short-term memory network LSTM model, embed LSTM layer in CEGCN model, build the post-processing PP-CEGCN-LSTM model that can consider numerical forecast precipitation data spatial feature and time correlation simultaneously, feature extraction is carried out to the whole graph using PP-CEGCN-LSTM model;Finally, using hydro-meteorological measured data and forecast data as model input data, using clustering algorithm divides the node in graph data into different clusters, the node in each cluster is carried out feature fusion to obtain more comprehensive and accurate precipitation result.The application can significantly improve the prediction accuracy of numerical weather forecast data, and can effectively prevent extreme precipitation weather warning.

Description

Technical Field

[0001] This invention relates to the field of precipitation forecasting, and in particular to an intelligent post-processing method and device for spatiotemporal errors coupled with multiple numerical weather forecasts, which is mainly used by water conservancy departments and meteorological departments to carry out forecasting and early warning work for extreme precipitation, floods and other events. Background Technology

[0002] Due to the complexity and high nonlinearity of atmospheric systems, as well as the limitations of forecasting systems themselves, numerical weather prediction results often contain certain systematic errors, especially for variables highly correlated with weather factors. Ensemble forecast products suffer from forecast errors and low dispersion of ensemble members due to imperfections in initial value perturbations, model design, and the number of ensemble members. In order to generate higher quality precipitation data, further research is needed on post-processing methods such as spatial downscaling of precipitation data, bias correction, and product fusion.

[0003] Currently, statistical post-processing methods for numerical precipitation forecasting have been extensively studied, such as methods for evaluating quantile mappings for seasonal rainfall prediction and parameterized post-processing methods like Bayesian joint probabilistic modeling. Deep learning methods, due to their superior learning and generalization abilities, and their unique advantages in handling nonlinear data and performing well in sparse data regions, are widely used in bias correction and post-processing research. For example, multi-task convolutional neural networks are used to consider the correlation between time series data and station data for precipitation forecasting.

[0004] However, the application of Cluster Enhanced Graph Convolutional Network (CEGCN) and Long Short-Term Memory (LSTM) models to precipitation forecast post-processing has been rarely mentioned in previous studies. While LSTM models have the ability to analyze and predict time-series data, they cannot effectively extract the spatial features of the data. The CEGCN model couples CNN and GCN models, using GCN to segment the image into a graph structure, and CNN to extract and analyze spatial features. Finally, the feature information of nodes and edges is summarized, and the CEGCN model can be embedded in LSTM layers. Therefore, the model combining CEGCN and LSTM can combine the advantages of both while considering the spatial characteristics and temporal correlation of forecast precipitation data. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the present invention aims to provide a spatiotemporal error intelligent post-processing PP-CEGCN-LSTM model that couples multiple numerical weather predictions, so as to correct the predicted precipitation data and improve the accuracy of precipitation forecasts.

[0006] To achieve the above objectives, the present invention provides the following technical solution: Firstly, the present invention provides an intelligent post-processing method for spatiotemporal errors coupled with multiple numerical weather forecasts, the method comprising the following steps:

[0007] Step 1: Obtain hydrological and meteorological measured data and numerical precipitation forecast data. Preprocess the numerical weather forecast precipitation data to obtain graph structure data.

[0008] Step 2: Couple the clustering enhancement graph convolutional network model CEGCN and the long short-term memory network LSTM model, embed the LSTM layer in the CEGCN model, and build a post-processing model PP-CEGCN-LSTM that can simultaneously consider the spatial characteristics and temporal correlation of numerical forecast precipitation data.

[0009] Step 3: Use the post-processing model PP-CEGCN-LSTM to extract features from the entire graph structure data. Extract features from each node and edge in the graph, including spatial location, precipitation elements and temporal information. Divide the nodes into different clusters and cluster them. Perform feature propagation on each cluster and information transmission on the edges. Finally, define the loss function and evaluation index for model training.

[0010] Step 4: Using the hydrological and meteorological measured data from Step 1 and the forecast data of all members of the three numerical precipitation forecasts as model input data, the model is trained by inputting the data into the post-processing model. The message passing mechanism is used to update the features by performing feature fusion on the nodes in each cluster and information passing between edges to obtain more comprehensive and accurate precipitation results.

[0011] Furthermore, the hydrological and meteorological data required in step 1 is CMA-CMOPRH precipitation fusion data; the numerical forecast precipitation data comes from the TIGGE dataset, including numerical precipitation forecast data released by the European Centre for Medium Weather Prediction (ECMWF), the National Center for Environmental Prediction (NCEP), and the China Meteorological Administration's CMA Forecasting Center; the data preprocessing specifically involves converting the three different numerical weather forecast precipitation data into graph structure data, where each node represents a spatial location and each edge represents the spatial relationship between two nodes.

[0012] Furthermore, the hydrological and meteorological measured data involved in step 1 are grid data with an accuracy of 0.1×0.1°, and the numerical forecast precipitation data has an accuracy of 0.5×0.5°. The numerical forecast precipitation data grid is downscaled to 0.1×0.1° using the inverse weight interpolation method, and the data is normalized.

[0013] Furthermore, in step 2, the node features output by the CEGCN model are treated as each time step in the sequence, input into the LSTM model, and then a fully connected layer and a Dropout layer are added for two-dimensional output.

[0014] Furthermore, in step 3, nodes represent precipitation data for each grid point of each member, and edges represent spatial relationships between grid points. Both nodes and edges can be embedded into the LSTM model to learn the dynamic relationships between nodes and edges.

[0015] Furthermore, in step 3, the SSIM metric is defined, and the PP-CEGCN-LSTM model is trained using a weighted average of the SSIM and MSE metrics. RMSE, MAE, CC, and Pbias are defined as evaluation metrics.

[0016] Furthermore, in step 4, the message passing mechanism is to use a clustering algorithm to divide the nodes in the graph data into different clusters, perform feature fusion on the nodes in each cluster, use graph convolution operation to propagate the features of each cluster, obtain information between nodes, and summarize the features of all clusters to obtain a new node representation.

[0017] Secondly, the present invention provides an intelligent post-processing device for spatiotemporal errors coupled with multiple numerical weather forecasts, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, when the processor executes the executable code, it implements the aforementioned intelligent post-processing method for spatiotemporal errors coupled with multiple numerical weather forecasts.

[0018] Thirdly, the present invention provides a computer-readable storage medium having a program stored thereon, characterized in that, when the program is executed by a processor, it implements the aforementioned intelligent post-processing method for spatiotemporal errors coupled with multiple numerical weather forecasts.

[0019] By employing the above-mentioned technical means, the beneficial effects of the present invention are as follows:

[0020] (1) The method of this invention proposes a newly defined model that couples CEGCN and LSTM, which can not only extract and analyze features from spatial data, but also has good analytical and predictive capabilities for time series data, making it suitable for precipitation post-processing. Furthermore, because the CEGCN model shares weights, it reduces the number of model parameters and improves the model's training efficiency.

[0021] (2) The method of this invention proposes a newly defined loss function as an image evaluation index, which can effectively consider the problem of grid spatial image comparison during model training, and maximizes the use of spatiotemporal information by ensembling all members of the numerical precipitation forecast. This innovative method is highly effective and is an important means to solve precipitation forecast errors.

[0022] (3) This invention directly utilizes the deep learning-based post-processing PP-CEGCN-LSTM model, which can significantly reduce the initial error of numerical weather prediction data and greatly improve the accuracy of precipitation forecasts. The method of this invention is simple and effective, has high practical value, and has high application prospects in operational forecasting. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the watershed in the case study watershed of the PP-CEGCN-LSTM model for intelligent post-processing of spatiotemporal errors in numerical weather forecasting based on deep learning, according to the present invention.

[0024] Figure 2 This is a schematic diagram of the PP-CEGCN-LSTM model of the present invention. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. A typical watershed is used as an example for specific implementation, including the following steps:

[0026] Step 1: Preparation of watershed geographic information, measured hydrological and meteorological data, and numerical weather prediction data:

[0027] ArcGIS is used to extract and process geographic information data of the target watershed, obtaining the watershed outline and the watershed catchment area, such as... Figure 1 .

[0028] For CMA-CMORPH fused data, inverse weighted interpolation is used to extract data with a resolution of 0.1°×0.1°. The principle is as follows:

[0029]

[0030]

[0031] Where D i For (x) i ,y i The distance Z from grid point (A,B) to grid point (A,B) (A,B) This is the estimated value at grid point (A,B), which is the sum of the estimated values ​​obtained by weighting the distances of the N grid points closest to (A,B). In this paper, N=4 is used for calculation.

[0032] The hydrological and meteorological data consisted of an hourly precipitation grid dataset fused from 30,000 automatic weather stations in China and CMORPH precipitation products. Hourly precipitation data at 0.1° resolution from 2007 to 2020 were generated using a probability density and optimal interpolation (PDF+OI) algorithm. The latitude and longitude range of the CMA-CMORPH fused data is 28°–29°30′N and 120°30′–122°. Due to missing data for 2007, hourly precipitation data from June to August 2007 and the flood season from April to October 2008–2019 were selected. The data were accumulated over 6 hours (starting at 08:00 each day). Missing and outlier values ​​in the time series were filled using linear interpolation, while missing values ​​in the spatial data were filled using inverse distance weighting. The min-max normalization method was used to convert the data into decimals in the (0,1) interval. Finally, the daily 6-hour cumulative data were sorted by date. The data normalization method uses the min-max normalization method to transform the data into decimals in the (0,1) interval.

[0033] The numerical precipitation forecast data comes from the TIGGE dataset, including numerical precipitation forecast data issued by the European Centre for Medium-Range Weather Forecasts (ECMWF), the National Centers for Environmental Prediction (NCEP), and the China Meteorological Administration (CMA) Forecasting Center. The ensemble precipitation forecast data comes from the TIGGE (THORPEX Interactive Grand Global Ensemble) dataset, including but not limited to short- and medium-range numerical precipitation forecast data issued by forecasting centers such as the European Centre for Medium-Range Weather Forecasts (ECMWF), the National Centers for Environmental Prediction (NCEP), and the China Meteorological Administration (CMA).

[0034] ECMWF numerical weather prediction precipitation data consists of 6-hourly cumulative precipitation data from 51 members. First, this data is converted into 6-hourly cumulative precipitation forecasts for forecast periods of 0–6h, 6–12h, …, 66–72h. For each forecast period, the data is processed as follows: linear interpolation and inverse distance weighted interpolation are used to fill the data; the above methods are then used to normalize each data point from the 51 members; finally, the resulting 51 member data are sorted by date. To ensure data consistency, observed and forecast data are matched based on date. CMA numerical weather prediction precipitation data consists of 6-hourly cumulative precipitation data from 14 members, and the data processing method is the same as for ECMWF data. NCEP numerical weather prediction precipitation data consists of 6-hourly cumulative precipitation data from 21 members, and the data processing method is the same as for ECMWF data.

[0035] Step 2: Couple the Cluster Enhanced Graph Convolutional Network (CEGCN) model and the Long Short-Term Memory (LSTM) model to build a post-processing PP-CEGCN-LSTM model that can simultaneously consider the spatial characteristics and temporal correlation of numerical precipitation data.

[0036] The CEGCN network model is a deep learning method capable of extracting features from graph data. Based on GCN and CNN, the CEGCN model also includes four types of layers: graph convolutional layers, convolutional layers, pooling layers, and fully connected layers. The CEGCN model uses graph convolutional layers to perform convolution operations on the input adjacency matrix and feature matrix, extracting features from aggregated nodes and updating node representations. Convolutional layers extract local structural information of node features and correlation features between nodes and edges; the CEGCN model uses one-dimensional convolution to process node features. Pooling layers perform pooling operations on the feature matrix, reducing the dimensionality of nodes and compressing the feature vectors of multiple nodes into a single vector. Finally, the fully connected layer outputs the intermediate processes of the model, used to couple with the LSTM model. The graph convolution operator and convolutional layer are as follows:

[0037]

[0038]

[0039] in: Let be the feature representation of node i in layer l+1; σ(·) is a non-linear activation function, such as ReLU(); N i For node i, its neighboring nodes include itself; c ij As the normalization factor, Let i be the connection weight between node i in layer l and node i+1 in layer l+1. Let A be an adjacency matrix, where A is a matrix describing the connection relationships between all nodes, and D is a matrix describing the adjacency relationships between nodes. The degree matrix, W l Let be the linear transformation matrix of the node features in the i-th layer.

[0040] LSTM is a special type of recurrent neural network (RNN) with temporal feedforward connections, allowing information from past time steps to influence the output at the current time step. LSTM has four gates: update gate, input gate, forget gate, and output gate. The input and output gates, as their names suggest, relate to the input and output data. The update gate stores accumulated past information, while the "forget gate" selectively "forgets" factors from the past that have little or no impact, preventing them from being propagated. This reduces the accumulated systematic error of the model, resulting in better performance than usual. The formulas are summarized below:

[0041] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0042] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0043]

[0044]

[0045] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0046] h t =o t *tanh(C t )

[0047] Where f t i t o t Represents the forget gate, input gate, and output gate; C t To update the gate, h t For the hidden vector, C t After scaling by the activation function (tanh) and o t Multiplying them together yields the result; [h t-1 ,x t ] indicates that the two vectors are connected; W and b are the basic weight matrices and correction terms.

[0048] The PP-CEGCN-LSTM model comprises two modules: CEGCN spatial feature extraction and concatenation, and LSTM temporal correlation analysis. The coupling step of the coupled model involves passing grid data through separate independent networks to obtain the current two-dimensional precipitation feature vector, and then inputting the precipitation vector at each time step into a bidirectional LSTM through a fully connected layer to extract the temporal correlation between forward and backward directions. The specific model structure is as follows... Figure 2 It represents the post-processing of precipitation data, from input to output.

[0049] Step 3: First, divide the model's input data into a training set and a validation set, and define the loss function and evaluation metrics for model training.

[0050] The loss function is defined as a weighted average of the SSIM and MSE indices, each with a weight of 0.5. Relative Bias, Mean Absolute Error (MAE), Correlation Coefficient (CC), and Root Mean Square Error (RMSE) are selected as evaluation metrics, and their mathematical definitions are as follows:

[0051]

[0052]

[0053]

[0054]

[0055] Where n is the number of observations at each grid point. This indicates that the grid predicts precipitation. This indicates the actual precipitation observed in this grid. This represents the average predicted precipitation for that grid. Y represents the average value of the observed precipitation in this grid. pred ,Y obs These represent the forecast precipitation sequence and the observed precipitation sequence, respectively.

[0056] SSIM is a newly defined loss function, which is a metric for evaluating image quality.

[0057]

[0058] Where μ x ,μ y Let σ be the average of x and y. x ,σ y Let σ be the variance of x and y. xy Let x be the covariance of x and y, and c1 and c2 be the stability constants.

[0059] Step 4: Using measured hydrological and meteorological data and forecast data from all members of the three numerical precipitation forecasts as point data as input precipitation data to the model, and obtaining the relationships between spatial grid points as edges, the data of each member of different numerical weather forecasts is fully coupled to enhance the accuracy of post-processing. The input is then used to train the post-processing model.

[0060] From a precipitation perspective, precipitation at each spatial location is not isolated but is related to the surrounding space. Extracting precipitation information from the area surrounding the target provides a reference for correcting the current target point, resulting in a dataset of (n, 3, 5, 5). All data sets from the rate period are then concatenated into a continuous time series dataset, which serves as the input data for the coupled model. This dataset includes forecast information elements from different members, as well as spatial and temporal information.

[0061] The above description is merely an implementation of the present invention in a typical watershed and is not intended to limit the invention. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the scope of the claims of this invention should be considered within the protection scope of this invention.

Claims

1. A method for intelligent post-processing of spatiotemporal errors coupled with multiple numerical weather predictions, characterized in that, The method includes the following steps: Step 1: Obtain hydrological and meteorological measured data and numerical precipitation forecast data. Preprocess the numerical weather forecast precipitation data to obtain graph structure data. Step 2 involves coupling the clustering-enhanced graph convolutional network model CEGCN and the long short-term memory network (LSTM) model. An LSTM layer is embedded within the CEGCN model to construct a post-processing model, PP-CEGCN-LSTM, that simultaneously considers the spatial characteristics and temporal correlations of numerical weather prediction precipitation data. Specifically: The CEGCN model includes graph convolutional layers, convolutional layers, pooling layers, and fully connected layers. The graph convolutional layers perform convolution operations on the input adjacency matrix and feature matrix, extracting features from aggregated nodes and updating node representations. The convolutional layers extract local structural information of node features and correlation features between nodes and edges, using one-dimensional convolution to process node features. The pooling layers pool the feature matrix, reducing node dimensionality and compressing the feature vectors of multiple nodes into a single vector. Node features are treated as each time step in the sequence and input into the LSTM model. Finally, the fully connected layers output the intermediate processes of the model, used to couple the LSTM model. Step 3: Use the post-processing model PP-CEGCN-LSTM to extract features from the entire graph structure data. Extract features from each node and edge in the graph, including spatial location, precipitation elements and temporal information. Divide the nodes into different clusters and cluster them. Perform feature propagation on each cluster and information transmission on the edges. Finally, define the loss function and evaluation index for model training. Step 4: Using the hydrological and meteorological measured data and the forecast data of all members of the numerical precipitation forecast in Step 1 as model input data, the model is trained by inputting the data into the post-processing model. The message passing mechanism is used to update the features by performing feature fusion on the nodes in each cluster and information passing between edges to obtain more comprehensive and accurate precipitation results.

2. The intelligent post-processing method for spatiotemporal errors coupled with multiple numerical weather predictions as described in claim 1, characterized in that, The hydrological and meteorological data required in step 1 is the CMA-CMOPRH precipitation fusion data; the numerical forecast precipitation data comes from the TIGGE dataset, including numerical precipitation forecast data released by the European Centre for Medium Weather Prediction (ECMWF), the National Center for Environmental Prediction (NCEP), and the China Meteorological Administration's CMA Forecasting Center; the data preprocessing specifically involves converting the three different numerical weather forecast precipitation data into graph structure data, where each node represents a spatial location and each edge represents the spatial relationship between two nodes.

3. The intelligent post-processing method for coupled spatiotemporal errors of multiple numerical weather forecasts as described in claim 1, characterized in that, The hydrological and meteorological data involved in step 1 are gridded data with an accuracy of 0.1×0.1°, and the numerical precipitation forecast data has an accuracy of 0.5×0.5°. The numerical precipitation forecast data gridded data is downscaled to 0.1×0.1° using the inverse weighted interpolation method, and the data is normalized.

4. The intelligent post-processing method for spatiotemporal errors coupled with multiple numerical weather predictions as described in claim 1, characterized in that, In step 3, nodes represent precipitation data for each grid point of each member, and edges represent spatial relationships between grid points. Both nodes and edges can be embedded into the LSTM model to learn the dynamic relationships between nodes and edges.

5. The intelligent post-processing method for spatiotemporal errors coupled with multiple numerical weather forecasts as described in claim 1, characterized in that, In step 3, the SSIM metric is defined, and the PP-CEGCN-LSTM model is trained using a weighted average of the SSIM and MSE metrics. RMSE, MAE, CC, and Pbias are defined as evaluation metrics.

6. The intelligent post-processing method for spatiotemporal errors coupled with multiple numerical weather predictions as described in claim 1, characterized in that, In step 4, the message passing mechanism is to use a clustering algorithm to divide the nodes in the graph data into different clusters, perform feature fusion on the nodes in each cluster, use graph convolution operation to propagate the features of each cluster, obtain information between nodes, and summarize the features of all clusters to obtain a new node representation.

7. A smart post-processing device for spatiotemporal errors coupled with multiple numerical weather forecasts, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the processor executes the executable code, it implements a spatiotemporal error intelligent post-processing method for coupling multiple numerical weather forecasts as described in any one of claims 1-6.

8. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements a spatiotemporal error intelligent post-processing method for coupling multiple numerical weather forecasts as described in any one of claims 1-6.

Images