Numerical weather forecast data correction method based on collaborative optimization

Abstract

The invention discloses a numerical weather forecast data correction method based on collaborative optimization, and the method comprises the steps: carrying out the grading processing of ANO, KF and RF according to error characteristics, forming a progressive correction path of system error-time sequence error-nonlinear residual error, combining the dynamic weight distribution of three correction methods, and carrying out the correction of the numerical weather forecast data. The method can effectively improve the accuracy of each parameter in numerical weather forecast data, supports real-time data stream processing, can be embedded into an existing forecast service system, improves the refinement level of weather forecast, provides weather information service with higher reference value for each industry, and enhances the ability to deal with meteorological disasters.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of meteorological data processing, and particularly relates to a design of a numerical weather prediction data correction method based on collaborative optimization. BACKGROUND

[0002] When a numerical model simulates the meteorological process in a complex terrain area, there are certain deviations in the simulation results due to initial field errors, imperfect parameterization schemes of the model, insufficient resolution of terrain data and other reasons. In order to improve the simulation accuracy, error correction technology has become a research hotspot. At present, the commonly used error correction methods include statistical correction, dynamic correction and machine learning correction. Statistical correction methods such as regression analysis and Kalman filtering correct the simulation results by establishing an error model based on historical data; the dynamic correction method optimizes the simulation process by adjusting the model parameters or the initial field. In recent years, with the development of artificial intelligence technology, error correction methods based on machine learning have gradually emerged, which can effectively reduce the simulation error by training the model with a large amount of data. Studies have shown that atmospheric changes have chaotic nature, and the numerical model used to simulate the atmosphere has uncertainty in the initial field, and the model itself also has inherent defects. These factors cause the long-term existence and difficulty to completely eliminate the prediction error of the numerical model. Therefore, it is of great theoretical and practical significance to correct the prediction results of the numerical model to improve the accuracy of the prediction in the field of meteorological research.

[0003] In the current numerical model prediction correction business practice, the full prediction method (PP) and the model output statistical method (MOS) have been widely popularized in practical application. The full prediction method corrects the prediction results of the model by constructing a linear statistical model between the observation values and the model prediction values; the model output statistical method adjusts the prediction results by constructing a linear statistical model between the observation values and the numerical prediction estimates of a set of related atmospheric variables. In the current numerical model prediction correction technology, the use of a single method often has limitations, and the combination of multiple methods usually adopts a parallel or serial form, lacks a dynamic collaborative mechanism, and causes error accumulation or calculation redundancy. SUMMARY

[0004] The purpose of the present application is to solve the above-mentioned problems existing in the current meteorological numerical model prediction correction technology, and to propose a numerical weather prediction data correction method based on collaborative optimization, which can significantly improve the reliability of the numerical simulation results, improve the fine level of meteorological prediction, provide more valuable meteorological information services for various industries, and enhance the ability to cope with meteorological disasters.

[0005] The technical scheme of the present application is: a numerical weather prediction data correction method based on collaborative optimization, comprising the following steps: S1, a mode anomaly integral prediction correction method is used to eliminate the systematic error in the original numerical weather prediction data, and a primary prediction field after anomaly correction is obtained.

[0006] S2, a Kalman filter recursive average method is used to filter the time sequence of the primary prediction field, eliminate the time sequence error in the primary prediction field, and obtain a filtered grid simulation prediction field.

[0007] S3, based on exogenous physical characteristics, a random forest algorithm is used to filter the nonlinear residual of the grid simulation prediction field, and a numerical weather prediction data correction result is obtained.

[0008] Further, the calculation formula of the primary prediction field after anomaly correction in step S1 is: Wherein represents the primary prediction field after anomaly correction, represents the climate state of the real meteorological data, represents the weather disturbance prediction quantity.

[0009] Further, the calculation formula of the weather disturbance prediction quantity is: Wherein represents the original numerical weather prediction data, represents time, represents the partial derivative of , represents the conjugate of , i represents a statistical variable, n represents a statistical total.

[0010] Further, step S2 includes the following sub-steps: S21, according to the primary prediction field, the prediction value t of each grid point at time is obtained.

[0011] S22, according to the prediction value , the error t of each grid point at time is calculated.

[0012] S23, according to the error , the recursive average error t at time is calculated.

[0013] S24, according to the prediction value and the recursive average error The filtered grid point simulation forecast field .

[0014] Further, the calculation formula of the error t of each grid point at the time is as follows: Wherein represents the observation value of each grid point at the time t .

[0015] Further, the calculation formula of the decreasing average error t of each grid point at the time is as follows: Wherein represents the decreasing weight coefficient, represents the decreasing average error of each grid point at the time .

[0016] Further, the calculation formula of the filtered grid point simulation forecast field is as follows: Further, step S3 comprises the following sub-steps: S31, constructing an original data set according to the filtered grid point simulation forecast field and the external source physical characteristics.

[0017] S32, using the bootstrap sampling method to sample from the original data set with replacement to obtain a plurality of sub-sample data sets.

[0018] S33, at each node of the decision tree, randomly selecting a feature from the sub-sample data set, and determining the splitting mode of the node according to the selected feature.

[0019] S34, constructing a decision tree according to the splitting mode of each node to obtain a numerical weather prediction data correction result.

[0020] Further, step S34 comprises the following sub-steps: S341, calculating the information loss of each selected feature: Wherein represents the information loss, Y represents the category set of the feature, represents the probability that the feature is of category y .

[0021] S342, selecting the feature with the lowest information loss as the splitting feature of the current node.

[0022] S343, according to the splitting feature of the current node, obtaining all unique values contained in the splitting feature, and creating a leaf node corresponding to each unique value.

[0023] S344, for each leaf node, repeating steps S342-S343 until a preset stop condition is met, and a decision tree is constructed.

[0024] S345, under the premise of ensuring sufficient growth of the decision tree, recursively traversing the structure of the decision tree according to the selected features until the corresponding leaf node is located.

[0025] S346, taking the class corresponding to each leaf node as the prediction result of the decision tree on the sub-sample data set.

[0026] S347, integrating the prediction results of all decision trees to obtain the numerical weather forecast data correction result.

[0027] Further, the numerical weather forecast data correction method further comprises dynamic weight distribution of the model anomaly integral prediction correction method, the Kalman filter recursive average method and the random forest algorithm: wherein represents the weight of the model anomaly integral prediction correction method, represents the weight of the Kalman filter recursive average method, represents the weight of the random forest algorithm, represents the forecast validity, a , b , c and d are fitting parameters according to the forecast elements and the forecast validity.

[0028] The beneficial effects of the present application are: the present application firstly classifies ANO, KF and RF according to error characteristics, forms a progressive correction path of systematic error-time series error-nonlinear residual, and combines dynamic weight distribution of the three correction methods, which can effectively improve the accuracy of each parameter in numerical weather forecast data, support real-time data stream processing, can be embedded in existing forecast business system, improve the fine level of meteorological forecast, provide more valuable meteorological information service for various industries, and enhance the ability to cope with meteorological disasters. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1Shown is a flow chart of a numerical weather forecast data correction method based on collaborative optimization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the accompanying drawings are merely exemplary and are intended to illustrate the principles and spirit of the present invention, rather than to limit the scope of the present invention.

[0031] The embodiment of the present invention provides a numerical weather forecast data correction method based on collaborative optimization, such as Figure 1 As shown, the following steps S1 to S3 are included: S1. The model anomaly integral correction (ANO) method is used to eliminate the systematic errors in the original numerical weather forecast data and obtain the anomaly-corrected primary forecast field.

[0032] Atmospheric motion exhibits remarkable, highly nonlinear characteristics. As a complex fluid dynamics system constrained by boundary conditions, its dynamic behavior is extremely complex. The evolution of atmospheric motion is not only strictly governed by its inherent internal dynamical mechanisms but is also significantly influenced by dynamic and thermal processes at the Earth's surface (including the ocean surface), as well as external forcing factors such as solar radiation. The interplay of these internal and external factors makes accurate predictions of atmospheric state changes extremely challenging.

[0033] Based on the basic principles of physics, any physical variable in the atmospheric system s The evolution over time follows the basic physical laws such as Newton's second law, the first law of thermodynamics and the continuity equation. These dynamic equations are combined with the state equations that describe the state of the atmosphere to construct a set of mathematical models that can perform numerical simulations and predictions on atmospheric changes. However, it is worth noting that this numerical model is essentially composed of a set of partial differential equations. Due to the complexity of this set of equations, it is difficult to obtain its exact analytical solution mathematically, and it can only be approximated with the help of numerical calculation methods. Therefore, in the embodiment of the present invention, the original numerical weather forecast data is used as the variable s , from the perspective of mathematical expression, the equations can be summarized as: in and represent the linear part and nonlinear part of the original numerical weather forecast data, respectively. represents a linear function, represents a nonlinear function, Represents model error. Original numerical weather forecast data s In a certain period of time, it can be decomposed into climate states and weather disturbances (or anomalies) relative to the climate state : Combining the above two formulas and taking into account the linear term , we can get: in Represents the original numerical weather forecast data s Linear function of The conjugate of Represents the original numerical weather forecast data s The conjugate of the climate state ), Representation mode error The conjugation of .

[0034] The above formula clearly shows a deviation between the model's output climate data and actual climate conditions. This deviation reflects the inherent mechanism of systematic error accumulation. This deviation is not simply due to the amplification of initial value errors; truncation errors introduced during the re-decomposition of the climate component are also amplified, severely limiting the accuracy and reliability of numerical forecasts.

[0035] Integrate the above formula to obtain the weather disturbance amount : in represents the original numerical weather forecast data, Indicates time, express right The partial derivative of express The conjugate of i represents statistical variables, n Indicates the total statistical amount.

[0036] According to the calculation formula of weather disturbance, the climate field represents the result of weather changes after many years of statistical averaging. When carrying out weather forecasting, it can be regarded as a relatively constant background condition because of its relative stability over a relatively long period of time. Based on this, we can deeply mine and make full use of historical meteorological data, and obtain climate states of real meteorological data that can reflect the true characteristics of the atmosphere through rigorous statistical analysis and scientific calculation methods. , and combined with the weather disturbance forecast Add them together to get the anomaly-corrected primary forecast field : Datasets constructed using existing observational data can accurately depict the true course of atmospheric evolution. Introducing these datasets into numerical forecast models, replacing the climate state generated by the model itself, can effectively improve the accuracy of numerical forecasts and eliminate systematic errors caused by deviations from the model's climate state. In practice, climate field information is obtained by analyzing historical observational data, and weather disturbance fields are forecasted using model integration. This method can partially correct for systematic errors in the model's climate state, making the resulting primary forecast field closer to the actual atmospheric state in terms of temporal and spatial distribution, effectively improving the reliability and practicality of numerical forecasts.

[0037] S2. Use the Kalman filter decreasing average method (KF) to perform time series filtering on the primary forecast field to eliminate the time series error in the primary forecast field and obtain the filtered grid point simulation forecast field.

[0038] The Kalman filter, a classic filtering algorithm, mathematically describes the filter using a linear stochastic system state-space model consisting of a state equation and a measurement equation. This method leverages the recursive nature of the state equation and, based on the linear unbiased minimum mean square error (LMSE) estimation criterion, employs recursive operations to optimally estimate the filter state variables, thereby extracting the most useful signal from the noisy signal that is closest to the true value.

[0039] In practical applications, Kalman filtering is primarily used to solve three core problems: filtering, prediction, and smoothing. In an embodiment of the present invention, Kalman filtering is used to perform decreasing mean error correction. Specifically, this approach estimates the decreasing mean deviation at the time of correction and applies a direct correction to each grid point in the forecast result. In statistics, when faced with the challenge of inaccurate state estimation, Kalman filtering employs a weighted averaging strategy, which, compared to other estimation methods, can make the resulting state estimate closer to the true state.

[0040] Step S2 includes the following sub-steps S21 to S24: S21, obtain the forecast of each grid point based on the primary forecast field t Forecast value at time .

[0041] S22, according to the forecast value Calculate each grid point t Time error : in Indicates that each grid point t Observed value at time.

[0042] S23, according to the error Calculated tThe decreasing average error of the time point : wherein represents a decreasing value, and the value range is [0, 1], represents The decreasing average error of the time point.

[0043] S24, according to the forecast value And the decreasing average error The filtered grid simulation forecast field is calculated .

[0044] S3, based on the exogenous physical characteristics, the random forest algorithm (Random Forest, RF) is used to filter out the nonlinear residual of the grid simulation forecast field, and the numerical weather prediction data correction result is obtained.

[0045] Machine learning, as one of the core research directions in the field of artificial intelligence, has shown strong penetration in many disciplines and industries, and has played a crucial role in promoting the development of many research fields. Machine learning, in essence, is to use pre-set algorithm models to systematically learn historical data, extract data rules, and apply these rules to future data analysis, so as to realize accurate identification and prediction of data. With the continuous enrichment and perfection of data in the field of ocean and meteorology, deep learning can effectively mine potential information from massive data due to its unique advantages. This feature has attracted the attention of researchers and business forecasters. At the same time, a large number of researches and practices have shown that combining deep learning technology with traditional model prediction technology can significantly improve the accuracy of prediction.

[0046] Random forest algorithm is a very flexible machine learning method. The "forest" constructed by this algorithm is integrated by multiple decision trees, and in most cases, bagging sampling method is used for training. Bagging method selects training data and features randomly with replacement, and constructs multiple decision trees based on these data and features, so as to enhance the prediction performance of the overall model. Random forest has excellent performance and broad application prospects in many fields, and can achieve good fitting effect for most types of data.

[0047] Step S3 includes the following sub-steps S31-S34: S31, constructing an original data set according to the filtered grid simulation forecast field and the exogenous physical characteristics.

[0048] In the embodiment of the application, the exogenous physical characteristics include terrain, sea temperature and precipitation.

[0049] S32, bootstrap sampling is adopted to sample from the original data set with replacement to obtain a plurality of sub-sample data sets.

[0050] In the embodiment of the application, each sub-sample data set is consistent in size with the original data set, but allows repeated selection of the same sample. This process can be specifically divided into the following steps: first, the number of training samples in the original data set is N; second, the bootstrap sampling method is used to extract N times from N training samples to form a sub-sample data set.

[0051] S33, at each node of the decision tree, m features are randomly selected from the sub-sample data set (m is much smaller than the total number of features M), and the splitting mode of the node is determined according to the selected features.

[0052] S34, a decision tree is constructed according to the splitting mode of each node to obtain the numerical weather forecast data correction result.

[0053] Step S34 includes the following steps S341-S347: S341, the information loss of each selected feature is calculated: Wherein represents the information loss, Y represents the category set of the feature, represents the probability that the feature is category y .

[0054] S342, the feature with the lowest information loss is selected as the splitting feature of the current node.

[0055] S343, according to the splitting feature of the current node, all unique values contained in the splitting feature are obtained, and for each unique value, a leaf node corresponding to the unique value is created. This step aims to further subdivide the data set according to different values of the splitting feature, laying a foundation for subsequent recursive construction of the decision tree.

[0056] S344, for each leaf node, steps S342-S343 are repeatedly executed until a preset stopping condition is met, and a decision tree is constructed.

[0057] In the embodiment of the application, the stopping condition includes but is not limited to the number of samples in the leaf node reaching a pre-set threshold, or the information loss reduction of all features when splitting the leaf node is less than a pre-set threshold. Through continuous recursive construction, the structure of the decision tree is gradually refined, so that it can more accurately classify or predict data.

[0058] S345, under the premise of ensuring that the decision tree is fully grown, recursively traversing the structure of the decision tree according to the selected features until the corresponding leaf node is located.

[0059] In the embodiment of the application, the decision tree continues to grow until the leaf node is empty, or the number of data instances contained in the leaf node is less than the pre-set minimum split sample number (min_samples_split). This fully grown manner helps the decision tree to fully learn the feature information in the data.

[0060] S346, the category corresponding to each leaf node is taken as the prediction result of the decision tree on the sub-sample data set.

[0061] S347, the prediction results of all decision trees are integrated to obtain the numerical weather forecast data correction result.

[0062] In the embodiment of the application, the prediction results of all decision trees are integrated by using the majority voting method.

[0063] In the embodiment of the application, the random forest algorithm has the following advantages: (1) The model has a lower requirement for data integrity, and the model accuracy is limitedly affected in the case of data missing.

[0064] (2) Through the test and verification of multiple data sets, the model shows excellent stability and accuracy in various data set scenarios.

[0065] (3) The model has strong high-dimensional feature processing capability and can effectively suppress overfitting. The final result is determined by calculating the mean value to ensure the robustness of the output.

[0066] (4) The model training has significant efficiency, especially when facing massive data, parallel computing technology can be used to simultaneously perform data extraction and decision sub-tree generation, greatly shortening the training period.

[0067] (5) The model has strong compatibility and is relatively loose in format and distribution requirements. Whether it is discrete data or continuous data, it can be effectively processed without the need for normalization preprocessing.

[0068] In the embodiment of the application, the pattern anomaly integral prediction correction method, the Kalman filter decreasing average method and the random forest algorithm are also dynamically weighted. Among them, RF has high weight in short-term prediction, and decreases with time effect; KF has dominant weight in medium-term prediction, showing a bell-shaped distribution; ANO gradually increases in weight in long-term prediction.

[0069] Based on this, the dynamic weight allocation formula is: wherein represents the weight of the pattern anomaly integral prediction correction method, represents the weight of the Kalman filter recursive average method, represents the weight of the random forest algorithm, represents the forecast age, divided into short (0-12h), medium (12-72h) and long (72h+) term, a , b , c and d are fitting parameters according to the forecast element and the forecast age.

[0070] According to the above dynamic weight distribution formula, short (0-12h): RF weight is 70%, KF weight is 20%, and ANO weight is 10%; Medium (48h): KF weight is 60%, RF weight is 30%, and ANO weight is 10%; Long (72h+): ANO weight is 50%, KF weight is 30%, and RF weight is 20%.

[0071] The weight distribution rule in the embodiment of the application is dynamic, can be automatically optimized based on historical performance, and the curve shape and parameters can be adjusted according to specific modes (such as temperature / rainfall).

[0072] Those skilled in the art will appreciate that the embodiments described herein are intended to help the reader understand the principles of the application and should be understood as not limiting the scope of protection of the application to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations according to the technical inspirations disclosed in the application without departing from the essence of the application, and these modifications and combinations are still within the scope of protection of the application.

Claims

1. A numerical weather forecast data correction method based on collaborative optimization, characterized in that: The following steps are involved: S1. Use the model anomaly integral forecast correction method to eliminate the systematic errors in the original numerical weather forecast data and obtain the anomaly-corrected primary forecast field; S2. Use the Kalman filter decreasing average method to perform time series filtering on the primary forecast field to eliminate the time series error in the primary forecast field and obtain the filtered grid point simulation forecast field; S3. Based on the external physical characteristics, the random forest algorithm is used to filter out the nonlinear residuals of the grid simulation forecast field to obtain the correction results of the numerical weather forecast data.

2. The method for correcting numerical weather forecast data based on collaborative optimization according to claim 1, characterized in that: The calculation formula of the primary forecast field after anomaly correction in step S1 is: in represents the primary forecast field after anomaly correction, represents the climate state of real meteorological data, Represents the weather disturbance forecast amount.

3. The method for correcting numerical weather forecast data based on collaborative optimization according to claim 2, characterized in that: The weather disturbance forecast The calculation formula is: in represents the original numerical weather forecast data, Indicates time, express right The partial derivative of express The conjugate of i represents statistical variables, n Indicates the total statistical amount.

4. The method for correcting numerical weather forecast data based on collaborative optimization according to claim 1, characterized in that: The step S2 comprises the following sub-steps: S21, obtain the forecast of each grid point based on the primary forecast field t Forecast value at time ; S22, according to the forecast value Calculate each grid point t Time error ; S23, according to the error Calculated t Decreasing mean error at time ; S24. According to the forecast value and decreasing mean error The filtered grid point simulation forecast field is calculated .

5. The method for correcting numerical weather forecast data based on collaborative optimization according to claim 4, characterized in that: In step S22, each grid point t Time error The calculation formula is: in Indicates that each grid point t Observed value at time.

6. The method for correcting numerical weather forecast data based on collaborative optimization according to claim 4, characterized in that: In step S23 t Decreasing mean error at time The calculation formula is: in Indicates a decreasing weight coefficient, express The decreasing mean error at time.

7. The method for correcting numerical weather forecast data based on collaborative optimization according to claim 4, characterized in that: The grid point simulation forecast field after filtering in step S24 The calculation formula is: 。 8. The method for correcting numerical weather forecast data based on collaborative optimization according to claim 1, characterized in that: The step S3 includes the following sub-steps: S31. Construct the original data set based on the filtered grid simulated forecast field and external physical characteristics; S32. Use the bootstrap sampling method to perform sampling with replacement from the original data set to obtain multiple subsample data sets; S33. At each node of the decision tree, randomly select features from the subsample data set, and determine the splitting method of the node based on the selected features; S34. Construct a decision tree according to the splitting method of each node to obtain the correction result of the numerical weather forecast data.

9. The method for correcting numerical weather forecast data based on collaborative optimization according to claim 8, characterized in that: The step S34 includes the following sub-steps: S341. Calculate the information loss of each selected feature: in Indicates information loss, Y represents a set of categories of features, Representing features as categories y probability; S342. Select the feature with the lowest information loss as the splitting feature of the current node; S343. According to the splitting feature of the current node, all unique values ​​contained in the splitting feature are obtained, and for each unique value, a corresponding leaf node is created; S344: Repeat steps S342 to S343 for each leaf node until a preset stopping condition is met, thereby constructing a decision tree. S345. Under the premise of ensuring that the decision tree grows sufficiently, recursively traverse the structure of the decision tree according to the selected features until the corresponding leaf node is located; S346. Taking the category corresponding to each leaf node as the prediction result of the decision tree for the sub-sample data set; S347. Integrate the prediction results of all decision trees to obtain the revised results of numerical weather forecast data.

10. The method for correcting numerical weather forecast data based on collaborative optimization according to claim 1, characterized in that: The numerical weather forecast data correction method further includes dynamically allocating weights to the model anomaly integral forecast correction method, the Kalman filter decreasing average method, and the random forest algorithm: in represents the weight of the model anomaly integral forecast correction method, represents the weight of the Kalman filter decreasing average method, represents the weight of the random forest algorithm, Indicates the forecast timeliness, a 、 b 、 c and d All are fitting parameters based on forecast elements and forecast time.

Images