Atmosphere data assimilation method, device, equipment and storage medium

Abstract

The application provides an atmospheric data assimilation method, device, equipment and storage medium. It relates to the technical field of computer science and atmospheric science. The method comprises the following steps: adding a cost function, a prior estimate and a gradient of the cost function into a gradient-based descent algorithm, adding a posterior estimate, and constructing a four-dimensional variational and ensemble Kalman filter combination model based on Bayesian prior information; and based on the four-dimensional variational and ensemble Kalman filter combination model, data assimilation is performed on atmospheric data. The application combines artificial intelligence technology, especially deep learning and Bayesian optimization adaptive algorithm, enhances and optimizes the traditional data assimilation algorithm, and can be widely applied to the prediction of extreme weather events such as high temperature and heat wave. Through the assimilation of historical data and real-time observation data, the prediction accuracy of future climate change trends is significantly improved, providing stronger support for extreme weather warning and decision making.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer science and atmospheric science, in particular to the intersection of data assimilation, deep learning, numerical analysis, and extreme climate analysis and prediction, and more particularly, to an atmospheric data assimilation method, device, equipment and storage medium. BACKGROUND

[0002] Data assimilation technology has been widely used in the field of atmospheric science, especially in weather forecasting, climate modeling, environmental monitoring, etc. By combining observation data with numerical models, data assimilation can effectively improve the prediction accuracy. Currently, in the field of atmospheric data assimilation, there are mainly the following several commonly used technologies:

[0003] Kalman filter is one of the earliest and most basic data assimilation methods, especially suitable for linear systems. Its basic idea is to adjust the predicted value by estimating the mean and variance of the system state combined with the current observation data. This method has a long history of application in numerical weather prediction, and is usually used for real-time monitoring and short-term prediction, but its application is usually limited to low-dimensional systems or approximately linear systems. For high-dimensional, large-scale problems, especially atmospheric models, its efficiency is low, and it may face numerical instability.

[0004] EnKF is a Kalman filter method based on the integration idea, which estimates the statistical properties of the system state by using multiple "samples" or "ensembles". It avoids the difficulties encountered by Kalman filter methods in high-dimensional problems. EnKF generates multiple state samples (ensembles) from a set of initial states, and uses these samples to calculate the covariance matrix and optimize the data results. EnKF method can effectively handle high-dimensional nonlinear systems and is more computationally efficient, especially suitable for real-time data assimilation process. It is widely used in meteorology, oceanography, environmental monitoring, etc., especially in numerical weather prediction. However, EnKF relies on a large number of ensemble members, requiring high computational resources. In some extreme weather situations, such as modeling complex weather systems, EnKF may face convergence problems.

[0005] Variational assimilation is a assimilation method based on optimization idea, its goal is to minimize the error between model prediction and observation data, usually through minimizing an objective function (such as cost loss function). In atmospheric data assimilation, variational assimilation methods are usually applied in numerical weather prediction models, which can handle more complex nonlinear and high-dimensional systems. The currently widely used are three-dimensional variational assimilation (3D-Var) and four-dimensional variational assimilation (4D-Var), but both methods require high computational resources. Therefore, some related variants have been proposed to improve efficiency.

[0006] The four-dimensional variational and ensemble Kalman filter combination (4DEnVar) method is an improved method that combines variational assimilation and ensemble Kalman filter (EnKF). It estimates the statistical properties of the system by introducing the ensemble method of EnKF, and combines the variational method to optimize the initial conditions, thereby improving the computational efficiency while ensuring high precision. At present, 4DEnVar is also considered as an excellent method that combines the precision of 4D-Var and the efficiency of EnKF, and it can achieve efficient data assimilation in complex systems such as atmospheric prediction. By using the integration idea in the variational framework, 4DEnVar can effectively reduce the computational overhead while providing high precision, so this method is also the most commonly used method in the field of data assimilation.

[0007] The main technologies in the current field of atmospheric data assimilation include classical Kalman filter, variational assimilation method, ensemble Kalman filter (EnKF), and the derived 4DEnVar method. These technologies have their own advantages and disadvantages, and are suitable for different application scenarios and computing resources. In recent years, with the improvement of computing power and the development of artificial intelligence technology, deep learning and machine learning methods have been introduced into the field of data assimilation, greatly expanding the application range and effect of assimilation technology. Therefore, the data assimilation technology based on artificial intelligence enhancement proposed in the present application will pay more attention to improving the computational efficiency, precision, and ability to handle nonlinear and high-dimensional systems, especially in the fields of high-temperature heat wave prediction and extreme climate change prediction, which has important application potential. SUMMARY

[0008] The present application provides an atmospheric data assimilation method, device, equipment and storage medium, aiming to enhance and optimize the traditional data assimilation algorithm by combining artificial intelligence technology, especially deep learning and Bayesian optimization adaptive algorithm, and can be widely applied to the prediction of extreme weather events such as high-temperature heat wave. Through the assimilation of historical data and real-time observation data, the prediction accuracy of future climate change trends is significantly improved, providing stronger support for extreme weather warning and decision-making.

[0009] In a first aspect, the present application provides an atmospheric data assimilation method, comprising:

[0010] The cost function, prior estimate and gradient of the cost function are added to the gradient-based descent algorithm, and the posterior estimate is added to construct a four-dimensional variational and ensemble Kalman filter combination model based on Bayesian prior information;

[0011] Based on the four-dimensional variational and ensemble Kalman filter combination model, the atmospheric data is assimilated.

[0012] In one possible design, the cost function, prior estimate and gradient of the cost function are added to the gradient-based descent algorithm, comprising:

[0013] The gradient of the cost function is given by

[0014]

[0015] where, denotes the gradient of the cost function, x0denotes an initial state vector for which an optimum is sought, B denotes the background error covariance matrix, x b denotes the prior distribution of the state vector at time 0, denotes a diagonal matrix composed of the observation error covariance matrices R of the overall time states 0 to N, denotes the observation data, denotes the observation data;

[0016] Using the following equation

[0017]

[0018] where, is the background field adjustment in the observation space, which drives the analysis field to converge to the optimal solution in the subsequent data assimilation process, represents the joint action item of the background error covariance and the observation bias in the gradient of the objective function, x represents the ordinary parameter, A represents the background observation value converted by the background field through the observation operator, f(x) represents the matching function between the background state, the observation data and the state error, and J(f) represents the minimization function required in the optimization process;

[0019] J is expressed as the Jacobian determinant:

[0020]

[0021] where, represents the cost function of the observation bias at the initial time, h0represents h0(x0), which is the observation space projection at the initial time; m0represents m0(x0), which is the time evolution of the model state; J(h i ) = h i represents the linearized observation data, J(m i ) = m i represents the tangent linear model data, that is, the values of J(h i ) and J(m i ) at x i ; J(h N (m 0→N (x0))) represents the observation bias cost function after the model integration; J(h N ) J(m N-1... J(m0) represents the Jacobian chain product in the gradient computation, which is used to adjust the initial field to minimize the overall error; N represents the time when the final assimilation ends;

[0022] According to formula (3):

[0023]

[0024] Wherein, M N,0 = M N-1 M N-2 ... M0

[0025] In the formula, represents a mathematical mapping, which maps the state of the model from the state space to the observation space; M represents the model error covariance matrix, which describes the error structure of the model itself, and 0~N respectively represent the error covariance matrix of the model at different time states; H0 represents the observation space projection at the initial time, H1 represents the observation operator at time t=1, H N represents the observation constraint at the end of the time window, M0 represents the single-time-step model integration operator, which describes the dynamic propagation of the initial error; M N,0 represents the integration chain within the time window, which supports the modeling of multi-time observation constraints and flow-dependent differences;

[0026] Define the pretreatment matrix U to satisfy B=UU T , after Taylor expansion of h(X b +U w ) in the cost function, the cost function is represented as:

[0027]

[0028] In the formula, J(w) represents the optimization objective function, w represents the optimization variable, and represents the state increment or disturbance;

[0029] The gradient function of the cost function is represented as:

[0030]

[0031] In the formula, represents the gradient operator for the objective function;

[0032] Given a set of m joint state parameter vectors, the expression of the perturbation matrix is defined as:

[0033]

[0034] In the formula, X' b is defined as the perturbation matrix, represents the mean of the model output data at multiple time states, and X b,iThis represents the model output data for the i-th time state;

[0035] The m set members X in equation (8) b From previous predictions or a known distribution N(X) b B), using X b The approximate calculation of the background or prior error covariance matrix is ​​expressed as:

[0036] B≈X′ b (X′ b ) T (9)

[0037] By defining the use of matrix X b This is used as a preprocessing matrix and transformed into the integration space; then... Where w is a vector of length m;

[0038] When the loss function and its gradient are as follows (10) and (11):

[0039]

[0040] For background state X b covariance matrix Approximately:

[0041]

[0042] The modified loss function and its gradient are obtained:

[0043]

[0044] In the formula, Y' b It represents the first derivative of the background state increment.

[0045] In one possible design, posterior estimation is incorporated in the following manner:

[0046] By analyzing the error covariance matrix P a The analysis yielded the following results:

[0047]

[0048] In the formula, K represents the Kalman gain matrix, I represents the identity matrix, and X' a P represents the first derivative of the analyzed state data. b This represents the background error covariance matrix, which describes the error and uncertainty of the background state estimation.

[0049] The Kalman gain matrix is ​​expressed as:

[0050]

[0051] According to equation (16), in all the following content, Use Y b To approximate and avoid calculating the tangent linear model and linearized observation operator, the analysis error covariance matrix becomes:

[0052]

[0053] Using formula (18):

[0054]

[0055] use Rewrite the original matrix to get:

[0056]

[0057] In the formula, X' a satisfy

[0058] The expression for the posterior perturbation matrix is ​​defined as follows:

[0059]

[0060] In the formula, X a,i Indicate the state X to be analyzed at time i. a The value;

[0061] By substituting the posterior set, we have:

[0062]

[0063] In the formula, X' a,i X represents a,i The first derivative.

[0064] Secondly, this application provides an atmospheric data assimilation device, the device comprising:

[0065] The model building module is configured to add the cost function, prior estimate, and gradient of the cost function to the gradient-based descent algorithm, and add the posterior estimate to build a four-dimensional variational and ensemble Kalman filter combined model based on Bayesian prior information.

[0066] The data assimilation module is configured to assimilate atmospheric data based on the combined four-dimensional variational and ensemble Kalman filter model.

[0067] Furthermore, the model building module is further configured as follows:

[0068] The gradient of the cost function is given by the following equation:

[0069]

[0070] In the formula, Let x represent the gradient of the cost function, x0 represent the expected initial state vector to be found, and B represent the background error covariance matrix. b This represents the prior distribution of the state vector at time 0. Let R be a diagonal matrix consisting of the observation error covariance matrices of the global time states 0 to N. Represents observation data, Represents observation data;

[0071] Using the following formula:

[0072]

[0073] In the formula, This serves as an adjustment for the background field in the observation space, driving the analysis field towards the optimal solution during the subsequent data assimilation process. denoted by , represents the joint action term of background error covariance and observation bias in the gradient of the objective function, x represents the ordinary parameter, A represents the background observation value after the background field is transformed by the observation operator, f(x) represents the matching function between the background state, the observation data and the state error, and J(f) represents the minimization function required in the optimization process.

[0074] Express J as a Jacobian determinant:

[0075]

[0076] In the formula, The cost function representing the observation bias at the initial moment, h0 represents h0(x0), which is the spatial projection of the observation at the initial moment; m0 represents m0(x0), which is the temporal evolution of the model state; J(h i ) = h i , representing linearized observation data, J(m i ) = m i , representing the tangent linear model data, i.e., J(h) i ) and J(m i ) in x i Values ​​are taken at the location; J(h) N (m 0→N (x0))) represents the observation bias cost function after model integration; J(h N )J(m N-1 ...J(m0) represents the Jacobian matrix chain product in gradient calculation, used to adjust the initial field to minimize the overall error; N represents the final moment of assimilation.

[0077] According to equation (3), we can conclude that:

[0078]

[0079] In the formula, represents a mathematical mapping that maps the model's state from the state space to the observation space; M represents the model error covariance matrix, which describes the error structure of the model itself, and its 0 to N values ​​represent the model's error covariance matrices at different time states; H0 represents the initial observation space projection, H1 represents the observation operator at time t=1, H... N The observation constraint represents the end of the time window; M0 represents the single-time-step mode integral operator, describing the dynamic propagation of the initial error; M N,0 This represents the integral chain within a time window, supporting multi-time observation constraints and flow dependency difference modeling;

[0080] Define the preprocessing matrix U such that B = UU T After analyzing h(X) in the cost function b +U w The Taylor expansion of ) expresses the cost function as:

[0081]

[0082] In the formula, J(w) represents the objective function, w represents the optimization variable, and represents the state increment or disturbance;

[0083] The gradient function of the cost function is expressed as:

[0084]

[0085] In the formula, This represents the gradient operator with respect to the objective function;

[0086] Given a set of m joint state parameter vectors, the expression for the perturbation matrix is ​​defined as follows:

[0087]

[0088] In the formula, X' b Defined as a perturbation matrix, X represents the mean of the model output data across multiple time states. b,i This represents the model output data for the i-th time state;

[0089] The m set members X in equation (8) b From previous predictions or a known distribution N(X) b B), using X b The approximate calculation of the background or prior error covariance matrix is ​​expressed as:

[0090] B≈X′ b (X′b ) T (9)

[0091] By defining the use of matrix X b This is used as a preprocessing matrix and transformed into the integration space; then... Where w is a vector of length m;

[0092] When the loss function and its gradient are as follows (10) and (11):

[0093]

[0094] For background state X b covariance matrix Approximately:

[0095]

[0096] The modified loss function and its gradient are obtained:

[0097]

[0098] In the formula, Y' b It represents the first derivative of the background state increment.

[0099] Furthermore, the model building module is further configured as follows:

[0100] By analyzing the error covariance matrix P a The analysis yielded the following results:

[0101]

[0102] In the formula, K represents the Kalman gain matrix, I represents the identity matrix, and X' a P represents the first derivative of the analyzed state data. b This represents the background error covariance matrix, which describes the error and uncertainty of the background state estimation.

[0103] The Kalman gain matrix is ​​expressed as:

[0104]

[0105] According to equation (16), in all the following content, Use Y b To approximate and avoid calculating the tangent linear model and linearized observation operator, the analysis error covariance matrix becomes:

[0106]

[0107] Using formula (18):

[0108]

[0109] use Rewrite the original matrix to get:

[0110]

[0111] In the formula, X' a satisfy

[0112] The expression for the posterior perturbation matrix is ​​defined as follows:

[0113]

[0114] In the formula, X a,i Indicate the state X to be analyzed at time i. a The value;

[0115] By substituting the posterior set, we have:

[0116]

[0117] In the formula, X' a,i X represents a,i The first derivative.

[0118] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to perform the atmospheric data assimilation method as described in the first aspect and various possible designs of the first aspect.

[0119] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the atmospheric data assimilation method described in the first aspect and various possible designs of the first aspect.

[0120] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the atmospheric data assimilation method described in the first aspect and various possible designs of the first aspect.

[0121] The atmospheric data assimilation method, apparatus, equipment, and storage medium provided in this application have at least the following beneficial effects:

[0122] This application addresses the challenges of current data assimilation methods, which typically involve high computational resource requirements and unsatisfactory final results. By incorporating information from the Bayesian posterior perturbation matrix, it enables better comparison and selection of assimilated data, thereby significantly improving the accuracy of the dataset in subsequent models and enhancing the training performance of the models.

[0123] Specifically, 4DEnVar introduces a Bayesian optimization process to make background error and observation error independent of each other, thus avoiding the neglect of extreme weather data during computation due to the model's pursuit of minimizing the loss function. Simultaneously, the AI-enhanced deep learning component employs a perturbation observation method, ensuring that each DA member uses different observation errors, adhering to statistical principles of observation errors. Compared to the high computational demands of traditional WRF-DA methods, this application, by incorporating AI deep learning into the loss function calculation process and utilizing Bayesian optimization and enhanced error statistical modeling, avoids the problems of missing extreme data and gradual error accumulation during loss function calculation, ensuring the stability and accuracy of the final results. While improving data assimilation accuracy, this application also significantly reduces the time and resource consumption associated with excessive assimilation model computation. Attached Figure Description

[0124] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0125] Figure 1 A flowchart of an atmospheric data assimilation method provided in an embodiment of this application;

[0126] Figure 2 This application provides a probability density distribution image of the prior, posterior, and true values ​​of four parameters in a data assimilation process.

[0127] Figure 3 A comparison chart of the root mean square error of the assimilation-iterated data and the reference data provided in the embodiments of this application; wherein, (a) the initial reference data (relatively smooth); and (b) the reference dataset contains extreme weather elements;

[0128] Figure 4 A comparison of RMSE and its probability density distribution images during the assimilation process of the extreme weather dataset provided in this application embodiment;

[0129] Figure 5 This is a schematic diagram of an atmospheric data assimilation device provided in an embodiment of this application.

[0130] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0131] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0132] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0133] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0134] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0135] This paper leverages artificial intelligence to combine deep learning with data assimilation, overcoming the excessive computational demands of current data assimilation processes and the need to improve convergence to achieve the minimum regional loss function. For example, WRF-DA requires high computational power and has excessively long execution times; PyTorch-based tools such as OpenDA, DAPPER, PDAF, and TorchDA exhibit significant biases in extreme data assimilation, neglecting the accuracy requirements for extreme meteorological elements. Crucially, these data assimilation tools are primarily suited for explicit prediction and observation of background states within the assimilation context; existing tools are largely incapable of handling prediction and observation models represented by deep neural networks. Therefore, this application provides an atmospheric data assimilation method—a novel approach combining the 4DEnVar data assimilation algorithm with deep learning—which will contribute to providing a more advanced, comprehensive, accurate, and convenient solution for data assimilation.

[0136] Specifically, Figure 1 A flowchart illustrating an atmospheric data assimilation method provided in an embodiment of this application. Figure 1 As shown, the atmospheric data assimilation method includes steps S10 to S20.

[0137] S10. The cost function, prior estimate, and gradient of the cost function are added to the gradient-based descent algorithm, and a posterior estimate is added to construct a four-dimensional variational and ensemble Kalman filter combined model based on Bayesian prior information.

[0138] S20. Based on the aforementioned four-dimensional variational and ensemble Kalman filter combined model, atmospheric data is assimilated.

[0139] The atmospheric data assimilation method provided in this embodiment primarily employs a combination of four-dimensional variational and ensemble Kalman filtering (4DEnVar). By combining Bayesian test information with atmospheric physical states, it generates background field data with smaller biases, improving the predictive ability of subsequent AI models for extreme weather events (such as extreme heat waves). This embodiment also utilizes meteorological data provided by ERA5, offering a wealth of research data for validating the data assimilation model.

[0140] The core of this method lies in constructing a four-dimensional variational and ensemble Kalman filter (4DEnVar) model based on Bayesian prior information. Initially, it optimizes the original minimum loss matrix in the 4D-Var algorithm. Subsequently, by incorporating the prior estimate, cost function, and its gradient into a gradient-based descent algorithm, a posterior estimate can be added to the 4DEnVar algorithm. Compared to the conventional 4DEnVar algorithm, this AI-enhanced 4DEnVar algorithm considers more of the differences between extreme and relatively stable data during the data assimilation process. Through deep learning iteration, it learns and predicts error statistics using the assimilated data obtained from training, particularly reducing the computational resource requirements in high-dimensional problems. It also effectively avoids the shortcomings of existing data assimilation methods that blindly pursue the minimum loss function during assimilation, leading to the acquisition of extreme weather data. This improves prediction fidelity and promotes dynamic updates of the model state.

[0141] First, by incorporating the cost function, prior estimate, and gradient of the cost function into the gradient-based descent algorithm, the diagonal matrix R in the loss function can be found in this part, and the gradient of the cost function is then given by the following equation:

[0142]

[0143] In the formula, Let x represent the gradient of the cost function, x0 represent an optimal initial state vector that the algorithm aims to find, and B represent the background error covariance matrix. b This represents the prior distribution of the state vector at time 0. This represents a diagonal matrix composed of R (observation error covariance matrix) representing the global time states from 0 to N. This represents the transformation of observed data (from state variables to the observation space). This represents observational data.

[0144] In this embodiment, the following formula is used:

[0145]

[0146] In the formula, This can be understood as a background field adjustment in the observation space, which drives the analysis field to converge toward the optimal solution during the subsequent data assimilation process. denoted by , represents the joint action term of background error covariance and observation bias in the gradient of the objective function, x represents the ordinary parameter, A represents the background observation value after the background field is transformed by the observation operator, f(x) represents the matching function between the background state, the observation data and the state error, and J(f) represents the minimization function required in the optimization process.

[0147] Here, J is the Jacobian determinant.

[0148]

[0149] In the formula, The cost function representing the observation bias at the initial moment, h0 represents h0(x0), which is the spatial projection of the observation at the initial moment; m0 represents m0(x0), which is the temporal evolution of the model state; J(h i ) = h i , representing linearized observation data, J(m i ) = m i , representing the tangent linear model data, i.e., J(h) i ) and J(m i ) in x i Values ​​are taken at the location; J(h) N (m 0→N (x0))) represents the observation bias cost function after model integration; J(h N )J(m N-1 ...J(m0) represents the Jacobian matrix chain product in gradient calculation, used to adjust the initial field to minimize the overall error; N represents the moment when assimilation finally ends.

[0150] Based on the above formula, we can conclude that:

[0151]

[0152] Among them, M N,0 =M N-1 M N-2 …M0

[0153] In the formula, represents a mathematical mapping that maps the model's state from the state space to the observation space; M represents the model error covariance matrix, which describes the error structure of the model itself, and its 0 to N values ​​represent the model's error covariance matrices at different time states; H0 represents the initial observation space projection, H1 represents the observation operator at time t=1, H... N The observation constraint represents the end of the time window; M0 represents the single-time-step mode integral operator, describing the dynamic propagation of the initial error; M N,0 It represents the integral chain within the time window, supporting the modeling of observation constraints and flow dependency differences at multiple time points.

[0154] In this embodiment, the preprocessing matrix U is defined to satisfy B = UU T After analyzing h(X) in the cost function b +U w The Taylor expansion of ) can further approximate the cost function as:

[0155]

[0156] In the formula, J(w) represents the objective function, w represents the optimization variable, and represents the state increment or perturbation.

[0157] The gradient function of the cost function is given as follows:

[0158]

[0159] In the formula, This represents the gradient operator with respect to the objective function.

[0160] Since the square root of a matrix is ​​not unique, there will be multiple choices for matrix U in the preprocessing part. In this embodiment, we choose to use AI-enhanced deep learning to iterate multiple times to select the most suitable preprocessing matrix in order to ensure the accuracy of Bayesian posterior information in the subsequent part.

[0161] Furthermore, given a set of m joint state parameter vectors in this embodiment, the expression for the perturbation matrix can be defined as follows:

[0162]

[0163] In the formula, X' b Defined as a perturbation matrix, X represents the mean of the model output data across multiple time states. b,i This represents the model output data for the i-th time state.

[0164] The above formula contains m set members X b It can come from previous predictions (in this case, X). b (It is the average of the m set members) or from a known distribution N(X) b B), such that X b =X b Using X b It can approximate the background or prior error covariance matrix by means of...

[0165] B≈X′ b (X′ b ) T (9)

[0166] Then matrix X can be used by definition. b This is used as the preprocessing matrix in this embodiment and transformed into the integration space. Where w is a vector of length m.

[0167] X0 defined in this way can effectively reduce problems when the state or parameter vector is much larger than the ensemble size, and can also regularize the problem when the state or parameter vector contains an order of magnitude larger than the comparison vector. Even if the loss function becomes as follows:

[0168]

[0169] And the gradient is:

[0170]

[0171] In this embodiment, the background state X is... b covariance matrix Approximately

[0172]

[0173] This leads to the modified loss function and its gradient:

[0174]

[0175]

[0176] In the formula, Y' b It represents the first derivative of the background state increment.

[0177] In the following section, this embodiment calculates the posterior information using a Bayesian optimization process to generate a reference posterior perturbation matrix. This is achieved by optimizing the error covariance matrix P. a The analysis includes

[0178]

[0179] In the formula, K represents the Kalman gain matrix, I represents the identity matrix, and X' a P represents the first derivative of the analyzed state data. b This represents the background error covariance matrix, which describes the error and uncertainty of the background state estimation.

[0180] Matrix K is the Kalman gain matrix, expressed as:

[0181]

[0182] Based on the above formula, in all subsequent content... Y can be used b To approximate the linear model and linearized observation operator, the analysis error covariance matrix becomes:

[0183]

[0184] Using formula (18):

[0185]

[0186] use To rewrite the original matrix

[0187]

[0188] And satisfy

[0189] Finally, in this embodiment, the expression for the posterior perturbation matrix is ​​defined as follows:

[0190]

[0191] In the formula, X a,i Indicate the state X to be analyzed at time i. a The value.

[0192] Therefore, in this embodiment, by substituting the posterior set, we have:

[0193]

[0194] In the formula, X' a,i X represents a,i The first derivative.

[0195] Therefore, in addition to avoiding the computation of linearized observation operators and tangent linear models, the 4DEnVar algorithm of this invention also provides data assimilation operators with more ways to compare the merits of model results, provided that the reference condition of providing posterior uncertainty information is provided.

[0196] The traditional 4DEnVar algorithm assumes that the error of the state variable follows a Gaussian distribution and uses a linear model to describe the error and observation error. However, in reality, the error may have a complex nonlinear structure. Therefore, this embodiment uses deep neural networks to learn the nonlinear distribution characteristics of the state variable error. These networks can automatically capture the complex patterns of the error through training samples (the difference between the assimilation model simulated data and the original baseline data); at the same time, generative adversarial networks are used to generate an error field consistent with the observed data, providing more accurate error estimation. The trained deep learning model is embedded into the 4DEnVar data assimilation process, replacing the traditional error modeling and optimization steps. During backpropagation, the neural network can adaptively adjust the learning rate and weights to accelerate the optimization of the objective loss function of the 4DEnVar algorithm, thereby optimizing the entire assimilation process.

[0197] Based on the four-dimensional variational and ensemble Kalman filter (4DEnVar) model constructed using the above method and based on Bayesian prior information, data assimilation is performed on meteorological data provided by ERA5. The results are as follows: Figures 2 to 4 As shown, where, Figure 2 This is the probability density distribution image of the prior, posterior, and true values ​​of four parameters during the data assimilation process, where the four parameters are p1, p2, k1, and k2. Figure 3 This is a comparison chart of the root mean square error between the data after assimilation iteration and the benchmark data. Figure 4 A comparison of RMSE and its probability density distribution images during the assimilation process of an extreme weather dataset.

[0198] Based on the above examples, it can be seen that the method provided in this embodiment can efficiently assimilate extreme weather phenomena (such as heat waves) and other related meteorological data. This model effectively combines Bayesian optimization, the 4DEnVar data assimilation model, and deep learning in AI to generate better assimilated post-processing data, thus overcoming the current problems of inaccurate assimilated data for extreme weather elements and excessive computational demands on the assimilation process. This innovation lies in ensuring the accuracy and usability of the data through multiple iterations of AI deep learning, playing a positive role in the subsequent prediction process of various meteorological elements in large AI models.

[0199] Furthermore, this application allows operators to train neural networks to replace traditional numerical models for state transformation functions and observation operator models. This novel data assimilation method provides the ability to perform four-dimensional variational and ensemble Kalman filtering (4DEnVar) data assimilation using non-explicit transformation functions (represented by neural networks). This tool addresses a significant limitation of current data assimilation tools, namely their difficulty in handling such non-explicit functions. By changing parameters such as the number of observations, observation network, ensemble size, observation error, numerical model, reference trajectory, dilation factor, and location length, operators can create their own custom data assimilation scenarios.

[0200] In summary, the key to this method lies in its specific structure and implementation, including the combination of deep learning models and the 4DEnVar data assimilation method incorporating Bayesian optimization analysis. A detailed description and explanation of these key elements will help ensure that this technology is adequately protected.

[0201] This application also provides an atmospheric data assimilation device, such as... Figure 5 As shown, the atmospheric data assimilation device includes:

[0202] The model building module 501 is configured to add the cost function, prior estimate and gradient of the cost function to the gradient-based descent algorithm, and add the posterior estimate to build a four-dimensional variational and ensemble Kalman filter combined model based on Bayesian prior information.

[0203] The data assimilation module 502 is configured to assimilate atmospheric data based on the four-dimensional variational and ensemble Kalman filter combined model.

[0204] In some embodiments, the model building module is further configured to:

[0205] The gradient of the cost function is given by the following equation:

[0206]

[0207] In the formula, Let x represent the gradient of the cost function, x0 represent the expected initial state vector to be found, and B represent the background error covariance matrix. b This represents the prior distribution of the state vector at time 0. Let R be a diagonal matrix consisting of the observation error covariance matrices of the global time states 0 to N. Represents observation data, Represents observation data;

[0208] Using the following formula:

[0209]

[0210] In the formula, This serves as an adjustment for the background field in the observation space, driving the analysis field towards the optimal solution during the subsequent data assimilation process. denoted by , represents the joint action term of background error covariance and observation bias in the gradient of the objective function, x represents the ordinary parameter, A represents the background observation value after the background field is transformed by the observation operator, f(x) represents the matching function between the background state, the observation data and the state error, and J(f) represents the minimization function required in the optimization process.

[0211] Express J as a Jacobian determinant:

[0212]

[0213] In the formula, The cost function representing the observation bias at the initial moment, h0 represents h0(x0), which is the spatial projection of the observation at the initial moment; m0 represents m0(x0), which is the temporal evolution of the model state; J(h i ) = h i , representing linearized observation data, J(m i ) = mi , representing the tangent linear model data, i.e., J(h) i ) and J(m i ) in x i Values ​​are taken at the location; J(h) N (m 0→N (x0))) represents the observation bias cost function after model integration; J(h N )J(m N-1 J(m0) represents the Jacobian matrix chain product in the gradient calculation, used to adjust the initial field to minimize the overall error; N represents the moment when the assimilation finally ends.

[0214] According to equation (3), we can conclude that:

[0215]

[0216] In the formula, represents a mathematical mapping that maps the model's state from the state space to the observation space; M represents the model error covariance matrix, which describes the error structure of the model itself, and its 0 to N values ​​represent the model's error covariance matrices at different time states; H0 represents the initial observation space projection, H1 represents the observation operator at time t=1, H... N The observation constraint represents the end of the time window; M0 represents the single-time-step mode integral operator, describing the dynamic propagation of the initial error; M N,0 This represents the integral chain within a time window, supporting multi-time observation constraints and flow dependency difference modeling;

[0217] Define the preprocessing matrix U such that B = UU T After analyzing h(X) in the cost function b +U w The Taylor expansion of ) expresses the cost function as:

[0218]

[0219] In the formula, J(w) represents the objective function, w represents the optimization variable, and represents the state increment or disturbance;

[0220] The gradient function of the cost function is expressed as:

[0221]

[0222] In the formula, This represents the gradient operator with respect to the objective function;

[0223] Given a set of m joint state parameter vectors, the expression for the perturbation matrix is ​​defined as follows:

[0224]

[0225] In the formula, X' b Defined as a perturbation matrix, X represents the mean of the model output data across multiple time states. b,i This represents the model output data for the i-th time state;

[0226] The m set members X in equation (8) b From previous predictions or a known distribution N(X) b B), using X b The approximate calculation of the background or prior error covariance matrix is ​​expressed as:

[0227] B≈X′ b (X′ b ) T (9)

[0228] By defining the use of matrix X b This is used as a preprocessing matrix and transformed into the integration space; then... Where w is a vector of length m;

[0229] When the loss function and its gradient are as follows (10) and (11):

[0230]

[0231] For background state X b covariance matrix Approximately:

[0232]

[0233] The modified loss function and its gradient are obtained:

[0234]

[0235] In the formula, Y' b It represents the first derivative of the background state increment.

[0236] In some embodiments, the model building module is further configured to:

[0237] By analyzing the error covariance matrix P a The analysis yielded the following results:

[0238]

[0239] In the formula, K represents the Kalman gain matrix, I represents the identity matrix, and X' a P represents the first derivative of the analyzed state data. b This represents the background error covariance matrix, which describes the error and uncertainty of the background state estimation.

[0240] The Kalman gain matrix is ​​expressed as:

[0241]

[0242] According to equation (16), in all the following content, Use Y b To approximate and avoid calculating the tangent linear model and linearized observation operator, the analysis error covariance matrix becomes:

[0243]

[0244] Using formula (18):

[0245]

[0246] use Rewrite the original matrix to get:

[0247]

[0248] In the formula, X' a satisfy

[0249] The expression for the posterior perturbation matrix is ​​defined as follows:

[0250]

[0251] In the formula, X a,i Indicate the state X to be analyzed at time i. a The value;

[0252] By substituting the posterior set, we have:

[0253]

[0254] In the formula, X' a,i X represents a,i The first derivative.

[0255] This application provides an electronic device. The electronic device may include a processor and a memory, wherein the processor and the memory can communicate; exemplarily, the processor and the memory communicate via a communication bus.

[0256] The processor executes computer execution instructions stored in memory, causing the processor to perform the schemes in the above embodiments. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0257] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0258] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0259] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the technical solution of the atmospheric data assimilation method described above.

[0260] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the atmospheric data assimilation method in the above embodiments.

[0261] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0262] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0263] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0264] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0265] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0266] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0267] Buses can be Industry Standard Architecture (ISA) buses, Peripheral Component Interconnect (PCI) buses, or Extended Industry Standard Architecture (EISA) buses, etc. Buses can be categorized into address buses, data buses, control buses, etc.

[0268] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0269] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0270] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0271] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method of atmospheric data assimilation, characterized by, The method comprises: adding the cost function, the prior estimate and the gradient of the cost function into a gradient-based descent algorithm, and adding the posterior estimate to construct a four-dimensional variational and ensemble Kalman filter combined model based on Bayesian prior information; based on the four-dimensional variational and ensemble Kalman filter combined model, data assimilation is performed on the atmospheric data; adding the cost function, the prior estimate and the gradient of the cost function into a gradient-based descent algorithm, including: the gradient of the cost function is given by the following formula: wherein denotes the gradient of the cost function, x0denotes an initial state vector for which an optimum is expected to be found, B denotes the background error covariance matrix, x b denotes the state vector at time 0, x0denotes the initial state vector, B denotes the background error covariance matrix, x denotes a diagonal matrix composed of the observation error covariance matrices R of the overall time states 0 to N, denotes the simulated observation values after the observation operator has mapped the initial state vector x0of the model to the observation space, denotes the set of actual observation data; J is expressed as a Jacobian determinant by using the following formula: wherein is the adjustment of the background field in the observation space, which drives the analysis field to converge to the optimal solution in the subsequent data assimilation process, represents the joint action term of the background error covariance and the observation bias in the gradient of the objective function, x represents the ordinary parameters, A represents the background observation value converted by the observation operator, f(x) represents the matching function between the background state, the observation data and the state error, and J(f) represents the minimization function required in the optimization process. According to formula (3), it is obtained that: wherein h0represents the cost function of the observation bias at the initial time, h0(x0) is the observation space projection at the initial time; m0represents the time evolution of the model state, m0(x0) is the model state at the initial time; J(h i ) = h i , represents the linearized observation data, J(m i ) = m i , represents the tangent linear model data, i.e., J(h i ) and J(m i ) take values at x i ; J(h N (m 0→N (x0))) represents the observation bias cost function after the model integration; J(h N ) J(m N-1 )...J(m0) represents the chain product of the Jacobian matrices in the gradient calculation, used to adjust the initial field to minimize the overall error; N represents the time at which the final assimilation ends. In the formula, J(w) represents an optimization objective function, w represents an optimization variable, and represents a state increment or a disturbance; wherein M N,0 = M N-1 M N-2 …M0 where represents a mathematical mapping from the state space of the model to the observation space; M represents the model error covariance matrix, which describes the error structure of the model itself, and 0 < N represent the error covariance matrices of the model at different time states; H0 represents the initial time observation space projection, H1 represents the observation operator at time t = 1, H N represents the observation constraint at the end of the time window; M0 represents the single-time-step model integral operator, which describes the dynamic propagation of the initial error, and M N,0 represents the integral chain within the time window, which supports the modeling of multi-time observation constraints and flow-dependent differences. The definition of the pre-processing matrix U satisfies B = UU T , the Taylor expansion of h(X b + U w ) in the cost function is carried out, and the cost function is represented as: the gradient function of the cost function is expressed as: Given a set of m joint state parameter vectors, the expression of the perturbation matrix is defined as: In the formula, denotes the gradient operator with respect to the objective function; In the case of the loss function and its gradient as shown in the following formula (10) and (11): where X b is defined as the perturbation matrix, represents the mean of the model output data in multiple time states, X b,i represents the model output data of the i-th time state; The m set members X in formula (8) b The distribution N(X b ,B) from previous predictions or known, approximated background or prior error covariance matrix, denoted as: b , B ≈ X' b (X' b ) T (9) By defining the use of a matrix X b as a pre-processing matrix and transforming into the integrated space; then the modified loss function and its gradient are obtained as follows: The covariance matrix of the background state X b is approximated as: ​ the posterior estimate is added by the following way: where Y b represents the first derivative of the background state increment; the Kalman gain matrix is expressed as: By analyzing the error covariance matrix P a we obtain: where K represents a Kalman gain matrix, I represents a unit matrix, X a represents a first derivative of the analysis state data, P b represents a background error covariance matrix, describing the error and uncertainty of the background state estimate; using formula (18): According to equation (16), in all that follows, with Y b to avoid computing the tangent linear model and linearized observation operator, where the analysis error covariance matrix becomes: the expression of the posterior perturbation matrix is defined as: Rewrite the original matrix with to get: In the formula, X' a satisfies by substituting the posterior set, it is obtained that: wherein X a,i represents the value of the analysis state X a at the i-th time state. The device comprises: where X a,i represents the first derivative of X a,i .

2. An atmospheric data assimilation apparatus characterized by comprising: a model construction module configured to add the cost function, the prior estimate and the gradient of the cost function into a gradient-based descent algorithm, and add the posterior estimate to construct a four-dimensional variational and ensemble Kalman filter combined model based on Bayesian prior information; a data assimilation module configured to perform data assimilation on the atmospheric data based on the four-dimensional variational and ensemble Kalman filter combined model; The model construction module is further configured to: the gradient of the cost function is given by the following formula: J is expressed as a Jacobian determinant by using the following formula: wherein denotes the gradient of the cost function, x0denotes an initial state vector for which an optimum is expected to be found, B denotes the background error covariance matrix, x b denotes the prior distribution of the state vector at time 0, denotes a diagonal matrix composed of the observation error covariance matrices R of the overall time states 0 to N, denotes the observation data, denotes the observation data; According to formula (3), it is obtained that: wherein The background adjustment is used to observe the space field, and the analysis field is driven to converge to the optimal solution in the subsequent data assimilation process. The background error covariance and the observation bias in the target function gradient represent the combined action item, x represents the general parameter, A represents the background observation value converted by the background field through the observation operator, f(x) represents the matching function between the background state, the observation data and the state error, and J(f) represents the minimum function required in the optimization process. In the formula, J(w) represents an optimization objective function, w represents an optimization variable, and represents a state increment or a disturbance; where represents the cost function of the observation bias at the initial time, h0represents h0(x0), the observation space projection at the initial time; m0represents m0(x0), the time evolution of the model state; J(h i ) = h i , represents the linearized observation data, J(m i ) = m i , represents the tangent linear model data, i.e., J(h i ) and J(m i ) take values at x i ; J(h N (m 0→N (x0))) represents the observation bias cost function after the model integration; J(h N ) J(m N-1 )...J(m0) represents the chain product of the Jacobian matrices in the gradient calculation, used to adjust the initial field to minimize the overall error; N represents the time at which the final assimilation ends. the gradient function of the cost function is expressed as: where represents a mathematical mapping from the state space of the model to the observation space; M represents the model error covariance matrix, which describes the error structure of the model itself, and 0 ~ N represent the error covariance matrices of the model at different time states, respectively; H0 represents the initial time observation space projection, H1 represents the observation operator at time t = 1, H N represents the observation constraint at the end of the time window, M0 represents the single time step model integral operator, and describes the dynamic propagation of the initial error; M N,0 represents the integration chain within the time window, supports the modeling of multi-time observation constraints and flow-dependent differences; The pre-processing matrix U is defined to satisfy B = UU T , the cost function is expressed as: b + U w ) is expanded in Taylor series Given a set of m joint state parameter vectors, the expression of the perturbation matrix is defined as: In the case of the loss function and its gradient as shown in the following formula (10) and (11): In the formula, denotes the gradient operator with respect to the objective function; the modified loss function and its gradient are obtained as follows: where X b is defined as the perturbation matrix, denotes the mean of the model output data in multiple time states, X b,i denotes the model output data in the i-th time state; The m set members X in formula (8) b The distribution N(X b , B) from previous predictions or known, with X b The background or prior error covariance matrix is approximated, denoted as: B ≈ X' b (X' b ) T (9) By defining the use of a matrix X b as a pre-processing matrix and transforming into the integrated space; then where w is a vector of length m; The model construction module is further configured to: The covariance matrix of the background state X b is approximated as: ​ the Kalman gain matrix is expressed as: where Y b represents the first derivative of the background state increment; using formula (18): By analyzing the error covariance matrix P a we obtain: where K represents a Kalman gain matrix, I represents a unit matrix, X a represents a first derivative of the analysis state data, P b represents a background error covariance matrix, describing the error and uncertainty of the background state estimate; the expression of the posterior perturbation matrix is defined as: According to equation (16), in all that follows, with Y b approximated to avoid computing the tangent linear model and linearized observation operator, where the analysis error covariance matrix becomes: by substituting the posterior set, it is obtained that: Rewrite the original matrix with to get: In the formula, X' a satisfies including: wherein X a,i represents the value of the analysis state X a at the i-th time state. a processor, and a memory connected with the processor in communication; where X a,i represents the first derivative of X a,i .

3. An electronic device, comprising: the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the atmospheric data assimilation method according to claim 1. The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the atmospheric data assimilation method according to claim 1.

5. A computer program product comprising a computer program, which, when executed by a processor, implements the atmospheric data assimilation method according to claim 1.

4. A computer-readable storage medium, characterized in that, ​ ​

Images