Causal-driven adaptive localization method based on ensemble Kalman filtering
By employing a causal-driven adaptive localization method, the Liang-Kleeman information flow is used to calculate causal relationships and dynamically adjust observation weights. This solves the problem of spurious correlations in small set data assimilation and achieves efficient and accurate data assimilation.
Patent Information
- Application Number
- CN202511651122.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-12
- Publication Date
- 2026-02-13
AI Technical Summary
Existing technologies suffer from spurious correlations in the assimilation of small sets of data, resulting in high computational costs and insufficient assimilation accuracy. Traditional localization methods rely on empirical parameters and have limited applicability.
A causal-driven adaptive localization method based on ensemble Kalman filtering is adopted. The causal relationship between observations and state variables is calculated through Liang-Kleeman information flow, the location radius and influence weight of the observation are dynamically determined, and the state is updated in combination with EAKF to reduce the impact of spurious correlations.
It significantly improves assimilation accuracy and reduces computational costs in sparse observation systems, is applicable to complex system models, improves assimilation accuracy by about 10%, and reduces computational resource consumption.
Smart Images
Figure CN121524480A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the field of new generation of marine information technology, and relates to a causal driving adaptive localization method based on ensemble Kalman filtering. BACKGROUND
[0002] EnKF, as a Monte Carlo approximation method of Kalman filtering, has been widely applied since it was introduced into the field of atmospheric and oceanic data assimilation in the 1990s. Its advantage is to capture the nonlinearity of the system by constructing the flow-dependent background error covariance of the background field through ensemble samples. However, this method faces two major challenges: large ensemble size can effectively reduce sampling errors, but will lead to exponential growth of computational cost; small ensemble is prone to produce pseudo-correlation between observations and state variables, which will distort the covariance matrix and affect the results.
[0003] To solve the pseudo-correlation, scholars have proposed covariance localization. Covariance localization is based on a naive assumption that the correlation between observations and state variables will decrease with distance under normal circumstances, limiting the influence of long-distance observations on state variables and reducing the probability of false estimation by small ensembles. Specifically, the concept of localization can be understood as adding additional prior information for effective sampling, so that a small ensemble can achieve the same effect as a large enough ensemble for sample estimation. Early localization uses rectangular functions or conical functions, but there are problems such as the best function may not be Gaussian, resource consumption for parameter adjustment, filtering out long-wave signals containing teleconnections, and limiting assimilation accuracy.
[0004] To overcome these limitations, scholars have proposed adaptive localization schemes, which are usually based on sample estimation rather than using physical distance and predefined functions to calculate localization. Subsequent adaptive localization does not require manual parameter adjustment, but the analysis process is complex and computationally intensive. Therefore, current localization is in a dilemma, with traditional schemes being simple to operate but relying on empirical parameters, and adaptive schemes overcoming parameter dependence but being computationally heavy.
[0005] The essence of localization problem is the spurious correlation between the observation and state variables caused by sampling error under the condition of small sample set. To fundamentally solve this problem, a new method is needed to solve the spurious correlation. Unlike correlation, causality refers to the change of one variable directly leading to the change of another variable. It can be said that there is causality if there is correlation, but there is no causality if there is correlation. Therefore, causality may be used as a means to filter spurious correlation and provide a new idea for the construction of localization scheme. Liang pointed out that the causality between two entities can be judged by information flow. Liang-Kleeman (LK) information flow theory reveals the essence of causality in physical systems through rigorous mathematical derivation. The core is to derive the quantitative expression of information transmission from the dynamic equation, without pre-setting the correlation model between variables, and the causality direction can be directly extracted from time series data, which has shown significant application potential in the field of earth system science. The causality analysis method of LK information flow has potential to solve the spurious correlation of data assimilation, but related research needs to be improved. SUMMARY
[0006] To solve the problems in the prior art, the present application provides a causally driven adaptive localization method based on ensemble Kalman filter, which calculates the causality between observation and state variables by using Liang-Kleeman information flow, thereby realizing the construction of localization method and improving assimilation accuracy. This method is mainly applied to small set data assimilation. By estimating the causality between observation and state variables in the analysis area, the positioning radius and influence weight of the observation are adaptively determined based on these relationships, thereby alleviating the spurious correlation in the background error covariance when dealing with limited ensemble data. Finally, the effectiveness of these methods is evaluated through the experiment of barotropic spectral model, and the improvement of state field estimation is checked and analyzed.
[0007] The technical scheme adopted by the present application to achieve the above-mentioned purpose is as follows: a causally driven adaptive localization method based on ensemble Kalman filter, comprising the following steps: Step one: data preparation; Step two: LK information flow calculation; Step three: significance test; Step four: observation determination; Step five: loop judgment; Step six: intersection operation; Step seven: weight calculation; Step eight: state update.
[0008] The step one specifically comprises: obtaining the time series data of the first set of state variables and observations as the basis for subsequent calculation and analysis.
[0009] The step two specifically comprises: calculating the LK information flow of the acquired time series data by using formula (1), for analyzing the causality condition between the two, to provide the basis for subsequent removal of pseudo-correlation. (1) In the formula, represents the sample covariance between the two time series, represents the covariance of the time series of variable 1 and the Euler forward difference of variable 1.
[0010] The step three specifically comprises: for a given confidence level and confidence interval coefficient , performing significance test on the calculated LK information flow by using formula (2); if the significance test is passed, IF=1; if the significance test is not passed, IF=0; this step is used to screen out data with significant characteristics; (2) In the formula, is the parameter of the approximate normal distribution, which is related to the Fisher information matrix; if the interval contains 0, it means is the cause of , otherwise it is considered that the causality is not significant.
[0011] The step four specifically comprises: determining the observed data for assimilation according to the value of IF, and preliminarily screening out effective data for subsequent assimilation analysis.
[0012] The step five specifically comprises: judging whether it is greater than or equal to the number of sets; if not, let , return to re-acquire the time series data of the state variable and the observation of the th set, and repeat the above steps; if yes, go to the next step; this cycle ensures that all set state variables are completely processed.
[0013] The step six specifically comprises: taking the intersection of the set results processed by the above steps, and further integrating the effective data.
[0014] The step seven specifically comprises: according to the intersection result obtained in step six, calculating the distance from the position of each state variable in the state grid to the farthest observation that can be assimilated, and substituting it into the G-C function formula (3) to obtain the weight of the observation. (3) In the formula, represents the state variable and the observation value Spatial distance, Represents the observed value The maximum radius of the influence that can be generated.
[0015] Step eight specifically includes: for a single observation In this regard, the EAKF method is implemented in two steps: first, the ensemble increment at the observation points is calculated. ; It consists of two parts: the increment of the sample mean of the set. and observation Collection disturbance at the location , respectively, are calculated using the following formulas (4) and (5): (4) (5) In the formula, Indicates the location The A prior set of samples, typically derived from the set of state variables. The position is obtained by linear interpolation; represent The mean of the prior set at that point, i.e. Mean; express The standard deviation of the prior set at that location, i.e. Standard deviation; Indicates the standard deviation of the observation error; The sum of the two calculated parts is the ensemble increment at the observation point. : (6) Next, the least squares fitting method is used to calculate the ensemble increment at the observation points. The linear projection onto the state set increment is calculated using the following formula: (7) In the formula, Indicates the first The first state variable Incremental samples of a set Indicates state change and observation The prior error covariance; By combining this localization scheme with EAKF, the state variables are updated to reduce the impact of spurious correlation observations. The state variable update formula in EAKF is shown in (8): (8) In the formula, Represents state variables and observation Localization factors between them; It is obtained according to formula (3). Indicates from the first The observation value to the first The information flow of each pattern grid point is obtained through steps three, five, and six, and can also be regarded as a binary filter.
[0016] This invention proposes a causal-driven adaptive localization method based on ensemble Kalman filtering. In terms of adaptability, it dynamically determines the localization range and weights through causal analysis, overcoming the limitations of traditional methods that rely on repeated manual experiments to adjust parameters, and solving the problem that fixed localization radii are difficult to adapt to complex systems. In terms of computational efficiency, it eliminates the large number of trial-and-error experiments required in traditional methods to determine the optimal localization radius, significantly reducing experimental resource consumption. In terms of assimilation accuracy, it achieves results comparable to traditional optimal solutions in full-grid observation systems, and in sparse observation systems that are closer to practical applications, it effectively distinguishes between true causal relationships and spurious correlations, reducing the assimilation error by approximately 10%, significantly outperforming traditional methods. In terms of universality, it is not only applicable to various ensemble Kalman filter variants but can also be extended to complex system models such as the atmosphere and ocean, breaking through the bottleneck of limited applicability of traditional localization methods. Attached Figure Description
[0017] Figure 1 This is a flowchart of the specific implementation steps of the present invention; Figure 2 This is a diagram of the observation system described in the implementation of this invention; Figure 3 This invention describes the spatiotemporal averaging of traditional methods at different radii under two observation systems. Renderings; Figure 4 This is the spatiotemporal average of the present invention compared with traditional methods. Renderings. Detailed Implementation
[0018] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments: Example 1
[0019] like Figure 1 The causal-driven adaptive localization method based on ensemble Kalman filtering, as shown, includes the following steps: Step 1: Data Preparation; Step 2: LK information flow calculation; Step 3: Significance test; Step 4: Observation and confirmation; Step 5: Loop through the conditional statements; Step six: intersection operation; Step seven: weight calculation; Step eight: state update.
[0020] The step one specifically comprises: obtaining the first set of state variables and observed time series data as the basis for subsequent calculation and analysis.
[0021] The step two specifically comprises: calculating the LK information flow of the obtained time series data by using formula (1), for analyzing the causal relationship between the two, and providing a basis for subsequent removal of pseudo-correlation; (1) In the formula, represents the sample covariance between the two time series, represents the covariance of the time series of variable 1 and the Euler forward difference of variable 1.
[0022] The step three specifically comprises: for a given confidence level and confidence interval coefficient , the significance test of the calculated LK information flow is carried out by using formula (2); if the significance test is passed, IF=1; if the significance test is not passed, IF=0; this step is used to screen out data with significant characteristics; (2) In the formula, is the parameter of the approximate normal distribution, which is related to the Fisher information matrix; if the interval contains 0, it means is the cause of , otherwise it is considered that the causal relationship is not significant.
[0023] The step four specifically comprises: determining the observed data for assimilation according to the value of IF, and preliminarily screening out effective data for subsequent assimilation analysis.
[0024] The step five specifically comprises: judging whether it is greater than or equal to the number of sets; if not, let , return to obtain the first set of state variables and observed time series data, and repeat the above steps; if yes, go to the next step; this loop ensures that all set state variables are completely processed.
[0025] The step six specifically comprises: taking the intersection of the set results processed by the above steps, and further integrating the effective data.
[0026] Step seven specifically includes: based on the intersection results obtained in step six, sequentially calculating the distance from the position of each state variable's state grid to the farthest observation that can be assimilated. Substitute it into the GC function formula (3) to obtain the weight of the observation; (3) In the formula, Represents state variables Compared with observed values Spatial distance, Represents the observed value The maximum radius of the influence that can be generated.
[0027] Step eight specifically includes: for a single observation In this regard, the EAKF method is implemented in two steps: first, the ensemble increment at the observation points is calculated. ; It consists of two parts: the increment of the sample mean of the set. and observation Collection disturbance at the location , respectively, are calculated using the following formulas (4) and (5): (4) (5) In the formula, Indicates the location The A prior set of samples, typically derived from the set of state variables. The position is obtained by linear interpolation; represent The prior set mean at that point, i.e. The mean; express The standard deviation of the prior set at that location, i.e. Standard deviation; Indicates the standard deviation of the observation error; The sum of the two calculated parts is the ensemble increment at the observation point. : (6) Next, the least squares fitting method is used to calculate the ensemble increment at the observation points. The linear projection onto the state set increment is calculated using the following formula: (7)
[0028] In the formula, Indicates the first The first state variable A set of incremental samples The state variable and the observation The prior error covariance; The localization scheme is combined with the EAKF to update the state variable, reduce the influence of pseudo-correlation observations, and the state variable update formula in the EAKF is shown in equation (8): (8) In the formula, The localization factor between the state variable and the observation ; is obtained according to equation (3), The information flow from the first observation value to the first grid point of the mode, which is obtained through steps three, five and six, and can be regarded as a binary filter.
[0029] Embodiment 2 (1) Twin experiment setting In order to compare the EAKF assimilation experiments of different methods, a twin experiment framework is set in this embodiment, and a real model and an assimilation model are set in the framework. The real model uses an unbiased initial field, and the assimilation model uses a biased initial field. In this embodiment, the 500-hPa zonal and meridional wind data of the Greenwich time on January 1, 1991 of the European Center for Medium-Range Weather Forecasts is used to calculate the atmospheric stream function, and the initial condition is used to forward integrate the real model and the assimilation model for 30 days (1440 time steps), that is, the initial state field of the real model and the assimilation model is obtained.
[0030] The present application adopts Fortran95, and carries out a twin experiment on a global barotropic spectral model. The global barotropic spectral model is applied in the twin experiment of the present application, and the model is based on the equation of potential vorticity conservation:
[0031] In the formula, and respectively represent relative vorticity and planetary vorticity (i.e. Coriolis force parameter), is the “equivalent depth” of the average atmosphere.
[0032] The equation can be rewritten in the form of atmospheric stream function :
[0033] In the formula, represents the influence of terrain, , represents the meridional distance from the equator to the north, It is a Jacobian matrix. This is the Cressman coefficient, which is the reciprocal of the square of the Rossby deformation radius. The global barotropic spectral model uses a diamond truncation method for the conversion between spectral coefficients and grid values. The state variables are time-stepped spectral coefficients (gas flow functions at 64×54 Gaussian grid points). A leapfrog scheme is used to integrate the model, and a Robert-Asselin time filter is employed to suppress spurious computational mode shapes. The time filter coefficients are dimensionless. In this embodiment, its value is taken as The integration time step of the mode is "half an hour" in the corresponding mode, that is, the mode integrates forward one day, which includes 48 time steps.
[0034] Within the twin experiment framework, the real model and the assimilation model were integrated forward for 200 days, and the integrated results of the real model were projected onto the observation points at 6-hour intervals (i.e., 12 time steps), with a superposition standard deviation of 10. 6 m 2 s -1 Gaussian white noise is used as the observation error to simulate the "observation". This embodiment designs two observation systems, such as... Figure 2 As shown, one is a full-grid ideal observation system, where the observation points are in the same position as the grid points in the state space (a total of 3456 observation points), and the other is a sparse observation system (a total of 709 observation points), where "+" represents a grid point and "·" represents an observation point.
[0035] For comparison, the traditional localization method sets 15 influence radii: 125 km, 250 km, 500 km, 750 km, 1000 km, 1250 km, 1500 km, 1750 km, 2000 km, 2500 km, 3000 km, 3500 km, 4000 km, 4500 km, and 5000 km. By comparing the results of twin assimilation experiments under each influence radius condition, the optimal influence radius is determined, and subsequently compared with the results of this embodiment.
[0036] (2) Implementation of LKIF adaptive localization scheme in EAKF As an improvement to the EAKF localization method, the EAKF workflow is first introduced. This is for single observations. In this regard, the EAKF method is implemented in two steps: first, the ensemble increment at the observation points is calculated. . It consists of two parts: the increment of the sample mean of the set. and observation Collection disturbance at the location are calculated by equations (4) and (5) respectively: (4) (5) where, represents the i-th ensemble sample at the location of , which is usually obtained by linearly interpolating the state variable ensemble at the location of ; represents the ensemble mean at the location of , i.e., the mean of ; represents the ensemble standard deviation at the location of , i.e., the standard deviation of ; represents the observation error standard deviation.
[0037] The two parts calculated above are superimposed to obtain the ensemble increment at the observation location (6) Next, the ensemble increment at the observation location is linearly projected onto the state ensemble increment by least squares fitting method, and the calculation formula of the state ensemble increment is as follows: (7) where, represents the i-th ensemble increment sample of the j-th state variable represents the prior error covariance between the state variable and the observation ; G-C localization scheme is widely used in ensemble filtering-based data assimilation methods, which uses a localization factor to weaken the influence of false correlation between observation data and model state variables. Using the estimated localization factor, each ensemble sample of the state variable increment in EAKF can be updated according to the following formula:
[0038] where, represents the localization factor between the state variable and the observation value ; is calculated according to the G-C function (Gaspari & Cohn, 1999), as follows:
[0039] where, Represents state variables Compared with observed values Spatial distance, Represents the observed value The maximum radius of the influence that can be generated.
[0040] A causal-driven adaptive localization method based on ensemble Kalman filtering, the process is as follows: Figure 1 As shown, the steps include: First, calculate the state variables from the observation to the state variables according to formula (1). The LK information flow of ")" is used to retain only the observations that pass the significance test; then, the state variables ( (To the farthest distance to retain the maximum observation distance) (defined as the farthest distance from the target grid point to the observations identified as having a causal relationship by the LK information flow method), and based on this distance, the GC function is used to assign corresponding weights to these observations. The update of the state variables in this localization method is shown in Equation (8).
[0041] (1) In the formula, This represents the sample covariance between two time series. This represents the covariance between the time series of variable 1 and the time series of the Euler positive difference of variable 1.
[0042] (8) In the formula, Indicates from the first The observation value to the first The information flow of each pattern grid point can also be viewed as a binary filter.
[0043] (3) Assimilation experiment As mentioned earlier, the assimilation effect of the GC-based localization scheme depends on the setting of the influence radius. Therefore, the 15 influence radii set for the GC localization scheme were tested sequentially, and the optimal influence radius was selected by comparing the final assimilation effects for comparison with the adaptive localization scheme. To evaluate the assimilation effect based on different influence radii, the assimilation results are statistically presented as the spatiotemporal average RMSE of the updated atmospheric flow function, as shown in the following formula.
[0044]
[0045] In the formula, Indicates the time step Calculated spatial average ; Indicates time hour the atmospheric flow function at the first model grid point; denotes the ensemble mean of the set of state variables; the superscript and respectively denote the updated (assimilated) value and the true value; denotes the number of time steps, denotes the number of grid points.
[0046] As shown in Figure 3 , the optimal radius under the two observation systems is 1250 km.
[0047] Figure 4 The posterior ensemble calculation of the atmospheric flow function of the last 30 days under the influence radius of 1250 km by the traditional method and the application is shown, and it can be found that the assimilation accuracy of the two is equivalent under the full grid observation system, and the assimilation accuracy of the application is higher under the sparse observation system, and the assimilation error is reduced by about 10%.
[0048] By adopting the causal driving adaptive localization method based on the ensemble adjustment Kalman filter provided by the application, the assimilation accuracy can be improved under the sparse observation system, which matches the actual needs.
Claims
1. A causal-driven adaptive localization method based on ensemble Kalman filtering, characterized in that, Includes the following steps: Step 1: Data Preparation; Step 2: LK information flow calculation; Step 3: Significance test; Step 4: Observation and confirmation; Step 5: Loop through the conditional statements; Step Six: Intersection Operation; Step 7: Weight Calculation; Step 8: Status Update.
2. The causal-driven adaptive localization method based on ensemble Kalman filtering according to claim 1, characterized in that, Step one specifically includes: obtaining the... The set of state variables and the observed time series data serve as the basis for subsequent calculations and analyses.
3. The causal-driven adaptive localization method based on ensemble Kalman filtering according to claim 1, characterized in that, The second step specifically includes: using formula (1) to calculate the LK information flow of the acquired time series data, which is used to analyze the causal relationship between the two and to provide a basis for subsequent removal of spurious correlations; （1） In the formula, This represents the sample covariance between two time series. This represents the covariance between the time series of variable 1 and the time series of the Euler positive difference of variable 1.
4. The causal-driven adaptive localization method based on ensemble Kalman filtering according to claim 1, characterized in that, Step three specifically includes: for a given confidence level and confidence interval coefficient The calculated LK information flow is subjected to a significance test using formula (2). If the significance test is passed, then IF=1; if the significance test is not passed, then IF=0. This step is used to screen out data with significant features. （2） In the formula, These are parameters of an approximate normal distribution, related to the Fisher information matrix; if the interval contains 0, it indicates... for If the cause is not obvious, then the causal relationship is considered insignificant.
5. The causal-driven adaptive localization method based on ensemble Kalman filtering according to claim 1, characterized in that, Step four specifically includes: determining the observation data to be assimilated based on the IF value, and initially screening out the effective data for subsequent assimilation analysis.
6. The causal-driven adaptive localization method based on ensemble Kalman filtering according to claim 1, characterized in that, Step five specifically includes: determining Is it greater than or equal to the number of sets? If not, then let... Return to retrieve the first one. For each set of state variables and observed time series data, repeat the above steps; if yes, proceed to the next step; this loop ensures that all set of state variables are fully processed.
7. The causal-driven adaptive localization method based on ensemble Kalman filtering according to claim 1, characterized in that, Step six specifically includes: taking the intersection of the set results processed by the above steps to further integrate the effective data.
8. The causal-driven adaptive localization method based on ensemble Kalman filtering according to claim 1, characterized in that, Step seven specifically includes: based on the intersection results obtained in step six, sequentially calculating the distance from the position of each state variable's state grid to the farthest observation that can be assimilated. Substitute it into the GC function formula (3) to obtain the weight of the observation; （3） In the formula, Represents state variables Compared with observed values Spatial distance, Represents the observed value The maximum radius of the influence that can be generated.
9. The causal-driven adaptive localization method based on ensemble Kalman filtering according to claim 1, characterized in that, Step eight specifically includes: for a single observation In this regard, the EAKF method is implemented in two steps: first, the ensemble increment at the observation points is calculated. ; It consists of two parts: the increment of the sample mean of the set. and observation Collection disturbance at the location , respectively, are calculated using the following formulas (4) and (5): （4） （5） In the formula, Indicates the location The A prior set of samples, typically derived from the set of state variables. The position is obtained by linear interpolation; represent The prior set mean at that point, i.e. The mean; express The standard deviation of the prior set at that location, i.e. Standard deviation; Indicates the standard deviation of the observation error; The sum of the two calculated parts is the ensemble increment at the observation point. : （6） Next, the least squares fitting method is used to calculate the ensemble increment at the observation points. The linear projection onto the state set increment is calculated using the following formula: （7） In the formula, Indicates the first The first state variable Incremental samples of a set Indicates state change and observation The prior error covariance; By combining this localization scheme with EAKF, the state variables are updated to reduce the impact of spurious correlation observations. The state variable update formula in EAKF is shown in (8): (8) In the formula, Represents state variables and observation Localization factors between them; It is obtained according to formula (3). Indicates from the first The observation value to the first The information flow of each pattern grid point is obtained through steps three, five, and six, and can also be regarded as a binary filter.