Four-dimensional variational assimilation method and system based on latent space projection and super set

Abstract

The application discloses a four-dimensional variational assimilation method and system based on hidden space projection and super set, comprising the following steps: projecting a high-dimensional model state to a low-dimensional hidden space by using a pre-trained deep autoencoder; secondly, generating super set members far exceeding the traditional scale by fast reasoning of a deep generative model; constructing a flow-dependent background error covariance matrix in the hidden space based on the super set; combining the flow-dependent background error with a static statistical background error, constructing a mixed four-dimensional variational assimilation objective function in the hidden space and optimizing and solving the function; and finally, reconstructing the optimal analysis hidden variable back to the original high-dimensional state space. The application greatly reduces the calculation dimension through hidden space projection, greatly enriches the flow-dependent information through the super set, and combines the static statistical stability with the flow-dependent adaptability through the mixed background error covariance, so that the assimilation precision and the prediction skill are significantly improved while the calculation efficiency is maintained.

Description

Technical Field

[0001] This invention relates to the field of four-dimensional variational assimilation, and more particularly to a four-dimensional variational assimilation method and system based on latent space projection and supersets. Background Technology

[0002] Four-dimensional variational assimilation (4D-Var) is a core data assimilation technique in fields such as numerical weather prediction, climate simulation, fluid mechanics, and Earth system modeling. Its core principle is to accurately estimate the optimal initial conditions of the model by fitting the trajectory of the numerical model with observational data within a specific time window, thereby providing a reliable foundation for subsequent simulations and forecasts. It plays an irreplaceable role in improving forecast accuracy and optimizing system simulation effects.

[0003] However, traditional four-dimensional variational assimilation methods face three major technical challenges that urgently need to be overcome in practical applications, severely limiting their effectiveness in high-dimensional, nonlinear systems. The high dimensionality itself introduces computational and storage bottlenecks. The state variable dimensions in target application scenarios such as Earth system models and atmospheric numerical models are typically as high as 10⁻⁶. 9 At this high-dimensional scale, the direct calculation, storage, and manipulation of the background error covariance matrix (B matrix) and its inverse matrix are practically infeasible in engineering. Existing techniques often circumvent this problem by introducing simplified approximations (such as assuming the covariance matrix is ​​a diagonal matrix or using localization processing), but such simplifications inevitably lose crucial spatial correlation information, leading to a decrease in assimilation accuracy and making it difficult to meet the needs of high-precision simulation and forecasting.

[0004] Secondly, the objective function of traditional four-dimensional variational assimilation generally exhibits non-convex characteristics in high-dimensional spaces and contains a large number of local minima. Traditional optimization algorithms based on gradient descent are prone to getting trapped in local optima in such objective functions, leading to low convergence efficiency or even convergence to incorrect solutions. This makes it impossible to effectively obtain globally optimal initial model conditions, further affecting the reliability of subsequent simulations and forecasts. Traditional 4D-Var methods use a static background error covariance matrix based on climatological statistics. This matrix can only reflect the long-term average error distribution characteristics and cannot dynamically adapt to changes in specific flow patterns such as current weather conditions and fluid motion states. This results in unreasonable direction and magnitude of analytical increments, making it difficult to accurately correct errors in the model's initial field. This deficiency is particularly prominent in scenarios such as extreme weather and rapidly evolving fluid systems.

[0005] To address the aforementioned issue of insufficient flow dependency, existing technologies have proposed ensemble variational assimilation methods (such as En4D-Var). The core idea is to utilize ensemble forecasts to provide uncertainty information about flow dependency, thereby optimizing the background error covariance matrix. However, these methods still have fundamental limitations: firstly, due to computational resource constraints, operational ensemble forecasts typically contain only 20-100 ensemble members. This limited sample size leads to significant sampling errors in the constructed sample covariance matrix, failing to accurately reflect the uncertainty structure of the system. Secondly, the limited ensemble members cannot comprehensively cover uncertainties across multiple dimensions, such as initial conditions, model parameters, and physical processes. This problem is particularly pronounced in extreme weather events or rapidly evolving systems, where insufficient ensemble quality makes it difficult to fully capture the complete characteristics of forecast uncertainty. Therefore, there is an urgent need for a four-dimensional variational assimilation method and system based on latent space projection and super-ensembles. Summary of the Invention

[0006] The purpose of this invention is to provide a four-dimensional variational assimilation method and system based on latent space projection and supersets that combines computational efficiency, flow dependency adaptability, and physical consistency, in order to solve the problems existing in the current assimilation analysis system, and to address the feasibility of high-dimensional computation, the sufficiency of flow dependency information, and the stability of optimization solutions.

[0007] This invention includes the following steps: S1 data acquisition steps: Obtain the background field of the high-dimensional numerical model, and the observation data sequence within the assimilation time window; S2 Latent Space Projection Step: Using the encoder of a pre-trained deep autoencoder, the background field is projected onto a low-dimensional latent space to obtain background latent variables. The deep autoencoder also includes a decoder for reconstructing the latent variables into high-dimensional states. S3 Superset Generation Steps: Using a pre-trained deep generative model, generate multiple superset members, and project each superset member onto the low-dimensional latent space through the encoder to obtain the corresponding set latent variables; S4 Flow Dependency Covariance Construction Steps: Based on the set of latent variables, calculate the background error covariance matrix of the flow dependency in the low-dimensional latent space; S5 Objective Function Construction Steps: Construct a four-dimensional variational assimilation objective function within the low-dimensional latent space. S6 Optimization Solution Steps: Within the low-dimensional latent space, the optimal analytical latent variables are obtained through an optimization algorithm. S7 State Reconstruction Step: Using the decoder, the optimal analytical latent variables are reconstructed back to the original high-dimensional state space to obtain the final analytical field.

[0008] Furthermore, the deep generative model mentioned in step S3 is a generative adversarial network, variational autoencoder, diffusion model, or normalized flow model, which is trained using historical forecast data or reanalysis data.

[0009] Furthermore, the generation of the superset members in step S3 can be carried out in any of the following ways: based on initial condition perturbation; based on model physical process parameter perturbation; or based on a hybrid perturbation strategy that simultaneously considers initial conditions, boundary conditions and model parameters.

[0010] Further, the dynamic model of the latent space described in step S6 is obtained through any of the following methods: Method 1: In the latent space, the historical sequence of the high-dimensional model state is projected to obtain the latent variable sequence, and a recurrent neural network, long short-term memory network, gated recurrent unit, or neural network with regular differential equations is trained based on this sequence as an alternative model; Method 2: The dynamic model Represented as ,in This is the original high-dimensional numerical model.

[0011] Furthermore, the depth autoencoder is a variational autoencoder, a convolutional autoencoder, or a physically constrained autoencoder.

[0012] Furthermore, the static statistical background error covariance matrix of the latent space described in step S6) is obtained through statistical analysis of the historical latent variable sequence, or can be simplified to a diagonal matrix or an identity matrix.

[0013] Further, the optimization algorithm described in step S5) is a quasi-Newton method, a conjugate gradient method, or a gradient-based stochastic optimization algorithm, constructing the objective function J(z) for four-dimensional variational assimilation, whose expression is:

[0014] in, For the control variables to be optimized, Let be the static statistical background error covariance matrix in the latent space. ∈ [0, 1] represents the mixing coefficient. To characterize the latent space dynamics model that represents the evolution of latent variable z from initial time t0 to latent variable zi at time ti. , where is the observation operator at time ti, and is denoted by . Indicates function composition. Let be the observation error covariance matrix at time ti.

[0015] Furthermore, the mixing coefficient is determined in the following ways: by determining a fixed value based on the statistical optimality of historical assimilation experiments; by dynamically adjusting it based on the statistical relationship between ensemble dispersion and forecast error; or by adaptively adjusting it based on machine learning models according to the current manifold characteristics.

[0016] Secondly, a system based on a four-dimensional variational assimilation method using latent space projection and supersets, characterized in that it includes: The data acquisition module is used to acquire high-dimensional background fields and observation time series data; The deep generative model module is used to quickly generate members of a super set; The autoencoder module stores pre-trained deep autoencoders for bidirectional mapping between high-dimensional states and low-dimensional latent variables. The flow-dependent covariance calculation module constructs the flow-dependent background error covariance in the latent space based on a superset. The latent space assimilation engine is used to construct a hybrid four-dimensional variational assimilation objective function in the latent space and perform optimization to obtain the optimal latent variables for analysis. The state reconstruction module is used to decode the optimal analysis latent variables into the final high-dimensional analysis field.

[0017] The beneficial effects of this invention are: This invention significantly reduces computational dimensionality through latent space projection, greatly enriches flow dependency information through supersets, and combines static statistical stability with flow dependency adaptability by mixing background error covariance, thereby significantly improving assimilation accuracy and forecasting skills while maintaining computational efficiency. Attached Figure Description

[0018] Figure 1 is an overall flowchart of the method of the present invention.

[0019] Figure 2 is a schematic diagram of the observation information introduction path of the method of the present invention.

[0020] Figure 3 is a simplified flowchart of the superset generation and flow-dependent covariance construction method of the present invention.

[0021] Figure 4 is a flowchart of the implicit space four-dimensional variational assimilation optimization solution based on automatic differentiation according to the method of the present invention. Detailed Implementation

[0022] The present invention will be further described below through specific embodiments. The illustrative embodiments and descriptions herein are used to explain the present invention, but are not intended to limit the present invention.

[0023] like Figure 1 As shown, the four-dimensional variational assimilation method based on latent space projection and supersets of the present invention includes the following steps: S1 data acquisition steps: Obtain the background field of the high-dimensional numerical model, and the observation data sequence within the assimilation time window; In this embodiment, regional mesoscale numerical weather prediction is used as the application scenario. The atmospheric state assimilation process over a 100km×100km area in a certain province is simulated. The specific operation process of the four-dimensional variational assimilation method based on latent space projection and superset is described in detail. The core parameters are set based on the requirements of regional mesoscale forecasting, as follows: Assimilation time window selection It contains 3 observation times, covering the key initial period of short-term forecast. The target area is discretized using a 100×100 grid, with 10,000 grid points corresponding to each type of variable. Therefore, the total dimension of the high-dimensional state vector x is 3×100×100=30000, i.e. x∈R30000. The latent space dimension is reduced from high-dimensional state to low-dimensional latent space by a pre-trained deep autoencoder. The dimension of the latent variable z is set to 500, i.e. z∈R500, which balances computational efficiency and preservation of dynamic information. The super set size is based on a deep generative model that generates 1000 set members, denoted as {x(1),x(2),...,x(1000)}, to fully capture forecast uncertainties; The mixing coefficient is based on the statistically optimal results of assimilation experiments over the same period over the past 10 years. The mixing coefficient α = 0.3 is set to balance the stability of static covariance and the adaptability of flow-dependent covariance. The optimization algorithm uses the quasi-Newton method (L-BFGS) to solve for the minimum value of the latent space objective function. The convergence threshold is set to the gradient norm ϵ=10-6 to ensure a balance between optimization accuracy and computational efficiency.

[0024] The high-dimensional background field xb is the initial field at time t0=0 h, provided by the numerical weather prediction results of the target area in the previous 6 hours. The statistical characteristics and specific values ​​of its variables are as follows: The average value of the temperature variable (T) is 18.5℃, the standard deviation of the grid is 2.3℃, and the physical value ranges from -5.2℃ (northern mountainous area) to 32.8℃ (southern plain area). The grid mean for relative humidity (RH) was 65.2%, the grid standard deviation was 12.1%, and the physical values ​​ranged from 23.5% (western arid region) to 98.7% (eastern coastal region). The average value of the horizontal wind speed variable (U) is 4.8 m / s, the standard deviation of the grid is 1.5 m / s, and the physical value ranges from 0.2 m / s (calm wind zone inland within the region) to 15.3 m / s (strong wind zone in the eastern coastal area of ​​the region). The vector form of the high-dimensional background field xb has a total dimension of 30000. The first 10000 elements correspond to the temperature values ​​of each grid, the 10001st to 20000th elements correspond to the relative humidity values ​​of each grid, and the 20001st to 30000th elements correspond to the horizontal wind speed values ​​of each grid. The first 5 elements (temperature values) are 18.2℃, 18.5℃, 17.9℃, 19.1℃, and 17.8℃, respectively; the 10001st to 10003rd elements (relative humidity values) are 62.3%, 65.1%, and 60.7%, respectively; and the 20001st to 20003rd elements (horizontal wind speed values) are 4.5m / s, 4.7m / s, and 5.1m / s, respectively.

[0025] The observation data comes from ground-based automatic weather stations and satellite remote sensing inversion within the target area. There are 500 ground-based automatic weather stations (covering major towns and key topographical areas) and 100 satellite remote sensing observation points (covering blank areas). Therefore, the observation vector dimension at each observation time is 600. Specific information about the observation data at each time point is as follows: The observation data yobs0 at t0=0 h includes temperature, relative humidity, and horizontal wind speed observations from ground automatic stations and temperature observations retrieved from satellite remote sensing. The standard deviation of the temperature observation error is 0.8℃, the standard deviation of the relative humidity observation error is 8%, and the standard deviation of the horizontal wind speed observation error is 0.5m / s. The first five elements of this observation vector (ground station temperature observations) are 18.0℃, 18.3℃, 17.7℃, 18.9℃, and 17.6℃, respectively, and the 501st to 503rd elements (satellite temperature retrieval values) are 17.5℃, 18.8℃, and 19.2℃, respectively. The observation data yobs1 for t1=3 h includes temperature, relative humidity, and horizontal wind speed observations from ground automatic weather stations and relative humidity observations retrieved from satellite remote sensing. The standard deviation of the temperature observation error is 0.7℃, the standard deviation of the relative humidity observation error is 7%, and the standard deviation of the horizontal wind speed observation error is 0.4m / s. The first 5 elements of this observation vector (ground station temperature observations) are 19.1℃, 19.4℃, 18.8℃, 20.0℃, and 18.7℃, respectively, and the 501st to 503rd elements (satellite humidity retrieval values) are 63.2%, 66.5%, and 61.8%, respectively. The observation data yobs2 for t2=6 h includes temperature, relative humidity, and horizontal wind speed observations from ground automatic stations and horizontal wind speed observations retrieved from satellite remote sensing. The standard deviation of the temperature observation error is 0.9℃, the standard deviation of the relative humidity observation error is 9%, and the standard deviation of the horizontal wind speed observation error is 0.6m / s. The first 5 elements of this observation vector (ground station temperature observations) are 20.2℃, 20.5℃, 19.9℃, 21.1℃, and 19.8℃, respectively, and the 501st to 503rd elements (satellite wind speed retrieval values) are 5.2m / s, 5.5m / s, and 4.9m / s, respectively.

[0026] For each observation time ti (i=0,1,2), construct the corresponding observation error covariance matrix Ri. This matrix is ​​a 600×600 diagonal matrix, and the elements on the diagonal are the squares of the errors of the corresponding observation variables, as follows: The elemental value of the ground station temperature observation corresponding to the diagonal of R0 (t0=0 h) is 0.82 = 0.64 (°C). 2 The corresponding element value for relative humidity observation at the ground station is 82 = 64 (%) 2 The corresponding element value for the horizontal wind speed observed at the ground station is 0.52 = 0.25 (m / s). 2 The corresponding element value from satellite temperature inversion observations is 0.82 = 0.64 (°C). 2 ); The elemental value of the temperature observation at the ground station corresponding to the diagonal of R1 (t1=3 h) is 0.72 = 0.49 (°C). 2 The corresponding element value for relative humidity observation at the ground station is 72 = 49 (%) 2 The corresponding element value for the horizontal wind speed observed at the ground station is 0.42 = 0.16 (m / s). 2 The corresponding element value from satellite humidity inversion observations is 72 = 49 (%) 2 ); The elemental value of the temperature observation at the ground station corresponding to the diagonal of R2 (t2=6 h) is 0.92 = 0.81 (°C). 2 The element value corresponding to the relative humidity observed at the ground station is 92 = 81 (%) 2 The corresponding element value for the horizontal wind speed observed at the ground station is 0.62 = 0.36 (m / s). 2 The corresponding element value from satellite wind speed inversion observations is 0.62 = 0.36 (m / s). 2 ).

[0027] S2 Latent Space Projection Step: Using the encoder of a pre-trained deep autoencoder, the background field is projected onto a low-dimensional latent space to obtain background latent variables. The deep autoencoder also includes a decoder for reconstructing the latent variables into high-dimensional states. like Figure 2 As shown, this embodiment uses a convolutional autoencoder as a mapping tool between high-dimensional states and low-dimensional latent spaces. This autoencoder has been pre-trained using reanalysis data of the target region over the past 10 years (a total of 3650 sets of high-dimensional state data). Its structure and performance parameters are as follows: The encoder E consists of 3 convolutional layers and 2 fully connected layers. The convolutional kernel sizes of the convolutional layers are 3×3, 3×3, and 5×5, respectively, and the stride is 2. The number of neurons in the fully connected layers is 1000 and 500, respectively. This encoder can map a 30000-dimensional high-dimensional state vector x to a 500-dimensional latent variable z, i.e., z=E(x). The decoder D consists of two fully connected layers and three transposed convolutional layers. The number of neurons in the fully connected layers is 1000 and 10000 respectively, and the kernel size of the transposed convolutional layers is 5×5, 3×3 and 3×3 respectively, with a stride of 2 for each layer. This decoder can reconstruct the 500-dimensional latent variable z into a 30000-dimensional high-dimensional state vector x′, i.e., x′=D(z). After the autoencoder performance training is completed, the reconstruction error of the validation set data meets the following requirements: the average reconstruction error of temperature variable ≤ 0.5℃, the average reconstruction error of relative humidity variable ≤ 5%, and the average reconstruction error of horizontal wind speed variable ≤ 0.3m / s, ensuring that the core dynamic information of the high-dimensional state is effectively preserved in the latent space.

[0028] The high-dimensional background field xb is input into the pre-trained encoder E. Through convolution and fully connected operations of the encoder, the corresponding background field latent variable zb (dimension 500) is obtained. The specific values ​​of the first 10 elements are 2.15, -1.32, 0.87, -0.56, 1.23, -0.91, 0.45, -1.18, 0.72, and -0.34, respectively. The subsequent elements are all real numbers that conform to the latent space distribution characteristics, with values ​​ranging from -3.5 to 3.8.

[0029] S3 Superset Generation Steps: Using a pre-trained deep generative model, generate multiple superset members, and project each superset member onto the low-dimensional latent space through the encoder to obtain the corresponding set latent variables; This embodiment uses a pre-trained variational autoencoder (VAE) as the deep generative model. This model has been trained using historical forecast data of the target region over the past 5 years (a total of 1825 sets of high-dimensional state data). It can quickly generate physically consistent superset members. The specific generation process is as follows: 1000 500-dimensional latent space perturbation vectors are randomly sampled from the standard normal distribution N(0,I), denoted as {z(1),z(2),...,z(1000)}, where the elements of each perturbation vector follow a normal distribution with a mean of 0 and a standard deviation of 1; The above 1000 latent space perturbation vectors are input into the pre-trained decoder D. Through the fully connected and transposed convolution operations of the decoder, 1000 members of a 30000-dimensional high-dimensional super set are obtained, denoted as {x(1),x(2),...,x(1000)}. The physical consistency verification of the superset members shows that among the 1000 generated set members, the temperature variable ranges from -6.0℃ to 33.5℃, the relative humidity variable ranges from 20.0% to 100.0%, and the horizontal wind speed variable ranges from 0.1m / s to 16.0m / s, which conforms to the laws of atmospheric physics. Furthermore, the average standard deviation of temperature among the set members is 2.4℃, the average standard deviation of relative humidity is 12.3%, and the average standard deviation of horizontal wind speed is 1.6m / s, which is consistent with the statistical characteristics of the background field, ensuring the rationality of the disturbance.

[0030] The 1000 high-dimensional super set members {x(1),x(2),...,x(1000)} generated in S3 are input into encoder E respectively. The encoder calculates the corresponding 1000 500-dimensional set latent variables, denoted as {z(1),z(2),...,z(1000)}. The first 5 elements of the first set latent variable z(1) are 2.13, -1.34, 0.89, -0.57, and 1.22, respectively. The first 5 elements of the 1000th set latent variable z(1000) are 2.10, -1.36, 0.88, -0.59, and 1.20, respectively. The value range of all set latent variables is between -3.6 and 3.9, which is consistent with the distribution characteristics of the background field latent variable zb.

[0031] According to the formula:

[0032] The computational flow depends on the background error covariance matrix, where N=1000 is the size of the superset. The average vector of the set's latent variables is shown below. The specific calculation process and results are as follows. The ensemble mean vector zˉ is calculated by averaging over each dimension of the 1000 ensemble latent variables {z(1),...,z(1000)}, resulting in a 500-dimensional ensemble mean vector. The first five elements are 2.12, -1.35, 0.89, -0.58, and 1.21, respectively, and the subsequent elements are the set mean of the corresponding dimension. Deviation vector computation: For each set of latent variables z(i), calculate its deviation vector relative to the set mean vector. deviation - This yields 1000 500-dimensional bias vectors; like Figure 3 As shown, the flow-dependent covariance matrix is ​​constructed by multiplying each bias vector by its transpose, resulting in 1000 500×500 matrices. Summing these matrices and dividing by N-1=999 yields the flow-dependent background error covariance matrix. (500×500); The mean of the diagonal elements (the error variance of the latent variables themselves) of this matrix is ​​1.8, and the standard deviation is 0.5. The correlation coefficients of the off-diagonal elements (the error covariance between latent variables) range from -0.3 to 0.6, reflecting the dynamic correlation characteristics between latent variables.

[0033] S4 Flow Dependency Covariance Construction Steps: Based on the set of latent variables, calculate the background error covariance matrix of the flow dependency in the low-dimensional latent space; Static statistical background error covariance matrix The data was obtained through statistical analysis of historical latent variable sequences (3650 latent variable samples) from the same period over the past 10 years in the target region. Specifically, the covariance of the error bias of the historical latent variable sequences was calculated to obtain a 500×500 static covariance matrix. The mean of the diagonal elements of this matrix is ​​2.2, and the standard deviation is 0.6, slightly greater than... The mean of the diagonal elements reflects the stable error characteristics in a climatological sense; the correlation coefficients of the off-diagonal elements range from -0.2 to 0.4, and the spatial correlation is relatively weaker than that of the mean of the diagonal elements. It is more conservative, ensuring the stability of static covariance.

[0034] According to the formula Calculate the mixed background error covariance matrix, where α = 0.3 is the mixing coefficient. The specific calculation results are as follows. The value of each diagonal element is α multiplied by α. The diagonal elements are multiplied by (1-α) The sum of the diagonal elements has a mean of 0.3×2.2+0.7×1.8=1.92 and a standard deviation of 0.3×0.6+0.7×0.5=0.53, which takes into account the characteristics of both steady-state error and flow-dependent error. The value of each off-diagonal element is α multiplied by... Corresponding off-diagonal elements multiplied by (1-α) The correlation coefficient for the sum of the corresponding off-diagonal elements ranges from 0.3×[-0.2,0.4]+0.7×[-0.3,0.6]=[-0.27,0.54], which retains the conservative correlation of static covariance while incorporating the dynamic correlation characteristics of flow-dependent covariance.

[0035] S5 Objective Function Construction Steps: Construct a four-dimensional variational assimilation objective function within the low-dimensional latent space. This embodiment uses a Long Short-Term Memory (LSTM) network as the latent space dynamics model Mi. This model has been trained using the latent variable evolution sequence of the target region over the past 5 years (a total of 1825 sets of latent variable time series data). It can predict the evolution of latent variable z from time t0 to time ti, i.e., zi = Mi(z). Here, M0 is the unit mapping (itself at time t0), M1 corresponds to the evolution from t0 to t1 = 3 h, and M2 corresponds to the evolution from t0 to t2 = 6 h. The prediction error of the model for the evolution of latent variables meets the following requirements: the prediction error of temperature-related latent variables ≤ 0.2, the prediction error of humidity-related latent variables ≤ 0.3, and the prediction error of wind speed-related latent variables ≤ 0.25, ensuring the accuracy of the evolution process.

[0036] Observation operator It is used to map high-dimensional states to the observation space. Its core function is to extract the variable values ​​corresponding to the observation points in the high-dimensional state, as detailed below. For time ti, Receive the high-dimensional state D output by decoder D ( (z)), based on the grid location of the observation point, extract the temperature, relative humidity, or horizontal wind speed values ​​of the corresponding grid to form a simulated observation vector with the same yobsi dimension (600 dimensions) as the observation data, i.e., Hi(D( (z))); For example, for the observation operator H0 at t0=0 h, it extracts the temperature, relative humidity, and horizontal wind speed values ​​of 500 ground station grids and the temperature values ​​of 100 satellite inversion grids from D(M0(z)), totaling 600 elements, to form the simulated observation vector.

[0037] Construct the objective function J(z) for the latent space four-dimensional variational assimilation based on the formula:

[0038] The first term is the background term, representing the deviation between the latent variable z and the background field latent variable zb, and its weight is determined by the inverse of the mixture covariance matrix (Bzhybrid)⁻¹. The second term is the observation term, representing the deviation between simulated and actual observations, and its weight is determined by the inverse of the observation error covariance matrix Ri⁻¹. The function is represented by the composite symbol "". "Indicates execution sequentially" (Hidden Space Evolution), D (High-Dimensional Reconstruction) (Observation extraction) calculation.

[0039] S6 Optimization Solution Steps: Within the low-dimensional latent space, the optimal analytical latent variables are obtained through an optimization algorithm. like Figure 4As shown, the objective function J(z) is minimized using the quasi-Newton method (L-BFGS). The initial iteration point is set as the hidden variable zb of the background field. The key data of the iteration process are as follows. The objective function value of the 0th iteration is J(z0) = 1.2 × 10⁴, and the gradient norm ||J(z0)|| = 15.6, which does not meet the convergence threshold. In the 10th iteration, the objective function value J(z10) = 3.5 × 10³, and the gradient norm ||J(z10)|| = 4.2. The objective function value decreases significantly, and the gradient norm shrinks. The objective function value of the 35th iteration is J(z35) = 8.2 × 102, and the gradient norm |J(z35)| = 0.9, which is close to the convergence threshold. The objective function value of the 52nd iteration is J(z52) = 4.1 × 102, and the gradient norm ||J(z52)|| = 9.8 × 10-7, which satisfies the convergence threshold ϵ = 10-6, and the iteration terminates. The latent variable obtained by the 52nd iteration of the optimal analysis latent variable za is the optimal analysis latent variable. Its first 10 elements are 2.14, -1.33, 0.88, -0.57, 1.22, -0.90, 0.46, -1.17, 0.73, and -0.33, respectively. The deviation from the background field latent variable zb is small, which meets the optimization convergence characteristics.

[0040] S7 State Reconstruction Step: Using the decoder, the optimal analytical latent variables are reconstructed back to the original high-dimensional state space to obtain the final analytical field.

[0041] The optimal latent variable za is input into the decoder D. Through the fully connected and transposed convolution operations of the decoder, a 30,000-dimensional high-dimensional analysis field xa is reconstructed, i.e., xa = D(za). The first 5 elements (temperature values) of this analysis field are 18.1℃, 18.4℃, 17.8℃, 19.0℃, and 17.7℃, respectively. The 10,001st to 10,003rd elements (relative humidity values) are 62.1%, 65.0%, and 60.5%, respectively. The 20,001st to 20,003rd elements (horizontal wind speed values) are 4.6m / s, 4.8m / s, and 5.0m / s, respectively. The numerical deviation from the background field xb is reasonable, reflecting the correction of the background field by assimilation.

[0042] The assimilation effect was verified by analyzing the goodness of fit between the field and the observed data. Specific indicators are as follows: The root mean square error (RMSE) between the background temperature field and the observed data was 1.8℃, while the RMSE between the analytical field and the observed data decreased to 0.9℃, improving the goodness of fit by 50%. The root mean square error between the background field and the observed data for relative humidity was 10.2%, while the root mean square error between the analytical field and the observed data decreased to 5.1%, resulting in a 50% improvement in the goodness of fit. The root mean square error between the background field and the observed data for horizontal wind speed was 1.2 m / s, while the root mean square error between the analyzed field and the observed data was reduced to 0.6 m / s, with a 50% improvement in fit. The validation results show that this method effectively reduces the cost of high-dimensional computation and improves the assimilation accuracy by using latent space projection and super ensemble techniques, thus meeting the initial field quality requirements for regional scale numerical weather prediction.

[0043] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A four-dimensional variational assimilation method based on latent space projection and supersets, characterized in that, Includes the following steps: S1 data acquisition steps: Obtain the background field of the high-dimensional numerical model, and the observation data sequence within the assimilation time window; S2 Latent Space Projection Step: Using the encoder of a pre-trained deep autoencoder, the background field is projected onto a low-dimensional latent space to obtain background latent variables. The deep autoencoder also includes a decoder for reconstructing the latent variables into high-dimensional states. S3 Superset Generation Steps: Using a pre-trained deep generative model, generate multiple superset members, and project each superset member onto the low-dimensional latent space through the encoder to obtain the corresponding set latent variables; S4 Flow Dependency Covariance Construction Steps: Based on the set of latent variables, calculate the background error covariance matrix of the flow dependency in the low-dimensional latent space; S5 Objective Function Construction Steps: Construct a four-dimensional variational assimilation objective function within the low-dimensional latent space; S6 Optimization Solution Steps: Within the low-dimensional latent space, the optimal latent variables for analysis are obtained through optimization algorithms; S7 State Reconstruction Step: Using the decoder, the optimal analytical latent variables are reconstructed back to the original high-dimensional state space to obtain the final analytical field; In step S4, a static statistical background error covariance matrix is ​​constructed: based on historical latent variable samples from the same period of the past 10 years in the target region, the covariance of the historical latent variable sequence error bias is calculated to obtain the static covariance matrix. The mixed background error covariance matrix is ​​calculated, and the corresponding elements of the static covariance matrix and the flow-dependent background error covariance matrix are weighted and summed. In step S5, a long short-term memory network is used as the latent space dynamics model. The model prediction error satisfies that the latent variable associated with temperature is ≤0.2, the latent variable associated with humidity is ≤0.3, and the latent variable associated with wind speed is ≤0.

25. The observation operator Hi is used to map the high-dimensional state to the observation space.

2. The four-dimensional variational assimilation method based on latent space projection and supersets according to claim 1, characterized in that, The deep generative model mentioned in step S3 is a generative adversarial network, variational autoencoder, diffusion model, or normalized flow model, which is trained using historical forecast data or reanalysis data.

3. The four-dimensional variational assimilation method based on latent space projection and supersets according to claim 1, characterized in that, The generation of the superset members in step S3 can be performed in any of the following ways: (1) Based on initial condition perturbation; (2) Based on perturbation of physical process parameters in the model; (3) Or based on a hybrid perturbation strategy that simultaneously considers initial conditions, boundary conditions and model parameters.

4. The four-dimensional variational assimilation method based on latent space projection and supersets according to claim 1, characterized in that, The dynamic model of the latent space described in step S5 is obtained through any of the following methods: Method 1: In the latent space, the historical sequence of the high-dimensional model state is projected to obtain the sequence of latent variables, and a recurrent neural network, long short-term memory network, gated recurrent unit, or neural network with regular differential equations is trained based on this sequence as an alternative model; Method 2: The dynamic model ,in This is the original high-dimensional numerical model.

5. The four-dimensional variational assimilation method based on latent space projection and supersets according to claim 1, characterized in that, The depth autoencoder is a variational autoencoder, a convolutional autoencoder, or a physically constrained autoencoder.

6. The four-dimensional variational assimilation method based on latent space projection and supersets according to claim 1, characterized in that, The static statistical background error covariance matrix of the latent space mentioned in step S6 is obtained by statistical analysis of the historical latent variable sequence and is simplified to a diagonal matrix or identity matrix. The optimization algorithm is a quasi-Newton method, a conjugate gradient method, or a gradient-based stochastic optimization algorithm.

7. The four-dimensional variational assimilation method based on latent space projection and supersets according to claim 1, characterized in that, In step S5, the objective function J(z) for four-dimensional variational assimilation is constructed, and its expression is: Where z is the control variable to be optimized. Let be the static statistical background error covariance matrix of the latent space, and α ∈ [0, 1] be the mixing coefficients. To characterize the latent space dynamics model that represents the evolution of latent variable z from initial time t0 to latent variable zi at time ti. Let be the observation operator at time ti, with the symbol . Indicates function composition. Let be the observation error covariance matrix at time ti.

8. The four-dimensional variational assimilation method based on latent space projection and supersets according to claim 7, characterized in that, The mixing coefficient is determined in the following ways: by determining a fixed value based on the statistical optimality of historical assimilation experiments; by dynamically adjusting it based on the statistical relationship between ensemble dispersion and forecast error; or by adaptively adjusting it based on machine learning models according to the current flow pattern characteristics.

9. A system for implementing the four-dimensional variational assimilation method based on latent space projection and supersets as described in any one of claims 1-8, characterized in that, include: The data acquisition module is used to acquire high-dimensional background fields and observation time series data; The deep generative model module is used to quickly generate members of a super set; The autoencoder module stores pre-trained deep autoencoders for bidirectional mapping between high-dimensional states and low-dimensional latent variables. The flow-dependent covariance calculation module constructs the flow-dependent background error covariance in the latent space based on a superset. The latent space assimilation engine is used to construct a hybrid four-dimensional variational assimilation objective function in the latent space and perform optimization to obtain the optimal latent variables for analysis. The state reconstruction module is used to decode the optimal analysis latent variables into the final high-dimensional analysis field.

Images