Data assimilation method and device, equipment and medium

Abstract

The invention relates to the technical field of data assimilation, and discloses a data assimilation method, device, equipment and medium, the method comprises the following steps: firstly obtaining observation data and meshing, the observation data comprising at least one variable; or, a background field of at least one time and corresponding observation data are obtained, the observation data are gridded, and the background field and the observation data both comprise at least one variable; next, the constructed data assimilation model is trained by using the grids corresponding to all the variables to obtain a trained data assimilation model; and inputting the to-be-processed gridding observation data or the to-be-processed background field and the gridding observation data into the trained data assimilation model, and outputting a corresponding analysis field. The data assimilation method, device, equipment and medium provided by the invention have the characteristics of high reasoning speed, high assimilation precision and the like, and are suitable for data assimilation in the fields of weather climate, ocean, land surface and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of data assimilation, and in particular to a data assimilation method, device, equipment and medium. Background Art

[0002] With the development of meteorology, dynamic meteorology, earth observation technology (especially satellite remote sensing), information and communication technology, data assimilation technology, etc., numerical weather forecasting has made great progress and has become an important basis for daily weather forecasting. Numerical weather forecasting is an initial (boundary) value problem. It predicts the evolution of the atmosphere through the most accurate initial field and the most perfect numerical model that describes the laws of atmospheric motion. It can be seen that the accuracy of numerical weather forecasting is mainly limited by the initial field and the numerical model. However, with the continuous improvement of numerical models, the accuracy of the initial field has become a bottleneck for further improving the level of numerical weather forecasting. In addition, weather forecasting based on large models also requires the most accurate initial field.

[0003] Data assimilation (DA) technology is a data fusion technology that combines observation data, short-term forecast data, and corresponding error information to provide the most accurate initial field for the numerical model. In order to continuously improve the level of numerical weather forecasting, scientists have successively developed data assimilation technologies such as step-by-step correction method, optimal interpolation method, three-dimensional and four-dimensional variational method, Kalman filter, ensemble Kalman filter, ensemble-variational method, hierarchical Bayesian method, particle filter algorithm, etc. In particular, three-dimensional and four-dimensional variational methods have become the mainstream data assimilation technology for business operations. However, the implementation of the above methods usually requires a lot of professional knowledge and assumptions, and there are problems such as slow processing speed and low assimilation accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a data assimilation method, device, equipment and medium, which can improve the reasoning speed and enhance the assimilation accuracy.

[0005] In order to solve the above technical problems, the present invention provides a data assimilation method, which comprises:

[0006] Obtain observation data and grid it; the observation data contains at least one variable;

[0007] Or, obtaining a background field and corresponding observation data of at least one time, and gridding the observation data to generate a grid consistent with the background field; wherein the background field and the observation data each contain at least one variable;

[0008] The constructed data assimilation model is trained using the grids corresponding to all variables to obtain a trained data assimilation model;

[0009] The gridded observation data to be processed or the background field to be processed and the gridded observation data are input into the trained data assimilation model, and the corresponding analysis field is output.

[0010] In a first aspect, in the above-mentioned data assimilation method provided by the present invention, the background field includes at least one variable among a plurality of ground layer variables, a plurality of atmospheric variables at a plurality of height layers, a plurality of ocean variables at a plurality of depth layers, and a plurality of soil variables at a plurality of depth layers;

[0011] Optionally, the observation data include infrared radiation brightness temperature observed by satellite, microwave radiation brightness temperature observed by satellite, infrared emissivity observed by satellite, microwave emissivity observed by satellite, wind field, ozone concentration and snow depth inverted by satellite remote sensing, navigation satellite radio occultation data, satellite scatterometer data, satellite altimeter data, ground station observation data, buoy station observation data, ship observation data, wind profile observed by wind profiler radar, radiosonde observation data, aviation observation data, weather radar estimated precipitation intensity, millimeter wave radar detection data, microwave link data or a combination thereof.

[0012] On the other hand, in the data assimilation method provided by the present invention, before the constructed data assimilation model is trained using the grids corresponding to all variables, the method further includes:

[0013] An image-to-image transformation type machine learning model or a sequence-to-sequence transformation type machine learning model is constructed as a data assimilation model; the data assimilation model includes an encoder and / or a decoder.

[0014] On the other hand, in the data assimilation method provided by the present invention, in the process of training the constructed data assimilation model, the process includes:

[0015] Divide the grid of all variables into non-overlapping patches;

[0016] Input the divided patches and their corresponding variable identities and spatiotemporal positions into the encoder;

[0017] Using the patch embedding module of the encoder, each patch corresponding to each variable is projected into a multidimensional feature vector, and the variable identity and spatiotemporal position of each patch are embedded into the corresponding multidimensional feature vector to obtain a token of the patch;

[0018] An encoder or a decoder is used to predict a patch of at least one time analysis field that has been pre-divided to obtain at least one time analysis field.

[0019] On the other hand, in the data assimilation method provided by the present invention, in the process of projecting each patch corresponding to each variable into a multidimensional feature vector, the process includes:

[0020] For each variable, when some grid points in the patch are null values, a learnable, shared scalar is assigned to each null value position;

[0021] Project the scalared patch into a multidimensional feature vector.

[0022] On the other hand, in the data assimilation method provided by the present invention, the constructed data assimilation model is trained using grids corresponding to all variables, including:

[0023] The historical analysis field or reanalysis field is used as the actual situation and combined with the grids corresponding to all variables to train the constructed data assimilation model;

[0024] Or, the historical analysis field or reanalysis field is used as the actual situation to train the constructed data assimilation model in combination with the grids corresponding to all variables, and the data assimilation model is fine-tuned using high-precision observation data;

[0025] Alternatively, during the training process, the output of the data assimilation model is interpolated to the high-precision observation data, and then the loss function is calculated to complete the training of the data assimilation model.

[0026] On the other hand, in the data assimilation method provided by the present invention, after only acquiring the observation data and before gridding the observation data, it also includes:

[0027] Conduct quality control on acquired observation data;

[0028] Normalize or standardize the observed data after quality control;

[0029] After obtaining at least one time period of background field and corresponding observation data, and before gridding the observation data, the method further includes:

[0030] Perform quality control on acquired background fields and / or observation data;

[0031] Normalize or standardize the background field and / or observation data after quality control.

[0032] In order to solve the above technical problems, the present invention further provides a data assimilation device, the device comprising:

[0033] A data acquisition module is used to acquire and grid observation data; the observation data includes at least one variable; it is also used to acquire at least one time background field and corresponding observation data, and grid the observation data to generate a grid consistent with the background field; wherein the background field and the observation data each include at least one variable;

[0034] The model training module is used to train the constructed data assimilation model using the grids corresponding to all variables to obtain a trained data assimilation model;

[0035] The model inference module is used to input the gridded observation data to be processed or the background field to be processed and the gridded observation data into the trained data assimilation model, and output the corresponding analysis field.

[0036] In order to solve the above technical problems, the present invention further provides a data assimilation device, the device comprising:

[0037] Memory for storing computer programs;

[0038] A processor is used to implement the steps of the above-mentioned data assimilation method when executing the computer program.

[0039] In order to solve the above technical problem, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above data assimilation method are implemented.

[0040] It can be seen from the above technical scheme that a data assimilation method provided by the present invention includes: obtaining observation data and gridding the observation data, wherein the observation data includes at least one variable; or obtaining a background field and corresponding observation data of at least one time, and gridding the observation data to generate a grid consistent with the background field, wherein both the background field and the observation data include at least one variable; using the grids corresponding to all variables to train the constructed data assimilation model to obtain a trained data assimilation model; inputting the gridded observation data to be processed or the background field to be processed and the gridded observation data into the trained data assimilation model, and outputting the corresponding analysis field.

[0041] The beneficial effect of the present invention is that the above-mentioned data assimilation method provided by the present invention first obtains only the observation data and grids it, or obtains the background field and corresponding observation data of at least one time and grids the observation data to generate a grid consistent with the background field, and then uses the grids corresponding to all variables to train the data assimilation model, and then uses the trained data assimilation model to infer the analysis field corresponding to the gridded observation data to be processed or the analysis field corresponding to the background field to be processed and the gridded observation data. The whole reasoning process is fast, the assimilation accuracy is high, and the hardware processing speed is improved. The above-mentioned data assimilation method can be combined with traditional numerical weather forecasting or with weather forecasting based on large models, and can be used for data assimilation in the fields of weather climate, ocean, land surface, etc.

[0042] In addition, the present invention also provides a corresponding data assimilation device, a data assimilation equipment and a computer-readable storage medium for the data assimilation method, which have the same or corresponding technical features as the above-mentioned data assimilation method and have the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A flow chart of a data assimilation method provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of the structure of a data assimilation device provided by an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the structure of a data assimilation device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The core of the present invention is to provide a data assimilation method, device, equipment and medium to solve the technical problems that the implementation of traditional data assimilation methods usually requires a lot of professional knowledge and assumptions, and has slow processing speed and low assimilation accuracy.

[0049] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 A flow chart of a data assimilation method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0050] S101, obtaining observation data; the observation data includes at least one variable;

[0051] S102, gridding the observation data;

[0052] It should be noted that the above steps S101 and S102 are to only obtain the observation data and grid the observation data, which is the first implementation method. The observation data corresponding to the same physical quantity at different levels and different times in the observation data can be considered as different variables. Next, steps S105 and S106 can be executed.

[0053] S103, obtaining a background field and corresponding observation data of at least one time period; wherein the background field and the observation data both contain at least one variable;

[0054] S104, gridding the observed data to generate a grid consistent with the background field.

[0055] It should be noted that the above steps S103 and S104 are to obtain the background field and corresponding observation data of at least one time and grid the observation data, wherein the background field and the observation data both contain at least one variable, which is a second implementation method. The variables corresponding to the same physical quantities at different levels and different times in the background field of at least one time can be considered as different variables. The observation data corresponding to the same physical quantities at different levels and different times in the observation data can be considered as different variables. Steps S105 and S106 can be executed next.

[0056] In a specific implementation, in the above data assimilation method provided in the embodiment of the present invention, after executing step S101 to obtain the observation data, before executing step S102 to grid the observation data, it may also include: first, quality control of the obtained observation data; then, normalization or standardization of the observation data after quality control. In implementation, the Min-Max method may be specifically used to normalize the observation data after quality control.

[0057] In addition, after executing step S103 to obtain at least one time of background field and corresponding observation data, before executing step S104 to grid the observation data, it may also include: first, quality control of the obtained background field and / or observation data; then, normalization or standardization of the background field and / or observation data after quality control. In implementation, the Min-Max method may be specifically used to normalize the background field and / or observation data after quality control.

[0058] When executing step S102 and step S104, the following method can be used for gridding the observation data: the value of each grid point is a statistical value of the observation data in a certain spatiotemporal neighborhood, such as the average value, median value, maximum value, minimum value, etc., which can also be obtained by interpolation. The present invention can unify the observation data into a grid consistent with the background field (such as a latitude and longitude grid with a resolution of 0.25°×0.25°), and can also unify the background field and the observation data into a grid of other required resolutions.

[0059] S105. Train the constructed data assimilation model using the grids corresponding to all variables to obtain a trained data assimilation model.

[0060] It should be noted that the data assimilation model is a model that improves the accuracy of the analysis field by fusing data from multiple sources. The data assimilation model of the present invention can be used in fields such as weather forecasting.

[0061] In specific implementation, the data assimilation model of the present invention can be trained using grids corresponding to all variables.

[0062] In executing step S105, the constructed data assimilation model is trained using the grids corresponding to all variables. The following three implementation methods can be used:

[0063] The first implementation method is to use the historical analysis field or reanalysis field (such as ERA5, CRA-40, GLORYS12) as the actual situation and combine it with the grids corresponding to all variables to train the constructed data assimilation model.

[0064] The historical analysis field can come from a numerical forecasting system or a multi-source data fusion analysis field in operational operation. The grids corresponding to all variables can be used as inputs to the data assimilation model, and the historical analysis field or the reanalysis field can be used as the actual situation (label). Specifically, the grids of all variables are input into the data assimilation model to obtain at least one time analysis field, and then the loss function is calculated.

[0065] The second implementation method is to first train the data assimilation model by combining the historical analysis field or reanalysis field with the grids corresponding to all variables as the actual situation, and then fine-tune the data assimilation model using high-precision observation data. During actual business operation, high-precision observation data (such as ground station observation data, buoy station observation data, ship observation data, radiosonde observation data, and aviation observation data) can be used from time to time to fine-tune the trained data assimilation model.

[0066] In the first implementation and the second implementation, when training the data assimilation model, patches in the historical analysis field or the reanalysis field may be normalized one by one to reduce the generalization error.

[0067] The third implementation method is: during the training process, the output of the data assimilation model is interpolated to high-precision observation data (such as ground station observation data, buoy station observation data, ship observation data, radiosonde observation data, and aviation observation data), and then the loss function is calculated to train the data assimilation model. Preferably, the mean square error loss function or the mean absolute error loss function can be used to train the data assimilation model. Optionally, when focusing on the analysis field of a specific variable in a specific area, in the loss function, the error evaluation index (such as absolute value of error, square of error) of the corresponding variable in the corresponding area is given a greater weight.

[0068] S106: input the gridded observation data to be processed or the background field to be processed and the gridded observation data into the trained data assimilation model, and output the corresponding analysis field.

[0069] Optionally, the output of the data assimilation model is denormalized to obtain the final required analysis field;

[0070] It should be noted that the data assimilation method of the present invention is an end-to-end data assimilation method based on machine learning.

[0071] In the above data assimilation method provided by the embodiment of the present invention, firstly, only the observation data is obtained and gridded, or, the background field and the corresponding observation data of at least one time are obtained and the observation data are gridded, so that the background field and the observation data are unified into a consistent grid, and then the grids corresponding to all variables are used to train the data assimilation model, and then the trained data assimilation model is used to infer the analysis field corresponding to the gridded observation data to be processed or the analysis field corresponding to the background field to be processed and the gridded observation data. The whole inference process is fast, the assimilation accuracy is high, and the hardware processing speed is improved. The above data assimilation method can be combined with traditional numerical weather forecasting or with weather forecasting based on large models, and can be used for data assimilation in the fields of weather, climate, ocean, land, etc.

[0072] Furthermore, in a specific implementation, in the above-mentioned data assimilation method provided in an embodiment of the present invention, the background field may include at least one variable among multiple ground layer variables, multiple altitude layer atmospheric variables, multiple depth layer ocean variables, and multiple depth layer soil variables.

[0073] In implementation, the background field may include multiple ground layer variables (such as 2-meter air temperature, 10-meter U wind component (east-west wind speed component), 10-meter V wind component (north-south wind speed component) and sea level pressure), multiple atmospheric variables at multiple altitude layers (such as potential, air temperature, U component of wind, V component of wind, relative humidity), multiple ocean variables at different depth layers (such as ocean temperature, salinity, latitudinal and longitudinal components of ocean currents), and multiple soil variables at different depth layers (such as soil temperature and water content).

[0074] The background field of at least one of the above time periods can come from the Global Medium-Term Numerical Forecast System (GRAPES_GFS) of the China Meteorological Administration, the Integrated Forecast System of the European Centre for Medium-Term Weather Forecasts (ECMWF IFS), the Guangdong Rapid Update Assimilation Numerical Forecast System (CMA-GD) of the China Meteorological Administration, the Mesoscale Weather Numerical Forecast System (CMA-MESO) of the China Meteorological Administration, the Shanghai Numerical Forecast Model System (CMA-SH9) of the China Meteorological Administration, the Hybrid Coordinate Ocean Model (HYCOM), the Mercator Ocean Physical System (PSY4), the Real-Time Ocean Forecast System (RTOFS), the Nucleus for European Modelling of the Ocean (NEMO), the Global Ice Ocean Prediction System (GOS), the Global Ocean Circulation Model (NEMO), the Global Ice Ocean Prediction System (GOS), the Global Ocean Circulation Model (NEMO), the Global Ocean Circulation Model (GOS ... The short-term forecasts can also come from the numerical models such as Pangu Weather Forecast Model (Pangu-Weather), Fengwu Weather Forecast Model (FengWu), Fuxi Weather Forecast Model (FuXi), FourCastNet Weather Forecast Model, Ocean Forecast Model (XiHe, AI-GOMS) and other large models.

[0075] The above-mentioned observation data may include infrared radiation brightness temperature observed by satellite, microwave radiation brightness temperature observed by satellite, infrared radiance observed by satellite, microwave radiance observed by satellite, wind field, ozone concentration and snow depth inverted by satellite remote sensing, radio occultation data of navigation satellites (such as GPS, Beidou, etc.) (such as Bending angle), satellite scatterometer data (such as Backscatter sigma0, soil moisture, wind field), satellite altimeter data (sea surface height), ground station observation data (such as 2-meter temperature, relative humidity, air pressure, snow depth), buoy station observation data (such as wind field, air pressure), ship observation data (such as air pressure, temperature, relative humidity, wind field), wind profile observed by wind profiler radar, radiosonde observation data (such as temperature, relative humidity and wind profile), aviation observation data (such as temperature, relative humidity, wind field), precipitation intensity estimated by weather radar, millimeter wave radar detection data (such as attenuation coefficient detected by vehicle-mounted and intelligent transportation millimeter wave radar), microwave link data (such as attenuation coefficient field of microwave link of mobile communication network), etc.

[0076] Furthermore, in a specific implementation, in the above-mentioned data assimilation method provided in an embodiment of the present invention, before executing step S105 to train the constructed data assimilation model using the grids corresponding to all variables, it may also include: using an image-to-image transformation type machine learning model or a sequence-to-sequence transformation type machine learning model to construct a data assimilation model; the data assimilation model includes an encoder and / or a decoder.

[0077] In implementation, the data assimilation model may adopt a machine learning model of image-to-image or sequence-to-sequence transformation type. Specifically, the data assimilation model may include an encoder and / or a decoder.

[0078] Optionally, the encoder may include a patch embedding module, a positional embedding module, a multi-head self-attention module, a cross attention module, a Swin Transformer module, a state space model, a Mamba module, a visual state space (Visual State Space, VSS) module, a vision mamba module (vision mamba block), a U-Mamba module, a Plain Mamba module, an Efficient Visual State Space (Efficient Visual State Space, EVSS) module, a LocalMamba module, a Mamba-ND module, an Arbitrary-Masked State Space (Arbitrary-Masked State Space, AMSS) module, a Temporal Mamba module, a ParallelMamba module, a convolutional layer, a pooling layer, a patch merging (Patch Merging) module, a downsampling module, a residual module, a layer normalization (Layer Normalization) module, a multi-layer perceptron, a linear projection (Linear Projection) module, etc. The combination of several functional modules.

[0079] Optionally, the encoder may adopt a vision transformer (ViT) based encoder.

[0080] Optionally, the above-mentioned decoder may include a combination of several functional modules including a linear projection module, a multi-head self-attention module, a multi-head cross-attention module, a visual state space (VSS) module, a Mamba module, a U-Mamba module, an Arbitrary-Masked State Space (AMSS) module, a deconvolution layer, a patch expansion module, an upsampling module, a residual module, a layer normalization module, a multi-layer perceptron, etc.

[0081] Optionally, the above decoder may adopt a decoder based on a vision transformer (ViT).

[0082] The specific implementation of each module or component of the encoder and decoder can refer to the prior art;

[0083] Optionally, the data assimilation model can adopt a CNN-based machine learning model (such as U-Net), a transformer-based machine learning model, a vision transformer (ViT)-based machine learning model, a U-Transformer-based machine learning model, a Visual State Space Model (VMamba)-based machine learning model, a vision mamba (Vim)-based machine learning model, a Swin Transformer-based machine learning model, a State Space Model-based machine learning model, a Masked Autoencoders (MAE)-based machine learning model, or a Cross-Attention Masked One or a combination of the following: machine model of Autoencoders (CrossMAE), machine learning model based on U-Mamba, machine learning model based on VM-UNet, machine learning model based on Swin-UMamba, machine learning model based on Mamba-UNet, machine learning model based on LightM-UNet, machine learning model based on VideoMamba, machine learning model based on MambaMIR, machine learning model based on SiMBA, machine learning model based on Vivim, machine learning model based on nnMamba, machine learning model based on Mamba-2, machine learning model based on SimMIM, etc.

[0084] Furthermore, in a specific implementation, the data assimilation method provided in the embodiment of the present invention may further include: allocating an identity document (ID) to each variable.

[0085] Correspondingly, in the process of executing step S105 to train the constructed data assimilation model, it can specifically include: dividing the grids of all variables into non-overlapping patches; inputting the divided patches and their corresponding variable identities, and spatiotemporal positions into the encoder; using the encoder's patch embedding module to project each patch corresponding to each variable into a multidimensional feature vector, and at the same time embedding the variable identity and spatiotemporal position of each patch into the corresponding multidimensional feature vector to obtain a patch token; using the encoder or decoder to predict the patch of at least one pre-divided time analysis field to obtain at least one time analysis field.

[0086] In implementation, the grids of all variables can be uniformly divided into non-overlapping patches of size h×w, and the height h and width w can take the same value. Then, all standard patches of all variables and their corresponding variable IDs, spatiotemporal positions can be input into the encoder, or, all standard patches of all variables and their corresponding variable IDs or spatiotemporal positions can be input into the encoder. The standard patches here can be all patches, patches where each grid point is not null, and patches where all grid point values ​​are not all null.

[0087] Afterwards, the encoder patch embedding module projects each patch of each variable into a multi-dimensional feature vector as a token of the patch. Specifically, the patch embedding module can include a patch embedding submodule for each variable; the patch embedding submodule for each variable can be a linear projection module or an independent patch embedding neural network; the patch embedding neural network can project the patch into a multi-dimensional feature vector.

[0088] In the specific implementation, in the above steps, in the process of projecting each patch corresponding to each variable into a multidimensional feature vector, it can specifically include: for each variable, when some grid points in the patch are null values, assigning a learnable, shared scalar to each null value position; projecting the patch assigned with the scalar into a multidimensional feature vector. In this way, sparse and irregularly distributed observation data can be fully utilized.

[0089] In implementation, after the patch is projected into a multidimensional feature vector, the variable ID and spatiotemporal position of each patch can be embedded into a corresponding multidimensional feature vector, and then added to the token of the patch, and the sum vector is used as the token of the patch. The spatiotemporal position multidimensional feature vector of the patch can be obtained by using a separable spatiotemporal position embedding method.

[0090] Optionally, after obtaining the token of the patch, the cross-attention module in the encoder can be used to aggregate the tokens corresponding to patches of different times and variables at the same spatial position to reduce the number of tokens and thus reduce the amount of computation. The query of the cross-attention module in the encoder is a learnable vector, with the key and value being the token vector corresponding to the patch at the same spatial position. It should be noted that the multi-head cross-attention module in the decoder can only query tokens that meet certain conditions (for example, tokens of non-empty patches, tokens of specific variables, tokens within a certain time and space range), thereby reducing the amount of computation.

[0091] The above encoder or decoder can predict the patch of at least one pre-divided time analysis field, so as to obtain at least one time analysis field. Optionally, each time analysis field includes multiple ground layer variables (such as 2-meter air temperature, 10-meter U wind component, 10-meter V wind component and sea level pressure), multiple atmospheric variables at multiple altitude layers (such as potential, air temperature, U component of wind, V component of wind, relative humidity), multiple ocean variables at different depth layers (such as ocean temperature, salinity, latitudinal and longitudinal components of ocean currents), and one or more variables of soil variables at different depth layers (such as soil temperature and water content). The division method of the variable patch in the analysis field can be consistent with the division method of the variable patch in the background field. The present invention can predict the patches of variables at different times and altitude layers at the same spatial position in the horizontal direction as a multi-channel patch, so that a more accurate prediction result can be obtained.

[0092] In the above embodiments, the data assimilation method is described in detail, and the present invention also provides embodiments corresponding to the data assimilation device and the data assimilation equipment. It should be noted that the present invention describes the embodiments of the device part from two perspectives, one is based on the functional module perspective, and the other is based on the hardware perspective.

[0093] Figure 2 This is a schematic diagram of the structure of the data assimilation device provided by an embodiment of the present invention. This embodiment is based on the perspective of functional modules, such as Figure 2 As shown, the device comprises:

[0094] The data acquisition module 10 is used to acquire and grid the observation data; the observation data includes at least one variable; it is also used to acquire at least one time background field and corresponding observation data, and grid the observation data to generate a grid consistent with the background field; wherein the background field and the observation data each include at least one variable;

[0095] The model training module 11 is used to train the constructed data assimilation model using the grids corresponding to all variables to obtain a trained data assimilation model;

[0096] The model inference module 12 is used to input the gridded observation data to be processed or the background field to be processed and the gridded observation data into the trained data assimilation model, and output the corresponding analysis field.

[0097] In the above-mentioned data assimilation device provided in the embodiment of the present invention, through the interaction of the above-mentioned three modules, only the observation data can be obtained and gridded, or the background field and the corresponding observation data of at least one time period can be obtained and the observation data can be gridded so that the background field and the observation data are unified into a consistent grid, and the grids corresponding to all variables are used to train the data assimilation model, and the trained data assimilation model is used to infer the analysis field corresponding to the gridded observation data to be processed or the analysis field corresponding to the background field to be processed and the gridded observation data. The whole inference process is fast, the assimilation accuracy is high, and the hardware processing speed is improved. The above-mentioned data assimilation device can be combined with traditional numerical weather forecasts or with weather forecasts based on large models, and can be used for data assimilation in the fields of weather climate, ocean, land surface, etc.

[0098] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part refer to the description of the embodiments of the method part, which will not be described here. And it has the same beneficial effects as the above-mentioned data assimilation method.

[0099] Furthermore, in a specific implementation, in the above-mentioned data assimilation device provided in the embodiment of the present invention, the data acquisition module 10 can also be used to acquire a historical analysis field or a reanalysis field.

[0100] Furthermore, in a specific implementation, in the above-mentioned data assimilation device provided in the embodiment of the present invention, the data acquisition module 10 can also be used to reserve some observation data with higher precision from the acquired observation data as high-precision observation data;

[0101] The model training module 11 can be specifically used to train the constructed data assimilation model by taking the historical analysis field or the reanalysis field as the actual situation in combination with the grids corresponding to all variables; it can also be specifically used to train the constructed data assimilation model by taking the historical analysis field or the reanalysis field as the actual situation in combination with the grids corresponding to all variables, and fine-tune the data assimilation model using high-precision observation data; it can also be specifically used in the training process to interpolate the output of the data assimilation model to the high-precision observation data, and then calculate the loss function to complete the training of the data assimilation model.

[0102] Furthermore, in a specific implementation, the data assimilation device provided in the embodiment of the present invention may further include:

[0103] A model building module is used to build an image-to-image transformation type machine learning model or a sequence-to-sequence transformation type machine learning model as a data assimilation model; the data assimilation model includes an encoder and / or a decoder.

[0104] The identification assignment module is used to assign an identity to each variable.

[0105] Furthermore, in a specific implementation, in the above-mentioned data assimilation device provided in an embodiment of the present invention, the model training module 11 can be specifically used to divide the grids of all variables into non-overlapping patches; the divided patches and their corresponding variable identities, and spatiotemporal positions are input into the encoder; the patch embedding module of the encoder is used to project each patch corresponding to each variable into a multi-dimensional feature vector, and at the same time, the variable identity and spatiotemporal position of each patch are embedded into the corresponding multi-dimensional feature vector to obtain a token of the patch; the cross-attention module of the encoder is used to aggregate the tokens corresponding to patches of different times and different variables at the same spatial position; the encoder or decoder is used to predict the patch of at least one pre-divided time analysis field to obtain at least one time analysis field.

[0106] Figure 3 This is a schematic diagram of the structure of the data assimilation device provided in an embodiment of the present invention. This embodiment is based on the hardware perspective, such as Figure 3 As shown, the data assimilation equipment includes:

[0107] A memory 20, for storing computer programs;

[0108] The processor 21 is used to implement the steps of the data assimilation method mentioned in the above embodiment when executing the computer program.

[0109] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU; the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0110] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the data assimilation method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data involved in the above-mentioned data assimilation method, etc.

[0111] In some embodiments, the data assimilation device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26. Those skilled in the art will appreciate that Figure 3 The structure shown in does not constitute a limitation on the data assimilation device, and may include more or fewer components than shown in the figure. The data assimilation device provided by the embodiment of the present invention includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the following method: data assimilation method, the effect is the same as above.

[0112] Finally, the present invention also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.

[0113] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code. The computer-readable storage medium provided by the present invention can implement the above-mentioned data assimilation method, and the effect is the same as above.

[0114] Finally, the present invention also provides an embodiment corresponding to a computer program product. The computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps described in the above data assimilation method embodiment are implemented. The computer program product provided by the present invention can implement the above-mentioned data assimilation method, and the effect is the same as above.

[0115] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0116] The data assimilation method, device, equipment and medium provided by the present invention are introduced in detail above. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A data assimilation method, characterized in that: The method comprises: Obtain observation data and grid it; the observation data contains at least one variable; Or, obtaining a background field and corresponding observation data of at least one time, and gridding the observation data to generate a grid consistent with the background field; wherein the background field and the observation data each contain at least one variable; The constructed data assimilation model is trained using the grids corresponding to all variables to obtain a trained data assimilation model; The gridded observation data to be processed or the background field to be processed and the gridded observation data are input into the trained data assimilation model, and the corresponding analysis field is output.

2. The data assimilation method according to claim 1, characterized in that: The background field includes at least one variable among a plurality of ground layer variables, a plurality of atmospheric variables at a plurality of height layers, a plurality of ocean variables at a plurality of depth layers, and a plurality of soil variables at a plurality of depth layers; The observation data include infrared radiation brightness temperature observed by satellite, microwave radiation brightness temperature observed by satellite, infrared emissivity observed by satellite, microwave emissivity observed by satellite, wind field inverted by satellite remote sensing, ozone concentration and snow depth, navigation satellite radio occultation data, satellite scatterometer data, satellite altimeter data, ground station observation data, buoy station observation data, ship observation data, wind profile observed by wind profiler radar, radiosonde observation data, aviation observation data, precipitation intensity estimated by weather radar, millimeter wave radar detection data, microwave link data, or one or a combination of these.

3. The data assimilation method according to claim 1, characterized in that: Before training the constructed data assimilation model using the grids corresponding to all variables, it also includes: An image-to-image transformation type machine learning model or a sequence-to-sequence transformation type machine learning model is constructed as a data assimilation model; the data assimilation model includes an encoder and / or a decoder.

4. The data assimilation method according to claim 3, characterized in that: The process of training the constructed data assimilation model includes: Divide the grid of all variables into non-overlapping patches; Input the divided patches and their corresponding variable identities and spatiotemporal positions into the encoder; Using the patch embedding module of the encoder, each patch corresponding to each variable is projected into a multidimensional feature vector, and the variable identity and spatiotemporal position of each patch are embedded into the corresponding multidimensional feature vector to obtain a token of the patch; An encoder or a decoder is used to predict a patch of at least one time analysis field that has been pre-divided to obtain at least one time analysis field.

5. The data assimilation method according to claim 4, characterized in that: The process of projecting each patch corresponding to each variable into a multi-dimensional feature vector includes: For each variable, when some grid points in the patch are null values, a learnable, shared scalar is assigned to each null value position; Project the scalared patch into a multidimensional feature vector.

6. The data assimilation method according to claim 1, characterized in that: The constructed data assimilation model is trained using the grids corresponding to all variables, including: The historical analysis field or reanalysis field is used as the actual situation and combined with the grids corresponding to all variables to train the constructed data assimilation model; Or, the historical analysis field or reanalysis field is used as the actual situation to train the constructed data assimilation model in combination with the grids corresponding to all variables, and the data assimilation model is fine-tuned using high-precision observation data; Alternatively, during the training process, the output of the data assimilation model is interpolated to the high-precision observation data, and then the loss function is calculated to complete the training of the data assimilation model.

7. The data assimilation method according to claim 1, characterized in that: After acquiring only the observation data, and before gridding the observation data, it also includes: Conduct quality control on acquired observation data; Normalize or standardize the observed data after quality control; After obtaining at least one time period of background field and corresponding observation data, and before gridding the observation data, the method further includes: Perform quality control on acquired background fields and / or observation data; Normalize or standardize the background field and / or observation data after quality control.

8. A data assimilation device, characterized in that: The device comprises: A data acquisition module is used to acquire and grid observation data; the observation data includes at least one variable; it is also used to acquire at least one time background field and corresponding observation data, and grid the observation data to generate a grid consistent with the background field; wherein the background field and the observation data each include at least one variable; The model training module is used to train the constructed data assimilation model using the grids corresponding to all variables to obtain a trained data assimilation model; The model inference module is used to input the gridded observation data to be processed or the background field to be processed and the gridded observation data into the trained data assimilation model, and output the corresponding analysis field.

9. A data assimilation device, characterized in that: The device comprises: Memory for storing computer programs; A processor, configured to implement the steps of the data assimilation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data assimilation method according to any one of claims 1 to 7 are implemented.

Images