Weather forecast downscaling correction method and device based on generative model

Abstract

The present application provides a method and device for downscaling and correcting weather forecasts based on a generative model, which relates to the field of data processing technology and solves the technical problem of low accuracy of weather forecast downscaling data. The method includes: generating low-resolution forecast samples and high-resolution actual analysis samples based on actual weather analysis data and historical weather forecast data; determining the high-resolution actual analysis samples as a first-stage training data set for a variational autoencoder, and determining the low-resolution forecast samples and specified actual constraints as a second-stage training data set for a diffusion model; constructing a downscaling and correction model based on the diffusion model and the variational autoencoder; using the first-stage training data set to perform first-stage model training on the variational autoencoder part to obtain a pre-trained network, and based on the pre-trained network, using the second-stage training data set to perform second-stage model training on the diffusion model part to obtain a trained target downscaling and correction model.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and device for downscaling and correcting weather forecasts based on a generative model. Background Art

[0002] At present, the way to downscale low-resolution weather forecasts is based on the convolutional neural network downscaling method, represented by SRCNN (Super-Resolution Convolution Neural Network), which directly learns the mapping relationship from low-resolution (LR) images to high-resolution (HR) images. The loss function is the mean square error, and what is optimized is the pixel-level difference between low-resolution and high-resolution images. In the fields of earth sciences such as meteorology and oceanography, the purpose of downscaling low-resolution weather forecasts is achieved by using such image super-resolution algorithms to establish the relationship between two meteorological data products with different resolutions.

[0003] However, the above-mentioned direct training model learns the pixel differences from low-resolution weather forecast grid fields to high-resolution actual reanalysis. The reconstructed image is prone to be too smooth and lacks high-frequency details and authenticity, resulting in low data accuracy of existing weather forecast downscaling methods. Summary of the Invention

[0004] The purpose of the present invention is to provide a weather forecast downscaling correction method and device based on a generative model to solve the technical problem of low data accuracy of existing weather forecast downscaling methods.

[0005] In a first aspect, the present application provides a method for downscaling and correcting weather forecasts based on a generative model, the method comprising:

[0006] Acquiring meteorological live analysis data and meteorological forecast historical data, and generating low-resolution forecast samples and high-resolution live analysis samples based on the meteorological live analysis data and the meteorological forecast historical data; wherein the resolution of the high-resolution live analysis samples is higher than that of the low-resolution forecast samples;

[0007] Determining the high-resolution live analysis samples as a first-stage training dataset for a variational autoencoder, and determining the low-resolution prediction samples and specified live constraints as a second-stage training dataset for a diffusion model, wherein the training labels corresponding to the second-stage training dataset are the high-resolution live analysis samples;

[0008] A downscaling correction model is constructed based on the diffusion model and the variational autoencoder; wherein the variational autoencoder portion of the downscaling correction model is used to encode and decode meteorological variables in a specified low-dimensional latent space, and the diffusion model portion of the downscaling correction model is used to refine the preliminary results generated by the variational autoencoder portion to improve the high-resolution correction results;

[0009] Performing first-stage model training on the variational autoencoder part using the first-stage training dataset to obtain a pre-trained network, and performing second-stage model training on the diffusion model part based on the pre-trained network using the second-stage training dataset to obtain a trained target downscaling correction model;

[0010] Based on the weather forecast data to be revised and the specified actual constraints, downscaling correction processing is performed using the target downscaling correction model to obtain revised high-resolution weather forecast data; wherein the resolution of the revised high-resolution weather forecast data is higher than that of the weather forecast data to be revised.

[0011] In one possible implementation, the loss function corresponding to the variational autoencoder part includes a reconstruction loss function and a KL divergence loss function; wherein the reconstruction loss function is used to measure the gap between the reconstructed data and the real high-resolution data corresponding to the high-resolution live analysis sample, and the KL divergence loss function is used to control the closeness of the potential distribution to the standard normal distribution N(0, I).

[0012] In one possible implementation, the diffusion process corresponding to the diffusion model part includes a forward process and a reverse process; wherein, the forward process includes gradually adding noise to the preliminary result generated by the variational autoencoder part, and the reverse process includes learning to gradually denoise the noise back to the high-resolution corrected result corresponding to the high-resolution live analysis sample.

[0013] In one possible implementation, the construction diffusion process corresponding to the forward process is performed by the following formula:

[0014] ;

[0015] in, For the forward process at time step t The data is used to represent the intermediate state to which a certain amount of noise has been added; For the forward process at time step t -1 data is used to represent the output of the previous step; is the noise attenuation coefficient in the forward process, which is used to determine the mixing ratio of the original data and the noise; is the standard normal distribution noise; The weights to hold for the data; is the weight of the noise.

[0016] In one possible implementation, the reverse process is performed by learning to gradually denoise the noise back to the high-resolution correction result using the following formula:

[0017] ;

[0018] in, t is the time step, is the noise state at time step t, is the noise state closer to the real data at time step t-1, is the probability distribution data, N () represents the Gaussian distribution form, represents the center value of the data after denoising, is the mean, is the covariance matrix, which is used to represent the control generation Diversity of time.

[0019] In a possible implementation, the input conditions corresponding to the diffusion model part include: using additional conditional variables as model inputs in each step, so that the model generation results meet physical constraints.

[0020] In one possible implementation, the additional condition variables include any one or more of the following: terrain, climate zone information, and recent conditions.

[0021] In one possible implementation, obtaining historical weather forecast data includes:

[0022] Acquiring ocean current data at multiple meteorological data collection locations at different time points in an ocean area; wherein the ocean current data includes at least one of seawater temperature, air temperature, wind speed, and water flow speed;

[0023] Comparing the plurality of ocean current data corresponding to the plurality of meteorological data collection locations to obtain a comparison result, and determining, based on the comparison result, at least two groups of target ocean current data having a data similarity greater than a preset similarity;

[0024] Determining target meteorological data collection locations and target time points corresponding to the at least two sets of target ocean current data, and determining ocean current dynamic data based on the at least two target meteorological data collection locations and the at least two target time points; wherein the ocean current dynamic data includes the dynamic direction and movement time difference of the target ocean current between the at least two target meteorological data collection locations;

[0025] Based on the ocean current dynamics data, historical weather forecast data for the ocean area is determined.

[0026] In a second aspect, the present application provides a weather forecast downscaling correction device based on a generative model, the device comprising:

[0027] an acquisition module, configured to acquire live meteorological analysis data and historical meteorological forecast data, and generate low-resolution forecast samples and high-resolution live analysis samples based on the live meteorological analysis data and the historical meteorological forecast data; wherein the high-resolution live analysis samples have a higher resolution than the low-resolution forecast samples;

[0028] a determination module, configured to determine the high-resolution live analysis samples as a first-stage training dataset for a variational autoencoder, and to determine the low-resolution prediction samples and specified live constraints as a second-stage training dataset for a diffusion model, wherein the training labels corresponding to the second-stage training dataset are the high-resolution live analysis samples;

[0029] A construction module is configured to construct a downscaling correction model based on the diffusion model and the variational autoencoder; wherein the variational autoencoder portion of the downscaling correction model is configured to encode and decode meteorological variables in a specified low-dimensional latent space, and the diffusion model portion of the downscaling correction model is configured to refine preliminary results generated by the variational autoencoder to improve high-resolution correction results;

[0030] a training module, configured to perform first-stage model training on the variational autoencoder portion using the first-stage training dataset to obtain a pre-trained network, and perform second-stage model training on the diffusion model portion based on the pre-trained network using the second-stage training dataset to obtain a trained target downscaling correction model;

[0031] A correction module is used to perform downscaling correction processing using the target downscaling correction model based on the weather forecast data to be corrected and the specified actual constraints to obtain corrected high-resolution weather forecast data; wherein the resolution of the corrected high-resolution weather forecast data is higher than that of the weather forecast data to be corrected.

[0032] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method described in the first aspect above.

[0034] This application brings the following beneficial effects:

[0035] The present application provides a weather forecast downscaling correction method and device based on a generative model, which can obtain weather actual analysis data and weather forecast historical data, and generate low-resolution forecast samples and high-resolution actual analysis samples based on the weather actual analysis data and the weather forecast historical data; wherein the resolution of the high-resolution actual analysis samples is higher than that of the low-resolution forecast samples; the high-resolution actual analysis samples are determined as the first-stage training data set for the variational autoencoder, and the low-resolution forecast samples and specified actual constraints are determined as the second-stage training data set for the diffusion model, wherein the training labels corresponding to the second-stage training data set are the high-resolution actual analysis samples; a downscaling correction model is constructed based on the diffusion model and the variational autoencoder; wherein the variational autoencoder in the downscaling correction model The encoder part is used to encode and decode the meteorological variables in a specified low-dimensional latent space, and the diffusion model part in the downscaling correction model is used to refine the preliminary results generated by the variational autoencoder part to improve the high-resolution correction results; the variational autoencoder part is trained on the first-stage model using the first-stage training data set to obtain a pre-trained network, and the diffusion model part is trained on the second-stage model based on the pre-trained network using the second-stage training data set to obtain a trained target downscaling correction model; based on the meteorological forecast data to be corrected and the specified actual constraints, downscaling correction processing is performed using the target downscaling correction model to obtain corrected high-resolution meteorological forecast data; wherein the resolution of the corrected high-resolution meteorological forecast data is higher than that of the meteorological forecast data to be corrected. In this scheme, not only is a diffusion model used to gradually generate real high-resolution meteorological data from noise, but live reanalysis data is also used to constrain the downscaling process, thereby simultaneously achieving downscaling and correction of meteorological forecasts. This not only downscales low-resolution forecasts, but also ensures that the details generated by the model are close to the real situation rather than fabricated details, thereby improving data resolution and details, thereby improving the accuracy of meteorological forecast downscaled data and solving the technical problem of low data accuracy of existing meteorological forecast downscaling methods.

[0036] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] Figure 1 A flow chart of a weather forecast downscaling correction method based on a generative model provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of a method for constructing a revised model for weather forecasting based on mobile observation points provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of the structure of a weather forecast downscaling correction device based on a generative model provided in an embodiment of the present application;

[0041] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0042] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0043] The terms "including," "having," and any variations thereof, as used in the embodiments of this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0044] Currently, downscaling methods for low-resolution weather forecasts also include those based on generative adversarial networks (GANs), such as the Super-Resolution Generative Adversarial Network (SRGAN) and the Enhanced Super-Resolution Generative Adversarial Network (ESRGAN). These methods essentially employ adversarial training between the generator and discriminator of a GAN. The generator is responsible for producing realistic high-resolution images, while the discriminator distinguishes between generated images and real images. These methods focus on optimizing image and perceptual quality rather than simply pixel-level errors. This allows for the generation of high-frequency details, resulting in sharper images that are closer to human perception. While visual performance is limited by traditional pixel-based models, these methods can be used to downscale meteorological data. However, these methods are difficult to train, and the adversarial loss can introduce artifacts that deviate from reality.

[0045] The above-mentioned downscaling method based on generative adversarial networks has many technical problems, including: the training process of generative adversarial networks using image data is already very unstable, and the loss function is more likely to become a Nan value when training with meteorological data, that is, the gradient disappears or explodes; the adversarial loss of generative adversarial networks is prone to introducing artificial details. In this case, although the data resolution is improved and there are more details, it often does not match the actual situation.

[0046] Based on this, the embodiment of the present application provides a weather forecast downscaling correction method and device based on a generative model, which can solve the technical problem of low data accuracy of existing weather forecast downscaling methods.

[0047] The embodiments of the present invention are further described below with reference to the accompanying drawings.

[0048] Figure 1 The following is a flow chart of a method for downscaling and correcting weather forecasts based on a generative model provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0049] Step S110 , obtaining meteorological live analysis data and meteorological forecast historical data, and generating low-resolution forecast samples and high-resolution live analysis samples based on the meteorological live analysis data and meteorological forecast historical data.

[0050] Among them, the resolution of high-resolution actual analysis samples is higher than that of low-resolution forecast samples.

[0051] As a possible implementation, Figure 2As shown in the figure, firstly, we collect the global meteorological actual reanalysis data and meteorological forecast historical data to make a training data set, that is, to establish low-resolution forecast samples and high-resolution actual reanalysis samples (that is, high-resolution actual analysis samples).

[0052] Step S120 : determining the high-resolution actual analysis samples as the first-stage training data set for the variational autoencoder, and determining the low-resolution prediction samples and the specified actual constraints as the second-stage training data set for the diffusion model.

[0053] Among them, the training labels corresponding to the second-stage training data set are high-resolution live analysis samples.

[0054] In an optional embodiment, high-resolution live reanalysis samples are used to establish a first-stage training dataset for the variational autoencoder; and low-resolution forecasts, live constraints, and live reanalysis data are used to construct a second-stage training dataset for the diffusion model.

[0055] The downscaling process, using real-world constraints, ensures that image resolution is enhanced while still generating details close to the real world. Furthermore, by integrating real-world constraints with low-resolution forecasts, the forecast is simultaneously corrected during downscaling, eliminating the need for a separate downscaling and correction step.

[0056] Step S130: construct a downscaling correction model based on the diffusion model and the variational autoencoder.

[0057] Among them, the variational autoencoder part in the downscaling correction model is used to encode and decode meteorological variables in a specified low-dimensional latent space, and the diffusion model part in the downscaling correction model is used to refine the preliminary results generated by the variational autoencoder part to improve the high-resolution correction results.

[0058] In this step, a downscaling correction model based on a variational autoencoder and a diffusion model is constructed. The variational autoencoder's primary task is to encode and decode meteorological variables in a low-dimensional latent space. The corresponding loss functions for the variational autoencoder include reconstruction loss and KL divergence loss. The reconstruction loss measures the difference between the reconstructed data and the true high-resolution data corresponding to the high-resolution ground truth analysis samples, typically using mean squared error (MSE) or other correlation metrics. The KL divergence loss controls the proximity of the latent distribution to the standard normal distribution N(0, I).

[0059] The diffusion model refines the initial results generated by the VAE (Variational Autoencoder), capturing more subtle patterns and improving the high-resolution correction. The diffusion process for this model consists of a forward process and a reverse process. The forward process involves gradually adding noise to the initial results generated by the VAE, while the reverse process involves learning to gradually remove the noise and return to the high-resolution correction results corresponding to the high-resolution ground truth analysis samples.

[0060] In an optional embodiment, the construction diffusion process corresponding to the forward process is performed by the following formula:

[0061] ;

[0062] in, For the forward process at time step t The data is used to represent the intermediate state to which a certain amount of noise has been added; For the forward process at time step t -1 data is used to represent the output of the previous step; is the noise attenuation coefficient in the forward process, which is used to determine the mixing ratio of the original data and the noise; is the standard normal distribution noise; The weights to hold for the data; is the weight of the noise. By using the above formula, the data processing process can be made more accurate.

[0063] In the embodiment of the present application, in the reverse process, the following formula is used to learn to gradually denoise the noise and return to the high-resolution correction result:

[0064] ;

[0065] in, t is the time step, is the noise state at time step t, is the noise state closer to the real data at time step t-1, is the probability distribution data, N () represents the Gaussian distribution form, represents the center value of the data after denoising, is the mean, is the covariance matrix, which is used to represent the control generation The above formula can make the data processing process more accurate.

[0066] It should be noted that this formula represents the reverse process, which describes how to use the current data at time step t to Estimate the last time step . Represents the conditional probability distribution of the reverse process, which means that at time step t, given the current noise state , calculate the previous time step The probability distribution of .

[0067] Since the transition at each time step is assumed to be Gaussian, it is a conditional normal distribution. Its goal is to learn how to Gradually denoise to get data closer to the real data .

[0068] For the Gaussian distribution , among which is the mean, which means that at time step t, Predict The most likely value of . It is a function learned by a neural network (usually U-Net or Transformer). Of course, it can also be understood intuitively as: Represents the center value of the data after denoising. The closer it is to the real data, the better. is the covariance matrix, which represents the generation The uncertainty in the time domain controls the diversity of the generated data. In addition, in many diffusion models, the covariance matrix It can be set to a diagonal matrix or even set to a constant (to simplify calculations). It can also be intuitively understood as determining the degree of "jitter" in the denoising process during the reverse process. If it is set too small, the generated data will lack diversity; if it is too large, the data will become unstable.

[0069] Furthermore, the input conditions for the diffusion model include the use of additional conditional variables as model inputs at each step to ensure that the model generates results consistent with physical constraints. These additional conditional variables include any one or more of the following: topography, climate zone information, and recent real-time conditions.

[0070] In step S140, the variational autoencoder part is trained in the first stage model using the first stage training data set to obtain a pre-trained network, and the diffusion model part is trained in the second stage model using the second stage training data set based on the pre-trained network to obtain a trained target downscaling correction model.

[0071] In practical applications, the training of the entire correction network is divided into two stages. The first stage is to train the autoencoder using live analysis data, and the second stage is to train the diffusion model using live constraints, low-resolution forecast data and high-resolution live reanalysis samples.

[0072] During the model training process, high-resolution live reanalysis data is first used to train the variational autoencoder, which is the first training stage, to obtain a pre-trained network. After the pre-training network, conditional constraints and low-resolution forecasts are used as model inputs, and live reanalysis data is used as labels to train the diffusion model, which is the second training stage.

[0073] In the embodiment of this application, Figure 2 As shown in the figure, a constrained generative downscaling correction algorithm based on diffusion (diffusion model) uses meteorological reanalysis data at continuous moments to train the variational encoder. After training, the diffusion model is trained to downscale the low-resolution meteorological forecast under the constraint of live reanalysis data at nearby moments, thereby obtaining high-resolution and realistically detailed results.

[0074] In practical applications, if the diffusion model is directly used to downscale meteorological data, this method can also obtain a downscaling result. However, this method lacks actual constraints and may lead to falsified details that are inconsistent with the actual situation and lack correction effect.

[0075] By constructing and training a data set for constrained downscaling correction using a diffusion model, we can use the reanalysis reality to constrain the use of a diffusion model to downscale meteorological forecasts, and achieve simultaneous downscaling and correction.

[0076] Step S150 , based on the weather forecast data to be corrected and the specified actual constraints, downscaling correction processing is performed using the target downscaling correction model to obtain corrected high-resolution weather forecast data.

[0077] Among them, the resolution of the revised high-resolution weather forecast data is higher than that of the weather forecast data to be revised.

[0078] After the model training is completed, the actual constraints and the low-resolution forecast data to be corrected are obtained. The actual constraints and the low-resolution forecast data to be corrected are used together as the input of the trained target downscaling correction model, and the high-resolution downscaled corrected forecast data is output.

[0079] In the embodiment of the present application, not only is a diffusion model used to gradually generate real high-resolution meteorological data from noise, but the downscaling process is also constrained by real-time reanalysis data, thereby simultaneously achieving downscaling and correction of meteorological forecasts. Not only can low-resolution forecasts be downscaled, but the details generated by the model can also be guaranteed to be close to the real situation rather than forged details, thereby improving data resolution and details, and thus improving the accuracy of meteorological forecast downscaled data.

[0080] Different from the downscaling method based on convolutional neural network in the prior art, the embodiment of the present application adopts a generative model, using a step-by-step diffusion algorithm, starting from real data, gradually adding Gaussian noise, and finally obtaining a distribution close to random noise, and then learning to gradually restore the noise to a clear image, so that the model can learn the complex distribution of data in more detail. Furthermore, the solution provided by the embodiment of the present application is also different from the downscaling method based on generative adversarial network in the prior art. In the embodiment of the present application, a variational encoder and a diffusion model are used. The entire training is stable and controllable, which can avoid the training collapse of the generative adversarial network. Moreover, by constructing a diffusion model with conditional constraints, real-world information is introduced while downscaling, guiding the model to approach the real situation while downscaling, and achieving downscaling and correction at the same time, solving the problem of forged details, and avoiding the problems of the details generated by the AI ​​downscaling model being too smooth, the generated forged details being inconsistent with the real situation, etc.

[0081] In some embodiments, the process of obtaining weather forecast historical data in the above step S110 can specifically include the following steps: obtaining ocean current data at multiple meteorological data collection points at different time points for the ocean area; wherein the ocean current data includes at least one of sea water temperature, air temperature, wind speed, and water flow speed; comparing the multiple ocean current data corresponding to the multiple meteorological data collection points to obtain a comparison result, and determining at least two groups of target ocean current data whose data similarity is greater than a preset similarity based on the comparison result; determining the target meteorological data collection points and the corresponding target time points corresponding to the at least two groups of target ocean current data, and determining ocean current dynamic data based on at least two target meteorological data collection points and at least two target time points; wherein the ocean current dynamic data includes the dynamic direction and moving time difference of the target ocean current between at least two target meteorological data collection points; and determining weather forecast historical data for the ocean area based on the ocean current dynamic data.

[0082] Through the above-mentioned dynamic processing method for ocean current data, the meteorological data for ocean areas in the acquired historical weather forecast data can be made more comprehensive and accurate, so that the data accuracy of subsequent model training process and weather forecast data processing process can be higher.

[0083] Figure 3 A schematic diagram of the structure of a weather forecast downscaling correction device based on a generative model is provided. Figure 3 As shown, the weather forecast downscaling correction device 300 based on the generative model includes:

[0084] An acquisition module 301 is configured to acquire live meteorological analysis data and historical meteorological forecast data, and generate low-resolution forecast samples and high-resolution live analysis samples based on the live meteorological analysis data and the historical meteorological forecast data; wherein the high-resolution live analysis samples have a higher resolution than the low-resolution forecast samples;

[0085] A determination module 302 is configured to determine the high-resolution live analysis samples as a first-stage training dataset for a variational autoencoder, and to determine the low-resolution prediction samples and specified live constraints as a second-stage training dataset for a diffusion model, wherein the training labels corresponding to the second-stage training dataset are the high-resolution live analysis samples;

[0086] A construction module 303 is configured to construct a downscaling correction model based on the diffusion model and the variational autoencoder; wherein the variational autoencoder portion of the downscaling correction model is configured to encode and decode meteorological variables in a specified low-dimensional latent space, and the diffusion model portion of the downscaling correction model is configured to refine preliminary results generated by the variational autoencoder to improve high-resolution correction results;

[0087] A training module 304 is configured to perform first-stage model training on the variational autoencoder portion using the first-stage training dataset to obtain a pre-trained network, and perform second-stage model training on the diffusion model portion based on the pre-trained network using the second-stage training dataset to obtain a trained target downscaling correction model;

[0088] The correction module 305 is used to perform downscaling correction processing based on the weather forecast data to be corrected and the specified actual constraints using the target downscaling correction model to obtain corrected high-resolution weather forecast data; wherein the resolution of the corrected high-resolution weather forecast data is higher than that of the weather forecast data to be corrected.

[0089] In some embodiments, the loss function corresponding to the variational autoencoder part includes a reconstruction loss function and a KL divergence loss function; wherein, the reconstruction loss function is used to measure the gap between the reconstructed data and the real high-resolution data corresponding to the high-resolution live analysis sample, and the KL divergence loss function is used to control the closeness of the potential distribution to the standard normal distribution N(0, I).

[0090] In some embodiments, the diffusion process corresponding to the diffusion model part includes a forward process and a reverse process; wherein, the forward process includes gradually adding noise to the preliminary result generated by the variational autoencoder part, and the reverse process includes learning to gradually denoise the noise back to the high-resolution corrected result corresponding to the high-resolution live analysis sample.

[0091] In some embodiments, the construction diffusion process corresponding to the forward process is performed by the following formula:

[0092] ;

[0093] in, For the forward process at time step t The data is used to represent the intermediate state to which a certain amount of noise has been added; For the forward process at time step t -1 data is used to represent the output of the previous step; is the noise attenuation coefficient in the forward process, which is used to determine the mixing ratio of the original data and the noise; is the standard normal distribution noise; The weights to hold for the data; is the weight of the noise.

[0094] In some embodiments, the reverse process is performed by learning to gradually denoise the noise back to the high-resolution correction result using the following formula:

[0095] ;

[0096] in, t is the time step, is the noise state at time step t, is the noise state closer to the real data at time step t-1, is the probability distribution data, N () represents the Gaussian distribution form, represents the center value of the data after denoising, is the mean, is the covariance matrix, which is used to represent the control generation Diversity of time.

[0097] In some embodiments, the input conditions corresponding to the diffusion model portion include: using additional conditional variables as model inputs in each step to ensure that the model generation results meet physical constraints.

[0098] In some embodiments, the additional condition variables include any one or more of the following: topography, climate zone information, and recent conditions.

[0099] The weather forecast downscaling correction device based on a generative model provided in the embodiment of the present application has the same technical features as the weather forecast downscaling correction method based on a generative model provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0100] An electronic device provided in an embodiment of the present application is Figure 4 As shown, the electronic device 400 includes a processor 402 and a memory 401 , wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps of the method provided in the above embodiment when executing the computer program.

[0101] See also Figure 4 The electronic device further includes: a bus 403 and a communication interface 404, a processor 402, a communication interface 404 and a memory 401 connected via the bus 403; the processor 402 is used to execute executable modules stored in the memory 401, such as computer programs.

[0102] Memory 401 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive. Communication between the system network element and at least one other network element is achieved via at least one communication interface 404 (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0103] The bus 403 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0104] Among them, the memory 401 is used to store programs, and the processor 402 executes the program after receiving the execution instruction. The method executed by the device defined by the process disclosed in any embodiment of the present application can be applied to the processor 402 or implemented by the processor 402.

[0105] The processor 402 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 402 or by instructions in the form of software. The above-mentioned processor 402 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 401, and processor 402 reads the information in memory 401 and, in conjunction with its hardware, completes the steps of the above method.

[0106] Corresponding to the above-mentioned weather forecast downscaling correction method based on the generative model, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned weather forecast downscaling correction method based on the generative model.

[0107] The weather forecast downscaling correction device based on the generative model provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, for the parts not mentioned in the device embodiment, reference can be made to the corresponding content in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0108] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0109] For another example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0112] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the weather forecast downscaling correction method based on the generative model described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0113] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0114] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. However, these modifications, changes, or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application. They should all be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A weather forecast downscaling correction method based on a generative model, characterized in that: The method comprises: Acquiring meteorological live analysis data and meteorological forecast historical data, and generating low-resolution forecast samples and high-resolution live analysis samples based on the meteorological live analysis data and the meteorological forecast historical data; wherein the resolution of the high-resolution live analysis samples is higher than that of the low-resolution forecast samples; Determining the high-resolution live analysis samples as a first-stage training dataset for a variational autoencoder, and determining the low-resolution prediction samples and specified live constraints as a second-stage training dataset for a diffusion model, wherein the training labels corresponding to the second-stage training dataset are the high-resolution live analysis samples; A downscaling correction model is constructed based on the diffusion model and the variational autoencoder; wherein the variational autoencoder portion of the downscaling correction model is used to encode and decode meteorological variables in a specified low-dimensional latent space, and the diffusion model portion of the downscaling correction model is used to refine the preliminary results generated by the variational autoencoder portion to improve the high-resolution correction results; The variational autoencoder is trained in the first stage using the first stage training data set to obtain a pre-trained network, and the diffusion model is trained in the second stage using the second stage training data set based on the pre-trained network to obtain a trained target downscaling correction model; the variational encoder is trained using continuous-time meteorological reanalysis data through a constrained generative downscaling correction algorithm based on the diffusion model, and after the variational encoder is trained, the diffusion model is trained to downscale and correct low-resolution meteorological forecasts under the constraints of near-time live reanalysis data, a dataset for constrained downscaling correction is constructed and trained using the diffusion model, and the meteorological forecast is downscaled using the diffusion model using the live reanalysis constraints, so that downscaling and correction are completed simultaneously; Based on the weather forecast data to be revised and the specified actual constraints, downscaling correction processing is performed using the target downscaling correction model to obtain revised high-resolution weather forecast data; wherein the resolution of the revised high-resolution weather forecast data is higher than that of the weather forecast data to be revised.

2. The method according to claim 1, characterized in that The loss function corresponding to the variational autoencoder part includes a reconstruction loss function and a KL divergence loss function; wherein, the reconstruction loss function is used to measure the gap between the reconstructed data and the real high-resolution data corresponding to the high-resolution live analysis sample, and the KL divergence loss function is used to control the closeness of the potential distribution to the standard normal distribution N(0, I).

3. The method according to claim 1, characterized in that The diffusion process corresponding to the diffusion model part includes a forward process and a reverse process; wherein, the forward process includes gradually adding noise to the preliminary result generated by the variational autoencoder part, and the reverse process includes learning to gradually denoise the noise back to the high-resolution corrected result corresponding to the high-resolution live analysis sample.

4. The method according to claim 3, characterized in that The construction diffusion process corresponding to the forward process is performed by the following formula: ； in, For the forward process at time step t The data is used to represent the intermediate state to which a certain amount of noise has been added; For the forward process at time step t -1 data is used to represent the output of the previous step; is the noise attenuation coefficient in the forward process, which is used to determine the mixing ratio of the original data and the noise; is the standard normal distribution noise; The weights to hold for the data; is the weight of the noise.

5. The method according to claim 3, characterized in that In the reverse process, the following formula is used to learn how to gradually denoise the noise and return to the high-resolution correction result: ； in, t is the time step, is the noise state at time step t, is the noise state closer to the real data at time step t-1, is the probability distribution data, N () represents the Gaussian distribution form, represents the center value of the data after denoising, is the mean, is the covariance matrix, which is used to represent the control generation Diversity of time.

6. The method according to claim 3, characterized in that The input conditions corresponding to the diffusion model part include: using additional conditional variables as model inputs in each step to make the model generation results meet physical constraints.

7. The method according to claim 6, characterized in that The additional condition variables include any one or more of the following: Topography, climate zone information, and recent conditions.

8. The method according to claim 1, characterized in that The obtaining of weather forecast historical data includes: Acquiring ocean current data at multiple meteorological data collection locations at different time points in an ocean area; wherein the ocean current data includes at least one of seawater temperature, air temperature, wind speed, and water flow speed; Comparing the plurality of ocean current data corresponding to the plurality of meteorological data collection locations to obtain a comparison result, and determining, based on the comparison result, at least two groups of target ocean current data having a data similarity greater than a preset similarity; Determining target meteorological data collection locations and target time points corresponding to the at least two sets of target ocean current data, and determining ocean current dynamic data based on the at least two target meteorological data collection locations and the at least two target time points; wherein the ocean current dynamic data includes the dynamic direction and movement time difference of the target ocean current between the at least two target meteorological data collection locations; Based on the ocean current dynamics data, historical weather forecast data for the ocean area is determined.

9. A weather forecast downscaling correction device based on a generative model, characterized in that: The device comprises: an acquisition module, configured to acquire live meteorological analysis data and historical meteorological forecast data, and generate low-resolution forecast samples and high-resolution live analysis samples based on the live meteorological analysis data and the historical meteorological forecast data; wherein the high-resolution live analysis samples have a higher resolution than the low-resolution forecast samples; a determination module, configured to determine the high-resolution live analysis samples as a first-stage training dataset for a variational autoencoder, and to determine the low-resolution prediction samples and specified live constraints as a second-stage training dataset for a diffusion model, wherein the training labels corresponding to the second-stage training dataset are the high-resolution live analysis samples; A construction module is configured to construct a downscaling correction model based on the diffusion model and the variational autoencoder; wherein the variational autoencoder portion of the downscaling correction model is configured to encode and decode meteorological variables in a specified low-dimensional latent space, and the diffusion model portion of the downscaling correction model is configured to refine preliminary results generated by the variational autoencoder to improve high-resolution correction results; A training module is configured to perform first-stage model training on the variational autoencoder portion using the first-stage training data set to obtain a pre-trained network, and perform second-stage model training on the diffusion model portion based on the pre-trained network using the second-stage training data set to obtain a trained target downscaling correction model; train the variational encoder using continuous-time meteorological reanalysis data through a constrained generative downscaling correction algorithm based on a diffusion model, and after completing the training of the variational encoder, train the diffusion model to downscale and correct low-resolution meteorological forecasts under the constraints of near-time live reanalysis data; construct and train a dataset for constrained downscaling correction using the diffusion model, and downscale the meteorological forecast using the diffusion model using the live reanalysis constraints, so that downscaling and correction are completed simultaneously; A correction module is used to perform downscaling correction processing using the target downscaling correction model based on the weather forecast data to be corrected and the specified actual constraints to obtain corrected high-resolution weather forecast data; wherein the resolution of the corrected high-resolution weather forecast data is higher than that of the weather forecast data to be corrected.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method according to any one of claims 1 to 8.

Images