Precipitation field space downscaling method

Abstract

The invention discloses a precipitation field space downscaling method, and relates to the technical field of meteorological data processing, and the method comprises the steps: obtaining high-resolution topographic information data and precipitation pairing data composed of high-resolution and low-resolution precipitation data, and carrying out the normalization processing of the high-resolution topographic information data and the precipitation pairing data; inputting the normalized precipitation pairing data and the high-resolution terrain information data into a pre-trained precipitation field space downscaling model, and outputting to obtain a high-resolution precipitation field; wherein the precipitation field space downscaling model comprises a convolutional neural network and a de-noising diffusion probability model, and the de-noising diffusion probability model is constructed based on residual calculation, forward noise addition, a multi-scale feature extraction fusion network and reverse de-noising. According to the method, the high-resolution precipitation field can be reconstructed from the low-resolution precipitation field.

Description

Technical Field

[0001] The present invention relates to the field of meteorological data processing technology, and in particular to a precipitation field spatial downscaling method suitable for scenarios such as high-resolution precipitation forecasting and disaster prevention. Background Art

[0002] Precipitation forecasting is crucial for disaster prevention and mitigation, urban management, agricultural production and many other fields. With the development of remote sensing technology and ground observation technology, the resolution and accuracy of meteorological data have gradually improved. However, due to the strong nonlinearity, local suddenness and complex spatial distribution of precipitation data, traditional downscaling methods have significant limitations in capturing the fine structure and local changes of heavy precipitation. They often lack accurate restoration of details and local features, and it is difficult to accurately reflect the microscopic distribution characteristics and short-term change trends of heavy precipitation. In addition, although models such as generative adversarial networks (GANs) have achieved certain results in image super-resolution, they still have problems such as optimization instability and noise amplification in the application of precipitation spatial downscaling. Summary of the invention

[0003] The present invention provides a precipitation field spatial downscaling method to reconstruct a high-resolution precipitation field.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A precipitation field spatial downscaling method comprises: obtaining high-resolution terrain information data and precipitation paired data consisting of high-resolution and low-resolution precipitation data, and normalizing both the high-resolution terrain information data and the precipitation paired data; inputting the normalized precipitation paired data and the high-resolution terrain information data into a pre-trained precipitation field spatial downscaling model, and outputting a high-resolution precipitation field; wherein the precipitation field spatial downscaling model comprises a convolutional neural network and a denoising diffusion probability model, and the denoising diffusion probability model is constructed based on residual calculation, forward denoising, reverse denoising and a multi-scale feature extraction fusion network based on a UNET architecture.

[0006] Preferably, low resolution refers to 3000m and 1000m resolution, and high resolution refers to 750m resolution.

[0007] Specifically, the convolutional neural network is used to process the low-resolution precipitation data in the precipitation pairing data to obtain the initial high-resolution precipitation data x cnn ; During model training, the frequency domain mixed loss function loss is used frequency Calculate the training loss of the convolutional neural network;

[0008] loss frequency =αloss FFT +βloss DWT

[0009] in, M and are the frequency domain values ​​obtained by applying fast Fourier transform to the true value and predicted value of the high-resolution precipitation field; H i , V i , D i It is the high frequency component after the true value i-times HAAR wavelet transform; It is the high-frequency component of the predicted value after HAAR wavelet transformation for i times.

[0010] Preferably, the optimization objective of the denoising diffusion probability model is defined as follows:

[0011]

[0012] Among them, cond terrain It is high-resolution terrain information data.

[0013] Specifically, the residual calculation is based on two Markov chains to generate the true value of high-resolution precipitation data x hr With the initial high-resolution precipitation data x cnn The residual x0 = x hr -x cnn .

[0014] Specifically, the forward noise addition formula is:

[0015]

[0016] Among them, β t is a hyperparameter, indicating the level of noise added to the data, which is related to the time step t, β t ∈(0,1), t=1,…,T; α t =1-β,

[0017] Specifically, the multi-scale feature extraction fusion network is used to add noise to the data x t and high-resolution terrain information datacond terrain Fusion is performed at time step t.

[0018] Specifically, the reverse denoising formula is:

[0019]

[0020] Among them, ∈ t_pred is the predicted value of the noise added to the UNET output corresponding to the forward time step t, ∈ is a randomly sampled Gaussian white noise;

[0021] By randomly sampling a Gaussian white noise and performing T-iteration denoising, the initial high-resolution precipitation data x cnn With high-resolution terrain information terrain To guide the gradual restoration of accurate high-frequency details of high-resolution precipitation data Then, the generated high-resolution precipitation high-frequency details are used and initial high-resolution precipitation data x cnn Synthesize accurate, high-resolution precipitation fields with high-frequency details.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] A pre-trained convolutional neural network (CNN) is used to estimate the initial high-resolution precipitation field, making full use of its advantages in spatial feature extraction and supporting multi-resolution downscaling, including non-integer multiples. Then, the residual learning ability of the residual network (ResNet) is used with the help of the encoder-decoder structure of UNET to capture contextual information of different scales, and the residual between the true value of the high-resolution precipitation field and the preliminary result is generated by guiding the frequency domain information and terrain information. The preliminary result is supplemented with details to improve the accuracy of the data distribution of the high-resolution precipitation field. In this process, the noise is effectively removed through the denoising process to ensure that the high-resolution precipitation field finally generated has higher accuracy and reliability.

[0024] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the embodiments of the present invention are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 It is the overall framework diagram of the present invention;

[0027] Figure 2 It is a schematic diagram of the framework of the spatial downscaling model of precipitation field of the present invention;

[0028] Figure 3 It is a schematic diagram of precipitation field data with different resolutions;

[0029] Figure 4 This is a comparison chart of super-resolution effects. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0031] Please refer to Figure 1 and Figure 2 , the present invention discloses a precipitation field spatial downscaling method, which is as follows:

[0032] 1. Obtain high-resolution terrain information data and precipitation pairing data consisting of high-resolution and low-resolution precipitation data.

[0033] Among them, the 750-meter high-resolution precipitation field data is obtained from the radar echo quantitative precipitation estimation product, and the precipitation pairing data is constructed by the interpolation downsampling method, including the pairing of 3000-meter low-resolution precipitation data with 750-meter high-resolution precipitation data, and the pairing of 1000-meter low-resolution precipitation data with 750-meter high-resolution precipitation data, see Figure 3 shown.

[0034] High-resolution terrain information data is extracted from 90-meter resolution elevation data through projection transformation, data resampling and other data processing to extract terrain feature information such as slope, slope aspect, terrain undulation, and generate 750-meter resolution altitude, slope, slope variability, slope aspect, slope variability, terrain undulation, and surface roughness data.

[0035] 2. Both high-resolution terrain information data and precipitation paired data are subjected to min-max normalization to improve the stability and generalization ability of the model.

[0036] 3. Input the normalized precipitation pairing data and high-resolution terrain information data into the pre-trained precipitation field spatial downscaling model, and output a high-resolution precipitation field. The precipitation field spatial downscaling model includes a convolutional neural network and a denoising diffusion probability model, such as Figure 1 As shown in the figure, it is divided into two stages. The first stage is to generate the initial high-resolution precipitation field with a convolutional neural network based on the frequency domain loss function. The second stage is to generate an accurate high-resolution precipitation field that conforms to the real data distribution based on the denoising diffusion probability model guided by high-resolution terrain information and high-frequency information.

[0037] In order to capture the global information and high-frequency information of the precipitation field, in the model training process of the convolutional neural network, the previous technology usually uses the difference between the true value and the model output value pixel as the main loss function, while the present invention abandons the loss term of pixel difference and directly uses the frequency domain mixed loss function loss frequency (including loss function based on fast Fourier transformFFT And the loss function loss based on discrete wavelet transform DWT ) Calculate the training loss of the convolutional neural network, enhance the ability of the convolutional neural network to capture global information and initial high-frequency information, and improve the accuracy of subsequent generation of high-resolution precipitation data;

[0038] loss frequency =αloss FFT +βloss DWT

[0039] in,

[0040]

[0041] α and β are adjustable hyperparameters, L is the number of discrete wavelet transforms; M and are the frequency domain values ​​obtained by applying fast Fourier transform to the true value and predicted value of the high-resolution precipitation field, including the real part and the imaginary part; H i , V i , D i It is the high frequency component after the true value i-times HAAR wavelet transform; It is the high-frequency component of the predicted value after HAAR wavelet transformation for i times.

[0042] The normalized precipitation paired dataset (e.g., 3000-750 m dataset) includes the low-resolution precipitation field x lr , high-resolution precipitation field x hr , x lr As the input of a one-stage convolutional neural network, x hr As the target.

[0043] The output of the convolutional neural network is x cnn , construct a dataset corresponding to the spatial scale (x cnn , x hr ), avoiding the time and resource consumption of convolutional neural networks.

[0044] The denoising diffusion probability model is constructed based on residual calculation, forward denoising, reverse denoising and a multi-scale feature extraction fusion network based on the UNET architecture. The second-stage denoising diffusion probability model takes the initial high-resolution precipitation field output by the first-stage convolutional neural network as input, introduces a multi-scale feature extraction mechanism, and combines the convolutional neural network (CNN) structure with the channel attention mechanism to extract high-resolution terrain information and spatial features of the initial high-resolution precipitation field. The residual convolution structure is used to extract different scale features with the help of the jump connection of the corresponding layer of UNET. Accurate prediction of the noise at the corresponding time step t is achieved. Gaussian white noise is randomly sampled from the standard normal distribution. The resolution of the Gaussian white noise is consistent with the resolution of the high-resolution precipitation field. Through the iterative denoising process, the noise is gradually reduced in each iteration, and the residual between the high-resolution precipitation field and the initial convolutional neural network output result is gradually restored. The residual is the high-frequency details of the high-resolution precipitation field, which is a detailed supplement to the initial prediction result.

[0045] The present invention uses high-resolution terrain information to guide the reverse denoising process to generate a more accurate high-resolution precipitation field. The optimization objective of the denoising diffusion probability model is defined as follows:

[0046]

[0047] Among them, cond terrain It is high-resolution terrain information data.

[0048] The residual calculation is based on two Markov chains to generate the true value x of high-resolution precipitation data. hr With the initial high-resolution precipitation data x cnn The residual x0 = x hr -x cnn .

[0049] The forward noise addition process gradually adds noise to the data:

[0050]

[0051] Among them, β t is a hyperparameter, indicating the level of noise added to the data, which is related to the time step t, β t ∈(0,1), t=1,…,T; α t =1-β t , The forward noise addition process is a parameter-free process. When T is large enough, x0 will be approximately Gaussian noise sampled from a standard normal distribution.

[0052] The multi-scale feature extraction fusion network is used to add noise to the data x t and high-resolution terrain information datacond terrain Fusion is performed at time step t.

[0053] Inverse denoising is used to obtain initial high-resolution precipitation data x cnn With high-resolution terrain information terrain To guide the gradual restoration of accurate high-resolution precipitation data Then, using accurate high-resolution precipitation data and initial high-resolution precipitation data x cnn Generate accurate, high-resolution precipitation fields with high-frequency details.

[0054] The reverse denoising process is the process of restoring the data to the real data distribution:

[0055]

[0056] Where θ represents the model parameter, p θ is the conditional probability transfer density function, represents normal distribution, μ θ ,Σ θ are the mean and variance respectively, and I is the identity matrix.

[0057] The multi-scale feature extraction and fusion UNET designed in this paper uses multi-scale convolution kernels and channel prior convolution attention mechanism to extract and fuse the features of high-resolution terrain information and high-resolution precipitation field information to accurately predict the corresponding time step t timestep The noise that needs to be removed∈ t_pred , the UNET forward noise process time step t timestep The added noise ∈ t is a true value, and ∈ t With model output ∈ t_pred The absolute error is used as the loss function:

[0058]

[0059] In the formula, t is equivalent to t timestep ,∈ is a randomly sampled Gaussian white noise.

[0060] The reverse denoising process is performed as follows:

[0061]

[0062] In the formula, ∈ is a randomly sampled Gaussian white noise. Through the above process, iterate T steps, starting from a randomly sampled Gaussian noise, the high-frequency details of the high-resolution precipitation field are gradually restored with the initial high-resolution precipitation field and high-resolution terrain information as the guide. Generates accurate high-resolution precipitation fields with high-frequency details. The results of generating high-resolution precipitation fields in the embodiment are visible Figure 4 .

[0063] The method of the present invention has been verified in actual cases, and the results show that the precipitation forecast after spatial downscaling is significantly better than the traditional method in resolution and accuracy, and can effectively meet the needs of high-resolution precipitation spatial distribution. Through the present invention, more refined precipitation spatial distribution support can be provided for large-scale events, urban waterlogging, geological disaster prevention and other fields. The present invention has good application prospects and market value, and can provide important technical guarantees for the intelligent processing and efficient use of meteorological data, helping to improve urban management and disaster response capabilities.

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A precipitation field spatial downscaling method, characterized in that: include: High-resolution terrain information data and precipitation paired data consisting of high-resolution and low-resolution precipitation data are obtained, and both the high-resolution terrain information data and the precipitation paired data are normalized; the normalized precipitation paired data and the high-resolution terrain information data are input into a pre-trained precipitation field spatial downscaling model, and a high-resolution precipitation field is output; wherein, the precipitation field spatial downscaling model includes a convolutional neural network and a denoising diffusion probability model, and the denoising diffusion probability model is constructed based on residual calculation, forward denoising, reverse denoising and a multi-scale feature extraction fusion network based on the UNET architecture.

2. The precipitation field spatial downscaling method according to claim 1, characterized in that: Low resolution refers to 3000m and 1000m resolution, and high resolution refers to 750m resolution.

3. The precipitation field spatial downscaling method according to claim 1, characterized in that: The convolutional neural network is used to process the low-resolution precipitation data in the precipitation pairing data to obtain the initial high-resolution precipitation data x cnn ; During model training, the frequency domain mixed loss function loss is used frequency Calculate the training loss of the convolutional neural network; loss frequency =αloss FFT +βloss DWT in, M and are the frequency domain values ​​obtained by applying fast Fourier transform to the true value and predicted value of the high-resolution precipitation field; H i , V i , D i It is the high frequency component after the true value i-times HAAR wavelet transform; It is the high-frequency component of the predicted value after HAAR wavelet transformation for i times.

4. The precipitation field spatial downscaling method according to claim 3, characterized in that: The optimization objective of the denoising diffusion probability model is defined as follows: Among them, cond terrain It is high-resolution terrain information data.

5. The precipitation field spatial downscaling method according to claim 4, characterized in that: The residual calculation is based on two Markov chains to generate the true value x of high-resolution precipitation data. hr With the initial high-resolution precipitation data x cnn The residual x0 = x hr -x cnn .

6. The precipitation field spatial downscaling method according to claim 5, characterized in that: The forward noise addition formula is: Among them, β t is a hyperparameter, indicating the level of noise added to the data, which is related to the time step t, β t ∈(0,1), t=1,…,T, T is an adjustable hyperparameter; α t =1-β, When T is large enough, after T steps of forward noise addition, it will approach a Gaussian white noise sampled from a standard normal distribution.

7. The precipitation field spatial downscaling method according to claim 6, characterized in that: The multi-scale feature extraction fusion network is used to add noise to the data x t , initial high-resolution precipitation data x cnn and high-resolution terrain information datacond terrain Fusion is performed at time step t.

8. The precipitation field spatial downscaling method according to claim 7, characterized in that: The reverse denoising formula is: Among them, ∈ t_pred is the predicted value of the noise added to the UNET output corresponding to the forward time step t, ∈ is a randomly sampled Gaussian white noise; By randomly sampling a Gaussian white noise and performing T-iteration denoising, the initial high-resolution precipitation data x cnn With high-resolution terrain information terrain To guide the gradual restoration of accurate high-frequency details of high-resolution precipitation data Then, the generated high-resolution precipitation high-frequency details are used and initial high-resolution precipitation data x cnn Synthesize accurate, high-resolution precipitation fields with high-frequency details.

Images