A radar echo extrapolation method for predicting refined structure based on dyconvgru and unet

Abstract

The application discloses a radar echo extrapolation method for predicting refined structure based on DyConvGRU and Unet, and belongs to the field of short-impending weather prediction. The method comprises the following steps in sequence: S1: pre-processing a radar gray image, and dividing a training sample set and a test sample set; S2: constructing a DyConvGRU and a discriminator network, inputting the training sample obtained in the step S1 into the constructed model and training, and obtaining a converged DyConvGRU and discriminator network; S3: inputting the training sample obtained in the step S1 into the converged model obtained in the step S2, and obtaining a prediction result O; S4: constructing a Unet and a discriminator network, inputting the output O in the step S3 into the constructed model and training, and obtaining a converged Unet model and discriminator network; and S5: inputting the test sample in the step S1 into the converged DyConvGRU and Unet model in the step S2 and the step S3 in sequence, and obtaining a radar echo extrapolation image. The application can effectively predict the radar extrapolation image, and provides effective technical support for short-impending weather prediction.

Description

Technical Field

[0001] This invention relates to a radar echo extrapolation method based on the prediction refinement structure of DyConvGRU and Unet, belonging to the field of short-term meteorological forecasting technology. Background Technology

[0002] Accurate weather forecasting plays a crucial role in agriculture, transportation, water supply, and all aspects of human life and production. However, due to the complexity of the atmospheric environment, accurate weather forecasting, especially short-term forecasting, is a highly challenging task. Short-term forecasting refers to forecasts within 0-2 hours, and its main research focus is on small- and medium-scale systems. Compared with medium- and long-term and short-term forecasts, short-term forecasts have higher requirements in terms of demand areas and forecast timeliness. Early traditional short-term forecasting methods included statistical forecasting, empirical extrapolation, and numerical model forecasting. Statistical methods use models such as ARIMA and LR to find mathematical relationships between time series; numerical forecasting relies on large amounts of computational resources and data to calibrate models, making it computationally complex and unsuitable for weather forecasting within two hours.

[0003] Radar echo extrapolation technology predicts the future location and intensity of radar echoes, enabling faster tracking and forecasting of severe convective systems. Furthermore, by utilizing the ZrR relationship, echo intensity is converted into precipitation, thus achieving precipitation prediction. Therefore, accurate radar extrapolation results often indicate accurate precipitation forecasts.

[0004] The classic method for radar extrapolation is optical flow. The basic principle of optical flow is based on two assumptions: the pixel intensity of a scene remains essentially constant between two adjacent radar echo images, and adjacent pixels exhibit similar motion. The prediction process of optical flow mainly consists of two steps: first, the motion vector field of the echo is obtained by calculating the optical flow field of the radar echo; second, assuming the vector field remains constant, the radar echo image can be extrapolated from the vector field. However, since the estimation and extrapolation of the vector field are separate, it is difficult to determine the model parameters. While this method can predict cloud movement trends, it cannot predict cloud dissipation.

[0005] With the development of computer hardware and deep learning theory, deep learning models have been used to solve various problems (such as image classification, object detection, speech recognition, and natural language processing) with remarkable results. Researchers have begun applying deep learning models to short-term weather forecasting. Radar echo extrapolation-based short-term precipitation forecasting plays a crucial role in this field. The radar echo extrapolation problem can be viewed as a problem of predicting spatiotemporal sequences (sequences containing both temporal and spatial information). Deep learning models extract low-level features through multi-layered network structures and nonlinear transformations, abstracting them into high-level representations and discovering the probability distribution of the data. This provides an end-to-end solution for short-term precipitation forecasting and has significant development potential.

[0006] Most existing deep learning-based radar extrapolation models are based on ConvRNN, such as ConvLSTM and TrajGRU proposed by Shi et al. [1-2]. ConvLSTM replaces the fully connected structures from input to state and from state to state in the fully connected LSTM (FC-LSTM) with convolutional structures. Compared with FC-LSTM, ConvLSTM greatly improves the model's ability to capture spatial correlation. TrajGRU proposes to use subnetworks to output the state-to-state connection structure before the state transition based on ConvGRU. This model can actively learn the changes in the spatial structure at the end of the loop connection (such as translation, rotation, etc.). Lin et al. proposed the SA-ConvLSTM model [3]. This model introduces a memory-based self-attention module (SAM) on the basis of ConvLSTM. SAM uses similarity scores to aggregate all positions of the input and memory features to generate new features. SA-ConvLSTM, which incorporates a SAM module, significantly improves the ability to capture spatial correlations in long-term sequences compared to ConvLSTM. However, due to the computational complexity of the self-attention mechanism, it requires substantial computational resources and a long time to predict high-resolution images. Previous models often suffer from difficulties in predicting radar echo images over long periods and the loss of features in high-echo regions. This invention proposes a radar echo extrapolation method based on a prediction refinement structure using DyConvGRU and Unet, improving the quality of radar echo prediction images, particularly the prediction quality for high-echo regions.

[0007] [1] Shi,

[0008] [2] Shi, Xingjian & Gao, Zhihan & Lausen, Leonard & Wang, Hao & Yeung, Dit-Yan & Wong, Wai Kin & WOO, Wang-chun. (2017). Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model.

[0009] [3] Lin, Zhihui & Li, Maomao & Zheng, Zhuobin & Cheng, Yangyang & Yuan, Chun. (2020). Self-Attention ConvLSTM for Spatiotemporal Prediction. Proceedings of the AAAI Conference on Artificial Intelligence. 34.11531-11538.10.1609 / aaai.v34i07.6819. Summary of the Invention

[0010] The purpose of this invention is to provide a radar echo extrapolation method based on a prediction refinement structure using DyConvGRU and Unet. The coarse prediction module uses the DyConvGRU network to predict future radar echo images, and the refinement module uses Unet to refine the prediction results of the coarse prediction module, thereby improving the quality of the generated radar echo images, especially the prediction quality of images in high echo regions.

[0011] The method of this invention adopts the following technical solution:

[0012] A radar echo extrapolation method based on DyConvGRU and Unet for predicting refined structures includes the following steps:

[0013] S1: The radar echo grayscale images in the meteorological radar image sequence are cleaned to obtain a radar echo image sequence. Samples are extracted from the radar echo image sequence and normalized to construct the first training set. Each sample in the first training set contains input data and label data. The input data consists of several radar echo images from historical times, and the label data consists of several radar echo images from future times.

[0014] S2: Construct a coarse prediction model formed by cascading a first generator and a first discriminator; the first generator adopts the DyConvGRU model, its input is several radar echo images from historical moments, and its output is a coarse prediction result composed of several coarse radar echo images from future moments; the first discriminator includes a temporal discriminator and a spatial discriminator; after initializing the network parameters of the first generator and the first discriminator, the first training set obtained in step S1 is input into the constructed coarse prediction model and trained based on the adversarial generative training framework to obtain the converged first generator and the first discriminator;

[0015] S3: Input the first training set obtained in step S1 into the converged first generator obtained in S2 to obtain the coarse prediction result for each sample; update the input data of each sample in the first training set with the coarse prediction result to construct the second training set.

[0016] S4: Construct a refined model formed by cascading a second generator and a second discriminator; the second generator adopts the Unet model, its input is the coarse radar echo image at any random time in the coarse prediction result output by the first generator, and its output is the refined radar echo image corresponding to the input coarse radar echo image; the second discriminator is a spatial discriminator; after initializing the network parameters of the second generator and the second discriminator, the second training set obtained in step S3 is input into the refined model and trained based on the adversarial generative training framework to obtain the converged second generator and the second discriminator;

[0017] S5: A radar echo extrapolation model is formed by cascading the first and second converged generators. Several historical radar echo images before the target time are used as model input. First, the DyConvGRU model outputs several coarse radar echo images of future times. Then, the Unet model performs a fine-tuning operation on each coarse radar echo image of future times, and finally outputs several fine-tuned radar echo images of future times as extrapolation results.

[0018] Preferably, step S1, constructing the first training set, specifically includes the following steps:

[0019] A sequence of grayscale images from a weather radar with a sampling interval of m is obtained, and the grayscale values ​​in the images are converted into radar echo intensities to obtain a radar echo image sequence. Radar echo images with a maximum radar echo intensity less than a set threshold are removed from the radar echo image sequence. Then, according to the total length l of the input data and label data, a sliding window of length l is used to slide across the radar echo image sequence. Each sliding window extracts l consecutive radar echo images from the sequence as a sample, where the first few radar echo images are used as the input data of the sample, and the remaining radar echo images are used as the label data of the sample. For each extracted sample, the pixel values ​​of the radar echo images are divided by the maximum echo intensity that the radar can detect to normalize the radar echo images. All normalized samples constitute the first training set.

[0020] Preferably, the set threshold is 15 dBz, and the maximum echo intensity that the radar can detect is 70.

[0021] Preferably, in the first training set, the input data for each sample consists of 5 radar echo images from historical moments, and the label data consists of 15 radar echo images from future moments.

[0022] Preferably, in step S2, the first generator in the coarse prediction model adopts a DyConvGRU model with an encoder-decoder structure; wherein the encoder part is composed of multiple sets of downsampling layers and DyConvGRU layers cascaded alternately; the decoder part, except for the last layer, is composed of multiple sets of DyConvGRU layers and upsampling layers cascaded alternately, and the last layer of the decoder part is a 3D convolutional layer used to convert the number of output image channels into the number of radar echo image channels; each DyConvGRU layer in the first generator is an improved ConvGRU network, which is implemented by replacing the convolution operation in the ConvGRU network with a dynamic convolution operation;

[0023] The first discriminator in the coarse prediction model consists of a temporal discriminator and a spatial discriminator. The temporal discriminator is composed of multiple layers of 3D dynamic convolution and multiple layers of fully connected layers cascaded together. Its input is the coarse radar echo image at all future times in the coarse prediction result output by the first generator, and its output is the discriminant result in the temporal dimension. The spatial discriminator is composed of multiple layers of 2D dynamic convolution and multiple layers of fully connected layers cascaded together. Its input is the coarse radar echo image at any random time in the coarse prediction result output by the first generator, and its output is the discriminant result in the spatial dimension.

[0024] Preferably, in the first generator of the coarse prediction model, the encoder part adopts a 6-layer network structure. The first layer is a downsampling layer used to downsample the input width and height to 1 / 4 of the original. The second layer is a DyConvGRU layer, the third layer is a downsampling layer used to downsample the input width and height to 1 / 2 of the original. The fourth layer is a DyConvGRU layer, the fifth layer is a downsampling layer used to downsample the input width and height to 1 / 2 of the original. The sixth layer is a DyConvGRU layer. The decoder part of the first generator adopts a 7-layer network structure. The first layer is a DyConvGRU layer, the second layer is an upsampling layer used to double the input width and height. The third layer is a DyConvGRU layer, the fourth layer is an upsampling layer used to double the input width and height. The fifth layer is a DyConvGRU layer, the sixth layer is an upsampling layer used to quadruple the input width and height. The seventh layer is a 3D convolutional layer used to reduce the number of output image channels to 1.

[0025] In the first discriminator of the coarse prediction model, the temporal discriminator employs a cascaded structure of three dynamic 3D convolutional layers and three fully connected layers. The first dynamic 3D convolutional layer has 1 input channel and 8 output channels, with a kernel size of 3 and a stride of (2, 8, 8). The second dynamic 3D convolutional layer has 8 input channels and 16 output channels, with a kernel size of 3 and a stride of (2, 4, 4). The third dynamic 3D convolutional layer has 16 input channels and 32 output channels, with a kernel size of 3 and a stride of (1, 2, 2). The first fully connected layer has 512 input channels and 256 output channels. The second fully connected layer has 256 input channels and 128 output channels. The third fully connected layer has 128 input channels and 1 output channel.

[0026] In the first discriminator of the coarse prediction model, the spatial discriminator employs a cascaded structure of three 2D dynamic convolutional layers and three fully connected layers. The first 2D dynamic convolutional layer has 1 input channel and 8 output channels, with a kernel size of 3 and a stride of 8. The second 2D dynamic convolutional layer has 8 input channels and 16 output channels, with a kernel size of 3 and a stride of 4. The third 2D dynamic convolutional layer has 16 input channels and 32 output channels, with a kernel size of 3 and a stride of 2. The first fully connected layer has 512 input channels and 256 output channels. The second fully connected layer has 256 input channels and 128 output channels. The third fully connected layer has 128 input channels and 1 output channel.

[0027] Preferably, the coarse prediction model is trained using the adversarial generative training method of WGAN.

[0028] Preferably, in step S4, the second generator in the refinement model adopts the Unet model, whose input is the coarse radar echo image at any random time in the coarse prediction result output by the first generator, and whose output is the refined radar echo image after the refinement operation of the input coarse radar echo image; the second discriminator in the refinement model adopts a spatial discriminator composed of multiple cascaded 2D dynamic convolutional layers and multiple fully connected layers, whose input is the refined radar echo image output by the second generator, and whose output is the spatial dimension discrimination result.

[0029] Preferably, in the second generator of the refined model, the spatial discriminator used by the second discriminator consists of three cascaded 2D dynamic convolutional layers and three fully connected layers. The first 2D dynamic convolutional layer has 1 channel input and 8 channels output, with a kernel size of 3 and a stride of 8; the second 2D dynamic convolutional layer has 8 channels input and 16 channels output, with a kernel size of 3 and a stride of 4; the third 2D dynamic convolutional layer has 16 channels input and 32 channels output, with a kernel size of 3 and a stride of 2; the first fully connected layer has 512 channels input and 256 channels output; the first fully connected layer has 256 channels input and 128 channels output; and the first fully connected layer has 128 channels input and 1 channel output.

[0030] Preferably, the refined model is trained using the adversarial generative training method of WGAN.

[0031] This invention cleans and normalizes radar datasets to obtain grayscale radar image datasets. Then, it constructs a DyConvGRU, a temporal discriminator, and a spatial discriminator, updating model parameters through backpropagation. Finally, it trains a converged DyConvGRU as a prediction model to obtain an initial predicted image. Next, it constructs a Unet and a spatial discriminator, using the initial predicted image as input, updating model parameters through backpropagation to obtain a converged Unet model as a refinement model. In use, the radar echo image is first input into the DyConvGRU for prediction, and then the prediction result is input into the Unet for refinement, resulting in the final radar echo image. Compared to traditional optical flow extrapolation and previous deep learning methods, this invention significantly improves the prediction capability for high-echo regions, and over longer prediction times, it enhances the CSI (Key Success Index) and reduces the FAR (False Alarm Rate) in high-echo regions. Attached Figure Description

[0032] Figure 1 This is a flowchart of the present invention.

[0033] Figure 2 This is a schematic diagram of the prediction module training process.

[0034] Figure 3 This is a schematic diagram of a dynamic convolution structure.

[0035] Figure 4 This is a schematic diagram of the Encoder-Decoder structure of the DyConvGRU network.

[0036] Figure 5 This is a schematic diagram of the discriminator structure.

[0037] Figure 6 A diagram illustrating the detailed module training process.

[0038] Figure 7 This is a schematic diagram of the actual prediction process of the model.

[0039] Figure 8 This is an example of the predicted effect. Detailed Implementation

[0040] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0041] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.

[0042] like Figure 1 As shown, in a preferred embodiment of the present invention, the radar echo extrapolation method based on DyConvGRU and Unet for predicting refined structures includes the following steps in sequence:

[0043] S1: The radar echo grayscale images in the meteorological radar image sequence are cleaned to obtain a radar echo image sequence. Samples are extracted from the radar echo image sequence and normalized to construct the first training set. Each sample in the first training set contains input data and label data. The input data consists of several radar echo images from historical times, and the label data consists of several radar echo images from future times.

[0044] S2: Construct a coarse prediction model formed by cascading a first generator and a first discriminator; the first generator adopts the DyConvGRU model, whose input is several radar echo images from historical times, and whose output is a coarse prediction result O composed of several coarse radar echo images from future times; the first discriminator includes a temporal discriminator and a spatial discriminator; after initializing the network parameters of the first generator and the first discriminator, the first training set obtained in step S1 is input into the constructed coarse prediction model and trained based on the adversarial generative training framework to obtain the converged first generator and the first discriminator.

[0045] S3: Input the first training set obtained in step S1 into the converged first generator obtained in S2 to obtain the coarse prediction result O for each sample; update the input data of each sample in the first training set to the coarse prediction result O, thereby constructing the second training set.

[0046] S4: Construct a refined model formed by cascading a second generator and a second discriminator; the second generator adopts the Unet model, its input is the coarse radar echo image at any random time in the coarse prediction result O output by the first generator, and its output is the refined radar echo image corresponding to the input coarse radar echo image; the second discriminator is a spatial discriminator; after initializing the network parameters of the second generator and the second discriminator, the second training set obtained in step S3 is input into the refined model and trained based on the adversarial generative training framework to obtain the converged second generator and the second discriminator.

[0047] S5: A radar echo extrapolation model is formed by cascading the first and second converged generators. Several historical radar echo images before the target time are used as model input. First, the DyConvGRU model outputs several coarse radar echo images of future times. Then, the Unet model performs a fine-tuning operation on each coarse radar echo image of future times, and finally outputs several fine-tuned radar echo images of future times as extrapolation results.

[0048] In this embodiment, the radar echo image obtained by the Doppler weather radar has a resolution of 501×501 grayscale, with each pixel representing a 1km×1km area. The image name is the time from radar echo detection to the generation of the original radar echo image. The sequence length of the model input radar echo grayscale image data is 5 images, with a total duration of 0.5 hours and an interval of 6 minutes between adjacent radar echo images. The sequence length of the target radar echo grayscale image data is 15 images, with a total duration of 1.5 hours and an interval of 6 minutes between adjacent radar echo images. The total radar echo image sequence length (model input radar echo image sequence length plus target radar echo image sequence length) is 20, with a coverage duration of 2 hours. For better image visualization and ease of use, this embodiment assigns different colors to different echo intensities when displaying the results, converting the radar echo grayscale image into a color radar echo image.

[0049] The specific implementation and technical effects of steps S1 to S5 in this embodiment are shown in detail below.

[0050] The core of step S1 above is data cleaning and normalization, specifically including: removing images whose maximum echo intensity is less than a set threshold; selecting a continuous weather radar image sequence with a length of l and a time interval of m using a sliding window; and using the formula... The radar image is normalized by using echo intensity (scale is the normalized value, and maxval is the maximum echo intensity).

[0051] Specifically, in this embodiment, step S1 includes the following specific steps:

[0052] Step S1.1: Obtain the sequence of grayscale images from the weather radar. Use the formula E = I ÷ 255.0 × 70 to convert the grayscale image values ​​in the sequence into radar echo intensity values ​​(E is the echo intensity value, I is the grayscale value of the grayscale image, and 70 is the maximum radar echo value that the radar that generated the radar echo image can detect). A total of 127,342 radar echo images are obtained.

[0053] Step S1.2: Remove radar echo images with a maximum echo intensity less than 15 dBz from the radar echo image sequence obtained in S1.1. Since there is no precipitation when the echo intensity is less than 15 dBz, removing radar echo images with no precipitation helps reduce the impact of rainless weather on model training. A total of 96,427 radar echo images were obtained.

[0054] Step S1.3: Based on the radar echo image sequence obtained in S1.2, a sliding window is used to slide on the sequence. Each sliding step selects a continuous image sequence with a length of 20 and a time interval of 6 minutes, resulting in a total of 78,837 radar echo image sequences with a length of 20.

[0055] In this embodiment, the sliding window length is 20 and the step size is 1. When the sliding window selects 20 radar echo images, it is determined whether the time interval between two adjacent radar echo images is 6 minutes. If the time interval is 6 minutes, the file names corresponding to the radar echo images in the sequence are written to the data.csv file line by line (i.e., each line of data represents the file name corresponding to each image in the continuous radar echo image sequence of length 20), and the sliding window moves one position to the right; otherwise, no processing is performed, and the sliding window moves one position to the right.

[0056] Step S1.4: For each group of 20 radar echo images obtained in S1.3, use the formula... (E is the echo intensity value; scale is the normalized echo intensity value; maxval is 70, which is the maximum radar echo intensity that the radar can detect) Normalize the radar echo image;

[0057] Step S1.5: Scale the normalized radar echo image from S1.4 to 128×128. Reducing the image resolution helps improve the training and prediction efficiency of the model.

[0058] Step S1.6: 80% of the radar echo image filename sequences in the obtained data.csv are used as training samples, resulting in 63,069 sequences of length 20, which are saved as train.csv; 20% are used as test set samples, resulting in 15,768 sequences of length 20, which are saved as test.csv. Of the 20 radar echo images in each sample, the first 5 are used as input data for model prediction, and the last 15 are used as prediction labels, i.e., radar echo images at 15 future time points.

[0059] Specifically, step S2 includes the following steps:

[0060] Step S2.1: Construct the first generator (using the DyConvGRU model) and the first discriminator (including the temporal discriminator and the spatial discriminator) and cascade them to form a coarse prediction model. Initialize the weights and biases of the first generator and the first discriminator, and train the relevant parameters.

[0061] Step S2.2: Read the train.csv file obtained in step S1.6 line by line (each line contains the filenames of 20 consecutive radar echo images with a 6-minute time interval), and read the radar echo images according to the filenames in each line, inputting them into the first generator. Each sample in the training set contains the training input image input = {x i |1≤i≤5} and label target={x i |6≤i≤20}, where xi Let i be the i-th image in the radar echo image sequence.

[0062] Step S2.3: The first generator obtains 15 generated radar echo images, and then inputs the generated images and the real labels into the first discriminator in sequence;

[0063] Step S2.4: Calculate the loss between the predicted image and the real image, as well as the loss of the first generator and the first discriminator;

[0064] Step S2.5: Based on the adversarial generative training framework, update the weights and biases of each layer of the first generator and the first discriminator network according to the calculated loss to obtain new weight and bias values;

[0065] Step S2.6: Repeat steps S2.2 to S2.5 until the model training is completed and the maximum number of iterations is reached, resulting in the final converged first generator (DyConvGRU model), temporal discriminator, and spatial discriminator, as shown below. Figure 2 As shown.

[0066] The first generator described in step S2.1 above employs a DyConvGRU model with an encoder-decoder structure. The encoder consists of multiple sets of downsampling layers and DyConvGRU layers cascaded alternately. The decoder, except for the last layer, consists of multiple sets of DyConvGRU layers and upsampling layers cascaded alternately. The last layer of the decoder is a 3D convolutional layer used to convert the number of output image channels into the number of channels in the radar echo image. Each DyConvGRU layer in the encoder and decoder parts of the first generator is an improved ConvGRU network. The difference between this and a standard ConvGRU network is that it uses dynamic convolution operations with added kernel attention mechanisms instead of the original conventional convolution operations in the ConvGRU network. Dynamic convolution operations are existing technologies, such as... Figure 3 As shown, it sets up K convolutional kernels of the same size and number of channels, with their respective weights π. k The convolution kernel parameters of this layer are obtained by merging the convolution kernels. π is then calculated. k The process is as follows: First, GlobalAvgPooling is applied to the input x to obtain global spatial features. Then, these features are mapped to K dimensions through two fully connected (FC) layers, and finally, softmax normalization is performed. The resulting K attention weights are then assigned to the K convolutional kernels of this layer. The results of these K convolutional kernels are then linearly summed. The sum is then convolved with the original input x, and finally passed through a BatchNormal layer and an activation function to obtain the final result y.

[0067] Specifically, the mathematical expression for dynamic convolution with added kernel attention mechanism is:

[0068]

[0069]

[0070]

[0071]

[0072] Where g is the activation function. It is the weight of the k-th perceptron. The bias value π of the k-th perceptron k (x) is the kth linear function The weight of z varies depending on the input; j for Figure 3 The output of the second fully connected (FC) layer in the attention branch is τ, where τ is the temperature (set to 30 in this embodiment) and K is set to 4. Compared to conventional convolution, dynamic convolution with added kernel attention enhances the model's expressive power through the fusion of multiple convolutional kernels. The resulting convolution is input-dependent, meaning different convolution operations are applied to different input data.

[0073] The mathematical expression for the DyConvGRU network is:

[0074] z t =σ(W xz *x t +W hz *h t-1 )

[0075] r t =σ(W xr *x t +W hr *h t-1 )

[0076]

[0077]

[0078] Where σ represents the Sigmoid activation function; * and ⊙ represent the dynamic convolution operation with added kernel attention mechanism and the Hadamard product, respectively; z t and r t f is the reset gate and the update gate; f is the activation function, and in this embodiment, f is the tanh function.

[0079] It should be noted that in the description of this invention, the DyConvGRU model and the DyConvGRU network are two different concepts. The DyConvGRU network is a part of the network layer in the DyConvGRU model, which together with other upsampling layers, downsampling layers, etc., constitutes the DyConvGRU model.

[0080] In this embodiment, the entire DyConvGRU model, which serves as the first generator, adopts an Encoder-Decoder structure, such as... Figure 4 As shown, the specific network structure parameters are as follows:

[0081] In the Encoder of the DyConvGRU model, the first layer is a downsampling layer, which downsamples the input width and height to 1 / 4 of the original. The second layer is a DyConvGRU layer, the third layer is a downsampling layer, which downsamples the input width and height to 1 / 2 of the original. The fourth layer is a DyConvGRU layer, the fifth layer is a downsampling layer, which downsamples the input width and height to 1 / 2 of the original. The sixth layer is a DyConvGRU layer.

[0082] In the Decoder of the DyConvGRU model, the first layer is a DyConvGRU layer, the second layer is an upsampling layer that doubles the input width and height, the third layer is a DyConvGRU layer, the fourth layer is an upsampling layer that doubles the input width and height, the fifth layer is a DyConvGRU layer, the sixth layer is an upsampling layer that quadruples the input width and height, and the seventh layer is a 3D convolutional layer that reduces the number of image channels to 1.

[0083] The DyConvGRU model's encoder consists of six stacked layers, with data passed sequentially from the first to the sixth layer. That is, the output of the first layer serves as the input to the second layer, the output of the second layer as the input to the third layer, and so on. Similarly, the decoder consists of seven stacked layers, with data passed sequentially from the first layer. The output of the first layer serves as the input to the second layer, the output of the second layer as the input to the third layer, and so on. The DyConvGRU model's encoder and decoder are cascaded, meaning the output of the decoder serves as the input to the decoder. In this embodiment, samples are input in batches; therefore, the input dimensions of the DyConvGRU network with the encoder-decoder structure are (B, T, C, W, H), where B is the batch size, T is the time step the coarse prediction model needs to predict, C is the number of radar echo image channels, W is the radar echo image width, and H is the radar echo image height. In this embodiment, the input dimensions (B, T, C, W, H) of the DyConvGRU network are (16, 15, 1, 128, 128), and the output dimensions are (16, 15, 1, 128, 128).

[0084] The coarse prediction model described above has two second discriminators: a temporal discriminator and a spatial discriminator, both composed of dynamic convolutional layers and fully connected layers, such as... Figure 5 As shown.

[0085] Specifically, the temporal discriminator in the second discriminator employs a cascaded structure of three dynamic 3D convolutional layers and three fully connected layers. The first dynamic 3D convolutional layer has 1 input channel and 8 output channels, with a kernel size of 3 and a stride of (2, 8, 8). The second dynamic 3D convolutional layer has 8 input channels and 16 output channels, with a kernel size of 3 and a stride of (2, 4, 4). The third dynamic 3D convolutional layer has 16 input channels and 32 output channels, with a kernel size of 3 and a stride of (1, 2, 2). The first fully connected layer has 512 input channels and 256 output channels. The second fully connected layer has 256 input channels and 128 output channels. The third fully connected layer has 128 input channels and 1 output channel. The input dimension of the time discriminator is (B, T, C, W, H), where B is the batch size, T is the time step, C is the number of channels, W is the width of the radar echo image, and H is the height of the radar echo image. In this embodiment, the input dimension of the time discriminator is (16, 15, 1, 128, 128), and the output dimension is (16, 1).

[0086] Specifically, the spatial discriminator in the second discriminator employs a cascaded structure of three dynamic 2D convolutional layers and three fully connected layers. The first dynamic 2D convolutional layer has 1 input channel and 8 output channels, with a kernel size of 3 and a stride of 8. The second dynamic 2D convolutional layer has 8 input channels and 16 output channels, with a kernel size of 3 and a stride of 4. The third dynamic 2D convolutional layer has 16 input channels and 32 output channels, with a kernel size of 3 and a stride of 2. The first fully connected layer has 512 input channels and 256 output channels. The second fully connected layer has 256 input channels and 128 output channels. The third fully connected layer has 128 input channels and 1 output channel. The input dimensions of the spatial discriminator are (B, C, W, H), where B is the batch size, C is the number of radar echo image channels, W is the radar echo image width, and H is the radar echo image height. In this embodiment, the input dimension of the spatial discriminator is (16,1,128,128), which is obtained by randomly selecting one step size (i.e., a future time) from the 15 step sizes in the prediction results of the DyConvGRU network (dimension (16,15,1,128,128)) using a random function. The output dimension is (16,1).

[0087] In this embodiment, the coarse prediction model is trained using the adversarial generative training method of WGAN. The training parameters can be set as follows: the first generator, the DyConvGRU model, is optimized using the Adam optimizer with an initial learning rate of 0.0002; the spatial discriminator is optimized using the Adam optimizer with an initial learning rate of 0.0001; the temporal discriminator is optimized using the Adam optimizer with an initial learning rate of 0.0001; the model weights and biases are initialized using Kaiming; and the training iterations are 100 epochs. Ultimately, in this embodiment, after 100 iterations, the training loss decreased from 311.77 to 84.76; and the test loss decreased from 218.95 to 90.50.

[0088] Furthermore, in this embodiment, during the adversarial generative training of the coarse prediction model, the loss function of the first generator DyConvGRU model is... G It can be calculated using the following formula:

[0089]

[0090]

[0091] Where D T D S represents the temporal discriminator and spatial discriminator in the first discriminator, respectively; output represents the radar echo image generated by the first generator; T represents the number of predicted radar echo images output by the first generator, i.e., the total prediction step size; H and W represent the height and width of the radar echo image, respectively; target i,j Output represents the value of the labeled pixel in the actual radar echo image; i,j This represents the pixel value of the radar echo image generated by the generator; weight i,j This represents the weight corresponding to each pixel.

[0092] Meanwhile, in this embodiment, during the training process of the coarse prediction model, the loss function of the first discriminator... D It can be calculated using the following formula:

[0093] loss D =D T (output)+D s (output)-D T (target)-D S (target)

[0094] Where D T D SThese represent the time discriminator and the spatial discriminator in the first discriminator, respectively. The target is the label, and the output is the radar echo image generated by the first generator (DyConvGRU model).

[0095] The radar echo image prediction method of this invention employs a two-step approach: first, a DyConvGRU model is used as the prediction model to obtain a coarse predicted image; then, a Unet model is used with the initial predicted image as input to perform a refinement operation to generate a refined image. Therefore, in order to train the Unet model, a new training set needs to be constructed.

[0096] In step S3 of this embodiment, the purpose is to use the converged first generator obtained after training in S2 to regenerate coarse prediction results for the input data of each sample in the first training set, and then replace the input data of the corresponding samples in the first training set with the coarse prediction results, while keeping the label data unchanged, thereby converting the first training set into a second training set. Based on this second training set, the Unet model can be trained.

[0097] Step S4 in this embodiment includes the following specific steps:

[0098] Step S4.1: Construct a second generator (using a Unet network) and a second discriminator (using a spatial discriminator) and cascade them to form a refined model; initialize the weights and biases of the second generator and the second discriminator.

[0099] Step S4.2: Sample batches of samples from the second training set in step S3. In each sample, randomly select one of the 15 radar echo images (coarse prediction result O) output by the DyConvGRU model and input it into the second generator Unet. Then input the image generated by Unet into the second discriminator.

[0100] Step S4.3: Calculate the loss of the Unet output image and the real image, as well as the loss of the second discriminator and the second generator;

[0101] Step S4.4: Based on the adversarial generative training framework, update the weights and biases of each layer of the second generator and the second discriminator according to the calculated loss to obtain new weight and bias values;

[0102] Step S4.5: Repeat steps S4.2 and S4.4 until the model training iterations are completed, finally obtaining the converged Unet model and spatial discriminator, as shown below. Figure 6 As shown.

[0103] In this embodiment, the refinement model employs the Unet model as the second generator. Its input is a coarse radar echo image at any random moment from the coarse prediction results output by the first generator, and its output is a refined radar echo image after refinement. The second discriminator in the refinement model is a spatial discriminator composed of multiple cascaded 2D dynamic convolutional layers and multiple fully connected layers. Its input is the refined radar echo image output by the second generator, and its output is the spatial dimension discrimination result.

[0104] Specifically, the U-Net network used as the second generator in the refinement model in this embodiment is existing technology. Refer to the model structure in the 2015 paper "U-Net: Convolutional Networks for Biomedical Image Segmentation," where both the input and output dimensions are (B, C, W, H), where B is the batch size, C is the number of radar echo image channels, W is the radar echo image width, and H is the radar echo image height. In this embodiment, the dimensions (B, C, W, H) are (16, 1, 128, 128). The U-Net input is one radar echo image randomly selected from the 15 radar echo images in the prediction result O (dimension (16, 15, 1, 128, 128)) from step S3 using a random function.

[0105] In this embodiment, the spatial discriminator, serving as the second discriminator in the refinement model, employs a cascaded structure of three dynamic 2D convolutional layers and three fully connected layers. The first 2D dynamic convolutional layer has a 1-channel input and an 8-channel output, with a kernel size of 3 and a stride of 8. The second 2D dynamic convolutional layer has an 8-channel input and a 16-channel output, with a kernel size of 3 and a stride of 4. The third 2D dynamic convolutional layer has a 16-channel input and a 32-channel output, with a kernel size of 3 and a stride of 2. The first fully connected layer... The input has 512 channels and the output has 256 channels; the second fully connected layer has 256 input channels and 128 output channels; the third fully connected layer has 128 input channels and 1 output channel; the input dimension of the spatial discriminator is (B, C, W, H), where B is the batch size, C is the number of channels, W is the width of the radar echo image, and H is the height of the radar echo image. In this embodiment, the input dimension (B, C, W, H) of the spatial discriminator is (16, 1, 128, 128).

[0106] In this embodiment, the refined model is trained using the WGAN adversarial generative training method. The training parameters are set as follows: the second generator Unet network is optimized using the Adam optimizer with an initial learning rate of 0.00005; the spatial discriminator is optimized using the Adam optimizer with an initial learning rate of 0.00005; the weights and biases of Unet are initialized using Xavier, and the weights and biases of the spatial discriminator are initialized using Kaiming; the training iterations are 100 epochs. In this embodiment, after 100 iterations, the training loss decreases from 88.41 to 65.79; the test loss decreases from 89.87 to 75.14.

[0107] Furthermore, in this embodiment, during the refinement of the adversarial generative training process, the loss function of the second generator is... G It can be calculated using the following formula:

[0108]

[0109]

[0110] Where D t ... i,j Represents the value of the actual image label pixel; output i,j This represents the pixel value of the radar echo image generated by the second generator; weight i,j This represents the weight corresponding to each pixel.

[0111] Furthermore, in this embodiment, during the refinement of the model's adversarial generative training process, the loss function of the spatial discriminator, which serves as the second discriminator, is... D Calculate using the following formula:

[0112] loss D =D S (output)-D S (target)

[0113] Where D s These represent the spatial discriminator as the second discriminator, respectively. The target is the real radar echo image in the label data, and the output is the radar echo image generated by the second generator (Unet).

[0114] The convergence model saving strategy in step 4.4 is to save the two models with the smallest training loss and test loss, and then select the model with the better performance as the final convergence model through validation set testing.

[0115] like Figure 7 As shown, once the convergent first generator (DyConvGRU model) and second generator (Unet) are obtained, a two-step prediction method can be used to predict the radar echo image at future times.

[0116] In this embodiment, step S5 includes the following specific steps:

[0117] Step S5.1: Input the first five radar echo images from the image sequence in the test set sample in step S1.5 into the converged DyConvGRU model in step S2.6 to obtain the coarse prediction result, namely 15 radar echo images for the next 1.5 hours.

[0118] Step S5.2: Input the 15 radar echo images from the coarse prediction results obtained in Step S5.1 into the converged Unet model in Step S4.5 sequentially, and perform a refinement operation on each image in the coarse prediction results to obtain the final refined radar echo image. The final partial radar image effect is as follows. Figure 8 As shown.

[0119] To quantify the quality of radar echo extrapolation image generation, this embodiment uses CSI (Critical Success Index) and FAR (False Alarm Rate). Forty radar image sequences were randomly selected from the test set for testing, and the CSI and FAR corresponding to thresholds of 20dBz, 30dBz, and 40dBz were calculated. The results are shown in Tables 1 and 2.

[0120] in This represents the ratio of correctly predicted events to the total number of events that actually occurred and the total number of false positives. This represents the proportion of events that were predicted to be true but did not occur. The meanings of TP, FP, FN, and TN are shown in Table 3. Among them, when image i,j <threshold is false; when image i,j It is true when it is greater than or equal to the threshold. i,j This represents the number of pixels in the image, and threshold is a user-defined threshold. In this embodiment, 20, 30, and 40 were selected respectively.

[0121] Table 1 CSI Indicators

[0122] T = 6min T = 30min T = 60min T = 90min Threshold = 20 0.85 0.81 0.81 0.79 Threshold = 30 0.69 0.62 0.61 0.58 Threshold = 40 0.49 0.41 0.41 0.30

[0123] Table 2 FAR Index

[0124] T = 6min T = 30min T = 60min T = 90min Threshold = 20 0.09 0.11 0.11 0.11 Threshold = 30 0.23 0.28 0.28 0.28 Threshold = 40 0.32 0.37 0.39 0.37

[0125] Table 3 Confusion Matrix

[0126]

[0127] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A radar echo extrapolation method based on DyConvGRU and Unet for predictive refinement structures, characterized in that, The steps are as follows: S1: The radar echo grayscale images in the meteorological radar image sequence are cleaned to obtain a radar echo image sequence. Samples are extracted from the radar echo image sequence and normalized to construct the first training set. Each sample in the first training set contains input data and label data. The input data consists of several radar echo images from historical times, and the label data consists of several radar echo images from future times. S2: Construct a coarse prediction model formed by cascading a first generator and a first discriminator; the first generator adopts the DyConvGRU model, its input is several radar echo images from historical moments, and its output is a coarse prediction result composed of several coarse radar echo images from future moments; the first discriminator includes a temporal discriminator and a spatial discriminator; after initializing the network parameters of the first generator and the first discriminator, the first training set obtained in step S1 is input into the constructed coarse prediction model and trained based on the adversarial generative training framework to obtain the converged first generator and the first discriminator; S3: Input the first training set obtained in step S1 into the converged first generator obtained in S2 to obtain the coarse prediction result for each sample; update the input data of each sample in the first training set with the coarse prediction result to construct the second training set. S4: Construct a refined model formed by cascading a second generator and a second discriminator; the second generator adopts the Unet model, its input is the coarse radar echo image at any random time in the coarse prediction result output by the first generator, and its output is the refined radar echo image corresponding to the input coarse radar echo image; the second discriminator is a spatial discriminator; after initializing the network parameters of the second generator and the second discriminator, the second training set obtained in step S3 is input into the refined model and trained based on the adversarial generative training framework to obtain the converged second generator and the second discriminator; S5: The radar echo extrapolation model is formed by cascading the first and second converged generators. Several historical radar echo images before the target time are used as model input. First, the DyConvGRU model outputs several coarse radar echo images of future times. Then, the Unet model performs fine-tuning operations on each coarse radar echo image of future times. Finally, several fine-tuned radar echo images of future times are output as extrapolation results. In step S2, the first generator in the coarse prediction model adopts a DyConvGRU model with an encoder-decoder structure; the encoder part consists of multiple sets of downsampling layers and DyConvGRU layers cascaded alternately; the decoder part, except for the last layer, consists of multiple sets of DyConvGRU layers and upsampling layers cascaded alternately; the last layer of the decoder part is a 3D convolutional layer, which is used to convert the number of output image channels into the number of radar echo image channels. Each DyConvGRU layer in the first generator is an improved ConvGRU network, implemented by replacing the convolution operations in the ConvGRU network with dynamic convolution operations. The first discriminator in the coarse prediction model consists of a temporal discriminator and a spatial discriminator. The temporal discriminator is composed of a cascade of multiple 3D dynamic convolutional layers and multiple fully connected layers. Its input is the coarse radar echo image at all future times in the coarse prediction result output by the first generator, and its output is the discriminant result in the temporal dimension. The spatial discriminator is composed of a cascade of multiple 2D dynamic convolutional layers and multiple fully connected layers. Its input is the coarse radar echo image at any random time in the coarse prediction result output by the first generator, and its output is the discriminant result in the spatial dimension. In the first generator of the coarse prediction model, the encoder part adopts a 6-layer network structure. The first layer is a downsampling layer, which is used to downsample the input width and height to 1 / 4 of the original. The second layer is a DyConvGRU layer, the third layer is a downsampling layer, which is used to downsample the input width and height to 1 / 2 of the original. The fourth layer is a DyConvGRU layer, the fifth layer is a downsampling layer, which is used to downsample the input width and height to 1 / 2 of the original. The sixth layer is a DyConvGRU layer. The decoder part of the first generator adopts a 7-layer network structure. The first layer is a DyConvGRU layer, the second layer is an upsampling layer, which is used to double the input width and height. The third layer is a DyConvGRU layer, the fourth layer is an upsampling layer, which is used to double the input width and height. The fifth layer is a DyConvGRU layer, the sixth layer is an upsampling layer, which is used to quadruple the input width and height. The seventh layer is a 3D convolutional layer, which is used to reduce the number of output image channels to 1. In the first discriminator of the coarse prediction model, the temporal discriminator employs a cascaded structure of three dynamic 3D convolutional layers and three fully connected layers. The first dynamic 3D convolutional layer has 1 input channel and 8 output channels, with a kernel size of 3 and a stride of (2, 8, 8). The second dynamic 3D convolutional layer has 8 input channels and 16 output channels, with a kernel size of 3 and a stride of (2, 4, 4). The third dynamic 3D convolutional layer has 16 input channels and 32 output channels, with a kernel size of 3 and a stride of (1, 2, 2). The first fully connected layer has 512 input channels and 256 output channels. The second fully connected layer has 256 input channels and 128 output channels. The third fully connected layer has 128 input channels and 1 output channel. In the first discriminator of the coarse prediction model, the spatial discriminator employs a cascaded structure of three 2D dynamic convolutional layers and three fully connected layers. The first 2D dynamic convolutional layer has 1 input channel and 8 output channels, with a kernel size of 3 and a stride of 8. The second 2D dynamic convolutional layer has 8 input channels and 16 output channels, with a kernel size of 3 and a stride of 4. The third 2D dynamic convolutional layer has 16 input channels and 32 output channels, with a kernel size of 3 and a stride of 2. The first fully connected layer has 512 input channels and 256 output channels. The second fully connected layer has 256 input channels and 128 output channels. The third fully connected layer has 128 input channels and 1 output channel.

2. The radar echo extrapolation method based on DyConvGRU and Unet for predictive refinement structure according to claim 1, characterized in that, Step S1, constructing the first training set, specifically includes the following steps: The sampling interval is obtained as The weather radar grayscale image sequence is obtained, and the grayscale values ​​in the images are converted into radar echo intensity to obtain a radar echo image sequence. Radar echo images with a maximum radar echo intensity less than a set threshold are removed from the radar echo image sequence. Then, the sequence is processed according to the total length of the input data and label data. Using a length of The sliding window moves across the radar echo image sequence, and each sliding window extracts a continuous sequence of images. The radar echo images are used as samples, with the first few radar echo images serving as the input data for the samples and the remaining radar echo images serving as the label data for the samples. For each extracted sample, the pixel value of the radar echo image is divided by the maximum echo intensity that the radar can detect, thereby normalizing the radar echo image. All the normalized samples constitute the first training set.

3. The radar echo extrapolation method based on DyConvGRU and Unet for predictive refinement structure according to claim 2, characterized in that, The set threshold is 15 dBz, and the maximum echo intensity that the radar can detect is 70.

4. The radar echo extrapolation method based on DyConvGRU and Unet for predictive refinement structure according to claim 1, characterized in that, In the first training set, the input data for each sample consists of 5 radar echo images from historical moments, and the label data consists of 15 radar echo images from future moments.

5. The radar echo extrapolation method based on DyConvGRU and Unet for predictive refinement structure according to claim 1, characterized in that, The coarse prediction model is trained using the adversarial generative training method of WGAN.

6. The radar echo extrapolation method based on DyConvGRU and Unet for predictive refinement structure according to claim 1, characterized in that, In step S4, the second generator in the refinement model adopts the Unet model. Its input is the coarse radar echo image at any random time in the coarse prediction result output by the first generator, and its output is the refined radar echo image after the coarse radar echo image has been refined. The second discriminator in the refinement model adopts a spatial discriminator composed of multiple cascaded 2D dynamic convolutional layers and multiple fully connected layers. Its input is the refined radar echo image output by the second generator, and its output is the spatial dimension discrimination result.

7. The radar echo extrapolation method based on DyConvGRU and Unet for predictive refinement structure according to claim 6, characterized in that, In the second generator of the refined model, the spatial discriminator used by the second discriminator consists of three cascaded 2D dynamic convolutional layers and three fully connected layers. The first 2D dynamic convolutional layer has 1 input channel and 8 output channels, with a kernel size of 3 and a stride of 8; the second 2D dynamic convolutional layer has 8 input channels and 16 output channels, with a kernel size of 3 and a stride of 4; the third 2D dynamic convolutional layer has 16 input channels and 32 output channels, with a kernel size of 3 and a stride of 2; the first fully connected layer has 512 input channels and 256 output channels; the first fully connected layer has 256 input channels and 128 output channels; the first fully connected layer has 128 input channels and 1 output channel.

8. The radar echo extrapolation method based on DyConvGRU and Unet for predictive refinement structure according to claim 1, characterized in that, The refined model is trained using the adversarial generative training method of WGAN.

Images