A Multimodal Radar Echo Extrapolation Method

Abstract

This invention discloses a multimodal radar echo extrapolation method, comprising: radar base data preprocessing: reading radar base data and performing data preprocessing, cropping and labeling the normalized grayscale data after data processing, and dividing the dataset; constructing and training a radar echo extrapolation model: using a Convolutional Long-Short Term Memory Network (ConvLSTM) to extract data features and perform feature fusion, training the network using multimodal fusion features, and saving the network parameters after training; and testing the model: inputting data from the test sample set into the network model to finally obtain the predicted radar echo image.

Description

Technical Field

[0001] This invention belongs to the field of ground meteorological observation technology in atmospheric sounding, and particularly relates to a radar echo extrapolation method based on multi-mode. Background Technology

[0002] Weather radar echo extrapolation refers to the prediction of the morphology, intensity, distribution, and other properties of echoes within the next 0-6 hours, given a historical echo sequence. Based on the extrapolation results, forecasters can forecast severe convective weather such as heavy precipitation, hail, squall lines, and tornadoes. Therefore, radar echo extrapolation is one of the most important and fundamental methods for short-term forecasting of severe convective weather.

[0003] Traditional echo extrapolation methods, such as centroid tracking and optical flow methods, are mainly based on the idea of ​​motion tracking. However, due to the complexity of the echo extrapolation model, they still have shortcomings in terms of forecast timeliness and accuracy. For example, the centroid tracking method based on a single entity obtains the motion vector of the centroid by fitting the position of the centroid coordinates, thereby achieving the tracking and prediction of the single entity; the region-based cross-correlation method divides the echo image into several sub-regions, calculates the motion vector field of the sub-regions, determines the motion vector of the echo sub-region by solving the correlation coefficient, and then extrapolates the motion and deformation of the echo; the optical flow method originated in the field of computer vision. By solving the constrained optical flow equation, the optical flow field of the continuous echo is obtained, and the motion vector of the echo can be indirectly obtained based on the optical flow field. In summary, the centroid tracking method requires first identifying storm cells and is suitable for extrapolating strong echo cells; the cross-correlation method struggles to accurately estimate the rapid motion and dramatic evolution of severe convective systems and is suitable for extrapolating general or mixed precipitation systems such as convective clouds or stratiform clouds; the optical flow method assumes that continuous echoes have fixed brightness over time and that the region moves slowly, making it suitable for weather systems with gradual changes. The basic idea of ​​traditional methods is to first estimate the vector field of the echo motion and then predict the echo motion through differences between time-series images. However, extrapolating the complex evolution of severe convective systems still has significant limitations, especially in its application to different types of severe convective weather systems.

[0004] In recent years, with the rapid development of computer hardware and the emergence of large-scale training data, deep learning has been widely applied in fields such as computer vision and natural language processing. It trains deep models capable of performing end-to-end tasks by learning multi-layered abstract representations of data. As a data-driven method, deep learning-based radar echo extrapolation can not only learn effective representations of data from historical data but also perform multi-scale analysis and reasoning within a spatiotemporal range through operations such as convolution. Its modeling capabilities and application potential are stronger than traditional radar echo extrapolation methods. In the radar echo extrapolation problem, a large amount of radar detection data can be used for training, and the long-accumulated historical radar echo data also implicitly contains the changing patterns of radar echoes. Therefore, from the perspective of deep learning, this problem can be solved by training a reasonable neural network model. Among many neural network models, the Convolutional Long Short-Term Memory (ConvLSTM) network, by replacing some fully connected operations with convolution operations, not only maintains the strong temporal information processing capabilities of LSTM but also enhances its spatial feature extraction capabilities, making it one of the effective models for solving spatiotemporal sequence learning problems. However, current deep learning-based radar echo extrapolation mainly extrapolates the reflectivity factor. A single reflectivity factor cannot fully reflect the properties and distribution of particles in the atmosphere, effectively utilize the correlation and complementarity of multiple radar products, and suffers from low prediction accuracy.

[0005] Based on the ConvLSTM network structure, to improve the accuracy and forecast timeliness of radar echo extrapolation, a multimodal radar echo extrapolation network is designed to address the motion and change characteristics of radar echoes. This network consists of an encoding layer, a feature fusion layer, a prediction layer, a state fusion layer, and a decoding layer. The feature fusion layer is composed of a convolutional long short-term memory model and an attention module. By fusing multimodal features to model the correlation and complementarity of various radar products, the network solves the problems of low extrapolation accuracy and short effective duration.

[0006] The trained network can effectively utilize the correlation and complementarity of radar multimodal data, memorize the nonlinear motion, generation and disappearance evolution of echoes, and fully learn the laws of radar echo motion change from historical echo data. This is of great significance for improving the accuracy of radar echo extrapolation and the timeliness of forecasts. Summary of the Invention

[0007] Objective of the Invention: The technical problem to be solved by this invention is the low accuracy and short forecast time of existing radar echo extrapolation methods. This invention proposes a multi-modality-based radar echo extrapolation method to achieve extrapolation prediction of radar echoes, comprising the following steps:

[0008] Step 1, Radar base data preprocessing: Read the radar base data of combined reflectivity factor CR, vertical cumulative liquid water content VIL, and echo top height TOP and perform data preprocessing. The normalized grayscale data obtained after data processing is cropped and labeled, and divided into training sample set, validation sample set and test sample set.

[0009] Step 2, train the radar echo extrapolation model: initialize the network model, extract multimodal data features from the combined reflectivity factor CR, vertical cumulative liquid water content VIL, and echo top height TOP sample set obtained in Step 1 and perform feature fusion, use the fused features to train the ConvLSTM network, and save the network parameters after training.

[0010] Step 3, test the model: input the data from the test sample set into the network model trained in Step 2, and finally obtain the predicted radar echo image.

[0011] Step 1-1, Read radar base data: According to the weather radar base data format, read and decode the base data. The data starts from the top left corner. Write the data according to the column number increasing with longitude and the row number decreasing with latitude to obtain data blocks. According to the data layer number and latitude and longitude coordinates in the file header, convert the data blocks into meteorological element regular grid data. Store the decoded data file according to the file header and data blocks respectively.

[0012] Step 1-2, Radar base data preprocessing: Read the data block from Step 1-1, convert the stored value into the actual value of meteorological element according to the conversion formula, map it to a gray value range, and then perform normalization processing to obtain normalized gray value data;

[0013] Steps 1-3, Data Product Processing: For the normalized grayscale data obtained in Steps 1-2, select the main echo area, crop the normalized grayscale data to the same size, and perform the same processing on the echo top height and vertical cumulative liquid water content data at the corresponding time.

[0014] Steps 1-4, Dataset Partitioning: For the normalized grayscale data obtained in Steps 1-3 after size cropping, set pixel thresholds according to the actual meteorological element values ​​and ranges, label the data and filter out data sequences that meet the conditions, and generate a data sample set according to a certain time series length and sliding window width. Divide the sample set into a training sample set, a validation sample set and a test sample set according to a certain ratio.

[0015] Step 1-2-1, Reading data blocks and data conversion: Read the data blocks from Step 1-1, and convert the stored values ​​of the data blocks into actual values ​​according to the record value conventions in the radar base data format specification file;

[0016] Step 1-2-2, Grayscale value mapping: The actual radar mosaic covers a latitude and longitude range of 20°*10°, with a longitude resolution of 0.01° and a latitude resolution of 0.01°. Based on this, the data block shape in Step 1-2-1 is reshaped to (1000, 2000). Then, the data block values ​​are mapped to a grayscale value range of 0 to 255 to obtain grayscale data P. This is then normalized to obtain normalized grayscale data P′. The grayscale mapping formula is as follows:

[0017]

[0018]

[0019]

[0020] The normalization formula is as follows:

[0021]

[0022] Among them, V CR V VIL V TOP P represents the actual value of the combined reflectivity factor, the actual value of the vertical cumulative liquid water content, and the actual value of the echo top height, respectively. CR P VIL P TOP These represent the combined reflectance grayscale value, vertical cumulative liquid water content grayscale value, and echo top height grayscale value after data mapping, respectively. P represents the grayscale data, and P′ represents the normalized grayscale data. Indicates rounding down;

[0023] The final normalized grayscale data P′ has a feature number of 1 and a resolution of 1000×2000.

[0024] Step 1-3-1, Combined reflectivity factor grayscale data cropping: For the combined reflectivity factor normalized grayscale data obtained in step 1-2-2, select the main echo areas according to the radar site location and radar detection range, and crop the normalized grayscale data to an X*Y size, where X is the width, Y is the height, 460≤X≤2000, 460≤Y≤1000;

[0025] Step 1-3-2, Vertical cumulative liquid water content and echo top height grayscale data cropping: The vertical cumulative liquid water content and echo top height grayscale data are cropped to an X*Y size. The data area retained after cropping is the same as the combined reflectivity factor grayscale data in Step 1-3-1.

[0026] Step 1-4-1, Threshold method for data labeling: For each frame of combined reflectivity factor normalized grayscale data with size X*Y, count the data value and number of elements of each frame of combined reflectivity factor normalized grayscale data, set a threshold for the data value and number of elements of echo information, and mark normalized grayscale data frames that exceed this threshold as 1, and other data as 0.

[0027] For the vertical cumulative liquid water content and echo top height normalized grayscale data, the data at the same time as the combined reflectivity factor normalized grayscale data marked as 1 are also marked as 1;

[0028] Step 1-4-2, Generate a data sequence sample set: Set the sample sequence length (sequence_length) to 20, the sliding window width (sliding_window_width) to 2 ≤ sliding_window_width ≤ 9, the step size (step_size_1) of the first 10 frames in the sample sequence to 1, and the step size (step_size_2) of the last 10 frames to 2. Use the first 10 frames as the model input sequence (input) and the last 10 frames as the label sequence (label). The sample sequence is represented as:

[0029] sequence = {input, label}

[0030] in, This indicates that the 1st to 10th data points in the input sequence are consecutive in time. The first to tenth data points in the label sequence are represented by t-9, t-8, t-7, t-6, t-5, t-4, t-3, t-2, t-1, t, t+2, t+4, t+6, t+8, t+10, t+12, t+14, t+16, t+18, and t+20, respectively representing the data points. The corresponding moment;

[0031] The 10th data in the input sequence. The first data in the label sequence. The time interval is 2; the time interval between the corresponding data of the next sequence and the previous sequence is the duration corresponding to the sliding window width (sliding_window_width); where i represents the modality category, I represents the modality set, i∈I, I={CR,VIL,TOP};

[0032] The total number of data sequences for the i-th modality is N. i Determined by the following formula:

[0033]

[0034] Where count represents the total number of data points in the normalized grayscale dataset;

[0035] For example, when the sliding window width sliding_window_width = 3, the above formula can be simplified to:

[0036]

[0037] From all the generated image sequences, the sequences containing images labeled 1 are selected as samples for the image sequence sample set;

[0038] Step 1-4-3, construct training sample set, validation sample set and test sample set: Randomly sort the image sequence sample set obtained in step 1-4-2 and divide it into training sample set, validation sample set and test sample set according to a certain ratio.

[0039] Step 2-1, Network Model Initialization: Construct the components of the ConvLSTM echo extrapolation network and define the model parameters. Use the Orthogonal method to initialize the training parameters. Train the model using the Adam optimizer. Initialize the model hyperparameters, set the learning rate (learning_rate), the number of input samples (train_batch_size) and the maximum number of training iterations (train_max_epoch) during the training phase.

[0040] Step 2-2, Multimodal data feature extraction and feature fusion: For the multimodal data of the input model, including combined reflectivity factor CR, vertical cumulative liquid water content VIL, and echo top height TOP, multimodal features are extracted through convolution operation, and then feature fusion is performed by attention-based feature concatenation.

[0041] Steps 2-3, Network Parameter Training and Update: During the model training phase, batch training is used. Each time, a training data set with dimension train_batch_size is taken from the training sample set obtained in step 1 and input into the network to train and optimize the network parameters. The parameters during the training process are denoted as the weights w and biases b in the convolution kernel. The parameters are learned through the backpropagation algorithm. After each iteration of the dataset, the current model performance is evaluated using the validation sample set, and the hyperparameter settings are optimized.

[0042] Step 2-1-1, Construct encoder C1: Encoder C1 is a convolutional layer, and the number of output feature maps of layer C1 is OutputMaps. C1 =8, OutputSize is the width of the output feature map. C1 =51, KernelSize of C1 layer convolution. C1=7, convolution stride C1 =5, padding size C1 =1;

[0043] Step 2-1-2, Construct the sequence coding layer C2: The sequence coding layer C2 consists of two ConvLSTM layers, denoted as ConvLSTM. 1 and ConvLSTM 2 ConvLSTM 1 Layers and ConvLSTM 2 Each layer contains 10 units. 10 units The data at time T correspond to respectively Where T∈{t-9,t-8,t-7,t-6,t-5,t-4,t-3,t-2,t-1,t};

[0044] With respect to time T The output of is used as Input; ConvLSTM unit kernel size Conv =3, Number of Output Feature Maps conv =64, Output feature map width OutputSize conv =51;

[0045] Step 2-1-3, Construct prediction layer P1: Prediction layer P1 consists of two ConvLSTM layers, denoted as ConvLSTM and P1 respectively. 3 and ConvLSTM 4 ConvLSTM 3 Layers and ConvLSTM 4 Each layer contains 10 units. 10 units The data at time T′ correspond to respectively Among them, T′∈{t+2,t+4,t+6,t+8,t+10,t+12,t+14,t+16,t+18,t+20};

[0046] With respect to time T′ The output of is used as Input; ConvLSTM unit kernel size Conv =3, Number of Output Feature Maps conv =64, Output feature map width OutputSize conv =51;

[0047] Step 2-1-4, Construct the state fusion layer F1: The state fusion layer F1 is a convolutional layer, and the number of input feature maps for layer F1 is InputMaps. F1 =128, kernel size F1 =1, OutputMaps F1 =8;

[0048] Step 2-1-5, Construct decoder T1: Decoder T1 is a deconvolutional layer, and the number of input feature maps in layer T1 is InputMaps. T1 =8, KernelSize of T1 layer convolution T1 =7, deconvolution stride T1 =5, OutputMaps T1 =1, padding size T1 =1, OutputSize T1 =256*256;

[0049] Step 2-1-6, Network Parameter and Hyperparameter Initialization: Orthogonally initialize all parameters in the network model; initialize the model hyperparameters, train the model using the Adam optimizer, set the learning rate learning_rate = 0.0001, beta_1 = 0.5, beta_2 = 0.999, use the StepLR learning rate decay mechanism, decay step size step_size = 10, decay factor gamma = 0.5, set the number of input samples each time in the training phase to train_batch_size, and the maximum number of training iterations to train_max_epoch, where 8 ≤ train_batch_size ≤ 32, 30 ≤ train_max_epoch ≤ 50.

[0050] Step 2-2-1, Multimodal Data Feature Extraction: For the multimodal data sample set obtained in Step 1-4-3, for each sample sequence... I = {CR, VIL, TOP}, and the combined reflectivity factor CR, vertical cumulative liquid water content VIL, and echo top height data feature TOP are extracted through the convolution operation of encoder C1 in step 2-1-1 to obtain the feature sequence. Encoder C1 inputs sample frames With corresponding convolution kernel Convolution, the result of which is then subjected to a bias parameter. After activation by the ReLU function, the output encoded features are obtained. The calculation formula is as follows:

[0051]

[0052] in, For the input sample frame, To output encoded features, For convolution kernel, Here are the bias parameters, * indicates the convolution operation, and ReLU(·) is the modified linear unit activation function;

[0053] Let the characteristic sequences of the combined reflectivity factor be denoted as follows: Let the characteristic sequences of vertical cumulative liquid water content and echo top height be denoted as follows: Where M = {CR} and N = {VIL, TOP};

[0054] Step 2-2-2, Feature Fusion: The feature fusion strategy adopts attention-based feature concatenation: based on the attention module CBAM, the combined reflectivity factor feature sequence is... Compared with the characteristic sequence of vertical cumulative liquid water content or the characteristic sequence of echo top height The feature maps are then stitched together after being weighted by the channel attention unit (CAM) and the spatial attention unit (SAM).

[0055] Step 2-2-2-1, Calculate channel attention: Let the combined reflectivity factor feature sequence be... Let the characteristic sequences of vertical cumulative liquid water content and echo top height be denoted as follows: For single-frame feature maps and The dimensions are H×W×C, where H is the height, W is the width, and C is the number of channels. Spatial average pooling and max pooling are performed respectively, denoted as the spatial average pooling of the combined reflectivity factor. Spatial max pooling of combined reflectivity factors Vertical accumulation of liquid water and spatial average pooling of echo top height Vertical accumulation of liquid water and spatial maximum pooling of echo top height Each yields two feature maps of size 1×1×C;

[0056] The two feature maps are then fed into a two-layer fully connected network, with shared parameters denoted as W0 and W1. During calculation, the number of feature map channels is first compressed to 1 / r of the original number, where, Let C be the set of real numbers, C be the number of channels, and r be the reduction rate. After passing through the ReLU(·) activation function, two activated results are obtained. Then, the feature map is expanded back to the original number of channels. The two outputs are added element-wise, and then passed through the Sigmoid(·) function to generate channel attention weight coefficients. Finally, these are multiplied by the original feature map to obtain the channel attention-based feature map. The combined reflectance factor feature map is denoted as... Vertical cumulative liquid water content and echo top height feature map are as follows The calculation process is expressed as follows:

[0057]

[0058]

[0059] Where m∈M, n∈N, and ⊙ is the Hadamard product;

[0060] Step 2-2-2-2, Calculate spatial attention: For the single-frame feature map obtained in step 2-2-2-1 and Perform average pooling and max pooling along the channel dimension separately, denoted as the channel average pooling of the combined reflectivity factor. Channel max pooling of combined reflectivity factors Vertical accumulation of liquid water content and channel-averaged pooling of echo top height Vertical accumulation of liquid water content and echo top height channel maximum pooling The number of channels in the feature map is reduced to 1, and the size becomes H×W×1; then the feature maps are concatenated along the channel dimension, denoted as... Two feature maps of size H×W×2 are obtained from each feature map. These two feature maps are then input into a convolutional layer with a kernel size of 3×3. The output feature map size remains unchanged, but the number of channels is reduced to 1. The convolution calculation process is denoted as follows: Two feature maps of size H×W×1 are obtained. Finally, the two feature maps are processed by the Sigmoid(·) function to generate spatial attention weight coefficients, which are then multiplied with the original feature maps to obtain spatial attention-based feature maps. The combined reflectance factor feature maps are denoted as follows: Vertical cumulative liquid water content and echo top height feature map are as follows The calculation process is expressed as follows:

[0061]

[0062]

[0063] Step 2-2-2-3, Feature Concatenation: Combine the two feature maps obtained based on attention. By splicing, a fused feature map S is obtained. [m；n] The calculation process is expressed as follows:

[0064]

[0065] [] indicates that the feature maps of two modal data are spliced ​​together. The length of the fused feature sequence and the size of the feature map remain unchanged, and the number of fused feature channels is expanded to twice the original.

[0066] Step 2-3-1, Train the network using fused features: Input the training sample set data obtained in Step 1-4 into the network model. For a multimodal sample sequence of length = 20... i represents the sample number, and the data from the first 10 frames. The training data is used for feature extraction and feature fusion, and is input into the convolutional long short-term memory model in step 2-1-2. The model outputs 10 frames of prediction data in the prediction layer P1 of step 2-1-3. Select the combined reflectivity factor data from the last 10 frames of data. Label data is used to calculate the loss function and supervise model training;

[0067] Step 2-3-2, Calculate the loss: Loss function F loss Using mean square error (MSE) loss, the combined reflectivity factor prediction data output from decoder T1 in step 2-3-1 is selected. With tag data The loss is calculated using the following formula:

[0068]

[0069] Where 0 ≤ i < length, i is the sample number, and length = 20 is the sample sequence length;

[0070] Step 2-3-3, Backpropagation of error to update parameters: based on the loss function The result is that automatic differentiation is used to achieve gradient backpropagation and update network parameters;

[0071] Steps 2-3-4, Optimize the selection of network hyperparameters: Based on the results of the model on the validation sample set during training, select the hyperparameters of the model that perform best as the final hyperparameters of the network model;

[0072] Step 2-3-1 includes the following steps:

[0073] Step 2-3-1-1, Sequence coding layer C2 processing: Sequence coding layer C2 = {ConvLSTM} 1 ConvLSTM 2The data at time T corresponds to ConvLSTM. 1 Layer unit Based on the input sample frame data Δ T ConvLSTM corresponding to time T-1 1 Layer unit The memory unit and hidden state are updated over time, and the hidden state is output.

[0074] ConvLSTM at time T 2 Layer unit The data is from ConvLSTM at time T. 1 Hidden state of input ConvLSTM at time T-1 2 Layer unit The memory unit and the hidden state are jointly determined. It updates its memory cells and hidden state over time and outputs the hidden state; when T = t-9, the memory cells and hidden state of the previous time T-1 are both 0; the update process is controlled by input gate i. T Forgotten Gate T Output gate o T Control, where input gate i T Control input and the addition of the previously hidden state, forget gate f T The output gate determines the degree of forgetting of memory cells from the previous moment. T Adjust the memory cell at that moment to output the hidden state at that moment;

[0075] The feature δ output by encoder C1 T δ T ∈{δ t-9:t} as input, output is Hidden state The forward propagation calculation formula is as follows:

[0076]

[0077]

[0078]

[0079]

[0080]

[0081] Where σ is the sigmoid(·) activation function, tanh(·) is the hyperbolic tangent activation function, * denotes convolution operation, and ° denotes the Hadamard product; δ Tyes Input, yes memory units, yes The hidden state, They are Input gate, forget gate, output gate; Input gate The parameters are respectively related to δ T Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Forgotten Gate The parameters are respectively related to δ T Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Output gate The parameters are respectively related to δ T Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias memory unit The parameters are respectively related to δ T Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias

[0082] Will Hidden state As input, The forward propagation calculation formula is as follows:

[0083]

[0084]

[0085]

[0086]

[0087]

[0088] in, Hidden state yes Input, yes memory units, yes The hidden state, They are Input gate, forget gate, output gate; Input gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Forgotten Gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Output gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias memory unit The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias

[0089] Step 2-3-1-2, Prediction layer P1 processing: Prediction layer P1 = {ConvLSTM} 3 ConvLSTM 4 ConvLSTM at time T′ 3 Layer unit The data is from ConvLSTM at time T′-2. 3 Layer unit The memory units and hidden states are determined. Update its memory unit and hidden state over time, and output the hidden state, where T′-2∈{t+2,t+4,t+6,t+8,t+10,t+12,t+14,t+16,t+18;

[0090] ConvLSTM at time T′ 4 Layer unit The data is from ConvLSTM at time T′ 3 Hidden state of input ConvLSTM corresponding to time T′-2 4 Layer unit The memory unit and the hidden state are jointly determined. It updates its memory units and hidden state over time and outputs the hidden state;

[0091] When T′=t+2, the memory cells and hidden state of the previous time T′-2 are the memory cells and hidden state of time T=t; the update process is controlled by input gate i T Forgotten Gate T Output gate o T Control, where input gate i T Control input and the addition of the previously hidden state, forget gate f T The output gate determines the degree of forgetting of memory cells from the previous moment. T Adjust the memory cell at that moment to output the hidden state at that moment;

[0092] The output is Hidden state The forward propagation calculation formula is as follows:

[0093]

[0094]

[0095]

[0096]

[0097]

[0098] in, yes memory units, yes The hidden state, They are Input gate, forget gate, output gate; Input gate The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations bias Forgotten Gate The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations bias Output gate The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations bias memory unit The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations bias

[0099] Will Hidden state As input, Hidden state As output The forward propagation calculation formula is as follows:

[0100]

[0101]

[0102]

[0103]

[0104]

[0105] in, Hidden state yes Input, yes memory units, yes The hidden state, They are Input gate, forget gate, output gate; Input gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Forgotten Gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Output gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias memory unit The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias

[0106] Step 2-3-1-3, State fusion layer F1 processing: The input F1 of the state fusion layer is the hidden state sequence output by the prediction layer P1. The number of channels is compressed through 1*1 convolution, and the output is a channel-compressed feature map sequence.

[0107] Steps 2-3-1-4, Decoder T1 processing: The input to decoder T1 is the predicted feature sequence output by the state fusion layer F1. Extract the CR portion from the predicted feature sequence With corresponding convolution kernel Perform deconvolution operation, and add the bias parameter to the convolution result. After activation by the ReLU(·) function, the output is the predicted echo sequence. The calculation formula is as follows:

[0108]

[0109] in, For the input feature frame, The output is the predicted echo frame. For convolution kernel, The bias parameter is used to obtain the output of the decoder after each input feature frame is processed by the corresponding decoder. That is, predicting the echo sequence.

[0110] Beneficial effects: The method of this invention is based on a multimodal radar echo extrapolation network. By fusing the features of multiple radar modal data, it explores the correlation and complementarity between multiple modes, trains on the radar echo dataset, and uses the trained network to perform radar echo extrapolation, which effectively improves the accuracy and forecast timeliness of radar echo extrapolation.

[0111] Specifically, the present invention has the following advantages compared with existing methods: 1. High extrapolation accuracy. Compared with other methods, the radar echo extrapolation method based on multi-mode in the present invention can effectively memorize the nonlinear motion, generation and disappearance evolution of the echo, etc., and has higher accuracy; 2. Long extrapolation timeliness. The extrapolation timeliness can be further improved by adjusting the network structure to increase the number of predicted sequence output data frames. Attached Figure Description

[0112] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention as described above or otherwise will become clearer.

[0113] Figure 1 This is a flowchart of the present invention.

[0114] Figure 2 The initialization model diagram for a multimodal radar echo extrapolation network is shown.

[0115] Figure 3 This is a structural diagram of encoder C1.

[0116] Figure 4 This is a schematic diagram of the ConvLSTM echo extrapolation network.

[0117] Figure 5 This is a diagram of the ConvLSTM cell structure.

[0118] Figure 6 This is a diagram of the channel attention unit (CAM) structure.

[0119] Figure 7 This is a structural diagram of the Spatial Attention Unit (SAM).

[0120] Figure 8 This is the F1 model for the state fusion layer.

[0121] Figure 9 This is a structural diagram of decoder T1. Detailed Implementation

[0122] This invention discloses a method for training a depth-spatial-prediction neural network for radar echo extrapolation, such as... Figure 1 As shown, it includes the following steps:

[0123] Step 1, Radar base data preprocessing: Read the radar base data of combined reflectivity factor CR, vertical cumulative liquid water content VIL, and echo top height TOP and perform data preprocessing. The normalized grayscale data obtained after data processing is cropped and labeled, and divided into training sample set, validation sample set and test sample set.

[0124] Step 2, train the radar echo extrapolation model: initialize the network model, extract multimodal data features from the combined reflectivity factor CR, vertical cumulative liquid water content VIL, and echo top height TOP sample set obtained in Step 1 and perform feature fusion, use the fused features to train the ConvLSTM network, and save the network parameters after training.

[0125] Step 3, test the model: input the data from the test sample set into the network model trained in Step 2, and finally obtain the predicted radar echo image.

[0126] Step 1-1, Read radar base data: According to the weather radar base data format, read and decode the base data. The data starts from the top left corner. Write the data according to the column number increasing with longitude and the row number decreasing with latitude to obtain data blocks. According to the data layer number and latitude and longitude coordinates in the file header, convert the data blocks into meteorological element regular grid data. Store the decoded data file according to the file header and data blocks respectively.

[0127] Step 1-2, Radar base data preprocessing: Read the data block from Step 1-1, convert the stored value into the actual value of meteorological element according to the conversion formula, map it to a gray value range, and then perform normalization processing to obtain normalized gray value data;

[0128] Steps 1-3, Data Product Processing: For the normalized grayscale data obtained in Steps 1-2, select the main echo area, crop the normalized grayscale data to the same size, and perform the same processing on the echo top height and vertical cumulative liquid water content data at the corresponding time.

[0129] Steps 1-4, Dataset Partitioning: For the normalized grayscale data obtained in Steps 1-3 after size cropping, set pixel thresholds according to the actual meteorological element values ​​and ranges, label the data and filter out data sequences that meet the conditions, and generate a data sample set according to a certain time series length and sliding window width. Divide the sample set into a training sample set, a validation sample set and a test sample set according to a certain ratio.

[0130] Step 1-2-1, Reading data blocks and data conversion: Read the data blocks from Step 1-1, and convert the stored values ​​of the data blocks into actual values ​​according to the record value conventions in the radar base data format specification file;

[0131] Step 1-2-2, Grayscale value mapping: The actual radar mosaic covers a latitude and longitude range of 20°*10°, with a longitude resolution of 0.01° and a latitude resolution of 0.01°. Based on this, the data block shape in Step 1-2-1 is reshaped to (1000, 2000). Then, the data block values ​​are mapped to a grayscale value range of 0 to 255 to obtain grayscale data P. This is then normalized to obtain normalized grayscale data P′. The grayscale mapping formula is as follows:

[0132]

[0133]

[0134]

[0135] The normalization formula is as follows:

[0136]

[0137] Among them, V CR V VIL V TOP P represents the actual value of the combined reflectivity factor, the actual value of the vertical cumulative liquid water content, and the actual value of the echo top height, respectively. CR P VIL P TOPThese represent the combined reflectance grayscale value, vertical cumulative liquid water content grayscale value, and echo top height grayscale value after data mapping, respectively. P represents the grayscale data, and P′ represents the normalized grayscale data. Indicates rounding down;

[0138] The final normalized grayscale data P′ has a feature number of 1 and a resolution of 1000×2000.

[0139] Step 1-3-1, Combined reflectivity factor grayscale data cropping: For the combined reflectivity factor normalized grayscale data obtained in step 1-2-2, select the main echo areas according to the radar site location and radar detection range, and crop the normalized grayscale data to an X*Y size, where X is the width, Y is the height, 460≤X≤2000, 460≤Y≤1000;

[0140] Step 1-3-2, Vertical cumulative liquid water content and echo top height grayscale data cropping: The vertical cumulative liquid water content and echo top height grayscale data are cropped to an X*Y size. The data area retained after cropping is the same as the combined reflectivity factor grayscale data in Step 1-3-1.

[0141] Step 1-4-1, Threshold method for data labeling: For each frame of combined reflectivity factor normalized grayscale data with size X*Y, count the data value and number of elements of each frame of combined reflectivity factor normalized grayscale data, set a threshold for the data value and number of elements of echo information, and mark normalized grayscale data frames that exceed this threshold as 1, and other data as 0.

[0142] For the vertical cumulative liquid water content and echo top height normalized grayscale data, the data at the same time as the combined reflectivity factor normalized grayscale data marked as 1 are also marked as 1;

[0143] Step 1-4-2, Generate a data sequence sample set: Set the sample sequence length `sequence_length` = 20, the sliding window width `sliding_window_with`, where 2 ≤ `sliding_window_width` ≤ 9, the step size `step_size_1` = 1 for the first 10 frames of the sample sequence, and the step size `step_size_2` = 2 for the last 10 frames of the sample sequence. Use the first 10 frames of data as the model input sequence `input`, and the last 10 frames of data as the label sequence `label`. The sample sequence is represented as:

[0144] sequence = {input, label}

[0145] in, This indicates that the 1st to 10th data points in the input sequence are consecutive in time. The first to tenth data points in the label sequence are represented by t-9, t-8, t-7, t-6, t-5, t-4, t-3, t-2, t-1, t, t+2, t+4, t+6, t+8, t+10, t+12, t+14, t+16, t+18, and t+20, respectively representing the data points. The corresponding moment;

[0146] The 10th data in the input sequence. The first data in the label sequence. The time interval is 2; the time interval between the corresponding data of the next sequence and the previous sequence is the duration corresponding to the sliding window width (sliding_window_width); where i represents the modality category, I represents the modality set, i∈I, I={CR,VIL,TOP};

[0147] The total number of data sequences for the i-th modality is N. i Determined by the following formula:

[0148]

[0149] Where count represents the total number of data points in the normalized grayscale dataset;

[0150] For example, when the sliding window width sliding_window_width = 3, the above formula can be simplified to:

[0151]

[0152] From all the generated image sequences, the sequences containing images labeled 1 are selected as samples for the image sequence sample set;

[0153] Step 1-4-3, construct training sample set, validation sample set and test sample set: Randomly sort the image sequence sample set obtained in step 1-4-2 and divide it into training sample set, validation sample set and test sample set according to a certain ratio.

[0154] Step 2-1, Network Model Initialization: Construct the components of the ConvLSTM echo extrapolation network and define the model parameters. Use the Orthogonal method to initialize the training parameters. Train the model using the Adam optimizer. Initialize the model hyperparameters, set the learning rate (learning_rate), the number of input samples (train_batch_size) and the maximum number of training iterations (train_max_epoch) during the training phase.

[0155] Step 2-2, Multimodal data feature extraction and feature fusion: For the multimodal data of the input model, including combined reflectivity factor CR, vertical cumulative liquid water content VIL, and echo top height TOP, multimodal features are extracted through convolution operation, and then feature fusion is performed by attention-based feature concatenation.

[0156] Steps 2-3, Network Parameter Training and Update: During the model training phase, batch training is used. Each time, a training data set with dimension train_batch_size is taken from the training sample set obtained in step 1 and input into the network to train and optimize the network parameters. The parameters during the training process are denoted as the weights w and biases b in the convolution kernel. The parameters are learned through the backpropagation algorithm. After each iteration of the dataset, the current model performance is evaluated using the validation sample set, and the hyperparameter settings are optimized.

[0157] Step 2-1-1, Construct encoder C1: Encoder C1 is a convolutional layer, and the number of output feature maps of layer C1 is OutputMaps. C1 =8, OutputSize is the width of the output feature map. C1 =51, KernelSize of C1 layer convolution. C1 =7, convolution stride C1 =5, padding size C1 =1;

[0158] Step 2-1-2, Construct the sequence coding layer C2: The sequence coding layer C2 consists of two ConvLSTM layers, denoted as ConvLSTM. 1 and ConvLSTM 2 ConvLSTM 1 Layers and ConvLSTM 2 Each layer contains 10 units. 10 units The data at time T correspond to respectively Where T∈{t-9,t-8,t-7,t-6,t-5,t-4,t-3,t-2,t-1,t};

[0159] With respect to time T The output of is used as Input; ConvLSTM unit kernel size Conv =3, Number of Output Feature Maps conv =64, Output feature map width OutputSize conv =51;

[0160] Step 2-1-3, Construct prediction layer P1: Prediction layer P1 consists of two ConvLSTM layers, denoted as ConvLSTM and P1 respectively. 3 and ConvLSTM 4 ConvLSTM 3 Layers and ConvLSTM 4 Each layer contains 10 units. 10 units The data at time T′ correspond to respectively Among them, T′∈{t+2,t+4,t+6,t+8,t+10,t+12,t+14,t+16,t+18,t+20};

[0161] With respect to time T′ The output of is used as Input; ConvLSTM unit kernel size Conv =3, Number of Output Feature Maps conv =64, Output feature map width OutputSize conv =51;

[0162] Step 2-1-4, Construct the state fusion layer F1: The state fusion layer F1 is a convolutional layer, and the number of input feature maps for layer F1 is InputMaps. F1 =128, kernel size F1 =1, OutputMaps F1 =8;

[0163] Step 2-1-5, Construct decoder T1: Decoder T1 is a deconvolutional layer, and the number of input feature maps in layer T1 is InputMaps. T1 =8, KernelSize of T1 layer convolution T1 =7, deconvolution stride T1 =5, OutputMaps T1 =1, padding size T1 =1, OutputSize T1 =256*256;

[0164] Step 2-1-6, Network Parameter and Hyperparameter Initialization: Orthogonally initialize all parameters in the network model; initialize the model hyperparameters, train the model using the Adam optimizer, set the learning rate learning_rate = 0.0001, beta_1 = 0.5, beta_2 = 0.999, use the StepLR learning rate decay mechanism, decay step size step_size = 10, decay factor gamma = 0.5, set the number of input samples each time in the training phase to train_batch_size, and the maximum number of training iterations to train_max_epoch, where 8 ≤ train_batch_size ≤ 32, 30 ≤ train_max_epoch ≤ 50.

[0165] Step 2-2-1, Multimodal Data Feature Extraction: For the multimodal data sample set obtained in Step 1-4-3, for each sample sequence... I = {CR, VIL, TOP}, and the combined reflectivity factor CR, vertical cumulative liquid water content VIL, and echo top height data feature TOP are extracted through the convolution operation of encoder C1 in step 2-1-1 to obtain the feature sequence. Encoder C1 inputs sample frames With corresponding convolution kernel Convolution, the result of which is then subjected to a bias parameter. After activation by the ReLU function, the output encoded features are obtained. The calculation formula is as follows:

[0166]

[0167] in, For the input sample frame, To output encoded features, For convolution kernel, Here, · represents the bias parameter, · denotes the convolution operation, and ReLU(·) is the modified linear unit activation function;

[0168] Let the characteristic sequences of the combined reflectivity factor be denoted as follows: Let the characteristic sequences of vertical cumulative liquid water content and echo top height be denoted as follows: Where M = {CR} and N = {VIL, TOP};

[0169] Step 2-2-2, Feature Fusion: The feature fusion strategy adopts attention-based feature concatenation: based on the attention module CBAM, the combined reflectivity factor feature sequence is... Compared with the characteristic sequence of vertical cumulative liquid water content or the characteristic sequence of echo top height The feature maps are then stitched together after being weighted by the channel attention unit (CAM) and the spatial attention unit (SAM).

[0170] Step 2-2-2-1, Calculate channel attention: Let the combined reflectivity factor feature sequence be... Let the characteristic sequences of vertical cumulative liquid water content and echo top height be denoted as follows: For single-frame feature maps and The dimensions are H×W×C, where H is the height, W is the width, and C is the number of channels. Spatial average pooling and max pooling are performed respectively, denoted as the spatial average pooling of the combined reflectivity factor. Spatial max pooling of combined reflectivity factors Vertical accumulation of liquid water and spatial average pooling of echo top height Vertical accumulation of liquid water and spatial maximum pooling of echo top height Each yields two feature maps of size 1×1×C;

[0171] The two feature maps are then fed into a two-layer fully connected network, with shared parameters denoted as W0 and W1. During calculation, the number of feature map channels is first compressed to 1 / r of the original number, where, Let C be the set of real numbers, C be the number of channels, and r be the reduction rate. After passing through the ReLU(·) activation function, two activated results are obtained. Then, the feature map is expanded back to the original number of channels. The two outputs are added element-wise, and then passed through the Sigmoid(·) function to generate channel attention weight coefficients. Finally, these are multiplied by the original feature map to obtain the channel attention-based feature map. The combined reflectance factor feature map is denoted as... Vertical cumulative liquid water content and echo top height feature map are as follows The calculation process is expressed as follows:

[0172]

[0173]

[0174] Where m∈M, n∈N, and ⊙ is the Hadamard product;

[0175] Step 2-2-2-2, Calculate spatial attention: For the single-frame feature map obtained in step 2-2-2-1 and Perform average pooling and max pooling along the channel dimension separately, denoted as the channel average pooling of the combined reflectivity factor. Channel max pooling of combined reflectivity factors Vertical accumulation of liquid water content and channel-averaged pooling of echo top height Vertical accumulation of liquid water content and echo top height channel maximum pooling The number of channels in the feature map is reduced to 1, and the size becomes H×W×1; then the feature maps are concatenated along the channel dimension, denoted as... Two feature maps of size H×W×2 are obtained from each feature map. These two feature maps are then input into a convolutional layer with a kernel size of 3×3. The output feature map size remains unchanged, but the number of channels is reduced to 1. The convolution calculation process is denoted as follows: Two feature maps of size H×W×1 are obtained. Finally, the two feature maps are processed by the Sigmoid(·) function to generate spatial attention weight coefficients, which are then multiplied with the original feature maps to obtain spatial attention-based feature maps. The combined reflectance factor feature maps are denoted as follows: Vertical cumulative liquid water content and echo top height feature map are as follows The calculation process is expressed as follows:

[0176]

[0177]

[0178] Step 2-2-2-3, Feature Concatenation: Combine the two feature maps obtained based on attention. By splicing, a fused feature map S is obtained. [m；n] The calculation process is expressed as follows:

[0179]

[0180] [] indicates that the feature maps of two modal data are spliced ​​together. The length of the fused feature sequence and the size of the feature map remain unchanged, and the number of fused feature channels is expanded to twice the original.

[0181] Step 2-3-1, Train the network using fused features: Input the training sample set data obtained in Step 1-4 into the network model. For a multimodal sample sequence of length = 20... i represents the sample number, and the data from the first 10 frames. The training data is used for feature extraction and feature fusion, and is input into the convolutional long short-term memory model in step 2-1-2. The model outputs 10 frames of prediction data in the prediction layer P1 of step 2-1-3. Select the combined reflectivity factor data from the last 10 frames of data. Label data is used to calculate the loss function and supervise model training;

[0182] Step 2-3-2, Calculate the loss: Loss function F lossUsing mean square error (MSE) loss, the combined reflectivity factor prediction data output from decoder T1 in step 2-3-1 is selected. With tag data The loss is calculated using the following formula:

[0183]

[0184] Where 0 ≤ i < length, i is the sample number, and length = 20 is the sample sequence length;

[0185] Step 2-3-3, Backpropagation of error to update parameters: based on the loss function The result is that automatic differentiation is used to achieve gradient backpropagation and update network parameters;

[0186] Steps 2-3-4, Optimize the selection of network hyperparameters: Based on the results of the model on the validation sample set during training, select the hyperparameters of the model that perform best as the final hyperparameters of the network model;

[0187] Step 2-3-1 includes the following steps:

[0188] Step 2-3-1-1, Sequence coding layer C2 processing: Sequence coding layer C2 = {ConvLSTM} 1 ConvLSTM 2 The data at time T corresponds to ConvLSTM. 1 Layer unit Based on the input sample frame data Δ T ConvLSTM corresponding to time T-1 1 Layer unit The memory unit and hidden state are updated over time, and the hidden state is output.

[0189] ConvLSTM at time T 2 Layer unit The data is from ConvLSTM at time T. 1 Hidden state of input ConvLSTM at time T-1 2 Layer unit The memory unit and the hidden state are jointly determined. It updates its memory cells and hidden state over time and outputs the hidden state; when T = t-9, the memory cells and hidden state of the previous time T-1 are both 0; the update process is controlled by input gate i. T Forgotten Gate T Output gate o T Control, where input gate i T Control input and the addition of the previously hidden state, forget gate f TThe output gate determines the degree of forgetting of memory cells from the previous moment. T Adjust the memory cell at that moment to output the hidden state at that moment;

[0190] The feature W output by encoder C1 T δ T ∈{δ t-9:t} as input, output is Hidden state The forward propagation calculation formula is as follows:

[0191]

[0192]

[0193]

[0194]

[0195]

[0196] Where σ is the sigmooid (·) activation function, tanh (·) is the hyperbolic tangent activation function, * denotes convolution operation, and ° denotes the Hadamard product; δ T yes Input, yes memory units, yes The hidden state, They are Input gate, forget gate, output gate; Input gate The parameters are respectively related to δ T Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Forgotten Gate The parameters are respectively related to δ T Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Output gate The parameters are respectively related to δ T Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias memory unit The parameters are respectively related to δT Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias

[0197] Will Hidden state As input, The forward propagation calculation formula is as follows:

[0198]

[0199]

[0200]

[0201]

[0202]

[0203] in, Hidden state yes Input, yes memory units, yes The hidden state, They are Input gate, forget gate, output gate; Input gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Forgotten Gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Output gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias memory unit The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias

[0204] Step 2-3-1-2, Prediction layer P1 processing: Prediction layer P1 = {ConvLSTM} 3 ConvLSTM 4 ConvLSTM at time T′ 3 Layer unit The data is from ConvLSTM at time T′-2. 3 Layer unit The memory units and hidden states are determined. Update its memory unit and hidden state over time, and output the hidden state, where T′-2∈{t+2,t+4,t+6,t+8,t+10,t+12,t+14,t+16,t+18;

[0205] ConvLSTM at time T′ 4 Layer unit The data is from ConvLSTM at time T′ 3 Hidden state of input ConvLSTM corresponding to time T′-2 4 Layer unit The memory unit and the hidden state are jointly determined. It updates its memory units and hidden state over time and outputs the hidden state;

[0206] When T′=t+2, the memory cells and hidden state of the previous time T′-2 are the memory cells and hidden state of time T=t; the update process is controlled by input gate i T Forgotten Gate T Output gate o T Control, where input gate i T Control input and the addition of the previously hidden state, forget gate f T The output gate determines the degree of forgetting of memory cells from the previous moment. T Adjust the memory cell at that moment to output the hidden state at that moment;

[0207] The output is Hidden state The forward propagation calculation formula is as follows:

[0208]

[0209]

[0210]

[0211]

[0212]

[0213] in, yes memory units, yes The hidden state, They are Input gate, forget gate, output gate; Input gate The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations bias Forgotten Gate The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations bias Output gate The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations bias memory unit The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations bias

[0214] Will Hidden state As input, Hidden state As output The forward propagation calculation formula is as follows:

[0215]

[0216]

[0217]

[0218]

[0219]

[0220] in, Hidden state yes Input, yes memory units, yes The hidden state, They are Input gate, forget gate, output gate; Input gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Forgotten Gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias Output gate The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias memory unit The parameters are respectively with Convolution kernel for performing convolution calculations With hidden state Convolution kernel for performing convolution calculations bias

[0221] Step 2-3-1-3, State fusion layer F1 processing: The input F1 of the state fusion layer is the hidden state sequence output by the prediction layer P1. The number of channels is compressed through 1*1 convolution, and the output is a channel-compressed feature map sequence.

[0222] Steps 2-3-1-4, Decoder T1 processing: The input to decoder T1 is the predicted feature sequence output by the state fusion layer F1. Extract the CR portion from the predicted feature sequence With corresponding convolution kernel Perform deconvolution operation, and add the bias parameter to the convolution result. After activation by the ReLU(·) function, the output is the predicted echo sequence. The calculation formula is as follows:

[0223]

[0224] in, For the input feature frame, The output is the predicted echo frame. For convolution kernel, The bias parameter is used to obtain the output of the decoder after each input feature frame is processed by the corresponding decoder. That is, predicting the echo sequence.

[0225] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding a multimodal radar echo extrapolation method and some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0226] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0227] This invention provides a multi-mode radar echo extrapolation method. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A multi-modal based radar return extrapolation method, characterized in that, The method comprises the following steps: Step 1, radar-based data preprocessing: reading radar-based data of combined reflectivity factor CR, vertical integrated liquid water content VIL and echo top height TOP and performing data preprocessing, cropping and marking normalized gray data obtained after data processing, and dividing into a training sample set, a verification sample set and a test sample set; Step 2, training of radar echo extrapolation model: the radar echo extrapolation model comprises an encoder C1, an attention module CBAM, a ConvLSTM echo extrapolation network and a decoder T1; The encoder C1 is used for feature extraction, and the attention module CBAM comprises a channel attention unit CAM and a spatial attention unit SAM; The ConvLSTM echo extrapolation network comprises a sequence encoding layer, a prediction layer and a state fusion layer; The network model is initialized, multi-modal data features of the combined reflectivity factor CR, the vertical integrated liquid water content VIL, and the echo top height TOP sample set obtained in step 1 are extracted and fused, and the feature fusion includes: the feature fusion strategy adopts attention-based feature splicing: based on the CBAM attention module, the combined reflectivity factor feature sequence and the vertical integrated liquid water content feature sequence or the echo top height feature sequence are respectively subjected to attention weighting through the channel attention unit CAM and the spatial attention unit SAM, and then the feature maps are spliced; the ConvLSTM network is trained using the fused features, and the network parameters are saved after the training is completed; Step 3, model testing: inputting data in the test sample set into the network model trained in step 2 to finally obtain a predicted radar echo image.

2. The method of claim 1, wherein, Step 1 comprises the following steps: Step 1-1, reading radar-based data: according to the weather radar-based data format, reading the base data and decoding, the data starting point starts from the upper left corner, the data is written according to the column sequence number increasing with the increase of longitude and the row sequence number decreasing with the decrease of latitude, a data block is obtained, the data block is converted into regular grid data of meteorological elements according to the data layer number and the longitude and latitude coordinates in the file header, and the decoded data file is stored according to the file header and the data block; Step 1-2, radar-based data preprocessing: reading the data block in step 1-1, converting the stored value into the actual value of the meteorological element according to the conversion formula, and mapping it to a gray value interval, then performing normalization processing to obtain normalized gray data; Step 1-3, data product processing: for the normalized gray data obtained in step 1-2, the main echo area is selected, the normalized gray data is cropped to the same size, and the echo top height and vertical integrated liquid water content data at the corresponding time are processed in the same way; Step 1-4, data set division: for the normalized gray data after size cropping obtained in step 1-3, a pixel threshold is set according to the actual meteorological element value size and range, the data is marked and the data sequence meeting the conditions is selected, and a data sample set is generated according to a certain time sequence length and sliding window width, and the sample set is divided into a training sample set, a verification sample set and a test sample set according to a certain proportion.

3. The method of claim 2, wherein, Step 1-4 comprises the following steps: Step 1-4-1, threshold method marking data: combining reflectivity factor normalized gray data for each frame, size is , statistics of data value and element number of each frame of combined reflectivity factor normalized gray data, setting threshold value of data value and element number of echo information, marking the normalized gray data frame exceeding the threshold value as 1, and marking other data as 0; For the vertical integrated liquid water content and echo top height normalized gray data, the data at the same time as the combined reflectivity factor normalized gray data marked as 1 is also marked as 1; Step 1-4-2, generate data sequence sample set: set sample sequence length , sliding window width , step of first 10 frames in sample sequence , step of last 10 frames in sample sequence , first 10 frames of data as model input sequence , last 10 frames of data as label sequence , sample sequence sequence is represented as: ， wherein, , , , , , , , , , , , denote the 1st to 10th data in the input sequence, which are continuous in time; , , , , , , , , , denote the 1st to 10th data in the label sequence, which are spaced by 2 in time; , , , , , , , , , , , , , , , , , , , denote the data , , , , , , , , , , , , , , , , , , , corresponding time points; Input sequence The 10th data in the middle And the label sequence The 1st data in the middle The time interval is 2; the time interval of the corresponding data of the previous sequence and the next sequence is the sliding window width The corresponding duration; wherein, Indicates the modal category, Indicates the modal set, , ; the total number of modal data sequences is determined by the equation:​ ， wherein represents the total number of data in the normalized gray scale data set; From all the generated image sequences, the sequences containing images marked as 1 are selected as samples of the image sequence sample set.

4. The method of claim 3, wherein, Step 2 comprises the following steps: Step 2-1, network model initialization: construct the components of the ConvLSTM echo extrapolation network and define the model parameters, initialize the training parameters using the Orthogonal method, train the model using the Adam optimizer, initialize the model hyperparameters, and set the learning rate , the number of samples input each time in the training stage , the maximum number of iterations for training ; Step 2-2, multi-modal data feature extraction and feature fusion: multi-modal features are extracted from the multi-modal data including combined reflectivity factor CR, vertical integrated liquid water content VIL and echo top height TOP input into the model through convolution operation, and then feature fusion is performed through attention-based feature splicing; Step 2-3, network parameter training and updating: in the model training stage, the batch training method is adopted, and each time a training data with a dimension of is taken from the training sample set obtained in step 1 to input the network, and the network parameters are trained and optimized. The parameters in the training process are recorded as the weights and the bias in the convolution kernel, and the parameter learning is performed through the error back propagation algorithm; when the data set is iterated for one round, the current model effect is evaluated using the verification sample set, and the hyperparameter setting is optimized.

5. The method of claim 4, wherein, Step 2-1 comprises the following steps: Step 2-1-1, constructing the encoder C1: the encoder C1 is a convolutional layer, the number of output feature maps of the C1 layer , the width of the output feature map , the size of the convolution kernel of the C1 layer , the convolution step , the size of the padding zero ; Step 2-1-2, constructing sequence coding layer C2: sequence coding layer C2 includes two layers in front and back, respectively denoted as and , layer and layer each contains 10 units , 10 units , data at time T respectively corresponds to , , wherein ;​ the output of time T as the input of time T+1; unit convolution kernel size output feature map number = 64, output feature map width ;​​ Step 2-1-3, constructing the prediction layer P1: the prediction layer P1 includes two layers in front and back, respectively denoted as , , , wherein ;​​​​​​ the output at time is taken as the input of the output at time ; unit convolution kernel size , output feature map number = 64, output feature map width 51;​ Step 2-1-4, constructing a state fusion layer F1: the state fusion layer F1 is a convolutional layer, the number of input feature maps of the F1 layer is , the convolution kernel size , the number of output feature maps ; Step 2-1-5, constructing decoder T1: the decoder T1 is a deconvolution layer, the input feature map number of the T1 layer is , the deconvolution kernel size of the T1 layer is , the deconvolution step is , the output image number is , the padding zero size is , the output data size is * 256.

6. The method of claim 5, wherein, Step 2-2 includes the following steps: Step 2-2-1, multi-modal data feature extraction: for the multi-modal data sample set obtained in step 1-4-3, the combined reflectivity factor CR, the vertical integrated liquid water content VIL and the echo top height data feature TOP are extracted respectively for each sample sequence , by the convolution operation of the encoder C1 in step 2-1-1, to obtain a feature sequence ; the encoder C1 convolves the input sample frame with the corresponding convolution kernel , adds a bias parameter to the convolution result, and then passes it through a activation function to output the encoded feature , and the calculation formula is as follows: )， wherein, is an input sample frame, is an output encoded feature, is a convolution kernel, is a bias parameter, denotes a convolution operation, is a rectified linear unit activation function; Respectively, record the combination reflectivity factor characteristic sequence as , record the vertical integrated liquid water content and echo top height characteristic sequence as , wherein , ; Step 2-2-2, feature fusion.

7. The method of claim 6, wherein, Step 2-2-2 includes the following steps: Step 2-2-2-1, calculating channel attention: let the combined reflectivity factor feature sequence be , let the vertical integrated liquid water content and echo top height feature sequences be , for single-frame feature maps and with the size of , where H is the height, W is the width, and C is the number of channels, spatial dimension average pooling and maximum pooling are respectively performed, and are respectively denoted as the spatial average pooling of the combined reflectivity factor , the spatial maximum pooling of the combined reflectivity factor , the spatial average pooling of the vertical integrated liquid water and the echo top height , and the spatial maximum pooling of the vertical integrated liquid water and the echo top height , each obtaining two feature maps with the size of ; The two feature maps are then fed into a two-layer fully connected network, with shared parameters, denoted as follows: , ,in , During calculation, the number of feature map channels is first compressed to the original value. ,in, Let C be the set of real numbers, and C be the number of channels. The reduction rate; after The activation function yields two activated results, which are then expanded back to the original number of channels. The two outputs are then summed element-wise, and then processed... The function generates channel attention weight coefficients, which are then multiplied by the original feature map to obtain a channel attention-based feature map. The combined reflectance factor feature map is denoted as... The vertical cumulative liquid water content and echo top height feature map are as follows: The calculation process is expressed as follows: ， ， wherein , , is d product; Step 2-2-2-2, calculate spatial attention: average pooling and max pooling are performed on the channel dimension of the single-frame feature map obtained in step 2-2-2-1 and respectively, denoted as channel average pooling of the combined reflectivity factor , channel max pooling of the combined reflectivity factor , channel average pooling of the vertical integrated liquid water content and the echo top height , channel max pooling of the vertical integrated liquid water content and the echo top height , the number of feature map channels is reduced to 1 and the size is changed to ; then the feature maps are spliced in the channel dimension respectively, denoted as , , each obtaining two feature maps with a size of ; then the two feature maps are respectively input into a convolutional layer with a convolution kernel size of , and the output feature map size remains unchanged and the number of channels is reduced to 1, and the convolution calculation process is denoted as , obtaining two feature maps with a size of ; finally, the two obtained feature maps are subjected to a function to generate spatial attention weight coefficients, and then multiplied with the original feature map to obtain a spatial attention-based feature map, denoted as for the combined reflectivity factor feature map, and for the vertical integrated liquid water content and the echo top height feature map, and the calculation process is represented as: Step 2-2-2-3, feature stitching: stitch the two feature maps obtained based on attention , to obtain a fused feature map The calculation process is represented as: ， wherein, represents the splicing of two modal data feature maps, the length and the feature map size of the spliced fusion feature sequence remain unchanged, and the fusion feature channel number is expanded to twice the original.

8. The method of claim 7, wherein, Step 2-3 includes the following steps: Step 2-3-1, training network using fusion features: input the training sample set data obtained in step 1-4 into the network model, for a length Multi-modal sample sequence , Sample number, first 10 frame data , as training data for feature extraction and feature fusion, input the convolutional long short-term memory model of step 2-1-2, and output 10 frames of prediction data in the prediction layer P1 of 2-1-3 ; select the combined reflectivity factor data in the last 10 frames of data as label data, used to calculate the loss function and supervise the model training; Step 2-3-2, calculate loss: loss function With mean square error (MSE) loss, select the combined reflectance factor prediction data output by the decoder T1 in step 2-3-1 , and the label data Calculate the loss, and the calculation formula is as follows: ， wherein, , is a sample sequence number, is a sample sequence length; Step 2 - 3-3, error backpropagation updates parameters: according to the results of the loss function Automatic derivation implements gradient backpropagation to update network parameters; Step 2-3-4, optimization of network hyperparameters: according to the results of the model on the validation sample set during the training process, the hyperparameters of the model with the best performance are selected as the final hyperparameters of the network model.

9. The method of claim 8, wherein, Step 2-3-1 includes the following steps: Step 2-3-1 includes the following steps: Step 2-3-1-1, sequence encoding layer C2 processing: sequence encoding layer , the data corresponding to T time Layer unit According to the input sample frame data and the data corresponding to T-1 time Layer unit The memory unit and hidden state of the layer unit are updated over time, and the hidden state is output. T time layer unit data is determined by T time input hidden state and T-1 time layer unit memory unit and hidden state, update their own memory unit and hidden state over time, and output the hidden state; when , the memory unit and hidden state of the previous time T-1 are both 0; the update process is controlled by the input gate , the forget gate , the output gate , wherein the input gate controls the addition of the input and the hidden state of the previous time, the forget gate determines the degree of forgetting of the memory unit of the previous time, and the output gate adjusts the memory unit at this time to output the hidden state at this time; The features output by the encoder C1 , As input, the output is Hidden state , The forward propagation is computed as follows: ， ， ， ， ， wherein, is an activation function, is a hyperbolic tangent activation function, represents a convolution operation, represents a product; is an input of is a memory cell of is a hidden state of , , , is an input gate, a forget gate, and an output gate of respectively; the parameters of the input gate are a convolution kernel for convolution calculation with , a convolution kernel for convolution calculation with the hidden state , and a bias ; the parameters of the forget gate are a convolution kernel for convolution calculation with , a convolution kernel for convolution calculation with the hidden state , and a bias ; the parameters of the output gate are a convolution kernel for convolution calculation with , a convolution kernel for convolution calculation with the hidden state , and a bias ; the parameters of the memory cell are a convolution kernel for convolution calculation with , a convolution kernel for convolution calculation with the hidden state , and a bias ; The hidden state of is given by as input, The forward propagation computation formula is as follows: ， ， ， ， ， Wherein, the hidden state is input, is a memory cell, is a hidden state, , , respectively input gate, forget gate, output gate of ; the parameters of the input gate respectively are convolution kernel for convolution calculation with hidden state for convolution calculation with bias ; the parameters of the forget gate respectively are convolution kernel for convolution calculation with hidden state for convolution calculation with bias ; the parameters of the output gate respectively are convolution kernel for convolution calculation with hidden state for convolution calculation with bias ; the parameters of the memory cell respectively are convolution kernel for convolution calculation with hidden state for convolution calculation with bias ; Step 2-3-1-2, prediction layer P1 process: prediction layer P1 , Time Layer unit The data of -2 Time Layer unit Memory unit and hidden state of the layer unit are determined, Update its own memory unit and hidden state over time, and output the hidden state, wherein, ; time layer unit data by time input hidden state and -2 time data corresponds layer unit memory unit and hidden state together determine, update their own memory unit and hidden state over time, and output hidden state; When the previous time -2 memory cells and hidden state for time t; the update process is controlled by the input gate , the forget gate , the output gate , where the input gate controls the addition of the input and the previous time hidden state, the forget gate decides the degree of forgetting of the previous time memory cell, and the output gate adjusts the memory cell at this time to output the hidden state at this time The output of the function is Hidden state , The forward propagation calculation formula is as follows: ， ， ， ， ， in, yes memory units, yes The hidden state, , , They are Input gate, forget gate, output gate; Input gate The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations Bias Forgotten Gate The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations Bias Output gate The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations Bias ; memory unit The parameters are respectively related to the hidden state. Convolution kernel for performing convolution calculations Bias ; The hidden state of is updated as follows: As input, the hidden state of is updated as follows: As output, the hidden state of The forward propagation computation formula is as follows: ， ， ， ， ， in, Hidden state yes Input, yes memory units, yes The hidden state, , , They are Input gate, forget gate, output gate; Input gate The parameters are respectively with Convolution kernel for performing convolution calculations and hidden state Convolution kernel for performing convolution calculations Bias Forgotten Gate The parameters are respectively with Convolution kernel for performing convolution calculations and hidden state Convolution kernel for performing convolution calculations Bias Output gate The parameters are respectively with Convolution kernel for performing convolution calculations and hidden state Convolution kernel for performing convolution calculations Bias ; memory unit The parameters are respectively with Convolution kernel for performing convolution calculations and hidden state Convolution kernel for performing convolution calculations Bias ; Step 2-3-1-3, state fusion layer F1 processing: the input F1 of the state fusion layer is the hidden state sequence output by the prediction layer P1, which is processed by Convolution compresses the number of channels, and the output is a sequence of channel-compressed feature maps ; Step 2-3-1-4, decoder T1 processing.

10. The method of claim 9, wherein, Step 2-3-1-4 includes: the input of the decoder T1 is the prediction feature sequence output by the state fusion layer F1 , the CR part in the prediction feature sequence is intercepted , the inverse convolution operation is performed on the intercepted CR part and the corresponding convolution kernel , the convolution result is added to the bias parameter , and then the function activation is performed , and the output is the prediction echo sequence , and the calculation formula is as follows: ， wherein, is an input feature frame, is an output predicted echo frame, is a convolution kernel, is a bias parameter, and each input feature frame is processed by a corresponding decoder to obtain a decoder output i.e., a predicted echo sequence.

Images