Weather radar echo frame-by-frame prediction neural network training state judgment method

Abstract

The invention discloses a weather radar echo frame-by-frame prediction neural network training state judgment method, which comprises the following steps of 1, collecting a weather radar echo data set, preprocessing data, and establishing a weather radar echo data sample; step 2, establishing a weather radar echo frame-by-frame prediction neural network; and step 3, establishing a weather radar echo frame-by-frame prediction neural network training state determination method, carrying out neural network training, and determining the training state until completion. According to the method disclosed by the invention, a training state judgment condition based on a prediction result and input-target time sequence sample similarity comparison method is added, so that the neural network can be ensured to have an effective prediction capability, and the method is particularly suitable for weather radar echo prediction with relatively small time sequence sample difference; comprising application scenes with high time resolution and large detection range, such as a phased array scanning weather radar with high time resolution or an S-band weather radar with large-range detection capability.

Description

Technical Field

[0001] The invention belongs to the field of weather radar echo prediction, and in particular relates to a method for determining a neural network training state of weather radar echo frame-by-frame prediction in the field. Background Art

[0002] Weather radar echo prediction (extrapolation) is extremely important in meteorological forecasting. It can timely discover and predict the occurrence and development trend of disastrous weather such as heavy rainfall and typhoons, and provide timely warning information to relevant departments and the public. It helps to reduce the threat of disastrous weather to people's lives and property. The weather radar echo prediction methods currently used include cross-correlation method, monomer centroid method, optical flow method and neural network prediction method. Among them, neural network prediction method, especially deep learning neural network prediction method, can achieve high-precision short-term prediction of weather radar echo based on massive big data and cumulative learning training.

[0003] For prediction neural networks, the training status (effect) of the network and whether the training is completed are usually judged by evaluating the similarity between the prediction results and the target samples. Specifically, for the weather radar echo frame-by-frame prediction neural network, continuous multi-frame (such as 10 frames) time-series echo data in the radar historical detection data are usually used as input samples, and the next frame echo data is used as the target sample. However, it should be noted that since the time corresponding to the echo to be predicted is very close to that of the previous frame echo, the change in its echo data is usually very small. At the same time, for weather radars with high temporal resolution and large detection range, the change in its continuously measured multi-frame time-series echo data is usually not significant. In the above case, a phenomenon that may occur is that after the neural network has been trained several times, although the similarity between the prediction result and the target sample is already very high, it is more similar to a certain (some) input sample. This result obviously fails to effectively reflect the predictiveness of the neural network. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for determining the training status of a neural network for frame-by-frame prediction of weather radar echoes, by examining the similarity between the prediction results and the input sample-target sample, evaluating the training status of the neural network, and accurately determining whether the training is completed.

[0005] The present invention adopts the following technical solution:

[0006] A method for determining the training state of a neural network for frame-by-frame prediction of weather radar echoes, the improvement of which is that it comprises the following steps:

[0007] Step 1: collect weather radar echo data sets, preprocess the data, and establish weather radar echo data samples;

[0008] Step 2, establish a weather radar echo frame-by-frame prediction neural network;

[0009] Step 3: Establish a method for determining the training status of a neural network for weather radar echo frame-by-frame prediction, conduct neural network training, and determine the training status until it is completed:

[0010] The weather radar echo frame-by-frame prediction neural network training state determination method includes a neural network loss function F L The method and prediction results based on S Y With input-target timing samples {S X ,S T}Similarity comparison method;

[0011]

[0012] In the above formula, F LT is the threshold, MSE represents the mean square error function, and when the above judgment conditions are met, the training is considered to be completed.

[0013] Furthermore, in step 1, collecting the weather radar echo data set refers to collecting the historical time series echo data set of the weather radar, and the total number of historical time series echo data of the weather radar is N P , the echo data is P(i P ), where i P =1,2,…,N P , then the weather radar echo data set S P for:

[0014] S P = {P(i P )}={P(1),P(2),…,P(N P )}.

[0015] Furthermore, in step 1, the data is preprocessed, including data quality control and data downscaling.

[0016] The data downscaling process, for the echo data in the polar coordinate system, includes reducing the azimuth scale and the radial scale. Let the scale of the original echo data be (M0, N0), then the original echo data is expressed as P0(i0, j0), i0=1, 2, ..., M0, j0=1, 2, ..., N0, the scale of the reduced echo data is (M1, N1), then the downscaled echo data is expressed as P1(i1, j1), i1=1, 2, ..., M1, j1=1, 2, ..., N1, the echo data after downscaling is valued according to the principle of proximity in distance, and there are:

[0017]

[0018] In the above formula, function D represents the distance between two data points.

[0019] Furthermore, in step 1, a weather radar echo data sample is established. First, the weather radar echo data set S P Sampling, let the input sample be S X , the input sample length is N X , let the target sample be S T For frame-by-frame prediction, the target sample length is 1, and the total sampling length is N X +1, where the first N samples X The first element is the input sample, and the last element is the target sample, which is:

[0020]

[0021] In the above formula, i S is the sample number, the total number of samples is N P -N X ; Divide the above samples into training samples and verification samples.

[0022] Furthermore, the sampling method adopts sliding sampling; the training samples and validation samples are randomly allocated with a distribution ratio of 90:10.

[0023] Furthermore, in step 2, the weather radar echo frame-by-frame prediction neural network adopts the sequential model under the Python language Keras artificial neural network library, sets four stacked two-dimensional convolutional long short-term memory layers and batch normalization layers, and a three-dimensional convolution layer, and the loss function F L Use the cross entropy loss function or the mean square error loss function.

[0024] The beneficial effects of the present invention are:

[0025] The method disclosed in the present invention adds a training state judgment condition based on the prediction result and the input-target time series sample similarity comparison method, which can ensure that the neural network has effective prediction ability, and is particularly suitable for weather radar echo prediction with small time series sample differences, including application scenarios with high time resolution and large detection range, such as phased array scanning weather radar with high time resolution, or S-band weather radar with large-range detection capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic flow diagram of the method of the present invention;

[0027] Figure 2 It is a schematic diagram of raw weather radar echo data;

[0028] Figure 3 It is a schematic diagram of weather radar echo data after quality control and downscaling;

[0029] Figure 4 This is a schematic diagram of a sample weather radar echo data input;

[0030] Figure 5 It is a schematic diagram of a target sample of weather radar echo data;

[0031] Figure 6 It is a graph of the change of the neural network loss function with the number of training times;

[0032] Figure 7 It is the similarity between the prediction result and the input-target time series sample when the number of training times is 160;

[0033] Figure 8 It is the similarity between the prediction result and the input-target time series sample when the number of training times is 320;

[0034] Fig. 9 This is a schematic diagram of the target sample after the training state is completed;

[0035] Fig.10 This is a schematic diagram of the prediction results after the training state is completed. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] Example 1: This example discloses a method for determining the training state of a neural network for weather radar echo frame-by-frame prediction. This method can make the neural network prediction results have better predictability, which is of great significance for short-term weather forecasts. Figure 1 As shown, the following steps are included:

[0038] Step 1: collect weather radar echo data sets, preprocess the data, and establish weather radar echo data samples;

[0039] Weather radar echo data refers to the reflectivity factor PPI (Plan Position Indicator) data detected by the weather radar.

[0040] Collecting weather radar echo data sets refers to collecting historical time series echo data sets of weather radars. Let the total number of historical time series echo data of weather radars be N. P , the echo data is P(i P ), where i P =1,2,…,N P , then the weather radar echo data set S P for:

[0041] S P = {P(i P )}={P(1),P(2),…,P(N P )}.

[0042] Echo data P(i P ) is usually in the form of a two-dimensional array.

[0043] Preprocess the data, including data quality control and data downscaling.

[0044] The data downscaling process refers to reducing the spatial resolution of the original echo data in order to reduce the consumption of computing resources and improve the training efficiency, thereby reducing the scale of the original echo data. For the echo data in the form of polar coordinate system, including reducing the azimuth scale and radial scale, let the scale of the original echo data be (M0, N0), then the original echo data is expressed as P0 (i0, j0), i0 = 1, 2, ..., M0, j0 = 1, 2, ..., N0, the scale of the reduced echo data is (M1, N1), then the downscaled echo data is expressed as P1 (i1, j1), i1 = 1, 2, ..., M1, j1 = 1, 2, ..., N1, the echo data after downscaling is valued according to the principle of proximity in distance, and there are:

[0045]

[0046] In the above formula, function D represents the distance between two data points.

[0047] The azimuth and radial scales of the original echo data are 360 ​​and 920 respectively. Figure 2 According to the method described in step 1, the original echo data is downscaled. The azimuth scale and radial scale of the downscaled echo data are 180 and 115 respectively. Figure 3 .also, Figure 3 A quality control process was also performed simultaneously.

[0048] To establish a weather radar echo data sample, firstly, the weather radar echo data set S P Sampling, the sampling method adopts sliding sampling; let the input sample be S X , the input sample length is N X , let the target sample be S T For frame-by-frame prediction, the target sample length is 1, and the total sampling length is N X +1, where the first N samples X The first element is the input sample, and the last element is the target sample, which is:

[0049]

[0050] In the above formula, i S is the sample number, the total number of samples is N P -N X ; The above samples are randomly divided into training samples and verification samples, with the allocation ratio of 90:10.

[0051] According to the method described in step 1, a set of weather radar echo data samples is given, where the input sample is as follows: Figure 4 As shown, the target sample is Figure 5 shown.

[0052] Step 2, establish a weather radar echo frame-by-frame prediction neural network;

[0053] The weather radar echo frame-by-frame prediction neural network adopts the sequential model under the Python language Keras artificial neural network library, setting four stacked two-dimensional convolutional long short-term memory layers and batch normalization layers, as well as a three-dimensional convolution layer, and the loss function F L Use cross entropy loss function or mean square error loss function etc.

[0054] Step 3: Establish a method for determining the training status of a neural network for weather radar echo frame-by-frame prediction, conduct neural network training, and determine the training status until it is completed:

[0055] The weather radar echo frame-by-frame prediction neural network training state determination method includes a neural network loss function F L The method and prediction results based on S Y With input-target timing samples {S X ,S T}Similarity comparison method;

[0056] The neural network loss function F L The method refers to examining the loss function F as the neural network is trained. L Is it less than (or not greater than) the threshold F? LT The threshold value can be determined based on experience.

[0057] The prediction result S Y With input-target timing samples {S X ,S T The similarity comparison method is to calculate the prediction result S of the neural network in the current training state as the neural network is trained. Y With input-target timing samples {S X ,S T Each element S in X-T The similarity of the prediction result S Y Is it consistent with the input-target timing sample {S X,S T} target sample S T The similarity is the highest. Among them, the similarity F S The mean square error function MSE can be used for evaluation. At the same time, all validation samples are examined and the similarity of all samples in the validation samples is averaged.

[0058]

[0059] The determination of the training state to completion means that the neural network loss function F is satisfied at the same time. L The judgment conditions of the method and the prediction results S Y With input-target timing samples {S X ,S T} The determination conditions of the similarity comparison method. The specific determination conditions are:

[0060]

[0061] In the above formula, F LT is the threshold, MSE represents the mean square error function, and when the above judgment conditions are met, the training is considered complete. Based on the neural network in this training state, the weather radar echo frame-by-frame prediction can be effectively carried out.

[0062] According to the method described in step 3, the change of the neural network loss function with the number of training times is obtained as follows Figure 6 If the threshold is set to 6, when the number of training times reaches 120, the judgment condition based on the neural network loss function method is met.

[0063] According to the method described in step 3, the similarity between the prediction result and the input-target time series sample is obtained, and the number of training times is 160 times (such as Figure 7 ) and 320 times (as shown Figure 8 When the number of training times reaches 320, the judgment condition based on the method of comparing the similarity between the prediction results and the input-target time series samples is met.

[0064] In summary, when the number of training times reaches 320, the judgment conditions based on the neural network loss function method and the judgment conditions based on the similarity comparison method between the prediction results and the input-target time series samples are met at the same time, so the training state is determined to be completed. Finally, the target sample after the training state is completed is given (such as Fig. 9 ) and the predicted results (as shown in Fig.10 shown).

Claims

1. A method for determining the training state of a neural network for frame-by-frame prediction of weather radar echoes, characterized in that: The steps include: Step 1: collect weather radar echo data sets, preprocess the data, and establish weather radar echo data samples; Step 2, establish a weather radar echo frame-by-frame prediction neural network; Step 3: Establish a method for determining the training status of a neural network for weather radar echo frame-by-frame prediction, conduct neural network training, and determine the training status until it is completed: The weather radar echo frame-by-frame prediction neural network training state determination method includes a neural network loss function F L The method and prediction results based on S Y With input-target timing samples {S X ,S T }Similarity comparison method; In the above formula, F LT is the threshold, MSE represents the mean square error function, and when the above judgment conditions are met, the training is considered to be completed.

2. The method for determining the training state of a neural network for weather radar echo frame-by-frame prediction according to claim 1 is characterized in that: In step 1, collecting weather radar echo data sets refers to collecting historical time series echo data sets of weather radars. Let the total number of historical time series echo data of weather radars be N. P , the echo data is P(i P ), where i P =1,2,…,N P , then the weather radar echo data set S P for: S P ={P(i P )}={P(1),P(2),…,P(N P )}。 3. The method for determining the training state of a neural network for weather radar echo frame-by-frame prediction according to claim 1 is characterized in that: In step 1, the data is preprocessed, including data quality control and data downscaling. The data downscaling process, for the echo data in the polar coordinate system, includes reducing the azimuth scale and the radial scale. Let the scale of the original echo data be (M0, N0), then the original echo data is expressed as P0(i0, j0), i0=1, 2, ..., M0, j0=1, 2, ..., N0, the scale of the reduced echo data is (M1, N1), then the downscaled echo data is expressed as P1(i1, j1), i1=1, 2, ..., M1, j1=1, 2, ..., N1, the echo data after downscaling is valued according to the principle of proximity in distance, and there are: In the above formula, function D represents the distance between two data points.

4. The method for determining the training state of a neural network for weather radar echo frame-by-frame prediction according to claim 2 is characterized in that: In step 1, a weather radar echo data sample is established. First, the weather radar echo data set S P Sampling, let the input sample be S X , the input sample length is N X , let the target sample be S T For frame-by-frame prediction, the target sample length is 1, and the total sampling length is N X +1, where the first N samples X The first element is the input sample, and the last element is the target sample, which is: In the above formula, i S is the sample number, the total number of samples is N P -N X ; Divide the above samples into training samples and verification samples.

5. The method for determining the training state of a neural network for frame-by-frame prediction of weather radar echoes according to claim 4 is characterized in that: The sampling method adopts sliding sampling; the training samples and validation samples are randomly allocated with a distribution ratio of 90:

10.

6. The method for determining the training state of a neural network for weather radar echo frame-by-frame prediction according to claim 1, characterized in that: In step 2, the weather radar echo frame-by-frame prediction neural network adopts the sequential model under the Python language Keras artificial neural network library, sets four stacked two-dimensional convolutional long short-term memory layers and batch normalization layers, and a three-dimensional convolution layer, and the loss function F L Use the cross entropy loss function or the mean square error loss function.

Images