A weather radar echo frame-by-frame prediction neural network training state determination method
By introducing a loss function and a similarity comparison method into the weather radar echo prediction neural network, the problem of difficulty in determining the training state is solved, and the prediction accuracy under high temporal resolution and large detection range is improved.
Patent Information
- Application Number
- CN202411411477.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2025-11-21
- Estimated Expiration
- 2044-10-10
AI Technical Summary
Existing neural networks for predicting weather radar echoes frame by frame have difficulty accurately determining whether training is complete during the training process, especially when the echo to be predicted has little change from the previous frame echo. The prediction result may be highly similar to the input sample but does not reflect predictive ability.
A method based on neural network loss function and prediction results and similarity comparison with input-target time series samples is adopted. By setting threshold and mean squared error function, it is determined whether the neural network training state is complete, including data preprocessing and neural network structure design, such as two-dimensional convolutional long short-term memory layer and batch normalization layer.
It improves the predictive power of neural networks, making it particularly suitable for weather radars with high temporal resolution and large detection range, ensuring the effectiveness and accuracy of prediction results.
Smart Images
Figure CN119535385B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of weather radar echo prediction, and specifically relates to a method for determining the training state of a neural network for frame-by-frame prediction of weather radar echoes. Background Technology
[0002] Weather radar echo prediction (extrapolation) plays a crucial role in meteorological forecasting. It can promptly detect and predict the occurrence and development trends of severe weather events such as heavy rainfall and typhoons, providing timely early warning information to relevant departments and the public. This helps reduce the threat of severe weather to people's lives and property. Currently used weather radar echo prediction methods include cross-correlation, single-unit centroid method, optical flow method, and neural network prediction methods. Among these, neural network prediction methods, especially deep learning neural network prediction methods, can achieve high-precision short-term weather radar echo prediction based on massive amounts of data and cumulative learning training.
[0003] For predictive neural networks, the training status (performance) and evaluation of whether training is complete are typically determined by assessing the similarity between the predicted result and the target sample. Specifically, for a frame-by-frame predictive neural network based on weather radar echoes, multiple consecutive frames (e.g., 10 frames) of time-series echo data from historical radar detection data are usually used as input samples, and the next frame of echo data is used as the target sample. However, it should be noted that because the time interval between the echo to be predicted and the previous frame is very close, the changes in its echo data are usually very small. Furthermore, for weather radars with high temporal resolution and large detection range, the changes in the continuously measured multiple frames of time-series echo data are usually not significant. Under these circumstances, a possible phenomenon is that after several training iterations, although the predicted result has a high similarity to the target sample, it is more similar to one or more input samples. This result clearly fails to effectively reflect the predictive ability of the neural network. Summary of the Invention
[0004] The technical problem to be solved by this invention is to provide a method for determining the training state of a neural network for frame-by-frame prediction of weather radar echoes. By examining the similarity between the prediction results and the input sample-target sample, the training state of the neural network is evaluated, and the completion of training is accurately determined.
[0005] The present invention adopts the following technical solution:
[0006] An improved method for determining the training state of a neural network for frame-by-frame prediction of weather radar echoes includes the following steps:
[0007] Step 1: Collect weather radar echo datasets, preprocess the data, and establish weather radar echo data samples.
[0008] Step 2: Establish a frame-by-frame prediction neural network for weather radar echoes;
[0009] Step 3: Establish a method for determining the training state of the weather radar echo frame-by-frame prediction neural network, conduct neural network training, and determine the training state until completion.
[0010] The method for determining the training state of the neural network for frame-by-frame prediction of weather radar echoes includes a method based on the neural network loss function F. L Methods and prediction results based on S Y With input-target time series samples {S X ,S T Similarity comparison methods;
[0011]
[0012] In the above formula, F LT The threshold is denoted by MSE, which represents the mean squared error function. When the above criteria are met, the training is considered complete.
[0013] Furthermore, in step 1, collecting the weather radar echo dataset refers to collecting the historical time-series echo data set of the weather radar, where the total number of historical time-series echo data is N. P The echo data is P(i P ), where i P =1,2,...,N P Then the weather radar echo dataset S P for:
[0014] S P ={P(i P )}={P(1),P(2),…,P(N P )}.
[0015] Furthermore, in step 1, the data is preprocessed, including data quality control and data downscaling.
[0016] The data downscaling process, for echo data in polar coordinates, includes reducing the azimuth and radial scales. Let the scale of the original echo data be (M0, N0), then the original echo data is represented as P0(i0, j0), i0 = 1, 2, ..., M0, j0 = 1, 2, ..., N0. Let the scale of the downscaled echo data be (M1, N1), then the downscaled echo data is represented as P1(i1, j1), i1 = 1, 2, ..., M1, j1 = 1, 2, ..., N1. The downscaled echo data uses the nearest neighbor principle, resulting in:
[0017]
[0018] In the above formula, function D represents the distance between two data points.
[0019] Furthermore, in step 1, weather radar echo data samples are established. First, the weather radar echo dataset S is processed. P Sampling, let the input sample be S X The input sample length is N X Let the target sample be S. T For frame-by-frame prediction, if the target sample length is 1, then the total sampling length is N. X +1, where the top N samples X The first element is the input sample, and the last element is the target sample.
[0020]
[0021] In the above formula, i S The sample number is N, and the total number of samples is N. P -N X The above samples are further divided into training samples and validation samples.
[0022] Furthermore, the sampling method adopts sliding sampling; training samples and validation samples are randomly allocated in a ratio of 90:10.
[0023] Furthermore, in step 2, the weather radar echo frame-by-frame prediction neural network adopts a sequential model from the Keras artificial neural network library in Python, setting up four stacked two-dimensional convolutional long short-term memory layers and batch normalization layers, as well as one three-dimensional convolutional layer, with a loss function F. L Use the cross-entropy loss function or the mean squared error loss function.
[0024] The beneficial effects of this invention are:
[0025] The method disclosed in this invention adds a training state determination condition based on the comparison of the prediction result and the similarity between the input and target time series samples. This ensures that the neural network has effective prediction capabilities and is particularly suitable for weather radar echo prediction with small differences in time series samples. This includes application scenarios with high time resolution and large detection range, such as phased array scanning weather radar with high time resolution or S-band weather radar with large-area detection capabilities. Attached Figure Description
[0026] Figure 1 This is a flowchart illustrating the method of the present invention;
[0027] Figure 2 This is a schematic diagram of the original weather radar echo data;
[0028] Figure 3 This is a schematic diagram of weather radar echo data after quality control and downscaling.
[0029] Figure 4 This is a schematic diagram of weather radar echo data input samples;
[0030] Figure 5 This is a schematic diagram of target samples from weather radar echo data;
[0031] Figure 6 This is a graph showing how the loss function of a neural network changes with the number of training iterations;
[0032] Figure 7 It is the similarity between the prediction result and the input-target time series sample after 160 training iterations;
[0033] Figure 8 It is the similarity between the prediction result and the input-target time series sample after 320 training iterations;
[0034] Figure 9 This is a schematic diagram of the target sample after the training process is completed;
[0035] Figure 10 This is a schematic diagram of the prediction results after the training process is completed. Detailed Implementation
[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0037] Example 1 discloses a method for determining the training state of a weather radar echo frame-by-frame prediction neural network. This method enables the neural network prediction results to have better predictive power, which is of great significance for short-term weather forecasting. Figure 1 As shown, it includes the following steps:
[0038] Step 1: Collect weather radar echo datasets, preprocess the data, and establish weather radar echo data samples.
[0039] Weather radar echo data refers to the reflectivity factor (PPI) (Plan Position Indicator) data detected by weather radar.
[0040] Collecting weather radar echo datasets refers to collecting historical time-series echo data sets from weather radars, where the total number of historical time-series echo data sets is N. P The echo data is P(i P ), where i P =1,2,...,N P Then the weather radar echo dataset S P for:
[0041] S P ={P(i P )}={P(1),P(2),…,P(N P )}.
[0042] Echo data P(i) P It is usually in the form of a two-dimensional array.
[0043] Data preprocessing includes data quality control and data downscaling.
[0044] The data downscaling process refers to reducing the spatial resolution of the original echo data, thereby reducing its scale, to decrease computational resource consumption and improve training efficiency. For echo data in polar coordinates, this includes reducing both the azimuth and radial scales. Let the scale of the original echo data be (M0, N0), then the original echo data is represented as P0(i0, j0), i0 = 1, 2, ..., M0, j0 = 1, 2, ..., N0. The scale of the downscaled echo data is (M1, N1), then the downscaled echo data is represented as P1(i1, j1), i1 = 1, 2, ..., M1, j1 = 1, 2, ..., N1. The downscaled echo data uses the nearest neighbor principle, resulting in:
[0045]
[0046] In the above formula, function D represents the distance between two data points.
[0047] The original echo data has azimuth and radial scales of 360 and 920, respectively. The echo data can be found in [link to original text]. Figure 2 Following the method described in step 1, the original echo data is downscaled. The azimuth and radial scales of the downscaled echo data are 180 and 115, respectively. See the echo data below. Figure 3 .also, Figure 3 Quality control procedures were also carried out at the same time.
[0048] To establish a weather radar echo data sample, firstly, analyze the weather radar echo dataset S. P Sampling is performed using sliding sampling; let the input sample be S. X The input sample length is N X Let the target sample be S. T For frame-by-frame prediction, if the target sample length is 1, then the total sampling length is N. X +1, where the top N samples X The first element is the input sample, and the last element is the target sample.
[0049]
[0050] In the above formula, i S The sample number is N, and the total number of samples is N. P -N X The above samples are then randomly divided into training samples and validation samples, with an allocation ratio of 90:10.
[0051] Following the method described in step 1, a set of weather radar echo data samples is provided, wherein the input samples are as follows: Figure 4 As shown, the target sample is as follows Figure 5 As shown.
[0052] Step 2: Establish a frame-by-frame prediction neural network for weather radar echoes;
[0053] The weather radar echo frame-by-frame prediction neural network uses a sequential model from the Keras artificial neural network library in Python. It consists of four stacked 2D convolutional long short-term memory layers and batch normalization layers, as well as one 3D convolutional layer. The loss function is F. L Use cross-entropy loss function or mean squared error loss function, etc.
[0054] Step 3: Establish a method for determining the training state of the weather radar echo frame-by-frame prediction neural network, conduct neural network training, and determine the training state until completion.
[0055] The method for determining the training state of the neural network for frame-by-frame prediction of weather radar echoes includes a method based on the neural network loss function F. L Methods and prediction results based on S Y With input-target time series samples {S X ,S T Similarity comparison methods;
[0056] The neural network loss function F L The method refers to examining the loss function F as the neural network is trained. L Is it less than (or not greater than) the threshold F? LT The threshold value can be determined based on experience.
[0057] The prediction result S Y With input-target time series samples {S X ,S T Similarity comparison methods refer to calculating the neural network's prediction result S under the current training state as the neural network is trained. Y With input-target time series samples {S X ,S T Each element S in} X-T The similarity is examined to determine the prediction result S. Y Is it related to the input-target time series sample {S}? X,S T The target sample S in} T The highest similarity was found in F. S The mean squared error (MSE) function can be used for evaluation. Simultaneously, all validation samples are examined, and the similarity of all samples within the validation samples is averaged. We have:
[0058]
[0059] The determination of the training state to completion refers to simultaneously satisfying the conditions based on the neural network loss function F. L The decision criteria of the method and the prediction results S Y With input-target time series samples {S X ,S T The criteria for similarity comparison methods. Specifically, the criteria are:
[0060]
[0061] In the above formula, F LT The threshold is set to MSE, which represents the mean squared error function. Training is considered complete when the above criteria are met. Based on this trained neural network, frame-by-frame prediction of weather radar echoes can be effectively performed.
[0062] Following the method described in step 3, the change of the neural network loss function with the number of training iterations is obtained as follows: Figure 6 As shown. If the threshold is set to 6, then when the number of training iterations reaches 120, the judgment condition based on the neural network loss function method is met.
[0063] The prediction result and the similarity between the input-target time series samples are obtained according to the method described in step 3, and the training times are 160 times (e.g., ...). Figure 7 (as shown) and 320 times (as shown) Figure 8 (As shown). When the number of training iterations reaches 320, the judgment condition based on the similarity comparison method between the prediction result and the input-target time series samples is met.
[0064] In summary, when the training iterations reach 320, both the criteria based on the neural network loss function method and the criteria based on the similarity comparison method between the predicted results and the input-target time-series samples are satisfied. Therefore, the training state is considered complete. Finally, the target sample (e.g., ...) after the training state is completed is given. Figure 9 (as shown) and prediction results (as shown) Figure 10 (As shown).
Claims
1. A method for determining the training state of a neural network for frame-by-frame prediction of weather radar echoes, characterized in that, Includes the following steps: Step 1: Collect weather radar echo datasets, preprocess the data, and establish weather radar echo data samples. To establish a weather radar echo data sample, firstly, analyze the weather radar echo dataset S. P Sampling, let the input sample be S X The input sample length is N X Let the target sample be S. T For frame-by-frame prediction, if the target sample length is 1, then the total sampling length is N. X +1, where the top N samples X The first element is the input sample, and the last element is the target sample. In the above formula, i S The sample number is N, and the total number of samples is N. P -N X The above samples are further divided into training samples and validation samples. Step 2: Establish a frame-by-frame prediction neural network for weather radar echoes; Step 3: Establish a method for determining the training state of the weather radar echo frame-by-frame prediction neural network, conduct neural network training, and determine the training state until completion. The method for determining the training state of the neural network for frame-by-frame prediction of weather radar echoes includes a method based on the neural network loss function F. L Methods and prediction results based on S Y With input-target time series samples {S X ,S T Similarity comparison methods; In the above formula, F LT The threshold is denoted by MSE, which represents the mean squared error function. When the above criteria are met, the training is considered complete.
2. The method for determining the training state of a weather radar echo frame-by-frame prediction neural network according to claim 1, characterized in that: In step 1, collecting the weather radar echo dataset refers to collecting the historical time-series echo data set of the weather radar, with the total number of historical time-series echo data sets being N. P The echo data is P(i P ), where i P =1,2,...,N P Then the weather radar echo dataset S P for: S P ={P(i P )}={P(1),P(2),…,P(N P )}。 3. The method for determining the training state of a weather radar echo frame-by-frame prediction neural network according to claim 1, characterized in that: In step 1, the data is preprocessed, including data quality control and data downscaling. The data downscaling process, for echo data in polar coordinates, includes reducing the azimuth and radial scales. Let the scale of the original echo data be (M0, N0), then the original echo data is represented as P0(i0, j0), i0 = 1, 2, ..., M0, j0 = 1, 2, ..., N0. Let the scale of the downscaled echo data be (M1, N1), then the downscaled echo data is represented as P1(i1, j1), i1 = 1, 2, ..., M1, j1 = 1, 2, ..., N1. The downscaled echo data uses the nearest neighbor principle, resulting in: In the above formula, function D represents the distance between two data points.
4. The method for determining the training state of a weather radar echo frame-by-frame prediction neural network according to claim 1, characterized in that: The sampling method adopted is sliding sampling; training samples and validation samples are randomly assigned in a ratio of 90:
10.
5. The method for determining the training state of a weather radar echo frame-by-frame prediction neural network according to claim 1, characterized in that: In step 2, the weather radar echo frame-by-frame prediction neural network adopts a sequential model from the Keras artificial neural network library in Python. It consists of four stacked two-dimensional convolutional long short-term memory layers and a batch normalization layer, as well as a three-dimensional convolutional layer. The loss function is F. L Use the cross-entropy loss function or the mean squared error loss function.