Meteorological model incremental training method based on asynchronous updating
By adopting an incremental training method of asynchronous update in the meteorological model, the model is updated using short-term and long-term buffers, and combined with the core set for optimization, the problem of quickly updating the meteorological prediction model under finite resources and time conditions is solved, and the effect of improving prediction effect and reducing calculation overhead is achieved.
Patent Information
- Application Number
- CN202411515712.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-02-25
AI Technical Summary
How to quickly update the meteorological prediction model under limited computing resources and time conditions to improve prediction results, especially when the amount of data is increasing, avoiding catastrophic forgetting caused by incremental learning.
The incremental training method of meteorological model based on asynchronous update is adopted. By constructing a deep neural network parallel to the meteorological backbone model as the residual model, the residual model and backbone model are updated respectively using the data in the short-term and long-term buffers, the model is updated in combination with the core set, and the updated backbone model and residual model are used for prediction during online prediction.
It realizes improving the prediction effect of meteorological models in the short-term and long-term incremental processes, reducing the calculation and storage overhead of incremental updates, and conducting online updates under limited resources and time conditions.
Smart Images
Figure CN119514646A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of artificial intelligence, and in particular relates to a meteorological model incremental training method based on asynchronous updating. Background Art
[0002] With the development of meteorological observation technology, the amount of data is increasing. Using the newly added data to quickly update the meteorological prediction model has become an urgent problem to be solved. In recent decades, meteorological observation stations and meteorological radars have been rapidly deployed, forming a data acquisition method for point and surface observation, which far exceeds the sum of many years in the past. The large-scale collection of data also poses a challenge to the continuous updating of the model. As the amount of training data increases, the accuracy of meteorological prediction is also improving. How to use the latest collected data to improve the prediction effect in the short-term and long-term incremental process, and minimize the computational and storage overhead of incremental updates.
[0003] Meteorological data is different from other image data in that it has a strong temporal correlation, that is, the meteorological patterns displayed within a certain time window are similar or have a certain continuity. For example, when a meteorological observation station collects a small batch of observation data, these data can be input into the existing model, and the model can be updated using appropriate strategies in this time step to obtain more accurate prediction results. The distribution of meteorological data is often non-stationary, and offline models based on the assumption of stationary data distribution are difficult to cope with such data changes. When meteorological data accumulates to a certain amount, the correlation of local time windows becomes lower and lower, and the model needs to absorb new knowledge from these large amounts of new data while retaining, even integrating and optimizing old knowledge.
[0004] Meteorological data differs from other similar data in the following two aspects: First, short-term data has strong similarity and contains the same meteorological patterns and forecasting rules, which provides a certain reference for model prediction in local time windows; second, long-term data contains more new knowledge that needs to be further learned. Summary of the invention
[0005] The purpose of the present invention is to provide a meteorological model incremental training method based on asynchronous updating, which utilizes the latest collected data to improve the model effect in the short-term and long-term incremental processes mentioned above, and reduces the catastrophic forgetting caused by incremental learning, so as to achieve the purpose of online updating of the model under the conditions of limited computing resources and time.
[0006] In order to achieve the above tasks, the present invention adopts the following technical solutions:
[0007] A meteorological model incremental training method based on asynchronous updating, comprising:
[0008] For the meteorological backbone model, a parallel deep neural network is constructed as a residual model;
[0009] Continuously collect historical meteorological data and their corresponding true values, and save them as meteorological data samples in a short-term buffer. When the number of samples in the short-term buffer reaches a first preset value, use the meteorological data samples in the short-term buffer to update the residual model, and save the updated residual model.
[0010] The meteorological data samples in the short-term buffer are placed in the long-term buffer, and the short-term buffer is cleared to continue collecting meteorological data samples; when the number of samples in the long-term buffer reaches a second preset value, the network parameters of the residual model are fixed, and the meteorological data samples in the long-term buffer and the core set are used to update the backbone model, and the updated backbone model is saved;
[0011] Using the current long-term buffer and core set, the meteorological data samples and sample weights in the core set are updated, and then the long-term buffer is cleared.
[0012] Furthermore, when performing online prediction, the meteorological data collected in real time is simultaneously input into the updated backbone model and the residual model, and the output results of the backbone model and the residual model are added together as the final prediction result.
[0013] Furthermore, the core set stores meteorological data samples obtained after screening. The data samples are based on the historical core set and all meteorological data samples in the long-term buffer. After the gradient direction angle is calculated on the backbone network, the first P meteorological data samples are selected from large to small as the updated core set, where P is the preset size of the core set.
[0014] Furthermore, when the backbone model and the residual model are updated, Adam is used as the optimizer.
[0015] Furthermore, the first preset value is 5, and the second preset value is 60.
[0016] Furthermore, the loss function used when updating the backbone model is:
[0017]
[0018] Where: Θ represents the network parameters of the backbone model; Represents the backbone model for the long-term buffer M c Meteorological data samples The calculation result of M c Indicates a long-term buffer; |M c | indicates the long-term buffer size; represents the meteorological data in the i-th meteorological data sample in the long-term buffer and the corresponding true value; x i ,y irepresents the i-th meteorological data sample in the core set and the corresponding true value; Representation sample The MSE loss, l(f Θ (x i ),y i ) represents the sample x i ,y i MSE loss; λ represents the core set replay loss hyperparameter; M0 represents the core set; |M0| represents the core set size, w i Represents the sample weights of the core set.
[0019] Furthermore, the use of the current long-term buffer and the core set to update the meteorological data samples and sample weights in the core set includes:
[0020]
[0021]
[0022] Where: M0 represents the core set, P is the size of the preset core set; g i represents the gradient direction of the ith core set meteorological data sample on the backbone network; L represents the union of the long-term buffer and the core set; |M0| represents the core set size; represents the updated sample weight of the i-th core set sample; <g i ,g j > represents the inner product of the ith core set sample and the ith core set sample; |g i | indicates g i The mold length.
[0023] Furthermore, the backbone model adopts FourCastNet, Pangu-Weather or ConvLSTM meteorological model; the residual model adopts a deep neural network model.
[0024] A terminal device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, the incremental training method of the meteorological model based on asynchronous update is implemented.
[0025] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the meteorological model incremental training method based on asynchronous updating is implemented.
[0026] Compared with the prior art, the present invention has the following technical features:
[0027] 1. It can quickly obtain the meteorological patterns of neighboring time windows and improve the local prediction effect;
[0028] 2. The model can be incrementally updated with low resource consumption, so as to achieve the purpose of updating the model under the conditions of limited computing resources and time.
[0029] 3. When making online forecasts, you can combine the latest weather patterns and historical weather knowledge to make forecasts at the same time to improve the forecast effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 This is a flow chart of the incremental learning method for streaming meteorological data based on long-term and short-term updates;
[0031] Figure 2 Schematic diagram of the model's long-term and short-term increments and online prediction. DETAILED DESCRIPTION
[0032] The present invention provides a meteorological model incremental training method based on asynchronous updating, which can use real-time data streams to update meteorological models online, and use the latest collected data to improve the prediction effect in short-term and long-term incremental processes, so as to achieve the purpose of updating the model under the conditions of limited computing resources and time.
[0033] The present invention integrates existing deep neural network meteorological backbone models, such as FourCastNet, Pangu-Weather, ConvLSTM and other meteorological models, and streaming meteorological data is continuously collected and stored; the residual model is a deep neural network model parallel to the backbone model, and the residual model is a model proposed in this scheme to fit the difference between the prediction result of the backbone model and the true value, for example, a multi-layer fully connected neural network regression model can be used; the samples in the short-term buffer and the long-term buffer are distinguished as data sets for updating the residual model and the backbone model, and the samples are stored as T all ×H×W×C,T all is the length of the short-term or long-term buffer dataset, where H is the longitude span count value at a fixed resolution, W is the latitude span count value at a fixed resolution, and C is the meteorological feature dimension; when used, it is divided into (τ+T)×W×H×C according to the time window; the first τ time length data is the meteorological data The time length data after T is the actual value corresponding to the meteorological data The samples in the core set are the samples with the largest angles in the first P gradient directions obtained after screening the data in the historical core set and the long-term buffer. The data is stored in the form of x, y pairs, x is stored as τ×W×H×C, y is stored as T×W×H×C, and the core set is initialized to empty; the sample weight of the core set represents the weight of the data sample lost when the backbone model is updated in the core set, and the weight is initialized to all 0.
[0034] In the first step, historical meteorological data and the corresponding true values are collected and stored in a short-term buffer M as meteorological data samples at regular intervals. One frame of meteorological data is a tensor of H×W×C. For example, for the entire earth, when the longitude and latitude resolution is 0.5 degrees, the meteorological data is a tensor of 720×360×1. The number of meteorological data samples stored in the short-term buffer M is denoted as |M|.
[0035] The second step is to determine whether the number of meteorological data samples |M| collected in the short-term buffer M is equal to the short-term increment threshold ε0, and set the short-term threshold to 5. That is, when the number of data samples in the short-term buffer M is not less than 5, the meteorological data in the short-term buffer M is used to enter the third step.
[0036] The third step is to use the meteorological data samples in the buffer data M to update the residual model. In this step, only the residual model is trained and the network parameters of the backbone model are fixed. The residual model training uses Adam as the optimizer and uses the meteorological data samples in the short-term buffer M for 12 rounds of training. The initial learning rate is set to 0.01 and is reduced to 0.1 times the original after every 4 epochs. The updated residual model is output and the updated residual model is used as the latest residual model.
[0037] The fourth step is to put the meteorological data samples in the short-term buffer M into the long-term buffer M c and clear the short-term buffer M, and the long-term buffer data M c Enter step five.
[0038] Step 5: Determine the long-term buffer zone M c Is the number of meteorological data samples in equal to the long-term threshold ε c In this example, the long-term threshold is set to 60, that is, when the data in the long-term buffer is not less than 60 frames, the long-term buffer data M is used. c Go to step 7, otherwise return to step 1.
[0039] Step 6: Update the backbone model.
[0040] Fixed the network parameters of the residual model and used the long-term buffer data M c Update the backbone model together with the core set; set the number of training batches to 8, the initial learning rate to 1e-4, and decrease it to 0.1 times of the original every 4 rounds, the core set size to 128, the number of incremental training batch samples to 30, use Adam as the optimizer, and the loss function is as shown in (1). The model updated in this step is used as the latest backbone model:
[0041]
[0042] Where: Θ represents the network parameters of the backbone model; Represents the backbone model for the long-term buffer M c Meteorological data samples The calculation result of M c Indicates a long-term buffer; |M c | indicates the long-term buffer size; represents the meteorological data in the i-th meteorological data sample in the long-term buffer and the corresponding true value; x i ,y i represents the i-th meteorological data sample in the core set and the corresponding true value; Representation sample The MSE loss, l(f Θ (x i ),y i ) represents the sample x i ,y i MSE loss; λ represents the core set replay loss hyperparameter; M0 represents the core set; |M0| represents the core set size, w i Represents the sample weights of the core set.
[0043] Step 7: Update the core set and the sample weights of the core set.
[0044] The core set is updated using the current core set and the long-term buffer. The specific method is to optimize formula (2) so that among all the samples in the union of the current core set and the long-term buffer, the first P samples (including the corresponding true values) whose gradient direction angles are sorted from large to small are used as meteorological data samples in the updated core set; P is the size of the preset core set; and the core set sample weights are updated using formula (3).
[0045]
[0046]
[0047] Where: M0 represents the core set; g i represents the gradient direction of the i-th core set sample on the backbone network; L represents the union of the long-term buffer and the core set; |M0| represents the core set size; w i represents the sample weight of the ith core set, represents the updated sample weight of the i-th core set sample, which is used as the new w i Participate in the update of the main model; <g i ,g j > represents the inner product of the ith core set sample and the ith core set sample; g i | indicates g i Note: Initially, since the core set is empty, P samples are selected from the long-term buffer as samples in the core set.
[0048] Step 8: Clear the long-term buffer M c At this time, a round of incremental training ends. If you want to continue incremental training, restart the first step.
[0049] When the meteorological model is deployed online, the method of the present invention can directly perform online prediction. When the latest meteorological data is obtained, the data format is τ×W×H×C, where τ is the time length of the meteorological data; the latest data is input into the two deep neural network models of the backbone model and the residual model proposed above for calculation, such as Figure 2 As shown in the figure, the parameters used by the model come from the latest backbone model and residual model. The output results of the backbone model and the residual model are added together to obtain the final prediction result data. The data type is T×W×H×C, where T is the length of the prediction time period.
[0050] The present invention is not only applicable to online incremental learning of meteorological models using streaming data, but also applicable to online incremental learning of other models with strong time correlation.
[0051] Example:
[0052] In order to verify the effect of the incremental training method of meteorological model based on asynchronous update proposed by the present invention, the present invention designs a meteorological incremental data set based on the ERA5 meteorological data set of Xi'an and Beijing and verifies the proposed method for actual application scenarios. The data set includes a spatiotemporal meteorological data set from January 2002 to December 2018, including 22 meteorological input factors including the highest and lowest temperatures, and the prediction factors include the highest and lowest temperatures. In the incremental learning task flow of meteorological forecasting, a basic backbone network trained on a historical data set is usually set, and incremental updates are performed based on the backbone network and newly collected data. The data set is divided by time, and the basic meteorological forecasting model is trained using 1000 days of data from January 1, 2002, and the model effect is evaluated using data from January 1, 2017 to December 31, 2017. The final effect of the model is tested using data from January 1, 2018 to December 31, 2018 as a test set. An incremental data set is set after the 1000-day basic model training set and before 2017.
[0053] The incremental data set is further divided according to the length of the incremental data:
[0054] 1. Short-term incremental data set: The short-term incremental data set uses a sliding window method, with a cycle of 6 days. The first 5 days are used as training sets, and the last day is used for testing. It focuses on model optimization within a shorter time window and is used to test the effect of the residual fitting incremental learning algorithm. The increment is 1195 rounds, and the total incremental data set size is 2000.
[0055] 2. Long-term incremental data set: The long-term incremental data set is incrementally trained every 60 days, focusing on the gain of long-term data increment on model prediction effect. It is used to test the incremental learning strategy based on core set playback. The total increment is 20 rounds, that is, the incremental data set size is 60×20=1200.
[0056] In the local short-term incremental data set, in order to evaluate the improvement effect of the model on short-term prediction, the weather forecast results of the last day are used for testing. In the test of the long-term incremental data set, it is necessary to test the gain of a larger amount of data on the overall prediction accuracy of the backbone model, not limited to the improvement of local effects. The data for the whole year of 2018 is used as the long-term incremental learning test set.
[0057] This verification backbone model uses a four-layer ConvLSTM, and the residual model uses a three-layer fully connected network model. For the short-term prediction model, this chapter uses Kaiming initialization to initialize the entire convolutional feature extraction network. The backbone network is frozen during training. 12 epochs are trained on a short-term incremental dataset with a batch size of 5. The sub-model is optimized using the Adam optimizer, and the initial learning rate is set to 1e-2, that is, it is reduced to the original 0.1 every 4 epochs. The model is implemented based on PyTorch and evaluated on a Linux server with two Nvidia1080Ti graphics cards.
[0058] The present invention outputs the prediction effect of the model without incremental learning and the prediction effect after incremental learning using the method proposed by the present invention for comparison. The specific experimental results are shown in Tables 1 and 2.
[0059] Table 1 Experimental results under short-term increment setting
[0060]
[0061] Table 2 Experimental results under long-term increment setting
[0062]
[0063]
[0064] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A meteorological model incremental training method based on asynchronous updating, characterized in that: include: For the meteorological backbone model, a parallel deep neural network is constructed as a residual model; Continuously collect historical meteorological data and their corresponding true values, and save them as meteorological data samples in a short-term buffer. When the number of samples in the short-term buffer reaches a first preset value, use the meteorological data samples in the short-term buffer to update the residual model, and save the updated residual model. The meteorological data samples in the short-term buffer are placed in the long-term buffer, and the short-term buffer is cleared to continue collecting meteorological data samples; when the number of samples in the long-term buffer reaches a second preset value, the network parameters of the residual model are fixed, and the meteorological data samples in the long-term buffer and the core set are used to update the backbone model, and the updated backbone model is saved; Using the current long-term buffer and core set, the meteorological data samples and sample weights in the core set are updated, and then the long-term buffer is cleared.
2. The method for incremental training of meteorological models based on asynchronous updating according to claim 1, characterized in that: When performing online prediction, the meteorological data collected in real time is simultaneously input into the updated backbone model and the residual model, and the output results of the backbone model and the residual model are added together as the final prediction result.
3. The method for incremental training of meteorological models based on asynchronous updating according to claim 1, characterized in that: The core set stores meteorological data samples that have been screened. This data sample is based on all meteorological data samples in the historical core set and the long-term buffer. After the gradient direction angle is calculated on the backbone network, the first P meteorological data samples are selected from large to small as the updated core set, where P is the size of the preset core set.
4. The method for incremental training of meteorological models based on asynchronous updating according to claim 1, characterized in that: When the backbone model and the residual model are updated, Adam is used as the optimizer.
5. The method for incremental training of meteorological models based on asynchronous updating according to claim 1, characterized in that: The first preset value is 5, and the second preset value is 60.
6. The method for incremental training of meteorological models based on asynchronous updating according to claim 1, characterized in that: The loss function used when updating the backbone model is: Where: Θ represents the network parameters of the backbone model; Represents the backbone model for the long-term buffer M c Meteorological data samples The calculation result of M c Indicates a long-term buffer; |M c | indicates the long-term buffer size; represents the meteorological data in the i-th meteorological data sample in the long-term buffer and the corresponding true value; x i ,y i represents the i-th meteorological data sample in the core set and the corresponding true value; Representation sample The MSE loss, l(f Θ (x i ),y i ) represents the sample x i ,y i MSE loss; λ represents the core set replay loss hyperparameter; M0 represents the core set; |M0| represents the core set size, w i Represents the sample weights of the core set.
7. The method for incremental training of meteorological models based on asynchronous updating according to claim 1, characterized in that: The use of the current long-term buffer and core set to update the meteorological data samples and sample weights in the core set includes: Where: M0 represents the core set, P is the size of the preset core set; g i represents the gradient direction of the ith core set meteorological data sample on the backbone network; L represents the union of the long-term buffer and the core set; |M0| represents the core set size; w i represents the updated sample weight of the i-th core set sample; <g i ,g j > represents the inner product of the ith core set sample and the ith core set sample; |g i | indicates g i The mold length.
8. The method for incremental training of meteorological models based on asynchronous updating according to claim 1, characterized in that: The backbone model adopts FourCastNet, Pangu-Weather or ConvLSTM meteorological model; the residual model adopts a deep neural network model.
9. A terminal device comprising a processor, a memory and a computer program stored in the memory; characterized in that: When the processor executes the computer program, it implements the meteorological model incremental training method based on asynchronous updating according to any one of claims 1-8.
10. A computer-readable storage medium, wherein a computer program is stored in the medium; characterized in that: When the computer program is executed by a processor, the incremental training method of a meteorological model based on asynchronous updating according to any one of claims 1 to 8 is implemented.