Multi-task timing prediction method, device, equipment, storage medium and program product

Abstract

The present invention provides a multi-task time series prediction method, device, equipment, storage medium and program product, which relate to the field of electronic digital data processing technology. The method includes: obtaining a target variable and a covariate; the target variable is a time series variable to be predicted, and the covariate is a time series variable used to assist in predicting the target variable; the target variable and the covariate are input into a multi-task time series model, and the target variable output by the multi-task time series model is obtained. The predicted value of the target variable within a preset time period in the future. Through the above method, the multi-task time series model can effectively use the information provided by the target variable and the covariate to jointly perform time series prediction, improve the generalization ability of the model, thereby improving the accuracy of time series prediction, and improving the model's utilization of important time series variable features, thereby facilitating the model's accurate prediction of the target variable.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a multi-task timing prediction method, device, equipment, storage medium and program product. Background Art

[0002] Time series forecasting is a technique that uses historical data to predict future trends. Time series forecasting is a real need in fields such as finance, meteorology, energy, and transportation. For example, in meteorology, weather data changes over time, and past weather data directly influences future weather data. Therefore, weather data is a predictable time series variable. By using recent historical weather data as input into a time series forecasting model, a time series forecast can be performed, producing forecasted weather data for the next day or week.

[0003] Traditional time series forecasting methods typically rely solely on historical data of the target variable for direct predictions, ignoring complex factors that indirectly influence future trends in the target variable. This leads to inaccurate forecasts when faced with nonlinear relationships or shifts in data distribution. For example, in the meteorological field, future changes in weather data are not only related to recent historical weather data but also indirectly to factors such as geographic location, season, and geological changes. However, traditional time series forecasting models are unable to effectively utilize this information, resulting in inaccurate weather data forecasts.

[0004] In the field of time series forecasting, factors that indirectly influence the future trend of the target variable are classified as covariates. Covariates are time series variables used to assist in predicting the target variable. Traditional time series forecasting methods cannot effectively utilize the information provided by covariates, resulting in limited model generalization capabilities and making it difficult to achieve accurate time series forecasting. Summary of the Invention

[0005] The present invention provides a multi-task time series prediction method, apparatus, device, storage medium and program product to address the defect that the existing technology cannot effectively utilize the information provided by covariates, resulting in limited model generalization ability and thus difficulty in achieving accurate time series prediction.

[0006] The present invention provides a multi-task time series prediction method, including: obtaining a target variable and a covariate; the target variable is a time series variable to be predicted, and the covariate is a time series variable used to assist in predicting the target variable; the target variable and the covariate are input into a multi-task time series model to obtain a predicted value of the target variable output by the multi-task time series model within a future preset time period; wherein the multi-task time series model is obtained based on training of sample time series variables, and the sample time series variables include sample target variables and sample covariates; the multi-task time series model is pre-trained using a preset loss function, and the preset loss function is obtained by performing a weighted aggregation operation based on the task loss of each task and the loss weight of each task loss, a task is determined based on a sample time series variable, a loss weight is determined based on the feature importance of a sample time series variable, and a feature importance is used to characterize the contribution of a sample time series variable to the prediction result of the multi-task time series model.

[0007] According to a multi-task timing prediction method provided by the present invention, the number of sample timing variables is M, the number of tasks is M, and M is a positive integer; the multi-task timing model is trained through the following steps: initializing the cache loss weights corresponding to the M tasks; based on the current number of iterations, judging whether the M cache loss weights need to be updated; if the M cache loss weights need to be updated, respectively determining the feature importance of the M sample timing variables; based on the M feature importances, respectively determining the loss weights of the M tasks, and updating the M loss weights to the cache loss weights of the M tasks; performing a weighted aggregation operation based on the M loss weights and the task losses of the M tasks to obtain a preset loss function; performing pre-training based on the M sample timing variables and the preset loss function, and updating the current number of iterations; if the pre-training is not completed, returning to the step of judging whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

[0008] According to a multi-task time series prediction method provided by the present invention, M sample time series variables include Q sample target variables and N sample covariates, where Q and N are positive integers; the importance of any feature is determined by the following steps: determining Q prediction importances corresponding to a sample time series variable; a prediction importance is the importance of a sample time series variable to a sample target variable; and performing an accumulation operation on the Q prediction importances to obtain the feature importance of a sample time series variable.

[0009] According to a multi-task time series prediction method provided by the present invention, a sample time series variable includes sample historical values of L historical time steps and sample predicted values of T future time steps, where L and T are positive integers; any prediction importance is determined by the following steps: determining the importance of multiple variables respectively; the importance of a variable is the importance of a sample historical value of a sample time series variable to a sample predicted value of a sample target variable; and performing cumulative operations on the importance of multiple variables to obtain a prediction importance.

[0010] According to a multi-task time series prediction method provided by the present invention, the importance of any variable is obtained by performing calculations based on a sample historical value of a sample time series variable and a sample predicted value of a sample target variable through an interpretability method; wherein the interpretability method is any one of a gradient saliency map method, an integral gradient method, and a SHAP algorithm.

[0011] According to a multi-task timing prediction method provided by the present invention, after determining whether M cache loss weights need to be updated based on the current number of iterations, the method further includes: if the M cache loss weights do not need to be updated, performing a weighted aggregation operation based on the M cache loss weights and the task losses of the M tasks to obtain a preset loss function; performing pre-training based on the M sample timing variables and the preset loss function, and updating the current number of iterations; if the pre-training is not completed, returning to the step of determining whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

[0012] The present invention also provides a multi-task time series prediction device, including: an acquisition module for acquiring target variables and covariates; the target variable is the time series variable to be predicted, and the covariate is the time series variable used to assist in predicting the target variable; a time series prediction module for inputting the target variable and the covariate into a multi-task time series model to obtain the predicted value of the target variable output by the multi-task time series model within a preset time period in the future; wherein, the multi-task time series model is obtained based on sample time series variable training, and the sample time series variables include sample target variables and sample covariates; the multi-task time series model is pre-trained using a preset loss function, and the preset loss function is obtained by weighted aggregation operation based on the task loss of each task and the loss weight of each task loss, a task is determined based on a sample time series variable, a loss weight is determined based on the feature importance of a sample time series variable, and a feature importance is used to characterize the contribution of a sample time series variable to the prediction result of the multi-task time series model.

[0013] The present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the multi-task timing prediction methods described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements any of the multi-task timing prediction methods described above.

[0015] The present invention also provides a computer program product, comprising a computer program, which implements any of the above-mentioned multi-task timing prediction methods when executed by a processor.

[0016] The multi-task time series prediction method, apparatus, equipment, storage medium and program product provided by the present invention introduce covariates and a multi-task learning framework in the process of time series prediction, and use the target variable and the covariate for auxiliary prediction as the input of the multi-task time series model, so that the multi-task time series model can effectively use the information provided by the target variable and the covariate to jointly perform time series prediction, improve the generalization ability of the model, and thus improve the accuracy of time series prediction; at the same time, the core of the multi-task learning framework is to simultaneously learn the features of multiple tasks through shared representation. When using sample target variables and sample covariates for model training at the same time, since the sample target variables and sample covariates are different types of sample time series variables, the multi-task time series model can perform feature learning of sample target variables and feature learning of sample covariates as different tasks respectively, thereby achieving the purpose of simultaneous multi-task learning of different types of sample time series variables, so that the trained model has the ability to process multiple time series variables at the same time, and thus can ensure model performance when introducing covariates for time series prediction, effectively improving the accuracy of time series prediction. Accuracy; In addition, in the traditional multi-task learning framework, the loss function used in model training focuses more on adjusting the loss balance between different tasks, that is, the loss weights of different tasks usually use fixed values, and the loss weights of each task tend to be balanced, so as to avoid the model from insufficiently learning some tasks and over-learning other tasks, thereby affecting the overall model performance. However, in the field of time series prediction, due to the different contributions and importance of different types of time series variables to the prediction of the target variable, directly using the traditional loss function for training may lead to insufficient utilization of the model's important time series variable features. Based on this, the multi-task time series model is trained using an improved preset loss function. The preset loss function can be obtained by weighted aggregation operation based on the task loss of each task and the loss weight of each task loss, and each loss weight is determined based on the feature importance of a sample time series variable, so that the tasks corresponding to the sample time series variables with high feature importance can get more attention from the model, improving the model's utilization of important time series variable features, which is conducive to the model's accurate prediction of the target variable. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flow chart of the multi-task timing prediction method provided by the present invention.

[0019] Figure 2 This is one of the training flow diagrams of the multi-task timing model provided by the present invention.

[0020] Figure 3 This is the second schematic diagram of the training process of the multi-task timing model provided by the present invention.

[0021] Figure 4 It is a structural diagram of the multi-task timing prediction device provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0024] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of the multi-task timing prediction method provided by the present invention. Figure 1 As shown, in this embodiment, the multi-task timing prediction method includes steps S110 to S120, and each step is specifically as follows:

[0025] S110: Obtain target variables and covariates.

[0026] The target variable is the time series variable to be predicted, and the covariate is the time series variable used to assist in predicting the target variable.

[0027] S120: Input the target variable and the covariate into the multi-task time series model to obtain the predicted value of the target variable output by the multi-task time series model within a future preset time period.

[0028] Among them, the multi-task time series model is trained based on sample time series variables, and the sample time series variables include sample target variables and sample covariates.

[0029] The multi-task time series model is pre-trained using a preset loss function. The preset loss function is obtained by weighted aggregation operation based on the task loss of each task and the loss weight of each task loss. A task is determined based on a sample time series variable, and a loss weight is determined based on the feature importance of a sample time series variable. A feature importance is used to characterize the contribution of a sample time series variable to the prediction result of the multi-task time series model.

[0030] It can be understood that there is at least one target variable and at least one covariate.

[0031] Specifically, assume that the number of target variables to be predicted is , the number of covariates is , The covariates represent indirect impact The input of the multi-task time series model is actually time series variables, and satisfy , 、 、 All are positive integers.

[0032] Will target variables and The covariates are input into the multi-task time series model for time series prediction, and the predicted value of each target variable output by the multi-task time series model within a preset future time period can be obtained.

[0033] The multi-task time series prediction method provided in this embodiment introduces covariates and a multi-task learning framework in the process of time series prediction, and takes the target variable and the covariate used for auxiliary prediction as the input of the multi-task time series model, so that the multi-task time series model can effectively use the information provided by the target variable and the covariate to jointly perform time series prediction, improve the generalization ability of the model, and thus improve the accuracy of time series prediction; at the same time, the core of the multi-task learning framework is to learn the features of multiple tasks simultaneously through shared representation. When using sample target variables and sample covariates for model training at the same time, since the sample target variables and sample covariates are different types of sample time series variables, the multi-task time series model can perform feature learning of sample target variables and feature learning of sample covariates as different tasks respectively, thereby achieving the purpose of performing multi-task learning on different types of sample time series variables at the same time, so that the trained model has the ability to process multiple time series variables at the same time, and thus can ensure model performance when introducing covariates for time series prediction, effectively improving the accuracy of time series prediction; in addition, In the traditional multi-task learning framework, the loss function used in model training focuses more on adjusting the loss balance between different tasks. That is, the loss weights of different tasks usually use fixed values, and the loss weights of each task tend to be balanced to avoid the model under-learning some tasks and over-learning other tasks, thereby affecting the overall model performance. However, in the field of time series prediction, since different types of time series variables have different contributions and importance to the prediction of the target variable, directly using traditional loss functions for training may lead to insufficient utilization of important time series variable features by the model. Based on this, the multi-task time series model is trained using an improved preset loss function. The preset loss function can be obtained by weighted aggregation operation based on the task loss of each task and the loss weight of each task loss. Each loss weight is determined based on the feature importance of a sample time series variable, so that the tasks corresponding to sample time series variables with high feature importance can receive more attention from the model, improving the model's utilization of important time series variable features, which is conducive to the model's accurate prediction of the target variable.

[0034] In some embodiments, the number of sample timing variables is M, the number of tasks is M, and M is a positive integer; the multi-task timing model is trained through the following steps: initializing the cache loss weights corresponding to the M tasks; based on the current number of iterations, judging whether the M cache loss weights need to be updated; if the M cache loss weights need to be updated, respectively determine the feature importance of the M sample timing variables; based on the M feature importances, respectively determine the loss weights of the M tasks, and update the M loss weights to the cache loss weights of the M tasks; perform weighted aggregation operations based on the M loss weights and the task losses of the M tasks to obtain a preset loss function; perform pre-training based on the M sample timing variables and the preset loss function, and update the current number of iterations; if the pre-training is not completed, return to the step of judging whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

[0035] The core concept of this embodiment is to simultaneously learn and predict different target variables and covariates using a multi-task time series model built on a multi-task learning framework. When calculating a preset loss function, the loss weights of each task are dynamically adjusted based on the feature importance of different sample time series variables. Based on this, this embodiment improves the training process of the multi-task time series model.

[0036] See also Figure 2 and Figure 3 , Figure 2 This is one of the training flow diagrams of the multi-task timing model provided by the present invention. Figure 3 This is the second schematic diagram of the training process of the multi-task timing model provided by the present invention.

[0037] The training process of the multi-task timing model includes the initialization phase and the cyclic training phase.

[0038] Specifically, if Figure 2 As shown, in the initialization phase, the time series prediction task of the multi-task time series model is first defined: Assume that the input data of the model in the application phase is time series variables, including target variables and covariates, the model also needs sample time series variables, including sample target variables and This modeling method using a multi-task learning framework can add both the target variable and the covariate to the time series prediction task of the model. The model can perform feature learning of the sample target variable and feature learning of the sample covariate as different tasks. Therefore, in the model training stage, For each sample time series variable, the model will have M corresponding tasks, with one task corresponding to one sample time series variable.

[0039] Furthermore, initialize the parameters of the multi-task timing model, that is, initialize the cache loss weights corresponding to the M tasks .

[0040] It should be noted that since the target variable itself is a direct factor affecting its future change trend, its contribution and importance to the prediction of the future change trend of the target variable are relatively large. Therefore, the cache loss weight of the task corresponding to the sample target variable is usually large. Similarly, the covariate is an indirect factor affecting the future change trend of the target variable. Its contribution and importance to the prediction of the future change trend of the target variable are relatively small. Therefore, the cache loss weight of the task corresponding to the sample covariate is usually small.

[0041] Optionally, The cache loss weights of the tasks corresponding to the sample target variables follow a uniform distribution with a sum of 0.8; The cache loss weights of the tasks corresponding to the sample covariates follow a uniform distribution with a sum of 0.2.

[0042] Furthermore, set the hyperparameters: set the weight update frequency K, that is, update the M loss weights every K iterations, and then update the preset loss function according to the updated M loss weights; at the same time, set the number of samples per iteration S, the minimum loss weight of the task corresponding to each sample target variable and the smoothing coefficient of the Exponential Moving Average (EMA) algorithm .

[0043] It should be noted that in each round of pre-training, for each sample target variable, the loss weight of the corresponding task must meet the minimum loss weight This is to ensure that during multiple rounds of training, the model will not gradually reduce or even ignore the learning of sample target variable features due to the dynamic update of loss weights, thereby avoiding knowledge forgetting.

[0044] Furthermore, all sample time series variable data are loaded and standardized or normalized.

[0045] After completing the above settings, you can enter the cycle training stage.

[0046] like Figure 3 As shown in FIG, in the cyclic training phase, for the tth iteration, first determine whether the M cache loss weights need to be updated based on the current iteration number t (t is initialized to 0).

[0047] Specifically, the weight update frequency K is modulo the current iteration number t.

[0048] If the result of the current number of iterations t modulo the weight update frequency K is 0, then (in, represents the remainder function), then the M cache loss weights need to be updated.

[0049] At this time, it is necessary to determine the feature importance of the M sample time series variables respectively, determine the loss weights of the M tasks based on the M feature importances, and update the M loss weights to the cache loss weights of the M tasks respectively.

[0050] Specifically, after determining the feature importance of the M sample time series variables, the M feature importances are first normalized, and the loss weight of the task corresponding to each sample target variable is guaranteed to meet the minimum loss weight. constraint.

[0051] It should be noted that in each iteration, the loss weights of different tasks usually change. The purpose of normalization is to ensure that the sum of the importance of M features is 1 for subsequent calculations.

[0052] Optionally, after normalization, the M feature importances are used as the loss weights of the M tasks, and the M loss weights are updated as the cache loss weights of the M tasks. .

[0053] Preferably, after normalization, the M feature importances are used as the loss weights of the M tasks, and the exponential moving average algorithm is used to smooth the loss weights of the M tasks to obtain the smoothed loss weights of the M tasks. Finally, the smoothed loss weights of the M tasks are updated as the cache loss weights of the M tasks. .

[0054] Among them, the loss weight for the mth task is , the formula for updating the corresponding cache loss weight using the moving average algorithm is as follows:

[0055] ;

[0056] in, Indicates the The cache loss weight of the mth task is updated using the moving average algorithm in the iteration; Indicates the The cache loss weight of the mth task is updated using the moving average algorithm in the iteration; is the smoothing coefficient.

[0057] At this time, the cache loss weights of the updated M tasks are That is, the M loss weights participating in the calculation in this round are weighted and aggregated based on the M loss weights and the task losses of the M tasks to obtain the dynamically weighted preset loss function.

[0058] Among them, the preset loss function The calculation formula is as follows:

[0059] ;

[0060] in, represents the loss weight of the mth task; represents the loss of the mth task.

[0061] Furthermore, pre-training is performed based on the M sample time series variables and the preset loss function to update the model parameters and the current number of iterations.

[0062] It can be understood that in each iteration, only M sample time series variables are used for training, rather than directly using a large number of sample time series variables for simultaneous training. This can effectively reduce the frequency of sample sampling in each round of pre-training, thereby reducing the computational overhead required for the model in each round of pre-training, improving the efficiency of each round of pre-training, and avoiding a sudden increase in model computational overhead leading to low training efficiency.

[0063] Furthermore, it is determined whether the pre-training is completed.

[0064] If the pre-training is not completed, the process returns to the step of determining whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

[0065] The multi-task time series prediction method provided in this embodiment can dynamically adjust the loss weights of different tasks according to the feature importance of different sample time series variables during the model pre-training process, optimize the feature learning of sample time series variables that contribute greatly to the prediction of the target variable, so that the tasks corresponding to sample time series variables with large prediction contributions and high feature importance can receive more attention from the model, improve the model's utilization of important time series variable features, and thus facilitate the model's accurate prediction of the target variable. This dynamic loss weighting mechanism enables the model to adapt to changes in data distribution or task requirements, thereby improving the model's adaptability. At the same time, since the model can fully utilize and learn the potential correlation between sample covariates and sample target variables, the model's prediction accuracy can be further improved.

[0066] In some embodiments, the M sample time series variables include Q sample target variables and N sample covariates, where Q and N are positive integers; the importance of any feature is determined by the following steps: determining Q prediction importances corresponding to a sample time series variable; a prediction importance is the importance of a sample time series variable to a sample target variable; and performing cumulative operations on the Q prediction importances to obtain the feature importance of a sample time series variable.

[0067] Specifically, for the mth sample time series variable, its importance to the Q sample target variables is calculated respectively as its corresponding Q prediction importances, and the Q prediction importances are accumulated to obtain its feature importance.

[0068] From the above steps, we can see that the feature importance of the mth sample time series variable The calculation formula is as follows:

[0069] ;

[0070] in, Indicates the sample target variables; Indicates the mth sample time series variable to the The importance of the target variable for each sample.

[0071] In some embodiments, a sample time series variable includes sample historical values of L historical time steps and sample predicted values of T future time steps, where L and T are positive integers; any prediction importance is determined by the following steps: determining the importance of multiple variables respectively; the importance of a variable is the importance of a sample historical value of a sample time series variable to a sample predicted value of a sample target variable; and performing cumulative operations on multiple variable importances to obtain a prediction importance.

[0072] Specifically, for the mth sample time series variable, it includes sample historical values of L historical time steps and sample predicted values of T future time steps.

[0073] like Figure 2 As shown in the figure, in each round of pre-training, for each sample time series variable, the learning goal of the multi-task time series model is to perform feature learning and prediction based on the sample historical values of the input L historical time steps, and to mine the correlation between the sample historical values of the output L historical time steps and the sample predicted values of T future time steps, so as to determine the impact of the sample historical values on the sample predicted values for subsequent prediction.

[0074] For the mth sample time series variable, since it includes the sample history values of L historical time steps, the sample history values of L historical time steps can be defined as a multivariate time series with a historical window length of L :

[0075] ;

[0076] in, ; Indicates the mth sample time series variable The sample history value of historical time steps, .

[0077] Since the M sample time series variables include Q sample target variables and N sample covariates, assuming that the first Q variables are sample target variables and the last N variables are sample covariates, the Q sample target variables can be recorded as , the N sample covariates are recorded as .

[0078] Similarly, for the mth sample time series variable, since it includes the sample prediction values of T future time steps, the sample prediction values of T future time steps can be defined as a multivariate time series with a window length of T :

[0079] ;

[0080] in, ; Indicates the mth sample time series variable The sample prediction value of future time steps, .

[0081] Furthermore, for the mth sample time series variable and the sample target variables, assuming the The target variable of the sample is The output of the future time step is , then the mth sample time series variable Sample historical values of historical time steps For the first The target variable of the sample is Sample prediction values for future time steps Importance , which can be obtained by interpretable method. The calculation formula is as follows:

[0082] ;

[0083] in, , represents the historical time step; , represents the future time step; , represents the index of the sample time series variable; Represents an interpretability method or model.

[0084] Furthermore, the mth sample time series variable has an effect on the Importance of target variables for each sample (i.e., the mth sample time series variable corresponds to the The importance of predictions) can be expressed by the following formula:

[0085] ;

[0086] Among them, when hour, Can represent the The contribution of the sample historical value of a sample target variable to the prediction of its own future change trend.

[0087] In some embodiments, the importance of any variable is obtained by performing calculations based on a sample historical value of a sample time series variable and a sample predicted value of a sample target variable through an interpretability method; wherein the interpretability method is any one of the gradient significance map method, the integral gradient method, and the SHAP algorithm.

[0088] The mth sample time series variable Sample historical values of historical time steps For the first The target variable of the sample is Sample prediction values for future time steps Importance , can be obtained by any one of the operations of gradient saliency map method, integral gradient method, and SHAP algorithm. Therefore, The calculation formula is as follows:

[0089] ;

[0090] in, , represents the historical time step; , represents the future time step; , represents the index of the sample time series variable; Represents an interpretability method or model.

[0091] In some embodiments, after determining whether the M cache loss weights need to be updated based on the current number of iterations, the method further includes: if the M cache loss weights do not need to be updated, performing a weighted aggregation operation based on the M cache loss weights and the task losses of the M tasks to obtain a preset loss function; performing pre-training based on the M sample time series variables and the preset loss function, and updating the current number of iterations; if the pre-training is not completed, returning to the step of determining whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

[0092] like Figure 3 As shown in , if there is no need to update the M cache loss weights, the M cache loss weights of the last iteration can be directly obtained as the M loss weights participating in the calculation in this round.

[0093] Furthermore, a weighted aggregation operation is performed based on the M cache loss weights and the task losses of the M tasks to obtain a preset loss function.

[0094] Furthermore, pre-training is performed based on the M sample time series variables and the preset loss function, and the current number of iterations is updated.

[0095] It can be understood that in each iteration, only M sample time series variables are used for training, rather than directly using a large number of sample time series variables for simultaneous training. This can effectively reduce the frequency of sample sampling in each round of pre-training, thereby reducing the computational overhead required for the model in each round of pre-training, improving the efficiency of each round of pre-training, and avoiding a sudden increase in model computational overhead leading to low training efficiency.

[0096] Furthermore, it is determined whether the pre-training is completed.

[0097] If the pre-training is not completed, the process returns to the step of determining whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

[0098] The multi-task time series prediction method provided in this embodiment does not choose to update the loss weight in all rounds of pre-training, but chooses to update the loss weight in some rounds. This can avoid the increase in model calculation overhead caused by too frequent loss weight updates, and can effectively improve the training efficiency of the model.

[0099] To facilitate understanding of the multi-task timing prediction method of the present invention, the present invention also provides a specific example of the multi-task timing prediction method.

[0100] In the field of meteorology, meteorological data changes over time. Past meteorological data directly influences future changes in meteorological data, making it a predictable time-series variable. Assuming the meteorological data to be predicted includes cloud trajectory, rainfall, and ultraviolet radiation intensity, and that factors that indirectly influence the future trends of these three factors include season and longitude and latitude, cloud trajectory, rainfall, and ultraviolet radiation intensity can be used as the three target variables to be predicted, and season and longitude and latitude can be used as covariates to assist in predicting the target variables.

[0101] By inputting cloud movement trajectory, rainfall, ultraviolet radiation intensity, season, and longitude and latitude into the multi-task time series model, the multi-task time series model can output the predicted value of cloud movement trajectory in the future preset time period, the predicted value of rainfall in the future preset time period, and the predicted value of ultraviolet radiation intensity in the future preset time period.

[0102] Among them, the multi-task time series model is trained based on sample time series variables, which include sample cloud movement trajectory, sample rainfall, sample ultraviolet radiation intensity, sample seasonal information and sample longitude and latitude.

[0103] The multi-task time series model uses a preset loss function for pre-training. The preset loss function is obtained by weighted aggregation operation based on the task loss of each task and the loss weight of each task loss. A task is determined based on a sample time series variable, and a loss weight is determined based on the feature importance of a sample time series variable.

[0104] In each round of training in the cyclic training phase, five sample time series variables are required, namely three sample target variables (i.e., sample cloud movement trajectory, sample rainfall, and sample ultraviolet radiation intensity) and two sample covariates (i.e., sample seasonal information and sample latitude and longitude).

[0105] After initializing the cache loss weights corresponding to the five tasks, it can be determined whether the five cache loss weights need to be updated based on the current number of iterations.

[0106] If the five cache loss weights need to be updated, the feature importance of the five sample time series variables is determined respectively.

[0107] In this embodiment, the five feature importances are the feature importance of the sample cloud movement trajectory, the feature importance of the sample rainfall, the feature importance of the sample ultraviolet radiation intensity, the feature importance of the sample seasonal information, and the feature importance of the sample latitude and longitude.

[0108] Among them, the importance of any feature is obtained by summing up its corresponding three prediction importances. A prediction importance is the importance of a sample time series variable to a sample target variable.

[0109] Taking the feature importance of a sample cloud trajectory as an example, this feature importance is calculated by accumulating the importance of the sample cloud trajectory to the prediction of the sample cloud trajectory, the importance of the sample cloud trajectory to the prediction of the sample rainfall, and the importance of the sample cloud trajectory to the prediction of the sample ultraviolet radiation intensity. The calculation of other feature importances can be deduced in a similar way and is not repeated here.

[0110] Each sample time series variable includes sample historical values of L historical time steps and sample predicted values of T future time steps, where L and T are positive integers. The importance of any prediction is obtained by cumulatively adding the importance of multiple variables. The importance of a variable is the importance of a sample historical value of a sample time series variable to a sample predicted value of a sample target variable, which can be obtained through interpretability calculations.

[0111] Taking the importance of sample latitude and longitude for the prediction of sample rainfall as an example, assuming that the sample latitude and longitude include sample historical values from two historical time steps, and the sample rainfall includes a sample predicted value from one future time step, we can calculate the importance of the first sample historical value of the sample latitude and longitude to the first sample predicted value of the sample rainfall as the first variable importance. Similarly, we can calculate the importance of the second sample historical value of the sample latitude and longitude to the first sample predicted value of the sample rainfall as the second variable importance. By adding the importance of the two variables, we can obtain the predictive importance of the sample latitude and longitude for the sample rainfall. The calculation of other predictive importances can be deduced in this way and will not be repeated here.

[0112] Furthermore, based on the importance of the five features, the loss weights of the five tasks are determined respectively, and the five loss weights are updated as the cache loss weights of the five tasks respectively.

[0113] Furthermore, a weighted aggregation operation is performed based on the 5 loss weights and the task losses of the 5 tasks to obtain a preset loss function; pre-training is performed based on the 5 sample time series variables and the preset loss function, and the current number of iterations is updated.

[0114] If the pre-training is not completed, the process returns to the step of determining whether the five cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

[0115] The present invention also provides a multi-task timing prediction device. Figure 4 , Figure 4FIG4 is a schematic diagram of the structure of the multi-task timing prediction device provided by the present invention. In this embodiment, the multi-task timing prediction device includes an acquisition module 410 and a timing prediction module 420.

[0116] The acquisition module 410 is used to acquire target variables and covariates.

[0117] The target variable is the time series variable to be predicted, and the covariate is the time series variable used to assist in predicting the target variable.

[0118] The time series prediction module 420 is used to input the target variable and the covariate into the multi-task time series model to obtain the predicted value of the target variable output by the multi-task time series model within a future preset time period.

[0119] Among them, the multi-task time series model is trained based on sample time series variables, and the sample time series variables include sample target variables and sample covariates.

[0120] The multi-task time series model is pre-trained using a preset loss function. The preset loss function is obtained by weighted aggregation operation based on the task loss of each task and the loss weight of each task loss. A task is determined based on a sample time series variable, and a loss weight is determined based on the feature importance of a sample time series variable. A feature importance is used to characterize the contribution of a sample time series variable to the prediction result of the multi-task time series model.

[0121] In some embodiments, the number of sample timing variables is M, the number of tasks is M, and M is a positive integer.

[0122] The multi-task timing model is trained through the following steps: initialize the cache loss weights corresponding to M tasks; based on the current number of iterations, determine whether the M cache loss weights need to be updated; if the M cache loss weights need to be updated, determine the feature importance of the M sample timing variables respectively; based on the M feature importances, determine the loss weights of the M tasks respectively, and update the M loss weights to the cache loss weights of the M tasks respectively; perform weighted aggregation operations based on the M loss weights and the task losses of the M tasks to obtain a preset loss function; perform pre-training based on the M sample timing variables and the preset loss function, and update the current number of iterations; if the pre-training is not completed, return to the step of determining whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

[0123] In some embodiments, the M sample time series variables include Q sample target variables and N sample covariates, where Q and N are positive integers.

[0124] The importance of any feature is determined through the following steps: determining Q prediction importances corresponding to a sample time series variable; a prediction importance is the importance of a sample time series variable to a sample target variable; and accumulating the Q prediction importances to obtain the feature importance of a sample time series variable.

[0125] In some embodiments, a sample time series variable includes sample historical values of L historical time steps and sample predicted values of T future time steps, where L and T are positive integers.

[0126] Any prediction importance is determined through the following steps: determine the importance of multiple variables separately; the importance of a variable is the importance of a sample historical value of a sample time series variable to a sample predicted value of a sample target variable; and perform cumulative operations on the importance of multiple variables to obtain a prediction importance.

[0127] In some embodiments, the importance of any variable is obtained by performing calculations based on a sample historical value of a sample time series variable and a sample predicted value of a sample target variable through an interpretability method; wherein the interpretability method is any one of the gradient significance map method, the integral gradient method, and the SHAP algorithm.

[0128] In some embodiments, after determining whether the M cache loss weights need to be updated based on the current number of iterations, the method further includes: if the M cache loss weights do not need to be updated, performing a weighted aggregation operation based on the M cache loss weights and the task losses of the M tasks to obtain a preset loss function; performing pre-training based on the M sample time series variables and the preset loss function, and updating the current number of iterations; if the pre-training is not completed, returning to the step of determining whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

[0129] The present invention also provides an electronic device. Figure 5 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call logic instructions in the memory 530 to execute the multi-task timing prediction method.

[0130] Furthermore, the logic instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0131] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the multi-task timing prediction method provided by the above methods is implemented.

[0132] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-task timing prediction method provided by the above methods.

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0134] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A multi-task time series prediction method, characterized in that: Applications in meteorological fields include: Obtain three target variables and two covariates; the target variable is a time series variable to be predicted, and the covariates are time series variables used to assist in predicting the target variable. The target variables include cloud movement trajectory, rainfall, and ultraviolet radiation intensity, and the covariates include season and longitude and latitude; Inputting the target variable and the covariate into a multi-task time series model, and obtaining a predicted value of the target variable within a future preset time period output by the multi-task time series model; the predicted value of the target variable within the future preset time period includes the predicted value of the cloud movement trajectory within the future preset time period, the predicted value of the rainfall within the future preset time period, and the predicted value of the ultraviolet radiation intensity within the future preset time period; The multi-task time series model is obtained by training based on sample time series variables, wherein the sample time series variables include sample target variables and sample covariates, and the sample time series variables include sample cloud movement trajectory, sample rainfall, sample ultraviolet radiation intensity, sample season information, and sample longitude and latitude. The sample target variable includes the sample cloud movement trajectory, the sample rainfall, and the sample ultraviolet radiation intensity, and the sample covariates include the sample season information and the sample longitude and latitude. The multi-task time series model is pre-trained using a preset loss function, where the preset loss function is obtained by performing a weighted aggregation operation based on the task loss of each task and the loss weight of each task loss, wherein one task is determined based on one sample time series variable, and one loss weight is determined based on the feature importance of one sample time series variable, where one feature importance is used to characterize the contribution of one sample time series variable to the prediction result of the multi-task time series model; The number of the sample time series variables is M, the number of the tasks is M, M is a positive integer, and M is 5; The multi-task timing model is trained by the following steps: Initialize the cache loss weights corresponding to the M tasks; Based on the current number of iterations, determining whether the M cache loss weights need to be updated; If the M cache loss weights need to be updated, the feature importance of the M sample time series variables is determined respectively; Based on the M feature importances, respectively determine the loss weights of the M tasks, and update the M loss weights to the cache loss weights of the M tasks; Performing a weighted aggregation operation based on the M loss weights and the task losses of the M tasks to obtain the preset loss function; Perform pre-training based on the M sample time series variables and the preset loss function, and update the current number of iterations; If the pre-training is not completed, return to the step of determining whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed; The M sample time series variables include Q sample target variables and N sample covariates, where Q and N are positive integers, Q is 3, and N is 2; The importance of any of the features is determined by the following steps: Determine Q prediction importances corresponding to one of the sample time series variables; one of the prediction importances is the importance of one of the sample time series variables to one of the sample target variables; The Q prediction importances are cumulatively added to obtain the feature importance of one sample time series variable.

2. The multi-task time series prediction method according to claim 1, characterized in that: A sample time series variable includes sample historical values of L historical time steps and sample predicted values of T future time steps, where L and T are positive integers; The importance of any of the predictions is determined by the following steps: Determining the importance of multiple variables respectively; the importance of one variable is the importance of a sample historical value of a sample time series variable to a sample predicted value of a sample target variable; A cumulative operation is performed on the importance of multiple variables to obtain the predicted importance.

3. The multi-task time series prediction method according to claim 2, characterized in that: Any of the variable importances is obtained by performing calculations based on a sample historical value of a sample time series variable and a sample predicted value of a sample target variable through an interpretability method; The interpretability method is any one of the gradient saliency map method, the integral gradient method, and the SHAP algorithm.

4. The multi-task time series prediction method according to claim 1, characterized in that: After determining whether the M cache loss weights need to be updated based on the current number of iterations, the method further includes: If the M cache loss weights do not need to be updated, performing a weighted aggregation operation based on the M cache loss weights and the task losses of the M tasks to obtain the preset loss function; Perform pre-training based on the M sample time series variables and the preset loss function, and update the current number of iterations; If the pre-training is not completed, the process returns to the step of determining whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed.

5. A multi-task timing prediction device, characterized in that: include: The acquisition module is used to obtain three target variables and two covariates; The target variable is a time series variable to be predicted, and the covariate is a time series variable used to assist in predicting the target variable. The target variable includes cloud movement trajectory, rainfall, and ultraviolet radiation intensity, and the covariate includes season and longitude and latitude. A time series prediction module, configured to input the target variable and the covariate into a multi-task time series model, and obtain a predicted value of the target variable within a future preset time period output by the multi-task time series model; The predicted value of the target variable in the future preset time period includes the predicted value of the cloud movement trajectory in the future preset time period, the predicted value of the rainfall in the future preset time period, and the predicted value of the ultraviolet radiation intensity in the future preset time period; The multi-task time series model is obtained by training based on sample time series variables, wherein the sample time series variables include sample target variables and sample covariates, and the sample time series variables include sample cloud movement trajectory, sample rainfall, sample ultraviolet radiation intensity, sample season information, and sample longitude and latitude. The sample target variable includes the sample cloud movement trajectory, the sample rainfall, and the sample ultraviolet radiation intensity, and the sample covariates include the sample season information and the sample longitude and latitude. The multi-task time series model is pre-trained using a preset loss function, where the preset loss function is obtained by performing a weighted aggregation operation based on the task loss of each task and the loss weight of each task loss, wherein one task is determined based on one sample time series variable, and one loss weight is determined based on the feature importance of one sample time series variable, where one feature importance is used to characterize the contribution of one sample time series variable to the prediction result of the multi-task time series model; The number of the sample time series variables is M, and the number of the tasks is M, where M is a positive integer and M is 5; the multi-task time series model is trained by the following steps: initializing the cache loss weights corresponding to the M tasks; based on the current number of iterations, judging whether the M cache loss weights need to be updated; if the M cache loss weights need to be updated, respectively determining the feature importance of the M sample time series variables; based on the M feature importances, respectively determining the loss weights of the M tasks, and respectively updating the M loss weights to the cache loss weights of the M tasks; performing weighted aggregation operations based on the M loss weights and the task losses of the M tasks to obtain the preset loss function; performing pre-training based on the M sample time series variables and the preset loss function, and updating the current number of iterations; if the pre-training is not completed, returning to the step of judging whether the M cache loss weights need to be updated based on the current number of iterations until the pre-training is completed; The M sample time series variables include Q sample target variables and N sample covariates, Q and N are positive integers, Q is 3, and N is 2; any one of the feature importances is determined by the following steps: determining Q prediction importances corresponding to one of the sample time series variables; one prediction importance is the importance of one of the sample time series variables to one of the sample target variables; and performing cumulative operations on the Q prediction importances to obtain the feature importance of one of the sample time series variables.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the multi-task timing prediction method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-task timing prediction method according to any one of claims 1 to 4 is implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-task timing prediction method according to any one of claims 1 to 4 is implemented.

Images