Integrated model training method and device, time sequence prediction method and device and time sequence prediction integrated model
Through the timing prediction integration model, combined with the basic model and expert network, the problems of low prediction accuracy and overfitting in high volatility and non-stationary data are solved, achieving higher prediction accuracy and robustness.
Patent Information
- Application Number
- CN202410797406.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-19
- Publication Date
- 2024-11-08
AI Technical Summary
Existing time series prediction methods have problems with low prediction accuracy and overfitting in high volatility and non-stationarity data. Traditional methods lack fine-grained feature extraction, while machine learning-based methods can easily lead to one-sidedness based on a single model.
A time series prediction ensemble model is proposed, including a base model group, an expert network group and a gated network. The initial prediction results of the base model are input to the expert network for feature extraction and encoding, and the gated network filters the adapted base model to generate result predictions. Sequences reduce the dependence of a single model and improve prediction accuracy.
Through the design of the integrated model, overfitting is reduced, prediction accuracy and robustness are improved, and it is suitable for time series data with high volatility and non-stationarity.
Smart Images

Figure CN118917356A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to technical fields such as deep learning, machine learning, and large models, and can be applied to scenarios such as artificial intelligence content generation, and in particular to a time series prediction integrated model, a time series prediction integrated model training method and device, a time series prediction method and device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art
[0002] The problem of time series prediction has always been the focus of researchers. Existing time series prediction analysis methods are mainly divided into two categories: traditional time series analysis methods and machine learning-based prediction methods. Traditional time series analysis methods analyze data trends and patterns to try to predict the future development of data. This method lacks the extraction of fine-grained features, so the prediction results are not accurate. The prediction method based on machine learning, which relies only on constructed features, may lead to one-sided predictions. In the prediction of highly volatile time series, a single model is prone to one-sided capture of time series signals, resulting in overfitting. Summary of the invention
[0003] The present disclosure provides a time series prediction integrated model, a time series prediction integrated model training method and device, a time series prediction method and device, an electronic device, a computer-readable storage medium, and a computer program product.
[0004] According to a first aspect, a time series prediction integrated model is provided, which includes: a base model group, in which the base models in the base model group are used to perform time series prediction on the input time series data to obtain an initial prediction sequence; an expert network group, in which an expert network in the expert network group is connected to a base model in the base model group, and the expert network is used to extract features and encode the initial prediction sequence output by the corresponding base model; a gating network, which is used to generate corresponding weights for the expert networks in the expert network group; and a prediction network, which generates a result prediction sequence based on the outputs of the expert network group and the gating network.
[0005] According to a second aspect, a method for training a time series prediction integrated model is provided, the method comprising: obtaining a sample set, the sample set comprising at least one sample, the sample comprising: time series data in a sample time period; obtaining an initial time series prediction integrated model, the time series prediction integrated model adopting the time series prediction integrated model as described in any implementation method of the first aspect; based on the sample set, training an expert network group, a gating network, and a prediction network in the time series prediction integrated model; in response to the expert network group, the gating network, and the prediction network satisfying a first training completion condition, obtaining a trained time series prediction integrated model.
[0006] According to the third aspect, a time series prediction method is provided, which includes: obtaining historical time series data and a time series prediction integration model for a historical time period, wherein the time series prediction integration model is a trained time series prediction integration model obtained by training using the time series prediction integration model training method described in any implementation method of the second aspect; inputting the historical time series data into the time series prediction integration model to obtain a result prediction sequence output by the time series prediction integration model.
[0007] According to a fourth aspect, a time series prediction integrated model training device is provided, the device comprising: a sample acquisition unit, configured to acquire a sample set, the sample set comprising at least one sample, the sample comprising: time series data in a sample time period; a model acquisition unit, configured to acquire an initial time series prediction integrated model, the time series prediction integrated model adopts the time series prediction integrated model as described in any implementation method of the first aspect; an expert training unit, configured to train an expert network group, a gating network and a prediction network in the time series prediction integrated model based on the sample set; and an obtaining unit, configured to obtain a trained time series prediction integrated model in response to the expert network group, the gating network and the prediction network satisfying a first training completion condition.
[0008] According to the fifth aspect, a time series prediction device is provided, which includes: an information acquisition unit, configured to acquire historical time series data and a time series prediction integrated model for a historical time period, the time series prediction integrated model being a trained time series prediction integrated model obtained by the time series prediction integrated model training method described in any implementation method of the fourth aspect; an input unit, configured to input historical time series data into the time series prediction integrated model to obtain a result prediction sequence output by the time series prediction integrated model.
[0009] According to the sixth aspect, an electronic device is provided, which includes: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation of the second aspect or the third aspect.
[0010] According to a seventh aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method described in any implementation of the second aspect or the third aspect.
[0011] According to an eighth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method described in any implementation of the second aspect or the third aspect.
[0012] The time series prediction integrated model provided by the embodiment of the present disclosure includes: a base model group, in which the base models in the base model group are used to perform time series prediction on the input time series data to obtain an initial prediction sequence; an expert network group, in which an expert network in the expert network group is connected to a base model in the base model group, and the expert network is used to extract features and encode the initial prediction sequence output by the corresponding base model; a gating network, which is used to generate corresponding weights for the expert networks in the expert network group; and a prediction network, which generates a result prediction sequence based on the output of the expert network group and the gating network. Thus, the initial prediction result of the base model is input into the expert network group, and the adapted base model is selected based on the screening mechanism of the gating network, which no longer relies on the structural features of a single base model, reduces the occurrence of overfitting caused by the one-sidedness of the model prediction, and improves the accuracy of the prediction.
[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0015] Figure 1 is a structural schematic diagram of an embodiment of the time series prediction integrated model disclosed in the present invention;
[0016] Figure 2 is a structural schematic diagram of another embodiment of the time series prediction integrated model disclosed in the present invention;
[0017] Figure 3 is a flowchart of an embodiment of a time series prediction integrated model training method disclosed in the present invention;
[0018] Figure 4 is a flow chart of an embodiment of a time series prediction method according to the present disclosure;
[0019] Figure 5 is a structural schematic diagram of an embodiment of a time series prediction integrated model training device disclosed in the present invention;
[0020] Figure 6 is a schematic structural diagram of an embodiment of a time series prediction device according to the present disclosure;
[0021] Figure 7 It is a block diagram of an electronic device used to implement the time series prediction integrated model training method or the time series prediction method of the embodiment of the present disclosure. DETAILED DESCRIPTION
[0022] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0023] The problem of time series prediction has always been the focus of researchers, and it plays an important role in many fields such as power consumption prediction, stock price prediction, and meteorological weather prediction. However, most time series have the characteristics of non-stationarity, high volatility, and low signal-to-noise ratio. Non-stationarity describes the fluctuation of the statistics of the time series over time; high volatility and low signal-to-noise ratio describe the unpredictability of these time series.
[0024] Modern forecasting analysis methods can be divided into two categories: traditional time series analysis methods and machine learning-based methods. Traditional time series analysis methods, such as autoregressive moving average model (ARIMA), autoregressive conditional heteroskedasticity model (ARCH), generalized autoregressive conditional heteroskedasticity model (GARCH), etc., all focus on the characteristics of data on the time axis. These methods try to predict future developments by analyzing past data trends and patterns. However, this method lacks the extraction of fine-grained features, so the forecast results are not satisfactory. On the other hand, with the advancement of data science and computer science, many forecasting methods based on machine learning have emerged, such as artificial neural networks (ANN), support vector regression (SVR), etc. These methods have achieved remarkable success in many applications. However, relying solely on constructed features may lead to one-sided predictions. In the prediction of time series with high volatility, a single model is prone to one-sided capture of time series signals, resulting in overfitting.
[0025] In the field of modern machine learning and data analysis, model integration has become a very popular strategy that combines multiple models to improve the accuracy of overall predictions. Unlike the single model approach, model integration can introduce diversity between different models, which helps to offset the possible defects and biases of a single model. Model integration strategies mainly include boosting, bagging, stacking and blending.
[0026] Boosting is an iterative ensemble method that aims to reduce bias. Boosting trains models in a sequential manner, with each training run on the residual or error of the previous run. This means that subsequent models pay more attention to data points where the previous model performed poorly. Common boosting algorithms include AdaBoost, GradientBoosting, and XGBoost. In AdaBoost, each data point is assigned a weight, and these weights are equal at the beginning. But with each iteration, the weights of data points that are misclassified increase, while the weights of data points that are correctly classified decrease, making subsequent models pay more attention to points that are difficult to classify. Secondly, the bagging ensemble strategy, or bootstrap pooling, has the main goal of reducing variance. This method trains the model by extracting multiple subsamples from the original data with replacement. Because the data used for each training is slightly different, multiple different models are obtained. The prediction result is the average or vote of the predictions of all individual models. The most famous bagging algorithm is random forest. Stacking is a more advanced integration strategy in which the predictions of multiple base models (also called primary models) are used as input to another model (also called secondary or meta-model) to make the final prediction. The basic idea is that the combination of multiple models may be better than any single model. Blending is similar to stacking, but different. Unlike stacking, which uses a meta-model, blending usually uses simple models such as linear regression or logistic regression for prediction. Another feature of blending is that it does not use all the training data to train the primary model, but divides the training data into two parts, one part is used to train the base model, and the other part is used as a training set for the meta-model.
[0027] Time series data often exhibits the characteristics of non-stationarity, high volatility and low signal-to-noise ratio, and the causal relationship in time series data is often time-varying. These characteristics make it difficult for traditional time series prediction models to model accurately. In addition, the modeling assumptions for stationary series further limit the scope of application of classical time series models. In the field of machine learning, problems such as overfitting, easy to fall into local optimal solutions, and insufficient generalization ability may be encountered. Although some models have relatively superior performance in generalization and are not easy to fall into local optimal solutions, their prediction ability may be limited when faced with input data with high-dimensional features. Therefore, it is usually necessary to reduce the dimension of the data before prediction. However, this dimensionality reduction process is highly dependent on feature engineering and has a high degree of dependence on prior knowledge, which limits the wide applicability of the model. For ensemble models, although they have indeed shown significant results in various prediction tasks, the existing ensemble strategies still have some shortcomings when dealing with time series prediction tasks. First, many ensemble methods are used to capture nonlinear relationships and complex patterns in data, but these methods often perform unsatisfactorily for the time dependency in time series data. Second, most current integration strategies are based on historical data to determine how to combine different base models. However, due to the dynamic characteristics of time series data, fixed base model combination strategies may no longer be applicable to newly emerging time periods. This means that when the distribution or pattern of data changes, the integration strategy may become ineffective.
[0028] Based on this, this paper proposes a time series prediction integration model. Figure 1 A structural schematic diagram 100 of an embodiment of a time series prediction integrated model according to the present disclosure is shown. The time series prediction integrated model 100 includes: a base model group 101, an expert network group 102, a gating network 103, and a prediction network 104.
[0029] Among them, the base model 1011 in the base model group 101 is used to perform time series prediction on the input time series data to obtain an initial prediction sequence. An expert network 1021 in the expert network group 102 is connected to a base model 1011 in the base model group 101, and the expert network 1021 is used to extract features and encode the initial prediction sequence output by the corresponding base model. The gating network 103 is used to generate corresponding weights for the expert network 1021 in the expert network group 102. The prediction network 104 generates a result prediction sequence based on the outputs of the expert network group 102 and the gating network 103.
[0030] In this embodiment, the base model 1011 is a time series prediction model that predicts past time series data to obtain future prediction data. The base model group 101 includes at least two base models 1011, such as Figure 1Base1~BaseN (N>2) are all base models in the base model group 101, among which Base1 is a base model, BaseN is also a base model, and each base model in Base1~BaseN is a mutually exclusive model. Since each base model 1011 will predict the time series data and obtain its own initial prediction sequence, the base model group 101 can output an initial prediction sequence group including at least two initial prediction sequences.
[0031] In this embodiment, the expert network group 102 includes at least two expert networks, each of which is used to extract and encode features of a corresponding base model. Figure 1 Expert1~ExpertN (N>2) are all expert networks in the expert network group, among which Expert1 is an expert network, ExpertN is also an expert network, and each expert network in Expert1~ExpertN is a mutually exclusive model.
[0032] In this embodiment, time series data refers to time series data. Time series data is a data column recorded in chronological order for the same unified indicator. Each data in the same data column must be of the same caliber and must be comparable. Time series data can be a number of periods or a number of time points. The purpose of time series analysis is to construct a time series model and perform out-of-sample prediction by finding the statistical characteristics and development regularity of the time series within the sample.
[0033] In this embodiment, the initial prediction sequence is a sequence of future development trends obtained after the model analyzes the time series data. For the time series data of the past M (M>1) days, the initial prediction sequence can be the time series data of the M+1th day, or M+1, M+2,...M+n (n>1).
[0034] In this embodiment, the result prediction sequence is the prediction value of the time series data obtained after processing the initial prediction sequence using the expert network group, the gated network and the prediction network. Compared with the initial prediction sequence, the result prediction sequence is the prediction value after optimizing the initial prediction sequence and with higher accuracy.
[0035] In this embodiment, when training the time series prediction integrated model, the initial prediction sequence and the result prediction sequence are both the predicted values of the labels in the samples, and the prediction results of the result prediction sequence are more accurate than those of the initial prediction sequence.
[0036] The time series prediction integrated model provided by the embodiment of the present disclosure includes: a base model group, each base model in the base model group is used to perform time series prediction on the input time series data to obtain an initial prediction sequence; an expert network group, an expert network in the expert network group is connected to a base model in the base model group, and each expert network is used to extract features and encode the initial prediction sequence output by the corresponding base model; a gating network is used to generate corresponding weights for each expert network in the expert network group; and a prediction network generates a result prediction sequence based on the output of the expert network group and the gating network. Thus, the initial prediction result of the base model is input into the expert network group, and the adapted base model is selected based on the screening mechanism of the gating network, which no longer relies on the structural features of a single base model, reduces the occurrence of overfitting caused by the one-sidedness of the model prediction, and improves the accuracy of the prediction.
[0037] Based on the non-stationary and high volatility of time series, in order to continue to deeply explore and screen features, Figure 2 A structural schematic diagram of another embodiment of the time series prediction integrated model according to the present disclosure is shown. The time series prediction integrated model 200 provided in this embodiment includes: a base model group 201, an expert network group 202, a gating network 203, a prediction network 204 and a reinforcement learning network 205.
[0038] The specific processing of the base model group 201, the expert network group 202, the gated network 203, and the prediction network 204 and the technical effects thereof can be referred to in detail. Figure 1 The relevant descriptions of the base model group 101, the expert network group 102, the gating network 103, and the prediction network 104 in the corresponding embodiment are not repeated here.
[0039] In this embodiment, the reinforcement learning network 205 is connected to the base model group 201 and the gating network 203 respectively, and the reinforcement learning network 205 controls the parameters of the gating network based on the output of the base model group.
[0040] like Figure 2 As shown, the expert network group 202 in the time series prediction integrated model 200 is used to further extract features and encode the input initial prediction sequence. The number of expert networks in the expert network group is consistent with the number of base models in the base model group; the gating network 203 in the time series prediction integrated model 200 is used to generate the weights of each expert network according to the input base model data, so as to integrate the base model results. The prediction network 204 in the time series prediction integrated model 200 is used to further fit the prediction label after the integrated model results. The signal flow of the entire time series prediction integrated model 200 is shown in the following formula:
[0041] x′ i =MLP(x i) i=1,2,3…N (1)
[0042] z=MLP(Concat(x1,x2,…,x N )) (2)
[0043]
[0044] y out =MLP(Concat(α1x′1,α2x′2,…,α N x′ N )) (4)
[0045] MLP(·) and Concat(·) in formula (1) and (2) represent the computation and connection layers of the multilayer perceptron, respectively. i represents the initial prediction sequence output by the i-th base model, x′ i represents the output after the nonlinear transformation of the i-th expert network. z is an N-dimensional vector, whose i-th element represents the score of the gating mechanism on the output of the i-th base model. The score is converted to α by the Softmax function. i , so that the sum of the weights is 1. Then the prediction value of each expert network is multiplied by the corresponding weight and passed to the prediction network, and finally the predicted value y of the label is output out , which is the result prediction sequence.
[0046] In this embodiment, considering the volatility and non-stationarity of the time series, this solution incorporates a reinforcement learning network 205 to dynamically adjust the parameters of the gating network according to the time series based on the expert network group 202, the gating network 203, and the prediction network 204. The state, action, and reward of the reinforcement learning network 205 are defined as follows: 1) The state is a time series composed of the output results of all base models for T (T>1) days in history; 2) The action is the parameter of the gating network output by the reinforcement learning network, and its output dimension is consistent with the dimension of the gating network parameter; 3) The reward is the MSE (Mean Squared Error) obtained based on the true label and the model prediction result.
[0047] In this embodiment, in the training phase, the trained base model group is used to first perform overall training on the expert network group 202, the gated network 203, and the prediction network 204. After the expert network group 202, the gated network 203, and the prediction network 204 converge, the parameters of the expert network group and the prediction network are fixed. Then, the reinforcement learning network is trained until the time series prediction integrated model converges.
[0048] The time series prediction integrated model provided in this embodiment also includes a reinforcement learning network that controls the parameters of the gating network. Taking into account the non-stationary and high volatility characteristics of the time series, deep reinforcement learning is further used to learn the signals of the time series in different time periods, and the integrated weights of the base model are output adaptively. Therefore, the current method continues to deeply mine and screen features on the basis of solving the long-term dependency problem in the time series, thereby significantly improving the accuracy of the integrated model prediction.
[0049] In some optional implementations of the present disclosure, the data input to each base model in the above-mentioned base model group is the same, and the data output by each base model is the same in dimension.
[0050] In this optional implementation, the input data is time series data, which is a sequence that changes over time. Specifically, the input data is x∈R B×L×D , where B is the batch size of the input data; L is the time window for inputting a single sample (i.e., time series data), that is, using the data from the past L days; and D is the feature dimension of each time series sample.
[0051] In this optional implementation, it is assumed that the trained base model group is M = {model1, model2…, model N}, where the base model model i ∈M. Here the model i It can be any model, it is worth noting that each base model model i The input is the same data, and the output data dimension is also the same. For example, each model i The output data is p i ∈R B ×LD .
[0052] The time series prediction integrated model provided by this optional implementation method sets each base model in the base model group to have the same input data and the same output data dimension, so that different base models can analyze the same data and give outputs with the same data structure, thereby improving the uniformity and reliability of data analysis of the time series prediction integrated model.
[0053] In some optional implementations of the present disclosure, each of the above-mentioned base models is a pre-trained deep learning or machine learning time series prediction model.
[0054] In this optional implementation, the base model is used to output the predicted value of the base model for future time series data, that is, the initial prediction sequence, which can be used for training the base model group. The base model can be any type of time series prediction model, such as a long short-term memory neural network based on deep learning or a regression moving average model based on traditional machine learning.
[0055] In this optional implementation, the Long Short Term Memory network (LSTM) has been widely used and studied. LSTM is a variant of RNN that uses a gating mechanism to control the inflow and loss of information and can use more distant historical memories. LSTM introduces three gates, namely the input gate, forget gate, and output gate, as well as memory cells with the same shape as the hidden state, to record additional information.
[0056] In this optional implementation, in the ARIMA model, AR stands for autoregression, with an order of p; MA stands for moving average, with a term of q, and d stands for the number of differences performed to make the time series stationary. The core idea of this model is to transform a non-stationary time series into a stationary one, and then establish a model based on the lag value and the random error term.
[0057] Optionally, the base model may also be RCLSTM.
[0058] The time series prediction integration model provided by this optional implementation, the base model in the model can be a time series prediction model of deep learning or machine learning, which provides a diverse implementation method for obtaining the base model group and improves the reliability of the prediction of the time series prediction integration model.
[0059] In view of the time series prediction integrated model provided in the above embodiment, the present disclosure also proposes a time series prediction integrated model training method. Figure 3 A process 300 of an embodiment of a time series prediction integrated model training method according to the present disclosure is shown. The time series prediction integrated model training method comprises the following steps:
[0060] Step 301: Obtain a sample set.
[0061] In this embodiment, the sample set includes at least one sample, and the sample includes: time series data in a sample time period.
[0062] In this embodiment, the time series data x∈R in the sample time period B×L×D , where B is the batch size of the input data; L is the sample time period, which is also the time window for inputting a single sample, that is, using the data of the past L days to predict the future label; D is the feature dimension of each time series sample. Assume that the dimension of the label information of the time data is Label∈R B×LD , where LD = {l1,l2…l n}, l n Represents the predicted (future) n Day label.
[0063] In this embodiment, the execution subject of the time series prediction integrated model training method can obtain the image to be detected in a variety of ways. For example, the execution subject can obtain the sample set stored in the database server through a wired connection or a wireless connection. For another example, the execution subject can also receive the sample set collected in real time by the terminal or other devices.
[0064] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of the sample sets involved are carried out after authorization and comply with relevant laws and regulations.
[0065] Step 302, obtaining an initial time series prediction integrated model.
[0066] In this embodiment, the time series prediction integration model adopts the time series prediction integration model shown in the above embodiment.
[0067] In this embodiment, the time series prediction integrated model is a model obtained by model integration, and there are at least two models in the time series prediction integrated model, and each model has different functions. The initial time series prediction integrated model is an integrated model that combines all models and has not been trained. By training the initial time series prediction integrated model, a trained time series prediction integrated model can be obtained.
[0068] In this embodiment, the execution subject of the time series prediction integrated model training method can obtain the initial time series prediction integrated model in a variety of ways. For example, the execution subject can obtain the initial time series prediction integrated model stored in the database server through a wired connection or a wireless connection. For another example, the execution subject can also receive the initial time series prediction integrated model sent by a terminal or other device in real time.
[0069] Step 303: Based on the sample set, the expert network group, the gating network and the prediction network in the time series prediction integrated model are trained.
[0070] In this embodiment, the above step 303 includes: selecting samples from the sample set, inputting the selected samples into the time series prediction integrated model, and obtaining the result prediction sequence output by the time series prediction integrated model. Based on the result prediction sequence, the loss function, and the labels in the selected samples, the loss of the time series prediction integrated model is calculated; based on the loss of the time series prediction integrated model, the parameters of the expert network group, the gated network, and the prediction network are adjusted, and the expert network group, the gated network, and the prediction network are detected to see whether they meet the first training completion condition. When the expert network group, the gated network, and the prediction network do not meet the first training completion condition, continue to select samples from the sample set, calculate the loss, adjust the parameters of the expert network group, the gated network, and the prediction network, and detect whether the expert network group, the gated network, and the prediction network meet the first training completion condition.
[0071] Among them, the selection method and the number of samples are not limited in the present disclosure. For example, at least one sample can be randomly selected, or a sample with good continuity can be selected from the sample set. The above adjustment of the parameters of the expert network group, the gating network and the prediction network can be to adjust any one or more parameters of the expert network group, the gating network and the prediction network. The above selection of samples from the sample set, calculation of losses, and adjustment of the parameters of the expert network group, the gating network and the prediction network is to complete an iterative training of the time series prediction integrated model.
[0072] Step 304, in response to the expert network group, the gating network, and the prediction network satisfying the first training completion condition, a trained time series prediction integrated model is obtained.
[0073] In this embodiment, the base model group in the time series prediction integrated model is a pre-trained model group, and a trained time series prediction integrated model can be obtained by training the expert network group, the gating network and the prediction network. Among them, each base model in the base model group can be obtained after pre-training in other systems.
[0074] In this embodiment, the first training completion condition is used to define whether the expert network group, the gated network and the prediction network have been trained. When the expert network group, the gated network and the prediction network meet the first training completion condition, it is determined that the expert network group, the gated network and the prediction network have been trained. Specifically, the first training completion condition includes at least one of the following: the loss value of the time series prediction integrated model corresponding to the expert network group, the gated network and the prediction network is less than the preset model loss threshold. For example, the model loss threshold is 0.05.
[0075] Optionally, in response to the expert network group, the gating network and the prediction network not satisfying the first training completion condition, any one or more parameters of the expert network group, the gating network and the prediction network are adjusted to make the model loss value converge.
[0076] The time series prediction integrated model training method provided by the embodiment of the present disclosure first obtains a sample set, the sample set includes at least one sample, and the sample includes: time series data in the sample time period; secondly, obtains an initial time series prediction integrated model; thirdly, based on the sample set, trains the expert network group, the gating network, and the prediction network in the time series prediction integrated model; finally, in response to the expert network group, the gating network, and the prediction network satisfying the first training completion condition, obtains a trained time series prediction integrated model. Thus, by training the expert network group, the gating network, and the prediction network, a trained time series prediction integrated model is obtained, which provides a reliable means of implementation for the training of the time series prediction integrated model and improves the accuracy and reliability of the recognition of the time series prediction integrated model.
[0077] In some embodiments of the present disclosure, the above-mentioned time series prediction integrated model also includes: a reinforcement learning network, and the above-mentioned time series prediction integrated model training method includes: obtaining a sample set, the sample set includes at least one sample, and the sample includes: time series data in the sample time period; obtaining an initial time series prediction integrated model, the time series prediction integrated model adopts the time series prediction integrated model of the above-mentioned embodiment; based on the sample set, training the expert network group, the gating network and the prediction network in the time series prediction integrated model; in response to the expert network group, the gating network and the prediction network satisfying the first training completion condition, fixing the parameters of the expert network group and the prediction network; based on the sample set, training the reinforcement learning network; in response to the reinforcement learning network satisfying the second training completion condition, obtaining a trained time series prediction integrated model.
[0078] In this embodiment, the base model group is a pre-trained model and can be directly used as a mature predictor. During the training process of the expert network group, the gated network and the prediction network, the parameters of the base model group remain unchanged. The loss values of the expert network group, the gated network and the prediction network are determined by calculating the loss functions of the expert network group, the gated network and the prediction network. Specifically, the loss functions of the expert network group, the gated network and the prediction network can adopt the mean square error (MSE), which is the most commonly used loss function in the field of machine learning and is used to measure the error degree between the output result (final prediction value) and the expected true value. It can minimize the loss of the model by reducing the difference between the predicted value and the true value during training, thereby improving the performance of the model.
[0079] In this embodiment, training the reinforcement learning network means training a DQN (Deep Q-Learning) for outputting the parameters of the gating network using the reinforcement learning method, on the premise that the base model group, the expert network group, and the prediction network have been trained and frozen.
[0080] The samples in the sample set are time series data of the business. When the business is the stock business, the loss function of the reinforcement learning network is reflected as the error between the predicted value and the true value in the future time of the stock prediction. The present disclosure adds a reinforcement learning network to the time series prediction integrated model and uses the Q-learning method to obtain the parameters of the gating network, thereby further adopting deep learning of time series data in different time periods and adaptively outputting the parameters of the gating network, thereby improving the prediction accuracy of the time series prediction integrated model.
[0081] The time series prediction integrated model training method provided in this embodiment further includes: a reinforcement learning network, and after the expert network group, the gating network and the prediction network meet the first training condition, the parameters of the expert network group and the prediction network are fixed; based on the sample set, the reinforcement learning network is trained; in response to the reinforcement learning network meeting the second training completion condition, a trained time series prediction integrated model is obtained; thereby, when the reinforcement learning network is trained, the parameters of the reinforcement learning network and the gating network are adjusted, so that the secondary training of the gating network is realized, and the generalization ability of the time series prediction integrated model is improved.
[0082] In some optional implementations of the present disclosure, the above-mentioned training of the reinforcement learning network based on the sample set includes: selecting a sample from the sample set; inputting the sample into the base model group of the time series prediction integrated model to obtain the action value function value output by the reinforcement learning network; based on the action value function value, obtaining the parameters of the gating network; based on the parameters of the gating network, adjusting the gating network to obtain the result prediction sequence output by the prediction network; based on the result prediction sequence and the sample, calculating the reward of the reinforcement learning network; based on the action value function value and the reward, calculating the loss of the reinforcement learning network; in response to the loss of the reinforcement learning network being greater than the reinforcement loss threshold, adjusting the parameters of the reinforcement learning network.
[0083] In this optional implementation, the DQN network can be used as a model of the reinforcement learning network. DQN is a neural network based on Q-learning, and its principle is to learn a strategy so that the action selected in each state can maximize the future cumulative reward. The value of each state-action pair is evaluated by approximating the Q function (which is also the reinforcement loss function of the reinforcement learning network), thereby guiding the agent to select the optimal action.
[0084] In this article, the input of DQN is a time series consisting of the output results of the model's history T days, and the output is the parameters of the gating network. The specific training process is as follows:
[0085] The initial prediction result of the base model group is used as the initial state, and the initial state is input into the DQN network;
[0086] The DQN network outputs the Q values (action-value function values) of all actions and selects an action (i.e., gating network parameters) based on a corresponding strategy (such as the epsilon-greedy strategy).
[0087] From the true value y true (label in the sample) and the predicted value y after passing through the prediction network out (Result prediction sequence) Calculate the mean square error and set it as the reward. Loss can be calculated based on the current Q value and the target Q value, where: The current Q value is the Q value of the action performed in the current state calculated by the Q network. The target Q value is the maximum Q value of the next state s' calculated by the target Q network, plus the current reward, as shown in formula (5);
[0088]
[0089] In formula (1), r is the current reward, Calculate the maximum Q value of the next state s' for the target Q network.
[0090] The loss of the reinforcement learning network is calculated according to the current Q value and the target Q value. Specifically, the loss of the reinforcement learning network can be calculated using formula (6). The DQN network updates the parameters according to the loss of the reinforcement learning network until convergence.
[0091] Loss=(Q 目标 -Q(s,a) 2 (6)
[0092] The method for training a reinforcement learning network provided by this optional implementation selects samples from a sample set, inputs the samples into the base model group of a time series prediction integrated model, obtains the action value function value output by the reinforcement learning network, obtains the parameters of the gating network based on the action value function value, adjusts the gating network, obtains the result prediction sequence output by the prediction network, determines the reward of the reinforcement learning network based on the result prediction sequence and the samples, and calculates the reinforcement loss of the reinforcement learning network based on the reward, thereby providing a reliable implementation method for the training of the reinforcement learning network and ensuring the reliability of the training of the time series prediction integrated model.
[0093] In some optional implementations of the present disclosure, the second training completion condition includes at least one of the following: the reinforcement loss of the reinforcement learning network is less than or equal to the reinforcement loss threshold; the number of training iterations of the reinforcement learning network reaches a second predetermined iteration threshold.
[0094] In this optional implementation, the reinforcement loss is the value calculated after inputting the output value of the reinforcement learning network and the corresponding value of the sample into the reinforcement loss function, wherein the reinforcement loss function is a loss function set for the reinforcement learning network, and the reinforcement loss threshold is a threshold set for training the reinforcement learning network. Setting the second training completion condition to the reinforcement loss being less than the reinforcement loss threshold can make the reinforcement learning network converge quickly.
[0095] The second training completion condition provided by this optional implementation method provides an effective restriction condition for the rapid convergence of the reinforcement learning network and improves the reliability of the training of the time series prediction integrated model.
[0096] In some optional implementations of the present disclosure, the above-mentioned training of the expert network group, the gating network and the prediction network in the time series prediction integrated model includes: selecting samples from a sample set; inputting the samples into the time series prediction integrated model to obtain a result prediction sequence output by the time series prediction integrated model; based on the result prediction sequence and the samples, calculating the optimization loss of the expert network group, the gating network and the prediction network in the time series prediction integrated model; in response to the optimization loss being greater than the optimization loss threshold, adjusting any one or more parameters of the expert network group, the gating network and the prediction network.
[0097] In this optional implementation, the optimization loss is a value calculated by the optimization loss function set for the expert network group, the gating network, and the prediction network. Specifically, the optimization loss function can use the cross entropy loss function, which can measure the difference between two different probability distributions in the same random variable. In machine learning, it is expressed as the difference between the true probability distribution and the predicted probability distribution. The smaller the value of the cross entropy loss function, the better the prediction effect of the time series prediction ensemble model.
[0098] In this optional implementation, the result prediction sequence and the labels in the sample are substituted into the optimization loss function to obtain the optimization loss. The optimization loss can be used to detect whether the expert network group, the gating network and the prediction network meet the first training completion condition. After the expert network group, the gating network and the prediction network meet the first training completion condition, the trained expert network group, the gating network and the prediction network are obtained.
[0099] In this embodiment, the first training completion condition includes: the total loss value of the time series prediction integrated model calculated by the optimization loss is less than the first loss value threshold. The first loss threshold can be determined based on specific training requirements. For example, the first loss threshold is 0.01, and the first loss threshold can be equal to the fixed value minus the optimization loss.
[0100] Optionally, in this embodiment, in response to the expert network group, the gating network and the prediction network not satisfying the first training completion condition, the relevant parameters in the expert network group, the gating network and the prediction network are adjusted so that the optimization losses of the expert network group, the gating network and the prediction network converge, and the optimization loss continues to be calculated based on the adjusted expert network group, the gating network and the prediction network until the first training completion condition is satisfied.
[0101] The method for training the expert network group, the gating network and the prediction network provided by this optional implementation is based on the optimization loss of the expert network group, the gating network and the prediction network, and the expert network group, the gating network and the prediction network are trained to improve the training efficiency and accuracy of the model.
[0102] In some optional implementations of the present disclosure, the first training completion condition includes at least one of the following: the optimization loss of the expert network group, the gating network and the prediction network is less than or equal to the optimization loss threshold; the number of training iterations of the expert network group, the gating network and the prediction network reaches a first predetermined iteration threshold.
[0103] In this optional implementation, the optimization loss is the value calculated after inputting the output results of the prediction network and the sample labels into the optimization loss function, wherein the optimization loss function is a loss function set for the expert network group, the gated network and the prediction network, and the optimization loss threshold is a threshold set for training the expert network group, the gated network and the prediction network. Setting the first training completion condition to the optimization loss being less than the optimization loss threshold can enable the expert network group, the gated network and the prediction network to converge quickly.
[0104] The first training completion condition provided by this optional implementation method provides effective restriction conditions for the rapid convergence of the expert network group, the gating network and the prediction network, and improves the reliability of the training of the time series prediction integrated model.
[0105] In order to improve model performance, speed up training, eliminate feature scale differences, avoid overfitting and bias, and ensure feature balance, the data set in the sample can be normalized to obtain the final sample. In some optional implementations of the present disclosure, the above-mentioned acquisition of the sample set includes: obtaining a time series data set of products in a first sample time period; calculating the mean and variance of the product based on the time series data in a second sample time period of the product, the second sample time period including the first sample time period; based on the mean and variance, normalizing the time series data set to obtain a sample set.
[0106] In this optional implementation, the time series data set includes at least one time series data, which refers to time series data. Time series data is a data column recorded in chronological order for the same unified indicator. The time series data set is the unprocessed business data obtained directly from the business data. Considering the characteristics of the time series data set such as the existence of outliers and the non-fixed maximum and minimum values, Z-score can be used for normalization processing to obtain a sample set.
[0107] In this optional implementation, Z-score normalization can also be called standardization, and the processed data is distributed with a mean of 0 and a standard deviation of 1. Specifically, the normalized sample x is calculated using the method shown in formula (8): norm :
[0108]
[0109] Wherein, in formula (8), μ is the mean, σ is the standard deviation, and both the mean and the standard deviation are calculated by the time series data in the second sample time period, for example, the second sample time period is the past year. x is the time series data in the time series data set in the first sample time, for example, the first sample time period is the past seven days.
[0110] The method for obtaining a sample set provided by this optional implementation calculates the mean and variance through the time series data in the second sample time period, and normalizes the time series data in the first sample time period, thereby improving the reliability and accuracy of the sample set and facilitating the time series prediction integrated model to identify the sample set.
[0111] Furthermore, based on the time series prediction integrated model training method provided in the above embodiment, the present disclosure also provides an embodiment of a time series prediction method. The time series prediction method of the present disclosure combines artificial intelligence fields such as machine learning and deep learning.
[0112] See also Figure 4 , shows a process 400 according to an embodiment of a time series prediction method disclosed in the present invention. The time series prediction method provided in this embodiment includes the following steps:
[0113] Step 401, obtaining historical time series data and a time series prediction integrated model.
[0114] In this embodiment, the historical time series data is the time series data to be predicted that is collected in the historical time period, wherein the time series data can be data in a variety of scenarios, for example, power consumption data, meteorological weather data, and the time series prediction is performed on the historical time series data through the time series prediction integration model to obtain a result prediction sequence of the historical time series data.
[0115] In this embodiment, the execution subject of the time series prediction method can obtain historical time series data in a variety of ways. For example, the execution subject can obtain the historical time series data stored in the database server through a wired connection or a wireless connection. For another example, the execution subject can also receive the historical time series data collected by the terminal or other devices in real time.
[0116] Step 402, input the historical time series data into the time series prediction integrated model to obtain the result prediction sequence output by the time series prediction integrated model.
[0117] In this embodiment, the execution entity may input the historical time series data obtained from step 401 into the time series prediction integration model to obtain a result prediction sequence output by the time series prediction integration model.
[0118] In this embodiment, the time series prediction integrated model can be adopted as described above Figure 1 The training completed time series prediction integrated model is obtained by training using the method described in the embodiment. For the specific training process, please refer to Figure 1 The relevant description of the illustrated embodiment will not be repeated here.
[0119] In this embodiment, the result prediction sequence is the result of predicting the future development of historical time series data, and the result prediction sequence includes: at least one predicted time series data, and the at least one predicted time series data can be the time series data of the time period after the historical time period.
[0120] The time series prediction method provided by the embodiment of the present disclosure obtains historical time series data and a time series prediction integrated model, inputs the historical time series data into the time series prediction integrated model, and obtains a result prediction sequence output by the time series prediction integrated model. Thus, the time series prediction integrated model is used to generate the detection results, which improves the reliability and accuracy of the time series prediction.
[0121] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a time series prediction integrated model training device, which is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0122] like Figure 5As shown, the time series prediction integrated model training device 500 provided in this embodiment includes: a sample acquisition unit 501, a model acquisition unit 502, an expert training unit 503, and an obtaining unit 504. Among them, the above-mentioned sample acquisition unit 501 can be configured to obtain a sample set, and the sample set includes at least one sample, and the sample includes: time series data in a sample time period. The above-mentioned model acquisition unit 502 can be configured to obtain an initial time series prediction integrated model, and the time series prediction integrated model adopts the time series prediction integrated model of the above-mentioned embodiment. The above-mentioned expert training unit 503 can be configured to train the expert network group, the gating network and the prediction network in the time series prediction integrated model based on the sample set. The above-mentioned obtaining unit 504 can be configured to obtain a trained time series prediction integrated model in response to the expert network group, the gating network and the prediction network satisfying the first training completion condition.
[0123] In this embodiment, in the time series prediction integrated model training device 500, the specific processing of the sample acquisition unit 501, the model acquisition unit 502, the expert training unit 503, and the obtaining unit 504 and the technical effects thereof can be referred to respectively. Figure 1 The relevant descriptions of step 101, step 102, step 103, and step 104 in the corresponding embodiment are not repeated here.
[0124] In some optional implementations of the present embodiment, the above-mentioned time series prediction integrated model also includes: a reinforcement learning network, and the above-mentioned device 500 also includes: a reinforcement training unit (not shown in the figure), and the above-mentioned reinforcement training unit can be configured to fix the parameters of the expert network group and the prediction network; based on the sample set, the reinforcement learning network is trained; in response to the reinforcement learning network satisfying the second training completion condition, a trained time series prediction integrated model is obtained.
[0125] In some optional implementations of the present embodiment, the reinforcement training unit is further configured to: select a sample from a sample set; input the sample into the base model group of the time series prediction integrated model to obtain the action value function value output by the reinforcement learning network; obtain the parameters of the gated network based on the action value function value; adjust the gated network based on the parameters of the gated network to obtain a result prediction sequence output by the prediction network; calculate the reward of the reinforcement learning network based on the result prediction sequence and the sample; calculate the loss of the reinforcement learning network based on the action value function value and the reward; and adjust the parameters of the reinforcement learning network in response to the loss of the reinforcement learning network being greater than the reinforcement loss threshold.
[0126] In some optional implementations of this embodiment, the above-mentioned second training completion condition includes at least one of the following: the reinforcement loss of the reinforcement learning network is less than or equal to the reinforcement loss threshold; the number of training iterations of the reinforcement learning network reaches a second predetermined iteration threshold.
[0127] In some optional implementations of the present embodiment, the above-mentioned expert training unit 503 is configured to: select samples from a sample set; input the samples into a time series prediction integrated model to obtain a result prediction sequence output by the time series prediction integrated model; based on the result prediction sequence and the samples, calculate the optimization loss of the expert network group, the gating network and the prediction network in the time series prediction integrated model; in response to the optimization loss being greater than the optimization loss threshold, adjust any one or more parameters of the expert network group, the gating network and the prediction network.
[0128] In some optional implementations of this embodiment, the above-mentioned first training completion condition includes at least one of the following: the optimization loss of the expert network group, the gating network and the prediction network is less than or equal to the optimization loss threshold; the number of training iterations of the expert network group, the gating network and the prediction network reaches a first predetermined iteration threshold.
[0129] In some optional implementations of the present embodiment, the sample acquisition unit 501 is configured to: acquire a time series data set of a product in a first sample time period; calculate the mean and variance of the product based on the time series data in a second sample time period of the product, where the second sample time period includes the first sample time period; and normalize the time series data set based on the mean and variance to obtain a sample set.
[0130] The time series prediction integrated model training device provided by the embodiment of the present disclosure, first, the sample acquisition unit 501 acquires a sample set, the sample set includes at least one sample, and the sample includes: time series data in the sample time period; secondly, the model acquisition unit 502 acquires the initial time series prediction integrated model; thirdly, the expert training unit 503 trains the expert network group, the gating network and the prediction network in the time series prediction integrated model based on the sample set; finally, the obtaining unit 504 obtains the trained time series prediction integrated model in response to the expert network group, the gating network and the prediction network satisfying the first training completion condition. Therefore, by training the expert network group, the gating network and the prediction network, the trained time series prediction integrated model is obtained, which provides a reliable implementation means for the training of the time series prediction integrated model and improves the accuracy and reliability of the recognition of the time series prediction integrated model.
[0131] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a time series prediction device, which is similar to Figure 4 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0132] like Figure 6As shown, the time series prediction device 600 provided in this embodiment includes: an information acquisition unit 601 and an input unit 602. The acquisition unit can be configured to acquire historical time series data and a time series prediction integrated model, and the time series prediction integrated model is a trained time series prediction integrated model obtained by using the time series prediction integrated model training method of the above embodiment; the input unit 602 can be configured to input historical time series data into the time series prediction integrated model to obtain a result prediction sequence output by the time series prediction integrated model.
[0133] In this embodiment, in the time series prediction device 600, the specific processing of the information acquisition unit 601 and the input unit 602 and the technical effects thereof can be referred to in Figure 4 The relevant descriptions of step 401 and step 402 in the corresponding embodiment are not repeated here.
[0134] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0135] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0136] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0137] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0138] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0139] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a time series prediction integrated model training method or a time series prediction method. For example, in some embodiments, the time series prediction integrated model training method or the time series prediction method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the time series prediction integrated model training method or the time series prediction method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the time series prediction integrated model training method or the time series prediction method in any other appropriate manner (for example, by means of firmware).
[0140] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0141] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable time series prediction integrated model training device or a time series prediction device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0142] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0144] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0145] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0146] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0147] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A time series prediction integrated model, the time series prediction integrated model comprising: A base model group, wherein the base models in the base model group are used to perform time series prediction on the input time series data to obtain an initial prediction sequence; An expert network group, wherein an expert network in the expert network group is connected to a base model in the base model group, and the expert network is used to extract features and encode an initial prediction sequence output by the corresponding base model; A gating network, used for generating corresponding weights for the expert networks in the expert network group; The prediction network generates a result prediction sequence based on the outputs of the expert network group and the gating network.
2. The model according to claim 1, further comprising: The reinforcement learning network is connected to the base model group and the gating network respectively, and the reinforcement learning network controls the parameters of the gating network based on the output of the base model group.
3. The model according to claim 1, wherein: The base models in the base model group have the same input data and the same output data dimensions.
4. A model according to any one of claims 1 to 3, wherein: Each base model is a pre-trained deep learning or machine learning time series prediction model.
5. A time series prediction integrated model training method, the method comprising: Acquire a sample set, the sample set including at least one sample, the sample including: time series data in a sample time period; Acquire an initial time series prediction integrated model, wherein the time series prediction integrated model adopts the time series prediction integrated model according to any one of claims 1 to 4; Based on the sample set, training the expert network group, the gating network and the prediction network in the time series prediction integrated model; In response to the expert network group, the gating network and the prediction network satisfying a first training completion condition, a trained time series prediction integrated model is obtained.
6. According to the method of claim 5, the time series prediction integrated model further comprises: Reinforce the learning network, before obtaining the trained time series prediction integrated model, the method further includes: Fixing parameters of the expert network group and the prediction network; Based on the sample set, training the reinforcement learning network; In response to the reinforcement learning network satisfying the second training completion condition, a trained time series prediction integrated model is obtained.
7. The method according to claim 6, wherein: The training of the reinforcement learning network based on the sample set includes: Selecting a sample from the sample set; Inputting the sample into the base model group of the time series prediction integrated model to obtain the action value function value output by the reinforcement learning network; Based on the action-value function value, obtaining the parameters of the gating network; Based on the parameters of the gating network, adjusting the gating network to obtain a result prediction sequence output by the prediction network; Calculating a reward of the reinforcement learning network based on the result prediction sequence and the sample; Calculating a loss of the reinforcement learning network based on the action-value function value and the reward; In response to the loss of the reinforcement learning network being greater than a reinforcement loss threshold, adjusting parameters of the reinforcement learning network.
8. The method according to claim 6, wherein: The second training completion condition includes at least one of the following: the reinforcement loss of the reinforcement learning network is less than or equal to the reinforcement loss threshold; the number of training iterations of the reinforcement learning network reaches a second predetermined iteration threshold.
9. The method according to claim 5, wherein: The training of the expert network group, the gating network and the prediction network in the time series prediction integrated model includes: Selecting a sample from the sample set; Inputting the sample into the time series prediction integrated model to obtain a result prediction sequence output by the time series prediction integrated model; Based on the result prediction sequence and the sample, calculating the optimization loss of the expert network group, the gating network and the prediction network in the time series prediction integrated model; In response to the optimization loss being greater than an optimization loss threshold, adjusting parameters of any one or more of the expert network group, the gating network, and the prediction network.
10. The method according to any one of claims 5 to 9, wherein: The first training completion condition includes at least one of the following: the optimization loss of the expert network group, the gating network and the prediction network is less than or equal to an optimization loss threshold; the number of training iterations of the expert network group, the gating network and the prediction network reaches a first predetermined iteration threshold.
11. The method according to any one of claims 5 to 9, wherein: The obtaining of the sample set comprises: Obtain a time series data set of products in a first sample time period; Calculating the mean and variance of the product based on time series data in a second sample time period of the product, where the second sample time period includes the first sample time period; Based on the mean and the variance, the time series data set is normalized to obtain a sample set.
12. A time series prediction method, the method comprising: Acquire historical time series data and a time series prediction integrated model, wherein the time series prediction integrated model is a trained time series prediction integrated model obtained by using the time series prediction integrated model training method described in claims 5-11; The historical time series data is input into the time series prediction integrated model to obtain a result prediction sequence output by the time series prediction integrated model.
13. A time series prediction integrated model training device, the device comprising: A sample acquisition unit is configured to acquire a sample set, wherein the sample set includes at least one sample, and the sample includes: time series data in a sample time period; A model acquisition unit, configured to acquire an initial time series prediction integrated model, wherein the time series prediction integrated model adopts the time series prediction integrated model according to any one of claims 1 to 4; An expert training unit is configured to train the expert network group, the gating network and the prediction network in the time series prediction integrated model based on the sample set; The obtaining unit is configured to obtain a trained time series prediction integrated model in response to the expert network group, the gating network and the prediction network satisfying a first training completion condition.
14. A time series prediction device, the device comprising: An information acquisition unit, configured to acquire historical time series data and a time series prediction integrated model, wherein the time series prediction integrated model is a trained time series prediction integrated model obtained by using the time series prediction integrated model training method according to claim 13; The input unit is configured to input the historical time series data into the time series prediction integrated model to obtain a result prediction sequence output by the time series prediction integrated model.
15. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 5 to 12.
16. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the method according to any one of claims 5 to 12.
17. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 5 to 12.
Citation Information
Patent Citations
Electric energy prediction method and system based on hybrid expert system and time sequence
CN116742618A
