Learning with Moment Estimation Using Different Time Constants

Abstract

A technique for training a model includes obtaining training examples for a model having model parameters stored on one or more computer-readable storage media operatively coupled to a hardware processor. The training examples include results and features for explaining the results. Gradients with respect to the model parameters of the model are calculated using the training examples. Two estimates of moments of the gradients are calculated using two different time constants for the same type of moment. The hardware processor is used to update the model parameters of the model using the two estimates of moments using the two different time constants, reducing an error between calculating the at least two estimates of the moments of the gradient.

Description

[Technical field]

[0001] The present disclosure relates generally to machine learning, and more particularly to computer-implemented methods, computer systems, and computer program products for training models based on moment estimation. [Background technology]

[0002] Stochastic Gradient Descent (SGD) is widely used in the field of machine learning, especially online learning. If the learning rate is set appropriately, the learning process is stable. However, the learning process often becomes unstable, especially when trying to learn quickly with a high learning rate. Finding a sufficient learning rate is generally difficult for large-scale tasks. Even when a high learning rate is used, stable performance is required. There are several application areas where obtaining fast convergence is more important than accuracy, and thus fast learning is required. As can be understood, such fast learning is suitable when it is necessary to dynamically adapt to new patterns of input.

[0003] To alleviate such difficulties, various techniques for adapting the learning rate have been developed, such as algorithms for first-order gradient-based optimization of stochastic objective functions based on adaptive estimates of lower-order moments. The method computes individual adaptive learning rates for the various parameters from estimates of the first and second moments of the stochastic gradient.

[0004] However, even if such methods can reduce the training time due to faster convergence characteristics, the learning process may still diverge when the base learning rate becomes higher. Therefore, all of the prior art techniques are insufficient in terms of achieving both stable learning and fast learning. Therefore, there is a need in the art to address the aforementioned problems. Summary of the Invention

[0005] Viewed from a first aspect, the present invention provides a computer-implemented method for training a model, the computer-implemented method comprising: obtaining training examples for a model having model parameters stored on one or more computer-readable storage media operatively coupled to a hardware processor, the training examples including outcomes and features for explaining the outcomes; calculating gradients with respect to the model parameters of the model using the training examples; calculating at least two estimates of moments of the gradients using at least two different time constants for a same type of moment, using the hardware processor to update the model parameters of the model using the at least two estimates of the moments with the at least two different time constants to reduce an error between calculating the at least two estimates of the moments of the gradients.

[0006] Viewed from another aspect, the present invention provides a computer system for training a model by executing program instructions, the computer system comprising: one or more computer readable storage media for storing the program instructions and training examples for a model having model parameters; and a processing circuit in communication with the memory for executing the program instructions, the processing circuit being configured to: obtain training examples for the model having model parameters from the one or more computer readable storage media, the training examples including results and features for explaining the results; calculate gradients with respect to the model parameters of the model using the training examples; calculate at least two estimates of moments using at least two different time constants for the same type of moment using the gradients; and update the model parameters of the model using the at least two estimates of the moments using the at least two different time constants to reduce an error between calculating the at least two estimates of the moments of the gradients.

[0007] Viewed from another aspect, the present invention provides a computer program product for training a model, the computer program product including a computer readable storage medium having program instructions and training examples for a model having model parameters embodied therein, the program instructions being executable by a computer to cause the computer to perform a computer implemented method including: obtaining training examples for the model having model parameters from the computer readable storage medium, the training examples including results and features for explaining the results; calculating gradients with respect to the model parameters of the model using the training examples; calculating at least two estimates of moments of the gradients using at least two different time constants for the same type of moment; and updating the model parameters of the model using the at least two estimates of the moments with the at least two different time constants to reduce an error between calculating the at least two estimates of the moments of the gradients.

[0008] According to one embodiment of the present invention, a computer-implemented method for training a model includes obtaining training examples for a model having model parameters stored on one or more computer-readable storage media operatively coupled to a hardware processor, the training examples including outcomes and features for explaining the outcomes; calculating gradients with respect to the model parameters of the model using the training examples; calculating at least two estimates of moments of the gradients using at least two different time constants for a same type of moment using the gradients; and updating the model parameters of the model using the at least two estimates of moments with the at least two different time constants using the hardware processor to reduce an error between calculating the at least two estimates of moments of the gradients.

[0009] Viewed from another aspect, the present invention provides a computer program product for training a model, the computer program product comprising a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit to perform a method for performing steps of the present invention.

[0010] Viewed from another aspect, the present invention provides a computer program stored on a computer readable medium and loadable into an internal memory of a digital computer, said computer program comprising software code portions for performing the steps of the present invention when said program is run on a computer.

[0011] In the computer-implemented method according to the embodiment of the present invention, the risk of making large errors in the gradient estimation due to the influence of outdated model parameters is expected to be reduced. Both moment estimations with different time constants are taken into account in the model parameter update. This allows for fast learning without reducing the stability of the learning process.

[0012] In one embodiment, each of the model parameters may be updated in a conservative manner by an amount determined individually by each component of at least two estimates of the gradient moments. Thus, aggressive updating of the model parameters in the case of a mismatch between at least two estimates of the gradient moments is suppressed. Thus, it is expected that the risk of mistakes in parameter updates due to aggressive updating is further reduced.

[0013] In an embodiment, in response to the at least two estimates of the moments having a mismatch in a component corresponding to the first model parameter, a first model parameter of the model may be updated by zero or a small amount. In response to the at least two estimates of the moments having a match in a component corresponding to the first model parameter, the first model parameter may be updated according to a value generated by combining the respective components of the at least two estimates of the moments corresponding to the first model parameter. Thereby, the update rule appropriately updates the model parameters having a match between the at least two estimates of the moments and does not change the model parameters having uncertainty. Thus, it is expected that the risk of errors due to parameter updates for uncertain directions is reduced.

[0014] In one embodiment, a first model parameter of the model may be updated according to the maximum or average value of the components of at least two estimates of the moment corresponding to the first model parameter, thereby adjusting the sensitivity of the parameter update by taking into account the at least two estimates of the moment.

[0015] In an embodiment, the moments may include different types: first moment of the gradient and second moment of the gradient. The first moment represents the average of the gradient. The second moment scales the individual learning rates for the model parameters of the model.

[0016] In one embodiment, the moments may include a first moment of gradient and a second moment of gradient as different types. A first model parameter of the model may be updated in a manner responsive to a mismatch between at least two estimates of the first moments of components corresponding to the first model parameters and a magnitude relationship between the at least two estimates of the second moments of the components.

[0017] In an embodiment, training examples may be provided in a streaming manner, and as new training examples arrive, the model to be trained may be updated, and the model is used to predict the value of the outcome based on the input features. In applications where data is provided in a streaming manner, it may be necessary to dynamically update the model parameters to adapt to new patterns observed in the input. In such applications, the feature of fast learning without reducing the stability of the learning process plays an important role.

[0018] In an embodiment, the input features may include multiple elements that represent fluctuations in past values ​​of the outcome observed over a period of time, or multiple elements that relate to the outcome.

[0019] In one embodiment, the gradient may be a stochastic gradient of an objective function at an iteration step. The objective function may evaluate the loss between the results on the training examples and the predictions made by the model using the current values ​​of the model parameters from the features on the training examples. The training example may include a single training example or a group of training examples.

[0020] According to another embodiment of the present invention, a computer system for training a model by executing program instructions includes one or more computer-readable storage media for storing the program instructions and training examples for a model having model parameters, and a processing circuit in communication with the memory for executing the program instructions, the processing circuit is configured to: obtain training examples for a model having model parameters from the one or more computer-readable storage media, the training examples including results and features for explaining the results, calculate gradients with respect to the model parameters of the model using the training examples, calculate at least two estimates of moments of the gradients using at least two different time constants for the same type of moment, and update the model parameters of the model using the at least two estimates of the moments using the at least two different time constants to reduce an error between calculating the at least two estimates of the moments of the gradients.

[0021] By using a computer system according to an embodiment of the present invention, it is expected that the risk of major errors in gradient calculation due to the influence of old model parameters before learning progress is reduced. Both moment estimations with different time constants can be taken into account in the model parameter update. In this way, fast learning can be performed without reducing the stability of the learning process.

[0022] Computer program products relating to one or more aspects of the present invention are also described and claimed herein.

[0023] Additional features and advantages are realized through the embodiments of the invention.Other embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed invention.

[0024] The above and other features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a block diagram of a forecasting system including a model training module for training a predictive model, according to one embodiment of the present invention. [Diagram 2] FIG. 2 is a detailed block diagram of a model training module according to one embodiment of the present invention. [Diagram 3] 1 is a flowchart illustrating a process for training a predictive model according to one embodiment of the present invention. [Figure 4] FIG. 2 illustrates pseudocode implementing a process for training a predictive model according to one embodiment of the present invention. [Diagram 5] 1 is a diagram illustrating updating model parameters using moment estimation with both long-term and short-term time constants, according to an embodiment of the present invention. [Figure 6] 1 is a schematic diagram of a computer system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] In the following, exemplary embodiments of the present invention are described. Those skilled in the art will understand that the embodiments described below are provided by way of example only and are not intended to limit the scope of the present invention.

[0027] One or more embodiments according to the invention are directed to computer-implemented methods, computer systems, and computer program products for training a predictive model, where model parameters of the predictive model are updated in a manner based on moment estimation and parameter update techniques.

[0028] First, referring to Figures 1-2, a computer system for training a predictive model used to forecast future outcomes from input features according to a stochastic gradient descent (SGD) technique using a novel adaptive moment estimation according to an embodiment of the present invention is described. Referring to Figures 3-5, a computer-implemented method and computer program product for training a predictive model according to a stochastic gradient descent technique using adaptive moment estimation according to an embodiment of the present invention is described. Furthermore, referring to Figure 6, a hardware configuration of a computer system according to an embodiment of the present invention is described.

[0029] Referring to FIG. 1, a block diagram of a forecasting system 100 including a model training module 130 is illustrated according to an embodiment of the present invention. The forecasting system 100 is configured to predict future outcomes based on input data 102 available at the time of the forecast. The outcome to be predicted may be any real-time or recorded measurable quantity obtained from a system, device, sample, or environment, or combination thereof, without or with the use of suitable hardware, which may be sensors measuring physical, chemical, biological, or environmental parameters, or interface devices that have access to such information. In an embodiment, the outcome may be:

[0030] Without being limited thereto, examples of measurable quantities may include any physical quantity, such as temperature, voltage, current, and an amount of a substance, a number of objects, a number of occurrences of a phenomenon or event such as sunspot number, the presence of an object, the occurrence of a phenomenon or event, meteorological observations such as precipitation or river level, a resource value, or an aggregate value (e.g., an index) assessing any combination of the foregoing.

[0031] It is contemplated that the results may be any demand or supply, and in embodiments may include network traffic, power demand or supply, manufacturing demand for parts subject to final product manufacturing, and the like.

[0032] As shown in FIG. 1, the forecasting system 100 may include an input acquisition module 110 for acquiring input data 102, a training data generation module 120 for generating training data based on current and historical data of the acquired input data 102, a model training module 130 for training a model based on the training data, and a predictive model 150 trained by the model training module 130.

[0033] The input acquisition module 110 is configured to acquire input data 102. The input data 102 may be provided to the forecasting system 100 in a streaming or sequential manner and may include observations of the outcome itself and, if necessary, information considered to be relevant to the outcome. The input data 102 may be stored in a suitable storage device. The actual observations of the outcome are used as the correct information (target variable) and may also be used as features (one or more explanatory variables) for training. If available, information considered to be relevant to the outcome is used as features (e.g., one or more explanatory variables) for training.

[0034] The training data generation module 120 is configured to generate training data for the predictive model 150 by extracting observed values ​​of outcomes as correct information and features to explain the outcomes from current and past data of the acquired input data 102 available at the time of prediction. Each training example in the training data includes extracted features, which may be given in the form of a feature vector, and the actual outcome observed in the past and extracted. The features may include past values ​​of the outcome itself, or information considered to be related to the outcome, or both. In an embodiment, the features may also include data obtained primarily by processing the input data with other models (e.g., sub-models).

[0035] The particular content of the training data may depend on the particular design of the predictive model 150. In an embodiment in which the predictive model 150 is designed to predict a future value of an outcome a predetermined time (e.g., 6 hours) from the time of prediction based on fluctuations in past values ​​of the outcome over a predetermined time period (e.g., 24 hours), the currently observed value of the outcome is used as the correct input, and past values ​​of the outcome over a predetermined time period (ending at a point prior to the predetermined time period from the time of prediction) (e.g., past values ​​from 6 to 30 hours ago) are used as features. In this way, the feature vector of the training example includes multiple elements representing past fluctuations of the outcome observed over the predetermined time period.

[0036] In embodiments where the predictive model 150 is designed to predict future values ​​of an outcome (such as demand over a given time from the present) based on relevant information, the currently observed value of the outcome (actual demand from a given time prior to the present) is used as the true information, and the currently available relevant information is used as the features. The currently available relevant information includes information believed to be relevant to the outcome. The currently available relevant information may also include past values ​​of the outcome itself. In one non-limiting embodiment, the feature vector includes multiple elements related to the outcome.

[0037] In the case where the outcome is power demand, it is contemplated that future power demand will be considered relative to the past demand itself, as well as other environmental parameters such as day of the week, season, weather, maximum and minimum temperatures (forecasted or actual), etc. Thus, power demand and observed values ​​of such environmental parameters may be used as features.

[0038] The model training module 130 is configured to train the predictive model 150 in a manner based on a novel moment estimation and parameter update technique. The predictive model 150 has model parameters 152 that are tuned by the model training module 130. It is envisioned that the training of the predictive model 150 may be based on Stochastic Gradient Descent (SGD) with adaptive moment estimation as described herein. In one non-limiting embodiment, the predictive model 150, and more specifically the model parameters 152, are updated each time a new training example arrives (becomes available). As can be appreciated, the training of the predictive model 150 may be considered as online learning.

[0039] The predictive model 150 is configured to predict future values ​​of the outcome based on the input features and output a forecast as a predicted outcome 104. The input acquisition module 110 also inputs features to the predictive model 150 based on the acquired input data 102. The specific content of the input features may depend on the specific content of the training data and the specific design of the predictive model 150. In one non-limiting embodiment, the most recent value of the outcome over a predetermined time period is used as the feature. It is contemplated that currently observed information that is considered relevant to the outcome, such as the day of the week, season, weather, maximum and minimum temperatures, etc., may be used as the input feature. As can be appreciated, in this specification, the term "future" indicates that the goal of the prediction is outside the time range of the input features.

[0040] Since the training process is based on SGD, it is contemplated that the predictive model 150 may be based on any of a variety of model architectures that can be trained by standard SGD. Example predictive model 150 architectures include, but are not limited to, regression analysis such as logistic regression, support vector machines (SVM), linear regression, graphical models, and artificial neural networks (ANNs) such as DNNs (deep neural networks), RNNs (recurrent neural networks), LSTMs (long short-term memory), and CNNs (convolutional neural networks), among others.

[0041] In one non-limiting embodiment, the predictive model 150 is generally described as a regression model (e.g., real-time forecasting) that predicts future values ​​of an outcome based on input features provided in a streaming manner. In an embodiment, moment estimation and parameter update techniques are applied to tasks that require dynamic adaptation to new patterns of inputs, and thus parameters are frequently updated. However, the predictive model 150 is not limited to a regression model, and the goal of the predictive model 150 is not limited to future values ​​of an outcome. In an embodiment, the predictive model 150 may be a classification model that predicts the category or class in which an observation characterized by the input features falls. It is envisioned that the classification model may be a binary classification model, or a multi-class classification model that may be implemented by combining multiple binary classification models and output multiple outcome values. As can be appreciated, the prediction is not limited to real-time forecasting. In an embodiment, the entire training dataset may be provided in advance. Even if the entire training dataset is available prior to learning, the model parameters are iteratively updated by using a single training example or a group of training examples randomly taken from the entire training dataset.

[0042] 2, a detailed block diagram of the model training module 130 is described. The model training module 130 includes an initialization submodule 132 for initializing settings for training the predictive model 150, a gradient computation submodule 134 for computing stochastic gradients of an objective function with respect to parameters of the training predictive model 150 using a single training example or a group of training examples (mini-batch) of training data, a moment estimation submodule 136, 142 for adaptive moment estimation for parameter update, and a parameter update submodule 148 for updating model parameters 152 of the predictive model 150 in a manner based on moment estimation.

[0043] The initialization submodule 132 is configured to initialize the hyper-parameters, model parameters 152 of the predictive model 150, and estimates of low-order moments of the stochastic gradient used for parameter update. The hyper-parameters may be set to pre-defined or default values, which may be empirically determined or optimized by any well-known parameter optimization method, such as Bayesian optimization, grid search, etc. The hyper-parameters may include a base step size to achieve a base learning rate fit and a base exponential decay rate for moment estimation. In an embodiment, the hyper-parameters include a factor (e.g., power or exponential) to modify the base exponential decay rate to an additional exponential decay rate for additional moment estimation with a different time constant or the additional exponential decay rate itself. It is contemplated that the initial model parameters may be set to random values ​​around zero and the estimates of the low-order moments may be initialized to zero.

[0044] The gradient computation submodule 134 is configured to compute a stochastic gradient of the objective function with respect to the model parameters 152 of the predictive model 150 using a single training example or a group of training examples (e.g., a mini-batch) of the given training data. As can be understood, the term "stochastic" generally refers to a process linked to random probability, but not necessarily meeting the independent and identically distributed (IID) requirement. A "stochastic" gradient in this specification means that an estimate of the gradient of the objective function is computed from a portion of the training dataset rather than the entire dataset. The objective function evaluates the loss between the outcome of the training examples and the prediction given by the predictive model 150 using the current values ​​of the model parameters 152 from the features of the training examples.

[0045] The moment estimation submodules 136, 142 are configured to use the stochastic gradients calculated by the gradient calculation submodule 134 to calculate estimates of the moments of the stochastic gradients with different time constants. As used herein, the time constant for moment estimation is determined by the time it takes for the smoothed response to deviate from the original value by a given value (1-e -1 ) is defined as the amount of time it takes for the moment to reach the desired moment. The moment to be estimated may include one moment or multiple moments. When there are multiple moments to be estimated, a different time constant is given for each moment of the same type.

[0046] In an embodiment, the moments to be estimated include the first moment of the stochastic gradient and the second moment of the stochastic gradient. As can be seen, the nth moment of a variable is defined as the expectation of the nth power of the variable. The first moment represents the mean or average value of the stochastic gradient and is related to the accumulation rate. The second moment represents the mean of the squared stochastic gradient or the (non-centered) variance of the stochastic gradient and serves to scale the individual learning rates for the model parameters 152 of the predictive model 150 and is related to the sensitivity. Thus, there are two moment estimation sub-modules, including the first moment estimation sub-module 136 and the second moment estimation sub-module 142.

[0047] The first moment estimation submodule 136 is configured to use the stochastic gradient to calculate multiple estimates of the first moment of the stochastic gradient with different time constants. In one non-limiting embodiment, the number of estimates and the number of time constants is two. Thus, the first moment estimation submodule 136 includes a long-term portion 138 for calculating an estimate of the first moment of the stochastic gradient for the long-term time constant and a short-term portion 140 for calculating an estimate of the first moment for the short-term time constant.

[0048] The second moment estimation submodule 142 is configured to use the stochastic gradient to calculate multiple estimates of the second moment of the stochastic gradient with different time constants. In this manner, the second moment estimation submodule 142 includes a long-term portion 144 for calculating an estimate of the second moment for a long-term time constant and a short-term portion 146 for calculating an estimate of the second moment of the stochastic gradient for a short-term time constant. The time constant for the second moment may or may not be equal to the time constant for the first moment.

[0049] In an embodiment, two estimates at two different time constants are obtained for each of the first and second moments, resulting in a total of four estimates. To minimize the difficulty of adjusting the hyper-parameters, the number of estimates and the number of time constants for each type of moment is set to two. However, it is contemplated that more than two time constants for each type of moment may be utilized. Furthermore, the number of estimates and the number of time constants for one type of moment may or may not be equal to the number of estimates and the number of time constants for the other type of moment.

[0050] The parameter update submodule 148 is configured to update the model parameters 152 of the predictive model 150 using a total of four estimates of the first and second moments with two different time constants. Each of the model parameters 152 is updated in a conservative manner by an amount determined individually by a corresponding component of the estimate of the moments of the stochastic gradient. As can be seen, the stochastic gradient or the moments of the stochastic gradient may be calculated as a vector whose elements correspond to the model parameters 152 of the predictive model 150.

[0051] In one non-limiting embodiment, each parameter in model parameters 152 is updated in a manner that is responsive to the mismatch between the two estimates of the first moment of the corresponding component and the magnitude relationship between the two estimates of the second moment of the corresponding component. Updating model parameters 152 of predictive model 150 using multiple estimates of the first and second moments is described in further detail below.

[0052] In an embodiment, the computation of stochastic gradients by gradient computation submodule 134, the computation of first and second moment estimates with different time constants by first moment estimation submodule 136 and second moment estimation submodule 142, and the updating of model parameters 152 by parameter update submodule 148 are performed iteratively as new training examples arrive.

[0053] It is contemplated that each of the aforementioned modules 110, 120, 130, and 150 shown in Figure 1 and each of the aforementioned sub-modules 132, 134, 136, 138, 140, 142, 144, 146, and 148 of the model training module 130 shown in Figure 2 may be implemented as a software module including program instructions and / or data structures, as a hardware module including electronic circuitry, along with hardware components such as a processor, memory, and the like, or as a combination thereof. It is contemplated that the modules 110, 120, 130, and 150 shown in Figure 1 and the sub-modules 132, 134, 136, 138, 140, 142, 144, 146, and 148 shown in Figure 2 may be implemented on a single computing device, such as a personal computer or server machine, or in a distributed manner across multiple devices, such as a cluster of computing nodes, a client-server system, an edge computing system, a cloud computing system, and the like.

[0054] In an embodiment, the input data 102 and the model parameters 152 may be stored in a suitable internal or external storage device or storage medium to which processing circuitry of a computer system implementing the model training module 130 is operatively coupled. The prediction results 104 generated by the predictive model 150 using current values ​​of the model parameters 152 may be stored in a suitable internal or external storage device or storage medium or output in any form from a device, which may include a display device, a speaker, an alarm, a light indicator, a printer, a network adapter, an interface device connected to a subsequent system.

[0055] 3-5, a process for training a predictive model 150 according to an exemplary embodiment of the invention will now be described.

[0056] 3 is a flow chart illustrating a process for training the predictive model 150. FIG 4 shows pseudocode for implementing the process for training the predictive model 150. It is contemplated that the process illustrated in FIGS. 3-4 may be performed by processing circuitry, such as a processing unit, of a computer system implementing at least the model training module 130 illustrated in FIG 1 and its sub-modules illustrated in FIG 2.

[0057] In an embodiment, the process shown in FIG. 3 begins at step S100, for example, in response to a request from an operator to begin the training process of the predictive model 150 or in response to the forecasting system 100 being started.

[0058] In step S101, the processing unit may obtain hyper-parameters including decay rates for the first and second moments and initialize estimates of the first and second moments for both short-term and long-term time constants. The model parameters 152 may also be initialized with appropriate initial values ​​(e.g., random values ​​around zero), which may depend on the architecture of the predictive model 150.

[0059] In the pseudocode shown in FIG. 4, lines 010-120 correspond to parameter and variable initialization in step S101. In the pseudocode, r represents the base step size or base learning rate hyper-parameter. bl1 and bl2 represent exponential decay rates for the first and second moments, respectively, contained within the interval [0,1) with a base time constant. p t Let yG(x;p) denote the model parameters of the predictive model G(x;p) at the current iteration step t, where x denotes the input feature vector and p denotes the model parameter vector. Thus, p0 denotes the initial model parameter vector of the predictive model G (at t=0). f(p) is the objective function with model parameter vector p. The objective function is the loss function {yG(x;p)} 2 where y represents the observation on one training example, and G(x;p) outputs the predicted outcome given the input feature vector x on the training example.

[0060] The objective function f(p) is differentiable or sub-differentiable with respect to the model parameters p. However, any function that is at least numerically differentiable by any known library may also be applicable as the objective function.

[0061] d1, d2 are each positive real numbers (d1, d2>0) and represent powers for changing the basic exponential decay rates bl1, bl2 to additional exponential decay rates bs1, bs2 for additional moment estimation with different time constants (referred to as additional time constants). In one non-limiting embodiment, as shown in lines 090-100 in the pseudocode shown in FIG. 4, d1 and d2 give the additional exponential decay rates bs1, bs2 as the d1 and d2 powers of bl1 and bl2, respectively. When d1, d2>1, the basic exponential decay rates bl1, bl2 represent the decay rates for the long time constant (i.e., the basic time constant is large), whereas the additional exponential decay rate bs1 (=bl1 d1 ), bs2(=bl2 d2 ) represent the decay rates for short-term time constants (i.e., the additional time constants are small). Instead of specifying factors d1, d2 for changing the basic time constants to additional time constants, the additional exponential decay rates bs1, bs2 themselves can be specified directly.

[0062] ml t represents the first moment estimate for the fundamental time constant at iteration step t, and vl t represents the estimate of the second moment for the fundamental time constant at iteration step t. ms t and vs t represent the first and second moment estimates for the additional time constant. In lines 070, 080, 110, and 120 of the pseudocode shown in Figure 4, the initial estimates ml0, vl0, ms0, and vs0 of the first and second moments with different base and additional time constants are initialized to zero (more precisely, a vector of zeros). In line 060, the iteration step t is also initialized to zero.

[0063] In step S102, the processing device may determine, via the model training module 130, whether the training process is to end. In response to determining that the training process is not to end in step S102, the process may proceed to step S103 and perform an iteration of steps S103-S108. When the input data 102 is provided in a streaming manner, the training process continues until no new data arrives or an explicit command to end the training process is received. In the pseudocode shown in FIG. 4, the while loop in lines 130-230 corresponds to the loop of steps S102 to 108. As can be appreciated, even though the process in FIG. 3 was described as continuing until no new data arrives or an explicit command to end is received, the pseudocode shown in FIG. 4 is described for convenience as continuing the training process until the model parameter vector p converges.

[0064] In step S103, the processing device may obtain new training examples (x, y), where x represents a feature vector and y represents an observation as the correct information, via the model training module 130. It is contemplated that the new training examples may be provided as single data points (pure SGD), or a group of predefined data points or subsets (mini-batch SGD).

[0065] In step S104, the processing device calculates, by the gradient calculation submodule 134, a stochastic gradient g of the objective function f(p) at the current iteration step t using the training examples (x, y) of the training data. t We can calculate the stochastic gradient g t is a set of partial derivatives of the objective function f(p), each of which is the derivative with respect to one variable corresponding to one element of the model parameter vector p. The objective function f(p) is 24, line 150 corresponds to step S104. For a single data point (or sample) or a given group of data points (or mini-batch), the stochastic gradient g t It is envisioned that can be calculated.

[0066] In step S105, the processing device derives both the basic time constant and the additional time constant ml by the first moment estimation submodule 136 and the second moment estimation submodule 142. t , vl t , ms t , and vs t In the pseudocode shown in FIG. 4, lines 160-190 correspond to step S105. As can be seen, g t 2 is the stochastic gradient g t 4 instructs the computer to calculate the exponentially moving average of the stochastic gradients and the squared stochastic gradients for the base time constant and the additional time constants (or decay rates). It is contemplated that the manner of calculating the moment estimates may be the same for the base time constant and the additional time constants, except that the decay rates are different.

[0067] In steps S106 to S108, the processing device updates the basic time constant and the additional time constant ml by the parameter update submodule 148. t , vl t , ms t , vs t The model parameter vector p can be updated based on the estimation of the first and second moments using the model parameter vector p. Each element of the model parameter vector p is the first moment ml t , ms t The discrepancy between the two estimates of and the second moment of the corresponding component vl t , vs tIt is updated by a moving average of the stochastic gradient adjusted by a moving average of the squared stochastic gradient depending on the magnitude relationship between the two estimates of

[0068] In step S106, the processing unit performs the parameter update submodule 148 to update the first moment estimates ml for the fundamental time constant and the additional time constants (long-term time constant and short-term time constant). t , ms t In the pseudocode shown in FIG. 4, line 200 corresponds to step S106. The final estimate of the first moment at the current iteration step M t can be calculated as follows:

number

[0069] The function H(a,b) can be defined as follows:

number

number

number

[0070] As we can see, the vector ml in the function H t / (1-bl1 t ) and the vector ms t / (1-bs1 t) represent bias-corrected estimates of the first moments with the fundamental and additional time constants, respectively. In an embodiment, bias correction is implemented to offset the initialization bias of the moment estimates when the estimates are initialized to a vector of zeros. Its denominator approaches unity progressively over time, and thus the bias-corrected estimates are approximately equivalent to biased estimates of the first moments with large t. Although generally described as being implemented, it is contemplated that bias correction may be omitted.

[0071] Final estimate of first moment M t is computed by the element-wise product (e.g., the Hadamard product) of the characteristic function I and the element-wise operator function H. The two estimates of the first moments ml t and ms. t When the corresponding components based on the characteristic function I are inconsistent, the final estimate of the first moment M t has elements set to zero (although in embodiments very small values ​​close to zero are also contemplated). Otherwise, the final estimate of the first moment M t The elements of are multiplied by the element-wise operator function H to find the two estimates of the first moment. t and ms. t The sigma-based filter is set to the value produced by combining the corresponding components of

[0072] Figure 5 shows a schematic diagram illustrating a method for updating model parameters using multiple moment estimates with both long-term and short-term time constants. Figure 5 shows a contour plot of an objective function in a two-dimensional parameter space, where in an embodiment, the model parameters include two parameters px and py. In Figure 5, the dashed lines represent diagrammatically the trajectory of the model parameters 152 evolving during the training process, the solid arrows represent the estimates of the first moments (or moving averages of the stochastic gradients) for the short-term time constants, the dashed arrows represent the estimates of the first moments for the long-term time constants, and the bold arrows represent the final estimates of the first moments.

[0073] As shown in Figure 5, as training progresses, the model parameters 152 are expected to approach the optimal solution (minimum error point). In the trajectory shown in Figure 5, two points (px i ,py i ) and (px j ,py j ) is picked up. The first point (px i ,py i ), the two estimates of the first moment with the long-term and short-term time constants are matched in both the px and py components. Thus, the vector of final estimates, represented by the bold arrows, has the values ​​of components px, py, each of which is generated by combining the respective components of the two estimates of the first moment (e.g., the mean).

[0074] However, the second point (px j ,py j ), the two estimates of the first moment with the long-term and short-term time constants are mismatched in the py component. Thus, one estimate is positive and the other negative. As a result, the component of the final estimate corresponding to the parameter py is set to zero, and the vector of the final estimate, represented by the bold arrow, has a new value (e.g., the mean) for the component px only.

[0075] In embodiments, more than two time constants for the first moments may also be utilized. In one non-limiting embodiment in which more than two time constants are utilized, a conservative rule may assign zero to a component of the final estimate if any of the corresponding components of the multiple estimates are inconsistent. When most of the corresponding components of the multiple estimates are inconsistent, a more relaxed rule may assign a non-zero value even if some corresponding components are inconsistent.

[0076] Referring again to FIG. 3, in step S107, the processing unit updates the second moment estimates vl of the long-term and short-term time constants by the parameter update submodule 148. t , vs tIn the pseudocode shown in FIG. 4, line 210 corresponds to step S107. The final estimate of the second moment at the current iteration step, V t can be calculated as follows:

number

[0077] In an embodiment, the final estimate of the second moment V t is calculated as a vector having the maximum of the components of each of the two estimates of the second moment. Instead of utilizing a magnitude relationship such as a maximum, an arithmetic or geometric mean of the components is also contemplated. In embodiments where more than two time constants are used for the second moment, the maximum or average of the three or more estimates may be utilized.

[0078] In step S108, the processing unit, by the parameter update submodule 148, calculates the final estimates M of the first and second moments according to the update rule expressed as follows: t and V t The current values ​​of the model parameters 152 may be updated based on the

number

[0079] As explained above with respect to the update rule, the final estimate of the first moment M t represents the gradient vector pointing in the direction of the gradient, and the final estimate of the second moment V t The inverse of serves as a sensitivity that serves to scale the individual learning rates for the model parameters p based on the base learning rate r.

[0080] As described herein, the final estimate of the first moment M t are the two estimates of the first moment, ml t and ms. t has elements that are set to zero when the corresponding components of are inconsistent. As can be seen, the update rule does not change uncertain parameters but keeps them current, thus the parameter update scheme is conservative.

[0081] Second moment estimate V t In embodiments where the components of are large, the individual learning rates for the parameters corresponding to the components are small. t Since θ is calculated as a vector with the maximum of each component of the two estimates of the second moment, for each model parameter a smaller learning rate is chosen and therefore the scheme of parameter update is conservative.

[0082] After the parameter update in step S108, the process may loop back to step S102. In response to no new data being acquired or in response to an explicit command to end the training process being received, the process may branch to step S109. In step S109, the processing unit may return final optimized values ​​of the model parameters 152 of the predictive model 150, and the process ends in step S110.

[0083] According to embodiments of the present disclosure, a computer-implemented method, computer system, and computer program product are provided for training a model capable of fast learning without reducing the stability of the learning process.

[0084] As can be seen, the moments of the stochastic gradients have a dependency on the current values ​​of the model parameters. Thus, as training progresses, the moments of the stochastic gradients change. In general, the current estimates of the moments include past estimates based on old parameters, and thus the moments tend to be biased towards past estimates. Therefore, it is useful to appropriately adjust the rate at which the training process forgets past estimates.

[0085] As can be seen, by taking into account at least two estimates of the moments of the stochastic gradient with at least two different time constants (which describe the rate at which past estimates are forgotten) in the model parameter update, the risk of making large mistakes in the gradient estimation due to the influence of outdated model parameters can be reduced. Thus, fast learning can be implemented without reducing the stability of the learning process.

[0086] Although one or more embodiments are described herein as being variations of a particular variant of SGD, it is contemplated that moment estimation may be performed using various other techniques. An embodiment of a variation of SGD using moment estimation may include stochastic gradient descent using moments and RMSProp (root mean square propagation), and the like. It is envisioned that the moment estimation and parameter update techniques according to the embodiments described herein may be applicable to any technique, as long as moment estimation is performed to train a model. In this manner, conventional moment estimation may be replaced with the moment estimation technique described herein, and conventional parameter update rules may be replaced with parameter update rules that take into account at least two estimates of moments of stochastic gradients with different time constants, as described herein.

[0087] Although advantages that may be obtained with respect to embodiments in accordance with the present invention have been described, it should be understood that some embodiments may not have such potential advantages, and that such potential advantages are not necessarily required for all embodiments.

[0088] 6, a schematic diagram of an example of a computer system 10 that may be used for the forecasting system 100 is shown. In an embodiment, the computer system 10 is implemented as a computer system. As can be appreciated, the computer system 10 is merely one example of a suitable processing device and is not intended to suggest any limitation as to the scope of use or functionality of the inventive embodiments described herein. Thus, the computer system 10 is configured to implement and / or perform any of the functionality described above.

[0089] In an embodiment, computer system 10 is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, that may be suitable for use with computer system 10 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, in-vehicle devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0090] Computer system 10 may be described in the general context of computer system executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types.

[0091] Computer system 10 is shown in the form of a general purpose computing device in Figure 6. Components of computer system 10 may include, but are not limited to, a processor (or processing unit) 12 and memory 16 coupled to processor 12 by a bus, including a memory bus or memory controller, and a processor or local bus, using any of a variety of bus architectures.

[0092] In an embodiment, computer system 10 may include a variety of computer system readable media. Such media may be any available media that is accessible by computer system 10, and it is contemplated that such media includes both volatile and nonvolatile media, removable and non-removable media.

[0093] The memory 16 may include a computer system readable medium in the form of a volatile memory such as random access memory (RAM). The computer system 10 may further include other removable / non-removable volatile / non-volatile computer system storage media. By way of example only, a storage system 18 may be provided for reading from and writing to a non-removable non-volatile magnetic medium. As further illustrated and described below, the storage system 18 may include at least one program product having a set (e.g., at least one) of program modules configured to implement the functions of the embodiments of the present invention described herein.

[0094] In one non-limiting embodiment, a program / utility having a set of program modules (e.g., at least one), as well as an operating system, one or more application programs, other program modules, and program data, may be stored in storage system 18. Each of the operating system, one or more application programs, other program modules, and program data, or any combination thereof, may include an implementation of a networking environment. As can be appreciated, the program modules are configured to perform the functions and / or methods of embodiments of the present invention as described herein.

[0095] In an embodiment, computer system 10 may also communicate with one or more peripheral devices 24, such as a keyboard, a pointing device, a car navigation system, an audio system, a display 26, one or more devices that allow a user to interact with computer system 10, or any device that allows computer system 10 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.), or a combination thereof. It is contemplated that such communication may occur via an input / output (I / O) interface 22. In an embodiment, computer system 10 may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, via a network adapter 20. In one non-limiting embodiment, network adapter 20 communicates with other components of computer system 10 via a bus. Although not shown, it is contemplated that other hardware and / or software components may be used with computer system 10, such as microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data archiving storage systems, and the like.

[0096] The present invention may be a computer system, a method, and / or a computer program product, which may include a computer readable storage medium having computer readable program instructions for causing a processor to implement aspects of the present invention.

[0097] A computer readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage media includes: portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory) static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, punch cards, or mechanically encoded devices such as raised structures in grooves with instructions recorded on them, and any suitable combination of the above. As used herein, computer-readable storage media should not be interpreted as signals that are transitory in nature, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses through a fiber optic cable), or electrical signals transmitted through wires.

[0098] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to the respective computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing / processing device.

[0099] The computer readable program instructions for carrying out the operations of the present invention may be either source or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or object code, including object oriented programming languages ​​such as Smalltalk®, C++, and traditional procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or a connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer readable program instructions to personalize the electronic circuitry by utilizing state information of the computer readable program instructions to perform aspects of the present invention.

[0100] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0101] Such computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the instructions, executed via the processor of the computer or other programmable data processing apparatus, produce means for implementing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams to create a machine. Such computer-readable program instructions may also be stored in a computer-readable storage medium that may direct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium having stored thereon the instructions includes an article of manufacture including instructions that implement aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0102] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams, causing the computer, other programmable apparatus, or other device to perform a series of operational steps to create a computer-implemented process.

[0103] The flowcharts or block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions that includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in the blocks may occur in an order other than that described in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It will also be noted that each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or a combination of dedicated hardware and computer instructions.

[0104] The terms used in the specification are merely for the purpose of describing particular embodiments and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that as used herein, the terms "comprises" and / or "comprising" specify the presence of stated features, steps, layers, elements, or components, or combinations thereof, but do not exclude the presence or addition of one or more other features, steps, layers, elements, components, or groups thereof, or combinations thereof.

[0105] Corresponding structures, materials, acts, and equivalents of all means or step and functional elements in the following claims are intended to include any structure, material, or act for performing a function in combination with other claimed elements as specifically claimed. The description of one or more aspects of the present invention has been presented for purposes of illustration and description and is not intended to be exhaustive or limited to the invention in the form disclosed.

[0106] Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terms used herein have been chosen to best explain the principles of the embodiments, practical applications or technical improvements over technology found in the marketplace, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for training a model, comprising: obtaining training examples for a model having model parameters stored on one or more computer-readable storage media operatively coupled to a hardware processor, the training examples including a first result and features for explaining the first result; calculating gradients of the model with respect to the model parameters using the training examples; using the gradient to calculate at least two estimates of the moments of the gradient using at least two different time constants for the same type of moment; using a hardware processor to update the model parameters of the model using the at least two estimates of the moment with the at least two different time constants to reduce an error between calculating the at least two estimates of the moment of the gradient; 23. A computer-implemented method comprising:

2. The method of claim 1 , wherein each of the model parameters is updated in a conservative manner by an amount determined individually by each of the at least two estimates of the components of the moment.

3. 3. The method of claim 2, wherein in response to the at least two estimates of the moments having a mismatch in a component corresponding to a first model parameter, the first model parameter of the model is updated by zero or a small amount.

4. 4. The method of claim 3, wherein in response to the at least two estimates of the moments matching in the component corresponding to the first model parameter, the first model parameter is updated according to a value produced by combining respective components of the at least two estimates of the moments corresponding to the first model parameter.

5. 5. The method according to claim 2, wherein the first model parameter of the model is updated according to the maximum or average value of the at least two estimates of the components of the moments corresponding to the first model parameter.

6. 6. The method of claim 1, wherein the moments include first and second moments as different types, the first moment representing an average of the gradients, and the second moment scaling individual learning rates for the model parameters of the model.

7. 6. A method according to claim 1, wherein the moments include first and second moments as different types, and a first model parameter of the model is updated in a manner responsive to a mismatch between at least two estimates of the first moments of the components corresponding to the first model parameters and a magnitude relationship between at least two estimates of the second moments of the components.

8. 8. The method of claim 1, wherein the time constants modify an exponential decay rate for moment estimation, the time constants including a first time constant and a second time constant greater than or less than the first time constant.

9. 9. The method of claim 1, wherein the computing of gradients, the computing of the at least two estimates of moments, and the updating of the model parameters are performed iteratively in response to new training examples.

10. 10. The method of claim 1, wherein the training examples are provided in a streaming manner, and each time a new training example arrives, the model to be trained is updated and the model is used to predict a value of the first outcome based on input features.

11. The method of claim 10 , wherein the input features include a plurality of elements representing fluctuations in past values ​​of the second outcome observed over a predetermined period of time.

12. 12. The method of claim 1, wherein the gradient is a stochastic gradient of an objective function at an iteration step, the objective function evaluating the loss between the first result on the training examples and a prediction made by the model using current values ​​of the model parameters from the features on the training examples, the training examples comprising a single training example or a group of training examples.

13. 1. A computer system for training a model by executing program instructions, comprising: one or more computer readable storage media for storing said program instructions and training examples for a model having model parameters; a processing circuit in communication with the memory for executing the program instructions, obtaining training examples for a model having model parameters from the one or more computer readable storage media, the training examples including outcomes and features to explain the outcomes; calculating gradients of the model with respect to the model parameters using the training examples; using said gradients to calculate at least two estimates of said moments with at least two different time constants for the same type of moment; updating the model parameters of the model using the at least two estimates of the moment with the at least two different time constants to reduce an error between calculating the at least two estimates of the moment of the gradient; The processing circuitry is configured to A computer system comprising:

14. 14. The computer system of claim 13, wherein the processing circuitry is configured to update each of the model parameters in a conservative manner by an amount determined individually by each component of the at least two estimates of the moment.

15. 15. The computer system of claim 14, wherein the processing circuitry is configured to update the first model parameter of the model by zero or a small amount in response to the at least two estimates of the moment mismatching in the component corresponding to the first model parameter, and to update the first model parameter according to a value generated by combining respective components of the at least two estimates of the moment corresponding to the first model parameter in response to the at least two estimates of the moment matching in the component corresponding to the first model parameter.

16. 15. The computer system of claim 14, wherein the processing circuitry is configured to update the first model parameter of the model according to a maximum or average value of the at least two estimates of a component of the moment corresponding to a first model parameter.

17. 17. The computer system of claim 13, wherein the moments include a first moment of the gradient and a second moment of the gradient as different types, the first moment representing an average of the gradients, and the second moment scaling individual learning rates for the model parameters of the model.

18. 18. A computer system according to claim 13, wherein the moments include first and second moments as different types, and a first model parameter of the model is updated in a manner responsive to a mismatch between at least two estimates of the first moments of a component corresponding to the first model parameter and a magnitude relationship between at least two estimates of the second moments of the component.

19. A computer program for training a model, the computer program causing a computer to carry out the method according to any one of claims 1 to 12.

Images