Performing time series predictions for computer control systems

Abstract

To provide a computer-implemented method of training a model for making time-series predictions of a computer-controlled system.SOLUTION: The model uses a stochastic differential equation (SDE) comprising a drift component and a diffusion component. The drift component has a predefined part representing domain knowledge that is received as an input to the training, and a trainable part. When training the model, values of the set of SDE variables at a current time point are predicted based on their values at a previous time point, and based on this, the model is refined. In order to predict the values of the set of SDE variables, the predefined part of the drift component is evaluated to get a first drift, and the first drift is combined with a second drift obtained by evaluating the trainable part of the drift component.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a computer-implemented method and corresponding system for training a model for performing time series forecasting based on a computerized control system. The present invention further relates to a computer-implemented method and corresponding system for using the trained model to perform time series forecasting for a computerized control system. The present invention further relates to a computer-readable medium including instructions for performing one or both of the above methods and / or including model data representing the trained model. [Background technology]

[0002] Background technology In many real-life applications, it is desirable to use models to perform time-series predictions for computer-controlled systems. As a practical example, in gasoline particulate filters for gasoline engines, soot particles accumulate on the filter. Therefore, to maintain the filter's functionality, it is necessary to predict the accumulation of soot particles so that a cleaning operation (e.g., using oxygen bursts) can be initiated without delay. More generally, in such computer-controlled systems, one or more sensors are typically used to obtain measurements of the system and / or its environment, and one or more actuators are used to perform actions that affect the environment. Computer-controlled systems include robotic systems in which a robot may perform one or more tasks automatically, e.g., under the control of an external device or an embedded controller. Further examples include vehicles and their components, home appliances, power tools, manufacturing machinery, personal assistants, access control systems, drones, nanorobots, and heating control systems.

[0003] Unfortunately, in many cases, it is not feasible to base such time series predictions on explicit physical models. While simulators are available that can provide reasonably accurate predictions for gasoline particulate filters, the computational cost of using such simulators is prohibitive for real-time applications such as control and monitoring. In general, the underlying physical models of computer-controlled systems and / or their environments are often too complex to be explicitly modeled or effectively simulated.

[0004] In such instances where it is not possible to use a physical model, a promising approach is to instead use machine learning techniques to train a model for time series forecasting. Such a model may be trained based on measurements of physical quantities. In this way, a model that matches the available observations can be automatically determined. Unless otherwise specified, in this specification, measurements of a given physical quantity can be direct (e.g., obtained from a sensor for the physical quantity, sometimes also referred to as an observable quantity) or indirect (e.g., calculated from measurements of other physical quantities, sometimes also referred to as a latent quantity).

[0005] The use of models based on stochastic differential equations (SDEs) is particularly advantageous because they can capture the actual occurrence probability of, for example, noise present in many systems. A. Look et al., "Differential Bayesian Neural Nets" (available at https: / / arxiv.org / abs / 1912.00796 and incorporated herein by reference) describes a trainable model for time series forecasting based on SDEs. Both the drift and diffusion components of the SDE are modeled as Bayesian neural networks. The posterior of their weights is inferred using stochastic gradient Langevin dynamics.

[0006] Unfortunately, existing trainable predictive models based on SDEs suffer from several drawbacks. When applied in practice, existing models are often not sufficiently accurate, making it difficult to effectively control or monitor the target computer-controlled system. Furthermore, achieving a certain level of accuracy often requires large amounts of training data. Because training data is typically based on actual measurements, it is often expensive or difficult to obtain. Training efficiency is also an issue, for example, because in many applications, existing models may take a long time to converge or may not converge at all. A final drawback is that existing trainable SDE models are typically black-box models that offer little or no interpretability, thus providing little insight into how predictions are determined, making debugging such models difficult and preventing trust in the results. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] "Differential Bayesian Neural Nets" by A. Look et al. (available at https: / / arxiv.org / abs / 1912.00796 and incorporated herein by reference) Summary of the Invention [Problem to be solved by the invention]

[0008] Summary of the Invention It is desirable to have trainable predictive models that can more accurately predict physical quantities, can be trained more efficiently based on fewer measurements, and / or provide more insight into how the model arrived at its predictions. [Means for solving the problem]

[0009] According to a first aspect of the present invention, there is provided a computer-implemented method and a corresponding system for training a model for performing time series forecasting of a computer-controlled system, as specified by claims 1 and 13, respectively. According to another aspect of the present invention, there is provided a computer-implemented method and a corresponding system for using the trained model for performing time series forecasting of a computer-controlled system, as specified by claims 8 and 14, respectively. According to one aspect of the present invention, there is provided a computer-readable medium, as specified in claim 15.

[0010] Various aspects relate to performing time-series predictions for computer-controlled systems. Such predictions may be expressed in terms of a set of variables, sometimes referred to herein as observation variables. A model may be trained based on a set of time-series observations of these observation variables. In general, the observation variables may represent one or more physical quantities of the computer-controlled system. For example, the state of a robotic arm may be represented as an angle, angular velocity, joint position, etc. Alternatively or additionally, the observation variables may represent physical quantities of the environment of the computer-controlled system. For example, the steering dynamics of an autonomous vehicle may be learned from a video camera attached to the autonomous vehicle and directed at the road ahead of the vehicle.

[0011] Stochastic differential equations (SDEs) are used to perform time series forecasting. SDEs are defined in terms of a set of SDE variables. The SDE variables may correspond to a set of observation variables and therefore may represent measurements (direct or indirect) of physical quantities of the computer control system and / or its environment. As explained in more detail below, the SDE variables may also be related to measurements via observation models, in which case the SDE variables themselves may or may not represent measurements of physical quantities.

[0012] In general, an SDE is a differential equation that includes a stochastic process. The SDE includes a drift component and a diffusion component. The stochastic process is typically defined by the diffusion component, which is the product of the diffusion component and a noise variable and is often the derivative of a Brownian motion or Wiener process. The remaining part of the SDE that does not form the stochastic process is referred to as the drift component. This drift component may be considered to model the deterministic dynamics of the system, while the diffusion component may be considered to model the stochasticity of the system. One advantage of using SDEs for time series forecasting, for example, compared to ordinary differential equations (ODEs), is that SDEs can more accurately model systems that exhibit stochasticity. SDEs can also provide an estimate of the uncertainty of a given prediction. This is important for many practical applications, such as in robotics to maintain a safety margin when maneuvering a robot.

[0013] Interestingly, various embodiments use a model in which the drift component of the SDE includes a trainable portion and a predefined portion. The predefined portion may be received as an input for training. The predefined portion of the drift component may represent domain knowledge about the computer-controlled system that can be used to improve the training and thereby the resulting model. For example, the predefined drift portion may be provided as ordinary differential equations derived from physics or based on a phenomenological model of the computer-controlled system. In many practical situations, such domain knowledge, e.g., a coarse and / or incomplete description of the computer-controlled system, may be available. For example, three-dimensional volume dynamics may be modeled as a flow through a single point, such as a center of gravity. As another example, a model on a subset of the system components may be provided. To evaluate the SDE at a given time, both the predefined portion r and the trainable portion f of the drift component may be evaluated and then combined (e.g., added) into a total drift.

[0014] In general, a wide variety of domain knowledge types can be integrated into trainable SDEs in this manner; for example, the techniques presented herein are not limited to some specific types of hard-coded dynamics, but can work with various types of predefined drift portions provided at runtime.

[0015] When training a model, a predefined portion r of the drift component may first be received as input, and then other portions of the model, including at least the trainable portion f of the drift component, may be trained using this predefined portion. During optimization of these other portions, the predefined portion of the drift component is typically left constant, meaning that the predefined portion is not parameterized by parameters trained during training. In particular, given a training set of time-series observations of a computerized control system and / or its environment, the model may be trained by using the model to perform time-series predictions, deriving a training signal by comparing the predictions with the observations, and adjusting a set of parameters of the trainable portion of the drift component based on the training signal. Other parameters may also be adjusted. For example, the diffusion component may be trainable to dynamically adapt to stochastic noise present in the computerized control system of interest. To perform time-series predictions, the predefined and trainable portions of the drift component may be evaluated, and the results may be combined into a total drift. Thus, the trainable portion of the drift component may be optimized to provide a drift contribution that best combines with the drift contributions of the predefined portions of the drift component to provide an optimal total drift for the SDE.

[0016] Similarly, applying the trained model to perform time series forecasting of a computerized control system may include using the SDE to forecast values ​​of a set of SDE variables at a current time based on values ​​of the set of SDE variables at a prior time. As part of this, the drift provided by the trained portion of the drift components and the predefined portion of the drift components may be combined into a total drift. Thus, during use, the drift contribution of the predefined portion of the drift components may be combined with the optimal (e.g., optimized) contribution of the trained portion of the drift components to obtain an optimal total drift. Time series forecasting may be used to control and / or monitor a computerized control system.

[0017] In many situations, including a predefined portion in the drift component of an SDE leads to a more easily trainable model and / or provides more accurate predictions. In effect, the trainable portion of the drift model only needs to calculate deviations from the values ​​provided by the predefined portion, which is an easier learning task than calculating the entire drift component. In particular, because the predefined portion is included, the initial estimate of the dynamics provided by the SDE can be more accurate than if the predefined portion were not included. This typically leads to faster convergence and better optimization. Also, less training data, and therefore fewer measurements, are required to achieve a given model accuracy.

[0018] Another advantage is that a trainable portion of the drift component selected to be well suited for optimization techniques such as stochastic gradient descent, e.g., a trainable portion having a neural network-type architecture, can be combined with a predefined portion described in any manner suitable for modeling the target computer control system, e.g., in closed form or as an external model (e.g., Simulink). For example, the predefined portion may not have efficiently computable derivatives. Using a trainable portion well suited for training and a predefined portion well suited for modeling the computer control system can also help achieve more accuracy and better training performance than, for example, using a neural network alone to calculate the drift.

[0019] Another advantage of using predefined parts is that it leads to models with improved interpretability. Because the model includes predefined parts, its output can also be understood as being based on the drift provided by the model. The contribution of the trainable parts may, for example, be considered as a black-box model that represents deviations from, for example, a physically motivated model used as a predefined part.

[0020] As discussed, an alternative to the presented technique is to model a dynamic environment by manually defining a system of differential equations (e.g., as formulated by a domain expert). While this approach allows for modeling clear regularities in physical phenomena, in practice, it is difficult to accurately simulate their behavior. Experience has shown that a relatively small set of simply related variables is sufficient to reach the Pareto frontier with reasonable effort and expertise. However, pushing the model beyond this frontier toward higher fidelity requires extensive expert time or significant computational resources. Interestingly, more accurate simulations are possible using the presented technique. Furthermore, physically motivated models often exhibit weak numerical behavior. We have observed that models trained according to the presented technique can achieve increased computational robustness due to their more generous computational properties.

[0021] Interestingly, the inventors have discovered that the predefined portion of the drift component does not need to be very accurate or complete in itself to achieve these benefits. For example, it is possible to use a predefined portion that models only a subset of the variables of the SDE. Alternatively, it is possible to use a very simplified model of the computer-controlled system, such as a linearized model, or a model in which a region or volume is modeled at a single point, as the predefined portion. Below, we provide several practical examples that demonstrate this. As long as the predefined drift component provides values ​​that correlate to some degree with the actual drift (e.g., the deterministic dynamics of the computer-controlled system), including the predefined drift component in the trainable SDE typically leads to a more accurate and / or more easily trainable model.

[0022] Optionally, the trainable portion of the drift component and the diffusion component may be Bayesian models. For example, the diffusion component may be trained along with the trainable portion of the drift component based on the same training signal. Using a Bayesian model (e.g., a Bayesian neural network) within the SDE allows for representing the uncertainty of the prediction, thereby obtaining a well-calibrated prediction uncertainty. For example, the inventors have discovered that using a Bayesian neural network can provide a much more accurate uncertainty score than using a conventional neural network in the SDE.

[0023] Optionally, the model may further include an observation model that provides a mapping between the set of variables of the SDE and the set of observed variables. This model can determine time series predictions in terms of the observed variables, and training observations may be expressed in terms of values ​​for the set of observed variables. The set of SDE variables may be a subset of the observed variables, or the set of observed variables may be projected onto a (typically smaller) set of SDE variables. This may lead to a more efficient and better trainable model, and therefore a more accurate model. To determine the time series predictions, values ​​for the SDE variables may be predicted according to the SDE, and these values ​​for the SDE variables may then be used by the observation model to predict values ​​of the observed variables.

[0024] The observation model can also be trainable, e.g., trained along the trainable portion of the drift component and / or the diffusion component. In this way, the mapping between the observation variables and the SDE variables and the corresponding interpretation of the predefined portion of the drift component with respect to the observation variables can be dynamically adjusted for a given computerized control system. For example, predefined drift components and an initial observation model may be available based on measurements from a computerized control system similar to the one presented. However, the predefined drift components may not be sufficiently calibrated for the given computerized control system. Having a trainable observation model allows for compensation for this calibration discrepancy by dynamically adjusting how measurements map to the SDE variables for which the predefined drift components are defined. For example, it may be beneficial to use an observation model that uses the same set of SDE variables and observation variables, is initially set to an identity function, and is trained to effectively calibrate the predefined drift components to the given computerized control system.

[0025] Optionally, a noise model may be used to define the probability of a noisy measurement of a given observation. For example, the noise model may be defined with respect to the SDE variables, or, if used, the observed variables of the observation model. In training, where a time series prediction is determined, the noise model may be used to determine the likelihood that a measurement of a selected observation is observed according to the determined prediction, e.g., log-likelihood. In use, performing a time series prediction may optionally include applying the noise model to predict the noisy measurements. Thus, the noise model may enable accurate modeling of measurement errors.

[0026] One approach to infer model parameters, such as the parameters of the trainable part of the diffusion component, is to apply posterior inference. In particular, time series prediction may be based on the posterior distribution of the variables of the drift and diffusion model. This posterior distribution can be approximated using, for example, variational inference or Markov Chain Monte Carlo (MCMC).

[0027] Interestingly, however, the inventors have found that applying model selection instead of posterior inference can improve results. Thus, a model may be trained based on a set of observations by performing maximum likelihood estimation. In a probabilistic setting, this means marginalizing latent (e.g., intermediate) variables apart from the parameters to be trained (e.g., parameters of a Bayesian neural network, if one is used; values ​​of SDE variables, if a noise / observation model is used; values ​​of observed variables, if a noise model is used), and maximizing the marginal likelihood for the parameters to be trained. Maximum likelihood estimation may, for example, be based on the likelihood (e.g., log-likelihood) that measurements are observed according to a noise model, as described above. The combination of model selection and applying a Bayesian model to at least the trainable portion of the drift component is referred to herein as the "empirical Bayes" method.

[0028] Applying model selection instead of posterior inference is advantageous for several reasons. Compared to applying Markov chain Monte Carlo, it avoids directly sampling global latent variables, such as Bayesian model parameters, which would result in larger variance in the estimator. Compared to variational inference, it avoids making posterior simplifying assumptions or introducing a second approximation via Monte Carlo sampling to ensure a closed-form computation of the transition density. In general, compared to posterior inference, model selection effectively directly models the marginal predictive distributions, for example, instead of first inferring the posterior distribution of the model parameters and then computing the posterior predictive distribution. This means that fewer simplifying assumptions need to be made along the way, while still being computationally more efficient and feasible. For example, it is not necessary to construct a Markov chain with global model parameters. It also improves training stability.

[0029] Optionally, to further improve training efficiency, the trainable portion of the drift component may be evaluated by sampling values ​​of parameter-free random variables and deterministically evaluating the trainable portion based on these sampled values. This is similarly performed when using a model. For example, when using gradient-based optimization, this avoids passing gradients to the parameters of the trainable portion of the drift component, which can be very large in the case of, for example, neural networks. In particular, the trainable portion of the drift component may be evaluated based on the sampled values ​​by determining statistical parameters of the linear activations of the nodes of the trainable portion and deterministically calculating these linear activations based on the statistical parameters and the values ​​of the random variables. This leads to a particularly large reduction in the variance of the marginal likelihood and therefore to a particularly fast convergence of the optimization.

[0030] Optionally, during the step of deriving the training signal, a regularization term can be evaluated to minimize the difference between a model with a trainable drift component and a model without a trainable drift component. Thus, the model may be encouraged to stay in the vicinity of a predefined drift component during training, which is expected to reduce overfitting and better generalize to unseen data. Such overfitting is prevented because the regularization term penalizes high complexity, e.g., the model is directed to explaining the data as simply as possible.

[0031] Interestingly, we have shown that using the Kullback-Leibler (KL) divergence between the posterior distributions of predictions for models with and without a trainable drift component as a regularization term is particularly beneficial. We have discovered that when the models used in SDEs are Bayesian models, this KL divergence can be determined by combining the divergence of the non-Bayesian model with the divergence between the prior and posterior distributions of the parameters of each Bayesian model. We have shown that by applying gradient-based optimization to a loss term that includes the log-likelihood loss and this particular Kullback-Leibler divergence, a training procedure can be obtained that can be shown to hold PAC (possibly approximately accurate) bounds. Such PAC bounds allow for the derivation of strong mathematical guarantees on learning ability, and therefore mathematically guarantee high generalization of the trained model. This particular choice of regularization term in combination with a Bayesian model in SDEs is referred to as the "empirical PAC Bayes" training method.

[0032] Optionally, the first drift and the second drift can be combined by combining values ​​of the first drift with corresponding values ​​of the second drift according to weight values. The weight values ​​can indicate the weight of predefined portions of the drift components in the combination. Different weight values ​​can be defined for different values. The weight values ​​can, for example, be predefined to completely or partially block the contribution of predefined portions of the drift components of a particular SDE variable. Interestingly, the weight values ​​can also be trained as part of training. In this way, it is possible to dynamically learn how to optimally combine predefined and trainable portions of the drift components. For example, some portions of the predefined drift components may be more accurate than others and therefore receive weight values ​​that can contribute more strongly to the total drift. Having trainable weights also allows predictions made during training to shift from initially relying heavily on the predefined portions, at least for some values, to later relying less heavily on the predefined portions when they are no longer needed.

[0033] Optionally, the predefined portion of the drift component may be determined by fitting a parameterized differential equation to a set of observations prior to training. For example, based on domain knowledge, a domain expert may formulate a differential equation, e.g., an ordinary differential equation for a set of SDE variables. Similarly, parameterized ordinary differential equations may be known that describe a computerized control system or a portion thereof, e.g., in the form of a phenomenological model. Such differential equations may be parameterized to fit various situations in which the computerized control system may be applied. By instantiating the differential equation based on training observations, a predefined portion of the drift component that adequately fits the data of interest may be obtained. Nevertheless, while training the trainable portion of the drift component, the predefined portion of the drift component remains fixed. Thus, for example, conventional techniques for fitting differential equations may be used to determine the predefined portion of the drift component, while the trainable portion may use a model that is more suited to the setting of a machine-learnable SDE, e.g., a model that enables optimization-based learning and / or a model that can provide an estimate of uncertainty.

[0034] Based on the time series predictions performed as described herein, output data can be generated that can be used to control and / or monitor a computerized control system. This output data can be the time series prediction itself. The output data can also include or be based on the uncertainty of the time series prediction, e.g., time-step uncertainty or global uncertainty. For example, such uncertainty may be derived from sampling multiple (e.g., at least 20 or at least 100) time series predictions based on the same initial set of SDE variables. Uncertainty can also be visualized, for example, by visualizing these multiple time series predictions, e.g., by plotting time series of one or more SDEs and / or observed variables. The output data can be provided via an output interface to an output device used to control or monitor the computerized control system.

[0035] Optionally, the output device is an actuator associated with a computer-controlled system, and the processor subsystem is configured to control the computer-controlled system by providing control data to the actuator based on the determined time series prediction. For example, the actuator may be used to control a vehicle, such as an autonomous or semi-autonomous vehicle, a robot, a manufacturing machine, a building, etc.

[0036] Optionally, the trained model can be used to train a computer-controlled system by serving as an environmental model that provides a time-series prediction of the environment of the computer-controlled system. Thus, interactions with the environment can be simulated, based on which the computer-controlled system can be trained using techniques known per se, for example, from model-based reinforcement learning. In this way, less environmental interaction may be required to train the computer-controlled system. For example, the computer-controlled system may be a robotic arm, where the environmental model may be used to predict the effect of performing a particular action (e.g., applying torque) on the behavior (e.g., of joint positions) of the robotic arm.

[0037] A particular application area of ​​the provided techniques is in the control systems of (semi-)autonomous vehicles. The model may be configured to perform time series predictions of the vehicle's mechanical parts. In many cases, the SDE variables can be selected to match the observed variables. In general, the time series predictions provided by the model can be used to monitor the vehicle's performance (e.g., to warn of excessively high emissions) or to control the vehicle (e.g., by adapting operating parameters to reduce emissions, by activating oxygen bursts to remove excessive soot, or by controlling the filling of a hydrogen motor).

[0038] In one example, the vehicle's mechanical component may be a vehicle's exhaust gas treatment component, such as a selective catalytic reduction (SCR), a three-way catalyst (TWC), or a diesel oxidation catalyst (DOC). The observed variables may in this case include one or more of pressure, temperature (upstream and / or downstream of the component), emissions (upstream and / or downstream of the component), and inputs to the component (e.g., NH3 metering in the case of an SCR catalyst). For example, a model may be used to determine predicted emissions, e.g., to adjust operating parameters. In this context, various models are available that can be used as this predefined drift component, e.g., the ETAS ASCMO model.

[0039] In another example, the vehicle's mechanical component may be a stack of multiple fuel cells in a hydrogen motor. The model can capture the dynamics of storing chemical reaction energy in the fuel cells. Observation variables in this case may include one or more of pressure, temperature, humidity, and electrical load (e.g., generated torque expressed as voltage). In practice, the number of cells can be large, for example, at least 100 or at least 1000. This is another application where currently only a coarse dynamic model is available, and therefore its accuracy can be improved by the provided technique. The provided technique also makes it possible to dynamically calibrate such models to new settings, reducing the calibration effort.

[0040] In yet another example, the vehicle's mechanical component may be a particulate filter, such as a gasoline particulate filter (GPF). The observed variables include one or more of engine pressure, mass flow rate, soot mass, and soot particle count. Currently, particulate filters can be modeled with moderate to high fidelity using simulators with high computational costs that make predictions prohibitive for real-time applications. Direct measurement of soot mass is expensive. By using a low-version model of such a simulator as a predefined drift for the inventive technique, a cheap and scalable approach to obtaining predictions can be achieved, and the predictions can be used by the vehicle's engine control unit. For example, if the soot mass is predicted to exceed a threshold, a cleaning action, such as an oxygen burst, can be activated or signaled.

[0041] It will be understood by those skilled in the art that two or more of the above-described embodiments, implementations, and / or any aspects of the invention may be combined in any manner deemed useful. Modifications and variations of any system and / or any computer-readable medium corresponding to the described modifications and variations of the corresponding computer-implemented methods may be made by those skilled in the art based on the description of the invention.

[0042] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described, by way of example, in the following specification and with reference to the accompanying drawings. [Brief explanation of the drawings]

[0043] [Figure 1] FIG. 1 illustrates a system for training a model for time series prediction. [Figure 2] FIG. 1 illustrates a system for using a trained model. [Figure 3] FIG. 1 shows a detailed example of a trainable model. [Figure 4]FIG. 1 shows a detailed example of training a model based on observations. [Figure 5] FIG. 1 illustrates a system for using a trained model as part of a (semi-)autonomous vehicle. [Figure 6] FIG. 1 illustrates a computer-implemented method for training a model for time series prediction of a computer-controlled system based on a set of observations. [Figure 7] FIG. 1 illustrates a computer-implemented method for using a trained model to perform time series predictions for a computer control system. [Figure 8] FIG. 1 illustrates a computer-readable medium containing data.

[0044] It should be noted that these drawings are purely diagrammatic and are not drawn to scale. In the drawings, elements that correspond to elements already described may have the same reference numerals. DETAILED DESCRIPTION OF THE INVENTION

[0045] Detailed Description of the Embodiments 1 shows a system 100 for training a model for time series prediction of a computerized control system based on a set of observations of the computerized control system and / or its environment. The model may use a stochastic differential equation (SDE) of a set of SDE variables. The SDE may include a drift component and a diffusion component.

[0046] System 100 may include a data interface 120 and a processor subsystem 140 that can communicate internally via data communication 121. Data interface 120 may be for accessing model data 040 that defines a model. The model data may include a set of parameters for at least a trainable portion of the drift component of the SDE. As shown, data interface 120 may optionally be for accessing a set of observations 030 of a computerized control system. Model data 040 may be for use, for example, by system 200 of FIG. 2 in performing time series forecasting of the computerized control system in accordance with methods described herein.

[0047] The processor subsystem 140 may be configured to access data 030, 040 during operation of the system 100 using a data interface 120. For example, as shown in FIG. 1, the data interface 120 may provide access 122 to an external data storage 021 that may contain the data 030, 040. Alternatively, the data 030, 040 may be accessed from internal data storage that is part of the system 100. Alternatively, the data 030, 040 may be received from another entity over a network. In general, the data interface 120 may take a variety of forms, such as a network interface to a local or wide area network, e.g., the Internet, a storage interface to internal or external data storage, etc. The data storage 021 may take any known suitable form.

[0048] During operation of the system 100, the processor subsystem 140 may be configured to receive a predefined portion of the drift component of the SDE as a training input using the data interface 120. The predefined portion of the drift component may be included in the model data 040, hard-coded into the instructions for training the model, specified by a user, or the like. The processor subsystem 140 may also obtain a set of observations 030 of the computerized control system. The observations may include time-series measurements of the computerized control system and / or its environment. While these observations are illustrated as being accessed via the data interface 120, this is not required; for example, the observations may be obtained via a sensor interface, as discussed in connection with FIG. 2.

[0049] The processor subsystem 140 can further train the model based on the set of observations. To train the model, the processor subsystem 140 uses the model for time series prediction. This may include using the SDE to predict values ​​of a set of SDE variables at a current time based on values ​​of the set of SDE variables at a previous time, which may include evaluating a predefined portion of a drift component to obtain a first drift and combining the first drift with a second drift obtained by evaluating the trainable portion of the drift component. The training may further include deriving a training signal by comparing the time series prediction with observed values ​​of the set of observations and adjusting a set of parameters of at least the trainable portion of the drift component based on the training signal.

[0050] The system 100 may further include an output interface for outputting trained model data representing the learned (or "trained") model. For example, as also shown in FIG. 1 , this output interface may be constituted by a data interface 120, which in these embodiments is an input / output ("IO") interface through which the trained model data 040 may be stored in data storage 021. For example, the model data 040 defining the "untrained" model may be at least partially replaced during or after training by model data of a trained model, where model parameters, such as neural network weights and other types of parameters, may be adapted to reflect the training in the training data 030. In other embodiments, the trained model data may be stored separately from the model data 194 defining the "untrained" dynamic model. In some embodiments, the output interface may be separate from the data interface 120, but may generally be of the type described above for the data storage interface 120.

[0051] Various details and aspects of the operation of system 100, including their operational aspects, are further elucidated with reference to FIGS.

[0052] In general, system 100 may be implemented as or within a single device or apparatus, such as a workstation, e.g., laptop or desktop-based, or server. This device or apparatus may include one or more microprocessors executing appropriate software. For example, the processor subsystem may be implemented by a single central processing unit (CPU) or graphical processing unit (GPU), but may also be implemented by a combination or system of such CPUs, GPUs, and / or other types of processing units. Software may be downloaded and / or stored in corresponding memory, e.g., volatile memory such as RAM or non-volatile memory such as flash. Alternatively, functional units of the system, e.g., data interfaces or processor subsystems, may be implemented in the device or apparatus in the form of programmable logic, e.g., field programmable gate arrays. In general, each functional unit of the system may be implemented in the form of a circuit. System 100 may also be implemented in a distributed manner, e.g., distributed servers including different devices and apparatus, e.g., in the form of cloud computing.

[0053] 2 shows a system 200 for using a trained model for time series prediction of a computerized control system to control and / or monitor the computerized control system. The model can use a stochastic differential equation (SDE) of a set of SDE variables. The SDE can include a drift component and a diffusion component.

[0054] System 200 may include a data interface 220 and a processor subsystem 240, which may communicate internally via data communication 221. Data interface 220 may be for accessing model data 040 that defines a model. The model data may include a set of parameters for at least a trainable portion of the drift component of the SDE. Model data 040 may further define a predefined portion of the drift component of the SDE. The model may be trained, for example, by system 100 of FIG. 1 according to a training method as described herein. System 200 may be configured to train the model in addition to applying it; for example, system 200 may be combined with system 100 of FIG. 1.

[0055] The processor subsystem 240 may be configured to access data 040 during operation of the system 200 using the data interface 220. For example, as shown in FIG. 2, the data interface 220 may provide access 222 to an external data storage 022 that may contain the data 040. Alternatively, the data 040 may be accessed from internal data storage that is part of the system 200. Alternatively, the data 040 may be received from another entity over a network. In general, the data interface 220 may take a variety of forms, such as a network interface to a local or wide area network, e.g., the Internet, a storage interface to internal or external data storage, etc. The data storage 022 may take any known suitable form.

[0056] During operation of system 200, processor subsystem 240 may be configured to acquire sensor data representing measurements of the computerized control system and / or its environment using data interface 220. Often, the sensor data is acquired from one or more sensors, as described below. Processor subsystem 240 may be configured to determine a time series prediction for the computerized control system based on the measurements. To determine the time series prediction, processor subsystem 240 may use the SDE to predict values ​​of a set of SDE variables at a current time based on values ​​of the set of SDE variables at a prior time, which may include evaluating a predefined portion of a drift component to obtain a first drift and combining the first drift with a second drift obtained by evaluating a trainable portion of the drift component. Processor subsystem 240 may be configured to generate output data based on the determined time series prediction for use in controlling and / or monitoring the computerized control system.

[0057] 2 further illustrates various optional components of system 200. For example, in some embodiments, system 200 may include a sensor interface 260 for directly accessing sensor data 224 acquired by sensors 072 in environment 082. This sensor interface 260 may be in internal communication with processor subsystem 240 via data communication 223. The sensors 072 may be, but are not necessarily, part of system 200. The sensors 072 may have any suitable form, such as an image sensor, a lidar sensor, a radar sensor, a pressure sensor, or a temperature sensor. In some embodiments, sensor data 224 may be acquired from two or more different sensors sensing different physical quantities, and thus may represent sensor measurements of different physical quantities. The sensor interface 260 corresponds to a specific type of sensor and may have any suitable form, including, but not limited to, a low-level communication interface based on I2C or SPI data communication, or a data storage interface of the type described for data interface 220 above.

[0058] In some embodiments, the system 200 may include an actuator interface 280 for providing control data 226 to actuators 092 in the environment 082. Such control data 226 may be generated by the processor subsystem 240 for controlling the actuators 092 based on time series predictions determined as described herein. For example, the actuators may be electrical, hydraulic, pneumatic, thermal, magnetic, and / or mechanical actuators. Specific, but non-limiting, examples include electric motors, electroactive polymers, hydraulic cylinders, piezoelectric actuators, pneumatic actuators, servos, solenoids, stepper motors, etc. Such types of control are described for (semi-)autonomous vehicles with reference to FIG. 5.

[0059] In other embodiments (not shown in FIG. 2 ), system 200 may include an output interface to a rendering device, such as a display, light source, speaker, vibration motor, or the like, that may be used to generate a sensory-perceptible output signal that may be generated based on the determined time series prediction. The sensory-perceptible output signal may be a direct indication of the time series prediction, but may also represent a sensory-perceptible output signal derived for use in guidance, navigation, or other types of control of a computer-controlled system, for example. Actuators may be part of system 200, but are not necessarily so.

[0060] Other types of output interfaces and devices are possible, for example, the output interface may be a data interface similar to or combined with data interface 120 for outputting the generated output data to storage that is also accessible by the output device, or it may be a communication interface, for example a direct communication interface such as USB or IEEE 1394, or a network communication interface such as a Wi-Fi or 4G connector. For example, system 200 may provide the determined time series predictions or data derived therefrom to other systems for controlling or monitoring the vehicle based on the determined time series predictions, for example other subsystems of the vehicle of which it is a part.

[0061] Various details and aspects of the operation of system 200, including any aspects thereof, are further elucidated with reference to FIGS.

[0062] In general, system 200 may be implemented as or within a single device or apparatus, such as a workstation, e.g., laptop or desktop-based, or server. The device may be an embedded device. The device or apparatus may include one or more microprocessors executing appropriate software. For example, the processor subsystem may be implemented by a single central processing unit (CPU) or graphical processing unit (GPU), but may also be implemented by a combination or system of such CPUs, GPUs, and / or other types of processing units. In particular, the processor subsystem may be implemented by one or more embedded processors. Software may be downloaded and / or stored in corresponding memory, e.g., volatile memory such as RAM or non-volatile memory such as flash. Alternatively, functional units of the system, e.g., data interfaces and processor subsystems, may be implemented in the device or apparatus in the form of programmable logic, e.g., field programmable gate arrays (FPGAs). In general, each functional unit of the system may be implemented in the form of a circuit. System 200 may be implemented in a distributed manner, such as across distributed servers comprising different devices or apparatuses, e.g., in the form of cloud computing. In some embodiments, system 200 may be part of a vehicle, robot, or similar physical entity and / or may represent a control system configured to control the physical entity.

[0063] FIG. 3 shows a detailed, non-limiting example of a trainable model for time series forecasting of a computer control system.

[0064] The model uses a stochastic differential equation, or SDE, of a set of SDE variables. Throughout this specification, P is used to represent the number of SDE variables. The number of SDE variables can be, for example, at most or at least 2, at most or at least 5, or at most or at least 10.

[0065] The model is parameterized by a set of trainable parameters PAR, 370. The number of parameters trained when training the model can be, for example, at most or at least 1000, at most or at least 10,000, or at most or at least 1,000,000.

[0066] Specifically, the figure shows how the model can be used to predict the noise value NOV-yt, 361 of a set of observed variables at the current time t, given the values ​​OV-xt-1, 301 of a set of observed variables at a previous time t-1.

[0067] In summary, in this example, a previous observation OV-xt-1 is first processed in an operation block Obs', 310, according to an observation model to determine the value SV-ht-1, 311, of the set of SDE variables at a previous time point. The SDE is then used to predict the value SV-ht, 341, of the set of SDE variables at the current time point. For this purpose, a predefined portion PD-r, 321, of the drift components of the SDE is evaluated to obtain a first drift FDt, 324. A trainable portion TD-f, 322, of the drift components of the SDE is evaluated to obtain a second drift SDt, 325. The first and second drifts FDt, SDt are combined in a combination operation block Cmb, 330, into a total drift OD, 331. This total drift, OD, and the diffusion, DFt, 326, calculated by the trainable diffusion component, TDF-g, 323, of the SDE are then used in the prediction computation block, Pred, 340, to obtain predicted values, SV-ht, of the set of SDE variables. In the computation block, Obs, 350, an observation model is used to predict values, OV-xt, 351, of the set of observed variables at the current time, based on the predicted values, SV-ht, of the set of SDE variables at the current time. A noise model, NM, 360, is finally used to predict noise measurements, e.g., noise values, NOV-yt, 361, of the set of observed variables at the current time.

[0068] As a specific example, the model shown in the figure may be implemented using the following system of stochastic differential equations:

number

number

number

number

number

number

[0069] More specifically, this figure shows the values ​​h of a set of SDE variables. t Given a set of observed variables, x t The probability p ψ (χ t |h t ) is defined as an observation model Obs. The observations are typically represented as a D-dimensional vector

number

[0070] As shown in the figure, the observation model Obs may be parameterized by parameters ψ from the parameter set PAR, but this is not required: for example, the observation model may be fixed (e.g., set to an identity) or manually defined.

[0071] The observation model Obs can be implemented in several ways. For example, the observation model can be an autoencoder (e.g., a variational autoencoder) in which the encoder of the computation block Obs' determines values ​​for a set of SDE variables given values ​​for the set of observation variables, and the decoder of the computation block Obs determines values ​​for the set of observation variables given values ​​for the set of SDE variables. This autoencoder can be trained end-to-end with other components of the model. Various known autoencoder architectures may be applied, for example, using neural networks. In general, the observation model can be implemented by, for example, a neural network, which determines values ​​for a set of SDE variables h t Given a set of observed variables x t In the computation block Obs, this function can be inverted, for example by performing an optimization to find values ​​for the SDE variables that provide a good approximation to the given observations. Here, it is also possible to use a probabilistic observation model, for example in the form of a Bayesian neural network or an autoencoder, a Bayesian encoder and / or decoder.

[0072] In various cases, only coarsely observed measurements may be available for one or more "special" quantities to be predicted, while finer observations are available for other "normal" quantities. In this case, a trainable model may be trained to predict the values ​​of the normal quantities as SDE variables (or equivalently, using the same observation model) and also to predict the values ​​of the special quantities as the output of the observation model. Thus, the model can be trained simultaneously with both fine-grained measurements of the normal quantities and coarse-grained measurements of the special quantities. This allows the model to be used to provide continuous predictions of the special quantities as well, with these coarse predictions aided by the coarse measurements.

[0073] The use of the observation model is optional. If the observation model is not used, the set of observation variables and the set of SDE variables can be made to match, and the mapping using the operation blocks Obs’ and Obs is unnecessary. Mathematically, this corresponds to using the identity function as the probability density function p ψ as described.

[0074] The operation blocks PD-r, TD-f, and Cmb represent determining the total drift OD of the SDE based on the value SV-ht-1 of the set of SDE variables at the previous time point.

[0075] The trainable part TD-f of the drift component can be well represented mathematically as

Number

[0076] Similarly, the diffusion component of the SDE manages (typically non-linear) diffusion dynamics and is also a trainable diffusion component TDF-g provided as a matrix-valued function f parameterized by the set of parameters θ

Number

Number

[0077] For example, the trainable portion of the drift component TD-f and / or the diffusion component TDF-g may be a neural network. Neural networks are also known as artificial neural networks. Examples include deep neural networks and convolutional neural networks. Neural networks are typically parameterized by the weights of the nodes of the neural network. For example, the number of layers of the model may be at least 5 or at least 10, and the number of nodes and / or weights may be at least 1000 or at least 10000. Depending on the particular application, various known architectures for neural networks or other types of machine-learnable models may be used.

[0078] Interestingly, the drift component trainable portion TD-f and / or the diffusion component TDF-g may optionally be Bayesian models, e.g., Bayesian neural networks. Thus, the parameters of these models (e.g., neural network weights) may themselves be random variables, whose probability distributions are parameterized by respective sets of parameters. For example, the figure

number

[0079] The predefined part of the drift component PD-r is generally expressed as, for example, an ordinary differential equation dh in terms of the SDE variables. t =r ξ (ht ,t)dt, we define a function r parameterized by a set of parameters ξ ξ The predefined portion may be represented as (·,·). The predefined portion can similarly be represented by a Bayesian model. The predefined portion PD-r can be provided as a function of a subset of the SDE variables. The predefined portion PD-r is typically non-constant, e.g., may even be non-linear or non-polynomial. Typically, the function defining the predefined portion does not represent the evaluation of a neural network or other type of machine learning model; e.g., the function defining the predefined portion often does not include an activation function such as softmax, ReLu, and / or a sigmoid or tanh function. The predefined portion is also typically selected based on the computerized control system of interest and is therefore application-dependent in the sense that it represents domain knowledge about the computerized control system. Although the predefined portion can be parameterized, these parameters are typically not trained as part of training the model.

[0080] A combination operation block Cmb is used to combine a first drift FDt determined by a predefined portion PD-r of the drift components and a second drift SDt determined by a trainable portion TD-f of the drift components into a total drift OD. It is also possible to combine the two drifts in a fixed way, for example as a (possibly weighted) sum or average. A preferred option is to combine the drifts according to trainable weight values ​​for each value given by a set of parameters PAR. For example, the weights are a free parameter vector γ∈[0,1] that governs the relative importance of prior knowledge on the actual learning problem. P These drifts may be specified as

number

number

[0081] Based on the total drift OD and the diffusion estimate DFt, the prediction component Pred can determine the prediction of the value SV-ht of the set of SDE variables at the current time. This can be done according to the following discrete-time stochastic model, for example, according to the Euler-Maruyama discretization of the SDE:

number

number

[0082] As mentioned above, given predicted values ​​SV-ht of a set of SDE variables, values ​​OV-xt of a set of observed variables can be predicted according to the observation model Obs (if used). Given values ​​OV-xt (or SV-ht if no observation model is used), noise values ​​NOV-yt of a set of observed variables can be predicted according to the noise model NM (if used). The noise model p(y t |x t) is a suitable likelihood function with no learnable parameters (e.g., the standard normal distribution).

[0083] When a Bayesian model is used for the drift component trainable part TD-f and / or the diffusion component TDF-g, the model must account for two sources of uncertainty about the set of SDE variables: the prior distribution

number

[0084] Next, an example of an algorithm for determining a time series forecast according to the model of Figure 3 is presented. The algorithm receives as input initial values ​​h0 for a set of SDE variables, determined, for example, based on sensor data indicative of the state of a computerized control system. For example, given values ​​for a set of observed variables, an observation model may be used in the computation block Obs', as described above. This example uses inheritance sampling to sample values ​​for the random variables based on sampled values ​​of the variables on which they depend. Thus, a sampled time series forecast is obtained.

number

[0085] For example, by repeatedly performing a sampling-based forecast of a time series according to the above algorithm, multiple forecasts can be obtained, based on which, for example, a forecast mean and standard deviation can be determined at each time point.

[0086] FIG. 4 shows a detailed, but non-limiting, example of training a model based on observations.

[0087] Shown in the figure is a trainable model TM, 400, for performing time series forecasting of a computerized control system. The model TM can use a stochastic differential equation (SDE) of a set of SDE variables. The SDE can include a drift component and a diffusion component. The drift component can include, for example, a predefined portion representing domain knowledge about the computerized control system and received as input for training, and a trainable portion whose parameters are determined during training. A detailed example of a trainable model TM is provided in FIG. 3.

[0088] Also shown is a set of parameters PAR,470 of the trainable model TM that are trained as part of training the model. These parameters include at least the parameters φ of the trainable part of the drift component. f As shown in the figure, parameters of various other parts of the model can also be trained. Optionally, the diffusion component of the model can be trained, and its parameters φ r are included in the set of parameters PAR. Optionally, the model includes a trainable observation model (e.g., in the form of an autoencoder as also described in relation to FIG. 3), the parameters ψ of which are included in the set of parameters PAR. Optionally, the drift determined by evaluating the predefined drift components of the model and the trainable drift components are combined according to trainable weight values ​​γ for each element of the drift, and these trainable weight values ​​are included in the set of parameters PAR.

[0089] Although the predefined portion of the drift component can be parameterized, those parameters are not adjusted by training and therefore are not included in the set of parameters PAR. In particular, although not shown in the figure, it is possible to determine the predefined portion of the drift component by fitting a parameterized differential equation to the set of observations OBS before training the model.

[0090] The model may be trained based on a set of observations OBS, 400. Such observations may include time series measurements of the computer control system and / or its environment. Mathematically, such time series can be expressed as:

number

[0091] For example, the number of time points of an observation can be at least 3, at least 10, or at least 100. The number of observations can be relatively large, for example, at least 1,000 or at least 10,000, although this is not necessary if, for example, the number of time points per observation is large. For example, training can be performed using just one observation. The total number of measurements of the observations in the set of observations OBS can be, for example, at least 1,000, at least 10,000, or at least 1,000,000.

[0092] In general, the model may be trained by using the model for time series prediction, deriving a training signal by comparing the determined time series prediction with observations from a set of observations OBS, and adjusting a set of parameters PAR based on the training signal.

[0093] Such training is typically performed by performing an optimization Opt,480 of the objective function. Typically, training is performed using a probabilistic approach such as stochastic gradient descent, e.g., the Adam optimizer disclosed in Kingma and Ba, "Adam: A Method for Stochastic Optimization" (available at https: / / arxiv.org / abs / 1412.6980, incorporated herein by reference). As is known, such optimization methods may be heuristic and / or may reach a local optimum. Different updates may adjust different subsets of the set of parameters PAR, e.g., a first iteration may adjust some subset of parameters φ f ,φ r ,ψ,γ, and a second iteration can update (at least partially) a different subset. Training may be performed instance-by-instance, or in batches of, for example, up to or at least 64 observations, or up to or at least 256 observations.

[0094] In the example shown in this figure, training is performed using maximum likelihood estimation, e.g., by marginalizing latent variables, comparing the marginal likelihoods of possible hypotheses, and selecting the one that provides the highest response. Specifically, a sampling-based approach may be used, given an initial measurement value MOV-y of an observation of a computerized control system, and the trained model TM may be used to determine a predicted value POV-xt, 451, of a set of observed variables at a given time point. This can be done by inheritance sampling. A noise model NM, 460, may be used to determine the likelihood POBSP(yt|xy), 462, that the corresponding measurement value MOV-yt of the observation is observed according to the determined time series forecast. The noise model NM may follow FIG. 3 and may use, for example, a (multivariate) normal distribution. This likelihood may be maximized over the parameter PAR.

[0095] In the observations, each time point can be predicted and compared to the observed value, but this is not necessary; for example, a subset of measurements can be predicted. It is also possible to use multiple prediction time steps between two subsequent measured observations. In general, using a longer time range is preferable for improved training stability. The time range used can vary from observation to observation or even from selected observation to observation.

[0096] As a specific example, when using a Bayesian model for the trainable portion of the drift component and the trainable diffusion component, training may involve selecting values ​​for the set of parameters PAR that maximize the marginal likelihood:

number

number

number

number

[0097] When applying the trained model TM during training, training performance can be improved by indirectly sampling the parameters of the Bayesian model (e.g., the Bayesian trainable portion of the drift component and / or the Bayesian diffusion component) based on sampled values ​​of parameter-free random variables. For example,

number

number

[0098] Another option is to use the sampled distributions for the parameters of the Bayesian model as parameter-free random variables.

number

number

number

number

number

[0099] In the case of Bayesian neural networks, to further reduce the variance of the estimators and enhance the training stability, the neural network may be evaluated by using a transfer function of the linear activations rather than sampling the neural network weights. Thus, the statistical parameters of the linear activations of the network's nodes may be determined, and the linear activations may be deterministically calculated based on the statistical parameters and the values ​​of the sampled parameter-free random variables. This avoids using a single sample of the weights that provides the same noise for all observations. Therefore, suboptimal training performance due to suppression of information in the gradient signal may be avoided. This is a variation of the local reparameterization trick known in state-space models for variational inference. Interestingly, this technique is applied here to the setting of SDEs using Bayesian models. As a specific example, the following condition is used:

number

number

number

number

number

number

[0100] Therefore, instead of sampling the weights individually, we use the linear activation

number

[0101] As shown, the optimization Opt may further include the evaluation of a regularization term Reg, 452, to minimize the difference between a model with a trainable drift component and a model without a trainable drift component. This generally improves training stability, penalizes more complex solutions, and improves the generalizability of the model. For example, for a model without a trainable drift component, the SDE

number

[0102] In particular, using the Kullback-Leibler (KL) divergence in the regularization term Reg is a good choice. Mathematically, this Kullback-Leibler divergence is given by 0→T , e.g., a model with a trainable drift component and a prior distribution P 0→T, for example, may represent the divergence between a model that does not include a trainable drift component. Computing the KL divergence of non-Bayesian SDEs is known per se in the context of variational inference. Interestingly, we show that such divergence of non-Bayesian SDEs can be adapted to settings with a Bayesian trainable part of the drift component and / or a trainable diffusion component, in particular, by applying a prior distribution p(θ) to the parameters of these Bayesian models. r ),p(θ f ) and the posterior distribution q(θ r ),q(θ f ) can be shown to be possible by including the divergence between . Specifically, this divergence can be calculated as follows when both models are Bayesian:

number

[0103] In a preferred embodiment, the optimization Opt performs gradient-based training on an objective function that includes the log-likelihood of the likelihood POBSp(yt|xt) and further includes a regularization term REG based on the KL divergence of the Bayesian SDE. In this case, the optimization Opt can be shown to provide PAC learning and therefore provide strong mathematical guarantees for learning ability. As a specific example, the following loss function can be used:

number

number

number

[0104] Here we present a detailed example of an algorithm that can be used to derive training signals for the optimization Opt. In this example, a Bayesian model is used for the trainable part of the drift component and the diffusion component. The algorithm calculates a loss term according to the loss function above. In this example, only one sample is drawn per observation, but the number of samples per observation can also be higher (e.g., at least 5 or at least 10). A set of parameters PAR, e.g., (φ r ,φ f , ψ,γ), the optimizer Opt optimizes a set of parameters given a set of observed trajectories OBS.

number

[0105] Above, we have given some examples of learning by maximum likelihood estimation, but it should be noted that this is not the only possibility. For example, the optimization Opt may learn a set of parameters PAR by posterior inference. This is done by learning p(θ r ,θ f ,H,,X|Y) and integrate H and X, then approximate the posterior distribution p(θ r ,θ f |Y) for prediction. For example:

number

[0106] 5 illustrates the above example where system 200 is shown to be a control system for a (semi-)autonomous vehicle 62 operating in an environment 082. The vehicle 62 may be autonomous in that it may include an autonomous driving system or a driver assistance system. The latter systems are also referred to as semi-autonomous systems. For example, the vehicle 62 may incorporate system 200, for example, in its engine control unit, for control of the operation of a gasoline particulate filter 042 based on sensor measurements of the vehicle 62 and / or its environment 082, for example, via sensors 075.

[0107] For example, the system may predict the soot mass of the particulate filter 042 and initiate an oxygen burst or other type of cleaning operation if the soot mass is predicted to exceed a given threshold. Soot mass is an example of a measurement that is typically only coarsely observed or not observed at all during use. Thus, in soot mass prediction, intermediate physical quantities may be continuously predicted according to the SDE (e.g., using an identity observation model), and the time evolution of soot mass may be predicted using physical calculations on these intermediate quantities as a second observation model. Because the SDE is simultaneously trained with both the finely observed intermediate values ​​and the coarsely observed soot mass, it can provide a continuous prediction of soot mass supported by infrequent measurements.

[0108] Implementation options for various aspects of the model will now be discussed on the basis of some concrete examples. In the first example, predictions are made for the following Lotka-Volterra system: dx t =(θ1x t -θ2x t y t )dt+0.2dβ t dy t =(-θ3y t +θ4x t y t )dt+0.3dβ t Here, θ=(2.0,1.0,4.0,1.0). In this example, the SDE and the set of observed variables are matched, and no trainable observation model is used. In this example, the trajectory can be predicted, for example, in the interval t = [0, 1] with a resolution dt = 0.01. We have performed experiments in which a predefined drift component portion is provided in the form of a PDE with parameters θ' sampled from a normal distribution centered on the true value θ with a standard deviation of 0.5.

[0109] Specifically, in the experiment, 10 5 The Euler-Maruyama step is performed with a time step size of 10. -4 The Bayesian neural network has a four-layer net with 50 neurons per layer and a ReLU activation function as the drift function. The predefined part of the drift component is determined by the true parameters (θ~N(θ,σ)). 2It is constructed by sampling from a normal distribution centered at I4). The model is trained for 50 epochs with the Adam optimizer and a learning rate of 1e-3.

[0110] By including this relevant but inaccurate prior knowledge, we observed significant improvements. Performing PAC learning using KL-based divergence terms further improved performance.

[0111] In a second example, a computer-controlled system is modeled by a Lorenz attractor (e.g., the computer-controlled system may include a laser, an electronic circuit, etc.). Interestingly, even in the deterministic form of this computer-controlled system, divergences are often observed in existing ODE solvers due to its chaotic nature, and therefore it has essentially unsolvable dynamics. The system dynamics can be described as follows: dx t =ζ(y t -x t )+dβ t dy t =x t (κ-z t )-y t +dβ t dz t =(x t y t -ρzt)+dβ t In this example, the SDE and the set of observed variables are matched, and no trainable observation model is used. In the experiment, ζ = 10, κ = 2.67, ρ = 28, and β tis a random variable following Brownian motion with unit diffusion. In the experiment, 2000 observations from the above dynamics were determined starting the system with (x(0) = 1, y(0) = 1, z(0) = 28), where the first half was for training and the rest was for testing. As a predefined part of the drift component, the parameters were provided in one of the three equations governing the system distorted by Gaussian noise:

[0112] For details, see 20 7 Euler-Maruyama steps, 10 -4 The dataset was run within a time step size of 0.001 and downsampled by a factor of 0.01. This resulted in a sequence of 2000 observations with a frequency of 0.01. The first half of the dataset was split into 20 sequences of length 50 and used for training, and the second half was split into 10 sequences of length 100 and used for testing. For all model variants, the Adam optimizer was used with a learning rate of 0.001, a mini-batch size of 2, a drift net with two hidden layers of 100 neurons each, and a diffusion net with 100 neurons. The models were trained for 100 epochs.

[0113] Despite the inaccuracy of this prior knowledge, a significant performance improvement was observed from providing this prior knowledge. Further performance improvements were achieved by implementing PAC learning with KL-based divergence terms.

[0114] As a third example, the CMU walking dataset was used. In this experiment, a model equipped with a Bayesian neural network and PAC learning was first trained on dataset MOCAP-1, which consisted of 43 motion capture sequences measured from 43 different subjects. The learned drift component of the trained model was then used as a predefined drift component to represent vague prior knowledge based on human walking dynamics. Then, dataset MOCAP-2, with 23 walking sequences from subject 35, was used to represent a high-fidelity, subject-specific modeling task.

[0115] This model is 10 -3 The autoencoder is trained with the Adam optimizer for 3000 epochs on seven randomly selected snippets at a time with a learning rate of . The snippet length is 30 for the first 1000 epochs, 50 up to epoch 2500, and 100 thereafter. As an observation model, the autoencoder is trained with a small additive noise σ to project the 50 observation variables onto six SDE variables. 2 =10 -8 Separate neural nets are used for the drift and diffusion terms, each with one hidden layer of 30 neurons.

[0116] Again, providing domain knowledge improved performance, with the best performance achieved using PAC learning in a Bayesian neural network.

[0117] 6 shows a block diagram of a computer-implemented method 600 for training a model for time series forecasting of a computerized control system based on a set of observations of the computerized control system. The model may use a stochastic differential equation (SDE) of a set of SDE variables. The SDE may include a drift component and a diffusion component. The method 600 may correspond to the operation of the system 100 of FIG. 1; however, this does not limit the method 600 from being performed using other systems, apparatus, or devices.

[0118] The method 600 may include, in an operation titled "Accessing Trainable Drift Parameters," accessing 610 model data defining a model, where the model data includes a set of parameters for at least a trainable portion of the drift component of the SDE.

[0119] The method 600 may include, in the operation entitled "Receive Predefined Drift," receiving 620 a predefined portion of the drift component of the SDE as input to training.

[0120] The method 600 may include a step 630 of acquiring a set of observations of the computerized control system in an operation titled “Acquiring Observations.” The observations may include time-series measurements of the computerized control system and / or its environment.

[0121] The method 600 may include step 640 of training a model based on a set of observations in an operation titled "Training a Model." Training the model may include using the model to perform time series predictions, as part of which, in operation titled "Predicting SDE Variables," the SDE may be used to predict 650 values ​​of a set of SDE variables at a current time based on values ​​of the set of SDE variables at a prior time. Operation 650 may include step 651 of evaluating a predefined portion of a drift component to obtain a first drift in an operation titled "Evaluating a Predefined Drift." Operation 650 may further include step 652 of combining the first drift with a second drift obtained by evaluating a trainable portion of the drift component in an operation titled "Combining with a Trainable Drift." Training operation 640 may further include step 660 of deriving a training signal by comparing the time series predictions to observations of a set of observations in an operation titled "Comparing with Observations." The training operation 640 may further include a step 670 of adjusting a set of parameters of at least the trainable portion of the drift component based on the training signal in an operation entitled "Adjusting Trainable Parameters."

[0122] 7 shows a block diagram of a computer-implemented method 700 for using a trained model to perform time series predictions for a computer-controlled system. Optionally, the predictions are made to control and / or monitor the computer-controlled system. The model may use a stochastic differential equation (SDE) of a set of SDE variables. The SDE may include a drift component and a diffusion component. The method 700 may correspond to the operation of system 200 of FIG. 2; however, this does not limit the method 700 from being performed using other systems, apparatus, or devices.

[0123] The method 700 may include, in an operation titled "Accessing Trained Drift Parameters," accessing 710 model data defining a model, the model data including a set of parameters for at least a trained portion of the drift component of the SDE. The model data may further define a predefined portion of the drift component of the SDE.

[0124] The method 700 may include, in an operation entitled "Acquiring Sensor Data," acquiring 720 sensor data representing measurements of the computerized control system and / or its environment.

[0125] The method 700 may include determining 730 a time series forecast for the computerized control system based on the measurements in an operation titled "Determining a Time Series Forecast." Operation 730 may include using the SDE to predict 750 a value for a set of SDE variables at a current time based on a value for the set of SDE variables at a prior time in an operation titled "Predicting SDE Variables." Operation 750 may include evaluating 751 a predefined portion of a drift component to obtain a first drift in an operation titled "Evaluating a Predefined Drift." Operation 750 may further include combining 752 the first drift with a second drift obtained by evaluating a trainable portion of the drift component in an operation titled "Combining with a Trainable Drift."

[0126] The method 700 may further include step 760, in an operation titled "Generating Control / Monitoring Data," generating output data for use in controlling and / or monitoring the computer control system based on the determined time series predictions.

[0127] In general, it will be appreciated that the operations of method 600 of Figure 6 and method 700 of Figure 7 may be performed in any suitable order, e.g., sequentially, simultaneously, or a combination thereof, and in some cases, where a particular order is dictated by, e.g., input / output relationships. Some or all of the methods may also be combined, e.g., method 700 for applying a trained model may be applied subsequent to the trained model being trained according to method 600.

[0128] The method may be implemented on a computer as a computer-implemented method, as dedicated hardware, or a combination of both. As also shown in FIG. 8 , instructions for a computer, e.g., executable code, may be stored on a computer-readable medium 800, e.g., in the form of a series of physical machine-readable marks 810 and / or in the form of a series of elements having different electrical, e.g., magnetic, or optical, properties or values. The executable code may be stored in a transitory or non-transitory manner. Examples of computer-readable media include memory devices, optical storage devices, integrated circuits, servers, online software, etc. FIG. 8 illustrates an optical disk 800. Alternatively, the computer-readable medium 800 may include transitory or non-transitory data 810 representing a model for time-series prediction of a computer control system, trained as described herein.

[0129] Examples, embodiments, or any features, whether or not indicated as non-limiting, should not be understood as limiting the invention as set forth in the claims.

[0130] It should be noted that the above-described embodiments are merely illustrative and not limiting of the present invention, and that those skilled in the art can design many alternative embodiments without departing from the scope of the appended claims. In the claims, reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" preceding an element does not exclude the presence of a plurality of such elements. The use of phrases such as "at least one" preceding a list or group of elements denotes the selection of all or any subset of the elements from that list or group. For example, the phrase "at least one of A, B, and C" should be understood to include A only, B only, C only, both A and B, both A and C, both B and C, or all of A, B, and C. The present invention may be implemented by means of hardware comprising several distinct elements, or by means of a suitably programmed computer. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. 1. A computer-implemented method (600) for training a model for time series prediction of a computerized control system based on a set of observations of the computerized control system, comprising: The model uses a stochastic differential equation (SDE) for a set of SDE variables, The SDE includes a drift component and a diffusion component, The method comprises: - accessing (610) model data defining a model, said model data including at least a set of parameters of a trainable part of said drift component of said SDE; - obtaining (630) a set of observations of a computerized control system, said observations comprising time-series measurements of the computerized control system and / or its environment; - training (640) a model based on a set of observations; Including, The training step (640) comprises: - using a model to perform time series forecasting, comprising using said SDE, for a step (650) of predicting values ​​of a set of SDE variables at a current time point based on values ​​of the set of SDE variables at a previous time point; - deriving (660) a training signal by comparing said time series forecast with observations of a set of observations; - adjusting (670) at least a set of parameters of the trainable part of said drift component based on said training signal; Including, Further, the method includes receiving (620) a predefined portion of the drift component of the SDE as input for training; The predicting step (650) comprises: evaluating a predefined portion of the drift components to obtain a first drift (651); combining (652) the first drift with a second drift obtained by evaluating the trainable portion of the drift component; Including, The method (600), wherein the step of combining the first drift and the second drift includes combining values ​​of the first drift and corresponding values ​​of the second drift according to weight values, the weight values ​​indicating weights of the predefined portions of the drift components in the combination, and the step of training further includes adjusting the weight values ​​based on the training signal.

2. The method (600) of claim 1, wherein the trainable portion of the drift component and the diffusion component are Bayesian models.

3. 3. The method (600) of claim 2, wherein the step of deriving the training signal comprises using a noise model to determine the likelihood that measurements of selected observations are observed according to a determined time series forecast, and using the likelihood for maximum likelihood estimation.

4. 4. The method (600) of claim 3, wherein evaluating the trainable portion of the drift component comprises sampling values ​​of parameter-free random variables, determining statistical parameters of linear activations of nodes in the trainable portion, and deterministically computing the linear activations based on the statistical parameters and the sampled values ​​of the random variables.

5. 5. The method (600) of claim 3 or 4, wherein the step of deriving the training signal comprises evaluating a regularization term to minimize the difference between a model that includes a trainable drift component and a model that does not include a trainable drift component.

6. 6. The method (600) of any one of claims 1 to 5, wherein the method comprises, prior to the training of the model, determining the predefined portion of the drift component by fitting a parameterized differential equation to a set of observations.

7. 1. A computer-implemented method (700) of using a trained model to perform time series predictions for a computerized control system for controlling and / or monitoring the computerized control system, the model using a stochastic differential equation (SDE) of a set of SDE variables, the SDE including a drift component and a diffusion component; The method comprises: - accessing (710) model data defining said model, said model data including a set of parameters for at least a trained portion of said drift component of said SDE; - acquiring (720) sensor data representative of measurements of said computer-controlled system and / or its environment; - determining (730) a time series forecast for said computer-controlled system based on said measurements, including using said SDEs for a step (750) of predicting values ​​of a set of SDE variables at a current time based on values ​​of said set of SDE variables at a previous time; - generating (760) output data for use in controlling and / or monitoring said computer-controlled system based on the determined time series forecast; Including, the model data further defines a predefined portion of the drift component of the SDE; The step of predicting (750) the values ​​of the set of SDE variables at the current time includes: evaluating (751) the predefined portion of the drift component to obtain a first drift; combining (752) the first drift with a second drift obtained by evaluating the trainable portion of the drift component; Including, The method (700), wherein the step of combining the first drift and the second drift includes combining values ​​of the first drift and corresponding values ​​of the second drift according to weight values, the weight values ​​indicating weights of the predefined portions of the drift components in the combination.

8. 10. The method (700) of claim 7, further comprising using an output interface to an output device used to control or monitor the computerized control system, and providing the output data to the output device to enable control or monitoring of the computerized control system based on the determined time series forecast.

9. 9. The method (700) according to claim 7 or 8, wherein the time series prediction is a time series prediction of a mechanical part of a vehicle, for example an exhaust gas treatment part and / or a stack and / or a particulate filter of a hydrogen fuel cell.

10. 10. The method (700) of claim 7, wherein the trained model is an environmental model that provides a time series prediction of an environment of the computerized control system, the method including training the computerized control system based on generating, as output data, control data for performing simulated interactions with the environment according to the environmental model.

11. 11. The method (700) of any one of claims 7 to 10, wherein the trainable portion of the drift component and the diffusion component are Bayesian models, the method further comprising determining uncertainty in a time series forecast.

12. 1. A system (100) for training a model for time series forecasting of a computerized control system based on a set of observations of the computerized control system, the model using a stochastic differential equation (SDE) of a set of SDE variables, the SDE including a drift component and a diffusion component; The system comprises: a data interface (120) for accessing model data (040) defining said model and comprising a set of parameters of at least the trainable part of the drift component of said SDE; a processor subsystem (140); Including, The processor subsystem (140) - obtaining a set of observations of the computerized system, the observations comprising time series measurements of the computerized system and / or its environment; - configured to train the model based on the set of observations; The training includes: - using the model to perform time series forecasting, comprising using the SDE to predict values ​​of a set of SDE variables at a current time based on values ​​of the set of SDE variables at a previous time; - deriving a training signal by comparing said time series forecast with observations of a set of observations; - adjusting at least a set of parameters of at least the trainable part of said drift component based on said training signal; Including, The processor subsystem (140) is further configured to receive a predefined portion of the drift component of the SDE as an input for training; Predicting values ​​of the set of SDE variables at the current time may include: evaluating a predefined portion of the drift components to obtain a first drift; combining the first drift with a second drift obtained by evaluating the trainable portion of the drift component; Including, The system (100) further includes: combining a value of the first drift and a corresponding value of the second drift according to a weight value; the weight value indicating a weight of the predefined portion of the drift component in the combination; and adjusting the weight value based on the training signal.

13. 1. A system (200) for using a trained model for time series prediction of a computerized control system to control and / or monitor the computerized control system, the model using a stochastic differential equation (SDE) of a set of SDE variables, the SDE including a drift component and a diffusion component; The system comprises: a data interface (220) for accessing model data (040) defining said model and comprising a set of parameters of at least the trained part of the drift component of said SDE; a processor subsystem (240); Including, The processor subsystem (240) - acquiring sensor data representing measurements of said computer-controlled system and / or its environment; determining a time series forecast for said computer-controlled system based on said measurements, and using said SDEs to predict values ​​of a set of SDE variables at a current time based on values ​​of the set of SDE variables at a previous time; - generating output data for use in controlling and / or monitoring said computer-controlled system based on the determined time series forecasts It is configured as follows: the model data further defines a predefined portion of the drift component of the SDE; Predicting values ​​of the set of SDE variables at the current time may include: evaluating a predefined portion of the drift components to obtain a first drift; combining the first drift with a second drift obtained by evaluating the trainable portion of the drift component; Including, The system (200) further includes combining values ​​of the first drift and the second drift according to weight values, the weight values ​​indicating weights of the predefined portions of the drift components in the combination.

14. A computer-readable medium (700) containing transitory or non-transitory data, comprising: The data is - instructions which, when executed by a processor system, cause said processor system to perform the computer-implemented method of any one of claims 1 to 6; - instructions which, when executed by a processor system, cause said processor system to perform the computer-implemented method of any one of claims 7 to 11; - a trained model for performing time series predictions for computer control systems; The model uses a stochastic differential equation (SDE) of a set of SDE variables, the SDE including a drift component and a diffusion component, and the model is configured to perform time series predictions for the computer-controlled system based on measurements by using the SDE to predict values ​​of a set of SDE variables at a current time based on values ​​of the set of SDE variables at a previous time, the model including evaluating a predefined portion of the drift component to obtain a first drift and combining the first drift with a second drift obtained by evaluating the trainable portion of the drift component. A computer-readable medium (700).

Images