Method and device for operating a technical system - Patents.com

Abstract

The present invention relates to an apparatus and computer-implemented method for machine learning using time series data representative of observations related to a technical system (102). The method includes providing (200) the time series data, model parameters for a distribution on the time series data and on a first latent variable and on a second latent variable, and variational parameters of an approximated distribution on the second latent variable, sampling (204-1) values ​​of the second latent variable from the approximated distribution on the second latent variable, and finding (204-2) values ​​of the first latent variable that are responsive to the density of the distribution on the time series data and on the first latent variable and on the values ​​of the second latent variable, in particular, that maximizes the density of the distribution on the time series data and on the first latent variable and on the values ​​of the second latent variable, and determining a Hessian responsive to a second order Taylor approximation of the distribution on the time series data and on the first latent variable and on the values ​​of the first latent variable evaluated at the values ​​of the first latent variable. determining a Laplace approximation of the distribution over the time series data adjusted with values ​​of the second latent variable in response to the Hessian determinant (204-5); determining the inverse Hessian (204-6); determining a Jacobian of the distribution over the time series data and the values ​​of the first latent variable and the second latent variable (204-7); evaluating an approximation lower bound dependent on the determined Laplace approximation for a plurality of values ​​of the second latent variable (202-1); determining a gradient of the Laplace approximation in response to the inverse Hessian and the Jacobian (202-2); and updating the model parameters and the variational parameters in response to the gradient (202-3).

Description

[Technical field]

[0001] The present invention relates to a method and apparatus for operating a technical system. [Background technology]

[0002] background Gaussian process state-space models use Gaussian processes as transition functions in state-space models to describe time series data in a fully probabilistic manner. These models have two types of latent variables: the time states, which are needed to model noisy continuous observations, and the so-called induced outputs, which are needed to efficiently process the Gaussian process part of the model.

[0003] "Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models," by Ialongo, Alessandro Davide, Mark Van Der Wilk, James Hensman and Carl Edward Rasmussen, International Conference on Machine Learning, 2019, discloses the use of variational inference to approximate the true posterior of Gaussian process models. The conditional dependence of the time states on the induction output is considered and a Markovian Gaussian model over the time states is assumed. The Markovian Gaussian model is parametric and allows for nonlinear transitions.

[0004] "Skaug, Hans Julius and David A. Fournier, "Automatic approximation of the marginal likelihood in non-Gaussian hierarchical models," Comput. Stat. Data Anal. 51, pp. 699-709, 2006, discloses the application of the Laplace approximation to general, i.e., non-Gaussian state-space models, in an efficient manner. This is possible by using the implicit function theorem and exploiting the sparsity and structure of the Hessian, i.e., the matrix required to apply the Laplace approximation. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Ialongo, Alessandro Davide, Mark Van Der Wilk, James Hensman, and Carl Edward Rasmussen, “Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models.”, International Conference on Machine Learning.2019 [Non-Patent Document 2] Skaug, Hans Julius and David A. Fournier, “Automatic approximation of the marginal likelihood in non-Gaussian hierarchical models.” Comput.Stat.Data Anal.51, pp. 699-709, 2006. Summary of the Invention [Problem to be solved by the invention]

[0006] Disclosure of the Invention The computer-implemented method and apparatus according to the independent claims provide a combination of models and inference methods by explicitly dealing with two different types of latent variables in a Gaussian process state space model, applying variational inference to the Gaussian process part of the model and applying Laplace approximation to the model's temporal states. The distinction between the two types of latent variables allows for efficient model processing. The method does not require continuous sampling of the temporal states during inference, but instead performs a Laplace approximation with joint optimization over those temporal states. This aspect helps to optimize the model. We further assume that the posterior approximation used in the model can locally linearly approximate the dynamics. The optimization improvement provided by this model also results in well-calibrated uncertainties for different time series forecasting tasks. [Means for solving the problem]

[0007] A computer-implemented method for machine learning using time series data representative of observations related to a technical system, comprising: providing the time series data, model parameters of a distribution on the time series data and on a first latent variable and on a second latent variable, and variational parameters of an approximated distribution on the second latent variable; sampling values ​​of the second latent variable from the approximated distribution on the second latent variable; depending on the density of the distribution on the time series data and on the first latent variable and on the values ​​of the second latent variable, finding a value of the first latent variable that in particular maximizes the density of the distribution on the time series data and on the first latent variable and on the values ​​of the second latent variable; determining a Hessian according to a second order Taylor approximation of the distribution over the values ​​of the second latent variable evaluated at the values ​​of , determining a determinant of the Hessian, determining a Laplace approximation of the distribution over the time series data adjusted with the values ​​of the second latent variable according to the determinant of the Hessian, determining the inverse of the Hessian, determining a Jacobian of the distribution over the time series data and the values ​​of the first latent variable and the second latent variable, evaluating an approximation lower bound dependent on the determined Laplace approximation for multiple values ​​of the second latent variable, determining a gradient of the Laplace approximation according to the inverse Hessian and the Jacobian, and updating the model parameters and the variational parameters according to the gradient. The method uses a distinction between two types of latent variables, uses a posteriori approximation, and assumes that the dynamics can be locally linearly approximated. The method provides an improved way of performing inference in Gaussian process state space models. The method has the following advantages: the method does not require continuous sampling of temporal states during inference, but instead performs a Laplace approximation with joint optimization over those temporal states.

[0008] Preferably, providing the time series data comprises receiving the time series data or receiving a sensor signal comprising information about the technical system and determining the time series data in response to the sensor signal.

[0009] The method preferably comprises determining instructions for activating the technical system in response to the time series data, the model parameters and the variational parameters, and outputting the instructions for operating the technical system.

[0010] Preferably, the technical system is a computer controlled device such as a robot, in particular a vehicle, a household appliance, a power tool, a manufacturing device, a personal assistant or an access control system.

[0011] The technical system may include an engine or part thereof, and the time series data may include speed and / or load as inputs to the technical system and emissions, engine temperature, or oxygen content in the engine as outputs of the technical system.

[0012] The technical system may include a fuel cell stack or a part thereof, and the time series data includes as inputs to the technical system the current in the fuel cell stack, the hydrogen concentration in the fuel cell stack, the stoichiometric ratio of the anode or cathode of the fuel cell stack, the volumetric flow of the coolant for the fuel cell stack, the anode pressure for the anode of the fuel cell stack, the cathode pressure for the cathode of the fuel cell stack, the coolant inlet temperature for the fuel cell stack, the coolant outlet temperature for the fuel cell stack, the anode dew point temperature for the anode of the fuel cell stack, the cathode dew point temperature for the cathode of the fuel cell stack, and as outputs of the technical system (102) the average cell tension across the cells of the fuel cell stack, the anode pressure drop at the anode of the fuel cell stack, the cathode pressure drop at the cathode of the fuel cell stack, the coolant pressure drop between the coolant inlet and outlet of the fuel cell stack, or the coolant temperature rise between the coolant inlet and outlet of the fuel cell stack.

[0013] The instructions preferably include a target operating mode for the technical system.

[0014] The method may include determining a determinant of the Hessian in response to a factorization that includes a strictly upper triangular portion of the Hessian, a strictly lower triangular portion of the Hessian, and a block diagonal matrix of recursively defined blocks of the matrix, which is a very computationally resource efficient method of determining the Hessian.

[0015] The method may include determining the inverse of the Hessian in response to a factorization that includes a strictly upper triangular portion of the Hessian, a strictly lower triangular portion of the Hessian, and a block diagonal matrix of recursively defined blocks of the matrix, which is a very computationally resource efficient method of determining the inverse of the Hessian.

[0016] Evaluating the approximate lower bound may include sampling with a sample of the second latent variable drawn from an approximate distribution on the second latent variable.

[0017] An apparatus for machine learning using time series data representative of observations related to a technical system comprises at least one processor and at least one memory, the at least one processor configured to execute instructions that, when executed by the at least one processor, cause the apparatus to perform steps in a method for operating a technical system. The apparatus provides advantages corresponding to those provided by the method.

[0018] The device may comprise an interface configured to receive information about a technical system and / or to output instructions for operating the technical system. The device is capable of interacting with the technical system.

[0019] The computer program may include computer readable instructions that, when executed by a computer, cause the computer to perform the steps of a method.

[0020] Further advantageous embodiments become apparent from the following description and the drawings. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 illustrates a schematic diagram of an apparatus for operating a technical system. [Diagram 2] 1A-1D diagrams illustrating steps in a method for operating a technical system; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] FIG. 1 illustrates generally an apparatus 100 for operating a technology system 102 .

[0023] The apparatus 100 comprises at least one processor 104 and at least one memory 106. The at least one processor 104 is configured to execute instructions that, when executed by the at least one processor 104, cause the apparatus 100 to perform steps in a method for operating the technology system 102.

[0024] The device 100 in this example comprises an interface 108. The interface 108 is configured, for example, to receive information related to the technical system 102. The interface 108 is configured, for example, to output instructions for operating the technical system 102. The technical system 102 may comprise an actuator 110. The actuator 110 may be at least temporarily connected to the interface 108 via a signal line 112.

[0025] FIG. 2 illustrates the steps of the method. The method is to y of

number

number

[0026] Time series data Y T For example, σ includes noisy observations from the technical system 102.

[0027] Time series data Y T In addition, the method

number

[0028] The technical system 102 may include an engine or a part thereof. The time series data may include speed and / or load as inputs to the technical system 102 and may include emissions, engine temperature, or oxygen content in the engine as outputs of the technical system 102.

[0029] The technical system 102 may include a fuel cell stack or a part thereof. The time series data may include, as inputs to the technical system 102, the current in the fuel cell stack, the hydrogen concentration in the fuel cell stack, the stoichiometric ratio of the anode or cathode of the fuel cell stack, the volumetric flow of the coolant for the fuel cell stack, the anode pressure for the anode of the fuel cell stack, the cathode pressure for the cathode of the fuel cell stack, the coolant inlet temperature for the fuel cell stack, the coolant outlet temperature for the fuel cell stack, the anode dew point temperature for the anode of the fuel cell stack, the cathode dew point temperature for the cathode of the fuel cell stack, and, as outputs of the technical system 102, the average cell tension across the cells of the fuel cell stack, the anode pressure drop at the anode of the fuel cell stack, the cathode pressure drop at the cathode of the fuel cell stack, the coolant pressure drop between the coolant inlet and outlet of the fuel cell stack, or the coolant temperature rise between the coolant inlet and outlet of the fuel cell stack.

[0030] The method involves computing the time series data Y for a given number I of iterations i and a given number N of samples n.T Operates on.

[0031] The method is based on a probability model and an approximation model. The approximation model is based on the fully independent training conditional FITC assumption. Details of this assumption are described, for example, in "Sparse Gaussian Processes using Pseudo-inputs" by Edward Snelson and Zoubin Ghahramani, In Advances in Neural Information Processing Systems, 2005.

[0032] The probabilistic model is a Gaussian process prior distribution on the latent state x t-1 to the next latent state x t It is based on a Gaussian process state space model that is put on the mean of a transition model to learn a mapping to

number

number

number

number

[0033] The method

number

number

[0034] The probabilistic model includes a first latent variable, which in this example is a function of a given dimension d x of

number

number

[0035] A posterior Gaussian process is a process where the posterior information is given by a pseudodataset (X M , F M ), which can be summarized by a sparse Gaussian process in X M is the inductive input, F M is the inductive output.

[0036] Inductive output F M and F T is a joint Gaussian distribution p Θ (F T ,F M ) The model uses a fully independent training conditional approximation that assumes independence of the latent GP evaluations given the induced outputs.

number

number

number

[0037] Inductive output F M is the second latent variable.

[0038] The method

number

number

[0039] The approximation model is the time series data Y T and the first latent variable, e.g., the temporal state

number

[0040] Variational inference is used to calculate the log marginal likelihood logp Θ (Y T ) as the lower limit, the induced output p Θ (F M |Y T ) it is possible to find an approximation to the true posterior.

number

[0041] The approximation model is a distribution approximation on the second latent variable, e.g., an induced output F with mean m and variance S. M The variational distribution q on Ψ (FM )=N(F M │m,S) for example given the initial variational parameters Ψ={m,S}.

[0042]

number

number

[0043] Inductive Input X M and induced output F M are referred to as pseudo data points. The prior distribution for the guided output is given by a Gaussian process prior: p(F M )=N(F M |0,K MM ),

number

[0044] The approximate model is the predictive distribution p(x t |x t-1 ,F M )=N(x t │x t-1 +μ(x t-1 ,F M ),Σ(x t-1 ) + Q), the average is

number

number

number

[0045] The method includes step 200 .

[0046] Step 200 is the time series data Y T Providing time series data T may be provided separately and used.

[0047] The method includes inputting time series data Y T The method may include receiving a

[0048] Step 200 includes receiving a sensor signal containing information about the technical system 102 and generating time series data Y in response to the sensor signal. T and determining the time series data U T may further be received or determined from the received sensor signals. t For example, the latent state x t and used as input to the transition model and kernel function.

[0049] Step 200 is the induction output F M Time series data Y subject to T and latent state

number

number

number

[0050] The method operates with given initial model parameters Θ and given initial variation parameters ψ={m, S}.

[0051] The method includes an outer loop 202 and an inner loop 204 .

[0052] The outer loop 202 is processed for iterations i=1,...,I. In the iterations, the model and variational parameters are optimized.

[0053] The inner loop 204 operates on samples n=1,...,N. The samples are used to obtain a probabilistic approximation to the log-likelihood.

[0054] The inner loop 204 includes a step 204-1.

[0055] Step 204-1 involves sampling values ​​of a second latent variable from an approximate distribution on the second latent variable.

[0056] In this example, the induced output sample

number

number

[0057] The inner loop 204 includes a step 204-2.

[0058] Step 204-2 includes finding a value for the first latent variable as a function of the density of the distribution over the time series data and the values ​​of the first latent variable and the second latent variable. In this example, step 204-2 includes finding a value for the first latent variable such that the density of the distribution over the time series data and the values ​​of the first latent variable and the second latent variable is maximized.

[0059] In this example, the mode with the maximum value of the logarithmic density

number

number

number

number

[0060] This is the mode in which the method maximizes the logarithmic density.

number

[0061] The inner loop 204 includes a step 204-3.

[0062] Step 204-3 is the first latent variable

number

number

number

[0063] In this example, the Hessian

number

number

[0064] In this example, the non-zero elements are the quantities

number

number

number

number

[0065] In this example, the nonzero elements of the Hessian are x The memory and time requirements are determined solely by the vector-Hessian product.

number

[0066] The inner loop 204 includes a step 204-4.

[0067] Step 204-4 involves determining the determinant of the Hessian.

[0068] The determinant of the Hessian is determined according to a factorization that includes, for example, a portion of the strictly upper triangular part of the Hessian, a portion of the strictly lower triangular part of the Hessian, and a block diagonal of a recursively defined block of the matrix.

[0069] In this example, the Hessian H(A t ,B t ) determinant detH(A t,B t ) is evaluated.

[0070] In this example, the Hessian H(A t ,B t ) determinant detH(A t ,B t ) is a factorization H(A t ,B t )=(Λ+B T )Λ -1 (Λ+B) is determined from B is a different B t The Hessian H(A t ,B t ), where Λ is a block diagonal matrix of recursively defined blocks,

number

number

number

number

[0071] The inner loop 204 includes a step 204-5.

[0072] Step 204-5 includes determining a Laplace approximation of the distribution over the time series data adjusted with the values ​​of the second latent variable according to the determinant of the Hessian.

[0073] In this example, the conditional

number

number

number

[0074] Inner loop 204 includes step 204-6.

[0075] Step 204-6 involves determining the inverse of the Hessian.

[0076] The inverse of the Hessian may be determined according to a factorization that includes, for example, a portion of the strictly upper triangular part of the Hessian, a portion of the strictly lower triangular part of the Hessian, and a block diagonal matrix of a recursively defined block of the matrix.

[0077] In this example, the inverse of the Hessian H -1 is determined.

[0078] The inverse of the Hessian is determined from the factorization. H -1 =(Λ+B) -1 Λ(Λ+B T ) -1

[0079] Inner loop 204 includes step 204-7.

[0080] Step 204-7 involves determining a Jacobian of the distribution over the values ​​of the time series data and the first latent variable and the second latent variable.

[0081] In this example, the function

number

number

[0082] The outer loop 202 includes a step 202-1.

[0083] Step 202-1 includes evaluating an approximation lower bound that depends on the Laplace approximation determined for multiple values ​​of the second latent variable.

[0084] In this example, the approximate lower bound L(Θ,Ψ)

number

number

number

[0085] In this example, the multiple values ​​of the second latent variable are samples of the second latent variable determined in step 204-1 when repeatedly processing the inner loop, which means that the approximate lower bound is evaluated as a function of samples of the second latent variable drawn from an approximate distribution on the second latent variable.

[0086] In this example, a parametric family is used to estimate and optimize this optimization objective using the approximate distribution q Ψ (F M ) is selected.

[0087] In this example, q Ψ (F M) is Gaussian distributed. This allows analytical evaluation of the KL terms. The other terms in L(Θ,Ψ) are analytically intractable. In this example, the other terms are optimized by sampling.

number

number

number

[0088] The outer loop 202 includes a step 202-2.

[0089] Step 202-2 involves determining the gradient of the Laplace approximation as a function of the inverse Hessian and the Jacobian.

[0090] In this example, the gradient of L(Θ,Ψ)

number

number

number

number

number

[0091] This trades potentially expensive automatic differentiation for a Hessian solution.

number

[0092] The outer loop 202 includes a step 202-3.

[0093] Step 202-3 involves updating the model parameters and the variational parameters as a function of the gradients.

[0094] In this example, the model parameters Θ and the variational parameters ψ={m, S} are updated.

[0095] Updating the model parameters Θ and the variational parameters Ψ={m,S} includes determining the model parameters Θ and the variational parameters Ψ={m,S} that minimize L(Θ,Ψ), which means that the model parameters Θ and the variational parameters Ψ={m,S} are determined if L(Θ,Ψ) is smaller than the other model parameters Θ and the variational parameters Ψ={m,S}.

[0096] The above steps of the method describe an inference method for learning the model parameters Θ and the variational parameters Ψ of a Gaussian process state-space model in training. These steps may be implemented, for example, by applying the Gaussian process state-space model to a given time series data Y T and optionally given additional time series data U T may be determined in an offline stage.

[0097] The following steps of the method may for example be performed for prediction in the online phase. These steps may be performed with the trained model, i.e. with the given model parameters Θ and variational parameters ψ. These steps may be performed independent of the training, i.e. without training or together with post-training training:

[0098] In this example, the model parameters Θ and the variational parameters Ψ determined in the last iteration of updating the model parameters Θ and the variational parameters Ψ are used for prediction.

[0099] The method may include step 206.

[0100] In step 206, the method comprises: T , the model parameters Θ, and the variational parameters ψ={m, S}, determining instructions for activating the technical system 102 according to m, S}.

[0101] Optionally, additional time series data U T may also be used.

[0102] For example, the time series data may include speed and / or load as inputs to an approximation model of the technical system 102. For example, the outputs of the approximation model of the technical system 102 may be emissions, engine temperature, or oxygen content in the engine.

[0103] For example, the time series data includes the current in the fuel cell stack, the hydrogen concentration in the fuel cell stack, the stoichiometric ratio of the anode or cathode of the fuel cell stack, the volumetric flow of the coolant for the fuel cell stack, the anode pressure for the anode of the fuel cell stack, the cathode pressure for the cathode of the fuel cell stack, the coolant inlet temperature for the fuel cell stack, the coolant outlet temperature for the fuel cell stack, the anode dew point temperature for the anode of the fuel cell stack, the cathode dew point temperature for the cathode of the fuel cell stack, etc. For example, the output of the approximation model of the technical system 102 is the average of the cell tension across the cells of the fuel cell stack, the anode pressure drop at the anode of the fuel cell stack, the cathode pressure drop at the cathode of the fuel cell stack, the coolant pressure drop between the coolant inlet and outlet of the fuel cell stack, or the coolant temperature rise between the coolant inlet and outlet of the fuel cell stack.

[0104] The instructions include, for example, a target operating mode for the technical system 102. The target operating mode can be determined as a function of the output of an approximation model, for example by a controller or a characteristic curve or a map that maps the output to the target operating mode.

[0105] The method may include step 208.

[0106] In step 208 , the method includes outputting instructions for operating the technology system 102 .

[0107] The instructions may include, for example, a target operating mode for the technology system 102 .

[0108] Time series data Y T may be processed for training in mini-batches. A mini-batch is a set of training vectors for any time index t 0 Length starting from T b Time series data Y T A subsequence of

number

number

[0109] The method involves the use of one-dimensional or multidimensional latent states x t The same applies to the multidimensional latent state x t In this case, we use independent Gaussian processes to calculate the latent state x t can be used for each dimension of

number

number

Claims

1. 1. A computer-implemented method for machine learning using time series data representative of observations related to a technical system (102), comprising: providing (200) the time series data, model parameters of distributions on the time series data and on a first latent variable and on a second latent variable, and variational parameters of an approximate distribution on the second latent variable; Sampling a value of the second latent variable from the approximate distribution on the second latent variable (204-1); Depending on the density of the distribution over the time series data and over the first latent variable and over the values ​​of the second latent variable, finding a value of the first latent variable that in particular maximizes the density of the distribution over the time series data and over the first latent variable and over the values ​​of the second latent variable (204-2); determining a Hessian in response to a second order Taylor approximation of the distribution over the time series data and the first latent variable and the values ​​of the second latent variable evaluated at the values ​​of the first latent variable (204-3); determining a determinant of the Hessian (204-4); and determining (204-5) a Laplace approximation of a distribution over the time series data adjusted with the value of the second latent variable according to the determinant of the Hessian; determining the inverse of the Hessian (204-6); and determining (204-7) a Jacobian of the distribution on the time series data and the values ​​of the first latent variable and the second latent variable; evaluating an approximation lower bound dependent on the Laplace approximation determined for a plurality of values ​​of the second latent variable (202-1); determining (202-2) a gradient of the Laplace approximation as a function of the inverse Hessian and the Jacobian; updating (202-3) the model parameters and the variational parameters in response to the gradient; A method comprising:

2. Providing the time series data (200) includes: receiving said time series data or receiving a sensor signal comprising information about said technical system (102); determining the time series data in response to the sensor signal; The method of claim 1 , comprising:

3. determining (206) instructions for operating the technical system (102) in response to the time series data, the model parameters and the variational parameters; outputting (208) the instructions for operating the technical system (102); The method according to claim 1 , characterized in that

4. 4. The method according to claim 1, wherein the technical system (102) is a computer-controlled device such as a robot, in particular a vehicle, a household appliance, a power tool, a manufacturing device, a personal assistant or an access control system.

5. said technical system (102) comprising an engine or part thereof; The time series data is Inputs to the technical system (102) include speed and / or load, The output of the technical system (102) includes emissions, the temperature of the engine, or the oxygen content in the engine.

5. The method according to claim 1, wherein the first and second electrodes are connected to a first electrode.

6. said technical system (102) comprising a fuel cell stack or part thereof, The time series data is inputs to said technical system (102) include the current in said fuel cell stack, the hydrogen concentration in said fuel cell stack, the stoichiometric ratio of the anode or cathode of said fuel cell stack, the volumetric flow of coolant for said fuel cell stack, the anode pressure for the anode of said fuel cell stack, the cathode pressure for the cathode of said fuel cell stack, the inlet temperature of the coolant for said fuel cell stack, the outlet temperature of the coolant for said fuel cell stack, the anode dew point temperature for the anode of said fuel cell stack, the cathode dew point temperature for the cathode of said fuel cell stack, The outputs of the technical system (102) include an average cell tension across the cells of the fuel cell stack, an anode pressure drop at the anode of the fuel cell stack, a cathode pressure drop at the cathode of the fuel cell stack, a coolant pressure drop between the coolant inlet and outlet of the fuel cell stack, or a coolant temperature rise between the coolant inlet and outlet of the fuel cell stack, 6. The method according to claim 1, wherein the first and second electrodes are connected to a first electrode.

7. the instructions include a target operating mode for the technical system (102); 7. The method according to claim 1, wherein the first and second electrodes are connected to a first electrode.

8. 8. The method of claim 1, further comprising determining (204-4) the determinant of the Hessian in response to a factorization comprising a strictly upper triangular portion of the Hessian, a strictly lower triangular portion of the Hessian, and a block diagonal of recursively defined blocks of matrices.

9. 9. The method of claim 1, further comprising determining (204-6) the inverse of the Hessian in response to a factorization comprising a strictly upper triangular portion of the Hessian, a strictly lower triangular portion of the Hessian, and a block diagonal of a recursively defined block of matrices.

10. Evaluating the approximation lower bound (202-1) may include: Sampling with a sample of the second latent variable drawn from the approximate distribution on the second latent variable.

10. The method according to claim 1, further comprising:

11. An apparatus (100) for machine learning using time series data representative of observations related to a technical system (102), comprising: The device (100) comprises: At least one processor (104); At least one memory (106); Equipped with The at least one processor (104) is configured to execute instructions that, when executed by the at least one processor (104), cause the apparatus (100) to perform steps in a method for operating the technical system (102).

12. The device (100) comprises: an interface (108) configured to receive information about the technical system (102) and / or to output instructions for operating the technical system (102); The apparatus (100) of claim 11, comprising:

13. A computer program comprising computer readable instructions for causing the computer to carry out the steps of the method according to any one of claims 1 to 12, when the computer program is executed by a computer.

Images