System and method for controlling operation of a device

Abstract

The present disclosure provides a feedback controller and method for controlling operation of a device at different control steps. The feedback controller includes at least one processor and a memory storing instructions that, when executed by the at least one processor, cause the feedback controller to collect, for a control step, measurements indicative of a state of the device at the control step, and recursively executing a probability solver parameterized on a control input of an actuator of the operating device until a termination condition is satisfied to generate a control input for the control step. The feedback controller is further configured to control an actuator of the operating device based on the generated control input.

Description

Technical Field

[0001] The present disclosure relates generally to control systems and, more particularly, to a system and method for controlling the operation of a device at different control steps based on a feedback signal. Background Art

[0002] Optimal control involves seeking control of a dynamic system over a period of time such that an objective function is optimized. It has wide applications in science, engineering, and operations research. For example, the dynamic system may be a spacecraft with controls corresponding to rocket thrusters, and the goal may be to reach the moon with minimal fuel expenditure. Similarly, the dynamic system may be a vehicle with controls corresponding to the acceleration of the vehicle. Model-based control methods such as linear quadratic regulators (LQR) or model predictive control (MPC) use a mathematical model of the dynamic system to determine the actuator commands / inputs. For numerical reasons, such a dynamic system model is simplified to facilitate numerical optimization, i.e., it is a "control-oriented" model. Such a control-oriented model is chosen to be a linear / nonlinear and continuous analytical function and is therefore suitable for gradient-based optimization.

[0003] As control applications become more complex and computational resources become more powerful, there is an increasing interest in and ability to use more advanced controllers to handle complex behaviors. For example, accurately controlling a dynamic system may require taking into account behaviors that are difficult to model analytically as explicit functions. These behaviors include contact dynamics, friction, inertia of complex shapes, flexible bodies such as in soft robotics, implicit differential equations, etc. The formulation of the optimal control problem taking into account complex dynamics presents many computational challenges that are difficult to address in real-time control applications. For example, dynamic programming for seeking the optimal solution to an optimal control problem may fail when the dynamic model includes discontinuous functions.

[0004] Therefore, there is a need for a system and method for solving optimal control problems that is suitable for control systems with complex dynamics. Summary of the invention

[0005] It is an object of some embodiments to provide a system and method suitable for feedback control of systems with complex dynamics. Additionally or alternatively, it is an object of some embodiments to provide feedback control for systems / devices with dynamics of unknown analytical form. Examples of such devices include legged robots, robots or systems made partially of compliant materials rather than rigid links, electric motors, etc. In practice, friction in the operation of many robotic systems may be difficult to capture in analytical form.

[0006] Some embodiments are based on the understanding that to achieve such a goal, alternative approaches to solving the optimal control problem are needed that differ from searching for analytical solutions and / or iterative optimization, such as gradient-based optimization.

[0007] An example of this alternative approach for solving complex optimization problems is based on the principles of training deep neural networks (DNNs) in the new technology field of physical information neural networks (PINNs). DNNs are trained by minimizing a loss function to build a model based on training data in order to make predictions or decisions without explicit planning to do so. Therefore, the use of DNNs is usually divided into two phases, namely, a training phase in which the parameters of the DNN are learned and a testing phase in which the trained DNN is used to make decisions. Therefore, the output generated by the DNN during the training phase is usually ignored because the output is only used to train the parameters of the DNN.

[0008] However, training a DNN to minimize a loss function can be viewed as finding a solution to the loss function, or specifically, finding DNN parameters that force the trained DNN to output a solution to the loss function given an input. Some embodiments are based on the understanding that the loss function can be formulated as an optimal control problem. In this regard, training a neural network using a loss function can be viewed as equivalent to solving an optimal control problem. For example, the output of a trained neural network can be the control input of a controlled device for a current control step. The input to the neural network can include a feedback signal indicating the state of the controlled device at the current control step. Therefore, the neural network needs to be retrained for each control step.

[0009] However, while this example provides an alternative approach for solving the optimal control problem, formulating the optimal control problem as training a DNN suffers from slow convergence, making it less suitable for real-time control applications. It is an objective of some embodiments to find different alternative approaches for solving the optimal control problem.

[0010] Some embodiments are based on the recognition that the reason why DNN training generated to solve the optimal control problem converges slowly is due to the deterministic nature of the DNN's output. Although the DNN can be interpreted as a multivariate statistical model for approximating an unknown desired function, the output generated by the output layer of the DNN is deterministic and finite, and does not require further modification. In addition, another reason for the slow convergence can be found in the deterministic update of the parameters of the DNN. This determinism slows down the back-propagation training of the DNN in searching for the DNN parameters that minimize the loss function.

[0011] Some embodiments are based on the recognition that the estimation of the solution of the optimal control problem and the iterative search for such a solution should be probabilistic. The probabilistic nature of the estimation of the solution and the probabilistic update of the current solution can increase the convergence of the estimated solution towards the optimal solution because it allows the solution space with possible optimal solutions to be explored.

[0012] To this end, some embodiments aim to find a probabilistic framework that allows probabilistically searching for control inputs for a control device provided by a solution to an optimal control problem. In addition, some embodiments aim to find a probabilistic framework that probabilistically updates the current probability estimate of the control input until a termination condition is met.

[0013] An example of such a probabilistic framework explored by various embodiments is a probabilistic solver, such as a Kalman filter. A probabilistic solver uses a series of measurements observed over time, including statistical noise and other inaccuracies, and generates estimates of unknown variables that are often more accurate than those based on a single measurement alone. A probabilistic solver is used to track the state of a controlled device. In other words, a Kalman filter is parameterized on the state of the controlled device. For example, a Kalman filter can be used to estimate the state of a vehicle from measurements of satellite signals as in GNSS applications. In this example, the Kalman filter is parameterized on the state of the vehicle.

[0014] The operational framework of the probabilistic solver includes two phases, namely, a prediction phase and a correction phase. For the prediction phase, the probabilistic solver generates an estimate of the current state variable and its uncertainty. To this end, the probabilistic solver adopts a prediction model that is subject to process noise. An example of a prediction model is a motion model of the controlled device, such as a motion model of a vehicle. Once the result of the next measurement is observed (which inevitably has some errors, including random noise), the estimate is updated using a weighted average of the measurements, where measurements with greater certainty are given more weight. To this end, the probabilistic solver adopts a measurement model that is subject to measurement noise. The measurement model links the measurement to the state estimate. For example, the measurement model links the GNSS measurement to the state of the vehicle.

[0015] Both process noise and measurement noise can be represented by a probability density function (PDF) showing the likelihood of possible changes to the predicted state and / or received measurements. The probabilistic solver is recursive and can operate in real time using only the current input measurements and the previously estimated state and its uncertainty matrix; additional past information is not required, but can be used if desired.

[0016] Some embodiments are based on the principle that a Kalman filter can provide a framework for searching for solutions to an optimal control problem in a probabilistic manner. In practice, instead of having the Kalman filter parameterized on state variables (e.g., the state of the vehicle), the Kalman filter can be parameterized on control inputs (e.g., acceleration values ​​of the vehicle). Therefore, the prediction model of the Kalman filter should predict the values ​​of the control inputs subject to process noise to generate a PDF of the predicted values ​​of the control inputs. An example of such a prediction model is the identity model. Another example is to make such predictions based on a control-oriented model.

[0017] The measurement model of this modified Kalman filter parameterized on the control input should relate the measurement to the current estimate of the control input. Furthermore, this relationship should indicate the mismatch between the controlled input derived from the measurement and the estimated control input in the context of the solution of the optimal control problem.

[0018] Some embodiments are based on the recognition that a cost function of an optimal control problem can be evaluated by simulating the operation of a device using predicted control inputs and current measurements of the device operation. For example, the simulation can be performed based on a digital twin of the controlled device to estimate a performance metric of the device operation under the evaluated control input. The simulation results are evaluated for the cost function and mapped in a control space subject to measurement noise. To this end, the measurement model of some embodiments relates the performance metric of the device operation to the control input subject to measurement noise to estimate the PDF of the measured value of the control input.

[0019] In addition, the correction step of the Kalman filter updates the PDF of the predicted value of the control input based on the PDF of the measured value of the control input to generate the PDF of the value of the control input for subsequent iterations. In this way, the control input is iteratively estimated and updated in a probabilistic manner. The iteration is repeated multiple times until each control step meets the termination condition.

[0020] Thus, one embodiment discloses a feedback controller for controlling operation of a device at different control steps based on a feedback signal, the feedback signal including measurements indicating the device state at the different control steps. The feedback controller includes at least one processor; and a memory storing instructions, the instructions when executed by the at least one processor, causes the feedback controller to: collect measurements indicating the device state at the control step; and recursively execute a probability solver parameterized on a control input of an actuator operating the device until a termination condition is satisfied to generate a control input for the control step, wherein during each execution, the probability solver is configured to estimate a PDF of a predicted value of the control input from a probability distribution function (PDF) of the value of the control input using a prediction model; evaluate a cost function of an optimal control problem for controlling the device based on simulations of the device operation using the measurements and values ​​of the control input sampled from the PDF of the predicted value of the control input to generate a performance measure of the device operation; and estimate a PDF of a simulated value associated with the control input based on a measurement model that associates the performance measure of the device operation with the control input; and correct the PDF of the predicted value of the control input based on the PDF of the simulated value associated with the control input to generate a PDF of the value of the control input. The feedback controller is further configured to control an actuator of the operating device using at least a mean value of the PDF of the values ​​of the control input.

[0021] Thus, another embodiment discloses a method for controlling the operation of a device at different control steps based on a feedback signal, the feedback signal including a measurement indicating the state of the device at the different control steps. The method includes the steps of collecting measurements indicating the state of the device at the control steps; and recursively executing a probability solver parameterized on a control input of an actuator operating the device until a termination condition is satisfied to generate a control input for the control step. During each execution, the probability solver is configured to estimate a PDF of a predicted value of the control input from a probability distribution function (PDF) of the value of the control input using a prediction model; evaluate a cost function of an optimal control problem for controlling the device based on at least one simulation of the device operation using the measurement and the value of the control input sampled from the PDF of the predicted value of the control input to generate a performance measure of the device operation; and estimate a PDF of a simulated value associated with the control input based on a measurement model that relates the performance measure of the device operation to the control input; and correct the PDF of the predicted value of the control input based on the PDF of the simulated value associated with the control input to generate a PDF of the value of the control input. The method also includes the steps of controlling the actuator operating the device using at least a mean of the PDF of the value of the control input.

[0022] Thus, another embodiment discloses a non-transitory computer-readable storage medium embodied with a program executable by a processor for performing a method for controlling operation of a device at different control steps based on a feedback signal, the feedback signal including a measurement indicating a device state at the different control steps. The method includes the steps of: collecting measurements indicating a device state at the control step; and recursively executing a probability solver parameterized on a control input of an actuator operating the device until a termination condition is satisfied to generate a control input for the control step. During each execution, the probability solver is configured to estimate a PDF of a predicted value of the control input from a probability distribution function (PDF) of the value of the control input using a prediction model; evaluate a cost function of an optimal control problem for controlling the device based on a simulation of the device operation using the measurement and the value of the control input sampled from the PDF of the predicted value of the control input to generate a performance metric for the device operation; and estimate a PDF of a simulated value associated with the control input based on a measurement model that relates the performance metric for the device operation to the control input; and correct the PDF of the predicted value of the control input based on the PDF of the simulated value associated with the control input to generate a PDF of the value of the control input. The method further comprises the step of controlling an actuator of the operating device using at least a mean value of the PDF of the values ​​of the control input.

[0023] Embodiments of the present disclosure will be further described with reference to the accompanying drawings. The drawings shown are not necessarily to scale, emphasis generally being placed upon illustrating the principles of embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] [ Figure 1 ] Figure 1 A block diagram of a feedback controller for controlling the operation of a device at different control steps according to some embodiments of the present disclosure is illustrated.

[0025] [ Figure 2 ] Figure 2 A schematic diagram illustrating steps performed by a probabilistic solver for generating control inputs according to some embodiments of the present disclosure is shown.

[0026] [ Figure 3 ] Figure 3 An example of a probability distribution function (PDF) according to some embodiments of the present disclosure is illustrated.

[0027] [ Figure 4A ] Figure 4A The process of how to use the distribution of control inputs to determine a simulation test for a simulation step according to some embodiments of the present disclosure is illustrated.

[0028] [ Figure 4B ] Figure 4B The process of how to use the distribution of control inputs to determine a simulation test for a simulation step according to some embodiments of the present disclosure is illustrated.

[0029] [ Figure 5 ] Figure 5 Gaussian distributions with different variances according to some embodiments of the present disclosure are illustrated.

[0030] [ Figure 6 ] Figure 6 A schematic diagram illustrating correction steps for generating a corrected PDF according to some embodiments of the present disclosure is shown.

[0031] [ Figure 7 ] Figure 7 Example termination conditions according to some embodiments of the present disclosure are illustrated.

[0032] [ Figure 8 ] Figure 8 An implementation of a probability solver using a Kalman filter as an iterative process according to some embodiments of the present disclosure is illustrated.

[0033] [ Fig. 9 ] Fig. 9 An example list showing objectives for a cost function according to some embodiments of the present disclosure.

[0034] [ Fig.10 ] Fig.10 Controlling a device by a feedback controller in an environment according to some embodiments of the present disclosure is illustrated. DETAILED DESCRIPTION

[0035] In the following description, for the purpose of illustration, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other cases, the devices and methods are shown only in block diagram form to avoid obscuring the present disclosure.

[0036] As used in this specification and claims, the terms "for example" and "such as" and the verbs "include," "have," "comprises," and other verb forms thereof, when used in conjunction with a list of one or more components or other items, should each be interpreted as open-ended, meaning that the list should not be viewed as excluding other additional components or items. The term "based on" means based, at least in part, on. In addition, it will be understood that the phraseology and terminology employed herein are for descriptive purposes and should not be viewed as limiting. Any headings utilized within this description are for convenience only and have no legal or limiting effect.

[0037] An object of some embodiments is to provide a system and method suitable for feedback control of systems with complex dynamics. Additionally or alternatively, an object of some embodiments is to provide feedback control for systems / devices with dynamics of unknown analytical form. Some embodiments are based on the understanding that in order to achieve such a goal, alternative methods for solving optimal control problems are needed that are different from searching for analytical solutions and / or iterative optimization, such as different from gradient-based optimization.

[0038] An example of this alternative approach for solving complex optimization problems is based on the principles of training deep neural networks (DNNs) in the new technology field of physical information neural networks (PINNs). DNNs are trained by minimizing a loss function to build a model based on training data in order to make predictions or decisions without explicit planning to do so. Therefore, the use of DNNs is usually divided into two phases, namely, a training phase in which the parameters of the DNN are learned and a testing phase in which the trained DNN is used to make decisions. Therefore, the output generated by the DNN during the training phase is usually ignored because the output is only used to train the parameters of the DNN.

[0039] However, training a DNN to minimize a loss function can be viewed as finding a solution to the loss function, or specifically, finding DNN parameters that force the trained DNN to output a solution to the loss function given an input. Some embodiments are based on the understanding that the loss function can be formulated as an optimal control problem. In this regard, training a neural network using a loss function can be viewed as equivalent to solving an optimal control problem. For example, the output of a trained neural network can be the control input of a controlled device for a current control step. The input to the neural network can include a feedback signal indicating the state of the controlled device at the current control step. Therefore, the neural network needs to be retrained for each control step.

[0040] However, while this example provides an alternative approach for solving the optimal control problem, formulating the optimal control problem as training a DNN suffers from slow convergence, making it less suitable for real-time control applications. It is an objective of some embodiments to find different alternative approaches for solving the optimal control problem.

[0041] Some embodiments are based on the recognition that the reason why DNN training generated to solve the optimal control problem converges slowly is due to the deterministic nature of the DNN's output. In fact, although the DNN can be interpreted as a multivariate statistical model for approximating an unknown desired function, the output generated by the output layer of the DNN is deterministic and finite, and does not require further modification. In addition, another reason for the slow convergence can be found in the deterministic update of the parameters of the DNN. This determinism slows down the back-propagation training of the DNN in searching for the DNN parameters that minimize the loss function.

[0042] Some embodiments are based on the recognition that the estimation of the solution of the optimal control problem and the iterative search for such a solution should be probabilistic. The probabilistic nature of the estimation of the solution and the probabilistic update of the current solution can increase the convergence of the estimated solution towards the optimal solution because it allows the solution space with possible optimal solutions to be explored.

[0043] To this end, some embodiments aim to find a probabilistic framework that allows probabilistically searching for control inputs for a control device provided by a solution to an optimal control problem. In addition, some embodiments aim to find a probabilistic framework that probabilistically updates the current probability estimate of the control input until a termination condition is met.

[0044] An example of such a probabilistic framework explored by various embodiments is a probabilistic solver, such as a Kalman filter. Some embodiments are based on the recognition that a probabilistic solver can be used to control a device with complex dynamics. For example, based on a probabilistic solver, a feedback controller can be formulated such that the feedback controller collects measurements indicating the state of the device at a control step, executes the probabilistic solver to generate a control input for the control step, and controls the device based on the generated control input. Figure 1This probabilistic solver-based feedback controller is illustrated in .

[0045] Figure 1 A block diagram of a feedback controller 100 for controlling the operation of a device 110 at different control steps according to some embodiments of the present disclosure is illustrated. The feedback controller 100 may be operably coupled to the device 110. Examples of the device 110 may include a vehicle (e.g., an autonomous vehicle), a robotic assembly, a legged robot, a motor, an elevator door, an HVAC (heating ventilation and air conditioning) system, etc. For example, the vehicle may be an autonomous car, an airplane, a spacecraft, a dynamically positioned vessel, etc. Examples of the operation of the device 110 may include, but are not limited to, operating a vehicle according to a specific goal, operating an HVAC system according to specific parameters, operating a robotic arm according to a specific task, and opening / closing an elevator door.

[0046] The feedback controller 100 may include at least one processor 120, a transceiver 130, and a bus 140. In addition, the feedback controller 100 may include a memory 150. The memory may be embodied as a storage medium such as a RAM (random access memory), a ROM (read only memory), a hard disk, or any combination thereof. For example, the memory 150 may store instructions that may be executed by the at least one processor 120. In an embodiment, the memory 150 is configured to store a simulation model 170 and a probability solver 160 of the device 110. The probability solver 160 is parameterized on the control input of the actuator operating the device 110. The simulation model 170 of the device 110 may approximate the physical behavior of the device 110. For example, the device 110 may be an electric motor, and the simulation model 170 may be a model of the electric motor. The probability solver 160 and the simulation model 170 are described in detail at a later stage. The at least one processor 120 may be embodied as a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. At least one processor 120 may be operatively connected to the memory 150 and / or the transceiver 130 via a bus 140 .

[0047] According to an embodiment, the feedback controller 100 may be configured to determine a sequence of control inputs to control the device 110. For example, the control inputs may be associated with physical quantities such as voltage, pressure, force, torque, etc. In an example embodiment, the feedback controller 100 may determine the sequence of control inputs so that the sequence of control inputs changes the state of the device 110 in order to perform a specific task (e.g., tracking a reference). Once the sequence of control inputs is determined, the transceiver 130 may be configured to submit the sequence of control inputs to the device 110 as an input signal 180. As a result, the state of the device 110 may be changed according to the input signal 180 to perform a specific task. For example, the transceiver 130 may be an RF (radio frequency) transceiver, etc.

[0048] In addition, the state of the device 110 may be measured using one or more sensors installed in the device 110. The one or more sensors may send a feedback signal 190 to the transceiver 130. The transceiver 130 may receive the feedback signal 190. In an example embodiment, the feedback signal 190 may include a measurement sequence corresponding to the control input sequence, respectively. For example, the measurement sequence may be a measurement of the state of the device 110 output according to the control input sequence. Therefore, each measurement in the measurement sequence may indicate the state of the device 110 caused by the corresponding control input. Each measurement in the measurement sequence may be associated with a physical quantity such as current, speed, position, etc. In this way, the feedback controller 100 may iteratively submit the control input sequence and receive the feedback signal. In an example embodiment, in order to determine the control input sequence of the control step, the feedback controller 100 uses the feedback signal 190 including the measurement sequence indicating the current state of the device 110.

[0049] To determine the control input for the control step, the processor 120 recursively executes the probability solver 160 until the termination condition is met to generate the control input for the control step. In an embodiment, the probability solver 160 may use the simulation model 170 of the device 110 to generate the control input for the control step. Figure 2 The steps performed by the probability solver 160 to generate the control inputs are described in FIG.

[0050] Figure 2 A schematic diagram 200 illustrating steps performed by the probability solver 160 for generating control inputs according to some embodiments of the present disclosure is shown. The steps performed by the probability solver 160 include a prediction step 270, a simulation step 280, and / or a correction step 290.

[0051] In a prediction step 270 , the probability solver 160 uses the prediction model to estimate a PDF 220 of the predicted value of the control input from a probability distribution function (PDF) 210 of the value of the control input. Figure 3 An example of a PDF 220 according to some embodiments of the present disclosure is illustrated. For example, the PDF 220 may correspond to a Gaussian distribution. The Gaussian distribution may be defined by a mean 310 and a variance 320, wherein the mean 310 defines the center position of the distribution 220 and the variance 320 defines the spread (or width) of the distribution 220.

[0052] Return to reference Figure 2, further performing at least one simulation of the operation of the device 110 using the measurements indicating the device states at the control step and the values ​​of the control input sampled from the PDF 220 of the predicted values ​​of the control input to generate a performance metric for the operation of the device 110. The probabilistic solver 160 evaluates a cost function 295 defining the performance metric for the optimal control problem based on the at least one simulation of the operation of the device 110.

[0053] In addition, the probability solver 160 estimates a PDF 240 of a simulated value associated with the control input based on a measurement model that relates a performance metric of the device operation to the control input. In a correction step 290, the probability solver 160 may use the PDF 240 of the simulated value and the PDF 230 of the cost function 230 that defines the prediction to determine a corrected PDF 250 of the predicted value of the control input. The correction step 290 is performed at Figure 6 Detailed description in.

[0054] The prediction step 270, simulation step 280 and correction step 290 are recursively performed 260 until a termination condition is met. Once the termination condition is met, the probability solver 160 outputs the value of the control input. The value of the control input defines the control strategy. In other words, the value of the control input corresponds to the parameters of the control strategy. The feedback controller 100 controls the device 110 according to the control strategy. In particular, the feedback controller 100 controls the actuator of the operating device based on the value of the control input.

[0055] In an embodiment, the cost function 295 may include the deviation of the state of the device 110 from the state reference value, the deviation of the control input from the input reference value, the penalty for not reaching a specific goal in time, etc. The cost function 295 may include the state of the device 110 at the current moment provided by the current measurement. Additionally or alternatively, the cost function 295 may include the predicted future state of the device 110. For example, the cost function 295 may use the squared 2 norm,

[0056] Cost function = ‖h ref -h(x(t),u(t))‖ Q

[0057] =(h ref -h(x(t),u(t))) T Q(h ref -h(x(t),u(t))) (1)

[0058] Wherein x(t) may represent the predicted future state of the device 110 , u(t) may represent the control strategy, and Q may be a positive definite matrix.

[0059] The predicted future state x(t) of the device 110 may be obtained using the simulation model 170 and the control strategy u(t),

[0060] 0 = f sim (x(t),u(t),t) (2)

[0061] The state x0 of the device 110 may be used to initialize the predicted future state of the device 110 at time zero, x(0)=x0.

[0062] Achievable Figure 2 The optimization-based process shown is to minimize the cost function (1) subject to constraints defining the operation of the simulation model 170 and the device 110 over a specific time horizon T hor The state within,

[0063]

[0064] Some embodiments use parameterization of the control strategy, i.e., the control inputs,

[0065] u(t)=φ(θ,t), (4)

[0066] where some parameters of the control strategy are θ. Examples of parameterized control strategies include zero-order hold

[0067] u(t)=φ(θ,t)=u k , for k≤t≤k+T s , (5)

[0068] in First-order hold

[0069]

[0070] Or use the implementation of basis functions

[0071] u(t)=φ(θ,t)=θ T χ(t). (7)

[0072] In this context, the parameters θ of the control strategy may define the control strategy u(t), and the control strategy may define the predicted future state x(t) of the device 110 via the simulation model 170 .

[0073] The probability solver 160 may update the parameters of the control strategy at each iteration i. The prediction model of the prediction step 270 may be given by the gradient of the cost function (1) with respect to the parameters θ of the control strategy,

[0074] θ i+1 =θ i +f(θ i ) (8)

[0075] where θ iDefine the parameters of the control strategy at each iteration i, f(θ i ) defines the gradient as

[0076]

[0077] The advantages of using the gradient as part of the prediction model 270 include that the optimization-based process can converge quickly, which can improve the sampling rate of the feedback controller 100.

[0078] Alternatively, the prediction model of the prediction step 270 may be given by an identity model,

[0079] θ i+1 =θ i (10)

[0080] The identity model is advantageous because no analytical model needs to be derived and no analytical model needs to be stored in memory 150 .

[0081] Figure 4A and Figure 4B The process of how to use the distribution of control inputs to determine the simulation test of simulation step 280 according to some embodiments of the present disclosure is illustrated. Distribution 410 may correspond to PDF 220. Distribution 410 may define a certain number of simulation tests. For example, distribution 410 may provide three control strategies. The three control strategies may be given by a first control input 420, a second control input 430, and a third control input 440. Probabilistic solver 160 may perform simulation step 480 and use the three control inputs 420, 430, and 440 to evaluate the performance of the simulation. For example, distribution 410 may be a PDF with mean θ and variance σ 2 In this example, the first control input 420 can be represented by θ1=θ-σ 2 Given, the second control input 430 can be given by θ2=θ, and the third control input 440 can be given by θ3=θ+σ 2 Then, simulation experiments can be performed using these three control inputs that define the three control strategies,

[0082] u1=φ(θ1,t), (11)

[0083] u2=φ(θ2,t), (12)

[0084] as well as

[0085] u3=φ(θ3,t). (13)

[0086] The results of the simulation test obtained from the three control inputs can then be used in the correction step 290 to determine the corrected PDF 250. For example, the weighted average of the three control inputs can be used to update the control inputs / parameters defining the control strategy.

[0087] θ=w1θ1+w2θ2+w3θ3. (14)

[0088] The control inputs θ1 420, θ2 430, and θ3 440 are assigned weights w1 450, w2 460, and w3 470, respectively. The weights w1 450, w2 460, and w3 470 may be selected based on the cost function 295,

[0089]

[0090] as well as

[0091]

[0092] as well as

[0093]

[0094] where c i is related to the parameter θ i Cost function for the evaluation of the associated simulation trials.

[0095] Some embodiments are based on the recognition that different variances of different Gaussian distributions may produce different control inputs evaluated in the simulation step 280. For example, a Gaussian distribution with a high variance may result in control inputs for a defined control strategy being further apart for simulation experiments than control inputs for a Gaussian distribution with a lower variance.

[0096] Figure 5 Gaussian distributions 510, 520, and 530 with different variances according to some embodiments of the present disclosure are illustrated. The Gaussian distributions 510, 520, and 530 may be predicted by the prediction step 270. Each of the Gaussian distributions 510, 520, and 530 has a different variance relative to each other, but the mean 540 of the Gaussian distributions 510, 520, and 530 may be the same. A Gaussian distribution with a small variance and a mean 540 with the highest probability among the other Gaussian distributions may be more confident of the correct control input of the defined control strategy. The different variances of the Gaussian distributions 510, 520, and 530 may produce different control inputs for the defined control strategy evaluated in the simulation step 280. For example, a Gaussian distribution 530 with a high variance may cause the control inputs for the defined control strategy of the simulation experiment to be further apart than the control inputs of the Gaussian distribution 510 with a lower variance.

[0097] Figure 6A schematic diagram illustrating a correction step 290 for generating a corrected PDF 250 according to some embodiments of the present disclosure is shown. The probability solver 160 corrects the PDF 220 of the predicted value of the control input based on the PDF 240 of the simulated value associated with the control input to generate the PDF 250 of the value of the control input. For example, the three control inputs 420, 430, and 440 in FIG. 4 that define three control strategies and result in three predicted future states of the device 110 can be evaluated using the cost function (1). These three evaluations can be used to determine how to update the control inputs that define the control strategies. For example, if the evaluation of the third control input 440 produces a lower cost than the first control input 420 and produces a lower cost than the second control input 430, the probability solver 160 can determine that the control input that defines the control strategy for the next iteration is closer to the third control input 440.

[0098] The prediction step 270 , the simulation step 280 and / or the correction step 290 are recursively performed until a termination condition is met.

[0099] Figure 7 Example termination conditions according to some embodiments of the present disclosure are illustrated. Figure 7 A first PDF 710, a second PDF 720, and a surface area 730 defining the difference between the first PDF 710 and the second PDF 720 are illustrated. The first PDF 710 may correspond to the PDF 230 defining the predicted cost function 230, and the second PDF 720 may correspond to the PDF 240 of the simulated value. The surface area 730 may be used as a metric to determine whether to stop the iteration or continue the recursive iteration. For example, the surface area 730 may be too large to terminate the recursive iteration, to end Figure 2 1. The optimization-based process shown. After further iterations, the probability solver 160 may generate a third PDF 740 and a fourth PDF 750. The third PDF 740 may correspond to the PDF of the cost function defining the prediction (e.g., PDF 230), and the fourth PDF 750 may correspond to the PDF 240 of the simulation value (e.g., PDF 240). In this example, the surface area 760 between the third PDF 740 and the fourth PDF 750 may be determined to be small enough to stop the recursive iteration to end the optimization-based process. In some alternative embodiments, the termination condition is based on a similarity measure between the PDF of the value of the control input of the current iteration and the PDF of the value of the control input of the previous iteration. If this similarity is less than a threshold, it is conceivable that the termination condition is met.

[0100] For example, the optimization-based procedure can use the Kullback-Leibler divergence measure as a termination criterion. For two distributions p i+1 (θ) and p i(θ), the Kullback-Leibler divergence can be defined as

[0101]

[0102] Alternatively, the optimization-based process can use a measure of the proximity of the control inputs such as

[0103]

[0104] Some embodiments use a Kalman filter as a probability solver 160. A Kalman filter is a process (or method) that generates estimates of unknown variables using a series of measurements observed over a period of time (containing statistical noise and other inaccuracies). In practice, these generated estimates of unknown variables (e.g., control inputs) may be more accurate than estimates of unknown variables generated using a single measurement. A Kalman filter generates estimates of unknown variables by estimating a joint probability distribution over the unknown variables. A Kalman filter is a two-step process that includes a prediction step and an update step. In the prediction step, the Kalman filter uses a prediction model to predict the current variable and its uncertainty controlled by process noise. For example, the prediction model may be designed so that the prediction model is subject to process noise while predicting the current variable to reduce the uncertainty of the variable. In practice, the predicted current variable may be represented by a joint probability distribution over the current variable.

[0105] Some embodiments are based on the realization that the Kalman filter should jointly adjust the parameters defining the control strategy because the cost function 295 and the control inputs are interdependent. One advantage of using a Kalman filter is that the interdependence of the control inputs is taken into account through their joint distribution.

[0106] Figure 8 The implementation of the probability solver 160 as an iterative process using a Kalman filter 800 according to some embodiments of the present disclosure is illustrated. According to an embodiment, the state of the Kalman filter 800 is defined by a control input. To this end, the goal of the Kalman filter 800 is to iteratively generate control inputs for different control steps. In an example embodiment, the Kalman filter 800 can iteratively generate control inputs using a prediction model 810 and a measurement model 840.

[0107] The prediction model 810 of the Kalman filter 800 may be used in the prediction step 270. The measurement model 840 of the Kalman filter 800 may be used as a combination of the simulation step 280 and the correction step 290. For example, the measurement model 840 of the Kalman filter 800 may be specified by a cost function 295. The cost function 295 may be interpreted as having a prior distribution given by a multivariate Gaussian distribution,

[0108]

[0109] Its logarithm is

[0110]

[0111] The probability solver 160 is based on the observation that maximizing the logarithm of the multivariate Gaussian distribution is similar to minimizing the squared 2 norm that defines the cost function 295. Therefore, the multivariate Gaussian distribution may define the prior distribution 850 of the cost function 295. For example, the prior distribution may be defined by the mean h ref and covariance Q -1 Given,

[0112] h(x(t),u(t))~N(h ref ,Q -1 ).(twenty two)

[0113] To generate the control input in the current iteration, the prediction model 810 may be configured to use the prior knowledge 820 of the control input to predict the value of the control input. For example, the prior knowledge 820 of the control input may be a measure of how fast the control input is expected or desired to change between iterations. For example, the control input may be expected to change according to an identity model or gradient relative to the cost function 295 and the process noise. The process noise may be a measure of how credible the prediction model 810 is. The prior knowledge 820 of the control input may be a joint probability distribution (or Gaussian distribution) over the control input. The process noise may be a joint Gaussian distribution with zero mean and a prior covariance P0. The process noise may be artificially designed.

[0114] The prediction model 810 may use the prior knowledge 820 of the control inputs and the joint probability distribution over the control inputs at the previous iteration 860 to generate a predicted joint probability distribution 830. For example, the joint probability distribution over the control inputs at the previous iteration 860 may be calculated by the mean θ calculated at the previous iteration (e.g., at iteration i-1). i-1|i-1 and variance (or covariance) P i-1|i-1 For example, the joint probability distribution over the control inputs at the previous iteration 860 may be generated based on the joint probability distribution generated at the previous iteration (e.g., at iteration i-2). The predicted joint probability distribution 830 over the control inputs may be generated by the predicted mean θ i|i-1 and the predicted variance (or covariance) P i|i-1 definition.

[0115] The predicted joint probability distribution 830 over the control inputs may be Figure 2 The PDF 220 of the prediction step 270 is shown. For example, the prediction model 810 with the identity model can be used to calculate the predicted mean θ i|i-1 and the predicted variance (or covariance) P i|i-1 ,

[0116] θ i|i-1 =θ i-1|i-1 (twenty three)

[0117] P i|i-1 =P i-1|i-1 +P0, (24)

[0118] where P0 is the prior covariance that is part of the prior knowledge 820 of the control input.

[0119] Alternatively, the predicted mean θ may be calculated using the prediction model 810 having a gradient relative to the cost function 295 i|i-1 and the predicted variance (or covariance) P i|i-1 ,

[0120] θ i|i-1 =θ i-1|i-1 +f(θ i-1|i-1 ) (25)

[0121] Where f(θ i-1|i-1 ) is the gradient of the cost function 295 as in (9), and

[0122]

[0123] in

[0124]

[0125] It will be appreciated that the present disclosure is not limited to these two examples of prediction models, and other prediction models may also be utilized.

[0126] Different embodiments use different implementations of the probability solver 160. Additionally or alternatively, different embodiments use different types of Kalman filters. For example, one embodiment uses the framework of an unscented Kalman filter. This embodiment is advantageous because the unscented Kalman filter performs updates during its correction step without estimating gradients. In this way, this embodiment can solve the optimization problem without estimating any gradients, which is beneficial for many power plants with complex dynamics.

[0127] Additionally, the unscented Kalman filter may indicate samples of the control inputs that facilitate simulation step 280. For example, the unscented Kalman filter utilizes sigma points to represent the PDF of the values ​​of the control inputs, and the sigma points are used throughout iterations of the unscented Kalman filter to perform an unscented transformation. Some embodiments are based on the recognition that the sigma points can be used as sampled control inputs to simulate their operation. In this way, the evaluation of the device operation is integrated into the probabilistic framework of the unscented Kalman filter.

[0128] The measurement model 840 of the Kalman filter 800 may use the predicted joint probability distribution 830 to determine the simulation trials as shown in FIG. 4. Some embodiments use an unscented Kalman filter implementation to determine the simulation trials. The unscented Kalman filter calculates sigma points representing the predicted joint probability distribution 830. Thus, the sigma points define an alternative definition of the predicted joint probability distribution 830. Each sigma point characterizes a simulation trial in the simulation step 280. Each sigma point represents the realization of control input,

[0129]

[0130] Therefore, the sigma point can be calculated using the simulation model 170. Perform simulation, which obtains the state trajectory of the sigma point based on the simulation

[0131] Given a state trajectory of sigma points based on simulation, the measurement model 840 may use the cost function 295 to evaluate the simulation performance of each sigma point,

[0132]

[0133] The weights of the unscented Kalman filter can be chosen differently. For example, the weights can be chosen to have equal weight,

[0134]

[0135] The sigma points can be calculated using the covariance matrix and Choletsky decomposition,

[0136]

[0137] Among them [A i ] j is a square matrix A i The jth column of

[0138] Unscented Kalman filters are advantageous because the estimates of the covariance matrix and the control inputs define both the simulation trials to be evaluated and how the results of the simulation trials are used to update the control inputs.

[0139] The measurement model 840 may use the simulated performance of each sigma point in order to perform the correction step 290. For example, the measurement model 840 may calculate a weighted average of the sigma points.

[0140]

[0141] and the weighted average of the simulation performance at the sigma point

[0142]

[0143] Some embodiments use sigma points and simulated performance of sigma points to iteratively update control inputs. For example, the measurement model 840 may use the Kalman gain K i

[0144]

[0145] and the innovation covariance matrix

[0146]

[0147] and the cross-covariance matrix

[0148]

[0149] To update the parameters / control inputs that define the control strategy

[0150]

[0151] and the covariance matrix of the parameters defining the control strategy

[0152]

[0153] It will be understood that the implementations presented represent example implementations. In practice, other implementations may be similarly selected, such as different choices of weights, different calculations of sigma points, or different prediction models.

[0154] The calibration step 290 of the measurement model 840 may then output an updated mean value θ of the control input 870 i|i and the updated covariance P i|i Then, if Figure 7 If the termination condition shown is met, the updated joint distribution of control inputs 880 may be used to determine input signal 180 for device 110. If the termination condition is not met, the next iteration 890 is performed.

[0155] Some embodiments are based on the recognition that an additional advantage of parameterizing the Kalman filter on the control input is that the Kalman filter can test multiple samples from the PDF 220 of the predicted value to accelerate convergence. Testing multiple samples results in multiple simulations. However, multiple parallel processors can be used to perform the testing and simulations simultaneously. In addition, different Kalman filters with different measurement noises can be used to test different samples.

[0156] Fig. 9295 according to some embodiments of the present disclosure. The cost function can be any combination of the elements in the list. For example, the cost function can include the cost 910 of the deviation of any number of states from the target state, h ref =x ref And h(x(t),u(t))=x(t); the cost of deviation of any number of inputs from the target input is 915, h ref =u ref And h(x(t),u(t))=u(t).

[0157] Additionally, the cost function 295 may be: cost c1, for a state 920 that exceeds a certain operating range, h ref =0 and

[0158]

[0159] Cost c2, for inputs exceeding a certain operating range 925, h ref =0 and

[0160]

[0161] Cost c3, penalizes oscillations 930 of one or more states, e.g.

[0162] h ref = 0, h(x(t),u(t)) = c3 · #sign changes in 3 seconds (x(t));

[0163] Cost c4, penalizes oscillations 935 of one or more states, e.g.

[0164] h ref = 0, h(x(t),u(t)) = c4 · #sign changes in 3 seconds (u(t));

[0165] Cost c5, if the reference / target state overshoots a specific value 940, e.g.

[0166]

[0167] Cost c6, if the input exceeds a certain operating area 945 within a certain time, e.g.

[0168]

[0169] If the cost of state change between time steps is 950, e.g.

[0170] h ref =0, h(x(t),u(t))=||x(t+t1)-x(t)||2;

[0171] If the input cost varies between time steps 955, e.g.

[0172] h ref =0, h(x(t),u(t))=||u(t+t1)-u(t)||2;

[0173] Cost c7, if the state changes between time steps faster than a certain value 960, e.g.

[0174]

[0175] Cost c8, if the input changes faster than a certain value 965 between time steps, e.g.

[0176]

[0177] Additionally or alternatively, the objectives may be selected to include the costs of specific control inputs themselves (rather than states or inputs), e.g.

[0178] h ref =θ nom ,h(x(t),u(t))=θ i ,

[0179] where θ nom Defines nominal values ​​for any or all control parameters.

[0180] Fig.10 The control of a device such as a robot 1010 in an environment 1000 according to some embodiments of the present disclosure is illustrated. The robot 1010 is communicatively coupled to a feedback controller 100. The environment 1000 may be an autonomous factory 1000 including obstacles 1020 and 1030. The robot 1010 needs to perform a task of traversing a path 1060 to reach a target location 1050. The path 1060 connects an initial location 1040 and a target location 1050 of the robot 1010 in the environment 1000.

[0181] In an embodiment, the feedback controller 100 controls the robot 1010 from the initial position 1040 to the target position 1050 so that the robot 1010 deviates from the path 1060 to a minimum. For example, the feedback controller 100 collects measurements indicating the state of the robot 1010, such as the current position of the robot 1010. In addition, the feedback controller 100 recursively executes the probabilistic solver 160 until a termination condition is met to generate control inputs that maintain the position of the robot 1010 along the path 1060. The feedback controller 100 also controls the robot 1010 based on the generated control inputs.

[0182] Additionally, in some embodiments, the feedback controller 100 can be used to control a crane that manipulates a load. For example, the feedback controller 100 can minimize oscillations of the load of the crane relative to a reference path while maneuvering the load.

[0183] The description provides only exemplary embodiments and is not intended to limit the scope, applicability or configuration of the present disclosure. Instead, the following description of the exemplary embodiments will provide a feasible description for implementing one or more exemplary embodiments to those skilled in the art. It is contemplated that various changes may be made to the function and arrangement of the elements without departing from the spirit and scope of the disclosed subject matter set forth in the appended claims.

[0184] Specific details are given in the above description to provide a thorough understanding of the embodiments. However, it will be appreciated by those of ordinary skill in the art that the embodiments may be practiced without these specific details. For example, the systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form to avoid obscuring the embodiments in terms of unnecessary details. In other cases, well-known processes, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments. In addition, similar reference numerals and designations in the various drawings indicate similar elements.

[0185] In addition, various embodiments may be described as processes, which are depicted as flow charts, data flow diagrams, structure diagrams, or block diagrams. Although flow charts may describe operations as sequential processes, many operations may be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process may terminate when its operations are completed, but may have additional steps that are not discussed or included in the accompanying drawings. In addition, not all operations in any specifically described process may appear in all embodiments. A process may correspond to a method, a function, a program, a subroutine, a subprogram, etc. When a process corresponds to a function, the termination of the function may correspond to the function returning to the calling function or the main function.

[0186] In addition, embodiments of the disclosed subject matter may be implemented at least in part manually or automatically. Manual or automatic implementation may be performed or at least assisted by the use of a machine, hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments that perform the desired tasks may be stored in a machine-readable medium. A processor may perform the desired tasks.

[0187] The various methods or processes outlined herein may be encoded as software that can be executed on one or more processors using any of a variety of operating systems or platforms. In addition, such software may be written using any of a variety of suitable programming languages ​​and / or programming or scripting tools, and may also be compiled into executable machine language code or intermediate code that is executed on a framework or virtual machine. Typically, in various embodiments, the functionality of the program modules may be combined or distributed as desired.

[0188] The embodiments of the present disclosure may be embodied as a method, examples of which have been provided. The actions performed as part of the method may be ordered in any suitable manner. Thus, embodiments may be constructed that perform the actions in a different order than shown, which may include performing some actions simultaneously, although shown as sequential actions in the illustrative embodiments.

[0189] In addition, the embodiments of the present disclosure and the functional operations described in this specification may be implemented in digital electronic circuits, in tangibly implemented computer software or firmware, in computer hardware including the structures disclosed in this specification and their structural equivalents, or in a combination of one or more of them. In addition, some embodiments of the present disclosure may be implemented as one or more computer programs, that is, one or more modules of computer program instructions that are implemented on a tangible non-transitory program carrier for execution by a data processing device or control the operation of the data processing device. In addition, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical signal, optical signal, or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0190] According to an embodiment of the present disclosure, the term "data processing device" may cover all types of devices, apparatuses and machines for processing data, including, by way of example, a programmable processor, a computer or a plurality of processors or computers. The device may include dedicated logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the device may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0191] A computer program (which may also be referred to or described as a program, software, software application, module, software module, script, or code) may be written in any form of programming language, including compiled or interpreted languages ​​or declarative or procedural languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may (but need not) correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files, such as files that store one or more modules, subroutines, or portions of code.

[0192] A computer program may be deployed to be executed on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network. As an example, a computer suitable for executing a computer program includes a central processing unit that may be based on a general-purpose microprocessor or a special-purpose microprocessor or both, or any other type. Typically, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for executing instructions and one or more memory devices for storing instructions and data.

[0193] Typically, a computer will also include one or more mass storage devices (e.g., magnetic, magneto-optical, or optical disks) for storing data or be operatively connected to receive data from or transfer data to or both of one or more mass storage devices (e.g., magnetic, magneto-optical, or optical disks) for storing data. However, a computer need not have these devices. In addition, a computer may be embedded in another device (e.g., a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, etc.).

[0194] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, verbal, or tactile input. In addition, a computer may interact with a user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser in response to a request received from a web browser on a user's client device.

[0195] Implementations of the subject matter described in this specification may be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), such as the Internet.

[0196] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0197] Although the present disclosure has been described with reference to certain preferred embodiments, it will be understood that various other adaptations and modifications may be made within the spirit and scope of the present disclosure. Therefore, the aspects of the appended claims cover all such changes and modifications that fall within the true spirit and scope of the present disclosure.

Claims

1. A feedback controller for controlling the operation of a device at different control steps based on a feedback signal, the feedback signal comprising a measurement indicative of a state of the device at the different control steps, the feedback controller comprising: at least one processor; and a memory storing instructions which, when executed by the at least one processor, cause the feedback controller to control the steps of: collecting measurements indicative of said state of said device during said controlling step; recursively executing a probability solver parameterized on a control input of an actuator operating the device until a termination condition is satisfied to generate a control input for the control step, wherein during each of the executions, the probability solver is configured to estimating a PDF of a predicted value of the control input from a probability distribution function PDF of the value of the control input using a prediction model; evaluating a cost function of an optimal control problem for controlling the device based on a simulation of the operation of the device using the measurements and values ​​of the control input sampled from the PDF of predicted values ​​of the control input to generate a performance metric for the operation of the device; and estimating a PDF of a simulation value associated with the control input based on a measurement model that relates the performance metric of the operation of the device to the control input; and correcting the PDF of the predicted value of the control input based on the PDF of the simulated value associated with the control input to generate the PDF of the value of the control input; and At least a mean of the PDF of the values ​​of the control input is used to control the actuator that operates the device.

2. The feedback controller according to claim 1, wherein: The prediction model is an identity model.

3. The feedback controller according to claim 1, wherein: The prediction model is given by the gradient of the cost function with respect to the control input.

4. The feedback controller according to claim 1, wherein: For each of the iterations performed for each of the control steps, the probabilistic solver is further configured to evaluate the cost function for a plurality of samples of simulation experiments defining the PDF of predicted values.

5. The feedback controller according to claim 4, wherein: To generate the PDF of the simulated values ​​associated with the control input from a plurality of performance metrics, the probabilistic solver is further configured to determine a weighted combination of estimates of a plurality of samples of the PDF of predicted values.

6. The feedback controller according to claim 4, wherein: The cost function is evaluated in parallel for multiple samples of the PDF of predicted values ​​using multiple processors.

7. The feedback controller according to claim 1, wherein: The termination condition is based on a similarity measure between the PDF of values ​​of the control input and a PDF of values ​​of the control input at a previous iteration.

8. The feedback controller according to claim 1, wherein: The probability solver is a Kalman filter.

9. The feedback controller according to claim 1, wherein: The probabilistic solver is an unscented Kalman filter that performs a gradient-free correction of the predicted value of the control input.

10. The feedback controller according to claim 9, wherein: To update the PDF of the predicted values ​​using the gradient-free correction, the unscented Kalman filter is configured to generate a PDF of the simulated values ​​associated with the control input using an evaluation of the cost function.

11. The feedback controller according to claim 9, wherein: The unscented Kalman filter evaluates the cost function at multiple sigma points of the PDF of the predicted values.

12. The feedback controller according to claim 11, wherein: To determine the sigma point, the unscented Kalman filter is configured to generate the sigma point based on a predicted mean of the PDF of predicted values ​​and a covariance matrix of the PDF of predicted values.

13. The feedback controller according to claim 12, wherein: The sigma point is determined based on the Cholesky decomposition.

14. A method for controlling the operation of a device at different control steps based on a feedback signal, the feedback signal comprising a measurement indicative of a state of the device at the different control steps, the method comprising the steps of: collecting measurements indicative of said state of said device during said controlling step; recursively executing a probability solver parameterized on a control input of an actuator operating the device until a termination condition is satisfied to generate a control input for the control step, wherein during each of the executions, the probability solver is configured to estimating a PDF of a predicted value of the control input from a probability distribution function PDF of the value of the control input using a prediction model; evaluating a cost function of an optimal control problem for controlling the device based on a simulation of the operation of the device using the measurements and values ​​of the control input sampled from the PDF of predicted values ​​of the control input to generate a performance metric for the operation of the device; and estimating a PDF of a simulation value associated with the control input based on a measurement model that relates the performance metric of the operation of the device to the control input; and correcting the PDF of the predicted value of the control input based on the PDF of the simulated value associated with the control input to generate the PDF of the value of the control input; and At least a mean of the PDF of the values ​​of the control input is used to control the actuator that operates the device.

15. The method according to claim 14, wherein: The prediction model is an identity model.

16. The method according to claim 14, wherein: The prediction model is given by the gradient of the cost function with respect to the control input.

17. The method according to claim 14, wherein: For each of the iterations performed for each of the control steps, the probabilistic solver is further configured to evaluate the cost function for a plurality of samples of simulation experiments defining the PDF of predicted values.

18. The method according to claim 17, wherein: To generate the PDF of the simulated values ​​associated with the control input from a plurality of performance metrics, the probabilistic solver is further configured to determine a weighted combination of estimates of a plurality of samples of the PDF of predicted values.

19. The method according to claim 14, wherein: The termination condition is based on a similarity measure between the PDF of values ​​of the control input and a PDF of values ​​of the control input at a previous iteration.

20. A non-transitory computer readable storage medium having a program embodied thereon, the program being executable by a processor for performing a method for controlling operation of a device at different control steps based on a feedback signal, the feedback signal comprising a measurement indicative of a state of the device at the different control steps, the method comprising the steps of: collecting measurements indicative of said state of said device during said controlling step; recursively executing a probability solver parameterized on a control input of an actuator operating the device until a termination condition is satisfied to generate a control input for the control step, wherein during each of the executions, the probability solver is configured to estimating a PDF of a predicted value of the control input from a probability distribution function PDF of the value of the control input using a prediction model; evaluating a cost function of an optimal control problem for controlling the device based on a simulation of the operation of the device using the measurements and values ​​of the control input sampled from the PDF of predicted values ​​of the control input to generate a performance metric for the operation of the device; and estimating a PDF of a simulation value associated with the control input based on a measurement model that relates the performance metric of the operation of the device to the control input; and correcting the PDF of the predicted value of the control input based on the PDF of the simulated value associated with the control input to generate the PDF of the value of the control input; and At least a mean of the PDF of the values ​​of the control input is used to control the actuator that operates the device.

Images