System and method for calibrating a feedback controller

Abstract

A system for controlling operation of a machine performing a task is disclosed. The system submits a sequence of control inputs to the machine and receives feedback signals. The system also applies, at each control step, a control policy that transforms a current measurement of a current state of the system into a current control input for controlling the machine based on current values of control parameters in a set of control parameters of a feedback controller, determines the current control input based on feedback signals including the current measurement. Further, the system can iteratively update a state of the feedback controller defined by the control parameters using a predictive model that predicts values of the control parameters and a measurement model that updates the predicted values to generate current values of the control parameters that specify a sequence of measurements according to a performance objective.

Description

Technical Field

[0001] This disclosure relates generally to control systems, and more specifically to systems and methods for calibrating feedback controllers. Background Technology

[0002] Currently, various dynamic machines exist that can operate in unstructured and uncertain environments. In fact, these dynamic machines are inherently more complex to operate in such environments. Because of their inherent complexity and operation in increasingly unstructured or uncertain environments, the need for automated design and calibration processes for dynamic machines becomes more critical. In particular, the control of many dynamic machines, such as autonomous vehicles or robots, involves various often conflicting specifications, thus requiring considerable manual calibration effort. Furthermore, calibration is typically performed during the production phase, and because the operating conditions of a dynamic machine vary throughout its lifespan, it is often difficult to adjust the controllers associated with the dynamic machine later on.

[0003] Several methods exist that aim to automate controller calibration and adapt controllers to the operating and operational conditions of powered machines. However, these methods focus on repetitive learning tasks that involve learning from human experts or through trial and error. Therefore, these methods may only be suitable for applications where repetitive learning is appropriate. For example, they can be used in robots to manipulate objects. However, these methods cannot provide controller calibration in inherently more continuous control applications such as autonomous driving. Furthermore, trial and error is often unsuitable for safety-critical machines. Additionally, the requirement for human demonstrators limits the amount of automation.

[0004] Therefore, a system is needed that can automatically calibrate the controller in an efficient and feasible manner. Summary of the Invention

[0005] One implementation aims to iteratively calibrate the controller in real time and use the calibrated controller to control the operation of the machine. Examples of machines may include vehicles (e.g., autonomous vehicles), robot components, motors, elevator doors, HVAC (heating, ventilation, and air conditioning) systems, etc. Examples of machine operation may include (but are not limited to) operating a vehicle according to a specific trajectory, operating an HVAC system according to specific parameters, operating a robot arm according to a specific task, and opening / closing elevator doors. Examples of controllers may include PID (proportional-integral-derivative) controllers, optimal controllers, neural network controllers, etc. Hereinafter, "controller" and "feedback controller" are used interchangeably to refer to the same thing.

[0006] To calibrate the feedback controller, some implementations use a Kalman filter. However, Kalman filters are typically used to estimate state variables that define the state of a machine, where the state can be a physical quantity such as position, velocity, etc. Therefore, some implementations aim to transform or adjust the Kalman filter compared to the machine's state variables to estimate the control parameters for the feedback controller used to control the machine. While state variables define the state of the controlled machine, control parameters are used to calculate control commands. Examples of control parameters are the gain of the feedback controller (e.g., the gain in a PID controller) and / or parameters of the machine's physical structure (similar to the mass of a robot arm or the friction between a vehicle's tires and the road). It is important to note that control parameters should not be confused with control variables (e.g., the voltage values ​​controlling actuators) that define the inputs and outputs of the control law or control strategy executed by the feedback controller. In other words, the control law defined based on the control parameters maps the input control variables to the output control variables. This mapping can be analytical or based on a solution to an optimization problem.

[0007] In many control applications, control parameters are known in advance and fixed, i.e., they remain constant during control. For example, the mass of a robot arm can be measured or known from the robot's specifications, tire friction can be limited or selected, and the gain of the controller can be adjusted in the laboratory. However, pre-fixing control parameters is suboptimal for some applications, and even impractical for others, which instead require controlling the machine with parameters that have uncertainty.

[0008] Some implementations are based on the understanding that the principle of tracking the state variables provided by a Kalman filter can be extended or adapted to track control parameters. In practice, although control is not a machine but a process, it is recognized that control can be viewed as a virtual machine with virtual states defined by control parameters. Based on this intuition, if the predictive model used by the Kalman filter during the prediction phase can predict the control parameters that can be measured to describe the state of the machine according to the measurement model, then the Kalman filter can iteratively track the control parameters.

[0009] It is worth noting that this flexibility, because the prediction and measurement models are provided by the Kalman filter's designer, allows the Kalman filter to be tuned for different types of control objectives. For example, in some implementations, the prediction model is a constant or identity model that predicts the control parameters will not change within the variance of the process noise. In practice, such predictions are common for many control applications with fixed control parameters. Additionally or alternatively, some implementations define prediction models that can predict at least some parameters based on their predetermined relationships with other parameters. For example, some implementations can predict changes in tire friction based on the vehicle's current speed. In this configuration of the Kalman filter, the process noise control parameters vary rapidly over time.

[0010] In any case, this predictive model places the primary work of tracking control parameters on the measurement model and adds the flexibility to update the measurement model based on changes in control objectives. Notably, this flexibility allows for variations in the measurement model to be used to control different machines, but also allows for variations in the measurement model at different times or under different states during the control of the same machine.

[0011] Therefore, in various implementations, the measurement model uses a performance objective that evaluates the performance of the online controlled closed-loop machine's operation, which is then used to adjust control parameters to improve the closed-loop machine operation measured relative to the performance objective. It is noteworthy that the performance objective has a highly flexible structure and can differ from the objective of the optimal controller. This is advantageous because the optimal control cost function has a structure constrained by its real-time application; for example, the cost function often needs to be differentiable and convex to make it suitable for numerical optimization. Furthermore, the performance objective can be changed at different control times according to the same optimal control objective. Additionally, the optimal control objective or other control parameters can be changed at different times or as a function of the machine state according to the same performance objective.

[0012] In this way, the advantages of the Kalman filter are extended to the recursive estimation of control parameters. These advantages include (i) online parameter adjustment during machine operation, (ii) robustness to noise due to the filter-based design, (iii) safety assurance for maintaining closed-loop operation, (iv) computational efficiency, (v) reduced data storage requirements due to recursive implementation, and (vi) ease of implementation, making it attractive for industrial applications.

[0013] Some implementations are based on the understanding that in many applications, multiple control parameters need to be collectively adjusted in a way that is interdependent. For example, the gain of a PID controller needs to be collectively adjusted to achieve the desired performance and ensure safe operation; the weights of the cost function of optimal control need to be collectively adjusted because they define trade-offs between multiple potentially conflicting objectives; and the weights for H... ∞ The filter coefficients of the controller or dynamic output feedback controller are adjusted to ensure performance and stability requirements.

[0014] Calibrating interdependent parameters is typically a more challenging problem because this interdependence adds another variable to consider. Therefore, having multiple interdependent parameters to calibrate can increase calibration complexity. However, some implementations are based on the understanding that this interdependence of the control parameters being calibrated can be naturally and statistically adjusted by varying the weights of the Kalman gain applied to the updates of different parameters.

[0015] Some implementations are based on the understanding that the control parameters used in a feedback controller depend on the machine's state. Some implementations address this state dependency using linear combinations of basis functions, which are functions of the machine's state. In practice, a Kalman filter can be implemented by adjusting the coefficients of the basis functions and then using them to generate the control parameters. Alternatively, some implementations use state-dependent regions in combination with the basis functions. In each region, the control parameters are calculated as linear combinations of basis functions. The Kalman filter can adjust the coefficients of the basis functions in each region and determine which set of basis functions is used to calculate the control parameters in that region.

[0016] In different implementations, the controlled machine exhibits linear or nonlinear dynamics and varying uncertainties in control parameters with different limits. Some implementations address these variations by selecting different types of Kalman filter implementations and / or varying variances of process and / or measurement noise.

[0017] For example, one implementation uses an extended Kalman filter (EKF) to compute the Kalman gain. The EKF numerically computes the gradient of the performance objective with respect to the control parameters. The EKF is useful for problems where the performance objective is differentiable with respect to the machine state because the gradient is computed using two gradients: (i) the gradient of the performance objective with respect to the machine state and (ii) the gradient of the dynamic machine state with respect to the control parameters. The gradient of the performance objective with respect to the machine state is computed by the designer. The gradient of the machine state with respect to the control parameters is computed using the structure of the feedback controller and a model defining the dynamic characteristics of the machine.

[0018] Alternatively, one implementation uses an unscented Kalman filter (UKF) to compute the Kalman gain. The UKF uses a functional evaluation of the performance objective to estimate the gradient of the performance objective relative to the control parameters. In this case, the UKF computes sigma points (realizations of the control parameters). The gradient is then estimated using an evaluation of the performance objective at all sigma points, combined with the joint probability distribution of the control parameters. The UKF is useful for both differentiable and non-differentiable performance objectives because it uses a functional evaluation to estimate the gradient.

[0019] Some implementations are based on the understanding that online iterative updates of control parameters in a feedback controller can improve control quality, but at the cost of additional challenges. For example, online updates of control parameters during machine operation can introduce control discontinuities. However, some implementations are based on the understanding that such discontinuities can be addressed by forcing control commands to satisfy constraints on machine operation. These constraints can be established by checking the control parameters to satisfy the properties of the established control theory.

[0020] Alternatively, some implementations are based on the understanding that online updates of control parameters can destabilize machine operation. For example, when the control law or control strategy is represented by a differential equation (ODE) of control parameters, changes in the control parameters may disrupt the equilibrium stability of the ODE. To address this new problem potentially introduced by Kalman filters in different implementations, some implementations perform safety checks, such as stability checks on the control strategy against the values ​​of the control parameters generated by the Kalman filter. Furthermore, the control parameters in the control strategy are only updated if the stability check is satisfied.

[0021] For example, a stability check is satisfied when a Lyapunov function exists for the control strategy updating the control parameters. The existence of a Lyapunov function can be checked in many ways. For example, some implementations solve optimization problems aimed at finding the Lyapunov function and / or proving its existence. Alternatively, one implementation checks whether the updated control parameters result in a decreasing cost of the state relative to the performance objective over the entire history of states and inputs. Another implementation checks whether the updated control parameters maintain the machine's proximity to its origin. It is recognized that maintaining proximity to the origin means that the cost associated with the end of the prediction range of the parameter-updated control strategy is defined by the ratio of the largest to the smallest eigenvalue of a positive definite matrix, such as the terminal cost.

[0022] Furthermore, some implementations are based on the understanding that when the control parameters generated by the Kalman filter fail to meet the safety check, the control parameters of the feedback controller should not be updated with the output of the Kalman filter. However, the Kalman filter itself should not restart; instead, it should continue its iteration with the newly generated control parameters, even if the control parameters of the Kalman filter will differ from those of the feedback controller. When the control parameters of the Kalman filter will meet the safety check during some subsequent iterations, the safety control parameters of the Kalman filter will update the old control parameters of the feedback controller. In this way, the implementation ensures control stability even with online updates of control parameters.

[0023] Therefore, one embodiment discloses a system for controlling the operation of a machine performing a task. The system includes a transceiver configured to submit a sequence of control inputs to the machine and receive feedback signals including corresponding measurement sequences, wherein each measurement indicates a machine state resulting from the corresponding control input. The system also includes a feedback controller configured to, at each control step, transform the current measurement of the machine's current state into a control strategy for controlling the machine based on the current values ​​of control parameters in a set of control parameters of the feedback controller, determining the current control input based on the feedback signals including the current measurements. Furthermore, the system includes a Kalman filter configured to iteratively update the state of the feedback controller, defined by the control parameters, using a prediction model predicting the values ​​of the control parameters subjected to process noise and a measurement model updating the predicted values ​​of the control parameters based on a measurement sequence subjected to measurement noise, to generate current values ​​of the control parameters describing the measurement sequence according to a performance objective.

[0024] Therefore, another embodiment discloses a method for controlling the operation of a machine performing a task. The method includes the following steps: submitting a control input sequence to the machine; receiving a feedback signal including a corresponding measurement sequence, wherein each measurement indicates a machine state caused by the corresponding control input; at each control step, transforming the current measurement of the machine's current state into a control strategy for controlling the machine's current control input by applying the current values ​​of control parameters in a set of control parameters based on the feedback controller, determining the current control input based on the feedback signal including the current measurement; and iteratively updating the state of the feedback controller defined by the control parameters using a prediction model predicting the values ​​of control parameters subjected to process noise and a measurement model updating the predicted values ​​of the control parameters based on the measurement sequence subjected to measurement noise, to generate current values ​​of the control parameters describing the measurement sequence according to a performance target.

[0025] Therefore, another embodiment discloses a programmable non-transitory computer-readable storage medium executable by a processor to perform a method for controlling the operation of a machine performing a task. The method includes the steps of: submitting a sequence of control inputs to the machine; receiving feedback signals including corresponding measurement sequences, wherein each measurement indicates a machine state resulting from the corresponding control input; at each control step, transforming the current measurement of the machine's current state into a control strategy for controlling the machine's current control input by applying the current values ​​of control parameters in a set of control parameters based on the feedback signal including the current measurements, determining the current control input based on the feedback signal including the current measurements; and iteratively updating the state of the feedback controller defined by the control parameters using a prediction model predicting the values ​​of the control parameters subjected to process noise and a measurement model updating the predicted values ​​of the control parameters based on the measurement sequence subjected to measurement noise, to generate current values ​​of the control parameters describing the measurement sequence according to a performance objective. Attached Figure Description

[0026] [ Figure 1 ] Figure 1 An overview of the principles of a Kalman filter according to some embodiments of the present disclosure is illustrated.

[0027] [ Figure 2A ] Figure 2A A block diagram illustrating a control system for controlling the operation of a powered machine according to some embodiments of the present disclosure is shown.

[0028] [ Figure 2B ] Figure 2B Kalman filters for generating control parameters according to some embodiments of the present disclosure are illustrated.

[0029] [ Figure 2C ] Figure 2C An example is illustrated by a Gaussian distribution representing a particular control parameter according to some embodiments of the present disclosure.

[0030] [ Figure 2D ] Figure 2D Examples of Gaussian distributions with different variances according to some embodiments of the present disclosure are shown.

[0031] [ Figure 2E ] Figure 2E The evolution of machine state over time is illustrated according to some embodiments of the present disclosure.

[0032] [ Figure 2F ] Figure 2F A schematic diagram illustrating the predicted values ​​of updated control parameters according to some embodiments of the present disclosure is shown.

[0033] [ Figure 3 ] Figure 3A block diagram illustrating a Kalman filter for calibrating multiple interdependent control parameters according to some embodiments of the present disclosure is shown.

[0034] [ Figure 4A ] Figure 4A Methods performed by a Kalman filter according to some embodiments of the present disclosure are illustrated.

[0035] [ Figure 4B ] Figure 4B Methods performed by a Kalman filter according to some other embodiments of this disclosure are illustrated.

[0036] [ Figure 5 ] Figure 5 Methods for calibrating state-related control parameters according to some embodiments of the present disclosure are illustrated.

[0037] [ Figure 6A ] Figure 6A A block diagram illustrating a system for controlling the operation of a machine according to some embodiments of the present disclosure is shown.

[0038] [ Figure 6B ] Figure 6B Examples of security check methods performed by a security check module according to some embodiments of the present disclosure are illustrated.

[0039] [ Figure 6C ] Figure 6C The evolution of machine states according to some other embodiments of this disclosure is illustrated.

[0040] [ Figure 7 ] Figure 7 A list of performance targets used in some embodiments of the Kalman filter according to this disclosure is provided.

[0041] [ Figure 8A ] Figure 8A A list of feedback controllers used in systems according to some embodiments of this disclosure is provided.

[0042] [ Figure 8B ] Figure 8B A block diagram illustrating a feedback controller for controlling a machine according to some embodiments of the present disclosure is shown.

[0043] [ Figure 9 ] Figure 9 A schematic diagram of a system for controlling an electric motor according to some embodiments of the present disclosure is shown.

[0044] [ Figure 10 ] Figure 10 A schematic diagram of a system for controlling a vehicle according to some embodiments of the present disclosure is shown. Detailed Implementation

[0045] In the following description, numerous specific details are set forth for illustrative purposes in order to provide a thorough understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure may be practiced without these specific details. In other instances, apparatuses and methods are shown only as block diagrams to avoid obscuring this disclosure.

[0046] As used in this specification and claims, the terms "for example" and "such as," as well as the verbs "comprising," "having," "including," and other verb forms thereof, when used in conjunction with a list of one or more components or other items, shall each be interpreted as open-ended, meaning that the list should not be considered as excluding other additional components or items. The term "based on" means at least partially based on. Furthermore, it will be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. Any headings used within this description are for convenience only and have no legal or limiting effect.

[0047] Figure 1 An overview of the principles of a Kalman filter according to some embodiments of the present disclosure is illustrated. A Kalman filter 100 is a process (or method) for generating estimates of unknown variables using a series of measurements observed over a period of time, which include statistical noise and other inaccuracies. In practice, these generated estimates of unknown variables can be more accurate than estimates generated using a single measurement. The Kalman filter 100 generates estimates of unknown variables by estimating the joint probability distribution over the unknown variables.

[0048] In an exemplary scenario, the series of measurements used by the Kalman filter 100 may be measurements 102 associated with the state variables of the powered machine. Therefore, in this exemplary scenario, the Kalman filter 100 can be used to generate a state estimate 104 of the powered machine. As used herein, state variables may be variables that mathematically describe the “state” of the powered machine. The state of the powered machine adequately describes the powered machine to determine its future behavior (e.g., motion) without any external forces affecting the powered machine. For example, state estimate 104 may be an estimate of physical quantities such as velocity, position, etc. In practice, these state estimates 104 are needed in applications such as navigation guidance and the control of vehicles (particularly aircraft, spacecraft, and dynamically positioned ships).

[0049] The Kalman filter 100 is a two-step process comprising a prediction step and an update step. In the prediction step, the Kalman filter 100 uses a prediction model to predict the current state and its uncertainty under process noise control. For example, the prediction model may be artificially designed to be subjected to process noise (e.g., assumption 108) while predicting the current state to reduce uncertainty in the state. In practice, the predicted current state may be represented by a joint probability distribution over the current state. In some example implementations, the prediction model may use model 106 of the power machine to predict the current state. As used herein, model 106 of the power machine may be a mathematical expression relating the state of the power machine to (i) the previous state of the power machine and (ii) the control input of the power machine. An example of model 106 is given below.

[0050] x k+1 =f(x) k ,u k )+w k ,

[0051] Where x k It is a vector or scalar that includes all states of the power machine, u k It is a vector or scalar that includes all control inputs to the power machine, f is the nonlinear function model of the power machine, and w k This illustrates the mismatch between the power machine and model 106.

[0052] In the update step, once the result of the next measurement is observed (which inevitably contains a certain amount of error, including random noise), the predicted state is updated according to the measurement model subjected to measurement noise. Measurement noise can control for errors in the measurement. Measurement noise can also be included in Assumption 108. The measurement model can be designed such that it aims to make the prediction consistent with the measurement. For example, the measurement model can use a weighted average to update the joint probability distribution over the current state, where estimates with higher certainty are given larger weights.

[0053] The output of the Kalman filter 100 can be a state estimate 104 that maximizes the probability of the received state measurement 102, given an assumption 108 about noise (e.g., process noise and measurement noise) and a model 106 of the power machine. For example, the assumption about noise 108 can include a mathematical noise model designed to reduce inaccuracies in the state and measurements. The Kalman filter 100 is a recursive process that can run in real time using only the current measurements and previously calculated states and their uncertainty matrices; no additional past information is required.

[0054] Some implementations are based on the understanding that the principles provided by the Kalman filter 100 for estimating the state of a power machine can be extended or adapted to estimate the virtual state of a virtual machine. In other words, the Kalman filter 100 for estimating the state of a power machine can be extended to a Kalman filter 110 for estimating the virtual state of a virtual machine. It is noteworthy that this flexibility allows the Kalman filter 100 to be adapted or extended to Kalman filter 110 because the prediction model and measurement model are provided by the designer of the Kalman filter 100.

[0055] In many control applications, the control parameters that define the state of the controller can be known in advance and fixed, i.e., kept constant during the control of the powered machine. Examples of control parameters include the controller gain (e.g., the gain in a PID controller) and / or parameters of the physical structure of the powered machine (such as the mass of a robotic arm or the friction between a vehicle's tires and the road). For example, the mass of a robotic arm can be measured or known from the robot's specifications, tire friction can be limited or selected, and the controller gain can be adjusted in the laboratory. However, fixing control parameters in advance is suboptimal for some applications, and even impractical for others, which instead require controlling the machine with control parameters that have uncertainty.

[0056] Therefore, some implementations aim to extend or adapt the Kalman filter 100 to a Kalman filter 110 that estimates control parameters 112 that define the state of the controller. In these implementations, the virtual state is the state defined by the control parameters, and the virtual state is the controller. To extend the Kalman filter 100 to the Kalman filter 110, in the prediction step, the prediction model subjected to process noise can be adapted to predict the control parameters using a transition model 116 of the control parameters 112. The process noise in the Kalman filter 110 controls how quickly the control parameters change over time, rather than the inaccuracies in the control state. Therefore, assumption 118 can be designed. Furthermore, the transition model 116 can also be designed manually.

[0057] In the update step, the measurement model subjected to measurement noise can be adjusted to evaluate the performance of the predicted control parameters in the control of the power machine based on performance target 114. Furthermore, the measurement model can be adjusted to update the predicted control parameters based on the evaluation. It is worth noting that performance target 114 has a highly flexible structure and can differ from the controller's target.

[0058] Thus, the Kalman filter 110 can estimate the control parameters 112 based on the assumption 118 of how rapidly the control parameters change, even when there is an error relative to the performance target 114. In effect, the output of the Kalman filter 110 is the control parameter estimate 112 that maximizes the probability of the received performance target 114, given (i) the assumption 118 and (ii) the transition model 116. For example, a control system using the principle of the Kalman filter 110 is... Figure 2A As described in the detailed description.

[0059] Figure 2A A block diagram illustrating a control system 200 for controlling the operation of a powered machine 202 according to some embodiments of the present disclosure is shown. Some embodiments are based on the understanding that the goal of the control system 200 is to control the powered machine 202 during the engineering process. For this purpose, the control system 200 is operatively coupled to the powered machine 202. Hereinafter, "control system" and "system" are used interchangeably to mean the same thing. Hereinafter, "powered machine" and "machine" are used interchangeably to mean the same thing. Examples of the machine 202 may include a vehicle (e.g., an autonomous vehicle), a robot component, a motor, an elevator door, an HVAC (heating, ventilation, and air conditioning) system, etc. For example, the vehicle may be a self-driving car, an aircraft, a spacecraft, a dynamically positioned ship, etc. Examples of the operation of the machine 202 may include (but are not limited to) operating a vehicle according to a specific trajectory, operating an HVAC system according to specific parameters, operating a robot arm according to a specific task, and opening / closing elevator doors.

[0060] System 200 may include at least one processor 204, a transceiver 206, and a bus 208. Additionally, system 200 may include memory. The memory may be implemented as a storage medium, such as RAM (Random Access Memory), ROM (Read-Only Memory), a hard disk, or any combination thereof. For example, the memory may store instructions executable by at least one processor 204. At least one processor 204 may be implemented as a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. At least one processor 204 may be operatively connected to the memory and / or the transceiver 206 via bus 208. According to embodiments, at least one processor 204 may be configured as a feedback controller 210 and / or a Kalman filter 212. Thus, the feedback controller 210 and the Kalman filter 212 may be implemented within a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. Alternatively, the feedback controller 210 may be implemented outside of system 200 and may communicate with system 200. In this configuration, system 200 is operatively connected to feedback controller 210, which in turn is connected to machine 202. For example, feedback controller 210 may be (but is not limited to) a PID (proportional-integral-derivative) controller, an optimal controller, a neural network controller, etc.

[0061] According to an implementation, the feedback controller 210 can be configured to determine a sequence of control inputs to control the machine 202. For example, the control inputs may be associated with physical quantities such as voltage, pressure, force, torque, etc. In an example implementation, the feedback controller 210 can determine a sequence of control inputs such that the sequence changes the state of the machine 202 to perform a specific task (e.g., tracking a reference). Once the control input sequence is determined, the transceiver 206 can be configured to submit the control input sequence as an input signal 214. As a result, the state of the machine 202 can be changed according to the input signal 214 to perform a specific task. For example, the transceiver 206 may be an RF (radio frequency) transceiver, etc.

[0062] Furthermore, the state of machine 202 can be measured using one or more sensors installed on machine 202. One or more sensors can send feedback signals 216 to transceiver 206. Transceiver 206 can receive feedback signals 216. In an example embodiment, feedback signals 216 may include measurement sequences, each corresponding to a control input sequence. For example, the measurement sequence may be a measurement of the state output by machine 202 according to the control input sequence. Thus, each measurement in the measurement sequence can indicate the state of machine 202 caused by the corresponding control input. Each measurement in the measurement sequence may be associated with a physical quantity such as current, flow rate, speed, position, etc. In this way, system 200 can iteratively submit control input sequences and receive feedback signals. In an example embodiment, to determine the control input sequence in the current iteration, system 200 uses feedback signals 216 that include a measurement sequence indicating the current state of machine 202.

[0063] To determine the sequence of control inputs in the current iteration, feedback controller 210 may be configured to determine the current control input for controlling machine 202 at each control step based on feedback signal 216, which includes the current measurement of the current state of the machine. According to embodiments, to determine the current control input, feedback controller 210 may be configured to apply a control strategy. As used herein, the control strategy may be a set of mathematical equations mapping all states or subsets of the machine 202 to control inputs. This mapping may be analytical or based on a solution to an optimization problem. In response to applying the control strategy, the current measurement of the current state may be transformed into the current control input based on the current values ​​of control parameters in the set of control parameters of feedback controller 210. As used herein, control parameters may be (i) the gain of feedback controller 210 and / or (ii) parameters of the physical structure of machine 202. For example, when feedback controller 210 corresponds to a PID controller, the set of control parameters includes the proportional gain, integral gain, and derivative gain of the PID controller. For example, parameters of the physical structure of machine 202 may include the mass of a robot arm or the friction between a vehicle's tires and the road. It is important to note that control parameters should not be confused with control inputs that are outputs of the control strategy. According to the implementation method, the current value of the control parameter can be generated by the Kalman filter 212. For example, the Kalman filter 212 that generates the control parameter is as follows: Figure 2B As described in the detailed description.

[0064] Figure 2B An example of a Kalman filter 212 for generating control parameters according to some embodiments of this disclosure is illustrated. (In conjunction with...) Figure 2A To explain Figure 2B According to an embodiment, the Kalman filter 212 can be configured to iteratively update the state of the feedback controller 210. According to an embodiment, the state of the feedback controller 210 is defined by control parameters. Therefore, the goal of the Kalman filter 212 is to iteratively generate the control parameters. In an example embodiment, the Kalman filter 212 can iteratively generate the control parameters using a prediction model 218 and a measurement model 220. For example, the prediction model 218 and the measurement model 220 can be manually designed.

[0065] To generate control parameters in the current iteration (e.g., at time step k), the prediction model 218 can be configured to predict the values ​​of the control parameters using prior knowledge 218a of the control parameters. For example, the prior knowledge 218a of the control parameters can be generated in a previous iteration (e.g., at time step k-1). The prior knowledge 218a of the control parameters can be a joint probability distribution (or Gaussian distribution) over the control parameters in previous iterations. The joint probability distribution over the control parameters in previous iterations can be derived from the mean θ calculated in the previous iterations. k-1|k-1 Sum of variances (or covariances) P k-1|k-1Definition. For example, the joint probability distribution of previous iterations may be generated based on the joint probability distribution generated in the previous iteration (e.g., at time step k-2) and / or the model of the feedback controller 210 (e.g., transition model 116).

[0066] According to the implementation, the values ​​of the control parameters predicted in the current iteration can also be a joint probability distribution 218b (or a Gaussian distribution 218b). For example, when the prediction model 218 is configured to predict multiple control parameters, the output of the prediction model 218 can be a joint probability distribution 218b. Alternatively, when the prediction model 218 is configured to predict a single control parameter, the output of the prediction model 218 can be a Gaussian distribution 218b. For example, the joint probability distribution 218a can be derived from the mean θ calculated in the current iteration. k|k-1 Sum of variances (or covariances) P k|k-1 Definition. For example, while predicting a single control parameter, the Gaussian distribution output by prediction model 218 is as follows: Figure 2C As shown.

[0067] Figure 2C An example is illustrated by a Gaussian distribution 224 representing a specific control parameter according to some embodiments of this disclosure. (In conjunction with...) Figure 2B To explain Figure 2C The Gaussian distribution 224 can be predicted by prediction model 218. For example, the Gaussian distribution 224 can correspond to the Gaussian distribution 218b. The Gaussian distribution 224 can be predicted by the mean 228 (e.g., the mean θ). k|k-1 ) and variance 226 (e.g., variance P) k|k-1 The distribution is defined as follows: mean 228 defines the center location of the Gaussian distribution 224, and variance 226 defines the measure of the spread (or width) of the Gaussian distribution 224.

[0068] Return to reference Figure 2B According to the implementation, prediction model 218 can withstand process noise. As used herein, process noise can be an assumption defining how quickly a control parameter changes over time (e.g., assumption 118). Process noise can control how quickly a control parameter changes over time within a variance defined by the process noise. Process noise can be artificially designed. For example, when prediction model 218 is subjected to process noise, prediction model 218 can output multiple Gaussian distributions for a particular control parameter, wherein the multiple Gaussian distributions can have different variances limited to the variance of the process noise. For example, the multiple Gaussian distributions output by prediction model 218 for a particular control parameter are as follows: Figure 2D As shown.

[0069] Figure 2D Examples of Gaussian distributions 230, 232, and 234 with different variances according to some embodiments of this disclosure are illustrated. (In conjunction with...) Figure 2B To explain Figure 2D Gaussian distributions 230, 232, and 234 can be predicted by prediction model 218. Each of these Gaussian distributions 230, 232, and 234 may have different variances from each other, but the mean 236 of the Gaussian distributions 230, 232, and 234 may be constant. A Gaussian distribution with (i) smaller variance and (ii) the highest probability among other Gaussian distributions of mean 236 can be a correct prediction of the control parameter. For example, Gaussian distribution 230 can represent a correct prediction of the control parameter.

[0070] Return to reference Figure 2B Thus, the prediction model 218, subjected to process noise, can be configured to predict the values ​​of the control parameters as outputs of the joint probability distribution 218b (or Gaussian distribution 218b). Once the prediction model 218 outputs the joint probability distribution 218b in the current iteration, the measurement model 220 can be configured to update the predicted values ​​of the control parameters based on the measurement sequence 220a to generate the current values ​​of the control parameters. In an example implementation, the measurement sequence 220a can be a measurement sequence received by the transceiver 206. For example, the measurement sequence 220a used by the measurement model 220 is as follows: Figure 2E As shown.

[0071] Figure 2E The evolution 238 of the state of machine 202 over time is illustrated according to some embodiments of this disclosure. (In conjunction with...) Figure 2A and Figure 2B To explain Figure 2E For example, the evolution 238 of the state of machine 202 can be obtained from one or more sensors installed in machine 202. For example, if the current time is t0, the measurement model 220 can use N state measurements 240 to update the predicted values ​​of the control parameters. The N state measurements 240 may correspond to a measurement sequence 220a. The N state measurements 240 may include measurements from past times t0. -N Related measurements Start with a measurement associated with the current time t0. The measurement is now complete. Here, in... Figure 2E In this context, consider a measurement model 220 that uses N state measurements 240 for only one state. However, when the machine 202 is associated with more than one state, the measurement model 220 can use N measurements for all states within the same time frame.

[0072] Return to reference Figure 2BSome implementations are based on the understanding that measurement sequences 220a obtained from one or more sensors may be inaccurate due to defects in the sensors, other noise (e.g., random noise), etc. Therefore, the measurement model 220 may be subjected to measurement noise. As used herein, measurement noise is a noise model that can be used to reduce inaccuracies in measurement 220a caused by defects in the sensors, other noise, etc. For example, measurement noise can be artificially designed.

[0073] In an example implementation, the measurement model 220 subjected to measurement noise can be configured to update the predicted values ​​of the control parameters based on the measurement sequence 220a. To update the predicted values, the measurement model 220 can be configured to calculate a model mismatch between the measurement sequence 220a and the model of machine 202 (e.g., model 106). Furthermore, the measurement model 220 can be configured to simulate the evolution of machine 202 (e.g., state measurements) using the predicted control values, the model of machine 202, and the calculated model mismatch. For example, the simulation evolution (i.e., state measurements) may resemble the measurement sequence 220a. Additionally, the measurement model 220 can be configured to evaluate the simulation evolution of machine 202 according to performance target 220b to generate current values ​​of the control parameters. Since the current values ​​of the control parameters are generated based on an evaluation that may resemble the simulation evolution of measurement sequence 220a, the current values ​​of the control parameters can describe the measurement sequence 220a. For example, in Figure 2F The figure shows a measurement model 220 for updating the predicted values ​​of the control parameters.

[0074] Figure 2F A schematic diagram 242 illustrates the predicted values ​​of updated control parameters according to some embodiments of this disclosure. (In conjunction with...) Figure 2B To explain Figure 2F Schematic diagram 242 includes a predicted Gaussian distribution 244, control parameters 246 (or values ​​of the control parameters), and an updated Gaussian distribution 248. For example, the predicted Gaussian distribution 244 could be derived from the mean θ. k|k-1 and variance P k|k-1 A Gaussian distribution 218b is defined. For example, control parameter 246 can be a control parameter that can be used to control machine 202 relative to performance target 220b to achieve a specific trajectory. Furthermore, control parameter 246 can be derived from a predicted Gaussian distribution 244, the measurement of which has a probability close to zero. To this end, measurement model 220 can update the predicted Gaussian distribution 244 such that the predicted Gaussian distribution 244 moves to be closer to the updated Gaussian distribution 248. In other words, measurement model 220 can update the mean and variance associated with the predicted Gaussian distribution 244 to the mean (e.g., mean θ) corresponding to the updated Gaussian distribution 248. k|k ) and variance (e.g., variance P) k|k ).

[0075] Return to reference Figure 2B Thus, measurement model 220 can update the predicted values ​​of control parameters based on measurement sequence 220a to generate current values ​​of control parameters according to performance target 220b. In the example implementation, performance target 220b may differ from the control strategy of feedback controller 210 used to determine control input. This is advantageous because the control strategy has a structure constrained by its real-time application; for example, the cost function often needs to be differentiable and convex so that the cost function is suitable for numerical optimization. However, performance target 220b may change at different control times according to the same control strategy.

[0076] According to the implementation method, the measurement model 220 can output the current value of the generated control parameters as a joint probability distribution 220d (or Gaussian distribution 220d), whose defined quantity 220c, such as the mean θ, is... k|k and variance P k|k The Kalman filter 212 can repeat the process in the next iteration 222 (e.g., at time step k+1) to generate control parameters.

[0077] In this way, the Kalman filter 212 can iteratively generate control parameters, which can be used to iteratively update the state of the feedback controller 210. The updated state of the feedback controller 210 can then be used to determine the control input for controlling the operation of the machine 202. Because the Kalman filter 212 generates the control parameters iteratively using the joint probability distribution of the control parameters (e.g., prior knowledge 218a), rather than recalculating the control parameters using the entire data history, the Kalman filter 212 can efficiently generate control parameters for controlling the operation of the machine 202. Furthermore, the amount of data to be stored in the memory of the system 200 can be reduced, since the system 200 may only need to store the prior knowledge of the control parameters, rather than the entire data history. Therefore, the memory requirements of the system 200 can be reduced.

[0078] Some implementations are based on the understanding that when one or more control parameters depend on another control parameter of the same type, the Kalman filter 212 should collectively calibrate the control parameters. For example, in a PID controller, the gain should be collectively calibrated because the gains of the PID controller are interdependent.

[0079] Calibrating these interdependent control parameters can often be challenging because the interdependence can introduce additional variables during calibration. In such cases, the Kalman filter 212 can be used as follows: Figure 3 The configuration is as described in the detailed description.

[0080] Figure 3A block diagram illustrating a Kalman filter 212 for calibrating multiple interdependent control parameters according to some embodiments of the present disclosure is shown. (In conjunction with...) Figure 2B To explain Figure 3 According to an implementation, when the control parameters correspond to multiple interdependent control parameters, the Kalman filter 212 can be configured to adjust the Kalman gain 300 for calibrating the control parameters. For example, if the control parameters include one or more control parameters that depend on other control parameters among the same control parameters, the control parameters may be referred to as multiple interdependent control parameters. As used herein, "adjusting the Kalman gain 300" may indicate setting different weights for the control parameters. To calibrate multiple interdependent control parameters, the Kalman filter 212 may adjust the Kalman gain 300 such that one or more control parameters that depend on other control parameters may be given more weight relative to the other control parameters. Furthermore, the Kalman filter 212 may be configured to simultaneously update the control parameters using the measurement model 220 for outputting calibrated interdependent control parameters 302. For example, the Kalman filter 212 may be configured as follows: Figure 4A and / or Figure 4B The Kalman gain of 300 was calculated as described in the detailed description.

[0081] Figure 4A Method 400a performed by Kalman filter 212 according to some embodiments of this disclosure is illustrated. (In conjunction with...) Figure 2B To explain Figure 4A When Kalman filter 212 corresponds to an unscented Kalman filter (UKF), Kalman filter 212 can perform method 400a. In step 402, Kalman filter 212 can be configured to compute the sigma point using a joint probability distribution 402a over control parameters and a preset weight set 402b. For example, the joint probability distribution 402a can be derived from the mean θ. k-1 and variance P k-1|k-1 Prior knowledge of the defined control parameters 218a. The preset weight set 402b can be the weights w selected by the designer of the Kalman filter 212. c,i and w a,i For example, the sigma point can be mathematically calculated using equation (1).

[0082]

[0083] Where L represents the number of control parameters, and A represents the number of control parameters. i This is the i-th column of matrix A. Matrix A can be decomposed using the Cholesky method from the variance P... k-1|k-1 =AA TCalculation. In the example implementation, these calculated sigma points can be an implementation of control parameters. Additionally, in step 402, the Kalman filter 212 can calculate the mean of the sigma points. For example, the mean of the sigma points can be mathematically calculated using equation (2).

[0084]

[0085] In step 404, the Kalman filter 212 can use the prediction model 218 to predict the mean and variance of the control parameters. For example, the mean and variance of the control parameters can be mathematically predicted using equations (3a) and (3b), respectively.

[0086] θ k|k-1 =θ k-1 Equation (3a)

[0087]

[0088] The notation C θ It is the variance matrix of the process noise.

[0089] In step 406, Kalman filter 212 can simulate the evolution of the machine (e.g., machine 202) for the calculated sigma points. To simulate the machine's evolution, in step 408, Kalman filter 212 can use measurement sequence 408a and machine model 408b to calculate model mismatch. For example, measurement sequence 408a may correspond to measurement 220a. For example, machine model 408b may be model 106 (in...). Figure 1 (See the explanation below). For example, model mismatch can be mathematically calculated using equation (4).

[0090]

[0091] Furthermore, in step 406, the Kalman filter 212 can simulate the evolution of the machine using the calculated model mismatch and the machine model 408 for the calculated sigma point. For example, the evolution of the machine can be a hypothetical evolution of the machine if the machine is controlled with control parameters defined by the calculated sigma point. For example, the evolution of the machine can be mathematically simulated using equation (6).

[0092]

[0093] Among the marks It is determined by the control parameter θ i The defined control strategy (or control law), denoted by w k This indicates a model mismatch, indicated by the symbol. Indicates time t -N Machine status, markings This represents the hypothetical state evolution used to achieve the control parameters defined by the sigma point.

[0094] In step 410, the Kalman filter 212 can evaluate the simulated evolution of each sigma point relative to performance target 410a. For example, performance target 410a can be performance target 220b. For example, evaluating each sigma point relative to performance target 410a can be performed mathematically using equation (7).

[0095]

[0096] The function “h” can be performance target 410a.

[0097] In step 410, the Kalman filter 212 may further calculate the weighted mean of the evaluation results. For example, the weighted mean of the evaluation results may be calculated mathematically using equation (8).

[0098]

[0099] Furthermore, in step 410, the Kalman filter 212 can use the evaluation results and the weighted average of the evaluation results to calculate the Kalman gain. For example, the Kalman gain can be mathematically calculated using equation (9).

[0100]

[0101] Among the marks mark Notation C v This is a measurement of noise. In some implementations, this can be achieved by modifying the preset weighting parameter w. c,i To adjust the Kalman gain K k .

[0102] In step 412, Kalman filter 212 can update the joint probability distribution of the control parameters using the evaluation results and the Kalman gain. For example, in step 412, Kalman filter 212 can update the mean and variance of the joint probability distribution of the control parameters predicted in step 404. For example, the mean and variance are mathematically updated using equations (10a) and (10b), respectively.

[0103] θ k =θ k-1 +K k (y k -h(θ k|k-1 Equation (10a)

[0104]

[0105] Furthermore, the Kalman filter 212 can output a value derived from the mean θ.k and variance P k|k The defined updated joint probability distribution is used as the control parameter for controlling the machine.

[0106] Figure 4B Method 400b performed by Kalman filter 212 according to some other embodiments of this disclosure is illustrated. (In conjunction with...) Figure 2B To explain Figure 4B When Kalman filter 212 corresponds to an Extended Kalman Filter (EKF), Kalman filter 212 can perform method 400b. In step 414, Kalman filter 212 can use the joint probability distribution 414a of the control parameters to predict the mean and variance of the control parameters. For example, the joint probability distribution 414a can be derived from the mean θ k-1 and variance P k-1|k-1 Prior knowledge 218a of the defined control parameters. According to an implementation, the prediction model 218 of the Kalman filter 212 can be configured to predict the mean and variance of the control parameters using a joint probability distribution 414a. For example, when the Kalman filter 212 corresponds to an EKF, the prediction model 218 can be designed (or declared) such that the prediction model 218 predicts the mean and variance of the control parameters that remain fixed (or constant) within the variance defined by the process noise. In other words, when the Kalman filter 212 corresponds to an EKF, the prediction model 218 can be designed such that the prediction model 218 predicts control parameters that do not change relative to the control parameters represented by the joint probability distribution 414a. In this case, the prediction model 218 can be an identity model. For example, the identity model can be an identity matrix. For example, the predicted mean and predicted variance of the control parameters can be as in equations (11a) and (11b), respectively.

[0107] θ k|k-1 =θ k-1 Equation (11a)

[0108] P k|k-1 =P k-1|k-1 +C θ Equation (11b)

[0109] The notation C θ It is the variance matrix of the process noise.

[0110] In step 416, Kalman filter 212 can simulate the evolution of the machine (e.g., machine 202) against the predicted mean. To simulate the machine's evolution, in step 418, Kalman filter 212 can use measurement sequence 418a and machine model 418b to calculate model mismatch. For example, measurement sequence 418a may correspond to measurement 220a. For example, machine model 418b may be model 106 (in...). Figure 1(See explanation below). For example, model mismatch can be mathematically calculated as:

[0111] Furthermore, in step 416, the Kalman filter 212 can simulate the evolution of the machine against the predicted mean using the calculated model mismatch and the machine's model 418b. For example, the evolution of the machine can be a hypothetical evolution of the machine if it is controlled with control parameters defined by the predicted mean. For example, the evolution of the machine can be mathematically simulated using equation (12).

[0112]

[0113] Among the marks It is determined by the control parameter θ k|k-1 Defined control strategy, notation w k This indicates a model mismatch, indicated by the symbol. Indicates time t -N Machine status, markings Represents the control parameter θ k|k-1 The hypothetical state evolution.

[0114] In step 420, the Kalman filter 212 may evaluate the simulated evolution of the predicted mean relative to performance target 420a. For example, performance target 420a may be performance target 220b. For example, evaluating the predicted mean relative to performance target 420a may be performed mathematically using equation (13).

[0115]

[0116] In step 422, the Kalman filter 212 can calculate the gradient of the performance target 420a relative to the predicted mean. In an example implementation, the gradient can define how the control parameters should be adjusted relative to the performance target 420a to optimize the performance of the closed-loop machine. For example, the gradient of the performance target 420a relative to the predicted mean can be mathematically calculated using equation (14).

[0117]

[0118] In step 422, the Kalman filter 212 may further use the gradient of the calculated performance target 420a to calculate the Kalman gain. For example, the Kalman gain may be calculated mathematically using equation (15).

[0119]

[0120] The notation C v This is the measurement noise of the Kalman filter 212.

[0121] In step 424, Kalman filter 212 can update the joint probability distribution of the control parameters using the Kalman gain, the evaluation result, and the gradient of performance target 420a. For example, in step 424, Kalman filter 212 can update the mean and variance of the joint probability distribution of the control parameters predicted in step 414. For example, the mean and variance can be mathematically updated using equations (16a) and (16b), respectively.

[0122]

[0123] P k|k =(IK k H k )P k|k-1 Equation (16b).

[0124] Furthermore, the Kalman filter 212 can output a value derived from the mean θ. k and variance P k|k The defined updated joint probability distribution is used as the control parameter for controlling the machine.

[0125] Figure 5 A method 500 for calibrating state-related control parameters according to some embodiments of this disclosure is illustrated. (In conjunction with...) Figure 2A and Figure 2B To explain Figure 5 Some implementations are based on the understanding that the set of control parameters for the feedback controller 212 may include at least some control parameters that depend on the state of the machine 202. For example, the friction of a vehicle's tires may depend on the vehicle's speed. Hereinafter, "at least some control parameters that depend on the state of the machine" and "state-dependent control parameters" are used interchangeably to mean the same thing. If the set of control parameters includes state-dependent control parameters, calibrating the control parameters may be challenging because these state-dependent control parameters may change continuously relative to the state of the machine. In these implementations, the Kalman filter 212 may perform a method 500 for calibrating the state-dependent control parameters.

[0126] In step 502, the Kalman filter 212 obtains the algebraic relationship between the state-dependent control parameters and the state of the machine 202. In an example embodiment, the algebraic relationship between the state-dependent control parameters can be obtained as a linear combination of the state-dependent control parameters and basis functions. For example, the basis functions can be functions of the state of the machine 202. The basis functions can be defined by one or more state-dependent regions. For example, the linear combination of the state-dependent control parameters and basis functions is shown in equation (17).

[0127]

[0128] Where the notation Θ represents the state-dependent control parameter of the feedback controller 210, the notation φ(x) is the basis function of the state (x) of the machine 202, and c(x)≤ψ defines the first state-dependent region. The notation represents the state-related control parameters of the first state-related region. This represents the state-dependent control parameters of the second region. The notation ψ represents the boundary separating the first and second state-dependent regions. In other words, and These are the coefficients of the basis functions.

[0129] In step 504, Kalman filter 212 can predict state-dependent control parameters within a variance defined by process noise based on an algebraic relationship with the state of machine 202. For example, prediction model 218 of Kalman filter 212 can be designed (or declared) such that prediction model 218 predicts state-dependent control parameters within a variance defined by process noise based on an algebraic relationship with the state of machine 202. For example, when the algebraic relationship of the state-dependent control parameters corresponds to a linear combination of the state-dependent control parameters and basis functions, prediction model 218 can be configured to check whether the basis functions are defined by more than one state-dependent region. If the basis functions are not defined by more than one state-dependent region, prediction model 218 can be configured to predict the coefficients of the basis functions.

[0130] If the basis functions are defined by more than one state-dependent region, the prediction model 218 can be configured to examine whether the boundary ψ of the separated state-dependent regions is fixed or variable. If the boundary ψ is fixed, the prediction model 218 can be configured to predict the coefficients of the basis functions within the variance defined by the process noise. and If the boundary v is variable, then prediction model 218 can be configured to predict the coefficients of the basis functions within the variance defined by the process noise. and And the boundary ψ.

[0131] In block 506, Kalman filter 212 can update state-dependent control parameters based on an algebraic relationship with the state of machine 202 for calibration purposes. For example, measurement model 220 of Kalman filter 212 can update state-dependent control parameters based on an algebraic relationship with the state of machine 202. For example, if the basis functions are not defined by more than one state-dependent region, measurement model 220 can be configured to update the coefficients of the basis functions (e.g., predicted coefficients). For example, if (i) the basis functions are defined by more than one state-dependent region and (ii) the boundary ψ is fixed, measurement model 220 can be configured to update the coefficients of the basis functions (e.g., predicted coefficients). and This is used to calibrate the state-dependent control parameter Θ. For example, if (i) the basis functions are defined by more than one state-dependent region and (ii) the boundary ψ is variable, then the measurement model 220 can be configured to update the coefficients of the basis functions. and And the boundary ψ is used to calibrate the state-dependent control parameter Θ.

[0132] Figure 6A A block diagram illustrating a system 200 for controlling the operation of a machine 202 according to some other embodiments of the present disclosure is shown. (In conjunction with...) Figure 2A and Figure 2B To explain Figure 6A Some implementations are based on the understanding that online updates of control parameters can destabilize the operation of machine 202. For example, when a control law or control strategy is represented by a differential equation (e.g., an ordinary differential equation (ODE)) with control parameters, changes (updates) to the control parameters may disrupt the equilibrium stability of the differential equation. Therefore, system 200 may also include a safety check module 600. For example, safety check module 600 may be specifically implemented within at least one processor 204. Alternatively, safety check module 600 may be a software module stored in memory that can be executed by at least one processor 204. According to an implementation, safety check module 600 may be configured to perform a safety check method using the values ​​of the control parameters generated by Kalman filter 212 to ensure the safe operation of machine 202. For example, the safety check method performed by safety check module 600 may be as follows: Figure 6B As described in the detailed description.

[0133] Figure 6B Examples of security check methods performed by the security check module 600 according to some embodiments of this disclosure are illustrated. (In conjunction with...) Figure 6A To explain Figure 6B In step 602, the security check module 600 can obtain the values ​​(e.g., current values) of the control parameters generated by the Kalman filter 212.

[0134] In step 604, the safety check module 600 may check whether the values ​​of the control parameters generated by the Kalman filter 212 meet the safety check according to the control strategy. In other words, when the feedback controller 210 controls the machine 202 according to the control strategy updated with the control parameters generated by the Kalman filter 212, the safety check module 600 may check whether the values ​​of the control parameters of the Kalman filter 212 provide stable control of the machine 202. To check whether the control parameters generated by the Kalman filter 212 meet the safety check, the safety check module 600 may use previous states, measurement sequences (e.g., measurement sequence 220a), and / or a model of the machine 202 (e.g., model 106).

[0135] For example, a safety check is satisfied when a Lyapunov function exists for the control strategy updated with control parameters generated by Kalman filter 212. In some embodiments, the existence of a Lyapunov function can be proven by solving an optimization problem aimed at finding the Lyapunov function. In one embodiment, a safety check is satisfied when, while controlling machine 202 with feedback controller 210 updated with control parameters generated by Kalman filter 212, the state of machine 202 is reduced in cost relative to the performance target over the entire state history and measurement sequence. In another embodiment, a safety check is satisfied when, while controlling machine 202 with feedback controller 210 updated with control parameters generated by Kalman filter 212, the state of machine 202 is made closer to the origin (or boundedness). In yet another embodiment, a safety check is satisfied when a combination of the reduced cost of the state of machine 202 and the closeness of the state of machine 202 to the origin is achieved. Therefore, a safety check may include one or a combination of the reduced cost of the state of machine 202 and the closeness of the state of machine 202. For example, a safety check module 600 performing the safety check is as follows: Figure 6C As described in the detailed description.

[0136] Figure 6C The evolution of the state of machine 202 according to some other embodiments of this disclosure is illustrated. (In conjunction with...) Figure 6A To explain Figure 6C .like Figure 6C As shown, state 604a of machine 202 can be within a radius around the origin. The process begins within the origin. After machine 202 evolves, state 604a arrives at a radius around the origin. Inside. In this example, radius Use defined functions The largest eigenvalue σ of the positive definite matrix P max and the smallest eigenvalue σ min Confirmed. Especially if... This ensures that the state 604a of machine 202 remains bounded.

[0137] Return to reference Figure 6BWhen the value of the control parameters generated by the Kalman filter 212 does not meet the safety check, in step 608, the safety check module 600 may maintain the control parameters of the feedback controller 210. In other words, when the safety check fails, the safety check module 600 may not update the control parameters of the feedback controller 210 with the control parameters generated by the Kalman filter 212. Furthermore, the Kalman filter 212 may be configured to iteratively generate new values ​​for the control parameters until the safety check is met. It is worth noting that even if the control parameters generated by the Kalman filter 212 do not meet the safety check, the Kalman filter 212 should not be restarted.

[0138] When the value of the control parameters generated by the Kalman filter 212 meets the safety check, in step 606, the safety check module 600 can update the control parameters of the feedback controller 210 using the control parameters generated by the Kalman filter 212. In this way, while the control parameters of the feedback controller 210 are updated online, the safety check module 600 can ensure the stability of the control.

[0139] Return to reference Figure 6A Once the control parameters of the feedback controller 210 are updated, the feedback controller 210 can be configured to determine the control input for controlling the operation of the machine 202 by applying a control strategy updated with the control parameters generated by the Kalman filter 212.

[0140] Some implementations are based on the understanding that online updates of the control parameters of the feedback controller 210 can improve control quality, but at the cost of additional challenges. For example, online updates of control parameters during operation of machine 202 can introduce control discontinuities. Some implementations are based on the understanding that control discontinuities can be addressed by forcing control commands to satisfy constraints on the operation of machine 202. To this end, the feedback controller 210 can be configured to determine the control input (e.g., the current control input) using control commands that satisfy constraints on the operation of machine 202. In other words, the feedback controller 210 can be configured to determine control inputs subject to constraints on the operation of machine 202, thereby addressing control discontinuities. For example, control commands satisfy constraints on the operation of machine 202 when control parameters (e.g., the current control parameters) satisfy control theory properties. For example, control theory properties can be specified by the designer.

[0141] Figure 7 A list of performance targets used by the Kalman filter 212 according to some embodiments of this disclosure is illustrated. (In conjunction with...) Figure 2B To explain Figure 7 Performance target 220b may include:

[0142] The cost function 700 defines the deviation between the machine's current state and its target state, y. k=x ref ,h(θ k )=x k ;

[0143] Cost function 702 defines the deviation between the control input and the reference control input, y. k =u ref ,h(θ k )=u k ;

[0144] Cost function 704, c1, for states exceeding a specific operating range, y k =0,

[0145] Cost function 706, c2, for control inputs exceeding a specific operating range, y k =0,

[0146] Cost function 708, c3, penalizes oscillations in one or more states, for example, y k =0, h(θ) k ) = c3·#3 seconds sign change (x) k );

[0147] Cost function 710, c4, penalizes oscillations of one or more control inputs, e.g., y k =0, h(θ) k ) = c4·#3 seconds sign change (u k );

[0148] Cost function 712, c5, if the reference / target state overshoots a specific value, for example, y k =0,

[0149] Cost function 714, c6, if the control input exceeds a specific operating range within a specific time period, for example, y k =0,

[0150] Cost function 716, if the state changes between time steps, for example, y k =0, h(θ) k )=||x k+1 -x k ||2;

[0151] Cost function 718, if the control input varies between time steps, for example, y k =0, h(θ) k )=||u k+1-u k ||2;

[0152] Cost function 720, c7, if the state changes faster than a certain value between time steps, for example, y k =0,

[0153] Cost function 722, c8, if the control input changes faster than a certain value between time steps, for example, y k =0,

[0154] Alternatively or concurrently, performance objective 200b may include a cost function of specific control parameters (rather than state or control inputs), for example, y k =θ nom ,h(θ k )=θ k , where θ nom Define the nominal value of any or all control parameters.

[0155] In some implementations, measurement model 220 may be configured to select one from a list of performance targets based on one or a combination of the machine's state and the state of the environment surrounding the machine. For example, based on the machine's state and / or the control parameters used to control the machine, measurement model 220 may identify a cost function from the performance target list that significantly degrades control performance compared to other cost functions in the list. Furthermore, measurement model 220 may select the identified cost function as performance target 220a. For example, if cost function 700 significantly degrades control performance compared to other cost functions in the performance target list, measurement model 220 may select cost function 700 as performance target 200a. Additionally, measurement model 220 may update the control parameters by optimizing (e.g., minimizing) cost function 700 while generating the control parameters.

[0156] Some implementations are based on the understanding that when the control parameters are independent of the machine's state, the limit of the uncertainty of the control parameters can be predetermined. Hereinafter, "control parameters independent of the machine's state" and "state-independent control parameters" are used interchangeably to refer to the same thing. In these implementations, the Kalman filter 212 can select one or a combination of performance targets, measurement noise, or process noise based on this limit. Furthermore, some possible state-dependent and state-independent control parameters include... Figure 8A As shown.

[0157] Figure 8A A list of feedback controllers used in system 200 according to some embodiments of this disclosure is illustrated. (In conjunction with...) Figure 2A To explain Figure 8AFor the PID controller 800, the Kalman filter 212 can estimate the proportional gain k. P Integral gain k I and differential gain k D The control parameters. For example, the control parameters of the PID controller 800 are...

[0158]

[0159] For the optimal controller 802, the Kalman filter 212 can estimate the control parameters, including the weights θ of the cost function. For example, the optimal controller 802 is...

[0160]

[0161] Make

[0162]

[0163] x0=x(0)

[0164] Where the constraint c(x) k ,u k The optimal control cost function is given by x(0) ≤ 0 and the initial machine state is x(0).

[0165]

[0166] The control parameters are:

[0167]

[0168] The function vec vectorizes matrices Q and R.

[0169] For the state feedback controller 804, the Kalman filter 212 can estimate control parameters including the gain.

[0170] u k =Kx k ,

[0171] The control parameters are:

[0172] θ = vec(K).

[0173] For the dynamic output feedback controller 806, which includes observer dynamics,

[0174]

[0175] in It is the observer's state estimate, C is the output matrix that maps the machine's state to the measured output, and the Kalman filter 212 can estimate control parameters including the feedback gain K and the observer gain (also known as the Luenberger gain) L.

[0176]

[0177] For the sliding mode controller 808, for example, with a control law

[0178] u k =K(r,x) k )-ρ·sign(r T x k ),

[0179] Where s k =r T x k It is a sliding surface, K(r,x) k It depends on the machine and ensures that the machine moves along the surface. k =r T x k The control law for sliding. Here, the control parameters are...

[0180]

[0181] For a neural network-based controller 810, for example, one with a control law

[0182] u k =σ y (W y σ x (W x x k )),

[0183] Where σ y and σ x These are activation functions; the control parameters estimated by the Kalman filter 212 include the weights of the neural network.

[0184]

[0185] For H ∞ The controller 812 and Kalman filter 212 estimate the filter coefficients of the pre-compensator and post-compensator used to determine the trade-off between performance and robustness. For example, H ∞ Controller 812 is shown in Figure 8B middle.

[0186] Figure 8BA block diagram illustrating a feedback controller 210 for controlling a machine 202 according to some embodiments of the present disclosure is shown. In an example embodiment, the feedback controller 210 may include H ∞ The controller 812, the front compensator 812b, and the rear compensator 812a are described. In the frequency domain, the front compensator 812a and the rear compensator 812b are...

[0187]

[0188] Where the integer n pre,nom n pre,denom n post,nom n post,denom Define the order of the pre-compensator 812a and the post-compensator 812b. The control parameter 812c estimated by the Kalman filter 212 is...

[0189]

[0190] Here, H ∞ The controller 812 can be calculated by minimizing the maximum amplitude in the frequency response of the machine 202.

[0191] Figure 9 A schematic diagram of a system 200 for controlling an electric motor 900 according to some embodiments of the present disclosure is illustrated. In this example, the feedback controller of the system 200 may be a PID controller for controlling the electric motor 900. The system 200 may receive a position or speed signal 904 (e.g., feedback signal 216) from a sensor 902 mounted on the electric motor 900. Furthermore, the system 200 may receive a desired position or speed signal 906 (e.g., a control command) to calculate an error signal 908. Additionally, the system 200 may calibrate the proportional gain k. P Integral gain k I and differential gain k D Furthermore, system 200 can determine control input 910 using error signal 908 by applying a control strategy. For example, the control strategy can be three-component (e.g., by adjusting the error with the calibrated proportional gain k). P The proportional component 912a obtained by multiplication is obtained by integrating the error and then combining the integral error with the calibrated integral gain k. I The integral component 912b obtained by multiplication, and the differential gain k obtained by differentiating the error with respect to time and then dividing the derivative with the calibration. D The sum of the differential components 912c obtained by multiplication. In addition, the system 200 can submit the determined control input 910 to the electric motor for controlling the electric motor 900.

[0192] Figure 10A schematic diagram of a system 200 for controlling a vehicle 1000 according to some embodiments of the present disclosure is illustrated. In this example, the system 200 can control the vehicle 1000 such that the vehicle 1000 remains in the center of lane 1002. The system 200 can receive position and / or speed signals from sensor 1004. The system 200 can further calibrate one or more control parameters associated with a control strategy. For example, the control parameter could be the friction between the tires of the vehicle 1000 and the road. Furthermore, the system 200 can determine a control input by applying a control strategy updated to the calibrated one or more control parameters. For example, the system 200 can determine a control input such that the control input keeps the vehicle 1000 in the center of lane 1002. For example, the determined control input could be a steering angle 1006, which causes the vehicle 1000 to move in a vehicle trajectory 1008 converging in the center of lane 1002.

[0193] The above description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the above description of the exemplary embodiments will provide those skilled in the art with a feasible description for implementing one or more exemplary embodiments. Various changes to the function and arrangement of the elements will be conceived without departing from the spirit and scope of the subject matter set forth in the appended claims.

[0194] Specific details are set forth in the foregoing description to provide a thorough understanding of the embodiments. However, it will be understood by those skilled in the art that embodiments may be practiced without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form to avoid obscuring the embodiments with unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments. Furthermore, similar reference numerals and designations in the various figures indicate similar elements.

[0195] Furthermore, the various embodiments can be described as processes, depicted as flowcharts, data flow diagrams, structural diagrams, or block diagrams. Although flowcharts can describe operations as a sequential process, many operations can be performed in parallel or simultaneously. Additionally, the order of operations can be rearranged. A process may terminate upon completion of its operations, but may have additional steps not discussed or included in the figures. Moreover, not all operations in any specifically described process may appear in all embodiments. A process may correspond to a method, function, program, subroutine, subroutines, etc. When a process corresponds to a function, the termination of the function may correspond to the function returning to the calling function or the main function.

[0196] Furthermore, implementations of the disclosed subject matter can be carried out, at least partially, manually or automatically. They can be performed, or at least assisted by, using machines, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, for manual or automatic implementation. When implemented in software, firmware, middleware, or microcode, program code or code segments that perform the required tasks can be stored in a machine-readable medium. The processor can then execute the required tasks.

[0197] The various methods or processes outlined herein can be encoded as software that can be executed on one or more processors employing any of a variety of operating systems or platforms. Furthermore, such software can be written using any of a variety of suitable programming languages ​​and / or programming or scripting tools, and can also be compiled into executable machine language code or intermediate code that executes on a framework or virtual machine. Typically, in various implementations, the functionality of program modules can be combined or distributed as needed.

[0198] Embodiments of this disclosure can be specifically implemented as a method, examples of which have been provided. Actions performed as part of this method can be ordered in any suitable manner. Thus, embodiments can be constructed that perform actions in a different order than those shown, which may include performing some actions simultaneously, although they are shown as sequential actions in the illustrative embodiments. Although this disclosure has been described with reference to specific preferred embodiments, it will be understood that various other adjustments and modifications can be made within the spirit and scope of this disclosure. Therefore, aspects of the appended claims cover all such variations and modifications that fall within the true spirit and scope of this disclosure.

Claims

1. A system for controlling the operation of a machine for performing a task, the system comprising: A transceiver configured to submit a sequence of control inputs to the machine and receive feedback signals including corresponding measurement sequences, wherein each measurement indicates the state of the machine as a result of the corresponding control input; A feedback controller configured to, at each control step, transform a current measurement of the current state of the machine into a control strategy for controlling the machine by applying current values ​​of control parameters in a set of control parameters of the feedback controller, and to determine the current control input based on the feedback signal including the current measurement; and A Kalman filter, configured to iteratively update the state of the feedback controller defined by the control parameters using a prediction model for predicting the values ​​of the control parameters subjected to process noise and a measurement model for updating the predicted values ​​of the control parameters based on the measurement sequence subjected to measurement noise, to generate the current values ​​of the control parameters associated with the measurement sequence according to a performance target. The measurement model is also configured to select from different performance targets based on one or a combination of the machine's state and the state of the environment surrounding the machine.

2. The system according to claim 1, wherein, The Kalman filter is also configured to adjust the Kalman gain for calibrating multiple interdependent control parameters.

3. The system according to claim 1, wherein, The prediction model is an identity model configured to predict that the control parameters remain constant within the variance defined by the process noise.

4. The system according to claim 1, wherein, The prediction model is configured to predict at least some control parameters within a variance defined by the process noise, based on an algebraic relationship with the state of the machine.

5. The system according to claim 1, wherein, The performance target used to update the control parameters is different from the control strategy of the feedback controller.

6. The system according to claim 1, wherein, The performance objective includes a cost function that defines the deviation of the machine's state from a reference state, and wherein the measurement model is configured to update the control parameters by optimizing the cost function.

7. The system according to claim 1, wherein, The performance objectives include (i) a cost function defining the deviation of the state from the reference state, (ii) a cost function for the state exceeding the optimal operating region, (iii) a cost function for the reference state overshooting by a specific value, (iv) a cost function for the oscillation of the state, and (v) one or a combination of cost functions for the state changing between time steps.

8. The system according to claim 1, wherein, The control parameters include (i) one or more gains of the feedback controller, (ii) one or more structural parameters of the machine, (iii) one or more coefficients of one or more filters used by the feedback controller, or (iv) one or more weights or combinations of the neural network controller.

9. The system according to claim 1, wherein, In order to generate the control parameters, the Kalman filter is configured to update the coefficients of the basis functions in one or more state-related regions.

10. The system according to claim 1, wherein, In order to generate the control parameters, the Kalman filter is configured to update the coefficients of the basis functions in multiple state-related regions and the boundaries separating the multiple state-related regions.

11. The system according to claim 1, wherein, The Kalman filter is an Extended Kalman Filter (EKF) configured to compute the Kalman gain by calculating the gradient of the performance target.

12. The system according to claim 1, wherein, The Kalman filter is an unscented Kalman filter (UKF) configured to calculate the Kalman gain by evaluating the control parameters relative to the performance target.

13. The system according to claim 1, wherein, The uncertainty of at least one of the control parameters is bounded, and the Kalman filter is further configured to select one or a combination of the process noise, the measurement noise, or the performance target based on the bound of the uncertainty.

14. The system according to claim 1, wherein, The feedback controller is configured to determine the current control input subject to constraints on the operation of the machine, thereby addressing control discontinuities.

15. The system according to claim 1, further comprising a security inspection module configured to: Perform a check based on the control strategy to determine whether the values ​​of the control parameters generated by the Kalman filter satisfy a security check; and When the security check is met, the control parameters of the feedback controller are updated with the control parameters generated by the Kalman filter.

16. The system according to claim 15, wherein, When the value of the control parameter generated by the Kalman filter does not meet the security check, the Kalman filter is further configured to iteratively generate new values ​​for the control parameter until the security check is met.

17. The system according to claim 15, wherein, The security check includes one or a combination of the boundedness of the state relative to the origin and the decreasing cost of the state.

18. A method for controlling the operation of a machine for performing a task, the method comprising the steps of: Submit a control input sequence to the machine; Receive feedback signals including corresponding measurement sequences, wherein each measurement indicates the state of the machine caused by a corresponding control input; In each control step, the current measurement of the machine's current state is transformed into a control strategy for controlling the machine by applying the current values ​​of control parameters in the control parameter set based on the feedback controller, and the current control input is determined based on the feedback signal including the current measurement; and The state of the feedback controller, defined by the control parameters, is iteratively updated using a prediction model that predicts the values ​​of the control parameters subjected to process noise and a measurement model that updates the predicted values ​​of the control parameters based on the measurement sequence subjected to measurement noise, to generate the current values ​​of the control parameters describing the measurement sequence according to a performance target. The measurement model is also configured to select from different performance targets based on one or a combination of the machine's state and the state of the environment surrounding the machine.

19. A non-transitory computer-readable storage medium having a program implemented thereon, the program being executable by a processor to perform a method for controlling operations of a machine performing a task, the method comprising the steps of: Submit a control input sequence to the machine; Receive feedback signals including corresponding measurement sequences, wherein each measurement indicates the state of the machine caused by a corresponding control input; In each control step, the current measurement of the machine's current state is transformed into a control strategy for controlling the machine by applying the current values ​​of control parameters in the control parameter set based on the feedback controller, and the current control input is determined based on the feedback signal including the current measurement; and The state of the feedback controller, defined by the control parameters, is iteratively updated using a prediction model that predicts the values ​​of the control parameters subjected to process noise and a measurement model that updates the predicted values ​​of the control parameters based on the measurement sequence subjected to measurement noise, to generate the current values ​​of the control parameters describing the measurement sequence according to a performance target. The measurement model is also configured to select from different performance targets based on one or a combination of the machine's state and the state of the environment surrounding the machine.

Images