Loop prediction and iterative correction method for fast solution for mixed integer optimal control

Through iterative pre-solving and correction methods, the MIOCP problem is converted into convex planning, which solves the problem of insufficient speed and inability to resolve in MIOCP, improves the feasibility and optimization of high-dimensional problems, and enhances the optimal control of mixed integers Real-time application capability.

CN118984977APending Publication Date: 2024-11-19MITSUBISHI ELECTRIC CORP
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202380032580.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-05-10
Filing Date
2023-02-10
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The existing mixed integer programming (MIP) solvers are insufficient in mixed integer optimal control (MIOCP), which limits their applicability in real-time applications, and the ability of supervised learning technology to expand high-dimensional MIOCP problems is limited, and Predicted solutions may not be feasible, especially in the case of multiple decision variables.

Method used

Using the correction method based on iterative pre-solving, by fixing discrete variables as predicted values ​​and updating other variables according to constraints, the mixed integer convex planning (MICP) problem is converted into convex planning (CP) problem, and machines such as recurrent neural networks are used. Learning methods predict and correct the feasibility and optimization of solutions.

Benefits of technology

It realizes the feasibility and optimization of solutions in high-dimensional MIOCP problems, reduces calculation time, enhances the real-time application ability of MIOCP problems, and ensures the feasibility of motion trajectory.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118984977A_ABST
    Figure CN118984977A_ABST
Patent Text Reader

Abstract

A controller uses a motion trajectory to control motion of a device to perform a constrained task. The controller evaluates a parameter function to output a predicted value for a set of discrete variables in a mixed integer convex programming (MICP) problem for performing a task defined by the parameter. The controller fixes a first subset of discrete variables in the MICP to predicted values output by the trained parametric function and updates at least some of the predicted values of the remaining subsets of discrete variables to values uniquely defined by the fixed values and constraints of the first subset of discrete variables. Accordingly, the controller transforms the MICP into a convex programming (CP) problem, solves the CP problem under constraints to produce a feasible motion trajectory, and controls the device in accordance with the motion trajectory.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates generally to mixed-integer convex optimization based control, and more particularly to loop prediction and iterative correction methods and apparatus for model predictive control of systems described by dynamics having continuous and discrete operating elements. Background Art

[0002] Optimization-based decision-making processes, planning and control techniques, such as model predictive control (MPC), allow for a model-based design framework in which system dynamics, system requirements and constraints can be directly considered. This framework has been extended to hybrid systems including both continuous and discrete decision variables, which provides a powerful technique to model a wide range of problems (e.g., including dynamic systems with mode switching or systems with quantized drives, problems with logic rules, time logic specifications or obstacle avoidance constraints). However, the resulting optimization problems are highly non-convex and therefore difficult to solve in practice because they contain variables that only take integer values. When a linear or linear quadratic objective is used in conjunction with linear system dynamics and linear inequality constraints, the resulting optimal control problem (OCP) can be formulated as a mixed integer linear program (MILP) or a mixed integer quadratic program (MIQP). More general convex inequality constraints can be included, such as quadratic inequality constraints, leading to mixed integer quadratic constrained quadratic programming (MIQCQP), or second-order cone constraints, leading to mixed integer second-order cone programming (MISOCP).

[0003] The goal of decision making, planning, or control for hybrid systems is to solve such a mixed integer program (MIP) at each sampling time. This is a difficult task, as mixed integer programs are generally NP-hard, and several methods for solving such sequences of MIPs have been explored in the literature. These methods can be divided into heuristic techniques that seek to efficiently find suboptimal solutions to the problem and optimization algorithms that attempt to solve the MIP to the optimum. Most mixed integer optimization algorithms are based on a variant of the branch and bound (B&B) technique to solve the MIP to the optimum. Variants of the branch and bound strategy have been combined with various methods for solving relaxed convex subproblems (e.g., with dual active set solvers, interior point algorithms, dual projected gradient methods, non-negative least squares solvers, and alternating direction method of multipliers (ADMM)). However, the combinatorial complexity of MIPs generally results in an exponential increase in the computational time of the B&B method for solving MIPs with an increasing number of discrete decision variables, limiting the applicability of MIP-based optimal control design in practice.

[0004] Examples of heuristic techniques can be based on rounding and pumping schemes using approximate optimization algorithms, approximate dynamic programming, or using data-based machine learning techniques such as supervised learning. Using supervised learning to replicate optimal and feasible MIP solutions from B&B methods obtained offline, and inferring these solutions online at high speed, has greatly improved the solution time of mixed integer optimal control problems (MIOCPs). Alternatively, reinforcement learning techniques have been used to learn tree search strategies to accelerate B&B methods, but these methods have limited applicability to real-time embedded systems in practice because they require passing a predictor (e.g., a neural network) forward at least once at each node of the B&B tree, and more importantly, in the worst case, these methods may still require enumerating the full B&B tree.

[0005] Existing supervised learning techniques can directly approximate the mapping from the problem parameters (i.e., the current state of the controlled system and the system environment) to the discrete variable solution of the MIOCP. This approach is based on the recognition that the mixed integer convex programming (MICP) problem can be solved very efficiently as a convex programming after fixing all discrete variables to a fixed set of values ​​provided by the learned predictor, i.e., after fixing all binary variables to 0 or 1 and fixing all integer variables to integer values. Examples of such supervised learning techniques include the use of kernel and nonparametric classification methods, deep neural networks for regression, and multi-class classification. Some existing methods are based on supervised learners that only predict discrete variables associated with one or more first time steps within the MIOCP control horizon; however, this is likely to lead to significant suboptimality in rolling horizon implementations. Alternatively, the policy obtained by the exact solver for the MIOCP can be directly imitated using a deep neural network, or the value function can be learned by utilizing data obtained through offline simulation.

[0006] Existing machine learning-based approaches for computing fast MIP solutions for MIOCPs suffer from two major drawbacks. First, the predictions of learning-based MIP solutions typically lack any guarantee of finding optimal or even feasible but suboptimal solutions, i.e., the predicted solution guesses may have a high probability of being infeasible with respect to one or more mixed integer inequality constraints. Second, such supervised learning techniques are effective for small-scale MIP problems, but they generally do not scale well to MIOCPs with more than 100 decision variables. In particular, existing machine learning approaches for MIPs require extensive data collection or sampling of the parameter space of high-dimensional MIOCPs.

[0007] Therefore, there is a need for a high-precision prediction and correction method for efficiently computing feasible but potentially suboptimal mixed-integer solutions to high-dimensional MIOCP problems, which is the purpose of the systems and methods described in the present invention. Summary of the invention

[0008] Mixed integer programming (MIP) problems, which include both continuous and discrete action elements, often arise in the optimal control of hybrid systems. Current state-of-the-art MIP solvers are often not fast enough for real-time applications of mixed integer optimal control, limiting the applicability of MIP-based controller design in practice. Although supervised learning techniques have been proposed to accelerate the solution of MIPs, supervised learning techniques generally have two major drawbacks. First, the prediction of learned MIP solutions may have a high probability of being infeasible for one or more mixed integer inequality constraints. Second, such supervised learning techniques do not scale well to optimal control problems with more than 100 decision variables. The present invention describes a recurrent architecture for scalable prediction of MIP solutions that exploits the temporal structure in mixed integer optimal control problems (MIOCPs) combined with an iterative pre-solve-based correction method to increase the likelihood of feasibility for each solution guess.

[0009] Embodiments of the present invention are based on the recognition that a mixed integer convex programming (MICP) problem can be transformed into a convex programming (CP) problem after fixing all discrete variables to a fixed set of values ​​provided by a machine learning based predictor, i.e., after fixing all binary variables to 0 or 1 and all integer variables to integer values. After the MICP is transformed, the resulting CP problem can be solved efficiently. Embodiments of the present invention are based on a predictor that predicts the discrete variables θ at each of the time steps i = 0, 1, ..., N within the control horizon of the MIOCP given a set of values ​​of the problem parameters θ. Examples of problem parameters θ may include, for example, the current state of the controlled system, a target or goal state, the location and / or orientation of obstacles, driving restrictions, weight values ​​in the objective function, and / or bound values ​​in the mixed integer inequality constraints of the MIOCP.

[0010] Some embodiments are based on the recognition that machine learning-based predictors (e.g., parameter functions such as neural networks) can be trained to predict the values ​​of a set of discrete variables in a MICP for performing a task defined by parameters. However, due to the constraints imposed on the device motion, these predicted values ​​may lead to infeasible solutions and / or infeasible motion trajectories for the device performing the task. For example, it is recognized that while constraints need to be enforced as hard constraints, replacing constrained non-convex optimization with such trained learning functions is problematic due to the nature of machine learning. Specifically, machine learning is closely related to computational statistics, which is probabilistic in nature. Therefore, trained parameter functions, such as neural networks, do not have the ability to handle hard constraints, but are designed to satisfy constraints with a certain probability. This concept is similar to or a direct replication of the concept of soft constraints, when violations of constraints are punished but not prohibited.

[0011] Therefore, it is impossible or at least impractical to train a neural network that predicts discrete values ​​of the solution to a non-convex optimization with hard constraints. It is possible to find such a solution with very high probability, but it is impractical given enough offline training data that closely approximates the online environment.

[0012] Since motion planning and control require constraints to be satisfied, some implementations are based on the realization that a learning function can be used to find initial values ​​for discrete variables, thereby converting a non-convex problem into a convex problem. However, such initial predictions need to be further processed to ensure that the final motion trajectory is feasible.

[0013] Some embodiments are based on the understanding that the predicted discrete values ​​can be checked for feasible solutions. However, such a check requires actually solving one or more convex problems, which is undesirable. Therefore, it is a goal of some embodiments to check the feasibility of the predicted variables without finding the corresponding motion trajectory. Additionally or alternatively, updating the predicted discrete values ​​to convert the non-convex optimization into a convex optimization to produce a feasible solution is another purpose of some embodiments.

[0014] Some embodiments are based on the recognition that predicted discrete values ​​can be checked against constraints on the motion of the device that result in constraints on discrete variables. For example, if a constraint states that the sum of two binary variables is not greater than 1, if the value of one of these variables is 1, then the value of the other variable must be zero to satisfy the constraint. It is worth noting that these constraint checks can be done without solving the corresponding convex problem, but only if the value of one of these variables is known in this example. These constraint checks, without solving the corresponding convex optimization problem, are generally referred to as pre-solve operations.

[0015] To this end, some embodiments fix the values ​​of discrete variables in the MICP problem to their predicted values, and use the given values ​​of the fixed variables to update other values ​​to satisfy the constraints. For example, when making confidence predictions for the values ​​of discrete variables, the confidence levels can be used to fix some variables and update the remaining variables. This increases the feasibility of solving the MICP problem without solving the corresponding convex problem.

[0016] Therefore, one embodiment discloses a controller for controlling the motion of a device to perform a task of changing the state of a constrained device, the controller comprising: a processor; and a memory having instructions stored thereon, which, when executed by the processor, cause the controller to perform the following operations: collect parameters of the task, including the state of the device; input the parameters of the task to evaluate a parameter function, the parameter function being trained to output predicted values ​​for a set of discrete variables in a mixed integer convex programming (MICP) problem for performing the task defined by the parameters; and convert the parameters in the MICP into predicted values ​​for a set of discrete variables. A first subset of discrete variables is fixed to predicted values ​​output by a trained parameter function; a pre-solve-based correction is performed, and the pre-solve-based correction is configured to update at least some of the predicted values ​​of the remaining subset of discrete variables in the MICP to values ​​uniquely defined by the fixed values ​​of the first subset of discrete variables and constraints; based on the fixed values ​​of the first subset of discrete variables and the updated values ​​of the remaining subset of discrete variables, the MICP is converted into a convex programming (CP) problem; the constrained CP problem is solved to generate a feasible motion trajectory for performing the task defined by the parameters; and the device is commanded to change state according to the motion trajectory.

[0017] For example, the predicted discrete value set can be used by the correction method based on iterative pre-solve In order to compute an updated set of values ​​with an increased likelihood of being feasible and / or optimal When the resulting convex optimization problem (CP) has a feasible optimal solution, the value set of the discrete variables is considered feasible. Finally, using the updated discrete value set With each discrete optimization variable fixed, the resulting CP can be efficiently solved to compute a feasible optimal control solution, including a trajectory of the state and values ​​of the control input variables at each time step in the control horizon, which is likely to be (close to) the optimal control solution of the original MIOCP.

[0018] In some embodiments of the present invention, the correction method based on iterative pre-solve can return one of the following results: the iterative correction method detects a predicted discrete value set is feasible, thus returning the value The iterative correction method calculates the updated discrete value set is feasible, thus returning the value Iterative correction methods may not be able to compute a feasible updated set of discrete values ​​within a predetermined time limit. Embodiments of the present invention are based on the recognition that correction methods based on iterative pre-solve can increase the likelihood of being feasible and / or optimal, for example, when the iterative correction method removes or prunes discrete variables that cause the learning-based predictor to provide incorrect predicted values ​​that would lead to an infeasible control solution.

[0019] In some embodiments of the present invention, each iteration of the iterative correction method performs a single pre-solve step, the purpose of which is to reduce the number of discrete optimization variables in the MIOCP by fixing each removed or pruned variable to an optimal and / or uniquely feasible value. The single pre-solve step preserves feasibility and optimality, that is, any feasible or optimal solution to the simplified MICP problem can be mapped to a feasible or optimal solution to the original MICP problem with the same objective value, and the simplified MICP problem is infeasible or unbounded if and only if the original MICP problem is infeasible or unbounded. Subsequently, if the single pre-solve step does not detect infeasibility and not all discrete variables have been fixed, the iterative correction method selects one or more of the remaining free discrete variables to fix them to the solution values ​​provided by the learning-based predictor and performs a single pre-solve step in the next iteration of the iterative correction method. After one or more iterations, if all discrete variables have been fixed, or infeasibility is detected, the correction method based on iterative pre-solve terminates.

[0020] Some embodiments of the invention are based on the recognition that if infeasibility is detected by an iterative pre-solve based correction method, an alternative control solution vector may be used to change the state of the plant.

[0021] In some embodiments of the invention, a single pre-solve step may reuse one or more computational results from a previous single pre-solve step to improve the computational efficiency of the iterative pre-solve based correction method. In some embodiments of the invention, the number of iterations in the iterative correction method is limited by a predetermined upper bound to improve computational efficiency. If the predetermined upper bound on the number of iterations has been reached and not all discrete variables have been fixed, all remaining free discrete variables may be fixed to the solution values ​​provided by the learning based predictor.

[0022] In some embodiments of the invention, a single pre-solve step itself includes an iterative process of one or more pre-solve operations to reduce the number of discrete and / or continuous optimization variables, tighten bounds on optimization variables, and / or tighten bounds on one or more inequality constraints. The iterative process of the single pre-solve step is then performed one or more iterations until the problem is detected to be infeasible, until insufficient progress is made between two consecutive iterations, or until a predetermined time limit is reached to ensure computational efficiency. Examples of pre-solve operations include domain propagation, bound strengthening, dual fixation, implicit variable substitution, coefficient strengthening, probing, detection, and removal of redundant variables and / or constraints. In addition, each of the pre-solve operations can be performed on a single variable, a single constraint, multiple variables, and / or multiple constraints.

[0023] Some embodiments of the present invention are based on the recognition that incorporating the temporal nature of the sequence of discrete values ​​in the MIOCP solution into the machine learning framework improves the relationship between the learning problem parameter θ and the discrete solution In some embodiments of the present invention, a recurrent neural network (RNN) is used to learn the relationship between θ and θ at each of the time steps i = 0, 1, ..., N in the control field of the MIOCP. For example, the RNN architecture can be based on a long short-term memory (LSTM) network or a gated recurrent unit (GRU). Embodiments of the present invention are based on the realization that by exploiting the temporal structure of the discrete MIOCP solution, the number of target labels in supervised learning can be greatly reduced, which can improve sampling efficiency and / or improve the performance of the predictor. For example, given n at each time step i = 0, 1, ..., N δ discrete variables, the maximum number of target labels is By exploiting the temporal structure of MIOCP, this can be reduced to a maximum target labels.

[0024] In some embodiments of the invention, supervised learning is used to train a predictor that, given a set of values ​​θ of the problem parameters, predicts the discrete variable θ at each of the time steps i = 0, 1, ..., N within the control horizon of the MIOCP. Based on the offline data generation step, embodiments of the present invention require a set of optimal control solutions corresponding to a set of values ​​of the problem parameters. Different techniques for sampling the parameter space can be used, including, for example, uniform sampling and smart sampling techniques, so that the data set of sampled parameter values ​​is representative of the optimal control problem of interest in practice. For these parameter values ​​θ j For each of , the corresponding MIOCP is solved, and if the problem is feasible, the optimal control solution is stored in the dataset for supervised training of the predictor. In addition, for each parameter value θj , for every unique set of discrete values Storage class tag value Where Δ represents the discrete optimizer set, that is, the union of all optimal discrete values ​​in the sampled data set.

[0025] In some embodiments of the present invention, the supervised learning based predictor is trained based on a regression problem, i.e., the predictor is trained to directly predict the discrete variable y at each time step i=0,1,…,N in the control horizon of the MIOCP. In some embodiments of the invention, the predictor is an RNN, for example, composed of one or more LSTMs and / or one or more feed-forward neural networks, with an input dimension equal to the number of problem parameters n θ , the output dimension is equal to the number of discrete variables n at each time step δ . For example, the training loss for a regression problem can be a binary cross entropy loss applied to the logit activations of the output layer of the network. In some embodiments of the invention, the regressor is used to fix only one or more discrete variables that give a learning-based predictor relatively high confidence, such that a simplified MIOCP needs to be solved to compute the remaining free discrete variables in order to compute a feasible but potentially suboptimal mixed integer solution to the high-dimensional MIOCP problem.

[0026] Some embodiments of the invention are based on the recognition that a subset of discrete variables can be fixed by a pre-solve based correction method that starts with predicted values ​​of a set of discrete variables in a MICP and, unless infeasibility is detected by the iterative pre-solve based correction, solves an approximate solution of the MICP with a reduced number of discrete optimization variables under constraints to obtain a feasible motion trajectory with an updated set of fixed values ​​of the subset of discrete variables. In some embodiments of the invention, an approximation of the MICP with a reduced number of discrete optimization variables can be solved using a branch and bound (B&B) optimization method in combination with one or more customized block sparse pre-solve operations that exploit the block structured sparsity of the MIOCP for performing tasks defined by the problem parameter values.

[0027] In some embodiments of the invention, a first subset of discrete variables in the MICP are fixed to predicted values ​​output by a trained parameter function, a second subset of discrete variables in the MICP are fixed to values ​​uniquely defined by the first subset of discrete variables and fixed values ​​of constraints through a pre-solve based correction, and a third subset of discrete variables in the MICP are selected as free optimization variables so that an approximation of the MICP with a reduced number of discrete optimization variables is solved under the constraints to produce feasible motion trajectories, for example, using a B&B optimization method in combination with one or more customized block sparse pre-solve operations that exploit the block structured sparsity of the MIOCP problem.

[0028] In some embodiments of the invention, the supervised learning based predictor is trained based on a classification problem, i.e., the predictor is trained to perform a multi-class classification on a discrete optimizer set (i.e., ) selects the best discrete value. In some embodiments of the present invention, the predictor is an RNN, for example, which is composed of one or more LSTMs and / or one or more feed-forward neural networks, and its input dimension is equal to the number of problem parameters n θ , the output dimension is equal to the number of unique discrete values ​​in the data set |Δ|. For example, the training loss for the classification problem can be a cross entropy loss. In some embodiments of the present invention, a classifier is used to select one or more discrete solution candidates corresponding to the highest confidence of feasibility and optimality, and a correction method based on iterative pre-solution can be used to potentially correct one or more of these discrete solution candidates in order to calculate a feasible but potentially suboptimal mixed integer solution to the high-dimensional MIOCP problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] [ Figure 1A ]

[0030] Figure 1A A block diagram of a mixed integer model predictive controller and feedback system is shown in accordance with some embodiments.

[0031] [ Figure 1B ]

[0032] Figure 1B A block diagram of a mixed integer model predictive controller and feedback system is shown in accordance with some embodiments.

[0033] [ Figure 1C ]

[0034] Figure 1C A block diagram illustrating a hierarchical integration between a mixed integer model predictive controller and a tracking controller to control a feedback system in accordance with some embodiments.

[0035] [ Figure 2A ]

[0036] Figure 2A A mixed integer model predictive control (MIMPC) method is shown according to some embodiments, which implements a predictive controller that calculates control signals given current problem parameter values ​​and control commands.

[0037] [ Figure 2B ]

[0038] Figure 2B A block diagram of a MIMPC method for solving an optimal control structured mixed integer convex program (MICP) given current problem parameter values ​​in accordance with some embodiments is shown.

[0039] [ Figure 2C ]

[0040] Figure 2C It is shown that after fixing all discrete variables to a fixed set of values, the optimal control structured MICP problem can be converted to an optimal control structured convex programming (CP) problem according to some embodiments, and can be solved very efficiently.

[0041] [ Figure 3A ]

[0042] Figure 3A A block diagram of a MICP solution method based on the solution of a prediction and optimal control structured CP for a fixed set of discrete values ​​according to some embodiments is shown.

[0043] [ Figure 3B ]

[0044] Figure 3B A block diagram of a MICP solution method based on prediction of a fixed set of discrete values, followed by correction and optimal control of the solution of a structured CP, according to some embodiments is shown.

[0045] [ Figure 3C ]

[0046] Figure 3C A block diagram of a MICP solution method based on multiple predictions for a fixed set of discrete values ​​followed by corrections after each prediction and solution of an optimally controlled structured CP is shown in accordance with some embodiments.

[0047] [ Figure 3D ]

[0048] Figure 3D A block diagram of a MICP solution method based on prediction for fixing a first subset of discrete variables, followed by pre-solved correction to fix the remaining subset of discrete variables, and solution of an optimally controlled structured CP is shown in accordance with some embodiments.

[0049] [ Figure 4A ]

[0050] Figure 4A A block diagram showing the offline data generation and supervised learning process for training a machine learning based predictor for the online variable fixing and optimal control solving process to solve the MICP.

[0051] [ Figure 4B ]

[0052] Figure 4BA flowchart showing the offline data generation process for MICP solving based on a set of sampled problem parameter values ​​and the supervised learning process for training a machine learning based predictor.

[0053] [ Figure 4C ]

[0054] Figure 4C is pseudo code for an offline data generation process for MICP solving based on a set of sampled problem parameter values ​​and a supervised learning process for training a learning-based predictor according to some embodiments.

[0055] [ Figure 4D ]

[0056] Figure 4D A flowchart of the online variable fixing process based on prediction and correction steps based on iterative pre-solution and the optimal control structured CP solving process for calculating feasible MICP solutions is shown.

[0057] [ Figure 4E ]

[0058] Figure 4E is a pseudo code for an online variable fixing process based on prediction and correction steps based on iterative pre-solution, and a CP solving process for computing a feasible MICP solution, according to some embodiments.

[0059] [ Figure 5A ]

[0060] Figure 5A An example of an online variable fixing process based on prediction and involving one or more pre-solve based iterations to calculate a set of fixed values ​​for the MICP discrete variables is shown.

[0061] [ Figure 5B ]

[0062] Figure 5B A flow chart of an online variable fixing process based on prediction and correction steps involving one or more pre-solve based iterations to calculate a set of fixed values ​​for MICP discrete variables is shown.

[0063] [ Figure 5C ]

[0064] Figure 5C is pseudo code for an iterative pre-solve based correction process for computing a fixed set of values ​​of discrete variables for a MICP solution according to some embodiments of the present invention.

[0065] [ Fig. 6A ]

[0066] Fig. 6AAn example of domain propagation for bound enforcement of MICP optimization variables that may be used for an iterative presolve based correction process according to some embodiments is shown.

[0067] [ Figure 6B ]

[0068] Figure 6B An example of redundant inequality constraint detection and dual fixing of MICP optimization variables that can be used for an iterative presolve-based correction process according to some embodiments is shown.

[0069] [ Figure 6C ]

[0070] Figure 6C An example of inequality constraint coefficient enforcement corresponding to MICP discrete optimization variables that may be used for an iterative pre-solve based correction process according to some embodiments is shown.

[0071] [ Fig.6D ]

[0072] Fig.6D An example of a binary variable detection process for MICP discrete optimization variables that may be used for an iterative pre-solve based correction process according to some embodiments is shown.

[0073] [ Fig. 7A ]

[0074] Fig. 7A is pseudocode for a block sparse forward-backward operation that may be used for iterative pre-solve based correction to implement domain propagation and bound strengthening procedures according to some embodiments.

[0075] [ Figure 7B ]

[0076] Figure 7B is pseudocode for efficient block-sparse computation of optimization-based bound tightening for MICP optimization variables that can be used for an iterative pre-solve based correction process according to some embodiments.

[0077] [ Figure 7C ]

[0078] Figure 7C is a pseudocode for block sparse redundant inequality constraint detection and dual fixation of MICP optimization variables that can be used for an iterative pre-solve based correction process according to some embodiments.

[0079] [ Fig.7D ]

[0080] Fig.7D is a pseudocode for block sparse inequality constraint coefficient enhancement corresponding to MICP discrete variables that may be used for an iterative pre-solve based correction process according to some embodiments.

[0081] [ Fig. 7E ]

[0082] Fig. 7E is pseudo code for a block-sparse binary variable detection process for MICP discrete variables that may be used for an iterative presolve based correction process according to some embodiments.

[0083] [ Figure 7F ]

[0084] Figure 7F is pseudo code for a single block sparse presolve step based on one or more presolve operations that may be used for an iterative presolve based correction process according to some embodiments.

[0085] [ Fig. 8A ]

[0086] Fig. 8A A block diagram is shown of a machine learning prediction method for directly predicting all discrete values ​​of the MICP solution based on regression according to some embodiments.

[0087] [ Figure 8B ]

[0088] Figure 8B A block diagram is shown of machine learning prediction using a temporal decomposition approach to directly predict discrete values ​​of one or more time steps of a MICP solution based on regression, according to some embodiments.

[0089] [ Figure 8C ]

[0090] Figure 8C A block diagram of a machine learning prediction method for predicting the probability of one or more discrete values ​​in a discrete optimizer set as a MICP solution based on classification according to some embodiments is shown.

[0091] [ Fig.8D ]

[0092] Fig.8D A block diagram of machine learning prediction using a temporal decomposition approach to predicting the probability of one or more discrete values ​​in a set of discrete optimizers at one or more time steps of a MICP solution based on classification according to some embodiments of the present invention is shown.

[0093] [ Fig. 9A ]

[0094] Fig. 9A An example of a machine learning algorithm for a predictive model according to some embodiments of the present disclosure is shown.

[0095] [ Fig. 9B ]

[0096] Fig. 9B A schematic diagram showing the layers and signals involved in a deep neural network for a machine learning-based prediction model according to some embodiments of the present disclosure.

[0097] [ Fig. 9C ]

[0098] Fig. 9C Schematic diagram of a deep neural network, hidden states, and single-layer or multi-layer recurrent neural network (RNN) for machine learning prediction using a temporal decomposition method according to some embodiments.

[0099] [ Fig. 10A ]

[0100] Fig. 10A A schematic diagram showing the generation of data using focused sampling of problem parameters according to some embodiments of the present disclosure.

[0101] [ Fig. 10B ]

[0102] Fig. 10B An example of a focused sampling method for data generation according to some embodiments of the present disclosure is shown.

[0103] [ Fig.11A ]

[0104] Fig.11A A schematic diagram showing an example of a binary control variable search tree representing a nested tree of search regions for integer feasible solutions to the MICP for a sample set of problem parameter values ​​according to some embodiments of the present invention is shown.

[0105] [ Fig. 11B ]

[0106] Fig. 11B A block diagram of a branch-and-bound mixed integer optimization algorithm for searching for integer feasible optimal solutions to a MICP based on a nested tree of search regions and corresponding lower / upper bound values ​​is shown in accordance with some embodiments.

[0107] [ Fig. 12A ]

[0108] Fig. 12A A schematic diagram of a vehicle including a predictive controller employing the principles of some embodiments is shown.

[0109] [ Fig. 12B ]

[0110] Fig. 12B A schematic diagram illustrating the interaction between a predictive controller and other controllers of a vehicle according to some embodiments.

[0111] [ Fig. 12C ]

[0112] Fig. 12C A schematic diagram of a path and / or motion planning method for a controlled vehicle employing the principles of some embodiments is shown.

[0113] [ Fig.12D ]

[0114] Fig.12D Exemplary traffic scenarios of a single vehicle or multiple vehicle decision making process modules according to some embodiments are shown.

[0115] [ Fig.13A ]

[0116] Fig.13A is a schematic diagram of a mixed integer predictive control problem formulation for a spacecraft using principles according to some implementations.

[0117] [ Fig. 13B ]

[0118] Fig. 13B is a schematic diagram of a mixed integer predictive control problem formulation for a spacecraft using principles according to some implementations.

[0119] [ Fig.14A ]

[0120] Fig.14A A schematic diagram of a vapor compression system controlled by a controller according to some embodiments is shown.

[0121] [ Fig. 14B ]

[0122] Fig. 14B Examples of configurations of signals, sensors, and controllers for use in a VCS are shown according to some implementations.

[0123] [ Fig.15 ]

[0124] Fig.15 A method for controlling a system according to an example implementation is shown. DETAILED DESCRIPTION

[0125] Some embodiments of the present disclosure provide systems and methods for controlling a system or operation of a system using a predictive controller. An example of a predictive controller is a mixed integer model predictive controller (MIMPC) that determines control inputs based on a dynamic model of a controlled system having continuous and discrete operating elements.

[0126] Figure 1A A block diagram of a predictive controller 110 and a feedback system 120 is shown according to some embodiments. Figure 1AAn example feedback system (or system) 120 is shown connected to a predictive controller 110 (or controller) through a state estimator 130 according to some embodiments. In some implementations, the predictive controller 110 is a MIMPC controller programmed according to a dynamic model 102 (or system model) of the system 120. The system model 102 can be a set of equations that represent the change of the state and output 103 of the system 120 over time as a function of current and previous inputs 111 and previous outputs 103. The system model 102 can include constraints 104 that represent the physical and operational limitations of the system 120. During operation, the controller 110 receives a command 101 that indicates a desired behavior of the system 120. For example, the command can be a motion command. In response to receiving the command 101, the controller 110 generates a control signal 111 used as an input to the system 120 including continuous operating elements and discrete operating elements. In response to the input, the system updates the output 103 of the system 120. Based on the measurement of the output 103 of the system 120, the estimator 130 updates the estimated state 121 of the system 120. The estimated state 121 of the system 120 provides state feedback to the predictive controller 110. Therefore, the predictive controller initially receives a feedback signal 121 of the system 120 through the estimator 130, wherein the feedback signal 121 includes a measurement of the state of the system 120.

[0127] As referred to herein, a system 120 may be any machine or device controlled by certain operational input signals, such as control signals 111 (inputs). Control input signals may include continuous elements such as voltage, pressure, force, torque, steering angle, speed, and temperature, as well as discrete elements such as energy levels, quantized valve inputs, gear shifting, on / off actuation, lane selection, and obstacle avoidance decision variables. The system 120 returns some controlled output signals 103 (outputs), which may include continuous elements such as current, flow, speed, position, temperature, heading, and steering angle, as well as discrete elements such as energy levels, quantized valve states, gear states, on / off states, and lane positions. The output values ​​are partially related to previous output values ​​of the system, and partially related to previous and current input values. The dependence on previous inputs and previous outputs is encoded in the state of the system. The operation of the system, for example, the movement of a component of the system, may include a series of output values ​​generated by the system after applying certain input values.

[0128] The system model 102 may include a set of mathematical equations that describe how the system output changes over time as a function of current and previous inputs and previous outputs. The mathematical equations may include one or more smooth equations that depend on continuous variables, and one or more mixed integer equations that depend on both continuous and discrete variables. Each function in the mathematical equation may be a linear function or a piecewise linear function. The state of the system 120 is any set of information that generally changes over time, such as a suitable subset of current and previous inputs and outputs, which together with the model of the system and future inputs can uniquely define the future motion of the system.

[0129] The system 120 may be constrained by physical limitations and specification constraints 104 that limit the range of possible operations allowed for the outputs, inputs, and likewise possible states of the system 120. The constraints may include one or more smooth equations that depend on continuous variables and one or more mixed integer equations that depend on both continuous and discrete variables. The constraint functions may be linear or piecewise linear functions, or may include one or more convex inequality constraint functions, such as convex quadratic inequality constraints, second-order cone constraints, or convex cone constraints. Some embodiments of the present invention are based on the recognition that after all discrete variables are fixed to a fixed set of values ​​provided by the predictor, i.e., after all binary variables are fixed to either 0 or 1 and all integer variables are fixed to integer feasible values, the mixed integer convex programming (MICP) problem may be solved very efficiently as a convex programming under the constraints of the system model 102 and the constraints 104.

[0130] The predictive controller 110 can be implemented as hardware or a software program executed in a processor (e.g., a microprocessor) that receives an estimated state 121 of the system 120 and a desired motion command 101 at a fixed or variable control cycle sampling interval, and uses this information to determine inputs, such as control signals 111, for operating the system 120. The predictive controller 110 further solves the optimal control structured mixed integer convex programming (MICP) problem using a prediction based on cyclic machine learning and a correction method based on iterative pre-solution. Some embodiments of the present invention are based on the recognition that the prediction and correction method can be used to efficiently calculate a feasible but possibly suboptimal solution to the MICP to generate the control signal 111 using a computation time that is significantly less than the time required to accurately solve the MICP. Therefore, the processor achieves fast processing speed as well as high accuracy. The predictive controller 110 further controls the system 120 based on the control signal 111 to change the state of the system 120.

[0131] The estimator 130 may be implemented in hardware or as a software program executed in the same or different processor as the controller 110 that receives the output of the system 103 at fixed or variable control cycle sampling intervals and determines an estimated state 121 of the system 120 using new and previous output measurements.

[0132] Figure 1B A block diagram of a mixed integer model predictive controller (MIMPC) 110 and a feedback system 120 is shown according to some embodiments. The predictive controller 110 drives the system 120 so that the estimated state 121 and output 103 of the system 120 follow the command 101. The controller 110 includes a computer (e.g., in the form of a single central processing unit (CPU) or multiple CPU processors 151) connected to a memory 152 for storing the system model 102 and constraints 104 on the operation of the system 120. The CPU processor 151 can be composed of a single core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 152 can include random access memory (RAM), read-only memory (ROM), flash memory, or any other suitable memory system.

[0133] Figure 1C A block diagram showing a hierarchical integration between a mixed integer model predictive controller 110 for computing a high-level control signal 111 and a tracking controller 115 intended to track the high-level control signal 111 and compute a low-level control signal 112 to directly control a feedback system 120 according to some embodiments of the present invention is shown. For example, the predictive controller 110 may be a MIMPC controller for computing a reference motion trajectory based on a dynamic model of a controlled system having continuous operating elements and discrete operating elements, and the tracking controller 115 is intended to execute (part of) the reference motion trajectory by sending control inputs 112 directly to actuators of the controlled system.

[0134] An example of a tracking controller 115 may be based on a proportional-integral-derivative (PID) controller to track a time-varying reference motion trajectory calculated by the MIMPC controller 110. In some embodiments of the present invention, the tracking controller 115 may be based on a model predictive controller (MPC) having only continuous decision variables, and thus requires relatively less computational complexity to solve the optimal control problem. For example, the MPC tracking controller 115 may be based on a linear quadratic objective function, a linear dynamic model, and linear equality and inequality constraints, which require solving a convex linear programming (LP) or convex quadratic programming (QP) problem that is computationally much easier than the MICP solved by the MIMPC controller 110.

[0135] Some embodiments of the present invention are based on the recognition that a relatively long prediction horizon can be used for the mixed integer model predictive controller 110 and a relatively short prediction horizon can be used for the tracking controller 115, so that a relatively fast sampling rate can be used for the tracking controller, such as a sampling period of 10-100 milliseconds, while a relatively slow sampling rate can be used for the mixed integer model predictive controller 110, such as a sampling period of 0.2-2 seconds.

[0136] Some embodiments of the present invention are based on the recognition that due to the computational complexity of solving the MICP, a relatively high-level, low-precision dynamic model 102 and constraints 104 may be used in the MICP formulation of the mixed-integer model predictive controller 110, while a relatively low-level, high-precision dynamic model 102 and constraints 104 may be used in the design and formulation of a computationally economical tracking controller 115.

[0137] Figure 2A A block diagram of a system and method for mixed integer model predictive control (MIMPC) to implement a predictive controller 110 that calculates a control signal 111 given a current problem parameter value 205 and a control command 101 according to some embodiments is shown. In some embodiments of the present invention, the problem parameters θ include, for example, the current state of the controlled system 121, the target or goal state, the location, size and / or orientation of obstacles, driving restrictions, weight values ​​in the objective function of the mixed integer optimal control problem (MIOCP), and / or bounds in the mixed integer inequality constraints. Specifically, MIMPC calculates a control solution by solving a constrained mixed integer convex programming (MICP) problem 250 at each control time step, for example, a solution vector 255 of a sequence of future optimal discrete and continuous control inputs over a predicted time horizon of the system 120. In the optimization problem 250, the MICP data 245 of the objective function, the equality constraints, and the discrete, continuous and mixed integer inequality constraints depend on the dynamic model, the system constraints 240, the current problem parameter value 205, the control target and the control command 101.

[0138] In some embodiments of the invention, the solution to the inequality constrained MICP problem 250 uses one or more states and control values ​​over the prediction time horizon, and possibly other MICP solution information 210 from the previous control time step, which can be read from memory. This concept is called a warm start or hot start of the optimization algorithm, and in some embodiments of the invention, it can reduce the amount of computation required by the MIMPC controller. In a similar manner, the corresponding solution vector 255 can be used to update a sequence of one or more optimal states and control values ​​over the prediction time horizon, and possibly other MICP solution information for the next control time step and store it in memory 235.

[0139] Figure 2B1 shows a block diagram of a MIMPC method that aims to find a feasible but possibly suboptimal solution vector 255 for an optimal control structured MICP 260 at each control time step, given the current problem parameter values ​​205 and the command 101, in order to calculate the control signal 111. The optimal control structured MICP 260 includes stage cost functions g0(x0, u0, δ0; θ), g1(x1, u1, δ1; θ), ..., g N-1 (x N-1 ,u N-1 , δ N-1 ;θ) and the end cost function g N (x N ,u N , δ N ;θ) (which depends on the state variable x 0:N =x0,x1,...,x N , continuous control input variable u 0:N =u0,u1,...,u N , discrete control variable δ 0:N =δ0,δ1,...,δ N and the problem parameter variables θ) is minimized or maximized 261. In some embodiments of the invention, the state variable x 0:N is a continuous optimization variable, controlling the input variable u 0:N is a continuous optimization variable, and the discrete variable δ 0:N Includes binary and / or integer optimization variables.

[0140] In some embodiments of the present invention, the constraints of the optimal control structured MICP 260 include the initial state variable x0=x init (θ), where the current state value of the system can be calculated by the state estimator 130 and can depend on one or more problem parameters. In other embodiments of the present invention, the initial state variables are free and the equality constraint 262 can be omitted. In some embodiments of the present invention, the constraints of the optimal control structured MICP 260 include a sequence of equality constraints that define the state variables of the next time step based on the state and control variables at the previous time step within the prediction time horizon, for example, a sequence of equality constraints x1=ψ0(x0,u0,60;θ), x2=ψ1(x1,u1,δ1;θ), ..., x based on the dynamic model 102 of the controlled system N =ψ N-1 (x N-1 ,u N-1 , δ N-1θ) 263. In some embodiments of the present invention, the constraints of the optimal control structured MICP 260 include a sequence of inequality constraints on one or more combinations of state and / or control variables at each time step within the prediction time horizon, for example, a sequence of inequality constraints over the prediction time horizon N Among them, n f is the number of inequality constraints at each time step.

[0141] In some embodiments of the present invention, one or more of the discrete variables are binary optimization variables, that is, for i=0, 1, ..., N and j in the index set of binary optimization variables, δ i,j ∈{0,1}265. In other embodiments of the present invention, one or more of the discrete variables are integer optimization variables, that is, for the index set of integer optimization variables i=0,1,…,N and Some embodiments of the present invention are based on the recognition that if the MICP function 247 is a convex function, that is, the stage cost function g i (·)261 is a convex, dynamic model function ψ i (·)263 is linear or piecewise linear, and the inequality constraint function f i,j (·) 264 is convex, then the optimal control structured MICP 260 can be solved relatively efficiently.

[0142] Some embodiments of the present invention are based on the recognition that if the stage cost function g i (·)261 is the linear, dynamic model function ψ i (·)263 is linear or piecewise linear, and the inequality constraint function f i,j (·) 264 is linear or piecewise linear, then the optimal control structured MICP 260 can be formulated as a mixed integer convex linear program (MILP). Some embodiments of the present invention are based on the recognition that if the stage cost function g i (·)261 is a linear quadratic form, dynamic model function ψ i (·)263 is linear or piecewise linear, and the inequality constraint function f i,j (·) 264 is linear or piecewise linear, then the optimal control structured MICP 260 can be formulated as a mixed integer convex quadratic program (MIQP).

[0143] In some embodiments of the present invention, the inequality constraint function f i,jOne or more of (·) 264 are convex quadratic inequality constraints, so that the optimal control structured MICP 260 can be formulated as a mixed integer quadratic constrained quadratic program (MIQCQP). In some embodiments of the invention, the inequality constraint function f i,j One or more of (·) 264 are convex second-order cone constraints, so that the optimal control structured MICP 260 can be formulated as a mixed integer second-order cone program (MISOCP). Some embodiments of the present invention are based on the recognition that MICP problems, including MILP, MIQP, MIQCQP or MISOCP, can be solved relatively efficiently, for example, using branch-and-bound methods, branch-and-cut methods, branch-and-price methods, or any other variant of a tree-search-based optimization algorithm. However, some embodiments of the present invention are based on the recognition that the combinatorial complexity of MICPs typically results in an exponential growth in the computational time of optimization algorithms for solving MICPs with an increasing number of discrete decision variables, thereby limiting the applicability of MICP-based optimal control designs in practice.

[0144] Figure 2C It is shown that after fixing all discrete variables to a fixed set of values ​​275, i.e., after fixing all binary variables to 0 or 1 and / or fixing all integer variables to integer feasible values, the optimal control structured MICP problem 270 can be converted and solved 260 to an optimal control structured convex program (CP) 280. Embodiments of the present invention are based on the recognition that after fixing the discrete variables to the optimal discrete solution that can be obtained by solving the original MICP 270, Afterwards, the optimal control structure CP 280's optimal state and control solution Equal to the optimal state and control solution of the optimal control structured MICP 270 To calculate the solution vector 255. Therefore, by accurately predicting the optimal discrete solution when the discrete optimization variables 275 are fixed The MICP problem 270 may alternatively be solved 260 by solving the CP 280, which is computationally much more economical. In some embodiments of the invention, a fixed set of values ​​275 of discrete variables are calculated by a cyclic machine learning based prediction and iterative pre-solve based correction approach to compute a feasible but potentially suboptimal solution vector 255.

[0145] The optimal control structure CP 280 is based on minimization or maximization 281 of the sum of the stage cost functions, and the constraints may include MICP functions 247g0,g1,...,g N ,ψ0,ψ1,…,ψ N-1 and The equality constraints 282 on the initial state variables, the state dynamic equality constraints 283 and one or more inequality constraints 284, wherein the discrete optimization variable δ i,j Each of the 275, where i = 0, ..., N and j = 1, ..., n δ Note that after fixing the discrete variables 275, the discrete optimization variable δ is removed from the optimal control structured CP 280. 0:N and discrete feasibility constraints 265. In some embodiments of the invention, the optimal control structured CP may be efficiently solved using a convex optimization algorithm including, for example, an active set method, an interior point method, a gradient-based method, an operator splitting method, or an alternating direction method of multipliers (ADMM).

[0146] Some embodiments of the invention are based on the recognition that if MICP 270 can be formulated as a MILP, then CP 280 is a convex linear program (LP). Some embodiments of the invention are based on the recognition that if MICP 270 can be formulated as a MIQP, then CP 280 is a convex quadratic program (QP). Some embodiments of the invention are based on the recognition that if MICP 270 can be formulated as a MIQCQP, then CP 280 is a convex quadratically constrained quadratic program (QCQP). Some embodiments of the invention are based on the recognition that if MICP 270 can be formulated as a MISOCP, then CP 280 is a convex second-order cone program (SOCP).

[0147] Figure 3A A block diagram of a MICP solution method 260 is shown for computing a feasible but potentially suboptimal solution vector 255 defining a control signal 111 based on prediction 305 of a fixed set of values ​​for discrete optimization variables and solution 280 of an optimal control structured CP approximation given a current problem parameter value θ205. In some embodiments of the invention, the prediction of a fixed set of values ​​for discrete optimization variables 305 given a current problem parameter value θ205 is based on a parametric function approximation of a mapping between the problem parameters θ205 and discrete values ​​of the optimal MICP solution 260. Some embodiments of the invention are based on the recognition that if all discrete optimization variables are binary optimization variables, then the discrete values ​​of the optimal MICP solution 260 are equal to 0 or 1. In some embodiments of the invention, the prediction method 305 is based on the problem parameters θ205 and the optimal MICP solution 260 for the binary optimization variables i=0,1,...,N and j=1,2,...,n. δ Each of or The parameter function approximates the mapping between the values ​​of .

[0148] In some embodiments of the present invention, the prediction model may be deterministic, for example, including multilayer perceptrons, convolutional neural networks (CNNs), recurrent neural networks (RNNs), kernel regression, support vector machines, and other machine learning algorithms or any combination of such deterministic prediction models. Alternatively, in some embodiments of the present invention, the prediction model may be stochastic, for example, including Bayesian neural networks, neural processes, Gaussian processes, Kriging interpolation, and other machine learning algorithms or any combination of such deterministic and / or stochastic prediction models.

[0149] Figure 3B A block diagram of the MICP solving method 260 is shown, where the optimal control structured CP approximation is solved 280 to compute a feasible but potentially suboptimal solution vector 255 defining the control signal 111, based on prediction 305 of a fixed set of discrete optimization variables, followed by correction 310 of the fixed set of discrete optimization variables to increase the likelihood of feasibility and optimality, given current problem parameter values ​​θ 205. The prediction method 305 can be implemented as a correlation between the problem parameters θ 205 and the optimal MICP solver 260 for binary optimization variables i = 0, 1, ..., N and j = 1, 2, ..., n. δ Each of or However, some embodiments of the present invention are based on the recognition that: or Prediction value 305 of may result in a likelihood that solution vector 255 computed from solver 280 of CP approximation has a relatively low feasibility and / or optimality in practice.

[0150] In some embodiments of the invention, if a fixed set of values ​​for the discrete optimization variables results in a feasible and / or optimal CP solution 280 for the original MICP optimization problem 270, the correction method 310 does not adjust or The predicted value of 305. When the obtained convex optimization problem (CP) has a feasible optimal solution, the value set of the discrete variable Alternatively, if the CP problem 280 is infeasible after fixing the discrete optimization variables to the set of predicted values ​​305, the correction method can adjust one or more of the predicted values ​​to better approximate the optimal MICP solution 260 for the binary optimization variables i=0,1,...,N and j=1,2,...,n δ The optimal discrete solution for each of Thereby increasing the possibility 310 of feasibility and optimality of embodiments according to the present invention.

[0151] Finally, using the updated discrete value set By fixing each of the discrete optimization variables 310, the resulting CP 280 can be efficiently solved to compute a feasible optimal control solution vector 255 (which includes the trajectory of the values ​​of the state and control input variables at each time step in the prediction time horizon) that is likely to be (close to) the optimal control solution of the original MIOCP 270.

[0152] In some embodiments of the present invention, the correction method 310 is an iterative pre-solve based correction method, which is based on an iterative process, wherein each iteration performs a single pre-solve step, followed by fixing one or more discrete optimization variables to a predicted set of fixed values ​​305. In some embodiments of the present invention, the single pre-solve step itself includes an iterative process of one or more pre-solve operations to reduce the number of discrete and / or continuous optimization variables, tighten the bounds of the optimization variables and / or tighten the bounds of one or more inequality constraints. The iterative process of the single pre-solve step is then performed one or more iterations until the problem is detected to be infeasible, until insufficient progress is made between two consecutive iterations, or until a predetermined time limit is reached to ensure computational efficiency. Examples of pre-solve operations include domain propagation, bound strengthening, dual fixation, implicit variable substitution, coefficient strengthening, probing, detecting and removing redundant variables and / or constraints. In addition, each of the pre-solve operations can be performed on a single variable, a single constraint, multiple variables and / or multiple constraints.

[0153] Figure 3C A block diagram of a MICP solving method 260 is shown, based on multiple different predictions 321-322 for a fixed set of discrete optimization variables given a current problem parameter value θ 205, correcting the fixed set of discrete optimization variables after each prediction 331-332 to increase the likelihood of feasibility and optimality, and solving 341-342 of an optimal control structured CP approximation by selecting a feasible control solution 350 with the best optimality from the CP solution of each of the n corrected discrete solution predictions 350 to calculate a feasible but possibly suboptimal solution vector 255 after each correction. In some embodiments, the calculation of each of the discrete solution predictions can be performed in parallel. For example, the first prediction of the discrete solution guess is 321 can be compared with one or more other predictions of the discrete solution guess (where k = 2, ..., n e )322 are calculated in parallel. Similarly, the first prediction of the discrete solution guess The correction steps of 331 can be predicted by other (where k = 2, ..., n e ) are calculated in parallel in one or more correction steps 332. Finally, The optimal control structured CP solution 341 and the fixed value set (where k = 2, ..., n e ) are calculated in parallel to select the optimal control solution 350 to calculate the MICP solution vector 255.

[0154] Figure 3D A block diagram of a MICP solving method 260 is shown, given a current problem parameter value θ 205, based on a prediction 305 for a fixed set of values ​​for discrete optimization variables, which is used to fix a first subset of discrete variables in the MICP to predicted values ​​311 output by a training parameter function, followed by a pre-solve based correction to fix the remaining subset of discrete variables to values ​​uniquely defined by the fixed values ​​for the first subset of discrete variables and based on constraints in the MICP to increase the likelihood 312 of feasibility and optimality of the updated fixed set of values ​​for the discrete variables 310, followed by a solve 280 of an optimal control structured CP approximation to compute a feasible but potentially suboptimal solution vector 255 defining the control signal 111.

[0155] Figure 4A 4 is a block diagram of an offline data generation and supervised learning process 400 according to some embodiments of the present invention, which trains a machine learning-based predictor 415 and then uses the machine learning-based predictor in an online variable fixing and optimal control solution process 420 to calculate a solution vector 255 for optimally controlling a structured MICP. The offline data generation and supervised learning process 400 includes a first step of performing a computation on a problem parameter value θ 1 ,θ 2 ,…,θ M The parameter values ​​are sampled 405 and represent the MICP problem to be solved online for real-time predictive control of the hybrid system. For example, if the problem parameters include the current state of the controlled system, the data sampling step 405 ensures that a sufficiently large number of sampled parameter values ​​are within one or more regions of the feasible state space of the controlled system.

[0156] The offline data generation and supervised learning process 400 includes a second step of solving the MICP to compute the parameter value θ for each sampled problem 1 ,θ 2 ,…,θ M The optimal control solution of 405 (x * ,u * ,δ *) 410. In some embodiments of the invention, each MICP may be solved exactly 410, for example, using a branch and bound (B&B) method, a branch and cut method, a branch and price method, or any other variant of a tree search based optimization algorithm. Alternatively, in some embodiments of the invention, each MICP may be solved approximately 410, for example, based on a rounding and pumping scheme, using an approximate optimization algorithm, approximate dynamic programming, or using a sequential convex programming (SCP) technique.

[0157] The offline data generation and supervised learning process 400 includes a third step of training a machine learning network using supervised learning based on a training data set of sampled parameter values ​​405 and corresponding MICP solutions 410 for performing a machine learning-based discrete optimal solution δ given a new set of problem parameter values. * 415. In some embodiments of the invention, training of a machine learning network involves computation of network weights φ that minimize a training loss function using, for example, stochastic gradient descent, Adam, AdaGrad, RMSProp, or any other variant of a gradient-based or gradient-free optimization algorithm.

[0158] The online variable fixing and optimal control solution process 420 includes a first step of evaluating a machine learning based predictor to calculate θ given the current problem parameter value θ 425. 430. Discrete solution guessing given predictions In the case of 430, the online variable fixing and optimal control solution process 420 includes performing a second step of correcting the discrete solution guess, thereby generating an updated solution guess. 435 to improve feasibility and / or optimality. Finally, the third step is based on The solution of the optimal control structured CP 440. In some embodiments of the present invention, the correction method 435 is an iterative pre-solve based correction method, which is based on an iterative process, in which each iteration performs a single pre-solve step and then fixes one or more discrete optimization variables to a predicted fixed value set 430.

[0159] Some embodiments of the present invention are based on the recognition that the offline data generation and supervised learning process 400 can be performed offline on a high-performance computer, so there are no strict requirements on the computational complexity and the worst-case computation time. On the other hand, embodiments of the present invention are based on the recognition that the online variable fixing and optimal control solution process 420 needs to be performed online, which has strict requirements on the computational complexity and the worst-case computation time, and the online process is usually performed on an embedded microprocessor with limited memory and limited computing power (for example, using an embedded control unit (ECU) of a vehicle).

[0160] Figure 4B and Figure 4C Flowcharts and pseudocode are shown for an offline data generation process 400 based on solving 410 the MICP for a set of sampled problem parameter values ​​405 and a supervised learning process to train a machine learning based predictor 415 based on a set of discrete optimizers Δ 453 and a training data set D 454, respectively. In some embodiments of the invention, the offline data generation 400 is an iterative process that checks whether the training data set D is large enough 455. If the training data set D is large enough 455, the process initiates the training 415 of the machine learning based predictor, i.e., calculates the network weights φ that minimize the training loss function, using stochastic gradient descent, Adam, AdaGrad, RMSProp, or any other variant of a gradient-based or gradient-free optimization algorithm. In some embodiments of the invention, the offline data generation process 400 returns the trained machine learning network h φ and discrete optimizer ensemble Δ456.

[0161] If the training data set D is not large enough, then the new sampling problem parameter value θ is calculated k 450 performs data generation 400 and solves the MICP problem to calculate the parameter value θ for the new sampling problem k The optimal control solution (x k,* ,u k,* ,δ k,* )451. If the MICP solution 451 is not feasible 452, then it is necessary to calculate a new sampling problem parameter value θ k 450. In some embodiments of the present invention, the new sampling problem parameter value θ k 450 can be obtained from the pre-computed batch problem parameter values ​​{θ k} k=1,…,M 445. At the beginning of the offline data generation process 400, the binary optimizer set Δ and the training data set D are each initialized to be equal to the empty set 401.

[0162] If the MICP solution 451 is feasible 452, then the optimal discrete solution value is Add to the discrete optimizer set Δ to perform data generation 400, and for each time step i=0,…,N in the prediction time horizon, for the optimal discrete solution value Identify unique class labels 453. In some embodiments of the invention, if the MICP solution 451 is feasible 452, then the sampled solution values ​​for each time step i=0, ..., N are Added to the training data set D454 for offline data generation 400.

[0163] Some embodiments of the present invention are based on the recognition that two or more sampling problem parameter values ​​θ k and θ l This can lead to the same optimal discrete solution value for one or more time steps i∈{0,1,…,N} within the prediction time horizon, that is, In this case, the corresponding class labels are equal to each other Because the class label value is the unique identifier of the discrete optimal solution value in the discrete optimizer set Δ.

[0164] Embodiments of the present invention are based on the recognition that by exploiting the temporal structure of discrete MIOCP solutions, the number of target class labels in supervised learning can be greatly reduced, which may lead to improved sampling efficiency and / or improved performance of machine learning-based predictors. For example, at each time step i=0,1,…,N given n δ In the case of discrete variables, the maximum number of target class label values ​​is By exploiting the temporal structure of MIOCP, this can be reduced to a maximum target class label value. In addition, some embodiments of the present invention are based on the recognition that a unique discrete value The number of can be significantly smaller than the theoretical maximum number of target class label values ​​due to sampling problem parameter values ​​that are representative of the optimal control problem of interest in practice and due to constraints on the feasibility of the MIOCP problem.

[0165] Figure 4D and Figure 4E 4 and 4. The flowchart and pseudo code of the online variable fixing process 420 based on prediction 430 and the correction step 435 based on iterative pre-solution and the optimal control structured CP solving process 440 for calculating the feasible MICP solution vector 255 are respectively shown according to some embodiments of the present invention. In some embodiments of the present invention, given the problem parameter value θ, the discrete optimizer set Δ, the trained machine learning network h φ and the maximum number of evaluations n evals In the case of 470, the trained machine learning network 415 can be used to predict one or more candidate discrete solutions 460. At the beginning of the online variable fixing and optimal control solution process 420, the initialization step 461 sets k←1, and the optimal control solution z * =(x * ,u * ) and δ * are all set to empty sets, and the optimal target value is set to infinity J * ←∞.

[0166] In some embodiments of the present invention, online variable fixing 420 is an iterative process, and each iteration includes guessing the solution based on the discrete solution. To calculate the new discrete solution value The iterative pre-solve based correction step 462. If the correction is unsuccessful 463, for example, because the discrete solution guessed using the prediction If no feasible solution is found, the process continues with the next iteration, and the number of iterations can be updated to k←k+1 468. If the correction successfully calculates the new value 462, which has an increased likelihood of feasibility and / or optimality, the online variable fixing process 420 is performed by fixing discrete values Solve the optimal control structured CP, and continue to calculate feasible and optimal control solutions and the corresponding target value 464.

[0167] For fixed discrete values The optimal control structured CP does not have a feasible solution or the corresponding target value is greater than the current optimal target value 465, the process continues to the next iteration, and the number of iterations can be updated k←k+1468. The feasible solution of the optimal control structured CP, and if the corresponding target value is less than the current optimal target value 465, then the process updates the optimal control solution Update the optimal discrete solution And update the optimal target value 466. In some embodiments of the present invention, a maximum number of evaluations n is defined. evals , so that the maximum number of evaluations reaches k+1>n evals The online variable fixing process 420 is terminated at 467 to limit the total computation time of the online variable fixing and optimal control solution process.

[0168] If the maximum number of evaluations has not been reached 467, the process continues with the next iteration and the number of iterations may be updated k←k+1 468. If the maximum number of evaluations has been reached 467, the optimal control solution vector (z * ,δ * ,J * )255 to implement a predictive controller according to an embodiment of the present invention.

[0169] In some embodiments of the invention, the iterative pre-solve based correction method 462 may return any of the following results:

[0170] The iterative correction method detects a discrete set of predicted values is feasible, thus returning the value

[0171]

[0172] Iterative correction method calculates a feasible set of updated discrete values Make the return value

[0173] • Iterative correction methods may not be able to compute a feasible set of updated discrete values ​​within a predetermined time limit.

[0174] Embodiments of the present invention are based on the recognition that an iterative pre-solve based correction method can increase the likelihood of feasibility and / or optimality, for example, when the iterative correction method removes or prunes a discrete variable that causes a machine learning based predictor to provide an incorrect predicted value that would result in an infeasible control solution.

[0175] Figure 5A An example of an iterative pre-solve based correction process 462 according to some embodiments of the present invention is shown. The online variable fixing and optimal control solution process 420 includes a first step of evaluating a machine learning based predictor to calculate given a current problem parameter value θ 425 for the MICP 430, thereby generating one or more candidate binary solutions 500, such as Figure 5A In some embodiments of the invention, the iterative pre-solve based correction process 462 begins with a first single pre-solve step 510 for the MICP, which may result in one or more fixations or trimmings of discrete optimization variables. Figure 5A In the first single pre-solve step 510, the fixed discrete variables are obtained and 505a, but the remaining free discrete variables have not been fixed, i.e. and 505b.

[0176] After fixing one or more discrete optimization variables, based on the first single pre-solve step 510, the remaining free discrete variables are fixed by selecting them and guessing 500 based on the predicted binary solution (e.g., 515), the iterative pre-solve based correction process 462 continues. If no remaining free discrete variables exist, then a complete updated binary solution guess 463 has been found. After 515, in some embodiments of the present invention, the iterative pre-solution based correction process 462 continues to perform a second single pre-solution step 511 for MICP. After the second single pre-solution step 511, all discrete variables fixed by the first single pre-solution step 510, i.e. and 505a remains fixed, and the discrete variable 515 has been fixed by the machine learning based predictor, and the second single pre-solve step 511 can obtain additional fixed discrete variables, e.g. 506a. For example, the second single pre-solve step can detect the predicted solution guess is not feasible, so it can be guaranteed that the discrete variable can be fixed to 511. Figure 5A For example, the remaining free discrete variables have not been fixed, that is, and 506b.

[0177] After the second single pre-solve step 511, the remaining free discrete variable is fixed by selecting it and guessing 500 the predicted binary solution (e.g. 516), the iterative pre-solve based correction process 462 continues. If no remaining free discrete variables exist, then a complete updated binary solution guess 463 has been found. After 516, the iterative pre-solve based correction process 462 continues in a similar manner with one or more additional single pre-solve steps and variable fixing 517 until a complete updated binary solution guess is found 463.

[0178] Some embodiments of the present invention are based on the recognition that any single pre-solve step may detect an infeasible solution, in which case the iterative pre-solve based correction process 462 may be terminated immediately without a successful correction 463 to reduce the overall computation time.

[0179] In some embodiments of the invention, each iteration of the iterative correction method performs a single pre-solve step, the purpose of which is to reduce the number of discrete optimization variables in the MIOCP by fixing each removed or pruned variable to an optimal and / or uniquely feasible value. The single pre-solve step preserves feasibility and optimality, i.e., any feasible or optimal solution to the simplified MICP problem can be mapped to a feasible or optimal solution to the original MICP problem with the same objective value, and the simplified MICP problem is infeasible or unbounded if and only if the original MICP problem is infeasible or unbounded. Subsequently, if the single pre-solve step does not detect infeasibility and not all discrete variables have been fixed, the iterative correction method selects one or more remaining free discrete variables to fix them to solution values ​​provided by the machine learning-based predictor, and performs a single pre-solve step in the next iteration of the iterative correction method. After one or more iterations, if all discrete variables have been fixed, or if infeasibility has been detected, the correction method based on iterative pre-solve terminates.

[0180] Some embodiments of the invention are based on the recognition that if infeasibility is detected by an iterative pre-solve based correction method, an alternative control solution vector may be used to change the state of the plant.

[0181] In some embodiments of the invention, a single pre-solve step may reuse one or more computational results from a previous single pre-solve step in order to improve the computational efficiency of the correction method based on iterative pre-solve. In some embodiments of the invention, the number of iterations in the iterative correction method is limited by a predetermined upper bound in order to improve computational efficiency. If the predetermined upper bound on the number of iterations has been reached and not all discrete variables have been fixed, all remaining free discrete variables may be fixed to the solution values ​​provided by the machine learning based predictor.

[0182] Figure 5B and Figure 5C Flowchart and pseudo code of an online variable fixing and optimal control solution process 420 according to some embodiments of the present invention are shown, which uses prediction 305 and a correction step based on one or more individual pre-solution steps 520 to calculate a solution vector 255 for an optimal control structured MICP 270 given a current problem parameter value θ 205. The first step of the online variable fixing process is to convert the index set and discrete value sets Initialized to an empty set 521. In some embodiments of the present invention, given a fixed discrete variable set in And in order to calculate the updated index set and the updated discrete value set Each iteration of the iterative pre-solve based correction process performs a single pre-solve step for the MICP 520. Some embodiments of the present invention are based on the recognition that the updated index set and the updated discrete value set must include at least the original index set and the original discrete value set, since each single pre-solve step can only result in fixing or pruning more discrete optimization variables.

[0183] If the single pre-solve step is unsuccessful 525, for example, if the single pre-solve step results in an infeasible solution, then the correction based on the iterative pre-solve fails to find a feasible solution 530. Alternatively, if the single pre-solve step is successful 525, and all discrete optimization variables have been fixed 535, then the resulting optimal control structured CP is solved to calculate the optimal control solution vector 280 based on the successful correction 536 of the fixed discrete values. Alternatively, if the single pre-solve step is successful 525, but not all discrete optimization variables have been fixed 535, then the optimal control solution vector 280 is calculated by updating the index set Sum The process continues 541 and the remaining free variables are selected to be fixed 540 based on the machine learning based predictions, i.e. And update the index set After fixing the remaining free variables 540 , in some embodiments of the invention, the iterative pre-solve based correction continues by performing one or more additional iterations, and each iteration may include another single pre-solve step 520 and a variable selection and fixing step 540 .

[0184] Some embodiments of the present invention are based on the recognition that a maximum number of individual pre-solve steps 520 can be performed in an iterative pre-solve based correction method, after which all remaining free variables are fixed to a machine learning based predictive discrete solution guess 540, so as to limit the maximum computation time of the online variable fixing and optimal control solution process 420 and allow real-time implementation of a predictive controller according to embodiments of the present invention.

[0185] Some embodiments of the present invention are based on the recognition that a trained parameter function approximation framework can provide a probability or confidence level for the predicted value of each discrete variable in the MICP solution, so the remaining free discrete variables that maximize the probability or confidence level of the trained parameter function approximation framework can be selected 540. Alternatively, in some embodiments of the present invention, the remaining free discrete variables can be randomly selected 540.

[0186] Fig. 6A An example of domain propagation for bound enforcement of optimization variables in the MICP problem 270 is shown, which may be used in a single pre-solve step 520 of the iterative pre-solve based correction process 435, according to some embodiments of the present invention. Fig. 6AAn example 601 of a MICP optimization problem with decision variables [x0, u0, δ0, x1, u1, δ1] and multiple equality and inequality constraints is shown. Assume that variables δ0, δ1 are binary decision variables, i.e., δ0, δ1∈{0, 1}. The first step of domain propagation 600 is to calculate the initial bounds 602 for each optimization variable, e.g., 0≤δ0≤1, 0≤δ1≤1, since they are binary decision variables δ0, δ1∈{0, 1}. In some embodiments of the invention, domain propagation is an iterative process, and each iteration updates the variable bounds 603 based on each inequality constraint and updates the variable bounds 604 based on each equality constraint, followed by another update 605 of the variable bounds based on each inequality constraint, and the process continues 606 until the termination condition is met.

[0187] For example, in some embodiments of the invention, the iterative process of domain propagation 600 terminates when all discrete optimization variables are fixed, or when the MICP problem is detected to be infeasible, or when insufficient progress is made from one iteration to the next, or when a maximum computation time is reached. The progress of domain propagation from one iteration to the next can be measured based on the tightening of one or more variable bounds from one iteration to the next. For example, in Fig. 6A , in the first update 603 of the variable bounds based on each inequality constraint, progress is made through domain propagation, resulting in the fixation of the binary decision variable δ0=1. In addition, the second update 604 of the variable bounds based on each equality constraint results in the tightening of the variable bounds from -∞≤x1≤∞ to the updated variable bounds -1≤x1≤1. Similarly, the third update 605 of the variable bounds based on each inequality constraint results in the fixation of the binary decision variable δ1=1.

[0188] Figure 6B An example of redundant inequality constraint detection and dual fixation 630 of optimization variables in a MICP problem 270 according to some embodiments of the present invention is shown, which may be used in a single pre-solve step 520 of an iterative pre-solve based correction process 435. Some embodiments of the present invention are based on the recognition that the detection and removal of redundant inequality constraints and dual fixation of optimization variables may result in additional fixation of discrete variables and / or it may reduce the computational cost of a single pre-solve step 520 of an iterative pre-solve based correction process 435. In some embodiments of the present invention, redundant inequality constraint detection and dual fixation of variables 630 may be used in conjunction with domain propagation 600 for bound enforcement and / or in conjunction with one or more other pre-solve operations in a single pre-solve step 520 of an iterative pre-solve based correction process 435.

[0189] Figure 6BAn example 631 of a MICP optimization problem with decision variables [x0, u0, δ0] and containing multiple inequality constraints is shown, where the variable δ0∈{0,1} is a binary decision variable. Since u0≤1 and δ0≤1, it can be detected that the following inequality constraint u0+δ0≤2 is redundant 632, and therefore the latter inequality constraint can be removed from the MICP optimization problem in order to reduce the computational cost of a single pre-solve step 520 in the iterative pre-solve based correction process 435. After removing the redundant inequality constraints, if the discrete variable does not appear in the equality constraints and it appears with the same sign in all inequality constraints, such as 2≤x0+Mδ0, the discrete variable δ0=1 633 can be fixed using dual fixing. In addition, this example of dual fixing assumes that the variable δ0 does not appear in the cost function, or it appears as the maximum of the values ​​of δ0∈{0,1}. In some embodiments of the invention, redundant inequality constraint detection and dual fixation 630 of optimization variables is an iterative process that continues 634 until all discrete optimization variables are fixed, or until the MICP problem is detected to be infeasible, or until insufficient progress is made from one iteration to the next, or until a maximum computation time is reached.

[0190] Figure 6C An example of inequality constraint coefficient enforcement 650 for the MICP optimization problem 270 according to some embodiments of the present invention is shown, which may be used in a single pre-solve step 520 of the iterative pre-solve based correction process 435. Some embodiments of the present invention are based on the recognition that inequality constraint coefficient enforcement may result in additional fixation of discrete variables and / or that it may reduce the computational cost of a single pre-solve step 520 of the iterative pre-solve based correction process 435. In some embodiments of the present invention, inequality constraint coefficient enforcement 650 may be used in conjunction with domain propagation 600 for bound enforcement, in conjunction with redundant inequality constraint detection and dual fixation 630 of variables, and / or in conjunction with one or more other pre-solve operations within a single pre-solve step 520 of the iterative pre-solve based correction process 435.

[0191] Figure 6CAn example 651 of a MICP optimization problem with decision variables [x0, u0, δ0] and containing multiple inequality constraints is shown, where the variable δ0∈{0, 1} is a binary decision variable. Using the inequality constraint x0≥1 and using the bound 0≤δ0≤1 of the discrete variable, the following inequality constraint 2≤x0+10060 can be replaced by the following dominant inequality constraint 2≤x0+δ0652 without removing any feasible solutions to the original MICP optimization problem. More specifically, it can be shown that any feasible MICP solution that satisfies the original inequality constraint 2≤x0+10060 also satisfies the stricter inequality constraint 2≤x0+δ0652. The inequality constraint coefficient is strengthened 652, which reduces the feasible space of the relaxed optimization problem after relaxing the discrete variable δ0∈{0, 1} to 0≤δ0≤1, and the resulting inequality constraint 2≤x0+60 is therefore considered to be the dominant inequality constraint. In some embodiments of the invention, inequality constraint coefficient enforcement is an iterative process that continues 653 until all discrete optimization variables are fixed, or until the MICP problem is detected to be infeasible, or until insufficient progress is made from one iteration to the next, or until a maximum computation time is reached.

[0192] Some embodiments of the invention are based on the recognition that inequality constraint coefficient enforcement can be used to replace one or more inequality constraints by one or more dominant inequality constraints, and that dominant inequality constraints can be used to reduce the feasible search space of the relaxed problem without removing integer feasible solutions to the MICP.

[0193] Fig.6D An example of binary variable detection 670 for the MICP optimization problem 270 is shown, which may be used in a single pre-solve step 520 of the iterative pre-solve based correction process 435, according to some embodiments of the present invention. Some embodiments of the present invention are based on the recognition that the binary variable detection 670 may lead to additional fixation of discrete optimization variables, which may improve efficiency and / or it may reduce the computational cost of the single pre-solve step 520 of the iterative pre-solve based correction process 435. In some embodiments of the present invention, the binary variable detection 670 may be used in conjunction with the domain propagation 600 for bound enforcement, in conjunction with the redundant inequality constraint detection and dual fixation 630 of variables, in conjunction with the inequality constraint coefficient enforcement 650, and / or in conjunction with one or more other pre-solve operations within the single pre-solve step 520 of the iterative pre-solve based correction process 435.

[0194] Fig.6DAn example 671 of a MICP optimization problem with decision variables [x0, u0, δ0, x1, u1, δ1] and containing multiple equality constraints and multiple inequality constraints is shown, wherein variables δ0∈{0,1} and δ1∈{0,1} are binary decision variables. In some embodiments of the invention, binary variable detection 670 selects binary variables, such as δ1∈{0,1}, and the process performs one or more pre-solve operations 672 on MICP subproblem 1 with a fixed discrete value δ1=0, and it additionally performs one or more pre-solve operations 673 on MICP subproblem 2 with a fixed discrete value δ1=1. Some embodiments of the invention are based on the recognition that one or more pre-solve operations 672 for MICP subproblem 1 and one or more pre-solve operations 673 for MICP subproblem 2 can be used to detect that MICP subproblem 1 and / or MICP subproblem 2 are infeasible 674. If only MICP subproblem 2 is detected as infeasible 674, δ1=0 may be fixed 675, and the bounds of each of the optimization variables and constraints may be updated to the bounds from the pre-solve operation 672 of MICP subproblem 1. If only MICP subproblem 1 is detected as infeasible 674, δ1=1 may be fixed 677, and the bounds of each of the optimization variables and constraints may be updated to the bounds from the pre-solve operation 673 of MICP subproblem 2. If both MICP subproblem 1 and MICP subproblem 2 are detected as infeasible 674, the original MICP optimization problem is detected as infeasible 676.

[0195] Some embodiments of the present invention are based on the recognition that binary variable detection 670 can be performed for one or more remaining free discrete variables in a single pre-solve step 520 of an iterative pre-solve based correction process 435. In some embodiments of the present invention, the number of binary variable detection 670 iterations is limited and the number of pre-solve operations in each iteration of binary variable detection 670 is limited in order to limit the maximum computation time of the online variable fixing and optimal control solution process 420 and to allow real-time implementation of a predictive controller according to embodiments of the present invention.

[0196] Some embodiments of the present invention are based on the recognition that one or more pre-solve operations within a single pre-solve step 520 of an iterative pre-solve based correction process 435 may result in a much lower computational cost to solve the optimal control structured CP 440 due to a reduced number of optimization variables, a reduced number of constraints, and / or a reduced feasible space of the optimization problem (e.g., due to a combination of bound hardening and constraint tightening).

[0197] Some embodiments of the invention are based on the recognition that each iteration of a single presolve step 520 may include one or more customized block sparse presolve operations that exploit the block structured sparsity of the mixed integer optimal control problem (MIOCP) to perform tasks defined by the problem parameters.

[0198] Fig. 7A is a pseudo code for implementing the block sparse forward-backward operation of the domain propagation and bounds enforcement process 600, which can be used in a single pre-solve step 520 of the iterative pre-solve based correction process 435. The input 701 of the domain propagation process 700 may include the current bounds of each of the optimization variables and constraints, and the block sparse matrices and vectors in the MIOCP data. In some embodiments of the invention, one iteration of the block sparse domain propagation process 700 includes a forward domain propagation process 705 for each time step i=0, 1, ..., N over the prediction time horizon and a backward domain propagation process 715 for each time step i=N-1, N-2, ..., 0 over the prediction time horizon.

[0199] Let us use the compact notation to optimize the variable z for each time step i=0,1,...,N over the forecast time horizon i , including all state variables x i , all control input variables u i and all discrete variables δ i In some embodiments of the invention, each iteration of the forward domain propagation process 705 may use an affine inequality constraint 706 is to optimize the variable z for each time step i=0, 1, ..., N in the prediction time horizon i,j , j = 1, ..., n z Each of performs optimization-based bound tightening. Specifically, the affine inequality constraints 706, combined with threshold Combined with the binary variable k∈B i Each constraint z in i,k ∈{0, 1}, can be used to calculate the optimization variable z for each time step i=0, 1, ..., N over the prediction time horizon. i,j , j = 1, ..., n z The minimum of each z i,j 707 and Max 708 Boundary value. Some embodiments of the present invention are based on the recognition that for each time step i=0, 1, ..., N in the prediction time horizon, the optimization variable z i,j , j = 1, ..., n zFor any optimization variable in It can then be detected that the MIOCP subproblem is infeasible 709.

[0200] In addition, each iteration of the forward domain propagation process 705 for each time step i=0, 1, ..., N-1 over the prediction time horizon may use the state dynamic equality constraint x i+1 =a i +F i z i 710 is used to perform the optimization of the variables x for each time step i=0, 1, ..., N-1 in the prediction time horizon. i+1,j , j = 1, ..., n x Optimization-based tightening of the bounds of each of . Specifically, the state dynamic equality constraint x i+1 =a i +F i z i 710 and threshold Combined, it can be used to calculate the optimization variable x for each time step i = 0, 1, ..., N-1 on the prediction time horizon i+1,j , j = 1, ..., n x The minimum of each x i+1,j and maximum 711 Boundary value. Some embodiments of the present invention are based on the following understanding: for each time step i=0, 1, ..., N-1 of the optimization variable x i+1,j , j = 1, ..., n x For any optimization variable in It can then be detected that the MIOCP subproblem is infeasible 712 .

[0201] In some embodiments of the invention, each iteration of the back-propagation process 715 may use the state dynamic equality constraint x i+1 =a i +F i z i 710 to perform the optimization of the variable z for each time step i=N-1, N-2, ..., 0 on the prediction time horizon i,j , j = 1, ..., n z Optimization-based tightening of the bounds of each of . Specifically, the state dynamic equality constraint x i+1 =a i +F i z i 710, combined with threshold And combined with the binary variable k∈B iEach constraint z in i,k ∈{0, 1}, can be used to calculate the optimization variable z for each time step i=N-1, N-2, ..., 0 over the prediction time horizon i,j , j = 1, ..., n z The minimum of each z i,j 716 and max 717 Boundary value. Some embodiments of the present invention are based on the recognition that for each time step i=N-1, N-2, ..., 0 in the prediction time horizon, the optimization variable z i,j , j = 1, ..., n z For any optimization variable in It can then be detected that the MIOCP subproblem is infeasible 718.

[0202] In some embodiments of the invention, the output of the domain propagation process may include updated bounds for each time step i=0, 1, ..., N over the prediction time horizon. z i and 719.

[0203] Figure 7B is a pseudo code of an optimized bound-tightening-based efficient block-sparse computation 720 for MICP optimization variables, which can be used as a basic computational step in one or more pre-solve operations within a single pre-solve step 520 of the iterative pre-solve-based correction process 435. For example, in some embodiments of the present invention, the optimized bound-tightening-based efficient block-sparse computation 720 can be used to compute the optimization variable z for each time step i=0, 1, ..., N over the prediction time horizon. i,j , j = 1, ..., n z The minimum of each z i,j Boundary value 707-716 and maximum Bounds 708-717. Inputs 721 to the optimization-based bound tightening block sparse computation 720 may include current bounds for each of the optimization variables and constraints, as well as block sparse matrices and vectors in the MIOCP data.

[0204] Some embodiments of the present invention are based on the recognition that the efficient block sparse computation 720 based on optimized bound tightening is effective for k=1,2,...,n c Considering affine inequality constraints alone Each 722 in order to calculate the optimization variable z for each time step i=0, 1, ..., N over the prediction time horizon i,j , j = 1, ..., n zThe updated threshold for each and 726. For each affine inequality constraint Each of 722 and the optimization variable z i,j For each of the above, an efficient block sparse calculation based on the optimized bound tightening 720 calculates the intermediate bound value d k , (If E k,j >∈)723 or d k , (If E k,j <-∈)724, such as Figure 7B Note that if |E k,j |<∈, then the intermediate boundary value is d k =-∞、 Wherein,∈>0 is defined as a small threshold. In some embodiments of the present invention, if the optimization variable z i,j is an integer variable, for example, if the optimization variable is a binary variable z i,j ∈{0,1}, then the intermediate value is executed and Rounding step 725.

[0205] Some embodiments of the present invention are based on the recognition that for each time step i=0,1,...,N over the prediction time horizon, the optimization variable z i,j , j=1,…,n z The minimum of each z i,j Boundary value 707-716 and maximum Cutoffs 708-717, updated cutoffs and 726 are potential conservative lower and upper bounds 727 respectively.

[0206] Figure 7C is pseudo code for block sparse redundant inequality constraint detection and dual fixation 730 of MICP optimization variables that may be used in a single pre-solve step 520 of an iterative pre-solve based correction process 435 according to some embodiments. The input 731 of the block sparse redundant inequality constraint detection and dual fixation 730 may include the optimization variables z i , The current limit of each of them and the constraints for each time step i=0,1,…,N in the prediction time horizon The current bounds for each of , and block sparse matrices and vectors in the MIOCP data.

[0207] In some embodiments of the present invention, each iteration of the block sparse redundant inequality constraint detection and dual fixation 730 process may use an affine inequality constraint 732 to calculate the intermediate value and like Figure 7C Some embodiments of the present invention are based on the following understanding: Based on the calculation of the intermediate threshold in 733, if or if Then the MIOCP may be detected as infeasible 734. In some embodiments of the present invention, each iteration of the block sparse redundant inequality constraint detection and dual fixation 730 process may use an affine inequality constraint 732 to calculate another intermediate value and like Figure 7C As shown in pseudo code 735, it can be used to detect that the upper bound constraint is redundant 736 and / or detect that the lower bound constraint is redundant 737, that is, so that the corresponding constraint limit values ​​can be updated as and / or

[0208] In some embodiments of the present invention, each iteration of the block sparse redundant inequality constraint detection and dual fixation 730 process may use the resulting non-redundant inequality constraints for the dual fixation step. Figure 7C As shown in the pseudo code of , if the optimization variable does not appear in any equality constraint, does not appear in the quadratic objective function, and appears with the same sign in all inequality constraints 741, then the dual fixation can result in fixing the optimization variable at its lower bound 742 or fixing the optimization variable at its upper bound 743. The output 745 of the block sparse redundant inequality constraint detection and dual fixation 730 may include the optimization variable for each time step i = 0, 1, ..., N over the prediction time horizon. z i , The updated bounds and constraints for each of The updated bound for each of .

[0209] Fig.7D is pseudo code for a block sparse inequality constraint coefficient enforcement 750 corresponding to discrete optimization variables in the MICP problem 270, which may be used in a single pre-solve step 520 of the iterative pre-solve based correction process 435, according to some embodiments. The input 751 of the block sparse inequality constraint coefficient enforcement 750 may include the optimization variables for each time step i=0,1,...,N over the prediction time horizon z i , The current bounds and constraints for each of The current bounds for each of , and block sparse matrices and vectors in the MIOCP data.

[0210] In some embodiments of the present invention, each iteration of the block sparse inequality constraint coefficient enforcement 750 process may use an affine inequality constraint 752 to calculate the intermediate value and like Fig.7D Some embodiments of the present invention are based on the recognition that for each of the integer or discrete optimization variables that have not been fixed to a fixed value, that is, 754, the block sparse inequality constraint coefficient enforcement 750 may be performed. In some embodiments of the present invention, if the lower bound is redundant The upper bound is non-redundant 755, or if the upper bound is redundant The lower bound is non-redundant 760, then we can use the affine inequality constraint Each of the execution blocks 752 performs sparse inequality constraint coefficient enforcement 750 .

[0211] If only the upper bound is non-redundant 755, the intermediate value can be calculated as shown in 756 It can be used to respectively k,j >∈ is positive or D k,j <-∈ is negative, strengthen the inequality constraint coefficient value D k,j and upper bound like Fig.7D 757-758 of the pseudocode. If only the lower bound is non-redundant 760, the intermediate value can be calculated as shown in 761 It can be used to respectively k,j >∈ is positive or D k,j <-∈ is negative, strengthen the inequality constraint coefficient value D k,j and lower bound like Fig.7D The output 765 of the block sparse inequality constraint coefficient enforcement 750 may include a constraint for each time step i=0, 1, ..., N in the prediction time horizon. and the constraint matrix D i The updated bound for each of .

[0212] Fig. 7Eis pseudo code for a block sparse binary variable detection process 770 for discrete optimization variables in the MICP problem 270, which can be used in a single pre-solve step 520 of the iterative pre-solve based correction process 435. The input 771 of the block sparse binary variable detection process 770 may include the optimization variables for each time step i=0,1,...,N over the prediction time horizon z i , The current bounds of each of , and block sparse matrices and vectors in the MIOCP data. Some embodiments of the present invention are based on the recognition that integer or discrete optimization variables (i.e., where j = 1, 2, ..., n δ ) and performs a block sparse binary variable detection process 770 for each time step i=0,1,…,N over the prediction time horizon 772.

[0213] In some embodiments of the present invention, based on one or more forward-backward operations 777-782 of the domain propagation and bound strengthening process 700, each iteration of the block sparse binary variable detection process 770 converts the binary optimization variable δ i,j ∈{0,1} to detect lower bounds 775 and upper bounds 780 in order to detect infeasibility 785 or to fix the binary optimization variable to an upper bound δ i,j = 1 786 or fix the binary optimization variable to the lower bound δ i,j =0 787.

[0214] In some embodiments of the present invention, when the binary optimization variable δ i,j ∈{0,1} After detecting the lower bound 775, the block sparse binary variable detection process 770 performs one or more iterations of the pre-solve operation 777 to update the intermediate bounds for each time step i=0,1,…,N in the prediction time horizon For example, each iteration 777 may include a forward and backward domain propagation and bound strengthening process 700, after which the infeasibility flag and the iteration number n←n+1 are updated 778. Some embodiments of the invention are based on the recognition that the iteration process may continue until insufficient progress from one iteration to the next is made, until infeasibility is detected by a pre-solve operation, or until a maximum computation time is reached, e.g., until n≥n probing , where n probing The maximum iteration limit is 776.

[0215] In some embodiments of the present invention, when the binary optimization variable δ i,j∈{0,1} After detecting the upper bound 780, the block sparse binary variable detection process 770 performs one or more iterations of the pre-solve operation 782 to update the intermediate bounds for each time step i=0,1,…,N in the prediction time horizon For example, each iteration 782 may include a forward and backward domain propagation and bound strengthening process 700, after which the infeasibility flag and the iteration number n←n+1 are updated 783. Some embodiments of the invention are based on the recognition that the iteration process may continue until insufficient progress from one iteration to the next is made, until infeasibility is detected by a pre-solve operation, or until a maximum computation time is reached, e.g., until n≥n probing , where n probing The maximum iteration limit is 781.

[0216] Some embodiments of the present invention are based on the recognition that if the binary optimization variable δ i,j ∈{0,1} detects the lower bound 775 and converts the binary optimization variable δ i,j ∈{0,1} detects infeasibility after detecting the upper bound 780, then it is detected that the MICP problem is infeasible 785. Some embodiments of the present invention are based on the recognition that if the binary optimization variable δ i,j ∈{0,1} detects infeasibility after detecting the lower bound value 775, then the binary optimization variable can be fixed to the upper bound δ i,j =1, and for each time step i=0,1,…,N on the prediction time horizon 786, all thresholds can be updated to the intermediate thresholds Some embodiments of the present invention are based on the recognition that if the binary optimization variable δ i,j ∈{0,1} detects infeasibility after detecting the upper bound value 780, then the binary optimization variable can be fixed to the lower bound δ i,j = 0, and for each time step i = 0, 1, ..., N on the prediction time horizon 787, all thresholds can be updated to the intermediate thresholds The output 788 from the block sparse binary variable detection process 770 may include the optimization variables for each time step i=0, 1, ..., N over the prediction time horizon. z i , The updated bound for each of .

[0217] Figure 7Fis pseudo code for a single block sparse pre-solve step 790 based on one or more pre-solve operations that may be used in the iterative pre-solve based correction process 435 according to some embodiments. The input 791 of the block sparse pre-solve step 790 may include block sparse matrices and vectors in the MIOCP problem data, and may include one or more flags for enabling or disabling one or more pre-solve operations, such as a flag for enabling or disabling the block sparse binary variable detection process 770. The output 795 from the block sparse pre-solve step 790 may include an infeasibility detection flag, an optimization variable detection flag, and a prediction time horizon for each time step i=0,1,...,N of the MIOCP problem data. z i , The updated bounds and constraints for each and the updated constraint matrix D i The updated bound for each of .

[0218] In some embodiments of the invention, the single block sparse pre-solve step 790 is an iterative process 794 that may continue until insufficient progress is made from one iteration to the next, until infeasibility is detected by the pre-solve operation, or until a maximum computation time is reached, e.g., until n ≥ n max , where n max is the maximum iteration limit 792. In some embodiments of the present invention, each iteration of the single block sparse pre-solve step 790 may include the forward-backward domain propagation and bounds enforcement process 700, the block sparse redundant inequality constraint detection and dual fixation 730, the block sparse inequality constraint coefficient enforcement 750 and / or the block sparse binary variable detection process 770. In some embodiments of the present invention, the block sparse binary variable detection process 770 is performed only when MICP infeasibility is not detected, the detection flag is enabled, the progress obtained from one iteration to the next iteration is insufficient, and the number of iterations is below a threshold 793, so as to reduce the total computation time of the single block sparse pre-solve step 790. In some embodiments of the present invention, the progress obtained from one iteration to the next iteration of the single block sparse pre-solve step 790 may be measured based on the tightening of one or more variable bounds from one iteration to the next iteration.

[0219] Fig. 8A It shows that all discrete values ​​of the MICP solution are directly predicted based on regression given the current problem parameter value θ425 according to some embodiments of the present invention. Block diagram of the machine learning based prediction method 800 of 430.

[0220] In some embodiments of the present invention, the supervised learning based predictor 800 is trained 415 based on a regression problem, i.e., the predictor is trained to directly predict the optimal value of the discrete variable at each time step i=0, 1, ..., N in the prediction time horizon of the MIOCP. In some embodiments of the invention, the predictor is a recurrent neural network (RNN), which, for example, is composed of one or more long short-term memory (LSTM) networks, gated recurrent units (GRUs) and / or one or more feed-forward neural networks, and whose input dimension is equal to the number n of problem parameter values ​​425 θ , the output dimension is equal to the total number of discrete variables N in the prediction time horizon of MIOCP δ =N×n δ .

[0221] For example, in some embodiments of the invention, the training loss for the regression problem can be a binary cross entropy loss of the logit activations of the output layer of a parametric function approximation architecture (e.g., a neural network). The resulting optimization problem, i.e., computing the network weights that minimize the training loss function, can be solved using stochastic gradient descent, Adam, AdaGrad, RMSProp, or any other gradient-based or gradient-free optimization algorithm. In some embodiments of the invention, the regression-based predictor 800 is used to fix only one or more of the discrete variables that give the learning-based predictor a relatively high confidence, such that the reduced MIOCP needs to be solved to compute the remaining free discrete variables in order to compute a feasible but potentially suboptimal mixed integer solution to the original high-dimensional MIOCP problem.

[0222] Figure 8B A block diagram of a machine learning based prediction using time decomposition methods 801-804 according to some embodiments is shown, which directly predicts discrete values ​​of the prediction time horizon of the MICP solution based on regression given the current problem parameter value θ425, that is, 430. In some embodiments of the invention, each machine learning based prediction step provides a prediction of a discrete value at one time step within a prediction time horizon. In other embodiments of the invention, each machine learning based prediction step provides a prediction of a discrete value at a plurality of consecutive time steps within a prediction time horizon.

[0223] Specifically, in some embodiments of the invention, machine learning-based prediction 430 includes multiple evaluations, sequential or in parallel, of a machine learning network for solving a regression problem in order to predict discrete variables. The value 801 at time step 0 in order to predict a discrete variable The value 802 at time step 1, and the process continues 803 until the discrete variable is predicted value 804 at time step N. In some embodiments of the invention, the predictor for each of the time steps is an RNN, which is composed of, for example, one or more LSTM networks, GRUs and / or one or more feed-forward neural networks, and whose input dimension is equal to the number n of problem parameter values ​​425 θ , the output dimension is equal to the total number of discrete variables n in each time step of the prediction time horizon of MIOCP δ .

[0224] Some embodiments of the present invention are based on the recognition that the hidden states of the RNN architecture can be used as memory in machine learning-based prediction to encode the discrete solution of the MIOCP given the problem parameter values ​​θ time structure.

[0225] Some embodiments of the present invention are based on the recognition that a subset of discrete variables can be fixed by a pre-solve based correction method, which starts with the predicted values ​​of the set of discrete variables in the MICP that allow the learning-based predictor to have relatively high confidence, and (unless infeasibility is detected by iterative pre-solve based correction) solves an approximation to the MICP with a reduced number of discrete optimization variables under constraints to obtain feasible motion trajectories with a fixed set of values ​​for the updated subset of discrete variables.

[0226] In some embodiments of the present invention, a branch and bound (B&B) optimization method may be used in conjunction with one or more customized block sparse presolve operations that exploit the block structured sparsity of a mixed integer optimal control problem (MIOCP) for performing tasks defined by problem parameter values ​​to solve an approximation of a MICP with a reduced number of discrete optimization variables. The block sparse presolve operations may be used to tighten and / or accelerate the solution of a convex relaxation in a root node of a B&B optimization method and / or one or more nodes of a B&B search tree by reducing the number of optimization variables and reducing the number of constraints, and by tightening bounds on the optimization variables and constraints.

[0227] In some embodiments of the present invention, a first subset of discrete variables in the MICP are fixed to predicted values ​​output by a trained parameter function, a second subset of discrete variables in the MICP are fixed to values ​​uniquely defined by the fixed values ​​of the first subset of discrete variables and constraints through a pre-solve-based correction, and a third subset of discrete variables in the MICP are selected as free optimization variables, so that an approximation of the MICP with a reduced number of discrete optimization variables is solved under constraints to produce feasible motion trajectories, for example using a B&B optimization method in combination with one or more customized block sparse pre-solve operations that exploit the block-structured sparsity of the MIOCP problem.

[0228] Figure 8C A block diagram of a prediction method 820 based on machine learning according to some embodiments of the present invention is shown. The prediction method 820 predicts one or more discrete values ​​(i.e., ), which can be used to predict the discrete variables solved by MICP The value is 430.

[0229] In some embodiments of the invention, the supervised learning based predictor 820 is trained 415 based on a multi-class classification problem, i.e., the predictor is trained as a set of discrete optimizers (i.e., ) selects one or more optimal discrete values. In some embodiments of the present invention, the predictor is an RNN, which is composed of, for example, one or more LSTMs, GRUs and / or one or more feed-forward neural networks, and whose input dimension is equal to the number n of problem parameter values ​​425 θ , the output dimension is equal to all discrete variables (δ0,δ1,…,δ N ) is the number of unique discrete values ​​in the training data set |Δ|.

[0230] For example, in some embodiments of the invention, the training loss for a multi-class classification problem can be a cross entropy loss applied to the output layer of a parametric function approximation architecture (e.g., a neural network). The resulting optimization problem, computing the network weights that minimize the training loss function, can be solved using stochastic gradient descent, Adam, AdaGrad, RMSProp, or any other gradient-based or gradient-free optimization algorithm. In some embodiments of the invention, a classifier 820 is used to select one or more discrete solution candidates 460 corresponding to the highest confidence in feasibility and optimality (i.e., the highest value for each class in the classes from the output layer of the network), and an iterative pre-solve-based correction method 462 can be used to potentially correct one or more of these discrete solution candidates in order to compute a feasible but potentially suboptimal mixed integer solution 255 for the high-dimensional MIOCP problem.

[0231] Fig.8D A block diagram of a machine learning based prediction using time decomposition methods 821-824 is shown according to some embodiments, which predicts one or more discrete values ​​(i.e., ) probability 430. In some embodiments of the invention, each machine learning based prediction step provides a prediction of a discrete value at one time step within the prediction time horizon. In other embodiments of the invention, each machine learning based prediction step provides a prediction of a discrete value at multiple consecutive time steps within the prediction time horizon.

[0232] Some embodiments of the present invention are based on the recognition that a trained parameter function approximation architecture, for example, a machine learning-based prediction model, can provide probabilities or confidence levels for one or more discrete values ​​in a discrete optimizer set at one or more time steps of a prediction time horizon, so that a prediction value 430 of a discrete variable can be calculated to maximize the probability or confidence level. In some embodiments of the present invention, the prediction value 430 is selected based on sampling according to a probability distribution, and the probability of each discrete value in the discrete optimizer set is equal to the probability or confidence level provided by the trained machine learning-based prediction model. Some embodiments of the present invention are based on the recognition that multiple discrete candidate solutions can be calculated by performing multiple samplings based on the probability distribution provided by a trained multi-class classification network architecture for one or more discrete values ​​in a discrete optimizer set at one or more time steps of a prediction time horizon.

[0233] Specifically, in some embodiments of the invention, machine learning-based prediction 430 includes multiple evaluations, either sequentially or in parallel, of a machine learning network for solving a multi-class classification problem to predict the discrete value of the discrete optimizer set at time step 0. 821, in order to predict the discrete value of the discrete optimizer set at time step 1 822, and the process continues 823 until the discrete optimizer set predicts the discrete value of N at time step N. 824. In some embodiments of the invention, the predictor for each of the time steps is an RNN, which is composed of, for example, one or more LSTM networks, GRUs, and / or one or more feed-forward neural networks, with an input dimension equal to the number n of problem parameter values ​​425 θ , the output dimension is equal to the discrete variable δ at a time step i∈{0,1,…,N} in the prediction time horizon of MIOCP i The number of unique discrete values ​​in the training data set |Δ|.

[0234] Some embodiments of the present invention are based on the recognition that, due to the reduction in the number of target class labels in supervised learning, the output dimension of a machine learning architecture for multi-class classification can be based on Fig.8D This can result in improved sampling efficiency and / or improved performance of machine learning-based predictors. For example, at each time step i = 0, 1, ..., N given n δ In the case of discrete variables, the maximum number of target class label values ​​is If the classifier provides predictions of discrete values ​​at one time step within the prediction time horizon, the number of target class label values ​​can be reduced to a maximum of target class label values. In addition, some embodiments of the present invention are based on the recognition that the number of unique discrete values ​​|Δ| can be much smaller than the theoretical maximum number of target class label values This is due to the sampling of problem parameter values ​​that are representative of the optimal control problem of interest in practice, and due to constraints on the feasibility of the MIOCP problem.

[0235] Some embodiments of the present invention are based on the recognition that the hidden states of the RNN architecture can be used as memory in machine learning-based prediction to encode the discrete solution of the MIOCP given the problem parameter values ​​θ time structure.

[0236] Fig. 9A An example of a machine learning algorithm for implementing a predictive model according to some embodiments of the present disclosure is shown. In particular, Fig. 9AVarious prediction models 901 are summarized that can be trained to generate predictions of solution values ​​for discrete optimization variables of a MICP problem. The prediction models can be deterministic 911 or stochastic 913 and can rely on neural architectures 921 / 931 or kernel representations 923 / 933. Examples of deterministic prediction models include multilayer perceptrons, recurrent and / or convolutional neural networks 921, kernel regression, and support vector machines 923, among others. Examples of stochastic prediction models include Bayesian neural networks, neural processes 931, Gaussian processes, Kriging interpolation 933, among others. Fig. 9B One embodiment described in uses a deep neural network as a prediction model.

[0237] Fig. 9B A schematic diagram of the layers and signals involved in a deep neural network 941 according to some embodiments of the present disclosure is shown. From the data collected for training, an input signal 951 is provided to the deep neural network 941 through an input layer 953, which is typically a feature extraction layer and can be linear or include convolutional filters to automatically extract useful features from the input signal 951. For a linear input layer, the transformation is linear and the feature

[0238]

[0239] Where W0 and w0 are the weights and biases of the input layer, and X is the input to the deep neural network 941. Therefore, the features pass through multiple hidden layers 955 (usually greater than 2 for deep neural networks), and they are transformed via activation functions α(·) such as rectified linear units (ReLU), sigmoid functions, or modifications and combinations thereof. The operations occurring in the hidden layers can be represented using a combination of functions, where the output of the final hidden layer is given by

[0240]

[0241] The output layer 957 is used here for regression or multi-class classification, and, for example, the output layer can be linear, so the predicted output 959 becomes

[0242]

[0243] And training the deep neural network 941 involves solving an optimization problem to obtain good weight and bias values ​​in the input layer 953, the output layer 957, and each hidden layer 955.

[0244] While common training methods for prediction models aim to minimize the training error of the prediction model equally in each direction, some embodiments of the present invention are based on the recognition that the training error should be minimized primarily in directions that cannot be fixed by the correction method based on iterative pre-solving. Specifically, some embodiments of the present invention are based on the recognition that the predictions of certain discrete optimization variables in the MICP problem by the trained machine learning network 430 can be simply corrected by the iterative pre-solving method 435. Alternatively, other predictions of certain discrete optimization variables in the MICP problem by the trained machine learning network 430 cannot be corrected by the iterative pre-solving method 435, and therefore, the training error for the latter discrete optimization variable should be minimized primarily.

[0245] Fig. 9C A schematic diagram of an example of a machine learning based prediction model using a time decomposition method for machine learning based prediction using a deep neural network, hidden state and a single or multi-layer recurrent neural network (RNN) for discrete MIOCP solution according to some embodiments of the present invention is shown. The current problem parameter value θ is provided to one or more deep neural networks 970-971-972 to extract features that can be used by one or more single or multi-layer RNN architectures 960-961-962. For example, given a feature input from a neural network 970 and a given initial hidden state value h0 965, a single or multi-layer RNN 960 calculates an updated hidden state value h1 966, which can be used by a single or multi-layer RNN 961 together with a feature input from a neural network 971 to calculate an updated hidden state value h2 967, which can again be used by a single or multi-layer RNN 962 together with a feature input from a neural network 972 to calculate updated hidden state values, which are used by one or more additional single or multi-layer RNN architectures 963. Each of the updated hidden state values ​​966-967 may be provided to a deep neural network 975-976 to compute a predicted discrete value for each time step i=0,1,...,N over the predicted time horizon of the MIOCP 980-981.

[0246] In some embodiments of the invention, each of the RNN architectures 960-961-962 is a single or multi-layer LSTM or GRU based network. Some embodiments of the invention are based on the recognition that each deep neural network used to extract features from parameters 970-971-972 can be the same neural network with the same weights and offset values. Similarly, some embodiments of the invention are based on the recognition that each RNN 960-961-962 for each time step within the prediction time horizon can be the same RNN model with the same weights and offset values. Finally, some embodiments of the invention are based on the recognition that each deep neural network used to calculate from the output of the hidden state values ​​975-976 can be the same neural network with the same weights and offset values.

[0247] Fig. 10A FIG. 4 is a schematic diagram of offline data generation 400 using centralized sampling according to some embodiments of the present disclosure. In particular, Fig. 10A The principle of concentrated sampling for collecting sampled problem parameter values ​​is demonstrated. According to some embodiments, the concentrated sampler is based on the recognition that it is necessary to generate more data samples of state values ​​that are located close to one or more obstacles 1001 in the environment 1011 of the controlled system. This makes it possible to explicitly minimize the training loss in the area near the obstacle 1001 where the optimal control solution may change rapidly and may be discontinuous with respect to the initial state of the system. To this end, a multi-step sampling strategy is developed, in which a predefined number of initial samples 1021 are generated using a standard sampling method, and data tuples are extracted from them. Some examples of sampling methods include uniform distribution, sparse grids, or low-discrepancy sequences defined on the state space of interest. Among these initial samples, some samples are infeasible because they overlap with obstacles: these infeasible samples 1023 are deleted from the data set because they do not cause training loss. After this first sampling step, another set of samples concentrated near the obstacles is generated: these samples 1031 are extracted from the concentrated sampler. Reference Fig. 10B Some embodiments of the centralized sampler 1051 are further described.

[0248] Fig. 10B Various methods of concentrated sampling according to embodiments of the present disclosure are shown. One embodiment employs distance-based concentrated sampling, where concentrated samples are generated by distance-based rejection sampling 1061. This involves generating samples on the allowed state space Determines a predefined neighborhood based on a predefined distance metric is within the neighborhood of any obstacle, and rejects the sample if it is outside the specified neighborhood. By repeatedly performing rejection sampling, the focused sampler produces a The sampled problem parameter values ​​include a regular sample set and a shifted sample set, wherein the shifted sample set is shifted toward the constraint.

[0249] Another embodiment of the concentrated sampler involves density-based importance sampling 1063 from a distribution designed with obstacle locations in mind. According to this embodiment, a distribution over the parameter space of interest is constructed such that the probability of selecting a data point increases, for example, as the concentrated sampler 1051 approaches an obstacle. For example, such a distribution can be efficiently generated by a kernel density estimation method, and extracting samples from such a skewed distribution will automatically concentrate the samples at locations that are in close proximity to obstacles.

[0250] Another embodiment of concentrated sampling involves active learning 1065, where an initial set of training data is used to build a learner, which then selects the next batch of samples that contain the most useful information (e.g., based on an entropy metric). Through repeated learning and batch selection, the collected data will include concentrated parameter samples around the most informative regions in parameter space.

[0251] Fig.11A A schematic diagram showing an example of a binary control variable search tree representing a nested tree of search regions for integer feasible solutions to a MICP subproblem according to some embodiments of the present invention. Fig.11A A schematic representation of a branch and bound method is shown by showing a binary search tree 1100 at a particular iteration of a mixed integer optimization algorithm for solving the MIOCP for a set of sampled values ​​of the problem parameters, which can be used in an offline data generation step for training a machine learning based predictor to implement a MIMPC controller in some embodiments of the present invention. The main idea of ​​the branch and bound (B&B) method is to sequentially create partitions of the original MICP problem and then attempt to solve these partitions, where each partition corresponds to a specific region of the discrete control variable search space. In some embodiments, the branch and bound method selects a partition or node and selects discrete control variables to branch the partition into smaller partitions or search regions, thereby producing a nested tree of partitions or search regions.

[0252] For example, partition P11101 represents a discrete search region, which can be split or branched into two smaller partitions or regions P21102 and P31103, i.e., a first region and a second region nested in a common region. The first region and the second region are disjoint, i.e., the intersection of the two regions is empty P2∩P3=φ1107, but they together constitute the original partition or region P1, i.e., the branched union P2∪P3=P11106 holds. The branch-and-bound method then solves the integer relaxed convex problem for both the first and second partitions or regions of the search space, and obtains two solutions (local optimal solutions), which can be compared with each other or with the upper bound value of the currently known optimal objective value. If the performance metric of the first and / or second partitions or regions is lower than the upper bound of the currently known optimal objective value of MICP, they can be pruned. If the first region, the second region, or both regions lead to a discrete feasible solution to the MICP, the upper bound value can be updated. The branch-and-bound method then continues by selecting the remaining regions in the nested tree of the current region for further partitioning.

[0253] Although solving each partition may still be challenging, it is quite efficient to obtain a local lower bound of the optimal objective value by solving a local relaxation of the mixed integer programming or by using duality. If the solution method of the MICP happens to obtain an integer feasible solution when solving the local relaxation, the branch and bound method can use it to obtain a global upper bound of the mixed integer control solution of the MICP, thereby finding a feasible and (local) optimal solution of the MIOCP for a set of sampled values ​​of the problem parameters. This can help avoid solving or branching certain partitions that have been created, that is, these partitions or nodes may be pruned. The general algorithmic concept of this partition can be represented as a binary search tree 1100, including a root node, such as P11101 at the top of the tree, and leaf nodes at the bottom of the tree, such as P41104 and P51105. In addition, nodes P21102 and P31103 are generally referred to as direct child nodes of node P11101, and node P11101 is referred to as the parent node of nodes P21102 and P31103. Similarly, nodes P41104 and P51105 are child nodes of their parent node P21102.

[0254] Fig. 11B A block diagram of a branch and bound optimization algorithm according to some embodiments of the present invention is shown, which searches for an integer feasible optimal solution to the MICP for a sample set of problem parameter values ​​based on a nested tree of search regions and corresponding lower / upper bound values. In some embodiments, Fig. 11BThe block diagram of the branch-and-bound mixed integer optimization algorithm shown can be used in the offline data generation step for training a machine learning-based predictor to implement a MIMPC controller. The branch-and-bound method is based on mixed integer convex programming (MICP) data 1165 consisting of matrices and vectors, and initializes 1110 the branch search tree information of the MICP at the current control time step. The initialization can additionally use the MICP solution information and branch search tree information from the previous iteration 1160 to generate a warm start initialization 1110 for the current control time step. The main goal of the optimization algorithm is to construct lower and upper bounds of the target value of the MICP solution. In step 1111, if the difference between the lower bound value and the upper bound value is less than a certain tolerance value, the mixed integer optimal control solution 1155 is found.

[0255] As long as the difference between the lower bound value and the upper bound value is greater than a certain tolerance value at step 1111, and the maximum execution time has not been reached by the optimization algorithm, the branch and bound method continues to iteratively search for the mixed integer optimal control solution of the MICP for the sampled set of problem parameter values ​​1155. Each iteration of the branch and bound method begins by selecting the next node in the tree (corresponding to the next region or partition of the integer variable search space), where possible variable fixation is performed based on a pre-solve branching technique 1115. After node selection, the corresponding integer relaxed convex problem is solved, where possible variable fixation is performed based on a post-solve branching technique 1120.

[0256] If the integer relaxed convex problem has a feasible solution, then the resulting relaxed control solution provides a lower bound on the objective value for that particular region or partition of the integer variable search space. In step 1121, if the objective is determined to be greater than a currently known upper bound on the objective value of the optimal mixed integer control solution for the MICP, the selected node is pruned or deleted from the branching tree 1140. However, in step 1121, if the objective is determined to be below the currently known upper bound, and the relaxed control solution is integer feasible 1125, then in step 1130, the currently known upper bound and the corresponding mixed integer control solution estimate are updated.

[0257] If the integer relaxed convex problem has a feasible solution and the objective is below the currently known upper bound 1121, but the relaxed controlled solution is not yet integer feasible 1125, then the global lower bound of the objective may be updated 1135 to be the minimum of the objective values ​​of the remaining leaf nodes in the branching tree, and the selected node is pruned from the tree 1140. In addition, starting from the current node, discrete variables with fractional values ​​are selected for branching according to a specific branching strategy 1145, thereby creating and appending the generated auxiliary MICP subproblem (which corresponds to a subregion or partition of the discrete search space of the original MICP) as a child of the node in the branching tree 1150.

[0258] An important step in the branch and bound method is how to create the partition, i.e., which node to choose 1115 and which discrete variable to choose for branching 1145. Some embodiments are based on branching on one of the binary control variables with fractional values ​​in the integer relaxed convex problem solution. For example, if a particular binary control variable u i,k ∈{0,1} as part of the integer relaxation convex problem solution with fractional values, some implementations add equality constraints u to a subproblem respectively i,k = 0 and add an equality constraint u to another subproblem i,k = 1 to create two partitions of the MICP subproblem. Some embodiments have used reliability-based branching strategies for variable selection 1145, which aims to predict future branching behavior based on information from previous branching decisions.

[0259] Some embodiments are based on a branch-and-bound approach using a depth-first node selection strategy, which can be implemented using a last-in, first-out (LIFO) buffer. The next node to be solved is selected as one of the children of the current node, and the process is repeated until the node is pruned (i.e., the node is infeasible, optimal, or preceded by the currently known upper bound value), and then the backtracking process is performed. In contrast, some embodiments are based on a branch-and-bound approach that uses a best-first strategy to select the node with the current lowest local lower bound. Some embodiments employ a combination of depth-first and best-first node selection methods, in which a depth-first node selection strategy is used until an integer feasible control solution is found, and then a best-first node selection strategy is used in subsequent iterations of the branch-and-bound based optimization algorithm. The motivation for the latter implementation is to find an integer feasible control solution (depth-first) early in the beginning of the branch-and-bound process so that early pruning can be performed, and then a better feasible solution can be searched more greedily (best-first).

[0260] The branch and bound method continues to iterate until one or more of the following conditions have been met:

[0261] The maximum execution time of the processor is reached.

[0262] All nodes in the branching search tree have been pruned so that no new nodes can be chosen to solve the convex relaxation or branch.

[0263] The optimality difference between the global lower bound and upper bound of the objective value solved by the MICP subproblem is less than the tolerance.

[0264] Fig. 12AA schematic diagram of a vehicle 1201 including a predictive controller 1202 employing the principles of some embodiments is shown. As used herein, the vehicle 1201 can be any type of wheeled vehicle, such as a passenger car, a bus, or a rover. In addition, the vehicle 1201 can be an autonomous or semi-autonomous vehicle. For example, some embodiments control the movement of the vehicle 1201. Examples of movement include lateral movement of the vehicle controlled by a steering system 1203 of the vehicle 1201. In one embodiment, the steering system 1203 is controlled by the controller 1202. Additionally or alternatively, the steering system 1203 can be controlled by a driver of the vehicle 1201.

[0265] The vehicle may also include an engine 1206, which may be controlled by the controller 1202 or by other components of the vehicle 1201. The vehicle may also include one or more sensors 1204 to sense the surrounding environment. Examples of sensors 1204 include distance rangefinders, radars, lidars, and cameras. The vehicle 1201 may also include one or more sensors 1205 to sense its current amount of motion and internal state. Examples of sensors 1205 include global positioning systems (GPS), accelerometers, inertial measurement units, gyroscopes, shaft rotation sensors, torque sensors, deflection sensors, pressure sensors, and flow sensors. The sensors provide information to the controller 1202. The vehicle may be equipped with a transceiver 1206 that enables the controller 1202 to obtain the ability to communicate through a wired or wireless communication channel.

[0266] Fig. 12BA schematic diagram of the interaction between a controller 1202 (e.g., a mixed integer model predictive controller (MIMPC)) and other controllers 1220 of a vehicle 1201 according to some embodiments is shown. For example, in some embodiments, the controllers 1220 of the vehicle 1201 are a steering 1225 and a brake / throttle controller 1230 that control the rotation and acceleration of the vehicle 1220, respectively. In this case, the predictive controller 1202 outputs control inputs to the controllers 1225 and 1230 to control the state of the vehicle 1201. The controller 1220 may also include a high-level controller, such as a lane keeping assist controller 1235, which further processes the control input of the predictive controller 1202. In both cases, the controller 1220 uses the output of the predictive controller 1202 to control at least one actuator of the vehicle 1201, such as the steering wheel and / or brakes of the vehicle 1201, so as to control the movement of the vehicle 1201. In addition, the predictive controller 1202 determines the input of the vehicle 1201 based on the mixed integer optimal control solution, wherein the input of the vehicle 1201 includes one or a combination of the acceleration of the vehicle 1201, the engine torque, braking torque and steering angle of the vehicle 1201, and discrete optimization variables that model one or a combination of discrete control decisions, system dynamics switching, gear shifting and obstacle avoidance constraints.

[0267] Fig. 12C A schematic diagram of a path and / or motion planning method for a controlled vehicle using the principles of some embodiments is shown. Fig. 12C A schematic diagram of an autonomous or semi-autonomous controlled vehicle 1250 is shown for which a dynamically feasible and often optimal trajectory 1255 can be calculated using embodiments of the present disclosure. The generated trajectory aims to keep the vehicle within a specific road boundary 1252 and aims to avoid other controlled and / or uncontrolled vehicles, i.e., these vehicles are obstacles 1251 to the specific controlled vehicle 1250. In some embodiments, each of the obstacles 1251 can be represented by one or more inequality constraints in a time or space formulation of a constrained mixed integer convex programming problem, which includes one or more additional discrete variables for each obstacle. For example, based on an embodiment configured to implement a mixed integer model predictive controller, the autonomous or semi-autonomous controlled vehicle 1250 can make discrete decisions in real time, such as passing another vehicle on the left or right, or staying behind another vehicle in the current lane of the road 1252, while also making continuous decisions in real time, such as controlling the speed, acceleration, or steering input of the movement of the vehicle 1250.

[0268] Fig.12D Exemplary traffic scenarios of a single vehicle or multiple vehicle decision making process modules according to some embodiments are shown. Fig.12DA scenario is depicted in which one or more vehicles are under control, referred to as ego vehicles 1271, where traffic consists of other vehicles shown similarly to 1272, lanes 1273, such as labeled L6, stop lines 1274, such as labeled S1, and intersections 1275, such as labeled I3. For a vehicle at location 1261, final destination 1262, the routing module provides a sequence of roads indicated by arrows 1263 and a sequence of turns indicated by arrows 1264. However, it should be noted that the sequence of roads 1263 and the sequence of turns 1264 do not in themselves specify the trajectory or path of the vehicle. Many discrete decisions need to be made, such as which lane the vehicle will travel in, whether the vehicle should change lanes or remain in the current lane, whether the vehicle should begin to slow down to stop at the stop line, whether the vehicle is allowed to cross the intersection, etc. In addition, many continuous decisions need to be made, such as the time sequence of positions and orientations that the vehicle should achieve during its journey from its initial point to its destination. These decisions are highly dependent on the current traffic conditions at the time the vehicle arrives at the corresponding location, which is generally unknown to the routing module due to the uncertainty of traffic movement and the uncertainty of the time the vehicle will arrive at the location. In some embodiments of the present disclosure, a motion plan can be calculated for one or more controlled ego vehicles 1271 (which may communicate to allow coordination between vehicles (V2V) and / or between smart infrastructure systems and vehicles (V2X)) by solving one or more connected mixed integer convex programming problems.

[0269] Fig.13A and Fig. 13B is a schematic diagram of a mixed integer model predictive control problem formulation for a spacecraft using the principles of some embodiments of the present disclosure. More specifically, Fig.13A and Fig. 13B A spacecraft 1302 is shown equipped with a plurality of actuators such as thrusters 1350 and momentum exchange devices 1351. Examples of types of momentum exchange devices include reaction wheels (RW) and gyroscopes. Spacecraft 1302 is a vehicle, container, or machine designed for flight in outer space, the operation of which is responsive to commands sent to actuators to change quantities such as the position of spacecraft 1302, its velocity, its attitude, or orientation. When commands are received, the actuators apply forces to spacecraft 1302 to increase or decrease the velocity of spacecraft 1302, thereby translating spacecraft 1302 in its position, and, when commands are received, the actuators also apply torques to spacecraft 1302 to rotate spacecraft 1302, thereby changing its attitude or orientation. As used herein, the operation of spacecraft 1302 is determined by the operation of the actuators that determine the motion of spacecraft 1302 that changes these quantities.

[0270] The spacecraft 1302 flies around, between, or near one or more gravitational bodies (such as the Earth 1361, the Moon, and / or other celestial bodies planets, stars, asteroids, comets) along an open or closed orbital path 1360. Typically, a desired or target position 1365 is given along the orbital path. A reference frame 1370 is attached to the desired position, where the origin of the reference frame, i.e., all zero coordinates in the reference frame at all times, are the coordinates of the desired position.

[0271] The spacecraft 1302 is subject to various perturbing forces 1314. These perturbing forces may include forces that were not accounted for when determining the orbital path of the spacecraft 1302. These perturbing forces act on the spacecraft 1302 to deviate from the desired position on the orbital path. These forces may include, but are not limited to, gravity, radiation pressure, atmospheric drag, a non-spherical center body, and leaking propellant. As a result, the spacecraft 1302 may be at a distance 1367 away from the target location.

[0272] Due to the presence of disturbance forces, it is not always possible to keep the spacecraft 1302 at a desired position along its orbit. Therefore, it is instead desirable to keep the spacecraft 1302 within a window 1366 of specified dimensions around the desired position 1364. To this end, the spacecraft 1302 is controlled to move along an arbitrary path 1380 included within the desired target window. In this example, the window 1366 has a rectangular shape, but the shape of the window may vary for different embodiments.

[0273] The spacecraft 1302 is also often required to maintain a desired orientation. For example, the reference frame 1374 to which the spacecraft is fixed needs to be aligned with a desired reference frame, such as an inertial reference frame 1371 that is fixed relative to distant stars 1372, or a reference frame 1373 that is always oriented in a manner pointing toward the earth. However, depending on the shape of the spacecraft 1302, different perturbation forces 1314 may act unevenly on the spacecraft 1302, thereby generating perturbation torques that cause the spacecraft 1302 to rotate away from its desired orientation. In order to compensate for the perturbation torque, a momentum exchange device 1351 such as a reaction wheel is used to absorb the perturbation torque so that the spacecraft maintains its desired orientation.

[0274] In order to prevent the momentum exchange device from being saturated and losing the ability to compensate for the disturbance torque, its stored momentum must be unloaded, for example by reducing the spin rate of the reaction wheel. Unloading the momentum exchange device will bring undesirable torque to the spacecraft. This undesirable torque is also compensated by the thrusters.

[0275] In some embodiments of the present invention, one or more equations are used to describe the linear or piecewise linear dynamics and / or convex constraints of the controlled system, and the MIMPC controller determines inputs to the spacecraft 1302 based on a mixed integer optimal control solution, wherein the inputs to the spacecraft 1302 drive one or a combination of thrusters and momentum exchange devices, and discrete optimization variables are used to model one or a combination of discrete control decisions, switching of system dynamics, integer values ​​of thruster commands, and obstacle avoidance constraints.

[0276] In some embodiments of the invention, spacecraft 1302 may be modeled as a hybrid system that includes both continuous and discrete operating elements, and a mixed integer model predictive controller (MIMPC) is used to compute commands to actuators. For example, in some embodiments, commands to thrusters 1350 may take only a set of discrete values, thus producing a set of binary or integer control input variables for each stage within the mixed integer control horizon.

[0277] In some embodiments of the invention, the predictive controller is designed to keep the spacecraft 1302 outside a specific region 1385 of specified size, close to the desired position along the track. The latter region can be fixed in time or time-varying, and is generally referred to as an exclusion zone 1385, for which corresponding logical inequality constraints can be modeled using an additional set of binary or integer control input variables for each stage within the mixed integer control horizon. In this example, the exclusion zone 1385 has a rectangular shape and it is located in the corners of the desired window 1366, but the shape of the exclusion zone and the location within the desired target window may vary for different embodiments.

[0278] Fig.14A A schematic diagram of a vapor compression system 1400 controlled by a controller 1460 is shown according to some embodiments. The controller 1460 includes a predictive controller, such as a controller implementing a mixed integer model predictive control (MIMPC). The components of the vapor compression system (VCS) 1400 may include an indoor heat exchanger 1420 located in an indoor space or area 1450, an outdoor unit heat exchanger 1430 located in the ambient environment, a compressor 1410, and an expansion valve 1440. A heat load 1415 acts on the indoor space or area 1450.

[0279] In addition, the VCS 1400 may include a flow reversing valve 1455 for directing high-pressure refrigerant flowing out of the compressor to the outdoor unit heat exchanger or the indoor unit heat exchanger, and directing low-pressure refrigerant returning from the indoor unit heat exchanger or the outdoor unit heat exchanger to the inlet of the compressor. In the case where the high-pressure refrigerant is directed to the outdoor unit heat exchanger, the outdoor unit heat exchanger acts as a condenser and the indoor unit acts as an evaporator, wherein the system discharges heat from the zone to the surrounding environment, which is operationally referred to as a "cooling mode." Conversely, in the case where the high-pressure refrigerant is directed to the indoor unit heat exchanger, the indoor unit heat exchanger acts as a condenser and the outdoor unit heat exchanger acts as an evaporator, extracting heat from the surrounding environment and pumping such heat into the zone, which is operationally referred to as a "heating mode."

[0280] Fig. 14B An example of a configuration of signals, sensors, and controllers used in VCS1400 according to some embodiments is shown. The controller 1460 reads information from a sensor 1470 configured to measure various temperatures, pressures, flows, or other information about the operation of the system, including measurable disturbances, such as ambient air temperature. The controller 1460 may be provided with a set value 1466, which represents the expected value of the measured signal of the process, such as the expected zone temperature. The set value information may come from a thermostat, a wireless remote control, or an internal memory or storage medium. The controller then calculates the control input so that some measured outputs are driven to their set values. These control inputs may include indoor unit fan speed 1480, outdoor unit fan speed 1481, compressor speed 1482, expansion valve position 1483, and flow reversing valve position 1484. In this way, the controller controls the operation of the vapor compression system so that the set value is reached in the presence of a disturbance 1468 (e.g., a heat load acting on the system).

[0281] In some embodiments, the VCS 1400 may be modeled as a hybrid system that includes both continuous and discrete operating elements, and uses a predictive controller such as a mixed integer model predictive controller (MIMPC) to calculate commands to the actuators. For example, in some embodiments, the commands to the valves and / or fans can only take a discrete set of values, thus resulting in a set of binary or integer control input variables for each stage within the mixed integer control horizon.

[0282] In some embodiments, the predictive controller determines inputs to the steam compression system based on a mixed integer optimal control solution, wherein the inputs to the steam compression system include one or a combination of an indoor unit fan speed, an outdoor unit fan speed, a compressor speed, an expansion valve position, and a flow reversing valve position, and discrete optimization variables are used to model discrete control decisions, switching of system dynamics, and one or a combination of integer values ​​of commands sent to valves and / or fans.

[0283] In some embodiments, the dynamic behavior of the VCS 1400 can change rapidly or even switch at certain time instances, depending on the current state of the system and the current control input values. The resulting hybrid VCS 1400 with switching dynamics can be modeled using a set of additional binary or integer control input variables for each stage within the mixed integer control horizon.

[0284] Fig.15 A method 1500 for controlling a system according to an example implementation is shown. In step 1501, the method includes receiving a feedback signal, the feedback signal including a measurement of a problem parameter, for example, including a state of the system and / or a state of the system environment. In step 1503, the method includes solving a mixed integer optimal control problem (MIOCP) using a prediction based on cyclic machine learning and a correction based on iterative pre-solve, the method searching for feasible and (local) optimal solutions to a mixed integer convex programming (MICP) problem. In step 1505, the system is controlled according to the control signal to change the state of the system.

[0285] The above-mentioned embodiments of the present invention can be implemented in any of a variety of ways. For example, hardware, software or a combination thereof can be used to implement the embodiments. When implemented in software, the software code can be executed on any suitable processor or processor set (whether provided in a single computer or distributed between multiple computers). Such a processor can be implemented as an integrated circuit with one or more processors in an integrated circuit assembly. However, the processor can be implemented using a circuit of any appropriate format.

[0286] In addition, the various methods or processes outlined herein may be encoded as software that can be executed on one or more processors using any of a variety of operating systems or platforms. In addition, such software can be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may also be compiled into executable machine language code or intermediate code that is executed on a framework or virtual machine. Typically, in various embodiments, the functionality of the program modules may be combined or distributed as desired.

[0287] In addition, embodiments of the present invention can be implemented as methods for which examples have been provided. The actions performed as part of the method can be ordered in any suitable manner. Therefore, embodiments can be constructed in which the actions are performed in a sequence different from that shown, which may include performing some actions simultaneously, although shown as sequential actions in illustrative embodiments.

[0288] Although the present invention has been described by examples of preferred embodiments, it should be understood that various other adjustments and modifications may be made within the spirit and scope of the present invention. Therefore, the purpose of the appended claims is to cover all such changes and modifications that fall within the true spirit and scope of the present invention.

Claims

1. A controller for controlling the movement of a device to perform a task of changing the state of the device under constraints, the controller comprising: processor; and a memory having instructions stored therein, which, when executed by the processor, cause the controller to: collecting parameters of the task, the parameters including the status of the device; Inputting parameters of the task to evaluate a parameter function trained to output predicted values ​​for a set of discrete variables in a mixed integer convex programming (MICP) problem for performing the task defined by the parameters; fixing a first subset of discrete variables in the MICP to the predicted values ​​output by the trained parameter function; performing a pre-solve-based correction configured to update at least some of the predicted values ​​of a remaining subset of discrete variables in the MICP to values ​​uniquely defined by the fixed values ​​of the first subset of discrete variables and the constraints; Converting the MICP into a convex programming (CP) problem based on the fixed values ​​of the first subset of discrete variables and the updated values ​​of the remaining subset of discrete variables; solving the CP problem under the constraints to generate a feasible motion trajectory for performing the task defined by the parameters; and The device is commanded to change state according to the motion trajectory.

2. The controller according to claim 1, wherein: The parameter function is trained to output predicted values ​​of a set of discrete variables at a single time step or at multiple consecutive time steps within a prediction time horizon of the MICP.

3. The controller according to claim 1, wherein: The parameter function is evaluated one or more times to generate multiple predicted values ​​for each discrete variable in the set of discrete variables in the MICP, thereby generating multiple candidate discrete solutions; wherein the controller performs a pre-solution-based correction on each candidate discrete solution to convert the MICP into multiple CP problems and determine multiple motion trajectories, and wherein the controller is further configured to select a feasible motion trajectory for performing the task from the multiple motion trajectories.

4. The controller according to claim 3, wherein: The feasible motion trajectory is selected from the plurality of motion trajectories so that the feasible motion trajectory is an optimal motion trajectory with the lowest objective value in a minimization problem or with the maximum objective value in a maximization problem.

5. The controller according to claim 3, wherein: The parameter function is trained based on a multi-class classification method to output a probability or confidence level for each of the predicted values ​​for each discrete variable in the set of discrete variables, wherein the first subset of discrete variables is selected based on the probability or confidence level of the predicted values.

6. The controller according to claim 5, wherein: A plurality of values ​​for each of the discrete variables in the MICP are selected to maximize a probability or confidence level output by the trained parameter function for each set of predicted values ​​for the discrete variable.

7. The controller according to claim 5, wherein: The multiple values ​​of each of the discrete variables in the MICP are generated by sampling a probability distribution at a single time step or multiple consecutive time steps within a prediction time horizon, and the probability of each fixed set of values ​​of the discrete variables in the discrete optimizer set at a single time step or multiple consecutive time steps within the prediction time horizon is equal to the probability output by the trained parameter function for performing the task defined by the parameters.

8. The controller according to claim 3, wherein: The parameter function is trained based on a regression approach to directly output a continuous approximation of feasible values ​​for one or more discrete variables at a single time step or multiple consecutive time steps within a prediction time horizon of the MICP for performing the task defined by the problem parameters.

9. The controller according to claim 8, wherein: The predicted values ​​for the set of discrete variables are generated by rounding the successive approximations output by the trained parameter function to the nearest integer feasible candidate solution.

10. The controller according to claim 3, wherein: The processor is at least one parallel processor, and finds a solution to the MICP problem by testing multiple candidate solutions in parallel to find the feasible motion trajectory.

11. The controller according to claim 1, wherein: To perform the pre-solved based correction, the controller iteratively selects discrete variables into the first subset and updates predicted values ​​in the remaining subset of discrete variables until infeasibility is detected or until the values ​​of all discrete variables are fixed to allow conversion of the MICP to the CP problem; If the infeasibility is detected, executing a backup control solution to change the state of the device; and On the contrary, if the values ​​of all discrete variables are fixed, the CP problem under the constraints is solved to generate the feasible motion trajectory.

12. The controller according to claim 11, wherein: The iteration of the pre-solution-based correction consists of one or more single pre-solution steps, and each single pre-solution step is an iterative process that performs one or more iterations until a termination condition is satisfied, and each iteration of the single pre-solution step includes one or a combination of pre-solution operations, and the pre-solution operations include: performing domain propagation for bound enforcement of continuous optimization variables and / or discrete optimization variables based on equality constraints and inequality constraints in the MICP for performing a task defined by problem parameters; Based on the cost function and based on the equality constraints and inequality constraints in the MICP, detecting and removing redundant inequality constraints, and dual fixing of continuous optimization variables and / or discrete optimization variables are performed; Inequality constraint coefficient strengthening for replacing one or more inequality constraints by one or more dominant inequality constraints, and the dominant inequality constraints are capable of reducing the feasible search space of the relaxed problem without removing integer feasible solutions of the MICP; and A binary variable probe is intended to fix the value of one or more discrete variables in the MICP by probing the feasibility of the fixed value of one or more discrete variables for performing a task defined by problem parameters.

13. The controller according to claim 12, wherein: Each iteration of the single pre-solve step comprises one or more customized block sparse pre-solve operations that exploit the block structured sparsity of a mixed integer optimal control problem (MIOCP) for performing the task defined by the parameters.

14. The controller according to claim 13, wherein: The customized block sparse pre-solve operation can include one or a combination of the following: a block sparse forward-backward operation for performing bound enforcement and domain propagation of continuous optimization variables and / or discrete optimization variables based on equality constraints and inequality constraints in a block structured MIOCP for performing a task defined by problem parameters; performing block sparse redundant inequality constraint detection and dual fixing of continuous optimization variables and / or discrete optimization variables based on a cost function and based on equality constraints and inequality constraints in the block structured MIOCP; block sparse inequality constraint coefficient strengthening for replacing one or more inequality constraints by one or more dominant inequality constraints in the block structured MIOCP; and A block sparse binary variable detection is provided, which aims to fix the value of one or more discrete variables by detecting the feasibility of fixing the value of one or more discrete variables in the block structured MIOCP.

15. The controller according to claim 2, wherein: The parameter function is a recurrent neural network (RNN), and each step of the RNN corresponds to a prediction of the values ​​of a set of discrete variables at a single time step or multiple consecutive time steps within a prediction time horizon of the MICP, such that sequential evaluations of the RNN produce predicted values ​​for each of the discrete variables of the MICP.

16. The controller according to claim 2, wherein: The parameter function approximation architecture can be composed of one or more layers of feed-forward deep neural network, one or more layers of long short-term memory LSTM network, one or more layers of gated recurrent unit GRU and / or one or more layers of convolutional neural network CNN.

17. The controller according to claim 1, wherein: A first subset of discrete variables in the MICP are fixed to predicted values ​​output by a trained parameter function, a second subset of discrete variables in the MICP are fixed to values ​​uniquely defined by the fixed values ​​of the first subset of discrete variables and the constraints through a pre-solve based correction, and a third subset of discrete variables in the MICP are selected as free optimization variables so that an approximation of the MICP with a reduced number of discrete optimization variables is solved under the constraints to produce feasible motion trajectories, for example, using a branch and bound (B&B) optimization method in combination with one or more customized block sparse pre-solve operations that exploit the block structured sparsity of the mixed integer optimal control problem (MIOCP) for performing the task defined by the parameters.

18. A hybrid dynamic system, comprising the controller according to claim 1, wherein: The control command generated by the controller specifies a target state of the hybrid dynamic system, the hybrid dynamic system comprising: A tracking controller is configured to generate one or more control inputs for actuators of the hybrid dynamic system to reduce an error between a current state and a target state of the hybrid dynamic system, wherein the tracking controller is a PID controller or a model predictive controller MPC.

19. The controller according to claim 1, wherein: The controller is a mixed integer model predictive controller (MIMPC), wherein the MIMPC calculates control signals based on parameters of the task, the parameters including the current state of the system and control commands, and wherein the MIMPC calculates a control solution by solving a constrained mixed integer optimization problem at each control time step, the control solution including a sequence of future optimal discrete control inputs and continuous control inputs over a predicted time horizon of the hybrid dynamic system.

20. The controller according to claim 1, wherein: The device is a vehicle, and wherein the controller determines inputs to the vehicle based on a MICP solution, wherein the inputs to the vehicle include one or a combination of vehicle acceleration, vehicle engine torque, braking torque, and steering angle; discrete optimization variables are used to model one or a combination of discrete control decisions, switching, gear shifting, and obstacle avoidance constraints of system dynamics; and the problem parameters of the task may include a current state of the vehicle, a target or target state, a position and / or orientation of one or more obstacles, one or more actuation restrictions, one or more weight values ​​in an objective function, and / or one or more bounds in a mixed integer inequality constraint of the mixed integer optimal control problem (MIOCP).

21. The controller according to claim 1, wherein: The device is a spacecraft, and wherein the controller determines inputs to the spacecraft based on a MICP solution, wherein the inputs to the spacecraft drive one or a combination of thrusters and momentum exchange devices; discrete optimization variables are used to model one or a combination of discrete control decisions, switching of system dynamics, integer values ​​of thruster commands, and obstacle avoidance constraints; and the problem parameters of the task can include a current state of the spacecraft, a target or target state, the position and / or orientation of one or more obstacles, one or more actuation restrictions, one or more weight values ​​in an objective function, and / or one or more bounds in a mixed integer inequality constraint of the MIOCP.

22. The controller according to claim 1, wherein: The device is a steam compression system, and wherein the controller determines inputs to the steam compression system based on a MICP solution, wherein the inputs to the steam compression system include one or a combination of indoor unit fan speed, outdoor unit fan speed, compressor speed, expansion valve position, and flow reversing valve position; discrete optimization variables are used to model one or a combination of discrete control decisions, switching of system dynamics, and integer values ​​of commands sent to valves and / or fans; and the problem parameters of the task may include a current state of the steam compression system, a target or target state, one or more actuation restrictions, one or more weight values ​​in an objective function, and / or one or more bounds in a mixed integer inequality constraint of the MIOCP.

Citation Information

Cited By

  • Coal blending optimization method, system and equipment for coal blending coking, medium and program product

    CN122046747A