Control method and processor programmed with instructions - Patents.com

Abstract

It allows for a network of hybrid controllers that can compute efficiently and adapt to changes in the controlled system(s). The present invention enables a network of hybrid controllers that can be computed efficiently and adapt to changes in a controlled system. The network includes at least one hybrid controller that includes a dynamic sub-controller and a learned system sub-controller. Information about the run-time performance of the controlled system is provided to both the hybrid controller and the over-controller, which provides one or more control inputs to the hybrid controller to modify the run-time behavior of the hybrid controller.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 609,244, filed December 21, 2017, which is incorporated herein by reference. [Background technology]

[0002] Controllers that control a particular system (e.g., maintain the output of the system at a desired level) can be configured in a variety of ways. In some examples, a controller may include components that are trained or otherwise specified to be models of the system to be controlled. A controller may also include dynamic components (e.g., filters or other dynamic control structures) in combination with such system modeling components to improve the overall dynamic response of the controller. However, such controllers, despite including components specified to model the internal structure of the controlled system, may not perform optimally under conditions that are different from those used to generate the training data used to train the system modeling components of the controller, for example. Summary of the Invention

[0003] To improve control of a system of interest, a hybrid controller may be provided that includes both dynamic components (e.g., filters or other dynamic controller elements) and components designated to model the controlled system (e.g., learned system models trained based on observed past behavior of the controlled system). Such hybrid controllers may provide improved output in terms of controlling the output of the system, or in terms of some other operating parameter of interest, compared to alternative controllers that include only dynamic or system modeling elements. Hybrid controllers may also be implemented cost-effectively and with minimal computational resources (e.g., cycles, memory, cores). Hybrid controllers may generate control outputs that may take into account learned information about the complex structure of the controlled system (via the system modeling components) while providing certain formal guarantees regarding their performance (due to the presence of analytically tractable dynamic components).

[0004] However, such hybrid controllers may exhibit suboptimal behavior under certain circumstances. For example, if the controlled system and / or the system environment differ significantly from the conditions used to develop the hybrid controller (e.g., to train the learned system model of the controller), the hybrid controller may produce suboptimal outputs. In such examples, an additional controller (e.g., an additional hybrid controller) may be provided to monitor and govern the hybrid controller. The additional controller may receive the output of the system being controlled by the hybrid controller and, based on the output, provide one or more outputs that control the operation of the hybrid controller. Such outputs may include one or more constraints of the hybrid controller (e.g., a target value used for feedback control of the system, a transmission latency limit, etc.). Such outputs may include values ​​of one or more parameters of the dynamic system model components of the hybrid controller and / or the learned system model components. For example, the output of the overall controller may be the magnitude or phase of the poles of the filter, the feedback gain, or some other operating parameter of the dynamic components of the hybrid controller. In another example, the overall controller may be a filter pole magnitude or phase, a feedback gain, or some other operating parameter of the dynamic components of the hybrid controller. The output of the controller may be a learning rate or other parameter of the trained system model component of the hybrid controller. In some examples, the output of the general controller may be a determination that a process should be executed on the hybrid controller, e.g., that the trained system model of the hybrid controller may be asynchronously updated and / or enter a learning mode. In another example, the output of the general controller may be a selection of a new learning model to be used in the trained system component, e.g., switching from a regularized regression model to a Bayesian model. In another example, the output of the general controller may indicate new actuators to be controlled by the hybrid controller, e.g., if new resources become dynamically configurable after the initial deployment of the hybrid controller.

[0005] Such an overall controller can be used to control multiple hybrid controllers. Additionally or alternatively, multiple overall controllers (e.g., multiple overall hybrid controllers) may commonly control one or more hybrid controllers. Each of the hybrid controllers controlled by the overall controller may control a respective subsystem that belongs to a larger overall system. For example, a first hybrid controller may control a battery charge / discharge circuit configuration of an electric vehicle, and a second hybrid controller may control a drive motor of the electric vehicle. In such an example, the overall controller may be considered to be effectively controlling the overall system via the lower hybrid controllers. The overall controller may control the overall system by applying inputs (e.g., target values ​​or other constraints, filter poles or other dynamic system parameters, learning rates or other learned system model parameters) to the lower hybrid controllers that themselves directly control their respective subsystems, or by otherwise modifying or controlling the operation of the lower hybrid controllers. In some examples, such an overall controller may itself be a hybrid controller.

[0006] One aspect of the present disclosure relates to methods implemented by one or more computers, controllers, microcontrollers, graphic processing units, tensor processing units, application specific integrated circuits, and / or some other variety of computational substrates. The method includes (i) sensing one or more outputs of the system during a first time period, (ii) determining a first performance metric of the system based on the one or more outputs sensed during the first time period, (iii) operating a first hybrid controller to generate a first hybrid controller output based on at least the first performance metric, (iv) controlling the system during a second time period according to at least the generated first hybrid controller output, (v) sensing one or more outputs of the system during a third time period, (vi) determining a second performance metric of the system based on the one or more outputs sensed during the third time period, (vii) operating a second hybrid controller based on at least the first performance metric and the second performance metric to generate an updated constraint value, (viii) updating the first constraint according to the updated constraint value, (ix) operating the first hybrid controller to generate a second hybrid controller output based on at least the second performance metric, and (x) controlling the system during a fourth time period according to at least the generated second hybrid controller output. The first hybrid controller includes a first constraint, a first dynamic system, and a first trained system model. Operating the first hybrid controller to generate a first hybrid controller output includes (1) determining a first difference between a first performance metric and the first constraint, (2) applying the determined first difference to the first dynamic system to generate a first dynamic system output, and (3) applying the generated first dynamic output to the first trained system model to generate a first hybrid controller output. The method includes generating a head controller output.

[0007] Another aspect of the disclosure relates to a method implemented by one or more computers, controllers, microcontrollers, graphic processing units, tensor processing units, application specific integrated circuits, and / or a number of other various computational substrates, the method including (i) detecting an output of a system during a first time period, (ii) determining a first performance metric of the system based on the output detected during the first time period, (iii) operating a first hybrid controller to generate a first hybrid controller output based on the first performance metric, (iv) controlling the system during a second time period in accordance with the generated first hybrid controller output, (v) detecting an output of the system during a third time period, (vi) determining a second performance metric of the system based on the output detected during the third time period, (vii) operating a second hybrid controller to control the system during a second time period in accordance with the generated first hybrid controller output, (v) detecting an output of the system during a third time period, (vii) determining a second performance metric of the system based on the output detected during the third time period, and (vii) operating a second hybrid controller to control the system during a second time period in accordance with the generated first hybrid controller output. (viii) updating the first hybrid controller according to the first hybrid controller update, (ix) operating the updated first hybrid controller based on the second performance metric to generate a second hybrid controller output, and (x) controlling the system during a fourth time period according to the generated second hybrid controller output. The first hybrid controller includes a first constraint, a first dynamic system, a first learned system model, and a first learned system update module. The first learned system update module is configured to update the first learned system model based on at least one output detected from the system and update the first learned system model according to at least one of a timing or a rate corresponding to the first learned parameter.The first dynamic system has a first dynamic parameter corresponding to an overall responsiveness of the first dynamic system. Operating the first hybrid controller to generate a first hybrid controller output includes (1) determining a first difference between the first performance metric and the first constraint, (2) applying the determined first difference to the first dynamic system to generate a first dynamic system output, and (3) applying the generated first dynamic output to the first trained system model to generate a first controller output.

[0008] These and other aspects, advantages, and alternatives will become apparent to those skilled in the art upon reading the following detailed description, with reference, as appropriate, to the accompanying drawings. Moreover, it should be understood that the description provided in this Summary section and elsewhere in this document is intended to illustrate the claimed subject matter by way of example, and not by way of limitation. [Brief description of the drawings]

[0009] [Figure 1A] 1 illustrates a schematic diagram of an example hybrid controller. [Figure 1B] 1 illustrates a schematic diagram of an example hybrid controller. [Diagram 2] 1 shows a schematic diagram of an example network of controllers. [Diagram 3] 1 is a flow chart of a method. [Figure 4] 1 is a flow chart of a method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Example methods and systems are described herein. It should be understood that the terms "exemplary," "example," and "demonstrative" are used herein to mean "serving as an example, instance, or illustration." Any embodiment or feature described as "is" should not necessarily be construed as preferred or advantageous over other embodiments or features. Moreover, the exemplary embodiments described herein are not intended to be limiting. It will be readily understood that certain aspects of the disclosed systems and methods are susceptible to arrangement and combination in a wide variety of configurations.

[0011] This application incorporates by reference the contents of Wang, Shu, et al., “Understanding and Auto-Adjusting Performance-Sensitive Configurations,” Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages ​​and Operating Systems. ACM, 2018; Zhang, Huazhe, and Henry Hoffmann, “Performance & energy tradeoffs for dependent distributed applications under system-wide power caps,” Proceedings of the 47th International Conference on Parallel Processing. ACM, 2018; and Santriaji, Muhammad Husni, and Henry Hoffmann, “Formalin: Architectural Support for Power & Performance Aware GPU,” 2018 IEEE Conference on Control Technology and Applications (CCTA). IEEE, 2018.

[0012] I. Overview The present disclosure discloses systems, methods, and apparatus that combine machine learning and control theory to monitor, control, improve, and / or optimize the performance of hierarchical, distributed, or other complex systems composed of one or more subsystems by creating one or more extended machine learning control systems that ensure such complex systems function optimally under dynamic operating conditions, where each of the one or more subsystems has similar, distinct, or common, local and / or global goals, and where each of the one or more controlled subsystem(s) of such complex systems is tightly or loosely coupled in hardware and / or software for the purpose of collectively optimizing and / or monitoring one or more or all of the subsystem(s).

[0013] Some of the disclosed embodiments relate to monitoring and / or optimizing such control system(s) or subsystem(s) by using a learning scheme(s) including, but not limited to, based on dynamically monitoring, learning, and changing the configuration of the control system(s) during application execution and data evaluation of system operational performance, and the system or subsystem(s) may dynamically adjust at least one of its controlled constraints and / or performance objectives and / or adjust the supervisory control system itself.

[0014] Some of the disclosed embodiments use metadata obtained from evaluating dynamic changes in system performance over time resulting from such optimization and / or monitoring of the targeted constraint(s) and / or performance goal(s), e.g., to establish beneficial constraints and performance goals; to prevent minima / maxima or other cascading failures, conflicts in one or more systems or subsystems; to develop advanced learning methods to make the generation of results more efficient; e.g., to use machine learning methods (e.g., artificial intelligence: neural networks) for time-sequential or other simulation of potential outcomes; The present disclosure relates to methods for performing adaptive neural networks (ANAs), such as replacing generative adversarial networks in whole or in part and / or otherwise improving their performance, for the purposes of operational evaluation and / or improvement of adaptive neural networks (ANAs, network-based neural networks, or other such methods).

[0015] Machine learning control systems (artificial intelligence) are being widely deployed and adopted across industries in applications such as financial trading, autonomous vehicles, logistics systems, chips and microprocessors, military systems, etc. Under the supervised component of unsupervised, supervised, or semi-supervised learning, each of these systems requires an initial "training" of large data sets that describe the environment to be controlled, which are then processed through neural networks of various complexities and tested to ensure that, once deployed, the control system "knows enough" to produce the desired outcome with a high probability. Under the unsupervised component of unsupervised, or semi-supervised, learning, this data is generated frequently.

[0016] Typically, when attempting to develop a control solution that can operate near optimally, the more data, the higher the probability of such a desired outcome. However, in practice, performance may not be optimal, the inner workings of neural networks or other trained predictive and / or control algorithms are not easily understood, data is often expensive to obtain or of limited breadth, and machine learning-based control systems must update their learning sets and operating logic as they operate in real time and can gather additional operating data of their own.

[0017] Traditional control theory approaches, where system operating parameters may be derived from known differential or difference models, produce probabilistic control outcomes and contrast with machine learning systems, which introduce varying degrees of uncertainty regarding the “completeness” of the training or other data set(s) and the risk of encountering unexpected operating conditions.

[0018] Machine learning and control theory techniques can be combined into what we call a "hybrid control system." Such a hybrid control system introduces the traditional idea of ​​dynamic optimization using one or more targeted constraints and / or performance objectives, and augments the capabilities of the machine-learned system by building deeply integrated, responsive prediction tables for system behavior around targeted constraints and / or performance objectives that drive the underlying machine-learned system more quickly toward its optimal global target when it encounters unlearned conditions, unexpected "poor behavior," or other unplanned or unexpected events.

[0019] For example, such composite machine-learned systems may be applied to speed up the learning process by creating various mathematical methods such as generative adversarial networks, e.g., DCGANs, Wasserstein GANs, and DCGAN ensembles, using Bayesian and other methods, in either supervised, unsupervised, or semi-supervised systems, including those using reinforcement learning, transfer learning, or active learning.

[0020] For example, these deep adversarial neural networks have proven to be more effective at both conditional and unconditional modeling of complex data distributions. Conditional generation allows for interactive control, but creating new controls or updating previous modeling often requires costly retraining. Either way, the goal is to develop better modeling (i.e., modeling that is faster, more economical, and leads to better predictions, among other things).

[0021] Training can be expensive and time-consuming, and can leave many learnings about the inner workings of neural networks. To improve understanding of machine learning, various techniques can be applied to reduce the requirements on the size and breadth of the required training set, for example by reducing the precision from 32-bit and 64-bit precision to 4-bit and 8-bit precision calculations. Most of these methods have been developed with the goal of improving understanding of computational efficiency, assessing risk or performance, improving the accuracy of unsupervised learning, reducing unintended consequences, and reducing time to market with reliable and understandable machine learning systems.

[0022] Neural networks have good empirical performance, but their inner workings are poorly understood. Other techniques can be applied to improve their empirical performance, including learning methods with better understood mathematical properties, such as various kernel and spectral learning methods.

[0023] Generally, prior work has failed to provide a system or method for simulating, creating, observing, learning from, contemporaneously monitoring, and / or optimizing a composite system that is comprised of one or more combinations of hierarchical, distributed, or other composite control systems, e.g., one or more hybrid control systems, in which loosely or tightly coupled, individually optimized subsystems operate and are configured together or make decisions using the same, related, or different constraint(s) and / or performance objective(s).

[0024] Additionally, this prior work does not provide a means to correct behavior when the dynamic operating environment violates these initial conditions or other conditions, nor does it provide a means to learn how to optimize the behavior of a distributed hierarchical system over time when abnormal operation of one or more subsystems becomes "normal" behavior over a period of time.

[0025] Furthermore, this prior work does not provide any static or dynamic system or method for establishing, monitoring during operation, or dynamically modifying optimal constraint(s) to improve the response time of the system relative to the global optimal performance goal(s), nor does it provide for using run-time monitoring data and / or optimization performance data to, for example, formulate or dynamically modify at least one constraint and / or performance goal(s) of the monitor itself.

[0026] Nor does this prior work provide a means to initially configure, then monitor, and control the actual performance of a system or method in a real-time operating environment, to develop better machine learning methods, or to simulate different outcomes using different constraints and / or performance objective targets, over time, sequentially, or otherwise.

[0027] A system, method and apparatus are described for: (1) Monitoring and / or optimizing, in hardware and / or software, individual complex systems or subsystems, or collections of complex systems or subsystems, whether tightly or loosely coupled, to create and control a complex control system of at least one distributed and / or hierarchical machine learning controlled control system(s) that satisfies at least one set of constraints and / or performance objectives, which may be the same, different, grouped, or otherwise set, for each system or subsystem.

[0028] (2) Monitoring the performance of a control system in a static or dynamic operating environment to satisfy at least one constraint and / or performance objective.

[0029] (3) Determining when a monitored system or subsystem is operating to optimally meet specified constraints or when it is not optimally meeting specified constraints.

[0030] (4) Dynamically tuning a learner, controller, or another system to respond to this behavior (e.g., adjusting goals or other operating parameters of such a system).

[0031] (5) Using data from the operation of a complex control system to monitor behavior and develop optimization goals when abnormal operation of one or more subsystems is the “new normal.”

[0032] (6) During the initial process of creating a machine-learned control system, enabling the creation of optimization parameters and / or monitoring parameters of the control system, and then monitoring and adjusting, statically or dynamically, machine-learned or otherwise, the dynamic performance of the distributed system globally, or at any one or more of its components, or in its own management.

[0033] (7) Enabling the system or any of its subsystems to dynamically adjust at least one of its constraints and / or performance objectives.

[0034] (8) Using information about the performance of the system or subsystem(s) to create databases and libraries of performance metrics or similar records that can be used to make decisions, predictions, and / or simulations for the underlying system or subsystem(s) and / or for other related and / or unrelated systems used by others.

[0035] (9) Using dynamic operational information (i.e., performance metadata) regarding optimization and / or supervisory actions to reduce requirements for supervised, unsupervised, or semi-supervised datasets, including but not limited to new methods of deep learning.

[0036] (10) To evaluate the feasibility of a hierarchical distributed system configuration in the context of real data on which the underlying control system operates over time, simulating the effects of various constraints and / or performance objectives being variously selected or configured, either by machine learning or otherwise.

[0037] Prior work has disclosed systems and methods for improving the robustness of composite, optimized, machine-learned systems in response to actual observations of dynamic events.

[0038] The power of machine learning and control theory can be applied to existing complex systems. A learning system measures the system's behavior and uses it to build an optimization model. The model is passed to a control system, which uses the model plus specified constraints and the measured system behavior to adjust parameters to ensure that the constraints are most appropriately met.

[0039] The advantage of this approach is twofold: First, the combination of learning and control deals with both complexity and dynamics, outperforming approaches that rely on only one or the other (purely for convenience, we will identify these control systems with machine-learned and control-theoretic components as "Hoffman Optimization Frameworks", "HOFs", "frameworks", "abstracted controllers", "abstracted control systems", "hybrid controllers", or "hybrid control systems").

[0040] Second, this combination makes the resulting system demonstrably robust: it is easy to determine and set system tolerances between the learned model and real-world behavior that still allow the combination to converge to the specified constraints.

[0041] In general, a complex optimized system using HOF is both observable and tunable. Specifically, the specified constraints are known, and the system is already instrumented to generate feedback that measures behavior over any range of constraints. Thus, the quality of the HOF control can be observed, including, but not limited to, the following: Is the system stable or oscillating? If the system does not oscillate, does it stabilize to the desired constraints? If not, how far is the stable behavior from the desired behavior? When an unexpected event causes the system to become unstable, how long does it take for the system to respond? For the same event, what is the maximum error that can occur on the way back to stabilization? Once the system has stabilized to the desired behavior, how far is the system from a state that is estimated to be optimal with respect to those constraints? The answers to all these questions can be quantified using the measured difference between the specified constraints and the feedback of the behavior.

[0042] With this insight that the optimized system can be further evaluated to provide monitoring as a standalone monitor or in combination with optimization strategies and / or tactics in a hierarchical and / or distributed complex control system or while in the field, the systems and methods described in this disclosure can provide various advantages.

[0043] A composite control system under the supervision of a HOF has at least one constraint and / or performance objective that the control system optimizes the system based on a response to a real-time dynamic influx of data regarding the operation of itself or other HOFs in the system. Also described herein are systems and methods for setting constraints and performance objectives that can themselves be evaluated and selected to further optimize the optimization of the system being controlled, whether it is a single HOF or a composite of multiple HOFs operating in a hierarchical system with one or more HOFs. The complexity of such analysis, in one example, may require the use of HOFs (machine-learned or not) that themselves use operational data to set optimal static or dynamic constraints and performance guarantees consistent with the system's performance objectives, or consideration of options.

[0044] The behavior of the HOF can be expressed as the difference between the achieved action and the constraints over a series of time steps. A well-behaved controller optimally corrects the deviations to keep performance close to the requirements.

[0045] A key insight in applying these mechanisms is that the time series of differences between desired and achieved behavior in any constrained range can be used to detect and drive adjustments in both learning and control behavior. More formally, the error e between any constraint c and measured behavior b at time t can be calculated as e(t)=c(t)-b(t).

[0046] For example, once the error at time t has been calculated, it can be used to detect a combined learning and control system that is misbehaving. If the HOF is not working properly, it may exhibit one or more of a variety of behaviors:

[0047] Oscillation: A system is never stable, but alternates between underperforming and overperforming the constraint(s) and / or performance goal(s). The pattern of oscillations can be very complex, passing through many intermediate states on the way to the extremes. Such oscillations can be indicated by errors going from positive to negative and back again.

[0048] Slow response to dynamic changes: The error suddenly becomes a large positive or negative number, but then takes many time steps to return to a magnitude near zero.

[0049] Steady underperformance: The system stabilizes at below-target performance, resulting in large positive errors.

[0050] Steady overperformance: The system stabilizes at exceeding the target performance, resulting in a large negative error.

[0051] Biased towards one side of the constraint: Instead of distributing measurements evenly to be slightly above and below the behavior constraint, the controller disproportionately weights one side of the constraint.

[0052] If these or other anomalous operations are detected, the learner, the controller, the target itself, or some other parameter of the control system may be modified as follows:

[0053] Modifying the learner: Previous work has assumed that the learning system eliminates local optima and that the controller can solve the convex optimization problem. However, expected or unexpected behavior may introduce new local optima, causing the controller to exhibit behaviors including, but not limited to, the "oscillation" or "side-weighting" described above. Thus, if oscillatory behavior or side-weighting is detected, the parameters of the learner can be adjusted to incorporate new data points. In one of the simplest cases, the selected configuration of the composite system and its resulting behavior may be recorded and added to the training data of the learner. In that case, for example, the learner can output a new model that removes the local optima and achieves the expected stable performance. If the learner is already built to perform online updates (e.g., some reinforcement learners), the system may adjust the learning rate to increase the rate at which new information about the local optima is incorporated into the model used by the control system.

[0054] Modification of the controller: All kinds of control systems expose parameters that trade sensitivity to errors for convergence speed, given a disturbance. For example, in a PID controller, these variables include one or more poles. Given a system that oscillates, or that heavily weights one of the constraints, these control parameters can be adjusted to increase convergence speed at the expense of a wider error tolerance. This adjustment prevents the system from violating the constraints, but the unconstrained behavior may still be suboptimal. Adjusting these control parameters can also be useful when a system is observed to be too slow to reconverge after a dynamic event. For example, the original system may have been designed too pessimistically, assuming error conditions that do not occur in real deployment, and adjusting the control parameters can allow the system to accommodate this better-than-expected behavior. In both cases, it can be useful to combine adjustment of the control parameters with adjustment of the learning parameters. For example, if an unexpected error is causing the oscillation, the control parameters can be adjusted to slow down the controller's action and eliminate the oscillation immediately, while the learning parameters can be adjusted more slowly to generate a new model that incorporates the new operating conditions.

[0055] Modifying targets: Additionally, the learning and control parameters themselves may be modified. Additionally or alternatively, new virtual targets may be created. Virtual targets are targets that cannot be detected otherwise. It can be used to correct behavior that was not achieved or anticipated at design time. For example, if the deployed environment is much noisier than expected (e.g., more variability than designed), the proposed system may override the specified constraints and set new constraints with a different operating range. For example, if a system has a 95th percentile quality of service constraint, but the runtime variability is greater than expected, a virtual target may be set that provides an even higher quality of service. In another example, the deployed system may become unable to meet one or more user-specified goals. The non-reachability of the goal may be tested using standard control methodologies (usually applied at design time, and may be modified here for runtime or other evaluation(s)). This controllability test may allow the system to formally conclude that it is no longer able to meet the specified goal. In this case, the system may set a virtual or other goal that is different from the desired goal, and then report the status.

[0056] Machine learning systems cannot easily handle the challenges inherent in many performance configurations, such as stringent constraints such as not running out of memory, and the indirect relationship between performance configuration and performance, which becomes even more complex and challenging when, for example, a system is designed that includes a hierarchical or distributed architecture of separate or related components or subcomponents.

[0057] Setting and adjusting performance configurations in such distributed architectures presents many challenges, including: (1) performance configurations that threaten hard performance constraints, such as out-of-memory or out-of-disk problems; (2) performance configurations that indirectly affect performance by setting thresholds for other system variables; (3) performance configurations that are tied to specific system events and therefore only take effect conditionally; (4) often various configurations simultaneously affect the same performance goals and require adjustments; (5) in hierarchical systems that control subsystems with local goals and master primary systems with global goals, conflicts arise requiring immediate response and adjustments; (6) real-time events may result in long-term changes from "normal" to a "new normal" state, rendering one or more constraints temporarily or permanently, partially or fully, inappropriate or even dangerous; and (7) resolving potential conflicts or adjustment requirements that arise between the optimizing and monitoring roles of the control system.

[0058] Unlike traditional configuration frameworks in which a user sets performance configurations at system startup, in certain embodiments, the Hoffman optimization framework automatically sets and dynamically tunes performance configurations during operation across distributed architectures that may include one or more HOFs in loosely or tightly coupled hardware and / or software control systems. The Hoffman optimization framework decomposes the problem of performance configuration setting.

[0059] HOFs allow performance constraints, goals, and / or formal guarantees to be specified without worrying about how to set and adjust configurations to meet those constraints and / or goals, with or without monitoring and / or optimizing other performance metrics. HOFs can do so, for example, on distributed complex systems, with or without one or more HOFs, and in the presence of other complex systems. HOFs provide these optimization capabilities to set performance configurations to constraints and / or goals dynamically (during run-time), statically, or by other means, but also provide the ability to act as a watchdog for a group of complex systems, or a group of one or more HOFs, to monitor, manage, or otherwise optimize performance against the goals and constraints.

[0060] As mentioned above, one of the main advantages of these control systems is that they are a composite, optimized system. The goal of the present invention is to make the system robust against, for example, abnormal operation, errors, or other suboptimal conditions or configurations that may arise during the execution of the system. Such conditions may arise from a number of causes, either alone or in combination, such as the system encountering an environment for which it was not originally designed, without the need to understand or detect the exact causes.

[0061] One limitation of prior work is that it may ignore real-world situations that often arise, for example, when multiple separate components (subsystems) contribute to the performance of the overall system, and each of these subsystems is optimized for its own local and system-wide requirements.

[0062] For example, in a three-tier web server, the overall latency and power is determined by the latency and power of (1) the load balancer, (2) the request manager, and (3) the backend database. In larger systems, these individual components can operate semi-autonomously and may be physically separated. Other examples include a complex fleet of drones and other delivery vehicles operating in concert with other physical assets in a complex logistics system, or autonomous vehicles with both internal and external performance goals.

[0063] Managing multiple HOF implementations can become even more complex during operation, and sometimes the constellation of global solutions, and the timing of their implementation, may need to be expanded or contracted so that the local or other goals (performance guarantees and / or constraints) of individual HOF implementations for specific subsystems meet the global performance objectives.

[0064] To optimize these distributed, complex control systems, supervisory HOF methods are needed. Disclosed herein are systems and methods in which a higher-level HOF controller (or multiple such higher-level HOF controllers) has at least one constraint and / or performance goal for its overall system performance, and tracks and manages progress toward a global goal managed by the lower level local HOFs and / or other means as they work to meet their respective local goals.

[0065] In some examples, a HOF may be used to control a distributed hierarchy that is made up of at least one HOF implementation.

[0066] In some examples, constraints and / or performance objectives may be used to control a HOF that is itself being used with similar or different constraints when the HOF controls single or multiple HOF architectures in simple or complex distributed systems.

[0067] In some examples, constraints and / or performance objectives may be used to control HOFs that control the HOFs themselves and / or manage resource utilization in single or multiple HOF architectures in simple or complex distributed systems.

[0068] In some examples, a HOF or a network composed of one or more HOFs can be adapted to the role of monitoring and / or optimizing a control system by monitoring operational data and collecting, reporting, and responding to data regarding the performance of the control system.

[0069] An example would be monitoring the number of pixels used by an image processor to make decisions for an autonomous vehicle. If the image does not meet the system requirements, the role of the monitor is to evaluate whether the number of available pixels from the image processor and other elements of the imaging system are within the tolerance range of that sensor, and then infer an action with an optimal solution based on established constraints and performance goals. The information is then used to later optimize the system. A learner is then available to make the decision.

[0070] In some examples, HOFs may be used in optimization and / or monitoring roles, alone or in combination, in simple or complex distributed control systems, where the HOF controls and / or optimizes single or multiple architectures comprised of at least one or more HOFs.

[0071] Another limitation of the Hoffman framework for single deployment is that the maximum tolerable error must be known or estimated at the time the system is designed: if the system is deployed and an unexpected large error occurs (e.g., due to an unexpected cascading system failure), there is no guarantee that the system meets the user-specified operational constraints.

[0072] To compensate for this limitation, the present disclosure provides a system and method for overcoming the above limitations, allowing a hybrid, optimized system to continue to run, for example, even when it encounters cascading run-time errors or when abnormal operation becomes the "new normal."

[0073] In some examples, one or more HOFs can be used to control and / or optimize one or more HOFs or other control methodologies, thereby creating operating boundaries, for example, by removing actuators from control of the original system or other systems.

[0074] In some examples, the behavior of hierarchical and / or distributed control systems is monitored to adjust constraints and / or performance guarantees as the system responds to static or dynamic operating data to further optimize the means to meet global or local goals.

[0075] In some examples, operational "metadata" derived from the operation of the composite control system and the performance of the underlying composite system being controlled is collected as operational data and used collectively or individually or in combination to inform the control system(s) whose responses are observed and recorded. Temporal and event sequence information (among others) is used dynamically to configure operational performance changes in response to run-time operating conditions and operational targets for both monitoring and / or optimization of the HOF(s), as well as for post-analysis and system performance improvement of the learner and / or controller, to analyze alternatives and create ideal optimization strategy(s) and / or static or dynamic tactic(s) for other applications using other data from the underlying system or other systems.

[0076] In some examples, tools derived from such analyses are used to create constraints and / or performance objectives for one or more HOFs that are dynamic in time and change in response to operational data evaluated over that same period of time, e.g., such that optimization and / or monitoring objectives at any point in time may be the same or different depending on operational performance.

[0077] These tools may be deployed initially, for example, as simulations using test data or training sets, or other simulations responsive to input data, and these tools may be deployed to a running system, a simulation, or a combination of simulation(s) and running system(s).

[0078] For example, these tools may use, for example, machine learning tools, derived electronically, with or without human involvement, and / or both methods, or from natural language generation. The system may be deployed in a combination of novel methods, such as the above-mentioned reports.

[0079] II. Hybrid Controller Example As used herein, the term "hybrid controller" refers to a controller that includes multiple distinct sub-components that cooperate to generate one or more control outputs (e.g., outputs applied to a system under control) based on one or more control inputs (e.g., inputs detected from or otherwise received from a system under control). The various components of such a hybrid controller receive respective inputs and generate respective outputs, and the output of one component may be provided as an input to another component according to a particular configuration of the hybrid controller. In certain examples, a hybrid controller may include a dynamic system component and a learned system model component. The dynamic system component may include one or more filters, integrators, differentiators, gains, or other elements to enable the controller to exhibit time dynamics (e.g., time dynamics characterized by one or more poles and / or zeros) and provide certain formal guarantees regarding the overall controller / system performance. The trained system model component may include lookup tables, polynomial functions, multivariate functions, arbitrary complex behaviors encoded in neural networks or other models, or some other elements based on information about the behavior and / or structure of the controlled system (e.g., training data) that enable the hybrid controller to provide control that is adapted to the observed past optimal (and suboptimal) behavior of the system under control, to exhibit complex nonlinear and / or multidimensional behavior, and to provide many outputs based on a small number of inputs. A hybrid controller may include multiple such trained system models, and the hybrid controller may switch between these models (e.g., based on its confidence in the accuracy of each trained system model given the data it receives about the operation of the controlled system).

[0080] The hybrid controller may also include a learned system update module configured to train or otherwise update the learned system model(s) based on newly received data regarding the operation of the controlled system. Such a learned system update module may operate to update the learned system model(s) according to an update schedule (e.g., to update the learned system module once every absolute number of seconds elapsed, every number of calculated controller time steps, or some other schedule) and / or according to timing determined from some other information or condition (e.g., to update the learned system model when the observed behavior of the controlled system deviates from the behavior predicted by the learned system model by more than a specified threshold). Such a learned system update module may operate to update the learned system model by an amount specified, for example, by a step size or step length associated with a gradient descent training method.

[0081] 1A shows an example of a hybrid controller 100a configured to control a system of interest 101a. The system may be any system of interest, such as one or more processors of a server or some other computing platform, a logistics hub or other facility of a logistics network, a joint motor or other subcomponent of a robotic system, or any other system that it is advantageous for the system to control. The hybrid controller 100a includes constraints 120a, a dynamic system 130a, a trained system model 140a, and a trained system update module 150a. The hybrid controller 100a also receives inputs 110a from the controlled system 101a.

[0082] A constraint may be, for example, a target value specified for the output of the system 101a. The controller 100a may then cause the output of the system 101a to reflect the value of such a constraint 120a. The input 110a may generate an output for controlling the system 101a to perform a desired operation. The output of the system 101a is detected and then compared to the constraint via the input 110a. In some examples, the input 110a may simply be a pass-through for the detected output of the system 101a. Additionally or alternatively, the input 110a may include a filter, a transfer function, or some other algorithm or other relationship to generate a useful input value (e.g., a performance metric representing the overall performance of the system) based on one or more detected outputs of the controlled system 101a. For example, the output of the system 101a may be the number of operations per second performed by each of several cores of a processor, and the input 110a may be a performance metric corresponding to the sum of the number of operations per second of all the cores. In another example, the output of the system may be a measure of the overall performance of the system 101a (e.g., the total number of operations per second performed by a server), and the input 110a may be a value corresponding to an increase or "speedup" of the server output over time, relative to a baseline performance level. In another example, the output of the system may include one or more derived measures of overall system performance and overall power consumption of the processor, and the input to the system may include two values ​​corresponding to relative changes in performance and power consumption. In this example, the inputs and outputs are vectors, and each vector component is a metric or signal.

[0083] The difference between the constraints 120a and the inputs 110a is then determined and the difference is applied to the dynamic system 130a to generate a dynamic system output. The dynamic system may include one or more differentiators, integrators, gains, summations, multi-pole filters, sample-and-holds, leaky integrators, feedback loops, target values, nonlinear functions, nonlinear filters, polynomial functions, finite impulse response filters, infinite impulse response filters, internal state variables, or other dynamic elements configured to provide a dynamic, time-dependent output based on a differential input. The behavior of the dynamic system component 130a may be characterized by one or more parameters (e.g., one of the filters, or one or more of its poles and / or zeros, or one or more parameters of other elements of the dynamic system 130a). For example, the dynamic system may include a filter characterized by the magnitude and phase of one or more poles.

[0084] The output of the dynamic system component 130a is applied to the trained system model component 140a to generate one or more controller 100a outputs that can be applied to the system 101a. The trained system model component 140a can generate one or more outputs that can then be applied to the controlled system 101a. The output of the trained system model may include one or more operating parameters for each one of a set of actuators or other elements of the controlled system 101a. For example, the controlled system 101a may be a computational substrate (e.g., a server, a GPU, a network of servers, a cloud computing system) that includes an array of computing units (e.g., discrete processors, processor cores, memory units, arithmetic logic units, server blades, texture processing units). In such an example, the output of the controller 100a may include one or more operating parameters for each of the computing units. Such operating parameters may include clock speed, memory bus speed or other bus speed, bit width, whether a particular function unit should be active or inactive, the number or identity of function unit subcomponents (e.g., ALUs, memory cells) that should be active, or some other operating parameter of one or more of the function units of the controlled system 101a.

[0085] The trained system model component 140a may include one or more artificial neural networks, polynomial functions, nonlinear functions, multivariate functions, piecewise defined functions, lookup tables, or other elements for converting the output of the dynamic system component 130a into one or more outputs applicable to control the control system 101a. In some examples, the trained system model component 140a may include multiple different trained system models (e.g., models having different underlying structures, models initialized to different starting conditions prior to training / learning). The learned system model component 140a may include a set of learned system models (models that differ in terms of the model, models exposed to different sets of training data, models that differ in terms of the training methodology used to train the models based on the available training data, and / or models that differ in terms of some other consideration). In such an example, the learned system model component 140a may be configured to select one of the set of learned system models and apply the output of the dynamic system component 130a to the selected learned system model to generate one or more outputs applicable to control the controlled system 101a. Such a selection may be performed, for example, based on an estimate of the accuracy of each of the learned system models with respect to predicting the behavior of the controlled system 101a, or based on some other determined confidence of the sub-elements of the learned system model component 140a. Such an estimate may be determined based on detected outputs from the controlled system 101a, for example, based on such outputs by the learned system update module 150a.

[0086] For example, FIG. 1B illustrates an example of a hybrid controller 100b configured to control a target system 101b, including constraints 120b, a dynamic system 130b, and a learned system model 140b, and receiving an input 110b from the controlled system 101b. The controlled system 101b includes four subsystems 105a, 105b, 105c, and 105d, which may be, for example, computing units of the controlled system 101b. The learned system model 140b includes a lookup table 145b. The lookup table 145b includes a number of entries, each of which includes at least one output that may be applied to control the system 101b. As an example, each entry of the lookup table 145b in FIG. 1B includes a value of the clock speed ("ClkA, ClkB, ClkC, ClkD") of each of the computing units 105a, 105b, 105c, and 105d of the controlled system 101b. Each entry of the lookup table 145b further includes an indication of how many cores of the first two computing units 105a, 105b are active ("CoresA, CoresB") and the bit width of the operation to be performed by the latter two computing units 105c, 105d ("WidthC, WidthD"). b The value of the output of the lookup table 145b is used to select a particular entry from the multiple entries of the lookup table 145b to output from the trained system model 140b, thereby controlling the system 101b. For example, each entry of the lookup table 145b may be a dynamic system component 130 b It may correspond to a range of values ​​for the output.

[0087] The learned system model components (e.g., 140a, 140b) of the hybrid controller described herein can be generated in a variety of ways. If a complete model of the system to be controlled is known, this complete model may be used to generate the learned system model (e.g., to generate lookup table entries and corresponding ranges of dynamic system output values). Additionally or alternatively, information about the past and / or ongoing observed behavior of the controlled system, or information about the behavior of a similar system (e.g., a server to be controlled or other instances of a particular model of another system), may be used to generate, update, and / or train the learned system model of the hybrid controller. For example, the learned system update module 150a may update and / or train the learned system model component 140a based on detected outputs from the controlled system 101a.

[0088] Updates to the trained system model based on acquired additional data regarding the behavior and / or structure of the controlled system may be performed locally by the same computational substrate (e.g., processor, server) used to implement the hybrid controller. Additionally or alternatively, such updates may be performed by a remote system (e.g., a cloud computing system) that then transmits information about the updated trained system model (e.g., updated weights of one or more neural networks, or The learning-based updates (e.g., values ​​of the control parameters, or other parameters, updated look-up tables) may be transmitted from the remote system to the system implementing the hybrid controller. Such a remote system may act as a service providing such learning-based updates for a number of different hybrid controllers. Such a remote system may generate such learning updates based on information about the controlled system (e.g., applied control outputs, sensed system outputs, and / or performance metrics) transmitted to the remote system from the system implementing the hybrid controller.

[0089] Such updates of the trained system model of the hybrid controller and / or learning about the behavior and / or structure of the controlled system may be made continuously, e.g., at a specified rate or according to a specified schedule. Additionally or alternatively, updates to the trained system model may be made in response to detecting some event, such as, for example, detecting that the performance of the hybrid controller has fallen below a threshold performance level, detecting an abrupt change in the performance of the controlled system, detecting that the controlled system has entered a particular undesirable operating state, or detecting some other event. The rate or timing at which the trained system model is updated and / or the magnitude or extent of such updates (e.g., step size of a gradient descent or other iterative update process) may be determined based on an estimate of the accuracy of the trained system model or based on some other determined confidence in the trained system model. For example, the trained system model may be updated more frequently and / or to a greater extent when the estimated accuracy of the model is low. Such an estimate may be determined based on, for example, the amount of error between a predicted system output generated by the trained system model and the observed output of the system.

[0090] Such determined confidence may also be used to update or control other aspects of the operation of the hybrid controller. For example, poles and / or zeros of dynamic system components may be adjusted to reduce the responsiveness of the dynamic system components, for example, when the confidence in the trained system model is low. This may be done, for example, to reduce the likelihood that the hybrid controller will generate control outputs that result in exceeding targets (e.g., maximum memory, maximum power usage) when the trained system model cannot accurately predict the behavior of the controlled system.

[0091] In some examples, a system (e.g., 101a, 101b) controlled by a hybrid controller (e.g., 100a, 100b) includes a computing host (e.g., a server, computer, graphics processor, or other system that includes one or more controllers, processors, or other computing elements). In these examples, the hybrid controller may be implemented using a portion of the computing resources provided by the computing host of the system. Additionally or alternatively, the hybrid controller may be implemented on another controller, processor, or other computing element(s) (e.g., an application specific integrated circuit made for a specific application).

[0092] III. Example of a Multi-Hybrid Controller Architecture As described above, hybrid controllers may be applied to control systems of interest. Such hybrid controllers offer many advantages, including the ability to incorporate observed information about the complex behavior and / or structure of the controlled system, and to provide time-dynamic controller outputs that can be formally shown to exhibit certain guarantees regarding overall system performance, e.g., by modifying the poles or other time dynamics of the dynamic controller based on a determined confidence in the accuracy of the trained system model, such that the overall controller behaves more "conservatively" when the model confidence is low. However, the performance of such hybrid controllers may vary under certain circumstances. For example, if the controlled system and / or the environment of such system deviates significantly from the system and / or environmental conditions used to generate the hybrid controller (e.g., to train the learned system model components of the hybrid controller), the hybrid controller may generate suboptimal outputs.

[0093] For example, the controlled system may be a network of servers, and a hybrid controller used to govern the network of servers may perform suboptimally if one of the servers experiences a critical failure. In another example, the controlled system may be an industrial robot, and a hybrid controller used to govern the robot may perform suboptimally if the robot's joint servos become inoperable or exhibit significant degradation in performance. In yet another example, the controlled system may be a logistics network for delivering packages, and a hybrid controller used to govern the network may perform suboptimally if an airport used by the network is closed due to inclement weather.

[0094] In such situations, the performance of the hybrid controller may degrade. However, the performance of the hybrid controller can be improved in these situations by changing the hybrid controller (e.g., by changing one or more constraints or other parameters of the hybrid controller, by updating the learned system model of the controller). This may include changing the constraints (e.g., 120a, 120b) of the controller (e.g., specifying a performance goal that is easier to achieve). In another example, the time dynamics of the controller may be modified by adjusting the filter poles or other parameters of the dynamic system elements of the hybrid controller (e.g., making the hybrid controller more or less responsive to changes in the output of the controlled system). In yet another example, the learned system model of the hybrid controller may be modified. This may include updating the learned system model (e.g., performing an asynchronous update of the learned system model) based on recent information about the controlled system, or adjusting the update rate of the ongoing learning process applied to the learned system model (e.g., update rate, step size of the backpropagation or other learning algorithm).

[0095] One or more supervisory controller(s) may be provided to determine that such adjustments to the operation and / or configuration of the hybrid controller are indicated and / or to generate such adjustments. Such supervisory controllers may receive as inputs the output(s) of the controlled system and generate as outputs values ​​of parameters or constraints of the hybrid controller, timing of updates to elements of the hybrid controller (e.g., a learned system model), or some other output related to the operation and / or configuration of the hybrid controller.

[0096] This is shown in FIG. 2 as an example. The first hybrid controller 200a is configured to provide a control output to the first system 201a and to sense one or more inputs from the first system 201a. The first hybrid controller 200a includes inputs 210a, constraints 220a (e.g., operating target values), a dynamic system 230a, and a learned system model 240a. The outputs of the system 201a are also provided as inputs to an overall controller 250. The overall controller 250 adjusts the constraints 220a, the dynamic system 230a (e.g., filter pole values ​​or some other dynamic parameters), and / or the learned system model 240a (e.g., updated learning rates or other parameters, learned system model) based on the sensed outputs from the first system 201a. The method is configured to generate values ​​and / or updates for one or more of the following:

[0097] The general controller 250 can be configured in a variety of ways. In some examples, the general controller 250 may include one or more control elements, such as filters, feedback loops, integrators, gains, differentiators, summers, polynomial functions. In some examples, the general controller 250 may itself be a hybrid controller. In some examples, the general controller 250 may be configured to detect one or more patterns or conditions in the output of the first system 201a and responsively adjust the operation and / or configuration of the hybrid controller 200a. This example may include the general controller 250 explicitly detecting the presence of such patterns or conditions (e.g., comparing the system output to a threshold, using a template matching algorithm to detect patterns) and / or elements of the controller (e.g., filters, lookup tables, feedback controllers) configured to effectively perform such detection.

[0098] Such detected events may include detecting the presence of one or more patterns or conditions in the system 201a output, in a filtered version of the system 201a output (e.g., in an integrated, bandpassed, differentiated, or otherwise filtered version of the system 201a output), in a target value-offset version of the system 201a output (e.g., a version offset by the values ​​of constraints 220a), or in an otherwise processed version of the system 201a output. Detecting a pattern, condition, or event based on the system 201a output may include (i) detecting that the output of system 201a is oscillating; (ii) detecting that the output of system 201a responds too slowly (e.g., slower than a threshold rate) or too quickly (e.g., faster than a threshold rate) in response to changes in the inputs and / or environment of system 201a; (iii) detecting that the output of system 201a overshoots a target value (e.g., beyond a threshold rate) in response to changes in the inputs and / or environment of system 201a; and / or (iv) detecting that the output of system 201a exhibits a consistent bias (e.g., beyond a threshold rate) relative to a target value (e.g., constraint 220a).

[0099] Such an overall controller may be applied to provide such control / monitoring to multiple different hybrid controllers. This is shown in FIG. 2 and also includes a second hybrid controller 200b configured to provide a control output to a second system 201b and to detect one or more inputs from the second system 201b. The second hybrid controller 200b includes inputs 210b, constraints 220b, a dynamic system 230b, and a learned system model 240b. The output of the second system 201b is also provided as an input to the overall controller 250. The overall controller 250 is further configured to generate values ​​and / or updates of one or more of the constraints 220b, the dynamic system 230b, and / or the learned system model 240b of the second controller 200b based on the detected output from the second system 201b.

[0100] In such an example, the different systems 200a, 200b may be subsystems of a single larger system. Thus, the general controller 250 may be considered to be effectively controlling the entire system through the subordinate controllers 200a, 200b. For example, the systems 201a, 201b may be different servers in a data center. In another example, the systems 201a, 201b may be the engines and hydraulic transducers of an airliner, systems that traditionally employ classical control and whose performance may be improved by applying the methods described herein. In yet another example, the systems The motors 201a, 201b may be battery management units and motors of an autonomous vehicle.

[0101] Additionally or alternatively, multiple general controllers may be applied to provide such control / supervision to one or more hybrid controllers under supervision. This is illustrated in FIG. 2, which also includes a second general controller 260. The second general controller 260 receives inputs from one or both of the hybrid controllers 200a, 200b under supervision, and the second general controller 260 may output updates and / or modifications to the learned system model(s) and / or the constraint(s), dynamic system parameter(s), and / or parameter(s) of the process used to train and / or update the learned system model(s).

[0102] In such an example, each governing controller 250, 260 may be configured to control a single system (e.g., the system 201a controlled by the first hybrid controller 200a) and / or a composite system including one or more subsystems (e.g., the first System 201a and the second System 201b In one embodiment, different performance objectives or constraints may be configured for a complex system (composite system including a

[0103] IV. Application Examples and Test Results The methods and systems described herein have been applied to a variety of applications to provide improved operation of the systems according to those applications. The control methods described herein are broadly applicable to any system in which efficient operation is desired in light of multiple competing goals. These control methods may be applied to the operation of mobile devices, equipment such as data center servers, distributed software systems, video distribution networks, autonomous vehicles, graphic processing units, handheld radios, satellite global positioning receivers, or other complex systems.

[0104] In one application, the control method described herein was applied to improve the performance of the Samsung Exynos 5422 processor shipped in the Samsung Galaxy S5 handset. In this application, the method achieved a 14% reduction in energy usage on complex benchmarks compared to alternative state-of-the-art optimization methods (including AI) at comparable or better performance levels. The method also achieved over 40% reduction in energy usage compared to stock Samsung / Android optimization software.

[0105] In another application example, the control methods described herein were applied to improve the performance of a dual Intel Xeon E5 2600 series processor with 64GB RAM running eight different applications, including web search, digital signal processing, clustering, image search, video encoding, and image tracking (which applications may run simultaneously). In this application example, the present method achieved an overall 8% reduction in energy usage (measured "from the wall", relative to the entire computer system) compared to alternative state-of-the-art optimization methods, including AI.

[0106] In yet another application, the control method described herein was applied to improve the performance of a composite software distributed storage and processing service running MapReduce, Cassandra, and Hadoop Database / Hadoop Distributed File System. In this application, the method achieved 34% speedup and 60% fewer crashes compared to alternative state-of-the-art optimization methods. This is particularly impressive, since the controller was not provided with any prior information about the underlying structure or capabilities of the storage and processing services. Instead, the controller The controller operated over time to learn this information about the controlled system.

[0107] In a further application, the control methods described herein were applied to improve the performance of an image processing system in an autonomous vehicle, where the methods enabled the image processing system to continue to function and react despite the failure of 75% of the system's hardware capabilities. In another example, the control methods described herein were applied to improve the performance of a Cray CX40 system running molecular dynamics simulations using the LAMMPS framework. The control methods were applied to increase the number of molecular dynamics analyses completed within a specified power budget. In this application, which illustrates the applicability of the methods herein to current and future exascale supercomputers and / or supercomputer environments, as well as others, application of the hierarchical and distributed control methods described herein resulted in an approximately 30% increase in the number of analyses completed within a power budget, as compared to state-of-the-art methods and other alternative power and / or performance management methods.

[0108] V. Method Examples FIG. 3 is a flow chart of a method 300 for controlling a system described herein. The method 300 includes detecting (302) an output of the system during a first time period. The method 300 further includes determining (304) a first performance metric of the system based on the detected output during the first time period. The method 300 further includes operating (306) a first hybrid controller to generate a first hybrid controller output based on the first performance metric. The first hybrid controller includes a first constraint, a first dynamic system, and a first learned system model. Operating the first hybrid controller to generate the first hybrid controller output includes (i) determining a first difference between the first performance metric and the first constraint, (ii) applying the determined first difference to the first dynamic system to generate a first dynamic system output, and (iii) applying the generated first dynamic output to the first learned system model to generate the first hybrid controller output. The method 300 further includes controlling (308) the system during a second time period according to the generated first hybrid controller output and detecting (310) an output of the system during a third time period. The method 300 further includes determining (312) a second performance metric of the system based on the detected output during the third time period and operating the second hybrid controller based on the first performance metric and the second performance metric to generate an updated constraint value (314). The method 300 further includes updating (316) the first constraint according to the updated constraint value, operating the updated first hybrid controller based on the second performance metric to generate a second hybrid controller output (318), and controlling (320) the system during a fourth time period according to the generated second hybrid controller output.

[0109] Method 300 may include additional elements or features. In some embodiments, the second hybrid controller may provide additional or alternative outputs to modify the operation and / or configuration of the first hybrid controller (e.g., modify dynamic parameters of the dynamic system of the first controller, change parameters, and / or initiate an update of the learned system model of the first controller). In some embodiments, the second hybrid controller may receive further inputs from the additional system and provide outputs related to the operation and / or configuration of the additional controller (e.g., hybrid controller) configured to control the additional system. Additionally or alternatively, the first hybrid controller may receive updated parameter values ​​from an additional supervisory controller (e.g., an additional supervisory hybrid controller).

[0110] 4 is a flow chart of a method 400 for controlling a system described herein. The method 400 includes sensing (402) an output of the system during a first time period. The method 400 further includes determining (404) a first performance metric of the system based on the output sensed during the first time period. The method 400 further includes operating (406) a first hybrid controller to generate a first hybrid controller output based on the first performance metric.

[0111] The first hybrid controller includes a first constraint, a first dynamic system, a first trained system model, and a first trained system update module. The first trained system update module is configured to update the first trained system model based on at least one output detected from the system and update the first trained system model according to at least one of a timing or a rate corresponding to the first trained parameter. The first dynamic system has a first dynamic parameter corresponding to an overall responsiveness of the first dynamic system. Operating the first hybrid controller to generate the first hybrid controller output includes (i) determining a first difference between the first performance metric and the first constraint, (ii) applying the determined first difference to the first dynamic system to generate the first dynamic system output, and (iii) applying the generated first dynamic output to the first trained system model to generate the first hybrid controller output.

[0112] The method 400 further includes controlling (408) the system during a second time period according to the generated first hybrid controller output and detecting (410) an output of the system during a third time period. The method 400 further includes determining (412) a second performance metric of the system based on the detected output during the third time period and operating the second hybrid controller based on the first performance metric and the second performance metric to generate a first hybrid controller update (414). The first hybrid controller update includes at least one of an updated first constraint value, an updated first dynamic parameter, or an updated first learning parameter. The method 400 further includes updating (416) the first hybrid controller according to the first hybrid controller update, operating the updated first hybrid controller based on the second performance metric to generate a second hybrid controller output (418), and controlling (420) the system during a fourth time period according to the generated second hybrid controller output.

[0113] Method 400 may include additional elements or features. In some embodiments, the second hybrid controller may further receive inputs from the additional systems and provide outputs related to the operation and / or configuration of additional controllers (e.g., hybrid controllers) configured to control the additional systems. Additionally or alternatively, the first hybrid controller may receive updated parameter values ​​from additional supervisory controllers (e.g., additional supervisory hybrid controllers).

[0114] VI. Application Examples Hybrid controllers / HOFs, whether applied to direct control, monitoring, simulation, development, or existing / legacy systems or other control methods, can fundamentally change and improve the performance and behavior of the systems in which they are placed. While HOFs are generalizable, HOFs deployed to one or more systems can change the performance and characteristics of the system. There is also a fundamental change in operation. The system now allocates available resources toward one or more goals, and the system responds to unknown or unexpected events, as well as events that were included and anticipated in previous models and methods used to "control" the system.

[0115] An example embodiment of a distributed or hierarchical HOF includes a top-level HOF implemented to control a cloud computing environment distributed across multiple data centers for the purpose of setting the overall system performance. This includes a distributed data warehouse and web servers, as well as several edge and other IoT devices that are asynchronously (and possibly unpredictably / irregularly) connected to a parent HOF(s). Some of these devices may be legacy systems that do not have a HOF controlling them, but are part of various HOF-based constraints and feedbacks.

[0116] Another exemplary embodiment includes a system with a distributed and / or hierarchical HOF, including a complex system that is part of one or more data centers or other distributed complex systems, which may communicate with and be a part of (whether as a whole, parent, child, or simply a contributor to) other complex systems that may or may not be run by the HOF. These other complex systems may include "edge", IoT, or on-premise devices such as cars, drones, phones, satellites, "connected" sensors, servers, cameras, etc. These systems may communicate synchronously and / or asynchronously with larger systems or other systems (where they may or may not be a part of, and may join or leave) in predictable and / or irregular ways.

[0117] In another exemplary embodiment, rather than changing constraints or knobs (tunable parameters) to meet a goal, the underlying model may be dynamically modified in addition to or instead of the knob(s) already exposed. This may be done for a variety of reasons, including adapting to a "new normal" due to system or external changes.

[0118] Yet another exemplary embodiment is a distributed (cloud-based) data framework such as Hadoop, where physically available memory is a key constraint and often a cause of crashes / errors. In this embodiment of the HOF, available memory is set as a "hard" target since the upper limit may not be violated by the system. Since many different behaviors and variables affect memory usage in ways that are not well represented by the models utilized in a single or multiple HOFs, instead of a dynamically changing target, or constraint, or model, the extremes of the dynamic system of the HOF(s) may be changed and may drift toward or away from the hard target, regardless of whether other abnormal behaviors (such as "new normals" or cascading failures) are detected. This may be done, for example, on a single HOF (regardless of the underlying system, distributed or not), hierarchical, monitored, or simulated one or more HOFs.

[0119] The approaches of the two embodiments described above may be extended so that a variety of "hard" and "soft" objectives may be mediated through multiple underlying models to modify the system or its performance.

[0120] The techniques described above can also be used in various ways or simultaneously in the event of various other types of external or internal failures, such as cascading failures caused by malicious actors or network effect demands.

[0121] Yet another exemplary embodiment is an ad-hoc system formed between autonomous cars and / or drones that share certain characteristics (e.g., time-space, ride-sharing network, communication provider, destination, etc.), some of which are themselves controlled by the HOF. Monitoring, targeting, as well as the integration of existing hierarchical and distributed HOFs. A novel (goal-driven) HOF-based system for control and learning from these is all part of this ad-hoc embodiment.

[0122] Yet another exemplary embodiment is a warehouse distribution system. The warehouse system itself is part of several larger complex systems (e.g. trucking, manufacturing, etc.) that are themselves often parts of larger complex systems that are interdependent. Within the warehouse, there may be many systems with competing and shared constraints and goals, such as package throughput (amount of packages arriving and departing in a given period of time), location within the warehouse (where, on which shelves), density (how closely / "tightly" the packages can be placed depending on their shape and weight). Other systems also affect the warehouse logistics system in ways that are continuous, dynamic, and not always predictable, such as robots, worker utilization, holidays, weather, etc., and all of this must be taken into account. In this embodiment, the HOF(s) controlling the warehouse can automatically optimize the tradeoffs, for example prioritizing density over high throughput when truck or air shipments are delayed.

[0123] Another exemplary embodiment is within the described warehouse environment, where a robot becomes unavailable or behaves erratically (e.g., when a human worker is laid off and damages the robot in response). Although these events are unexpected, this becomes the new normal (i.e., complete or partial loss of functionality of the damaged robot) over a period of time. In this embodiment, the supervisory HOF can modify the goal(s), constraint(s), and other resources available for the human workers and robots to achieve a short-term goal (e.g., move people and robots to the "unload" local HOF to achieve the local goal of unloading containers), and then adjust other local system and / or global goal(s) (package throughput, density, position ) to make this “new normal” last as long as it can (until the next shift or until the new robot arrives).

[0124] Another exemplary embodiment is of the same logistics complex, but this time as a global operator offering guaranteed delivery times (e.g., within 2 days) to a portion of its customers and having a network of warehouses, transportation modalities (e.g., planes, trains, trucks, drones, various third parties). Each system operates with its own goals and constraints and is also part of a larger complex system, some of which the HOF controls and some of which it does not. As weather, roads, and other conditions change, the HOF or one or more managers including the HOF (whether automated, semi-automated, human, or some other form of control) can update the overall or complex system goal (e.g., 2 days to 4 days) and all the rest of the HOF-enabled systems can then optimize their resources to the new goal(s).

[0125] The above embodiment (a complex global logistics system) includes one or more HOFs that generate updates to goals, constraints, models, extremes, hard and soft goals, "virtual" goals and constraints, etc. in response to unexpected events or occurrences. For example, a natural or man-made disaster may occur, making deliveries to some addresses temporarily impossible and significantly delaying deliveries to other addresses. Excess available resources can then be allocated to these disaster deliveries or to normal deliveries, allowing goals, constraints, and models to be variably updated to new normals and various failure modes of the previously functioning local and / or global complex system.

[0126] Another exemplary embodiment involves applying one or more HOFs to one or more systems (during development, deployment, or other stages) for testing or informational (rather than control) purposes, where the information is generated using various features of the one or more HOFs to generate various Performance and configuration parameters are collected and the resulting information can be used to inform both configuration management and target setting for the same, similar, or different systems.

[0127] One or more HOFs may have one or more performance objectives such that one or more models are dynamically selected (dynamically swapped or changed) based on operating conditions, regardless of the configuration of the system's parameters or constraints at the time of selection. In other words, the underlying models of the system are themselves available as tunable optimization parameters that serve one or more performance objectives.

[0128] One or more HOFs may control a "real-time" system (at the O / S or some other level) that has various hard constraints (which themselves may or may not change dynamically or over time depending on the needs of the user(s) and the external environment and performance capabilities of such system).

[0129] The HOF(s) may be used to test and evaluate the performance envelope of various configuration settings on an existing complex system without the underlying complex system having its own HOF(s).

[0130] One or more HOFs may be used to test and evaluate the performance envelopes of various performance targets on an existing complex system and to establish the operating and out-of-bounds envelopes for various configurations.

[0131] One or more HOFs may be used to test and evaluate the performance envelope of various performance targets on an existing complex system and to establish the operating envelope of various underlying performance models.

[0132] One or more HOFs may be used to test and evaluate the performance envelope of various performance targets on an existing complex system, which may change over time, and to establish operating envelopes for various underlying performance models.

[0133] The information gained from the above techniques may be collected in a library and deployed in one or more HOFs to be applied and leveraged in existing systems.

[0134] The information gained from the above techniques may be collected in a library and deployed as one or more HOFs for application and exploitation in systems under development.

[0135] The information collected from testing various composite systems at the HOF can be used in the design and development of new systems, including the performance envelopes and constraints of such systems.

[0136] Information obtained and collected using a monitored HOF on one or more complex systems can be applied to the design and development of new systems and models that may or may not include the original complex system or systems under monitoring.

[0137] Simulation may be used to develop a new HOF or HOFs based on information obtained and collected from testing with one or more HOFs, running a complex system including one or more HOFs, monitoring with one or more HOFs, or any or all combinations thereof.

[0138] Simulation may be used to develop new systems based on information obtained and collected from testing with one or more HOFs, running a complex system including one or more HOFs, monitoring with one or more HOFs, or any or all combinations thereof.

[0139] Simulation can be combined with monitoring HOF to improve the performance of a composite system or systems.

[0140] Simulation may be applied to improve the performance of a system or systems managed by one or more HOFs or that implement one or more HOFs.

[0141] A composite system may include multiple HOFs, one or more of which are themselves taught to set performance goals and / or configuration parameters for any time or under any conditions using artificial intelligence and / or neural networks, or other advanced learning methods.

[0142] One or more HOFs can be applied to the evaluation / discrimination network and / or the generative network of a generative adversarial network (GAN) to constrain or otherwise improve performance (e.g., speed or accuracy).

[0143] Information generated by one or more operational HOFs (e.g., performance metadata) can be applied to improve the underlying machine learning model, or a new machine learning model, or an AI model.

[0144] HOF can be used to control a composite system only when the composite system is detected to be out of range, utilizing information generated from its use to improve future and current performance of machine learning or AI models.

[0145] Multiple HOFs can be deployed on a smartphone or other device as a distributed control system in which one HOF establishes one or more key performance objectives and dynamically configures other HOFs that are uniquely related to the operational parameters of one or more applications that are currently active at that time.

[0146] Any of the disclosed methods can be applied to replace traditional control-based solutions (such as PID) with HOF whenever the traditional control solutions become obsolete or otherwise provide inferior performance.

[0147] VII. Conclusion In the above detailed description, various features and functions of the disclosed systems, devices, and methods are described with reference to the accompanying drawings. In the figures, like reference numerals typically identify similar components unless the context dictates otherwise. The exemplary embodiments described in the detailed description, figures, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter presented herein. It will be readily understood that aspects of the present disclosure, as generally described and illustrated herein, can be arranged, substituted, combined, separated, and designed in a wide variety of configurations, all of which are expressly contemplated herein.

[0148] Message flow diagrams, scenarios, and flowcharts in the figure As such and as discussed herein, each step, block, and / or communication may represent a process of information and / or a transmission of information, according to the exemplary embodiments. Alternative embodiments are included within the scope of these exemplary embodiments. In these alternative embodiments, for example, functions described as steps, blocks, transmissions, communications, requests, responses, and / or messages may be performed out of order from that shown or discussed, including substantially simultaneously or in reverse order, depending on the functionality involved. Furthermore, more or fewer steps, blocks, and / or functions may be used with any of the message flow diagrams, scenarios, and flowcharts discussed herein, and these message flow diagrams, scenarios, and flowcharts may be combined with each other, either in part or in whole.

[0149] The steps or blocks representing the processing of information may correspond to circuitry that may be configured to perform certain logical functions of the methods or techniques described herein. Alternatively or additionally, the steps or blocks representing the processing of information may correspond to a module, segment, or portion of program code (including associated data). The program code may include one or more instructions executable by a processor to implement certain logical functions or actions in a method or technique. The program code and / or associated data may be stored in any type of computer-readable medium, such as a storage device, including a disk drive, hard drive, or other storage medium.

[0150] The computer readable medium may also include non-transitory computer readable media, such as a register memory, a processor cache, and / or a computer readable medium that stores data for a short period of time, such as a random access memory (RAM). The computer readable medium may also include non-transitory computer readable media that stores program code and / or data for a longer period of time, such as a secondary or permanent long-term storage device, such as a read only memory (ROM), an optical or magnetic disk, and / or a compact disk read only memory (CD-ROM). The computer readable medium may also be any other volatile or non-volatile storage system. The computer readable medium may be considered, for example, a computer readable storage medium, or a tangible storage device.

[0151] Additionally, steps or blocks depicting one or more information transfers may correspond to information transfers between software and / or hardware modules within the same physical device, however, other information transfers may be between software and / or hardware modules in different physical devices.

[0152] While various aspects and embodiments are disclosed herein, other aspects and embodiments will be apparent to those of ordinary skill in the art. The various aspects and embodiments disclosed herein are intended to be illustrative and not limiting, with the true scope being indicated by the following claims.

Claims

1. 1. A method implemented as one or more actions performed by one or more devices, comprising: determining a first performance metric of the system based on an output of the system detected during a first time period; operating a first hybrid controller based on the first performance metric to generate a first hybrid controller output, the first hybrid controller operating according to a first constraint to operate with a first dynamic system and a first learned system model, and operating the first hybrid controller to generate the first hybrid controller output. generating a first dynamic system output as a function of a first difference between the first performance metric and the first constraint; and providing the generated first dynamic system output to the first trained system model to generate the first hybrid controller output; controlling the system during a second time period according to the generated first hybrid controller output; determining a second performance metric of the system based on an output of the system detected during a third time period; operating a second hybrid controller based on the first performance metric and the second performance metric to generate updated constraint values; updating the first constraint according to the updated constraint value; operating the first hybrid controller to generate a second hybrid controller output based on the second performance metric after updating the first constraint. , and controlling the system during a fourth time period according to the generated second hybrid controller output. A method comprising:

2. the first trained system model includes a lookup table; 2. The method of claim 1 , wherein applying the generated first dynamic system output to the first learned system model to generate the first hybrid controller output comprises selecting an entry in the lookup table based on a value of the first dynamic system output.

3. the first dynamic system being characterized by poles; and determining a confidence in the first trained system model based on the output of the system detected during the first time period; determining updates to the poles of the first dynamic system based on the determined confidence level; and updating the first dynamic system such that the poles of the updated first dynamic system have values ​​corresponding to the determined update values. The method of claim 1 , comprising:

4. 2. The method of claim 1, further comprising updating the first trained system model based on the detected outputs of the system during the first time period.

5. 5. The method of claim 4, further comprising determining to operate the second hybrid controller to update the first learned system model based on the first performance metric and the second performance metric, wherein updating the first learned system model based on the output of the system detected during the first time period is performed in response to the second hybrid controller determining to update the first learned system model.

6. updating the first trained system model based on the output of the system detected during the first time period; transmitting a representation of the output of the system detected during the first time period to a remote system; and receiving an indication of an update to the first trained system model from the remote system; The method of claim 4 , comprising:

7. 5. The method of claim 4, wherein updating the first trained system model based on the outputs of the system detected during the first time period comprises selecting a first trained system model from a set of different first trained system models.

8. The system includes a computing host having two or more computing units, each of which has a respective operational state; The method of claim 1 , wherein the first hybrid controller output comprises an indication of an operational state of each of the two or more computing units of the computing platform.

9. The method of claim 8 , wherein the at least one operating state of the computing unit of the computing host comprises a clock speed.

10. the second hybrid controller includes a second constraint, a second dynamic system, and a second trained system model; operating the second hybrid controller based on the first performance metric and the second performance metric to generate updated constraint values; applying at least one of the first performance metric or the second performance metric to the second dynamic system to generate a second dynamic system output; and applying the generated second dynamic system outputs to the second trained system model to generate the updated constraint values. The method of claim 1 , comprising:

11. determining a third performance metric of the second system based on a second output detected during the first time period; operating a third hybrid controller based on the third performance metric to generate a third hybrid controller output, the third hybrid controller operating according to a third constraint to operate with a third dynamic system and a third learned system model, and operating the third hybrid controller to generate the third hybrid controller output. generating a third dynamic system output as a function of a second difference between the third performance metric and the third constraint; and providing the generated third dynamic system output to the third trained system model to generate the third hybrid controller output; controlling the second system during the second time period according to the generated third hybrid controller output; determining a fourth performance metric of the second system based on a second output of the second system detected during a third time period; operating the second hybrid controller based on the third performance metric and the fourth performance metric to generate further updated constraint values; updating the third constraint according to the further updated constraint value; operating the third hybrid controller based on the fourth performance metric to generate a fourth hybrid controller output; and controlling the second system during the fourth time period according to the generated fourth hybrid controller output. The method of claim 10 further comprising:

12. Operating the second hybrid controller based on the first performance metric and the second performance metric to generate the updated constraint value includes: The method of claim 1 , further comprising detecting that the output of the system is in oscillation based on the first performance metric and the second performance metric.

13. Operating the second hybrid controller based on the first performance metric and the second performance metric to generate the updated constraint value includes: The method of claim 1 , comprising detecting that the output of the system responds slower than a threshold rate based on the first performance metric and the second performance metric.

14. Operating the second hybrid controller based on the first performance metric and the second performance metric to generate the updated constraint value includes: detecting that the output of the system exhibits a bias with respect to the first constraint based on the first performance metric and the second performance metric.

2. The method according to claim 1.

15. The method of claim 1 , wherein the system is configured in a vehicle and the first hybrid controller output comprises an indication of an operating state of the vehicle.

16. A processor programmed with instructions that, when executed, cause the processor to: determining a first performance metric of the system based on an output of the system detected during a first time period; operating a first hybrid controller based on the first performance metric to generate a first hybrid controller output; the first hybrid controller operates according to a first constraint to operate with a first dynamic system, a first trained system model, and a first trained system update module; the first trained system update module is configured to update the first trained system model based on at least one output detected from the system; the first trained system update module is configured to update the first trained system model according to at least one of a timing or a rate corresponding to a first training parameter; the first dynamic system having a first dynamic parameter corresponding to an overall responsiveness of the first dynamic system; operating the first hybrid controller to generate the first hybrid controller output; determining a first difference between the first performance metric and the first constraint; generating a first dynamic system output as a function of a first difference between the first performance metric and the first constraint; and providing the generated first dynamic system output to the first trained system model to generate the first hybrid controller output; controlling the system during a second time period according to the generated first hybrid controller output; determining a second performance metric of the system based on the detected outputs during a third time period; operating a second hybrid controller based on the first performance metric and the second performance metric to generate a first hybrid controller update, the first hybrid controller update including at least one of an updated first constraint value, an updated first dynamic parameter, or an updated first learning parameter; updating the first hybrid controller according to the first hybrid controller update; operating the updated first hybrid controller based on the second performance metric to generate a second hybrid controller output; and controlling the system during a fourth time period according to the generated second hybrid controller output. The processor.

17. The first dynamic system is characterized by poles, the poles being related to the first dynamic parameter, and the processor is configured to execute instructions that, when executed, cause the processor to: The first learning is performed based on the output of the system detected during the first time period. determining a confidence level for the completed system model; determining updates to the poles of the first dynamic system based on the determined confidence level; and 17. The processor of claim 16, further programmed with instructions to perform operations including updating the first dynamic parameters such that the poles of the updated first dynamic system have values ​​corresponding to the determined update values.

18. the second hybrid controller includes a second constraint, a second dynamic system, and a second trained system model; operating the second hybrid controller based on the first performance metric and the second performance metric to generate a first hybrid controller update; applying at least one of the first performance metric or the second performance metric to the second dynamic system to generate a second dynamic system output; and 20. The processor of claim 17, further comprising an operation of applying the generated second dynamic system output to the second learned system model to generate the first hybrid controller update.

19. When the instruction is executed, the processor: determining a third performance metric of the second system based on a second output detected during the first time period; and operating a third hybrid controller to generate a third hybrid controller output based on the third performance metric, the third hybrid controller including a third constraint, a third dynamic system, a third trained system model, and a third trained system update module, the third trained system update module configured to update the third trained system model based on at least one output detected from the second system, the third trained system update module configured to update the third trained system model according to at least one of a timing or a rate corresponding to a third learned parameter, the third dynamic system having a third dynamic parameter corresponding to an overall responsiveness of the third dynamic system, and operating the third hybrid controller to generate the third hybrid controller output. determining a second difference between the third performance metric and the third constraint; applying the determined second difference to the third dynamic system to generate a third dynamic system output; and generating the third hybrid controller output including applying the generated third dynamic system output to the third learned system model to generate the third hybrid controller output; controlling the second system during the second time period according to the generated third hybrid controller output; detecting the second output of the second system during the third time period; determining a fourth performance metric of the second system based on the second output detected from the second system during the third time period; operating the second hybrid controller based on the third performance metric and the fourth performance metric to generate a third hybrid controller update, the third hybrid controller update including at least one of an updated third constraint value, an updated third dynamic parameter, or an updated third learning parameter; updating the third hybrid controller according to the third hybrid controller update; operating the third hybrid controller based on the fourth performance metric to generate a fourth hybrid controller output; and controlling the second system during the fourth time period according to the generated fourth hybrid controller output.

20. The processor of claim 18, further programmed with instructions to perform operations including:

20. When the instruction is executed, the processor: operating a fourth hybrid controller based on the first performance metric and the second performance metric to generate a further first hybrid controller update, the further first hybrid controller update including at least one of an updated first constraint value, an updated first dynamic parameter, or an updated first learning parameter; Updating the first hybrid controller according to the further first hybrid controller update, wherein operating the updated first hybrid controller to generate a second hybrid controller output based on the second performance metric comprises operating the first hybrid controller updated with both the first hybrid controller update and the further first hybrid controller update to generate the second hybrid controller output.

20. The processor of claim 19, further programmed with instructions to perform operations including:

21. When the instruction is executed, the processor: operating a third hybrid controller based on the first performance metric and the second performance metric to generate a further first hybrid controller update, the further first hybrid controller update including at least one of an updated first constraint value, an updated first dynamic parameter, or an updated first learning parameter; Updating the first hybrid controller according to the further first hybrid controller update, wherein operating the updated first hybrid controller to generate a second hybrid controller output based on the second performance metric comprises operating the first hybrid controller updated with both the first hybrid controller update and the further first hybrid controller update to generate the second hybrid controller output.

21. The processor of claim 20 further programmed with instructions to perform operations including:

Images