Autonomous control of complex engineered systems

Abstract

A decided sequence of steps selected from a set of defined sequences of steps for the control system is executed to implement control by writing control variables to actuators. At each time step, it is redetermined whether to execute the decided sequence of steps or an alternative sequence of steps.

Description

Autonomous control of complex engineering systems

[0001] Cross-references to other applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 287,342, filed December 8, 2021, entitled “AUTONOMOUS CONTROL OF COMPLEXENGINEERED SYSTEMS,” which is incorporated herein by reference for all purposes. Technical Field

[0003] This application relates to autonomous control of complex engineering systems, and more specifically, to a control system for autonomous control of complex engineering systems. Background Technology

[0004] Complex engineered systems (such as those piloting air-based, water-based, space-based, or ground-based vehicles) are designed for manual / human control. Humans are prone to operator errors due to the monotony of controlling such systems, lack of experience in controlling them, and / or human impairments related to their operation. Humans may also be inefficient in their control due to poor training or deficiencies. Partial or fully autonomous control of complex engineered systems can improve their safety, reliability, quality, and / or efficiency by reducing the need for human intervention, and / or reduce the cost of operating such systems. Summary of the Invention

[0005] In a first aspect, a control system for autonomous control of complex engineering systems is disclosed. The control system includes a time sequencer configured to execute a determined sequence of steps selected from a set of step sequences defined for the control system to achieve rational control of an actuator. The control system also includes an immediate re-decisioner configured to periodically: receive sensor input reflecting the current state of the engineering system, and autonomously re-determine, based on the sensor input, whether the time sequencer should execute the determined sequence of steps or an alternative sequence of steps.

[0006] In a second aspect, a method for autonomous control of complex engineering systems is disclosed, comprising: executing a determined sequence of steps selected from a set of step sequences defined for the control system to achieve rational control of an actuator; and periodically receiving sensor inputs reflecting the current state of the engineering system; and autonomously re-determining, based on the sensor inputs, whether to execute the determined sequence of steps or an alternative sequence of steps.

[0007] In a third aspect, a computer program product comprising instructions which, when executed by at least one processor of a computer, cause the computer to perform the method according to the second aspect described above. Attached Figure Description

[0008] Various embodiments of the invention are disclosed in the following detailed description and accompanying drawings.

[0009] Figure 1 is a functional diagram illustrating a programmed computer / server system for autonomous control of a complex engineering system according to some embodiments.

[0010] Figure 2 is a block diagram illustrating an embodiment of a control system connected to a controlled system, the control system having sensor inputs and control outputs to actuators.

[0011] Figure 3A is a block diagram illustrating an embodiment of a control system configured as multiple decoupled control systems, one control system for each actuator.

[0012] Figure 3B is a block diagram illustrating an embodiment of a control system configured with less decoupling.

[0013] Figure 4A is a block diagram illustrating an embodiment of a control system with a temporal sequencer and an immediate redecider structure.

[0014] Figure 4B is a block diagram illustrating an embodiment of a control system with a partially decoupled TSIR structure.

[0015] Figure 5 is an illustration of an example timeline that determines the re-determiner of the time series at each time step.

[0016] Figure 6 illustrates a simplified tile-based implementation of a three-input / 3D system, where the inputs are labeled X-in, Y-in, and Z-in.

[0017] Figure 7 is a flowchart illustrating an embodiment of the process of selecting tiles in a time step.

[0018] Figure 8 is a diagram of the boundary tiles of a simplified example of an autonomous aircraft with the input logic object "Cruise".

[0019] Figure 9 is a block diagram illustrating an embodiment of a redeterminer implemented using multiple tile sets.

[0020] Figure 10 is an illustration of the overlapping area between take-off and climb of an autonomous aircraft.

[0021] Figure 11 is an illustration of an embodiment of a redeterminer / sequencer pair constructed as a hierarchical architecture.

[0022] Figure 12 is a diagram of a portion of a decision tree that maps inputs to associated tile labels.

[0023] Figures 13A and 13B are flowcharts illustrating an embodiment of the process of generating a decision tree from a set of tiles.

[0024] Figures 14A and 14B are flowcharts illustrating an embodiment of the process of expanding the input range.

[0025] Figure 15 is a diagram illustrating the expansion of the input dimension range.

[0026] Figure 16 is a flowchart illustrating an embodiment of the process of adding input to a tile set.

[0027] Figure 17 is a diagram of delegation used to simplify complex control.

[0028] Figure 18 illustrates a delegation-based control used to simplify complex control by a time sequencer-immediate redeterminer.

[0029] Figure 19 is a diagram of the redeterminer-sequencer timeline.

[0030] Figure 20 is a flowchart illustrating an embodiment of a process for autonomous control of complex engineering systems. Detailed Implementation

[0031] This invention can be implemented in a variety of ways, including as a process; an apparatus; a system; a material composition; a computer program product embodied on a computer-readable storage medium; and / or a processor, such as a processor configured to execute instructions stored on and / or provided by memory coupled to the processor. In this specification, these implementations or any other form of the invention may take the form of a technique. Generally, within the scope of this invention, the order of the steps of the disclosed process may be changed. Unless otherwise stated, a component such as a processor or memory described as being configured to perform a task may be implemented as a general component temporarily configured to perform that task at a given time, or manufactured as a specific component to perform that task. As used herein, the term "processor" refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

[0032] The following detailed description of one or more embodiments of the invention, together with the accompanying drawings illustrating the principles of the invention, provides for clarity. The invention has been described in conjunction with these embodiments, but is not limited to any particular embodiment. The scope of the invention is defined only by the claims, and the invention encompasses numerous alternatives, modifications, and equivalents. Numerous specific details are set forth in the following description to provide a thorough understanding of the invention. These details are provided for illustrative purposes, and the invention may be practiced according to the claims without requiring some or all of these specific details. For clarity, technical materials known in the art related to the invention have not been described in detail so as not to unnecessarily obscure the invention.

[0033] The engineering systems discussed in this paper are physical systems that are carefully designed to perform specific tasks reliably and efficiently. Each task can be categorized as being designed to achieve a logical objective, depending on the application requirements. The logical objective mentioned in this paper is one of a few discrete goals or purposes that a controlled / engineering system is designed to achieve.

[0034] For example, for an autonomous airborne vehicle, such as an aircraft, the logical objective is "transit from the current airport to airport B." It is logical in the sense that it is not a specific action or numerical control value, but rather an instruction to the control system to accomplish a task. As another example, a lower-level logical objective for an aircraft is the objective of "acquiring takeoff airspeed." It is also logical in the sense that it is not a specific numerical value, but rather associated with the objective of "being able to take off from the runway." The exact numerical value depends on many factors, including the number of passengers, cargo, runway altitude, and even temperature, and these factors may also affect the time taken to achieve the objective. The term "objective" is used in this paper because the controller is instructed to attempt to achieve what, i.e., the objective, rather than "how" to perform the task to achieve the objective.

[0035] The complex engineering systems discussed in this article react to their environment and failures in complex ways and have multiple actuators that can be controlled to achieve overall application objectives. For example, an aircraft has actuators to control the power sent to its engines, ailerons, elevators, rudders, flaps, brakes, landing gear, and possibly other actuators, all of which need to be controlled to enable the aircraft to fly properly. It must also react to changes in the environment, such as other traffic in the area, rather than simply “shutting down.” In contrast, a furnace is an engineering system designed to efficiently heat a given enclosed space. However, it is a simpler complex system because it essentially has only one control or actuator—opening or closing the furnace—and reacts to its environment solely by opening and closing.

[0036] Engineering systems are designed to have specific operating domains and are therefore typically expected to have one or more normal modes, referred to herein as normal modes, during which they perform their tasks. For example, autonomous air-based, water-based, and land-based vehicles are all designed to transport personnel and / or cargo. For each of these applications, the normal mode is cruising between the origin and destination. Similarly, a production line is designed to produce a large quantity of a given product, so the normal mode is when the production line is operating smoothly. That is, the normal mode is the state in which it performs its task without interference from abnormal conditions such as malfunctions. This normal mode is specified as part of the engineering design process.

[0037] Engineered systems are typically designed to allow for manual control, which, as discussed in this article, refers to manual operation through human-set controls. This manual operation requirement necessitates engineering to achieve predictable, stable, and accurate control. Specifically:

[0038] 1. A predictable control model exists within its defined operational domain;

[0039] 2. Within its typical operating domain, it is stable, meaning that small changes in the input do not require immediate and significant changes in the control. In other words, as mentioned in this paper, it primarily exhibits continuous behavior; and

[0040] 3. Exceptions to the normal condition (referred to as discontinuity or singularity in this document) are appropriately characterized as being relatively few in number, known to the operator / control system designer, and typically characterized by operational constraints to avoid these singularities.

[0041] Using an aircraft as an example, predictability means, for instance, that increasing the angle of attack increases the rate of climb. To illustrate stability in this example, small changes in pitch or airspeed do not require a sudden change in control. As an example of singularity in this domain, an aircraft will stall when its airspeed is low relative to its angle of attack. Stall conditions are well-known to aircraft designers and pilots and are an example of an exception to the normal course of smooth behavior because the lift from the wings decreases immediately.

[0042] It is important to note that if a large number of such singularities exist or if the singularities are unknown, then human operators will be unable to reliably operate the controlled system. In other words, the predictable and stable operating domain of the controlled system and that prediction are known, as are the domains in which its behavior is unpredictable or discontinuous.

[0043] Beyond this fundamental characteristic of predictability, there are typically several means to predict the behavior of engineered systems in greater detail, i.e., with greater accuracy. Such more accurate predictions can be based on mathematical formulas and / or computer simulations. For example, in the case of aircraft, sufficiently accurate simulations are needed to validate the design after manufacturing and later to train human pilots—the associated flight simulator. In other words, this detailed simulation is a crucial part of the engineering process and an important component of training the control system for the pilot.

[0044] Automated control of a system reduces operating costs and often improves reliability and efficiency by minimizing the need for human intervention. For a simple, familiar example, a thermostat automatically controls the heating system to open the furnace when the temperature is low and close it when the temperature is high enough, without human intervention. Even manually operating this simple system would be time-consuming and laborious, and likely inefficient.

[0045] As a more complex example, autopilot reduces the demands on the pilot during normal flight and often improves the aircraft's efficiency in flight. It can also provide better / safer operation during landings in low visibility. Fully autonomous aircraft would completely eliminate the overhead of having a pilot. However, these benefits depend on the automatic control functioning correctly in all situations within its operating domain and requiring and allowing for operator intervention when outside its operating domain.

[0046] In nontrivial systems, testing ensures correct and safe operation. Designing software for testability (sometimes referred to as test-driven development) is an established best practice. This is analogous to the hardware concept of designing for testability. For safety-critical systems such as autonomous aircraft, extensive testing is crucial. In fact, aerospace standards (such as DO-178B / C "Software Considerations in Airborne Systems and Equipment Certification") specify the tests and documentation required to obtain certification as avionics software.

[0047] The complex control systems of interest in this paper are those that support “autonomous” operation. The autonomous control system discussed here is one that can operate in a critical operational domain without intervention, enabling the system to detect when it is outside its operational domain and to intervene when it is outside that domain. For example, for aircraft, the operational domain is often referred to as its “flight envelope.”

[0048] This definition of "autonomy" differs from the dictionary definition because engineered systems are implemented to perform one or more tasks and rarely possess "free will." Therefore, it is designed to counteract human intervention, first defining the one or more tasks it is to perform. For example, an autonomous aircraft should be instructed to fly in a specific location.

[0049] Furthermore, engineered systems may have internal faults or experience external conditions they cannot handle. During the operation of a controlled system, there are situations requiring human intervention. When the system eventually goes out of its operating domain, the control system should be able to bring it into a safe state (if possible), allowing the operator time to be notified and take control, except in catastrophic situations. Advances in computer technology and reductions in the cost of such computing and sensors have made autonomous control cost-effective for many applications, and techniques for constructing software to implement such complex controls are disclosed.

[0050] Predictability and control become complex because the environment of a controlled system can change significantly. For example, in the case of an autonomous aircraft, the positions and velocities of other aircraft around the controlled system may change continuously, and their positions may be critical for the control system to react correctly. Furthermore, one or more components of the controlled system may fail or become miscalibrated. The control system should handle these situations at least to the extent that it recognizes one of them has occurred and seeks operator intervention. As is evident in the case of an autonomous aircraft, simply shutting down the controlled system in response to a failure is insufficient. Even in the context of simpler controlled systems like HVAC systems, the environment can be dynamic, and components may fail. In situations where sufficient “intelligence” in the control system can be used to reasonably handle the situation, there are risks and costs associated with simply shutting down the system.

[0051] Control theory formalizes how to control dynamic systems in engineering processes and machines. These systems are inherently continuous because the physical systems do not undergo discrete changes in behavior. For example, increasing the throttle of an aircraft continuously increases the thrust from the engine, which in turn continuously increases the aircraft's speed until the engine and aircraft reach their limits. As a field of continuous mathematics, control theory provides the foundation for the design of control systems for existing technologies and the physical devices mentioned in this paper. It relies on setting control variables to apply this control. In a closed-loop control system, it can use feedback inputs and a transfer function based on differential equations to one or more output control values ​​that are intended to control the device to achieve predictable, stable, and efficient performance and minimize overshoot / undershoot. As mentioned in existing technologies of control theory and this paper, the transfer function is a continuous function of one or more input parameters that calculates new values ​​for the control values. Simple control systems with a single input and a single output are referred to as SISO systems in existing technologies of control theory and this paper. For example, the PID (Proportional-Integral-Derivative) controller is suitable for many SISO systems.

[0052] The control systems of interest in this paper require complex control, thus responding to multiple inputs or sensors and generating multiple outputs to control the multiple actuators needed to control the controlled system. They may also require multiple inputs to monitor the behavior of the controlled system. Systems with multiple inputs and outputs are referred to as MIMO systems in existing techniques of conventional control theory and in this paper. An autopilot (e.g., a full-function autopilot) is an example of a MIMO system because it has many inputs, including airspeed, angle of attack, target trajectory, and / or altitude. It also has multiple outputs, including those going to actuators for throttle, stick position, and / or rudder angle. That is, these outputs correspond to different actuators controlling the controlled system.

[0053] The problem of designing a MIMO system can be transformed into the problem of designing multiple MISOs (referred to in this paper as multi-input single-output control systems) by decoupling—thus having a separate control system / subsystem for each actuator. Decoupling a MIMO system into an MISO system addresses the complexity of MIMO control design from a design understanding perspective. For example, a PID controller may not be suitable as a solution to the MIMO control problem because it only handles a single input and a single output. Meanwhile, traditional control theory also fails to handle MISO systems well, especially when they exhibit discontinuous behavior, as is common to all complex real-world systems.

[0054] Another problem with traditional control is that many engineering systems have singularities or discontinuities that are difficult or impossible to handle with continuous control models. As illustrated in the aircraft example above, increasing the angle of attack increases the climb rate, except when the aircraft stalls, at which point it loses all lift and its climb rate may suddenly become negative.

[0055] Even worse discontinuities can arise from unpredictable and uncontrolled elements outside the controlled system. For example, autonomous aircraft require a control system that rapidly detects and reacts to malfunctions that occur within the aircraft, potentially causing discontinuities in its flight behavior / capabilities. For instance, a sudden downdraft or wind shear could abruptly alter the aircraft's altitude or attitude. As another example, an unexpected headwind could cause the flight time to a destination to exceed the available fuel, resulting in a discontinuity in control when that threshold is crossed. In such cases, the autonomous control system needs to significantly and rapidly (and therefore discontinuously) change its behavior. For example, if an aircraft is flying over mountains or ocean when it is low on fuel, it might need to reverse direction and fly to another airport rather than simply incrementally choosing a slightly closer destination, because incrementally closer landing strips or airports with fuel sources are typically absent on mountainous / oceanic terrain.

[0056] A further improvement for achieving autonomous control is discretization. In computer implementations, the control system may at best re-evaluate the input at discrete time intervals, thus the implementation may use / depend on continuous time. Specifically, the computer control program calls the control decision process every T seconds, which takes input reflecting the current state and returns a control vector indicating the values ​​to be set for the device's control variables. Therefore, it is disclosed to modify the control formula and extend the theory to function correctly when implemented in discretized time.

[0057] Traditional work in discretization describes the transformation from continuous to discretized models for discretized time, addressing both step-invariant and ramp-invariant models. It also describes the inverse process from discretization to continuous. However, this traditional transformation still requires the use of continuous values ​​in the computation of the control decision process, and this transformation may only be applicable to linear models.

[0058] A further need arose for discretization of the input because digital computers can only handle digital inputs, i.e., discrete values. Therefore, computer control systems can use analog-to-digital converters (ADCs) that convert analog inputs into digital inputs. Traditionally, this conversion is designed to provide a digital output that approximates a continuous / analog input with an accuracy matching the capabilities of the associated sensors, and thus provides an approximation of the continuous value. ADCs can only provide samples of continuous input values ​​at discrete time intervals (sampling periods). Similarly, digital outputs are converted to analog outputs by digital-to-analog converters (DACs) because computer control can only provide digital outputs directly, while the controlled system or actuator may require analog values. It can also only update the digital output at discrete time periods (i.e., the periodicity of the computer process). Therefore, this traditional form of discrete-time with a continuous formula is referred to in this paper as “partial discretization.”

[0059] Other methods—LUTs. Discretization of control can be achieved using lookup tables (LUTs), and in some cases, this handles partially linear (PL) controlled systems. Traditionally, LUT methods have been used to replace expensive or inflexible computations with tables / empirical results that provide similar outputs. As a table in memory, it can be dynamically modified. However, applying LUT methods to determine control actions leads to an exponential explosion in space and processing costs, thus limiting its application to controlled systems where a small number of input dimensions are sufficient. Therefore, it cannot be applied to complex control systems with many different inputs and thus many dimensions.

[0060] Other methods—ML. So-called machine learning (ML) is a method that can be used to solve the challenge of determining the transfer function for more complex control systems. In this approach, the control decision process is implemented as a neural network with floating-point values ​​of weights associated with the input for each node or neuron. This neural network is then trained using a training set of input data labeled with expected behavior to determine the appropriate tuning of the weights. After sufficient training, the neural network can exhibit reasonable behavior in many situations. However, this training cannot guarantee anything about its behavior on input data that is even slightly different from the input data contained in the training set (as occurs in the real world). Furthermore, because the inputs are continuous, thoroughly testing a system built in this way is infeasible / impossible. Generating a well-trained control system can be expensive, requiring large amounts of training data, and may not necessarily be adaptable to another controlled system with slightly different behavior. For example, the control decision process of an autopilot developed for an aircraft may not be usable or even adaptable to different aircraft with different flight characteristics without retraining. Therefore, while ML can address the difficulty of explicitly programming control systems to some extent, albeit at the cost and difficulty of generating training data sets, it cannot solve the testability problem and thus cannot provide the required reliability.

[0061] Overview. This paper discloses the construction and implementation of autonomous control systems for complex engineering systems that provide sufficient control. The autonomous control system is testable, predictable, and adaptable to different instances of the controlled system. It can be implemented in the discretized reality of a digital computer system, can handle dynamic environments and certain fault conditions, and can be extended to address new requirements and functionalities without rendering previous control systems and their testing invalid. Sufficient control, as referred to herein, is control that enables the controlled system to respond quickly and appropriately to discontinuities, avoids false positives regarding discontinuities, and provides efficient and stable operation in the absence of discontinuities.

[0062] Figure 1 is a functional diagram illustrating a programmed computer / server system for autonomous control of a complex engineering system according to some embodiments. As shown, Figure 1 provides a functional diagram of a general-purpose computer system programmed to provide autonomous control of a complex engineering system according to some embodiments. As will be apparent, other computer system architectures and configurations can be used for autonomous control of complex engineering systems.

[0063] Computer system 100, which includes various subsystems as described below, includes at least one microprocessor subsystem, also referred to as a processor or central processing unit (“CPU”) 102. For example, processor 102 may be implemented by a single-chip processor or multiple cores and / or processors. In some embodiments, processor 102 is a general-purpose digital processor that controls the operation of computer system 100. Using instructions retrieved from memory 110, processor 102 controls the reception and manipulation of input data, as well as the output and display of data on output devices, such as display and graphics processing unit (GPU) 118.

[0064] Processor 102 is bidirectionally coupled to memory 110, which may include a first main storage device, typically random access memory (“RAM”), and a second main storage area, typically read-only memory (“ROM”). As is known in the art, the main storage device may be used as a general-purpose storage area and as scratch-pad memory, and may also be used to store input data and processed data. In addition to other data and instructions for processes operating on processor 102, the main memory may also store programming instructions and data in the form of data objects and text objects. Also as is known in the art, the main storage device typically includes basic operating instructions, program code, data, and objects used by processor 102 to perform its functions, such as programming instructions. For example, main storage device 110 may include any suitable computer-readable storage medium as described below, depending on whether, for example, data access needs to be bidirectional or unidirectional. For example, processor 102 may also directly and very quickly retrieve and store frequently needed data in cache memory (not shown). The processor 102 may also include a coprocessor (not shown) as an auxiliary processing component to assist the processor and / or memory 110.

[0065] Removable mass storage device 112 provides additional data storage capacity to computer system 100 and is coupled to processor 102 bidirectionally (read / write) or unidirectionally (read-only). For example, storage device 112 may also include computer-readable media such as flash memory, portable mass storage devices, holographic storage devices, magnetic devices, magneto-optical devices, optical devices, and other storage devices. Fixed mass storage device 120 may also provide additional data storage capacity, for example. An example of mass storage device 120 is an eMMC or microSD device. In one embodiment, mass storage device 120 is a solid-state drive connected via bus 114. Mass storage devices 112, 120 typically store additional programming instructions, data, etc., that are typically not actively used by processor 102. It will be appreciated that, if desired, information retained in mass storage devices 112, 120 can be incorporated into main memory 110 in a standard manner, such as as virtual memory (RAM).

[0066] In addition to providing processor 102 with access to the storage subsystem, bus 114 can also be used to provide access to other subsystems and devices. As shown, these may include a display monitor 118, a communication interface 116, a touch (or physical) keyboard 104, and one or more auxiliary input / output devices 106, including audio interfaces, sound cards, microphones, audio ports, audio input devices, audio cards, speakers, touch (or pointing) devices, and / or other subsystems as needed. Besides a touchscreen, auxiliary device 106 may also be a mouse, stylus, trackball, or tablet, and is used for interacting with a graphical user interface.

[0067] Communication interface 116 allows processor 102 to be coupled to another computer, computer network, or telecommunications network using the network connection shown. For example, through communication interface 116, processor 102 can receive information (e.g., data objects or program instructions) from or output information to another network during the execution of method / process steps. Information (typically represented as a sequence of instructions to be executed on the processor) can be received from and output to another network. An interface card or similar device and suitable software, executed / implemented by, for example, processor 102, can be used to connect computer system 100 to an external network and transfer data according to standard protocols. For example, the various process embodiments disclosed herein can be executed on processor 102 or can be executed in conjunction with a remote processor sharing a portion of the processing across a network such as the Internet, intranet, or local area network. Throughout this specification, "network" means any interconnection between computer components, including the Internet, Bluetooth, WiFi, 3G, 4G, 4G LTE, GSM, Ethernet, intranet, local area network ("LAN"), home area network ("HAN"), serial connection, parallel connection, wide area network ("WAN"), Fibre Channel, PCI / PCI-X, AGP, VLbus, PCI Express, Expresscard, Infiniband, ACCESS.bus, wireless LAN, HomePNA, fiber optic, G.hn, infrared network, satellite network, microwave network, cellular network, virtual private network ("VPN"), universal serial bus ("USB"), FireWire, serial ATA, 1-Wire, UNI / O, or any form that connects homogeneous and / or heterogeneous systems and / or groups of systems together. Additional mass storage devices, not shown, may also be connected to processor 102 via communication interface 116.

[0068] An auxiliary I / O device interface, not shown, can be used in conjunction with computer system 100. The auxiliary I / O device interface may include general-purpose and custom interfaces that allow processor 102 to send data and, more typically, receive data from other devices such as microphones, touch-sensitive displays, transducer card readers, tape readers, voice or handwriting recognition devices, biometric readers, cameras, portable mass storage devices, and other computers.

[0069] Furthermore, the various embodiments disclosed herein further relate to computer storage products having a computer-readable medium, which includes program code for performing various computer-implemented operations. A computer-readable medium is any data storage device capable of storing data that can subsequently be read by a computer system. Examples of computer-readable media include, but are not limited to, all of the media described above: flash memory media, such as NAND flash, eMMC, SD, compact flash; magnetic media, such as hard disks, floppy disks, and magnetic tapes; optical media, such as CD-ROM discs; magneto-optical media, such as optical discs; and specially configured hardware devices, such as application-specific integrated circuits (“ASICs”), programmable logic devices (“PLDs”), and ROM and RAM devices. Examples of program code include, for example, machine code generated by a compiler, or files containing higher-level code (e.g., scripts that can be executed using an interpreter).

[0070] The computer / server system shown in Figure 1 is merely an example of a computer system applicable to the various embodiments disclosed herein. Other computer systems suitable for this purpose may include additional or fewer subsystems. Furthermore, bus 114 illustrates any interconnection scheme for linking subsystems. Other computer architectures with different subsystem configurations may also be utilized.

[0071] Figure 2 is a block diagram illustrating an embodiment of a control system connected to a controlled system, the control system having sensor inputs and control outputs to actuators. In one embodiment, the control system (208) of Figure 2 may be a programmed computer / server system as shown in Figure 1.

[0072] The controlled system (201) is coupled to one or more sensors (202a), (202b)...(202k). Examples of these sensors may be an altimeter, a rudder position sensor, and an airspeed detector. The sensors (202a), (202b)...(202k) report readings via ADCs (204a), (204b)...(204k). These readings are preprocessed by a sensor preprocessing unit (206) and then provided to a control system (208), such as an autonomous control system for complex engineering systems. The control system (208) may accept intervention (209), such as higher-level instructions (e.g., “fly the aircraft to LAX”) or human intervention in the event of a rare emergency. The control system (208) outputs digital control values ​​to each of a plurality of DACs (210a), (210b)...(210m), which in turn control actuators (212a), (212b)...(212m).

[0073] Figure 3A is a block diagram illustrating an embodiment of a control system configured as multiple decoupled control systems, one control system for each actuator. In one embodiment, the diagram of Figure 3A is associated with the sensors (202a), (202b)...(202k) and actuators (212a), (212b)...(212m) of Figure 2. In one embodiment, one or more of the control systems (322a), (322b)...(322m) of Figure 3A may be a programmed computer / server system as shown in Figure 1.

[0074] As shown in Figure 3A, for k inputs, there are k sensors / ADCs / preprocessors (302a), (302b), ... (302k), which are then coupled to m control systems (322a), (322b), ... (322m). For m actuators, each actuator (342a), (342b), ... (342m) corresponds to one control system. Each control system (322a), (322b), ... (322m) periodically writes values ​​to one or more of its associated control variables to control its associated actuators (342a), (342b), ... (342m). Inputs from the sensors and sensor preprocessors (302a), (302b), ... (302k) are provided to each control system instance (322a), (322b), ... (322m) that requires such inputs.

[0075] For example, for an autonomous aircraft, an airspeed value (e.g., (302b)) can be provided to both a throttle-based control system (e.g., (322a)) and an elevator-based control system (e.g., (322b)). These common inputs allow coordination between these decoupled control systems to achieve coordinated control of the controlled system. For example, for an autonomous aircraft, the airspeed (e.g., (302b)) as a common input allows the elevator control (e.g., (322b)) to determine a safe rate of climb, and also allows the throttle control system (e.g., (322a)) to adjust the throttle to maintain speed during climb.

[0076] Figure 3B is a block diagram illustrating an embodiment of a control system configured with less decoupling. In one embodiment, the diagram of Figure 3B is associated with the sensors (202a), (202b)...(202k) and actuators (212a), (212b)...(212m) of Figure 2. In one embodiment, the control system (372) of Figure 3B may be a programmed computer / server system as shown in Figure 1.

[0077] For example, in one embodiment, the decoupling shown in FIG3A may be optional to avoid multiple queries to the decision-making mechanism. As shown in FIG3B, for k inputs, there are k sensors / ADCs / preprocessors (302a), (302b)...(302k), which are further coupled to fewer than m control systems, such as a control system (372) controlling m actuators (342a), (342b)...(342m). As shown in FIG3A, one or more control systems (372) periodically write values ​​to one or more of their associated control variables to control the associated actuators (342a), (342b)...(342m).

[0078] Figure 4A is a block diagram illustrating an embodiment of a control system with a time sequencer and an immediate redeterminer structure. In one embodiment, the structure of Figure 4A is of the type of autonomous control system shown in Figures 2 and 3A. In one embodiment, the control system (322j) of Figure 4A, or one or more of its subsystems, can be a programmed computer / server system as shown in Figure 1.

[0079] In one embodiment, the control system j (322j) from Figure 3A is configured as a time sequencer, immediate re-decisioner (TSIR) having at least one time sequencer (402) and at least one immediate re-decisioner (404). The time sequencer (402) herein refers to any system that implements a sequence of “steps” over time, which provides control values ​​at each step to control its associated actuators (210j), (212j). For example, if the step is “tilt the controlled aircraft,” then that step at least partially provides control values ​​for controlling one or more ailerons. The immediate re-decisioner (404) herein refers to any system that re-determines which time sequence the sequencer should execute at each time step, either continuing the current time sequence or switching to a different sequence. As used herein, the term re-decisioner is used to indicate that the component (404) potentially makes a new decision at each time step, thus repeatedly and immediately re-determining which sequence to execute. As mentioned herein, immediately means “as soon as new input becomes available,” i.e., each time step. Therefore, if conditions necessitate doing so, the redeterminer "redetermines" subsequent time series when there is a time series currently being executed.

[0080] As illustrated in Figure 4A, for control system j (322j), each of these layers (402), (404) receives sensor inputs (202a), (202b)...(202k) in a manner similar to Figure 2. The re-decisioner (404) uses these inputs (202a), (202b)...(202k) as the basis for its decisions. The sequencer (402) uses these inputs (202a), (202b)...(202k) to determine when it can move to the next step and how it is progressing toward achieving the goal of the current step. It also uses these inputs (202a), (202b)...(202k) to modify the control values ​​it provides to its actuators (210j), (212j) in order to attempt to achieve its input goals.

[0081] In one embodiment, the redeterminer (404) specifies a logical goal to the sequencer, which directly or indirectly selects the sequence to be executed by the time sequencer (402). Because the redeterminer (404) is outputting a logical goal, it is only responsible for determining the correct goal for the controlled system (201) at the current time based on higher-level goals / instructions / interventions (209) and its inputs (202a), (202b)...(202k) from the controlled system and / or environment. Outputting a goal rather than indicating a specific sequence gives the time sequencer (402) / immediate redeterminer (404) more flexibility to determine the best way to achieve the specified goal.

[0082] In one embodiment, the time sequencer (402) functions to implement time sequence control, that is, to execute a sequence of steps over time according to a predefined sequence, namely a sequence directly or indirectly selected by the re-determiner (404). Multiple sequences implemented by the sequencer (402) may exist. Each sequence implemented by the sequencer (402) is executed when the sequence is selected by the re-determiner (404) or is suitable for the input logic target from the re-determiner (404).

[0083] At each time step, the sequencer (402) provides corrected control values ​​directly or indirectly to the actuator (212j) as part of the execution sequence. These sequences, or some abstraction thereof, are specified to some extent as part of the engineering design of the controlled system. That is, the designer / engineer of the controlled system (201) can specify the time sequence for achieving each application objective to ensure that the controlled system (201) can achieve the required application-level objectives. For example, ensuring that a new aircraft design can land on a conventional runway requires specifications (sometimes parametric specifications) of the landing sequence of inputs (such as airspeed and altitude) and actuator outputs (such as throttle control and elevator settings).

[0084] By implementing multiple sequences, in contrast to attempting to handle different behaviors with a single sequence, the “decision-making” within the sequence is simplified. For example, an aircraft takeoff sequence only needs to focus on sequencing the aircraft through that takeoff sequence. It does not need to concern itself with decisions involving aborting takeoff, climbing to a higher altitude, and / or changing the aircraft's course. This simplification may be the same reason why these sequences are individually identified for human pilot commands. In one embodiment, the sequence is a single set of steps, where the only “decision” of the time sequencer (402) is when it can proceed from one step to the next. That is, the time sequencer (402) has three possibilities: maintain the current step; proceed to the next step; or have the re-decisioner (404) switch to a different sequence. The TSIR architecture relies on the re-decisioner (404) switching the current sequence to enable complex control behaviors, such as, for example, aircraft takeoff, takeoff abort, obstacle avoidance, and other behaviors required by the autonomous aircraft. This delegation to the re-decisioner (404) simplifies the time sequencer (402) to enable simpler control and decision-making.

[0085] In one embodiment, this timing sequence is necessary because the controlled system (201) can typically only achieve its objective over time and through multiple steps; it may be necessary to sequence the changes in control values ​​(210j), (212j) over time based on the constraints or requirements of the controlled system (201). For example, if the aircraft is stopped on the runway, it may be inefficient / damaging to the engine and / or uncomfortable for passengers to instantaneously reach full throttle in a single step when the logical target for airspeed changes to takeoff speed. Therefore, in this case, the sequencer (402) is responsible for implementing the steps of incrementally increasing throttle to avoid this problem. The sequencing is performed as a sequence of discrete steps compatible with computer implementation. That is, the processing is invoked at discrete intervals, and therefore the processing is performed at these discrete steps.

[0086] In one embodiment, the immediate re-decisioner (404) functions to: quickly re-determine the time series to be executed; select a given sequence from the set of sequences implemented by the sequencer (402); and / or instruct its selection as input to the sequencer (402). The re-decisioner (404) typically re-evaluates its decision at each re-decisioner time step (404). Thus, if conditions in the controlled system or its environment (201) change as indicated by its inputs (202a), (202b)...(202k), it can produce different decisions and thus select different sequences for the sequencer (402) to execute with very little delay.

[0087] For example, the aircraft's autonomous pilot can reassess its decisions every 50 milliseconds, allowing it to detect and react to changes in the available time of the corrected inputs (202a), (202b), ... (202k). In this example, the re-decisioner (404) in the autonomous pilot can detect a problem with the aircraft during the takeoff sequence, and / or a higher-level intervention (209) can be altered to indicate aborting takeoff. The re-decisioner (404) can then change the logical objective provided to the sequencer (402). The sequencer (402) is then instructed to immediately change to the sequence of steps associated with the new logical objective. That is, the sequencer (402) supports preemptible sequences, thus replacing the current sequence in execution with a different sequence when so instructed by the re-decisioner (404). The sequencer (402) can also restart the sequence if some parameters of the current sequence change significantly. For example, if the target airport that the aircraft is cruising to changes, the sequencer (402) can restart the cruise sequence even if the logical target of the cruise has not changed.

[0088] However, if the re-determiner (404) does not change the logical objective or associated parameters, the sequencer (402) continues the sequence at the current step in the current sequence, thus typically executing multiple subsequent time steps to achieve its logical objective. That is, it continues from the current step it is processing without interruption. This continuation in the case of the current time series is important for efficient and stable operation. Frequent responses to phantom discontinuities (such as false positives) would lead to unnecessary abrupt changes in control, which would be an excessive stress on the controlled system (201) and its processing. Furthermore, even unnecessarily restarting the time series can lead to unnecessary control variability and / or additional control processing.

[0089] Figure 4B is a block diagram illustrating an embodiment of a control system with a partially decoupled TSIR architecture. In one embodiment, the architecture of Figure 4B is of the type of autonomous control system shown in Figures 2 and 3B. In one embodiment, the control system (372) of Figure 4B, or one or more subsystems thereof, can be a programmed computer / server system as shown in Figure 1.

[0090] As shown in Figure 4B, instead of complete decoupling as shown in Figure 4A, a single redeterminer module (404) can be used to provide results to multiple timing sequencer modules (402a), (402b)...(402m). This decoupling is optional because two or more actuators can be controlled by a single redeterminer.

[0091] In contrast to Figure 4A, the system in Figure 4B has a control system (shown here as a single control system (372)) with fewer actuators (here shown as m actuators (212a), (212b)...(212m)). The control system (372) has multiple time sequencer modules (402a), (402b)...(402m) coupled to the corresponding DACs (210a), (210b)...(210m), which are then coupled to the corresponding actuators (212a), (212b)...(212m).

[0092] For Figure 4A, complete decoupling can have the advantage of avoiding multiple decisions in a single lookup in the decision tree for the re-determiner module (404) in each control system (322j). In the embodiment illustrated in Figure 4B, this problem can be addressed by mapping the decision results to a vector of decision values, one entry for each actuator (212a), (212b)...(212m).

[0093] In one embodiment, the system of Figure 4B may have an advantage in certain domains where each individual control agent may want some indication of what other control agents might be doing. Based on what other control agents are doing, there may be dependencies between different control agents in terms of the effectiveness with which each control agent is working toward a goal. For example, the elevator agent might want to know whether the throttle agent is increasing the throttle.

[0094] In one embodiment, the extended redeterminer provides each control agent with a reasonable range of parameters. For example, the throttle can be changed by ±10%, and / or the aileron can be adjusted by +10 / -10 degrees. The timing sequencer can refine the actual throttle value to be used based on a simulation of the controlled system over a relevant time period (e.g., by simulating the next five seconds). Therefore, the redeterminer provides a limited range of values ​​to try, so the timing sequencer only needs to try a few values, rather than all possible values. The simulation can be reused from dynamic simulations used by the designer of the engineering system of interest. Thus, each timing sequencer can use this simulation and / or refinements of that simulation without having to carefully program the parameters for each time series.

[0095] For many different parameter choices, simulation methods can encounter significant costs in running the simulation. Therefore, limiting the range of values ​​to be tried is a practical improvement to reduce the cost of simulation at each time step, which is achieved in part by providing a range of parameters for each actuator to utilize in the simulation. For example, if the logical goal is climb, but a high positive vertical velocity already exists, the ranges for throttle and elevator are more or less the same as they are now. However, if there is no vertical velocity or a negative vertical velocity exists, the ranges for throttle and elevator are much larger.

[0096] In any case, the range specified for each actuator is much smaller than the total possible range for each control variable, so each control agent can simulate far fewer possibilities. For example, in the latter case, it will at least not simulate a decrease in throttle or elevator. The throttle agent can use the midpoint of the range provided to other control agents, or the value that each other agent set its control variable to in a previous time step, if that value is included in the current range. For example, it can be assumed that the elevator agent is setting the elevator to the same position as in the last time step, or to the midpoint of its new range. Therefore, each control agent does not have to run the cross product of the simulation across all control value ranges, but rather for the possible values ​​within its own range.

[0097] In this approach, the time sequencer still provides the time series, and the redeterminer still provides an indication of the logical objective. The redeterminer has decision results mapped to a range vector of control variables, not just the logical objective. Moreover, using this range vector, a single redeterminer for all control agents may actually be sufficient.

[0098] Figure 5 is an illustration of an example timeline that determines the re-determiner of the time series at each time step. In one embodiment, the illustration in Figure 5 is performed by a programmed computer / server system as shown in Figure 1.

[0099] As shown in Figure 5, the immediate re-decisioner (404) of Figure 4A or Figure 4B re-evaluates its decision at each time step, thereby allowing the time sequencer (402) of Figure 4A or the time sequencer (402m) of Figure 4B to: continue the existing sequence if the re-evaluation does not change its decision—that is, the same logical goal or sequence of the sequencer (402) / (402m); or immediately change the sequence being executed if the re-decisioner (404) does change its output logical goal.

[0100] The improvement in control simplicity is an advantage of delegating control functions to the decision module (404), which quickly re-determines while delegating simple decisions involving time sequencing to a set of preemptible sequences, where the sequencer (402) / (402m) "decision" typically involves simply deciding when a particular sequence in execution should proceed to the next step in that sequence. That is, the decision conditions are specific to a particular sequence and are therefore further simplified. Thus, complex decision-making is located in the re-determiner (404), but time sequencing and fine-grained / micro-decision making are offloaded to the time sequencer (402) / (402m).

[0101] As described herein, preemptibility means pausing the current task / sequence and switching to execute a different task / sequence. This typically requires saving the state associated with the current task so that it can be restored later. However, as discussed herein, the sequence-related state is primarily the state of the controlled system (201) of Figure 2 reported by sensors (202a), (202b)...(202k), and therefore it is not sequence-specific and is preserved independently of the sequence.

[0102] Furthermore, preempted sequences are rarely restored. For example, if an autonomous aircraft is performing takeoff when an obstacle appears on the runway, causing it to preempt the sequence to begin aborting takeoff, it's unnecessary to restore the takeoff sequence at the same point as when it was preempted, as the aircraft is not physically in the same state. If the obstacle is immediately cleared, causing the aircraft to reconsider restoring the takeoff sequence, the sequence can be reconsidered based on the aircraft's current state to restart, which will be slower and further down the runway than a previous instance of the takeoff sequence. Therefore, even with a considerable airspeed, the available takeoff sequences are improved with a sufficiently large operational domain for invocation. Generally, each sequence can be designed to take over in a wide range of scenarios to support its immediate reconsideration, which might abruptly invoke the sequence in different scenarios. Therefore, designing a sequence to be preemptible primarily means designing it to be invoked across a reasonably broad and specified operational domain, and not leaving a sequence-specific residual state upon preemption.

[0103] The re-decisioner (404) effectively characterizes a scenario where the current sequence is selected based on its decision criteria. The decision criteria then provide a basis for the re-decisioner (404) to determine at the next time step whether the new situation at that next time step matches the characterization and therefore the current sequence should be allowed to continue, or whether the re-decisioner (404) needs to select a different sequence based on the changes in the new situation.

[0104] The logical objective output by the redeterminer (404) provides a simple criterion for determining whether a new scenario fits the representation that forms the basis of previous choices for the current sequence: if the same logical objective is the same as in previous time steps, it is the same representation; otherwise, it is not the same representation. In contrast, traditional sequence-based control lacks a redetermination mechanism, and conventional AI (artificial intelligence) planning methods have almost no representation of previous input scenarios, thus lacking clear criteria for determining whether previously generated plans will still be effective at the next time step or in general subsequent time steps.

[0105] In one embodiment, the redecisioner (404) overrides higher-level interventions (209) that it receives as inputs. For example, for an autonomous aircraft, if the redecisioner (404) detects an obstacle on the runway or a problem with the aircraft via one of its inputs, it may decide not to take off even when manually instructed / authorized. It may also decide not to take off again, given the short length of that particular runway or excessive cargo weight, if there is insufficient time to reach takeoff speed.

[0106] As described in the prior art and this document regarding manual action selection, an action is selected at each time step by determining a logical objective, mapping that logical objective to a time series in a sequencer (402) / (402m), and then executing the action at the current or next step of that series (which could be the first step if a new series is selected). The action can simply update the control value for its associated actuator (212j). This is efficiently achieved by simply dividing the process between an immediate re-determiner (404) and the time sequencer (402) / (402m).

[0107] The re-determiner (404) is often the most challenging aspect to implement because the time sequencer (402) / (402m) is typically specified as part of the system's engineering and human operator training, especially when the system can also be adapted for manual operation. Traditional manual action selection does not address two issues: the need for immediate response to significant changes in input; and the need for a smooth time series of control execution.

[0108] As stated earlier, the timing sequencer (402) / (402m) is generally simpler to implement because it is implementing a control sequence that may have been specified as part of the engineering design of the controlled system. For example, an aircraft training manual may describe how to land the aircraft based on feedback from the engineer who built it. The timing sequencer (402) / (402m) takes one of a relatively small number of logical objectives, which designate one of a plurality of sequences, as input and proceeds to execute the sequence associated with that input logical objective.

[0109] For example, an autonomous aircraft may have the following logical objectives: parking, taxiing, takeoff, climb, descent, alignment for landing, and landing. Each can be encoded as a coroutine / sequence, i.e., a program that executes the steps and then waits / pauses as needed for a specified condition to be true before proceeding to the next step. For example, a takeoff coroutine for elevator control might wait for the aircraft to reach takeoff speed and then raise the elevator to increase pitch to actually take off. More practically, the coroutine will wait for a period of time and recheck the airspeed to determine if the aircraft has made sufficient progress toward the required airspeed. In one embodiment, the sequencer (402) / (402m) is responsible for both: implementing the sequence of actions over time; and detecting whether the sequence has not progressed sufficiently over time to meet the time requirements of the input logical objective. The waiting condition can be relatively simple to implement, as it typically only determines whether the condition for proceeding from a particular current step to the next step is met. Given a change in the state of the controlled system and environment, it does not need to care whether the current sequence is still the correct time series to be executed, which is instead delegated to the re-determiner (404).

[0110] In one embodiment, the sequencer (402) / (402m) has a fixed set of procedures or processing sequences for verification, thus making the test feasible. In other words, engineered systems (such as aircraft) are designed with a relatively small number of logical objectives, and therefore, a small, fixed number of coordinators are sufficient. It should be noted that if an engineered system does not have the relatively small number of sequences / coordinators required to control it, it would be challenging for a typical human operator to know all the sequences and be able to select and execute the correct sequence as needed; therefore, the disclosed techniques can be used in systems driven by typical human operators.

[0111] In one embodiment, the coordinator sequence dynamically determines when to execute the next step, but the “decision” about which step is typically simple by design. For example, if the coordinator is iterating, incrementally increasing the target airspeed at each iteration, the “decision” to terminate the iteration is a simple “if” condition that the aircraft reaches the target speed and a condition to check if the sequence has taken too long to reach the target speed. If more complex decisions are required, a re-decisioner (404) is expected to handle the re-decision and thus change one or more current sequences being executed. For example, if the aircraft fails to increase its airspeed in time to reach takeoff speed, the re-decisioner (404) will make a decision to select an alternative sequence based on its input. In this example, the airspeed sequencer (402) / (402m) could simply indicate that it did not reach the target airspeed as quickly as expected, as feedback to a higher level. Due to this deferral of the decision to the re-decisioner (404), the sequencer (402) / (402m) is relatively easier to implement.

[0112] An automatic control system using a TSIR architecture is disclosed, wherein complexity is divided between an immediate re-decisioner (404) and a time sequencer (402) / (402m). In one embodiment, the key challenge is implementing the re-decisioner (404) because each decision it makes is based on a complex set of inputs (202a), (202b)...(202k) and is important for making the correct decision.

[0113] For example, when an aircraft has problems, the decision to abort or proceed with takeoff depends on many factors, such as current airspeed, remaining distance / time on the runway, and / or the nature of the detected mechanical problem and its impact on the aircraft's airworthiness. Making the wrong decision can be extremely dangerous.

[0114] In one embodiment, implementing the re-decisioner (404) involves designing each decision to map onto efficient, predictable, and stable control of the controlled continuous system, while still remaining highly responsive to changing conditions. Specifically, the re-decisioner (404) can allow the current time series to continue at any appropriate time for efficient stable control, and also immediately induce time series changes if necessary. For example, if a previous decision of the autonomous autopilot is no longer consistent with the constraints, it may react to changing conditions. However, it should be designed not to oscillate rapidly between deciding to abort and continuing takeoff.

[0115] In one embodiment, implementing the redecimalizer (404) includes allowing the control system to evolve to expand its operating domain. That is, as a software module, each redecimalizer (404) is designed to evolve or expand efficiently or safely across multiple versions to handle increasing amounts of input and thus make more complex decisions. For example, an autonomous autopilot might need to evolve / expand to handle new inputs indicating cabin pressure loss. The method used to do this needs to be implemented without compromising the generally well-tested processing of the situation it is currently dealing with.

[0116] As illustrated in the example in Figure 5, the re-determiner (404) can select one of a plurality of sequences for the time sequencer (402) / (402m), where each type of sequence is shown in Figure 5 with a different cross-shading pattern. For the aircraft example following Figure 5, the diagonal " / / / " pattern can indicate the takeoff sequence for step (502), and the vertical "|||" pattern can indicate the collision avoidance sequence step (504), which, when resolved, returns the aircraft to the takeoff sequence (506). Once the aircraft has successfully taken off, it can proceed to the cruise sequence (508) and then to the landing sequence (510). After landing, the aircraft can return to takeoff (512) and then cruise again (514). In one embodiment, the immediate re-determiner (404) knows the time taken by the time sequencer (402) / (402m) to complete a given sequence.

[0117] Tile-based implementation. Figure 6 illustrates a simplified tile-based implementation of a three-input / 3D system, where the inputs are labeled X-in, Y-in, and Z-in.

[0118] In one embodiment, a re-determiner (404) is implemented using an associated set of predefined K-dimensional tiles and a decision process. As described herein, a tile refers to a K-dimensional closed shape such that any point in K-dimensional space is either inside or outside the shape. The decision process takes a set of K input values ​​as input, which indicate the state of the controlled system at the current time, and matches those values ​​with one or more predefined tiles from the associated set of K-dimensional tiles. Specifically, if the K-dimensional point corresponding to those K input values ​​is contained in T, then those K input values ​​match tile T, for example, if the tile is constrained to a hyperrectangle, the first value is within the first dimension of the tile, the second value is within the second dimension given the first dimension, and so on, up to the Kth input value.

[0119] Each tile can be labeled to indicate an associated decision value that indicates a logical objective. The labels of matching tiles can be used to indicate the logical objective decision and potentially some of its parameters to provide a given sequencer, which then executes the sequence corresponding to that objective, thereby starting a new sequence or continuing an existing sequence (if the sequence is correct).

[0120] In this tile-based approach, each input can have a permissible total range. For example, for an autonomous aircraft, the airspeed range could be 0 to 500 km / h, the altitude range could be 0 to 15,000 meters, and the roll range could be 0 to 359 degrees. The K-dimensional space of the input is the cross product of these permissible total ranges.

[0121] Continuing the previous example, the input combination [200, 1500, 0]—where the triplet is [airspeed, altitude, roll]—is a point in this input space, which is a normal combination for a cruise aircraft. However, other input combinations (such as [0, 14000, 90]) are also included in this input space. This case of zero airspeed at high altitude and perpendicular to the ground is completely outside the aircraft's normal operating domain (also referred to herein as the flight envelope). However, an aircraft can have zero airspeed on the ground, can fly at high altitudes in some cases, or can be perpendicular to the ground when making a sharp descent turn. That is, each of these extreme values ​​may occur individually within the normal operating domain, but their combinations are outside the normal operating domain.

[0122] The tile set (referred to in this paper as the set of tiles used by the redeterminer) covers the K-dimensional input space. Specifically, each combination of input values ​​appearing in the cross product of the total allowed range of the input matches the corresponding tile.

[0123] In one embodiment, each tile has a tag, which is typically static at least within the decision cycle, thus limiting the set of possible tags to a small set of discrete values ​​or identifiers, i.e., an enumeration of sequences / parameters. For example, a tile tag for airspeed could indicate one of five values ​​corresponding to a target / sequence: stop, taxi, takeoff, cruise, and fast cruise. That is, a sequence is identified by specifying its associated target.

[0124] Figure 6 illustrates four tiles in a simple three-input / 3D system labeled X-in, Y-in, and Z-in, where K equals 3. When the input is within the boundaries of the X, Y, and Z values ​​of tile 1, tile 1 (602) generates output label -1. When the input is within the boundaries of the X, Y, and Z values ​​of tile 2, tile 2 (604) generates output label -2. When the input is within the boundaries of the X, Y, and Z values ​​of tile 3, tile 3 (606) generates output label -3. When the input is within the boundaries of the X, Y, and Z values ​​of tile 4, tile 4 (608) generates output label -4.

[0125] As shown in Figure 6, these tiles may have different lengths, widths, and heights, or be K-dimensional equivalents. For example, in Figure 6, tile 4 (608) has a larger X dimension than the other tiles. There exist tiles for every possible combination of input values ​​(i.e., X-in, Y-in, and Z-in in this figure). That is, the tiles “cover” the K-dimensional input space, and therefore there exist tiles and tile labels for every valid set of input values.

[0126] As indicated in this document, parentheses are used within a range to indicate an open interval that is strictly less than the maximum value. Square brackets are used within a range to indicate a closed interval that is greater than or equal to the minimum value.

[0127] To further illustrate, consider an example of a decision module for controlling the roll of a simple aircraft, with the following inputs: expected maneuver, airspeed, and altitude. These three inputs define a three-dimensional input space. Assume the integer / discrete input value indicating a rapid right turn is 13. As a specific tile definition, if there are airspeed thresholds at 150 km / h and 200 km / h and altitude thresholds at 200 feet and 500 feet, then there exist three-dimensional tiles corresponding to the following dimensions: maneuver [13, 14), airspeed [150, 200), and altitude [200, 500]. This tile is labeled with the roll output value associated with it, such as "hard roll". Thus, if the input is, for example, (13, 175, 375), these inputs are contained within the boundaries of this tile, and the decision module outputs a logical target of "hard roll", which might be mapped to 30 degrees for the aircraft, suitable for the requested rapid right turn indicated by the expected maneuver value (13 in this example). Then, the sequencer (402) / (402m) calls the sequence to achieve the specified roll angle within the appropriate time period.

[0128] There exist combinations of input values ​​that may not be part of the normal operating domain of the controlled system. For example, for an autonomous aircraft instructed to cruise via a higher-level target (209), there exists a certain trade-off between airspeed and altitude. Specifically, the higher the airspeed, the easier it is for the controlled system to reach the appropriate cruise altitude from a low altitude. Similarly, the higher the altitude, the easier it is for the aircraft to reach the appropriate airspeed from a very low airspeed. For example, if the aircraft is at a high altitude, the control system can simply orient the aircraft nose-down to obtain significant speed. On the other hand, it is unreasonable to expect a "cruise" sequence to handle extreme and dangerous situations, such as when both airspeed and altitude are low or when airspeed is very high at a very low altitude. Therefore, boundary tiles can be designed to match input situations that are outside the situations that the sequence can handle / outside its operating domain. As described herein, boundary tiles are tiles that indicate a portion of the K-dimensional input space outside the operating domain of the controlled system, and are provided with decision labels to ensure the existence of a match for every possible combination of input values. These tiles are labeled with instructions to seek intervention—either by a separate emergency sequence or by a human operator—and thus typically log erroneous indications. These boundary tiles explicitly define when an input indicates that the controlled system appears to be outside the operational domain of the input's logical target.

[0129] Figure 7 is a flowchart illustrating an embodiment of the process of selecting tiles in a time step. In one embodiment, the process of Figure 7 is executed by the re-decisioner / decision module (404) of Figures 4A and 4B. In another embodiment, the process of Figure 7 is executed by a programmed computer / server system as shown in Figure 1.

[0130] As shown in Figure 7, the processing may be feasible. In step (702), an input vector of values ​​is accepted. In step (704), the input vector of values ​​is matched to tiles containing the input vector. This matching step can be performed by iterating over all tiles, checking whether the K-dimensional point defined by the input value falls within a tile, and if so, returning the associated label. In one embodiment, an optimization method is used, as described below using a traditional decision tree. In step (706), the label associated with the matched tile is output.

[0131] In one embodiment, implementing a re-determiner involves defining an associated set of tiles, namely: dimensions; tile boundaries; and a label associated with each tile, such that there exists an actual number to store and an efficient means of matching them, as well as a limit on the number of matches required. When limiting the number of matches, a decision tree or an equivalent if-then-else structure requires a single decision for each match.

[0132] In one embodiment, the set of K-dimensional tiles for a given actuator is specified by a threshold for each input dimension (i.e., parameter). That is, each tile is a K-dimensional hyperrectangular / orthotope. Specifically, each tile is specified by its minimum and maximum values ​​for each dimension. For example, if airspeed is the input dimension / parameter, and thresholds of 0, 3, 100, 200, and 300 KMPH are included, then values ​​ranging from 0 to 3, 3 to 100, 100 to 200, 200 to 300, and 300 and higher define the tile boundaries in the associated tile set. This example includes a range of 0 to 3 to handle cases where the aircraft is essentially stationary.

[0133] As another example, a specific tile T can have its angle of attack dimension specified as a minimum of 5 degrees and a maximum of 21 degrees. Therefore, if the angle of attack input value is greater than or equal to 5 degrees and less than 21 degrees, and other input values ​​are contained within the corresponding dimension of tile T, then tile T is a match. The output of the decision module is a label associated with tile T. This label indicates the logical objective or sequence to be performed to one or more associated sequencers.

[0134] By using these thresholds, the resulting tile set covers all possible combinations of input values. Continuing the example above, the legal airspeed range is from 0 to some large value, such as 500 km / h used earlier. The first range starts from 0, and each range is necessarily continuous based on the threshold, with the final range between the maximum allowed value and the maximum threshold for that input type. Given that this is true for each input dimension, any legal combination of input values ​​is mapped to a tile. Here, "legal" in this paper refers to the values ​​allowed in the input of that input source.

[0135] In one embodiment, one dimension of the tiles in the tile set is time, which indicates for each tile the amount of time available to make the decision associated with that tile a good choice. For example, for an autonomous aircraft autopilot, if the time required to reach takeoff speed exceeds the time before reaching the end of the runway, then re-deciding on takeoff is meaningless. Due to the multidimensional aspects of the tiles, the time requirement for a given decision / tile is specific to the input values ​​within a specific range defined by that tile. For example, the time required to reach the takeoff speed indicated by the tile is defined by airspeed and acceleration. This addresses the aspect that the time requirement depends on these and other factors.

[0136] In one embodiment, the input dimension is treated as continuous, even if its values ​​are discrete or categorical, by treating any value greater than or equal to an integer value I but less than I+1 as corresponding to I. That is, integer values ​​are treated as the lower bound of a continuous interval. For example, suppose an enumeration of six airspeed values: stop, slow taxi, fast taxi, takeoff speed, landing speed, and cruise speed are represented as values ​​0, 1, 2, 3, 4, and 5, respectively. If this enumeration is provided as input to the decision module, i.e., the target airspeed, any value in the range 3.0 to 3.9999 is mapped to takeoff speed. Thus, in the tile set for airspeed, there exist tiles with dimensions corresponding to a logical target in the range 3.0 to 3.9999. Conversely, for process variables for airspeed, the input is actually discretized by mapping the actual airspeed to one of several ranges defined according to thresholds that elicit different decisions. As mentioned herein and in the art, process variables refer to a measure of a quantity controlled by a particular actuator.

[0137] Figure 8 is a diagram of the boundary tiles for a simplified example of an autonomous aircraft with the input logical object "Cruise". Figure 8 illustrates the boundary between the cruise operating domain and the area outside the operating domain, taking into account both altitude and airspeed. Curve (802) indicates the actual boundary calculated using a continuous function. The transparent area (804) is covered with tiles indicating cruise.

[0138] As shown in Figure 8, higher airspeeds allow cruise logic targets to take over at lower altitudes, while lower airspeeds require higher altitudes. Within the full tile set, very low airspeed tiles (806) may require intervention by selecting a "stall handling" sequence. Similarly, within the full tile set, very low altitudes at high airspeed tiles (808) may require intervention by selecting an "emergency climb" sequence.

[0139] As shown in Figure 8, the shaded tiles indicate the boundary tiles that approximate curve (802). As rectangular tiles, they can provide an approximation of the curve. As an approximation, there are airspeed and altitude values ​​that the aircraft and cruise sequences can actually handle but are not permitted by the tile-based method. These areas are indicated by the regions in which the tiles are located above the curve, for example, the region shown by element (810).

[0140] As shown in Figure 8, there exist regions of airspeed and altitude where switching to cruise is not permitted under a tile-based decision-making approach, but would be practically acceptable for the aircraft. It is important to note that this could lead to suboptimal control in some senses. However, it is worth noting that the controlled system improves when operating away from these boundary conditions, especially in dynamic environments with a persistent risk of component failure or miscalibration. Furthermore, it is important to note the relatively small efficiency difference between the sequence chosen in the boundary conditions and the chosen cruise logical objective. For example, at high altitude and low airspeed, both stall handling and cruise sequences will cause the aircraft's nose to tilt downwards to reduce altitude and gain airspeed. It's just that the stall handling sequence is expected to do this more aggressively.

[0141] In one embodiment, if a better approximation of the actual continuous curve becomes necessary, finer-grained boundary tiles are used to approximate the curve more closely. However, it should be noted that finer-grained tiles increase the cost of storing and matching the tiles, thus requiring a trade-off between these costs and the benefits of more accurate control.

[0142] Tile set reduction. The disclosed tile set-based method provides predictable / deterministic behavior because the immediate re-determiner makes the same decision whenever the input falls within the parameters of a given tile. Making the tiles as large as possible in K-dimensional space, while providing acceptable control, stabilizes control over most of the range of input values ​​corresponding to the vertices of the selected tiles. That is, the immediate re-determiner reselects the same logical target most of the time, and selects the same logical object if this is the correct decision to be made. Exponential tile explosion is avoided compared to traditional lookup table controller methods. In decoupled systems such as those described in Figures 3A and 4A, each tile set produces a single output, so they can be compiled into a decision tree for efficient lookup to determine new logical targets. In partially decoupled systems such as those described in Figures 3B and 4B, decision trees can be used where decision outcomes are mapped to vectors of decision values, one entry per actuator.

[0143] In general, for an immediate re-determiner module for a given actuator, there are N K-dimensional tiles and M possible tile labels. Without tile set reduction, the number of tiles used for assignment, storage, and matching can be even larger. For example, for a typical 16-bit ADC, 2-bit range, and I inputs, the number of input value combinations is 2^32. I×14 For values ​​of I greater than three, 2 I×14The memory space used to store the tiles and the matching overhead for matching the inputs to the tiles are both larger than practically possible. It's important to note that, as mentioned herein, a "2-bit range" means each range corresponds to four distinct numeric values. In one embodiment, the memory space and matching overhead can be reduced by using a per-input combination threshold and by delegating to the LUT controller on the sequencer.

[0144] This paper discloses a method to reduce the number of tiles and tile tags required and to facilitate the generation of a reduced tile set. The disclosed technique reduces the number of tiles required for each decision module while still providing sufficient control, such as safe and efficient control of continuously controlled systems. The decoupling described in Figures 3A and 4A reduces the number of inputs, outputs, and tile set size by focusing on inputs and input thresholds and the output logic objectives associated with the actuators. This decoupling also allows a single output value to be sufficient, thus allowing for more efficient tile matching. An instance of the TSIR structure for each actuator is referred to as a tactical control module (TCM), where "tactical" as used herein means that the control module directly controls the actuator. In decoupled control, there is one TCM for each actuator in the control system for each actuator required by the controlled system. It should be noted that this decoupling differs from some traditional control partitions, such as those by speed (both horizontal and vertical), where associated control modules control multiple actuators.

[0145] In one embodiment, the immediate redeterminer only needs to redetermine the logical objective and thus, directly or indirectly, the sequence to be executed, and therefore its parameters can be simplified to a tile set. In contrast, it's important to note that if a tile set is used to determine the next control value to be written to the actuator, it may require more inputs and finer-grained thresholds, significantly increasing the number of tiles.

[0146] Several additional techniques can be used to reduce the size and complexity of each tile set, including:

[0147] 1. Dynamic adaptive time sequencing;

[0148] 2. Divide the control decision-making mechanism based on different logical objectives to reduce the complexity of tile sets;

[0149] 3. Decision-making based on the current state;

[0150] 4. Delegate the damping out of potential rapid oscillations in control decisions to the pre-matching step;

[0151] 5. Delegate to input preprocessing to generate the "need-to-know" inputs for decision-making;

[0152] 6. By recognizing that operational constraints disallow certain input combinations, default to these combinations; and / or

[0153] 7. Use a decision tree representation for each tile set to reduce space and search costs.

[0154] This technique can be used to further reduce the number of input value combinations, i.e., by broadening the range of inputs that need to be individually labeled and reducing the number of output tile labels required, and thus reducing the number of tiles required per control module, and consequently the number of test cases. They also facilitate first-match semantics, which then allows for efficient mapping of inputs to matching tiles.

[0155] Dynamic adaptive time sequencing. In one embodiment, the time sequencer (such as (402) / (402m) in Figures 4A / 4B respectively) implements dynamic adaptive time sequencing. As mentioned herein, dynamic adaptive time sequencing means that the sequencer (402) / (402m) can change specific parameters and states of the time series to adapt to specific circumstances before and during the execution of the sequence.

[0156] For example, for an autonomous aircraft, the immediate re-decisioner (404) of Figure 4A / 4B can decide to fly from airport A to airport B and pass this logical goal to the sequencer (402) / (402m). The sequencer (402) / (402m) then maps the goal to a sequence of flights passing through waypoints, where each step or subsequence of steps is from one waypoint to the next. The sequencer (402) / (402m) can calculate what these waypoints are based on a routing method to find the shortest path that meets specific application criteria. This method could be to compute various paths between airport A and airport B and select the one with the lower cost. Therefore, the “decision” about which path to take can be feasible and easily implemented.

[0157] Using dynamic adaptive time sequencing, the sequencer only needs to implement a single sequence for each logical target. Continuing with the example above, there might be a single cruise sequence that progresses through a parameterized list of intermediate points. This contrasts with less dynamic adaptive sequencing, which must implement sequences specific to each pair of origin and destination airports. Therefore, dynamic adaptive time sequencing may require far fewer sequences. As a result of the redeterminer (404), it needs to make decisions between fewer sequences and thus requires fewer tiles. In particular, in this example, the current and destination airports can be parameters for the sequencer (402) / (402m), which do not constitute an additional dimension to the tile set used by the redeterminer (404).

[0158] As another example of an autonomous aircraft, when receiving an input logic objective to change course, the controller portion for controlling roll can pre-calculate the aileron angles to be taken at each step in a series of steps to perform a turn in the aircraft, and then execute each step by providing the associated values ​​to the aileron actuators. This can be considered micronavigation, which is similar to actual navigation, where the route to the destination is determined as a sequence of midpoints, and can be referred to as trajectory planning in this document and in the art. Here, the “route” for changing course is determined as a set of roll “midpoints”.

[0159] The time series is therefore adaptive because it adapts the control values ​​it provides to the behavior of the controlled system, as indicated by sensor feedback regarding the state of the controlled system. For example, in the case of an autonomous aircraft, the controller adapts the values ​​it instructs the aileron actuators to achieve and maintain a target roll, thus reacting to the actual roll over time as indicated by the associated sensors.

[0160] In one embodiment, the sequencer (402) / (402m) also selects from a subset of sequences suitable for achieving a particular logical objective. For example, if the input logical objective indicates a rapid climb, the sequencer (402) / (402m) can determine whether the objective is best achieved via a normal climb sequence or via a separate emergency climb sequence. This dynamic adaptation is based on extrapolation. For example, in the first example, each path is extrapolated to the destination, where the shortest path is selected. In another example, at the roll actuator level, this extrapolation is used to determine the path required to perform a turn and its associated midpoint setting for the control values.

[0161] In this embodiment, the time sequencer (402) / (402m), in addition to pre-calculating the parameters of the steps in the sequence, can also determine the execution time / expected time of the sequence by combining estimates of the execution time of each step. For example, pre-calculation of intermediate points provides an estimate of the flight time to a designated destination, and may also provide the required amount of fuel. As another example, plotting can determine whether the aircraft can reach takeoff speed within the finite length of the runway, and whether it can reach a certain altitude after the end of the runway to avoid obstacles.

[0162] Given a mechanism for pre-calculating sequence parameters, it is simpler to use that mechanism during the execution of the sequence to make it more dynamically adaptive. Specifically, sequence values ​​can be pre-calculated at steps within the sequence to update the parameters of the remainder of the sequence as needed to achieve the objective. Therefore, the parameter values ​​determined through this pre-calculation can be updated during the execution of the sequence to reflect the actual state of the controlled system as it progresses through the sequence. For example, if an aircraft encounters a strong headwind, it may use more fuel and take longer to reach the next intermediate point than previously determined. This results in the estimated flight time needing to be updated.

[0163] Generally, pre-compiling the parameters to be used in the time series has several advantages:

[0164] 1. It allows time series to be more highly parameterized, thereby reducing the number of time series required and simplifying the steps. Each step can use these pre-computed values. For example, instead of a separate series for each destination, an autonomous aircraft can have a midway point follow-up series where the pre-computed values ​​are filled in with precise midway points;

[0165] 2. It allows the time required for time series analysis to be estimated in advance, along with potential other resources (such as fuel); and / or

[0166] 3. It allows for the refinement of extrapolation / estimation mechanisms used to pre-compute sequence parameters during sequence execution and for subsequent sequence execution. This extrapolation mechanism reduces the reliance on fast correction feedback during sequence execution.

[0167] In one embodiment, pre-calculation and updating of sequence parameters provide values ​​that are then input back into the re-decisioner (404). For example, the amount of fuel required for the flight can be calculated and fed back to the re-decisioner, allowing it to determine whether the flight is feasible given the available fuel amount. Specifically, an indication of whether the difference (delta) between the required fuel amount and the amount of fuel available in the aircraft is positive can be provided as input to the re-decisioner. If the input indicates a negative difference, the re-decisioner can reconsider continuing the current flight plan and searching for a nearby airport, as it is low on fuel. This situation could also arise due to a leak in the fuel tank.

[0168] This embodiment can be considered an improvement over a conventional PID controller because it offers the aforementioned improvements and also makes it more resilient, including to feedback wait times, because the extrapolation can be corrected at many time steps of the sequence. Furthermore, the extrapolation can be coupled to a specific sequence, such as that specific to an aircraft performing a banking turn. In contrast, a conventional PID controller might only correct its error at each time step based on the updated process variable. Therefore, it may overshoot if the process variable is not updated in a timely manner, and it tends to be misled if there is any misreading of the process variable.

[0169] In one embodiment, the sequencer (402) / (402m) uses conventional adaptive controller components to provide actual control values ​​as outputs instead of directly specifying them. For example, in the airspeed control subsystem, the control values ​​could be implemented by a PID controller. The PID controller setpoint (SP) is specified to the controller by the sequencer based on the current step in its sequence. The process variable is the actual airspeed measured by the airspeed indicator.

[0170] The tile set for each redeterminer is partitioned based on a logical objective. In one embodiment, the tile set for the redeterminer (404) is partitioned based on a logical objective. For example, for an aircraft, the taxiing logical objective is very different from the takeoff and stopping logical objectives. An autonomous aircraft autopilot can use one instance of the tile set for control during taxiing and then switch to another instance during takeoff. This transition between tile sets can occur when the aircraft is positioned on the runway and takeoff is permitted, because the aircraft's expected state for takeoff after taxiing is the same as the state achieved during taxiing, i.e., stopping or decelerating and aligning at the beginning of the runway.

[0171] This logical target partitioning can be achieved using a traditional "switch" statement, which branches on the input values ​​of the logical target by taking advantage of the fact that logical targets can be indicated by a relatively small range of integers. Therefore, the re-determiner (404) can be implemented using the tile set of each input logical target.

[0172] Figure 9 is a block diagram illustrating an embodiment of a redeterminer implemented using multiple tile sets. The redeterminer (404) accepts a designation of a logical target (902) and other inputs (904) as input. The redeterminer (404) then selects the tile set corresponding to the input logical target (908) from a variety of tile sets (906a), (906b)...(906l). It then uses this tile set to match other inputs with the tiles to generate a decision (910), i.e., a tile label, which determines the sequence to be used. Its associated sequencer (402) then executes the sequence indicated by the generated decision.

[0173] Figure 9 illustrates a single sequencer (402), which may be appropriate because in many applications, the same sequence can be selected with different logical objectives, such as when the decision overrides the input logical objective. In particular, for a tile set of one logical objective, it can still be possible to select a sequence that would typically be selected by another tile set. For example, for an autonomous aircraft, the tile set of the logical objective for climbing might override that objective for certain inputs to avoid stall, and instead re-determine the sequence for handling level flight. However, whether to provide separate sequencers or a single shared sequencer in each TCM is an engineering choice.

[0174] In logical target partitioning, a replicated tile is one that is not specific to a particular logical target. In some applications, this results in a low degree of tile replication. For example, for autonomous aircraft, climbing is significantly different from descending. The former works against gravity, while the latter works by utilizing or benefits from gravity. In the case of autonomous vehicles, acceleration is achieved using a separate system, such as a throttle system, while deceleration is achieved using a braking system.

[0175] In logical target partitioning, the tile set may not need to process all inputs relevant to the overall redeterminer. For example, altitude above ground may be irrelevant to stopping or taxiing. However, when in the air, altitude may be relevant to performing tilt turns and landings. Reducing the number of inputs to the tile set leading to each logical target further reduces the complexity of that tile set.

[0176] Given a prior logical objective and the state of the controlled system, the code that opens the input logical objective checks whether a transition to that logical objective is permitted. For example, for an autonomous aircraft, a transition from taxiing to climbing might not be permitted, while a transition from takeoff to climbing is permitted if takeoff has already placed the aircraft in the air at a reasonable airspeed. As another example, a production line can transition from an initial state to startup to operation, but cannot transition directly from an initial state to operation. In general, a controlled system may have the concept of neighboring logical objectives as in the examples above, and only transitions from one logical objective to a neighboring logical objective are permitted. As mentioned in this paper, neighboring logical objectives are neighboring in the sense that the operational domain of the tile set of one logical objective overlaps with the operational domain of the tile set of another logical objective. The transition is only permitted if the controlled system is within this overlapping region.

[0177] Figure 10 is an illustration of the overlapping area between takeoff and climb of an autonomous aircraft. The diagonal “ / / / ” patterned portion (1002) indicates the operational domain for takeoff, covering from 0 airspeed and no altitude / on the ground up to takeoff speed, and then to higher altitudes, to allow the takeoff target to be used during aborted landing.

[0178] The diagonal “\\\” patterned portion (1006) indicates the operational domain for climbing, where the altitude is, for example, at least 200 meters and the airspeed is equal to or higher than the takeoff speed.

[0179] The crosshatched patterned portion (1004) indicates the overlapping area between tile sets / operational domains. The unpatterned portion (1008) is the boundary tile between two tile sets. As shown in Figure 10, the airspeed at takeoff speed overlaps with the minimum airspeed required for climb, and the same applies to altitude. Therefore, a switch from takeoff to climb logical target is allowed in this overlapping area. Similarly, the takeoff abort logical target tile set overlaps with the takeoff logical target tile set, at least in the early to middle stages of takeoff, thus allowing the aircraft to make a transition to abort takeoff.

[0180] In one embodiment, the next logical target can be pre-indicated, but the lookup instruction for the next logical target in the tile set occurs when the controlled system operates within its domain, i.e., when it does not match a boundary tile. In this embodiment, the module providing the next logical target only needs to know the adjacency at the logical target level, rather than the specific region where the transition is permitted.

[0181] This logical objective partitioning of the tile sets described in this paper can be applied to other different control scenarios. For example, when a fault detection system provides input indicating the presence of an internal fault to address, a separate tile set can be used in conjunction with the TCM and the given input logical objective. For instance, if one of the engines has failed or stopped, a twin-engine aircraft can switch to a separate tile set to handle different flight characteristics.

[0182] In one embodiment, the instance of the redeterminer (404) makes a redetermination based on one or more inputs regarding which actual tile set to use. For example, for an aircraft autopilot, this instance could choose between normal cruise and stall handling. This pre-selection of the tile set reduces the number of tiles to consider in the matching process, thereby reducing matching costs. For example, the system disclosed herein supports engineered choices between redeterminer instances when the scenario changes, or switching the tile set being used within a redeterminer instance, as illustrated in Figure 9.

[0183] In contrast to modularity in typical software engineering, appropriate modularity disclosed has a significant impact on the ability to handle more than a few inputs (a limitation of existing LUT controllers). Modularity also improves the complexity, variability, and testability of control systems. Therefore, practicality disclosed in many domains of interest is important.

[0184] Decision-making based on the current state. Without limitation, for clarity of example, it is assumed that in the given re-decisioner (404) of Figures 4A, 4B, and 9, there exists a separate set of tiles for each distinct logical objective. This does not preclude the use of the same set of tiles for two or more logical objectives when appropriate. In one embodiment, the re-decisioner (404) makes decisions based on the current state of the controlled system and its environment, rather than past states. This contrasts with planning methods, where decisions are based on previously generated plans. This current-state decision-making method is made more practical by having the sequencer (402) / (402m) implement a subsequence SS corresponding to steps S continuing from step i to the end of S for each multi-step sequence S. That is, it is a suffix of S in terms of steps.

[0185] For example, an autonomous aircraft could have a sequence Si for flying above a storm, involving climbing to a suitable altitude above the storm, crossing the storm, and then descending to a preferred altitude. However, it should also have a sequence Sj for handling situations where it is above the preferred altitude but above a storm (or other obstacle), so it needs to cross the obstacle and then descend to the preferred altitude. Therefore, once Si has completed the first step, the re-decisioner (404) may simply switch to Sj and still achieve the original objective. That is, the re-decisioner (404) does not need to distinguish between being in the middle of the Si sequence and simply finding itself above the storm with a preferred altitude lower than its current altitude.

[0186] It is important to note that re-determining the sequence is consistent with the overall requirement to be prepared to change the sequence at any time based on changing circumstances. The additional decision-making logic required to remain in Si instead of switching to Sj can increase complexity and risk; it increases risk because it introduces the possibility of deciding differently for a given situation based on what was re-determined earlier. There is a risk to the re-determiner's decision-making by considering the past, beyond what is encoded into the current state, as this consideration may interfere with the re-determiner's (404) immediate response to the current conditions, as required in some changes to the scenario.

[0187] Continuing with the earlier example, the storm may dissipate or move away, so the current conditions are different from those that prompted and justified the choice of sequence Si. In this case, the redeterminer (404) chooses a different sequence, regardless of what was redetermined at an earlier time step, except for its effect on the current state (i.e., the aircraft is at a higher altitude).

[0188] From a complexity perspective, simplifying the redecision based solely on the current state and the expected future state, without considering the past, is an improvement. The past is fully encoded in the current conditions, such as the current airspeed, climb rate, etc. Past states can also add additional input dimensions to the redecisioner (404), thereby increasing the number of tiles required.

[0189] It is important to note that in some cases, providing subsequences in the sequencer (402) / (402m) is crucial because the controlled system may end up in a particular state for reasons other than executing a specific multi-step sequence. For example, as in the earlier example, the aircraft might end up above a storm and at an altitude higher than the preferred altitude because the preferred altitude has changed and the storm has moved below the aircraft; it is not necessarily the case that the re-determiner (404) previously selected the sequence Si above. Nevertheless, it is possible in some controlled systems that the only way for the controlled system to end up in a given current state is by executing the initial M steps in the multi-step sequence Sk. It is still necessary to have a tile corresponding to this current state, indicating that the current state is consistent with continuing the sequence Sk.

[0190] It is important to note that, given a continuation subsequence that is a suffix to a step in Sk, there may be a relatively small implementation difference between continuing Sk in step M+1 and executing a new sequence Sl (which implements the Sk steps from step M+1 to the end). In this case, providing subsequences also avoids having to determine with complete determinism that associated conditions may rarely occur when not in the middle of the current sequence. Therefore, having tiles for each subsequence and for each step of a multi-step sequence is an improvement.

[0191] It is important to note that the subsequence method relies on selecting the next step based on the current state and a goal independent of the original logical goal, assuming that the conditions have not changed to invalidate the selection. The only way to have an action sequence in which the next step is compelling only with respect to the original goal is if there is a "hidden" cost of not doing so or a "hidden" benefit of continuing the original sequence.

[0192] For example, if an autonomous aircraft finds itself at a high altitude above a storm, it should fly above the storm and then descend if necessary under the current conditions, whether it ends up in that state by following the sequence described in the example above, or by the storm moving below it while it is at that altitude and that altitude eventually increases due to the change in target altitude. Typically, the costs and benefits of previous decisions and sequences are encoded in the current state. For example, the fuel cost of reaching a higher altitude is represented in the aircraft at that higher altitude. In some cases (such as financial investment control systems), "sunk" costs are implicit in the gains or returns on the investment, taking into account the costs of acquiring the investment and the costs of selling it. Overall, using subsequences as described herein ensures the selection of a given logical objective determined from the current input state, along with the costs and benefits of the selected logical objective, with little or no hidden costs or benefits.

[0193] Pre-matching is used to dampen oscillations. The risk in the current state decision is that slight oscillations in the input across a threshold can cause oscillations in the decision-making process from one time step to the next. With more stringent application of discrete input thresholds, control systems implemented using tile sets as described in Figure 6 may be at risk of oscillations between two or more tile set matches due to slight variations in one or more inputs.

[0194] For example, if the controlled system is an aircraft and one of the inputs is airspeed, the airspeed may oscillate around 500 km / h due to changes in headwind, changes in readings from the airspeed sensor, or actual changes in the aircraft's speed. If there exists a tile threshold for that input at 500 km / h, such a small change in airspeed can cause oscillations, described herein as rapid switching between two or more tiles from one time step to another, between tiles matched for 500 km / h or higher and tiles matched for less than 500 km / h. This can result in different tile labels being output from one time step to the next, and thus may lead to rapid changes in the logical target assigned to a given sequencer (402) / (402m).

[0195] If there are significant differences in the control settings between these tiles, such oscillating behavior can be unstable, potentially inefficient, or even dangerous. In other words, tile set methods without further mechanisms may require more thresholds and therefore more tiles to minimize the differences between neighboring tiles in order to achieve acceptable behavior in the event of oscillation.

[0196] In one embodiment, when a single tile match is expected, a pre-matching step is provided to attenuate the current input value before it is matched with tiles in the tile set. This step determines whether the current input value is within the tile boundary of the tile and a previously matched tile, which is extended by a margin in one or more of its dimensions.

[0197] For example, the airspeed threshold for the current tile could be 300 km / h as the minimum and 500 km / h as the maximum. In this pre-matching step, the maximum value might be extended with a margin of 50 km / h, so the input airspeed will continue to be matched with the current tile until it exceeds 550 km / h.

[0198] If a previously matched tile is matched using the extended boundary during the pre-matching step, the tile label on that previously matched tile is output, and the input value does not match the complete tile set. Otherwise, the process continues to match the current input to the tile set in the usual manner. Pre-matching is simple to implement because it only requires preserving an indication of the tile boundary of a tile matched in a previous step, expanding that boundary to allow for the associated margin, and then determining whether the current input value is contained within the expanded tile boundary. Using this pre-matching step, small oscillations at airspeeds around 500 km / h do not result in matches with different tiles.

[0199] If the airspeed does indeed exceed 550 km / h, pre-matching fails, and the input value matches to a single tile with a minimum airspeed threshold of 500 km / h. However, once this occurs, pre-matching introduces a margin again, such that if the margin for that minimum threshold is, for example, 20 km / h, the match may not switch back to another tile until an airspeed is measured at least below 480 km / h. Therefore, the airspeed must oscillate at speeds greater than 70 km / h to cause oscillations in tile matching, and it may have to oscillate so rapidly that it causes harmful oscillations.

[0200] In one embodiment, the margin is determined per input dimension and can be a function of either a minimum or maximum threshold. It can also be a threshold or threshold range and a function of the actual input value. For example, a margin of 500 km / h for the maximum airspeed threshold of the tile can be calculated as 10% of its value, thus 50 km / h as described above. However, at the maximum threshold of 200 km / h, the margin is 20 km / h, as 10% of 200.

[0201] Although the examples above focus on only one dimension without restrictions, the pre-matching step can use a margin computed separately on each input dimension, which potentially separates the minimum and maximum margins for each input dimension.

[0202] In one embodiment, this margin is not applied when a neighboring tile indicates a singularity condition. For example, for an aircraft, if the input airspeed is less than the minimum required to cause the aircraft to stall, the margin may not be applied to the boundary. Therefore, once the input indicates a neighboring tile, the control immediately reacts with a corrective action.

[0203] Conversely, singular tiles may have a margin of overlap with non-singular tiles or another singular tile. Therefore, control can function to significantly extricate the controlled system from singular situations, rather than altering its behavior merely by barely escaping them. This approach may not tend to lead to oscillations between non-singular and singular states because, as defined herein, singularities produce discontinuous transitions in the state of the controlled system. Thus, when this new state is reflected in the inputs of one or more decision modules, it maps to tiles that are not adjacent to the original tile. Furthermore, using a margin for singular tiles means that the controlled system must be significantly outside the singular state before matching to a tile that handles the non-singular situation.

[0204] In one embodiment, multiple matches are allowed per time step within the same tile set, and matched tiles from previous time steps are used to suppress matched tiles in the current time step following the new match. A new tile Tj is suppressed if its label corresponds to the same control as a previously matched tile Ti, and the input also matches Ti with an expanded margin. For example, if the tile set provides overlapping tiles, and multiple tile matches are used to handle both airspeed and roll simultaneously, previously matched tiles for airspeed can suppress new tiles for airspeed, but not tiles whose labels correspond to the roll angle.

[0205] Using a pre-matching step with margin is an improvement because it avoids rapid oscillations between tiles / different logical targets. Therefore, logical targets can be significantly different while still achieving smooth, efficient, and safe control. This also avoids the full cost of matching input values ​​to tiles at every time step, assuming the tiles are reasonably sized relative to the normal dynamics of the controlled system. Thus, under the expected conditions, the same tiles are matched across multiple consecutive time steps.

[0206] The above technique can produce a small number of discrete output values / labels for each redeterminer (404). The number of tiles required to output these distinct discrete output values ​​is not large for each sequencer and each actuator redeterminer as described in Figure 3A. That is, if there are D discrete output values, then D is a lower bound on the number of tiles required, and it is small in count, for example, a few dozen, rather than the millions for some practically complex systems. It should be noted that, instead, the combinations of input values ​​tend to be large, potentially significantly increasing the number of tiles required.

[0207] The input preprocessing is delegated to provide the "needs-to-know" re-decisioner input. In one embodiment, each tile set is further simplified by having input preprocessing that produces values ​​designed to be specific and minimal content that the tile set needs as input to make its decision.

[0208] For example, to determine whether an aircraft has sufficient runway ahead for takeoff, one approach is to provide its position on the runway, the runway length, its current airspeed, acceleration, and the required takeoff speed. However, instead, the selected takeoff time series can provide an estimate of the time required to reach the takeoff speed as feedback, and input preprocessing for the tile set can pre-calculate the time before reaching the runway end, assuming acceleration at the takeoff speed. The preprocessing can then provide the difference (delta) between these two values ​​as a single input to the re-determiner. It can even simply provide an indication of whether this difference is positive or negative as input.

[0209] Therefore, this input preprocessing can reduce the number of inputs required for the tile set to a much smaller number and a coarser range, thereby significantly reducing the complexity of the tile set. Furthermore, some parts of this input preprocessing may be specific to the TCM and the tile set used, i.e., the tile set in the module of the current input logic target.

[0210] For example, continuing from the above, the indication of time to complete takeoff is only calculated when the takeoff sequence has been selected. On the other hand, the difference between the remaining fuel and the fuel required to reach the destination can only be calculated when the aircraft has already taken off and is cruising to the next intermediate point.

[0211] The sensor preprocessor (206) in Figures 4A and 4B illustrates an embodiment where the actual sensor input data is preprocessed from its raw input data form into a form more suitable for decision-making by the re-determiner (404). This can be viewed as extended analog preprocessing / processing. Conventional examples of this preprocessing exist. For example, a radar sensor provides a data point cloud as its raw input data. These data points can be preprocessed into distance detections in different directions, indicating distances to objects from a controlled system. The distance to the controlled system can be discretized into multiple ranges using thresholds to accommodate tiling methods. For example, an object at 1050 meters might correspond to a range of 1000 to 1100 meters away. The input can also be preprocessed into differences or percentage differences before being fed into the tile matching process.

[0212] Another example of conventional input preprocessing is a system with redundant sensors for reliability. In this case, the values ​​provided to the control system are those calculated based on the redundant sensor set. For example, in an embodiment with three redundant airspeed sensors, the airspeed reported as input to the control decision process is the average of the actual sensor values ​​after discarding any outliers far outside the expected range. For instance, if the previous airspeed was 200 km / h, and at the current time, sensor 0 reads 202 km / h, sensor 1 reads 204 km / h, and sensor 2 reads 50 km / h, then the reading of sensor 2 is discarded as an erroneous reading, and the average of sensors 0 and 1 (i.e., 203 km / h) is passed to the tile set matching.

[0213] Generally, conventional input preprocessing provides the re-decisioner (404) with the resulting values, making the data more reliable and in a form best suited for decision-making. This input processing is typically a transformation of the actual sensor input. It also provides protection against mistuning and erroneous sensors. There is less range of decisions for the re-decisioner (404) beyond deciding to ignore certain erroneous sensor values. Therefore, per-sensor input preprocessing is required in most (if not all) real-world control systems.

[0214] The two delegations to the input preprocessing in order to provide the "need-to-know" input to the re-determiner (404) include:

[0215] A. Delegate root cause failure analysis and detection to separate subsystems; and / or

[0216] B. Delegating input preprocessing to separate layers and stages, partly based on the time frame of the decision.

[0217] Root cause failure analysis is delegated to separate subsystems.In one embodiment, the determination of fault indications in the controlled system is delegated to a separate automated root cause analysis system, which is considered as input preprocessing. This subsystem can receive a large number of inputs that allow it to detect significant symptoms and thereby determine the root cause of the fault, if any.

[0218] U.S. Patent No. 10,761,921, filed May 8, 2018, entitled "AUTOMATIC ROOT CAUSE ANALYSIS USING TERNARYFAULT SCENARIO REPRESENTATION," is an embodiment of such an automated root cause analysis system, which is incorporated herein by reference for all purposes. The output of this discrete system can be a set of discrete current root cause faults, which are then provided to the control system, particularly to the set of re-determiners tiles associated with the faults. The same analysis mechanism can also provide the effects of the faults, i.e., the effects of the root cause faults associated with the control system.

[0219] Delegating the determination of root cause failures to separate subsystems reduces the number of inputs required to the tile set. In particular, instead of having all the inputs required to perform the root cause analysis, the decision mechanism instead has inputs that provide indications of the root cause failure or implication associated with each re-determiner tile set.

[0220] For example, in an aircraft, if the rudder actuator malfunctions, it affects the aircraft's control when making turns. This might also mean that the aircraft needs to use the engines and tilt to maintain its flight path during cruise. Therefore, the control system needs inputs indicating this malfunction. However, it doesn't need all the inputs required to detect this malfunction and countless other potential malfunctions. Furthermore, it only needs to provide a specific malfunction indication to the set of tiles associated with that particular malfunction. The number of inputs required to reliably detect some malfunctions can be large because there are no direct sensors to detect every possible malfunction. For example, a rudder malfunction might require tracking deviations in the flight path that shouldn't occur, both regarding straight flight and regarding turns. Moreover, experience suggests that sensors can be a primary source of malfunctions, so additional / redundant inputs might be needed to avoid reacting to false positives.

[0221] Delegating perception to separate subsystems In one embodiment, perception is delegated to separate subsystems because a large amount of input may be required to perceive relevant conditions in the environment. For example, an autonomous aircraft might have several radar, camera, and lidar units for monitoring other traffic when in the air. It also needs to detect ground obstacles when on the ground.

[0222] Instead of having the re-decisioner (404) directly receive those sensor inputs, the separate sensing system can "determine" what those inputs indicate and provide a simplified summary to the re-decisioner (404) in the control system with fewer inputs.

[0223] For example, for an autonomous aircraft, an in-flight perception system can simply provide indications of any closing traffic at the same altitude, as well as at the next nearest altitude and at different distances, in each of the four quadrants surrounding the aircraft. As mentioned herein, “nearby” refers to the next higher and the next lower, as defined by the airspace the aircraft is expected to follow, and “closing traffic” refers to traffic whose distance to the aircraft appears to decrease over time. For instance, the aircraft only needs to know that there is rapidly approaching traffic in the left-forward quadrant in order to take evasive action. By delegating the handling of malfunctions and environmental conditions to separate subsystems, actuator tile sets can be significantly reduced in size and complexity.

[0224] If localization and perception are separate, the same approach can be used to delegate to the localization subsystem. That is, the localization of an aircraft associated with a given set of tiles can be provided by a separate subsystem that has further preprocessing within the module to provide the tile set with minimal "needs to know" information. Referring back to the preprocessing example above, the preprocessing provides the difference between the time to reach the runway end and the time required to reach takeoff speed, which is based in part on the aircraft's localization on the runway.

[0225] Time-based delegation In one embodiment, the strategy, as the determination of long-term behavior, is delegated to a separate subsystem that provides strategic input to the "real" control system. The term "real control system," as used herein, refers to the portion of the control system that actually sets control values ​​in the actuators. This reference is partly because controlling the actuators is the true control of the controlled system. This layer, comprised of the TCM for each actuator, is considered the tactical and operational layer because it reacts to changes in input within short timeframes. For example, for an autonomous aircraft, the tactical level needs to be re-determined every 50 milliseconds based on changes in the environment immediately surrounding the aircraft (such as other approaching aircraft).

[0226] This strategic information is considered input to the actual control system because the control system does not necessarily execute the strategy indicated by that input. In other words, the strategy subsystem does not govern or control what the tactical / operational layer does; it merely "suggests" actions. This is because considerations within short timeframes may override the strategy indicated by the input.

[0227] For example, for an autonomous aircraft, the strategy subsystem can provide inputs that "suggest" a change of course, but if the aircraft is dealing with a potential stall, collision, or mechanical failure—making it unsafe to attempt the suggestion—the TCM layer can override it. This suggestion structure is consistent with the view of the tactical layer delegating the processing of some inputs to these other layers, thus treating the result of such delegation as simply another input to the tactical layer.

[0228] The tactical layer is improved by delegating policy determination to the policy layer, because the tactical tile set only needs the obtained policy recommendation as input, rather than all the inputs associated with determining the policy. The latter types of inputs are generally unsuitable for the tactical layer because they are inputs from longer frames and are typically coarser-grained compared to the inputs required for the tactical layer. Therefore, this delegation to the policy subsystem reduces the size of each tile set at the tactical layer.

[0229] Delegation to a strategy subsystem allows the strategy subsystem to be shared across multiple TCMs, which arise from decoupling across multiple actuators. Sharing is feasible because the strategy typically applies to the entire controlled system, not just a single actuator. For example, for an autonomous aircraft, the strategy subsystem could provide input suggesting trajectory corrections to avoid storms, rather than strictly following a shortest path "sequence" to the next midpoint. The strategy applies to the entire aircraft and requires coordinated action across all TCMs. Therefore, a single strategy subsystem is appropriate.

[0230] The inputs to the strategy subsystem are necessarily longer frames than those at the tactical / operations layer because strategic objectives require more time to achieve. Continuing the previous example, a strategy to fly around a storm might take several minutes, while a response to closing inbound traffic might occur within subseconds. Therefore, different time perspectives are used to identify in advance the appropriate and feasible time to execute the strategy within the available time.

[0231] As another example illustrating the opposite scenario, an autonomous vehicle might have a strategy of overtaking a slower vehicle ahead from the left and then merging back. However, this strategy might not be suitable for situations where the autonomous vehicle needs to exit from the right exit before it can safely complete the strategic maneuver. To make the correct decision, in some cases, the input to the strategy subsystem needs to indicate conditions such as the time to reach the next exit several minutes in advance. In contrast, at the tactical level, time is typically in the sub-second range and uses a finer-grained unit than, for example, "kilometers to the next exit." Separating the strategy and tactical parts, due to the differences in the time frames processed by each part, avoids complicating longer time frames with time metrics that are too fine-grained relative to their requirements, and avoids complicating shorter ranges with time ranges that are longer than required.

[0232] In one embodiment, a TSIR structure is used to implement the strategy subsystem. Specifically, for a given controlled system and environment, there is typically a finite number of strategies. For example, for an autonomous vehicle flying into a severe storm system, one strategy is to turn around and return. Another strategy is to land as quickly as possible. Yet another strategy is to bypass the storm or fly over it. Each of these strategies requires a sequence of steps, which can therefore be implemented by a strategy sequencer. Strategy parameters can be adjusted during strategy execution by a re-determiner or sequencer that determines different sequences or different sequence parameters. Finally, the task of the strategy re-determiner is to receive input and determine the best strategy to execute. For example, for an autonomous vehicle, conditions at a point might indicate flying over the storm, but environmental conditions may change, so the strategy re-determiner determines a new strategy / time sequence corresponding to flying eastward around it. One or more tile sets used by the strategy subsystem can re-determine different strategies based on the changed input by matching different tiles, which is the same processing used at the tactical level.

[0233] In one embodiment, the sequencer portion of the policy module provides different input logical objectives to different TCMs. For example, for an autonomous aircraft, the policy layer could instruct the airspeed TCM to increase, instruct the flap TCM to half-deploy, and instruct the roll TCM to roll slightly to the left to prepare for landing. By providing these separate objectives, the policy layer further simplifies the TCM layer redeterminer and tile set, because each TCM only needs to decide how to achieve its specific objective, rather than determining what its objective is from a higher-level objective that the policy agent decides on to coordinate with or how to coordinate with other TCMs.

[0234] Therefore, when two or more TCMs are instructed differently at a single step, that step translates the logical objective of that step into a logical objective for each TCM. For example, for an autonomous aircraft, if the sequence is to perform turns and climbs, the step might specify logical objectives based on airspeed, roll, and pitch to coordinate tactical TCMs. Thus, the policy module is not decoupled in the same way as at the tactical layer. However, this coupling may not complicate one or more policy redeterminer tile sets because the coupling is strictly within the sequencer. Individual actuators may be relatively independent in how they are set up, and their associated TCMs receive relatively independent, relatively low-level objectives, so this coupling may not be handled in the sequencer section. Therefore, coupling at the tactical layer would typically require input logical objectives that are essentially the cross product of the input logical objectives of two coupled actuators, which significantly complicates and thus increases the memory size and processing cost of the tile set.

[0235] The policy layer itself can be further delegated to separate meta-policy subsystems. This layer involves higher-level policies on how the controlled system should operate according to application objectives, thereby outputting a path from the current state to a designated final target state to the policy layer. For example, for an aircraft autopilot, it determines how to fly from the current position to the indicated destination by providing the policy layer with a sequence of intermediate points.

[0236] In one embodiment, the layer includes a redeterminer and a sequencer, i.e., a TSIR structure, that executes a sequence designed to achieve the logical objective selected by the redeterminer.

[0237] Figure 11 is an illustration of an embodiment of a re-determiner / sequencer pair constructed as a hierarchical structure. As shown in Figure 11, the hierarchical structure includes a tactical layer (1102) with multiple instances of a TSIR structure, namely, a TCM (1104) that receives input from a policy layer (1106), and a policy layer (1106) that receives input from a meta-policy layer. Recalling Figures 4A / 4B, each sequencer may also have additional inputs, which are not shown in Figure 11 for clarity and without limitation. Figure 11 indicates two layers of policies, namely, a basic policy and a meta-policy, but is not limited thereto, for the sake of simplicity and clarity. Control systems using the disclosed content can have more than two layers of policies, and, if appropriate, multiple policy modules per layer, thus extending the example structure in Figure 11. That is, a policy module at layer m can provide inputs to policy modules at layer m-1 or smaller.

[0238] It is important to note that Figure 11 may differ from traditional philosophy, which views meta-policy as a higher level than policy, and policy as a higher level than tactics / operations. For example, traditional AI subsumption architectures place lower-level behaviors corresponding to operations at the bottom, where higher-level controls, such as policy suppression or blocking, control the lower-levels.

[0239] In the disclosed delegation model, placing the tactical / operations layer at the top is more appropriate because the tactical level (1102) makes the final control decisions and treats the policy input (1106) as another input it can use or override when conditions require. For example, if the input to the autonomous aircraft's tactical control indicates that the aircraft's airspeed is approaching a stall condition, then the autonomous aircraft's tactical control should not follow a climb policy. That is, unlike traditional phased computation, the tactical level (1102) does not need to synchronously wait for the policy level (1106) or meta-policy (1108) modules to provide their inputs to the tactical layer. Furthermore, the policy (1106) and meta-policy (1108) modules can be executed at a lower frequency than the tactical / operations layer because the tactical layer (1102) provides a much faster response.

[0240] It is important to note that a key aspect of the structure described in Figure 11 is that the policy (1106) and meta-policy (1108) layers do not need to “command” the tactical-level modules, but instead “suggest” in their inputs what the tactical layer (1102) should attempt to accomplish, thus recognizing that it can still override these suggestions if necessary.

[0241] It is important to note that the improvement in the disclosed structure is that the tactical layer (1102) does not strictly depend on the policy (1106) and meta-policy (1108) modules. Furthermore, these latter modules (1106) and (1108) can prepare the tactical layer (1102) for not performing the proposed content.

[0242] For example, the meta-policy (1108) and policy (1106) layers can prepare for a landing and retry approach that is about to be aborted. In this sense, this structure is similar to the traditional OSI (Open Systems Interconnection) network layers: the network layer IP (Internet Protocol) on the Internet is "best-effort" in the sense that it delivers packets if it can. The transport layer TCP (Transmission Control Protocol) is prepared to verify whether the action was successful (e.g., whether the packet was received) and retry if necessary. Furthermore, the application layer is expected to detect whether the transport layer indicates that it may have failed and then take action to process and recover. In this analogy, the tactical layer corresponds to the IP network layer, the policy layer is similar to the TCP transport layer, and the meta-policy layer corresponds to the application layer.

[0243] It is important to note that, by utilizing this delegation, the complexity and cost of the tile set in the tactical layer (1102) are reduced / improved because the number of inputs required for each layer is reduced. For example, tactical decisions require inputs reflecting short-term conditions, while strategic decisions require inputs indicating long-term conditions. In this case, the delegation means that the tactical control layers (1102), (1104) only require short-term condition inputs plus strategic inputs (1106), rather than the inputs required to make the strategic decision.

[0244] For example, instantaneous roll angle, pitch, airspeed, and / or wind speed may be irrelevant to the aircraft's strategic decisions, thus the decision module (1106) has fewer dimensions than when it is incorporated with a short-term tactical control module (1102). Similarly, the tactical TCM (1104) does not need to know aspects of the next intermediate point, airspace restrictions, and air traffic control inputs. Therefore, the number of inputs going to each TCM is reduced compared to the case where it directly has inputs that consider longer-term factors, and thus the size of the tile set is reduced.

[0245] The reduction in the number of inputs in the layer improves / reduces dimensionality, and thus exponentially improves / reduces the number of tiles. Therefore, the K inputs required to make a strategic decision (1106) can be removed from the combined strategic-tactical layer when it is reduced to tactical control only (1102). One or more strategic inputs (1110) are reduced by this module to a single input (1106) provided by the strategic sequence output (i.e., its input logical target) entering the tactical layer (1104). This reduction in the number of inputs lowers the cost of generating tiles and matching inputs to tiles.

[0246] Another improvement of modularity is that separate subsystems can be developed, tested, and updated independently. For example, a complete version of strategic control (1106) can be developed without changing the tactical layer, and therefore does not require retesting / recertification of that layer.

[0247] A further improvement to the disclosed delegation is that a single instance of the strategy subsystem can be shared across all modules in the tactical layer. This represents a reduction / improvement in processing cost, memory, and communication compared to having the strategy determined separately in each tactical module.

[0248] A further improvement to the disclosed modularity is that different layers can be constructed as separate processes, so they can be executed in parallel and independently restarted after a failure for high availability.

[0249] In one embodiment, the modularity of decision-making can also be based on other criteria. For example, an autonomous aircraft autopilot system can be constructed as three layers: a meta-policy layer (1108) determines whether reaching a designated destination is feasible and then provides midway points as input to the flight instructor (1106) (i.e., the policy layer). The flight instructor provides a strategy for reaching the next midway point, thus providing input to the tactical layer (1102). The autopilot's tactical control layer then determines how to maintain safe flight and implements the maneuvers suggested by the input from the flight instructor.

[0250] In this example, a separate instance of the control system / TCM (1102) can be used to control each actuator to achieve short-term safe flight following a specified trajectory (if following that trajectory is safe). Another instance / TCM (1106) can be used to generate suggested changes to the trajectory to follow a route identified by the conventional navigation system. The output of the latter (1106) is provided as input to the former (1102) in the form of a logical objective to guide it on the trajectory. For example, the latter module (1106) can determine a new heading that requires a right turn. This decision is input to the former module (1102), which then tilts the aircraft to initiate the turn, assuming it is safe to do so.

[0251] Overall, it's important to note that delegating to separate subsystems improves / reduces the number of inputs, thereby improving / reducing the tile set size for each decision module. It also improves / reduces the cost of tile matching by allowing matching on each tile set to be performed at different frequencies (often improved / lower frequencies) and in parallel.

[0252] Each re-determiner's choice of input and threshold Minimizing the actual number of inputs going to the redeciter is an improvement in order to: minimize the number of tiles required; minimize the cost of generating the tile set; minimize the spatial cost of the tile set; and / or minimize the cost of tile matching. Making the per-tile range of the input as large as possible is also an improvement / beneficial for the redeciter to make decisions that lead to acceptable control. In particular, it is important to note that the range of the input dimension ID for tile T may mean that more extreme values ​​for that input could lead to different logical objectives / sequences.

[0253] For example, for an autonomous aircraft, the tile corresponding to "cruise" as a logical objective can reasonably correspond to the entire flight envelope of the aircraft in terms of airspeed and load factor. As illustrated above in Figure 8, the tile approximates a continuous function. Utilizing tiles of varying widths along the same dimension may be most efficient. Therefore, it is noteworthy that in Figure 8, the width of the tile along the airspeed dimension and thus within the airspeed range is varied as needed to optimally approximate a continuous curve.

[0254] Therefore, it's important to note that when considering the logical objective first, a redeterminer might need its own logical objective as input so it knows what it should achieve. However, in many cases, it may not need the logical objective of another redeterminer as input. Having corresponding process variables instead may be sufficient for it.

[0255] For example, in the case of an autonomous aircraft, the roll re-determiner may need to know the actual airspeed, i.e., the process variable, but does not need to know the logical target provided to the roll re-determiner. This is because: the actual effect on roll is based on the actual airspeed, not the target, and / or; the airspeed target may only change over time, so the roll re-determiner has time to adapt to changes in airspeed due to changes in the logical target input to the airspeed TCM, if necessary.

[0256] Excluding other logical targets as input reduces the number of dimensions and thus the number of tiles, but also avoids considering situations that should not occur, such as logical targets for switching from cruise speed to taxi speed or from cruise speed to stop airspeed.

[0257] For each re-determiner, process variables and environmental inputs can also be examined to determine which process variables and environmental inputs are necessary, such as which process variables and environmental inputs a particular actuator is actually sensitive to. For example, a roll re-determiner might be sensitive to airspeed and altitude, so these two inputs could be included as inputs to the roll re-determiner. However, it might be insensitive to crosswinds under airspeed and altitude conditions, so it is prepared to switch to a significant roll angle.

[0258] A roll redeterminer may not need input I because the effect of input I is reflected in another input it already has. For example, the roll redeterminer example above may not need climb rate variables and / or pitch process variables because any significant pitch is expected to manifest as an effect on airspeed. Similarly, it may not need inputs about faults because they are adequately represented by their effect on airspeed. Therefore, these inputs do not need to be included.

[0259] Other inputs can be directly provided to the TCM for its orderer, but not for its redeterminer tile set. Therefore, the TCM's behavior is sensitive to this input, but it is not required as input to the redeterminer and is therefore not an input dimension of that tile set. In one case, the redeterminer might need this input, but only coarsely.

[0260] For example, in the roll re-determiner example above, the re-determiner might only need to know altitude to know whether the aircraft is high enough to perform a roll, hard roll, and / or emergency hard roll, so there are actually only four categories / different ranges to handle, including cases where the altitude is too low. On the other hand, a sequencer could use altitude as a more granular set of values, or even essentially the actual process variable values.

[0261] In summary, carefully selecting the input for each redeterminer tile set is one way to minimize the number of tiles required for the redeterminer, which improves / reduces the size and complexity of the tile set.

[0262] Carefully discretize the input. Choosing a threshold for each selected input (i.e., discretizing the input) is also a technique to improve / reduce the size and complexity of the tile set.

[0263] The tile set method described in this paper implicitly discretizes each input into a set of ranges for each input, where each range corresponds to the length of the edge of the tile in the dimension corresponding to that input. For example, for a small aircraft with normal load and altitude, the takeoff speed can be defined for a given tile within a range of 120 km / h or higher. The selection of these ranges and the resulting number of tiles has a significant impact on the cost of generation, storage space, and matching.

[0264] There are five input categories, each with a different criterion for selecting the discretization method. These categories are:

[0265] 1. Input logical target;

[0266] 2. Process variable feedback;

[0267] 3. Environmental input;

[0268] 4. Time input; and / or

[0269] 5. Input fault conditions.

[0270] Input logical objectives. The input logical objectives of the redeterminer are discretized, and in a preferred embodiment, simply one of a plurality of tile sets is selected for use, as illustrated above in Figure 9. For example, an autopilot might have discrete logical objectives such as: cruising at the current altitude / speed / direction, queuing for a landing approach; landing; and / or taxiing.

[0271] Generally, there are a small number of objectives because these objectives need to be understood and achieved by a human operator under manual control. That is, there might be dozens of objectives, rather than thousands or even hundreds. Small integers can be assigned to each objective, just as in traditional computer implementations of enumeration. For the tile set of each logical objective, the total number of tiles is the sum of the tiles required for each logical objective. Because there is a single high-level objective at any given time, the amount by which the high-level objective input is multiplied by the number of combinations of input values ​​is relatively small, such as 10x.

[0272] It's important to note that a small number of high-level objectives depend on the parameterized logical objective. For example, a high-level objective like "fly from the current location to airport A" (where A is a parameter) means it's the same logical objective and therefore the same sequence choice, regardless of which airport. The sequencer can record these parameters, and thus if the parameters change, the current sequence is canceled and a new sequence is started, even if the logical objective hasn't changed.

[0273] Process variable feedback. In addition to high-level target inputs, the re-decision-maker may also need to know some parameters associated with the current target. For example, it may need to know whether it maintained the correct altitude, speed, and direction during cruise. This falls under process variable feedback.

[0274] In one embodiment, each re-decisioner may require input from feedback from the controlled system, instructing how the controlled system should respond to a given sequencer objective. For example, an aircraft has a specified takeoff speed. This takeoff speed is a threshold that allows the re-decisioner to make a takeoff decision—that is, to raise the elevator and take off. The associated re-decisioner needs to know whether the aircraft has reached that takeoff speed. However, knowing that the actual airspeed is within the takeoff speed range is sufficient. A coarse range as feedback on process variables is adequate because the associated sequencer controls the actual actuators and provides the sequence. For example, the re-decisioner does not need to handle throttle position or aircraft acceleration. Instead, the re-decisioner needs sufficient input to make its discrete decisions correctly.

[0275] In one embodiment, the range used to discretize the process variables is made as coarse as possible to achieve adequate control. Using a coarse range means there are fewer ranges and therefore far fewer tiles.

[0276] In one embodiment, for input process variables corresponding to a logical objective, input preprocessing provides feedback as an indication of the difference between the logical objective and the process variable. For example, for an autopilot, if the aircraft is instructed to take off, the input is an indication of the difference between the takeoff speed and the current speed. Therefore, a re-decision may not result in takeoff until the input indicates a positive difference.

[0277] It's important to note that using this difference means the re-determiner may not require both the target and current values ​​as inputs. It can also increase the accuracy of control over the same number of discrete feedback input values. Continuing the example above, the discrete values ​​of the difference between the target airspeed and the corresponding process variable could be: much lower, slightly lower, equivalent, slightly higher, much higher. Therefore, the input is mapped to a range, thus providing a relatively small number for that input. This difference categorization approach recognizes that, similarly, human operators make decisions based on the category of the difference, rather than on the exact absolute value, or even the exact absolute value of the difference.

[0278] The relevant approach is to identify and designate “channels” as techniques for discretizing feedback, such as those described in U.S. Patent Application No. 16 / 795,236, filed February 19, 2020, entitled “USING A LANE-STRUCTURED DYNAMIC ENVIRONMENT FOR RULE-BASEDAUTOMATED CONTROL,” which is incorporated herein by reference for all purposes.

[0279] When using channels, the environmental input for a location can be reduced to the location within the channel, and further reduced to the difference between the actual and desired locations within the channel. For example, if an aircraft is guided by air traffic control to a specific air channel, including altitude, one or more autopilot tile sets could have the difference between their current location and trajectory and the location and trajectory of the current air channel as input from input preprocessing, typically with separate differences for each dimension. Instead, the redeterminer may need to re-determine how to reduce the difference from the target location and trajectory—this difference is discretized into a small number of difference categorical values ​​as described in this paper, rather than dealing with a large number of absolute values ​​for location, altitude, and trajectory.

[0280] In one embodiment, these discrete values ​​indicate percentage differences rather than absolute differences. For example, feedback might indicate that the aircraft is at a speed 5% lower than its takeoff speed. This percentage difference takes into account the fact that acceptable differences may vary depending on the actual values. For example, exceeding the intended speed by 20 km / h while taxiing might be a problem when navigating a corner, while exceeding the takeoff speed by 20 km / h is not. Using the percentage difference between the actual airspeed and the intended airspeed, earlier suggested discrete values ​​were mapped to percentages. For example, "equivalent" might correspond to within 3% of the intended speed, "slightly below" to at most 7% below, and "far below" to more than 7% below.

[0281] In one embodiment, the percentage difference method, along with separate logical target inputs indicating the operating mode (such as taxiing, takeoff, cruise, or landing), allows each redeterminer to make different decisions that indirectly take into account the absolute value of a metric (such as airspeed). For example, if the mode is "takeoff," the "equivalent" value as feedback on airspeed indicates that the aircraft is close enough to takeoff speed to take off.

[0282] It's important to note that by using the discretization range of the percentage difference between the process variable and the intended value (SP), the number of discrete values ​​required for each actuator is relatively small, such as the five used in the example above. This fact, coupled with the finite number of such process variable input combinations required by the redeterminer, means that the amount by which the process variable feedback multiplies the number of input value combinations is limited and practical. For example, for an aircraft, there might be altitude, airspeed, pitch, roll, and yaw; therefore, for discrete values, a multiplier of 625 could be used for the remaining inputs. That is, 5 different inputs—each with 5 different possible values—equal 625 different combinations of input values.

[0283] The tile generation method described below essentially begins with the fully known normal operating domain of the controlled system and then incrementally expands the tile set to handle more extreme cases without necessarily requiring intervention. In some cases, existing tiles are simply expanded. In others, tiles are expanded and then split based on which subregions share a common label or output logic goal. During this incremental expansion process, a tile may have a different width on the same input than another tile. It is important to note that this variability in tile width or any dimension is a key difference between the tile set method and the lookup table method, where entries are organized into rows and columns, thus requiring the same input range for each entry in a given row, and similarly for columns.

[0284] Environmental Inputs. Inputs in the Environment category indicate a specific aspect of the environment of the controlled system. For example, outside air temperature, head wind speed, and cross wind speed are all examples of environmental inputs in the case of an aircraft autopilot.

[0285] As described above, environmental perception is delegated to a separate perception system that provides simplified input to an appropriate set of re-determiner tiles. For example, this perception could indicate turbulence, storm activity, and other traffic in various quadrants around an aircraft at the same or adjacent altitudes.

[0286] For environmental inputs not processed by perception, discretization involves identifying a threshold at which different decision outcomes may occur between values ​​below and above that threshold. For example, if crosswinds are indicated as either absent or present, and "absent" is defined as less than 5 km / h, then when the crosswind changes from 4 km / h to 8 km / h, the aircraft must respond as if the crosswind has changed to a certain maximum value (such as jet velocity). Therefore, it is likely to overreact.

[0287] The solution is to provide enough discrete values ​​so that when environmental input values ​​change between one range and another, there are no unsafe, uncomfortable, or inefficient transitions in the control. In the example above, crosswinds can be categorized as very low, low, moderate, high, and very high, each with an associated threshold.

[0288] The threshold for one input may depend to some extent on the values ​​of other inputs. For example, the airspeed required during a turn may depend on both the headwind and the crosswind. As a continuous function, this can be visualized as a 3D surface, where the X and Y dimensions correspond to the headwind and crosswind, and the Z dimension corresponds to the throttle setting. The discretization of these inputs and this surface may need to be fine-grained enough to adequately approximate the surface, or it may need to be as coarse-grained as possible to minimize the number of tiles. This approximation is illustrated in Figure 8 using boundary tiles.

[0289] Altitude is an example of an input that relates both to process variables reflecting the target setpoint and to environmental inputs leading to a re-determiner. However, as an environmental input leading to a re-determiner other than an altitude re-determiner, it needs to be considered as an absolute value, not just a difference from the expected level. For example, flying below 200 feet in altitude might mean that any significant turns would be too dangerous to execute, and would therefore be immediately excluded by that threshold.

[0290] Therefore, altitude can be discretized into "on the ground," "low altitude," and one or more values ​​corresponding to higher altitudes. For example, at altitudes of 200 to 1000 feet, the feasibility of making a safe bank turn might depend on an airspeed above a certain threshold, while at altitudes above 1000 feet, a bank turn might only depend on not being in a stall state. As illustrated in this example, the semantics of range and extent may differ between different tile sets. The extent may also vary within the same tile set for different values ​​of other inputs. Preprocessing of the actual input can compute differences or percentage differences, or provide absolute values ​​as required for each tile set.

[0291] It should be noted that because there are relatively few environmental inputs, and they can be discretized into a relatively small number of discrete values ​​per input, the total multiplier for the combination of input values ​​provided by the environmental inputs is also “small” for automation, that is, about 100 to 1000.

[0292] Time input. In one embodiment, the tile set includes a dimension corresponding to the difference between the time required to achieve the objective and the time available to achieve the objective. For example, for an autonomous aircraft deciding whether to proceed with a takeoff objective, the available time is the time it takes to reach takeoff speed based on its distance to the runway end, its speed, and expected acceleration, while the required time is the expected time to reach takeoff speed given the current airspeed and acceleration.

[0293] When the time sequencer re-determines the parameters of the takeoff sequence at the time step, it can calculate the required time. Sensor input preprocessing can calculate the available time based on the distance to the runway end at the current airspeed, and then calculate the difference between these values ​​and provide it as input to the re-determiner's tile set. Therefore, the re-determiner can map the current input to tiles indicating whether to continue the takeoff sequence or abort takeoff. If an obstacle appears on the runway ahead of the aircraft, the available time for takeoff is significantly reduced, so the matched tile will indicate abort takeoff.

[0294] It's important to note that the calculation of the required time is delegated to the time sequencer because, generally, the required time is highly dependent on the specific steps to be taken in the time series. For example, for an autonomous vehicle, the journey to a specific destination depends not only on determining intermediate points and how far apart each intermediate point is, but also on other factors already determined by the time sequencer. Furthermore, it may need to be periodically recalculated as the sequence is processed to determine the remaining time.

[0295] Like other inputs, the time input, such as acceleration, is discretized by a threshold. In one embodiment, the threshold is conservative. Therefore, it is possible that takeoff might be aborted if it is not strictly necessary. However, it is also possible that the current acceleration might suddenly decrease, thus making the decision optimistic. The inaccuracy introduced by discretizing the time input can be comparable to the inaccuracy in the "required time" estimate due to unexpected changes in the environment or the behavior of the controlled system (such as engine failure) if not included in the inaccuracies in the "required time" estimate.

[0296] The initial input for the time required for a given logical target can be determined by passing the takeoff target to the time sequencer so that it calculates the required time, and then re-determining in the next time step based on that feedback input and the available time (if the difference is negative). This approach takes advantage of the fact that almost nothing needs to be done for the start of the takeoff sequence within the first 100 milliseconds, or regardless of the re-determiner's cycle.

[0297] In this example of takeoff, each range of current speed effectively defines a subsequence of the overall takeoff sequence. Specifically, in a normal takeoff sequence, the aircraft departs at zero airspeed at the runway terminus. As takeoff progresses, it both reduces its distance to the runway end and thus its available time, and increases its airspeed. Therefore, under normal circumstances, it advances through the sequence of tiles that sustain the takeoff decision. However, if it fails to accelerate as expected, this airspeed is less than the airspeed required relative to the time available to match the tiles used to continue the takeoff decision. Therefore, it matches the tile for aborting takeoff, causing the sequence to change to aborted takeoff. These takeoff tiles can also be matched during aborted landing, as there are “takeoff” tiles for considerably high airspeeds and relatively short takeoff times, which would occur if landing were aborted after the aircraft had already landed but had not yet significantly reduced its speed.

[0298] It's important to note that this subsequence behavior is similar to what happens when using the system in more complex sequences. For example, for an autonomous vehicle, to bypass a slower vehicle ahead, it needs to move to the passing lane, accelerate to overtake the slower vehicle, and merge back into its original lane. The time requirement for making this decision is the sum of the time allocated to each step.

[0299] It's important to note that the same subsequence behavior applies. That is, once a vehicle is in the overtaking lane, the time requirement decreases to the time required for overtaking and merging. Therefore, the matching tile is a different tile corresponding to the overtaking-merging sequence with different time requirements. However, under normal circumstances, it selects the subsequence that is completing the original sequence, assuming that the circumstances still indicate it should do so. As an example of a changed circumstance, a vehicle attempting to overtake might accelerate to the point that it takes too long to overtake or doesn't need to overtake at all. In this case, the matching tile indicates it should simply merge back.

[0300] It is important to note that multiple actions may occur concurrently. For example, a landing sequence on a runway involves aligning with the runway, descending to a landing point near the runway terminus, and reducing speed to taxi speed before the runway terminus; that is, alignment and airspeed reduction may occur concurrently. In this case, the required time is the maximum of the time required for a concurrent sequence.

[0301] Fault condition input. In one embodiment, the fault condition input provided by a separate root cause analysis subsystem is discretized by that subsystem as part of identifying the fault or effect. Preprocessing of the actual output of this separate subsystem can map its output to actual input values ​​suitable for the re-determiner. For example, various faults leading to engine failure can be mapped to a single value corresponding to "zero thrust," allowing the re-determiner to operate on individual effects rather than having tiles for each associated fault condition.

[0302] Input preprocessing within the control module itself can specialize faults that are relevant to the module and even to the logical objective being processed. For example, for an autonomous aircraft, a simple indication of an engine problem while the aircraft is taxiing is sufficient to instruct a return to the hangar, whereas if the aircraft is cruising in flight, more details may be helpful in determining how urgent the situation is and how best to handle it.

[0303] In summary, by using one or all of the above five techniques, the number of possible combinations of input values ​​is typically in the millions or less, making automated exhaustive testing feasible.

[0304] Identify operational constraints. It is important to note that for many controlled systems, there exist combinations of logical objectives and other inputs that are not permitted or "impossible." These combinations are excluded by operational constraints, which are identified as part of the engineering design process.

[0305] For example, for an autonomous aircraft, if the airspeed is less than the required speed for flight or the altitude is too low, the roll re-determiner should not accept the logical objective of tilting. As another example, if the aircraft is stationary on the ground or gliding, tilting to the left makes no sense. The word "impossible" is in quotes because even if a certain combination should not occur, the re-determiner might still receive such an input combination in the event of unpredictable failure.

[0306] Operational constraints can also indicate situations of discontinuous behavior occurring in a controlled system, referred to as discontinuities in this paper. For example, in the case of an aircraft, when the wing stalls, the lift being provided by the wing decreases significantly and rapidly. The autopilot needs to respond immediately and appropriately to this anomalous condition. In these cases, the control typically needs to restore the sequence rather than a smooth modification of the existing control. Furthermore, overriding normal behavior is often necessary.

[0307] To further illustrate this example with wing stall, the detection of wing stall can override the normal elevator control settings to ensure a nose-down orientation of the aircraft in order to restore airspeed and lift. This recovery sequence is typically defined as part of the engineering process and is therefore known as the appropriate label for one or more tiles corresponding to that condition. It does not need to be "discovered" by a prediction / simulation mechanism.

[0308] For example, for nuclear power plants, it is unnecessary to simulate core explosions to know when a shutdown sequence needs to be initiated when the reactor overheats. Furthermore, prediction / simulation mechanisms may not be highly accurate in simulating the behavior of a controlled system outside its operational constraints because they are not designed to operate in a predictable manner outside those constraints.

[0309] It is important to note that tile labels may indicate behavior that is completely different from that of its neighboring tiles. Thus, neighboring tiles may indicate a normal operating sequence, while the current tile may indicate a recovery sequence, since being crossed to enter one or more thresholds in that current tile indicates that the system is outside its operating constraints and potentially represents a discontinuity in its behavior.

[0310] For example, a tile corresponding to a stall condition may have a completely different output than its neighboring tiles that do not correspond to a stall condition. This is because there is no requirement for continuity between neighboring tiles in the system described in this paper. In contrast, conventional control systems based on continuous mathematics may require and assume continuity because they use continuous mathematics.

[0311] It is important to note that engineered systems may not possess a large number of discontinuities or unknown discontinuities. Otherwise, human operator control would be infeasible. For example, a human operator might assume that a slight increase in throttle would result in a slight increase in climb rate and airspeed, rather than a sudden, large increase or decrease, or other destabilizing behavior. Therefore, because these discontinuities are known and relatively few in number, they do not add significant complexity to the tile set and can even be manually specified. Furthermore, the handling of transitions away from discontinuities is handled by an overlap margin associated with the tile, such that, in the case of an aircraft, the aircraft is significantly outside of stall conditions and therefore overlaps with the tile corresponding to normal operation before the control transition returns to normal control behavior.

[0312] These operational constraints can define an entire subspace of tiles that should not appear or are not permitted and therefore can be immediately marked as boundary tiles. For example, for a roll redeterminer, the entire subspace of low airspeed or low altitude, as well as logical targets other than level, are in the disallowed category. Operational constraints applied to a given input logical target can define the final boundary tiles of the corresponding tile set. The term "final" is used in this paper because the initial version of the tile set may use a more restrictive set of boundary tiles, since the full range of input values ​​allowed by the operational constraints has not yet been specified and tested.

[0313] In one embodiment, the tile labels for all tiles in the "disallowed" subspace correspond to a neutral output, which corresponds to a neutral sequence augmented with an error indication. For example, for a roll redeterminer, the neutral output is level, so zero roll and an error indication simply report a logically disallowed target. If the altitude is not zero, the neutral output of the airspeed redeterminer could be the cruise speed, or alternatively, if on the ground, the neutral output of the airspeed redeterminer could be the current airspeed.

[0314] It's important to note that operational constraints can also limit transitions between input logical objectives. For example, for an autonomous aircraft, a direct transition from cruise to stop or vice versa is meaningless. The aircraft needs to transition from cruise to landing, to taxiing, and then to stop. In a multi-tile set redeterminer where tile sets are partitioned based on input logical objectives, the code used to select the tile set can easily check whether the new input logical objective is the same as the previous one, or otherwise check whether the new input logical objective corresponds to a permissible transition from the previous one. This is another benefit of the multi-tile set approach. Checking the transition in explicit code is feasible and avoids complicating the tile set by having a previous logical objective as input, as would occur in the case of a single-tile set approach.

[0315] It's important to note that when using these operational constraints identified during engineering design (if not the driving common sense in the example above), the entire subspace of tiles for each re-determiner can be pre-labeled / tagged, reducing the number of tiles that need to be labeled based on more in-depth / more expensive evaluations. Furthermore, all tiles in the "disallowed" subspace can be merged into a single tile or fewer tiles, reducing tile space and matching costs. Therefore, leveraging operational constraints reduces the cost of generating tile sets and also allows for efficient merging of tiles, thus reducing space and matching costs.

[0316] Matching based on decision trees and First Match Semantics. In the decision tree representation, the processing of a subspace can be represented as a single-node subtree corresponding to that subspace.

[0317] The technique described above allows the entire control system to be constructed as multiple re-decisioners, such that each re-decisioner requires a single result from the lookup of each tile set. This approach makes first-match semantics acceptable, meaning that the first matching tile is returned because it must be unique. For a large number of tiles, the simple approach of matching an input vector to one or more tiles by iterating through a set of N tiles is expensive. First-match semantics make it feasible to transform each such tile set into a decision tree implementation, thereby reducing the memory required to store the tile sets and also reducing the cost of combining input values ​​to match corresponding tiles.

[0318] It's important to note that the initial matching semantics are expected in the case of a re-determiner for each actuator, because control applications (such as aircraft autopilots) need to generate a control decision for each actuator at each time interval—for example, whether to raise, lower, or keep the ailerons constant at each time interval. It would be meaningless for it to "decide" to both raise and lower them. If the TCM generates two labels using a given set of inputs, it cannot execute both sequences simultaneously if they are inconsistent, and this is a problem to be corrected. If two consistent sequences exist for the same actuator, they are repetitive and should also be corrected. This contrasts with coupled or MIMO control, such as autopilots, which need to generate two actions for different control variables, such as raising the elevator and increasing the throttle. Decoupling means that each TCM re-determiner has a decision output.

[0319] In one embodiment, starting from the root node, the tile set is transformed into a decision tree. Input dimensions are selected and associated with each node. Each child node of a given node corresponds to a range of input values ​​in the dimension associated with its parent. Each leaf node stores the tile label corresponding to the tile reached via the path from the root of the tree to that leaf.

[0320] Figure 12 is a diagram of a portion of the decision tree that maps inputs to associated tile labels. As shown in Figure 12, the altimeter node (1202) is generated with dimensions corresponding to altitude and ranges corresponding to greater than or equal to 200 meters. The input is a vector of input values, with one vector for each of the K input dimensions. In this example, the tile is defined by throttle, altimeter, and airspeed values, which are within the range of each internal node on the path from the root to this leaf node, and the label is "cruising airspeed".

[0321] The resulting decision tree is called a comparison-based decision tree (CBDT) because each internal node performs only comparisons, which can be multi-way comparisons. It's important to note that the term "comparison-based decision tree," as used in this paper, differs from a classification decision tree because the latter strictly branches on categories, rather than on continuous values ​​as CBDTs can achieve. As described earlier, CBDTs are able to incorporate categories by treating category identifiers as continuous values. Conversely, a classification decision tree can be viewed as a restricted form of CBDT, where the only allowed form of comparison is equality comparison. CBDTs efficiently incorporate the discretization of the input by mapping each input value to a range (a discrete concept).

[0322] A CBDT can represent a set of tiles because each tile can be mapped to a path through the CBDT from the root to the leaf that is tagged using the tile's label, where each dimension appears as a node along that path, and the matching range involves the next child node on that path. A CBDT representation of a set of tiles typically reduces the required space significantly because dimensions and thresholds are specified fewer times compared to a list of tile descriptions. For example, as a simple example, if the root of a decision tree for an autonomous aircraft specifies airspeed and thresholds to determine which airspeed ranges map to which child nodes, then the airspeed and thresholds are specified only once for that decision tree, instead of once per tile as required if each tile were specified independently.

[0323] In one embodiment that only supports binary comparisons, multiple comparisons in an internal node can be replaced by a binary search tree that maps the input value of that dimension to the comparison of the right child node.

[0324] Note that the CBDT has the following property: Test cases can be hierarchically enumerated and are proportional to the number of leaf nodes in the CBDT. In particular, tests are first classified into subsets of tests for each child of the root of the CBDT, where each subset corresponds to the range mapped to that child. For example, as shown in FIG. 12, tests are first classified into two categories corresponding to the subset where the throttle is less than the maximum value (1204) and the subset where the throttle is equal to the maximum value (1206). Then, the subset of tests corresponding to the throttle being less than the maximum value is divided into a sub-subset corresponding to the altimeter reading being less than 200 meters (1208) and a sub-subset corresponding to the altimeter reading being greater than or equal to 200 meters (1210). This subdivision continues until each leaf node, such as (1212) corresponding to an individual test case.

[0325] Note that the restrictions on the CBDT are important for testability because if the decision tree nodes use any decisive expressions or calculations that are not equivalent to comparisons, the above test properties do not necessarily hold. For example, if the decision tree node uses the expression "input1 + input2 < fooThreshold" and input1 and input2 can take any floating-point values, then dividing the test cases into test cases below the threshold and test cases above the threshold does not eliminate input 1 or input 2 from further consideration in the resulting subsets. More complex decision expressions may be even more difficult to determine the partitioning. Therefore, each subset needs to consider a large number of values for each input, so by considering such subsets, the number of test cases may not be significantly reduced. Note that if an internal node branches on an expression such as "input1 + constant4 < barThreshold" that uses only one variable, this is equivalent to "input1PlusContant4 < barThreshold", where input1PlusConstant4 is a new threshold equal to "input1 + constant4". Therefore, the above test classification property still holds.

[0326] Decision tree generation. FIGS. 13A and 13B are flowcharts illustrating an embodiment of the process of generating a decision tree from a tile set. In one embodiment, the process of FIGS. 13A and 13B is performed by a control system such as (322j) of FIG. 4A, (372) of FIG. 4B, or any system (such as the system of FIG. 1).

[0327] In step (1302), a tagged tile set is received and in step (1304) it is established as the "current" tile set. If the current subset is not considered to be sufficiently reduced in step (1306), control passes to step (1308).

[0328] The dimension selected in step (1308) for branching at each internal node can be determined by developer input based on domain knowledge, experiential information, and / or by a conventional decision tree generation algorithm such as Amany Abdelhalim, Ilssa Traore, and Bassam Sayed's "RBDT-1: a New Rule-based Decision Tree Generation Technique". Each internal node performs a decision or branch based on comparing the input value of that dimension with a possible range.

[0329] After selecting a dimension in step (1308), a new internal node is instantiated in step (1310), creating the node as a child of the current parent and setting its dimension attribute to the dimension attribute selected in step (1308). The threshold is an ordered set of all thresholds for that dimension appearing in the tile subset of the child subtree.

[0330] In step (1312), the partitioning of the current subset of tiles is performed by creating a subset for each range of that dimension and adding to each subset those tiles in the current subset that match that range of that dimension. Each such subset is recorded with its parent node and its associated range relative to that parent for subsequent processing. For example, referring to Figure 12, the altimeter node (1202) is generated with a dimension corresponding to altitude and a range (1210) corresponding to 200 meters or more. In step (1314), the next “current” subset is selected, and control is passed to step (1306).

[0331] If the current subset is sufficiently reduced in step (1306), control is passed to step (1316), where a leaf node is instantiated for the current subset. If there are more subsets (1318) to process, control is passed to the next current subset in step (1314); otherwise, the process ends.

[0332] In one embodiment, assuming some criteria are available to select the order of dimensions in the tree for each path, the decision tree is incrementally built as tile labels are generated. For example, repeating an earlier example, consider a three-dimensional tile corresponding to a roll decision for dimensions—a logical target [13,14] labeled at 30 degrees, airspeed [150,200), and altitude [200,500). For this example, the first dimension of the tree is the logical target, then airspeed, and then altitude. When generating a label for the tile corresponding to 30 degrees, a path is defined in the decision tree from the logical target root node, which has child nodes corresponding to [13,14), where the child nodes are selected based on airspeed, and if the child node does not exist, it is created.

[0333] In this example, the child node has a sub-child node corresponding to an airspeed in the range [150, 200), and this sub-child node is compared based on altitude. This sub-child node itself can have another sub-sub-child node corresponding to an altitude in the range [200, 500). This sub-sub-child node is a leaf with a value or label corresponding to 30 degrees. After the path is generated in the decision tree, tile records can be discarded, thus avoiding the need to store a large number of labeled tiles, as required when pre-generating the tile set.

[0334] In this example, if a subsequent tile label is generated for a logical target in the range [13,14), the path uses the same child of the root as the earlier tile for its path. If the subsequent tile label has a different logical target, a new child node of the root is instantiated using the correct / different tile label. If a new tile is determined to be adjacent to an existing path and has the same label, the path for the new tile can be merged into the existing path. For example, if a new tile specifies a logical target [13,14) labeled at 30 degrees, an airspeed [150,200), and an altitude [500,1000), the decision node for the altitude of the first path described above can be expanded to have a range [200,1000) and mapped to the original leaf labeled at 30 degrees. Note that this situation may occur because a 500-meter threshold is required for different maneuvers or a set of subconditions. By merging in the path in this way, the number of nodes in the CBDT is reduced, with the improvement being the saving of space and time for lookups.

[0335] In this example, if a tile has a range in the current dimension that overlaps with the current range, the current node comparison can be modified to include additional ranges, including those corresponding to thresholds in the current dimension of that tile. For example, if the current tile has an airspeed range from [100, 300), then processing the current node comparison of [150, 200) is modified to process [100, 150), [150, 200), and [200, 300). Then, a path is generated using the current tile for each child node selected from these ranges.

[0336] Concurrently building a CBDT with tile generation can offer improvements in space saving and the processing of generating and storing all tiles. However, determining the optimal dimension to choose at each internal node based on the best information gain, as actually done in RBDT-1, may not be feasible. This is because not all tiles may be available for review when the next attribute or input dimension needs to be selected. Therefore, the efficiency of the CBDT obtained by this concurrent approach depends more on other means of selecting dimensions at each internal node, such as knowledge of the application domain. This knowledge can be assigned to the decision tree generator as a dimension priority. In many applications, it is feasible to store each complete set of tiles and then regenerate the decision tree for that set whenever the tile set changes.

[0337] If generated efficiently, the matching using a decision tree can be logarithmic as a function of the number of tiles N, rather than linear as with simple iterative methods. For example, if N = 1 million, or 1 million tiles, the search cost using a balanced binary decision tree is roughly 20 levels, while the worst case is 1 million using simple iteration. Improved search efficiency is important for, for example, enabling rapid responses to changes. This is also important for overall efficiency if decision-making is a significant part of the overall control processing overhead.

[0338] Decision tree implementation. Decision trees can be implemented by translating a sequence of nested "if...then...else" statements into executable code. For example, the tree in Figure 12 can be viewed as a flowchart implemented with partial pseudocode, as follows:

[0339] .

[0340] For those skilled in the art, this code generation is straightforward because the decision tree is equivalent to the expression tree used in standard compiler or interpreter implementations, and / or the basis for generating code, including various optimizations. One potential drawback of this approach is that the code size can be large, leading to processor i-cache misses and thus limiting performance.

[0341] An alternative implementation is a tree data structure implementation with a fixed process that traverses the tree data structure to locate the leaf corresponding to the input value. For example, each node could store a dimension along with a threshold and a list of pointers to its child nodes. This fixed process begins at the root node and then determines the dimension and the current input value associated with that dimension. It maps the input value to a range defined by a threshold Ti below or equal to that value and a threshold Ti+1 above it, and selects pointers to the child nodes associated with Ti and Ti+1, which are stored between these two thresholds. This approach may be more efficient because the number of processor d-cache misses is proportional to the depth of the tree (log N), which can be more difficult to implement given the generated code proportional to the number of leaves in the decision tree. Various further optimizations can be made using conventional techniques for both the data structure representation and the processing code.

[0342] The decision tree generated from the tile set differs from traditional decision trees generated from rules because the tile set guarantees that all valid values ​​for each input are covered. The generation of this decision tree also differs from traditional generation, which generates decision trees from statistical data rather than using a strict range from the tile set.

[0343] Sequencer Design and Implementation. Each sequencer (402) in Figures 4A and 4B is specified as part of the engineering design of the controlled system, or as part of the general operation of a category of controlled systems. In particular, the designer may identify the logical objectives required by the application, and how the controlled system can achieve these objectives using human / manual and / or automatic controllers, i.e., the sequence of steps for each objective.

[0344] For example, when the controlled system is an aircraft, designers consider airspeed objectives for the following: taxiing, takeoff, climb, cruise, approaching, and landing. For each objective, the engineering design outlines the sequence of steps to achieve it. This set of logical objectives can be expanded with additional application requirements. For instance, to avoid hazardous traffic conditions, an emergency climb, emergency descent, and / or emergency left / right roll might be necessary. High cruise speeds might be required to meet schedules when there are delays or attempts to avoid developing weather systems. Designers can also define the sequence of actions to achieve each logical objective. Otherwise, it cannot be guaranteed that the engineered system can meet every objective. In other words, unless the procedures for an emergency descent are defined within the constraints of the aircraft design, how can one know that the aircraft can handle an emergency descent?

[0345] It is important to note that in the case of automatic control, there is no need for arbitrary values. For example, a fully autonomous aircraft is not operated at will by a human pilot who may have old habits of flying at a specific airspeed. Instead, if the airspeed it chooses is a safe and efficient choice for the aircraft to achieve the operator's intentions, then the automatic control performance is considered sufficient. Therefore, if discrete values ​​are chosen to provide the key levels of functionality, as suggested in the aircraft example in this article, then tile labels are sufficient.

[0346] In a typical design process, engineers / designers also ensure that the controlled system can achieve the desired logical objectives and describe the constraints applicable in doing so. For example, different takeoff airspeeds may be required depending on passenger / cargo load, temperature, and runway altitude. Furthermore, there are constraints on the weight the aircraft can withstand during takeoff. Therefore, if the autonomous autopilot has access to these parameters via temperature sensors and an altimeter, its airspeed sequencer can calculate the takeoff speed using formulas employed by the designer. The sequencer can then generate a series of steps to achieve this takeoff speed and the time required to do so. The sequencer can then incrementally change the throttle at an acceptable rate and amount to converge to the specified airspeed or higher.

[0347] Other controlled systems may have similar logical objectives. For example, an autonomous vehicle could use a sequencer for lateral positioning that receives a logical objective to change lanes. This objective is logical, rather than specified as a precise value, because lane width can vary. The logical objective could include a desired beat / rhythm for achieving that objective, such as “normal” or “fast.”

[0348] As cited above, a sequence can be implemented by a cooperating thread or as an actual thread, allowing it to wait or pause for a specified time period and condition during the sequence. A sequence can also be implemented as a traditional state machine, where a state tracks the steps it participates in within the sequence. However, sequences are typically sequences of steps, and state machine structures are more general and therefore less indicative of the actual sequence being executed. That is, the transition function for a given state must be examined to determine the next step from that state, and it is also required that there are no multiple transitions leaving that state, making it effectively a sequence. A sequence can also be programmed as a suspendable thread that uses thread suspension and resumption to execute these steps, waiting for a fixed time period after each step, or otherwise until a given condition becomes true.

[0349] In general, the logical objectives of each actuator-level sequencer can be specified by the controlled system designer, including the means to achieve these logical objectives, in order to ensure that the system meets application requirements. Therefore, the associated sequencers can be implemented by those skilled in the art.

[0350] To make a sequence preemptible, it may be sufficient to release any resources associated with the current sequence. For example, if a data structure exists that stores the trajectory of the current sequence, that structure can be released or reused for a new sequence when the current sequence is preempted. Most of the state associated with the sequence can be the state of the controlled system as reported by input processing, and is therefore independent of the specific sequence being executed.

[0351] Sequences can evolve from simple to more complex. For example, for an autonomous vehicle traveling on a highway, a policy sequencer could initially be developed with a sequence that achieves the objective of switching to the overtaking lane, overtaking, and then merging back. Then, the "overtake-merge" sequence simply performs the overtaking maneuver and then decides to merge back. The entire process is then accomplished by switching to the overtaking lane and then performing the "overtake-merge back" sequence. The time required for the entire sequence is the sum of the times for each of the "switch to overtaking lane," "overtake," and "merge back" sequences. One reason for identifying the complete sequence of "switch to overtaking lane," "overtake," and "merge back" is that this sequence provides feedback data on the time cost of the entire sequence.

[0352] When a sequence achieves its objective, it can simply terminate, relying on a re-determiner to select a new objective and thus a new sequence. However, some objectives are continuous. For example, for an autonomous aircraft, level cruise is a sequence that continues indefinitely until it is cancelled; that is, it is not terminated. It is important to note that a level cruise sequence may require iteration to continuously detect whether level flight is impossible due to some malfunction; it cannot simply stop there.

[0353] In one embodiment, the sequences are made non-terminating because each sequence causes the controlled system to converge to its logical objective and then simply maintains it indefinitely until it is cancelled or a problem is reported. This is achieved by specifying the logical objective as absolute rather than relative. For example, a target airspeed is specified, rather than indicating an increase or decrease relative to the current airspeed. Therefore, after the controlled system achieves its logical objective, it stabilizes there and awaits a change in the logical objective, where there is a possibility that the new logical objective primarily indicates maintaining that logical objective. For example, if the selected sequence is "climb to cruise altitude," then once the aircraft reaches cruise altitude, it has already achieved its logical objective, so it simply continues control to maintain that altitude. Therefore, if the redeterminer changes to the "level flight" sequence, there may be no change to the aircraft's behavior at this point. In this approach, non-terminating sequences continue until they are cancelled and replaced by another sequence. Therefore, there is no gap or race condition between the end of the current sequence and the redeterminer selecting the next sequence.

[0354] As a dynamic adaptive time series, the sequencer can adjust its actual output control values ​​based on additional inputs besides the associated decision module inputs, such as as part of a step. For example, depending on passenger / cargo load, temperature, and runway altitude, different takeoff airspeeds may be required. Similarly, an automated HVAC system may have day and night labels, both of which are converted into actual configured temperature values ​​for each setting via a simple table. It is important to note that this conversion may not involve the decision itself. It primarily requires conversion using pre-specified calculations specified in the design of the controlled system and / or, in the case of HVAC, in its configuration.

[0355] In summary, the logical target / tile label is determined by the designer / engineer of the controlled system and the requirements of the application of that controlled system. The sequence of steps associated with the logical target, along with any associated parameters and constraints, is also specified.

[0356] Tile generation. Tile generation involves generating a set of labeled K-dimensional tiles for each re-determiner with K inputs. These K-dimensional tiles cover a range of possible K-dimensional vector input values, where each tile is assigned a label indicating the correct logical target to be provided to the associated sequencer for any combination of input values ​​within that tile. This range can be extended with margin, except for edges adjacent to singularities. This technique reduces the number of tiles required in the tile set and makes tile matching efficient. Now consider how to generate the tile set.

[0357] As an illustration, consider a fully autonomous control system that begins at the tactical level, then the strategic level, and then the meta-strategy level, each assumed to be implemented according to what is described herein, for example, a TSIR with a re-decisioner using one or more tile sets. Without limitation, the description of control in the meta-strategy, strategic, and tactical aspects using conventional examples of autonomous aircraft is merely an example presented; this approach is not limited to vehicle control. A meta-strategy typically identifies points in a P-dimensional space that denote the state of the controlled system. The navigation aspect to the denoteed destination attempts to bring the controlled system to a corrected state, recognizing the constraints on the controlled system and requiring "intermediate points" as intermediate states to reach the destination state from its current state, while recognizing the constraints and costs for the controlled system in doing so.

[0358] In other applications, such as the control of a manufacturing plant, a meta-strategy may be needed to navigate it from the production of one type of product to the production of different types of products, recognizing that there may be a required "midway point" between the current state of the production line and the next desired state.

[0359] In the following sections, it is assumed that the sequencer has already been implemented, or can be extended as needed to handle additional sequences. For example, for an autonomous aircraft, it will be implemented based on normal pilot procedures, the aircraft's flight commands, and engineering specifications and constraints.

[0360] Tactical layer tile set generation One task is to develop a tile set for each input logic target per actuator to provide the re-determiner section of the TCM for each actuator. This assumes that the sequencer section for each actuator in the controlled system has already been designed and implemented based on the specifications and engineering design of the controlled system.

[0361] In one embodiment, the method for generation begins with a single input logic target corresponding to the “normal” and / or common operation of the controlled system, a corresponding normal range of inputs, and a fixed / trivial re-determiner for the TCM of each actuator. The method for generation continues to incrementally expand the range of inputs and add additional input logic targets.

[0362] For example, for an autonomous aircraft, the "normal" operating mode is cruising at a reasonable altitude and a specified cruising speed, utilizing reasonable / initial inputs for cruising, in the absence of malfunctions or other traffic. Similarly, for an autonomous vehicle, "normal" operation is driving along a straight lane at a speed limit, provided there are no other vehicles or obstacles around and no malfunctions in the system. It's important to note that the logical goal of the "normal" input is often not the initial logical goal. For example, the initial state of an autonomous aircraft is being stationary and on the ground. However, the "normal" state is flying / cruising from one intermediate point to the next.

[0363] Because the redeterminer is initially fixed, the sequencer in each TCM is fixed when executing a single sequence. For example, for an autonomous aircraft, the sequence for a roll / aileron TCM during cruise would be to identify when the target roll differs significantly from the current roll angle and then adjust the ailerons to achieve that roll. Therefore, it relies on its sequencer to keep the roll angle close to the target roll angle. The sequence can also detect when there is excessive delay in achieving the target roll. However, it might not "determine" to effectively overtake the input logical target, such as resorting to level flight instead of a roll angle, because performing that target roll would be unsafe due to the presence of other traffic. Similarly, it might not decide to overtake the target roll because the current airspeed is too low or the aircraft pitch is too high. Any of these constitutes a different sequence in that TCM that might not be selected by the initially fixed redeterminer.

[0364] Initial Tactical Control Layer Tile Set. A fixed redeterminer for an input logical objective (e.g., "cruising in an aircraft") in each TCM can be replaced by a tile set that takes all or a subset of the K inputs as input dimensions and has tiles defined by the "normal" range of each input, which are labeled using the same sequence as the corresponding original fixed redeterminer.

[0365] For example, for an autonomous aircraft, there exists a cruise airspeed range, a normal roll angle range for cruise, and a normal altitude range for cruise. Multidimensional hyperrectangles or tiles, defined by these ranges on each of the corresponding input dimensions and labeled using the same sequence as the fixed redeterminer, will produce the same behavior in each TCM as the original fixed redeterminer. Therefore, it is feasible to generate an initial set of tiles—one initial set per TCM—that replaces the original fixed redeterminer with a redeterminer using the corresponding initial tile set for the current single input logic target.

[0366] It's important to note that these initial "normal" ranges may be conservative and not necessarily represent the extreme values ​​the controlled system is designed for for the current input logic objective. For example, the minimum airspeed for cruise might be 100 km / h, and the maximum cruise roll angle might be 30 degrees. However, a conservative safe subrange might be 150 km / h or higher, with roll angles less than 10 degrees. Using a safe subrange for each input avoids trade-offs that might require multiple tiles. For example, continuing the example above, 100 km / h might only be safe at roll angles of 5 degrees or less. An alternative is to define multiple tiles in a tile set to handle this trade-off. However, the next step in developing the TCM layer is to expand the input range, so a preferred approach is to start with a limited input range that requires only one tile and expand accordingly.

[0367] The normal range can include input scenarios in which multiple corrections to the controlled system are required. For example, for an autonomous aircraft, the scenario might be a slightly lower airspeed and a slightly lower altitude. In this case, sequencers for airspeed and for altitude / pitch might react concurrently to correct for both airspeed and altitude, resulting in reasonable control. As described herein, “reasonable control” includes efficient, stable, reliable, comfortable, industry-standard, established practices, and / or substantial control. In one embodiment, reasonable control is autonomous-level control that produces a response similar to that expected by a human operator in a similar scenario.

[0368] In contrast, continuing the example, if the airspeed is too low to attempt any increase in altitude, a time series might be needed for pitch TCM that waits for an increase in airspeed before attempting to correct for that altitude. In extreme cases, pitch TCM might temporarily sacrifice altitude to help restore airspeed and avoid a stall. These scenarios are outside the “normal” operating domain and are handled as part of an extended input range, as described below.

[0369] In one embodiment, each initial redeterminer tile set is extended using boundary tiles to provide complete coverage of the input range. Each boundary tile indicates an error and requests intervention when the input value is outside the initial input range. For example, continuing the example above, if the airspeed as input is below 150 km / h, the boundary tiles will match. Thus, a scenario where the control system is receiving an input outside its operating domain is detected. Using these boundary tiles is preferred over a simple limit check for each input because, in some cases, the boundaries are defined across multiple inputs. For example, the lower limit regarding airspeed might depend on altitude rather than a strictly constant boundary, as shown in Figure 8.

[0370] It's important to note that it's not necessary to incorporate all inputs at this initial stage. This is because if we assume the input is within a normal range, it's essentially fixed and therefore equivalent to a single "normal range" for that input; thus, if it's within that range, it won't change the redeterminer output. Furthermore, in some cases, for certain input logic targets, the input has no effect on the redeterminer output.

[0371] For example, for an autonomous aircraft with a set of tiles for gliding, the flap position may be irrelevant because its value does not change the decision for the tile set for any input set. Alternatively, no input is needed for a given tile set because the associated TCM may react to the effect of that input without treating it as a direct input dimension. For instance, for an autonomous aircraft, if the aircraft pitches down, this means the airspeed will increase, causing the sequencer mechanism for throttle to decrease the throttle to maintain the target airspeed, even without pitch as input. Similarly, pitching up means the airspeed will decrease, causing an increase in throttle, again without pitch as input. Therefore, for the airspeed TCM, responding to changes in airspeed without explicitly knowing the degree of pitch may be sufficient. Furthermore, if the airspeed TCM fails to control the airspeed, it should report an error. If the pitch is too severe, it may violate operational constraints beyond the aircraft's ability to control its airspeed, causing the sequencer mechanism to report an error. Therefore, although pitch is related to throttle settings, airspeed TCM can set the throttle based on airspeed process variables, without using pitch as the actual input to its tile set. Minimizing the input dimension reduces the number of tiles, and thus reduces the resource cost of matching and the difficulty of generating tile sets.

[0372] For the initial version, each input is constrained to a manually limited range, thus defining its initial operating domain. For example, for an autonomous aircraft, airspeed can be constrained to between the lower limit for cruise and the maximum normal cruise speed. Similarly, altitude can be constrained to a reasonable range for cruise; roll and pitch are also constrained to be relatively close to horizontal. Boundary tiles cover the supplement to the initial operating domain, as shown in Figure 8. Therefore, inputs outside the initial operating domain are matched to boundary tiles that indicate errors and the need for intervention. The definition of boundary tiles is feasible; it is the region in K-dimensional space not covered by the operating tiles. For example, if the initial tiles for an autonomous aircraft only cover airspeeds greater than or equal to 100 km / h, then the airspeed range from 0 to 100 is part of this boundary. Therefore, it is feasible to redefine the boundary tiles as the operating domain of the tile set is expanded.

[0373] By utilizing these constraints that define the initial operating domain, the initial control system can handle only the control of the aircraft from the initial conditions within these constraints. Therefore, the control system can be presented as an initial state indicating that the aircraft has an airspeed below the target airspeed, but not so low that some emergency action is needed to recover. Similarly, it can be presented as an initial condition where the aircraft is above or below the cruising altitude and then corrects for that discrepancy, but not so low or high that a different sequence, such as emergency altitude correction, is required.

[0374] It is important to note that the actual values ​​of the process variables used for the actuator are input to the TCM of its sequencer component, but the input to the tile set for that dimension can be a discretized difference between the process variable and the target for the input logic target, which may be expressed as a percentage difference, as discussed in the previous section.

[0375] This initial version of the tile set is feasible for each actuator and input logic target because each must be specified as part of the engineering of the controlled system, for example, the engineering of a controlled system for a human pilot to fly an aircraft. Such engineering typically involves determining the operational constraints for the controlled system and how it behaves when within those constraints. Furthermore, the initial input logic targets for the “normal” operation of the controlled system are specified most fully as part of the engineering of the controlled system.

[0376] For example, for an autonomous aircraft, "normal" operation involves cruising at a constant speed while continuously tracking towards the next midway point, assuming no system malfunctions and no external environmental conditions necessitating a change in normal flight behavior. For a manufacturing plant, "normal" operation occurs when an assembly line is started and running for a specific product without malfunctions or external obstacles.

[0377] At this stage, the control system can be tested to control the controlled system when it is within its initial operating domain. Typically, this initial test is performed using a simulation of the controlled system. However, this test may subsequently be validated on a real system.

[0378] For example, for an autonomous aircraft, assuming it allows an operator to control it locally or remotely, the operator can handle takeoff and climb to a reasonable altitude, attitude, and airspeed, and then transfer control to this initial version of the control system, thereby monitoring the aircraft and intervening if the control system behaves improperly or if certain faults or environmental conditions occur beyond the control system's handling capabilities. Assuming that inputs for detecting all such conditions are provided, this intervention is triggered by boundary tiles defining the operational domain, as described above.

[0379] Once this initial version of tactical control is implemented and tested, the next phase involves incrementally expanding the operating domain of the control system by incrementally performing the following steps: 1) adding input logical targets; 2) adding to the set of inputs; and 3) adding to the range of inputs. This process may also require introducing additional thresholds for the inputs. The expansion to additional input logical targets will be considered later.

[0380] Extending tactical control using additional input logical objectives may require adding a new set of tiles associated with the new input to each TCM, assuming each input logical objective in each TCM has a separate set of tiles. For example, for an autonomous aircraft, its initial tactical layer might support the "cruise" logical objective, as described above. A "climb" input logical objective could be added to extend this initial operational domain. Each actuator TCM may need to be extended to be able to receive the "climb" logical objective as input from the policy subsystem.

[0381] Some input logical objectives may be adjacent to existing logical objectives in the sense that the system's engineering supports a transition from a logical objective to an adjacent objective. For example, for an autonomous aircraft, the logical objective of "climb" is adjacent to "cruise" because the aircraft typically climbs to its cruising altitude and then transitions to cruise, where it simply maintains the cruise altitude. Similarly, "descent" is adjacent to "cruise" because it follows "cruise" and is used to lower the aircraft to a lower altitude, typically in preparation for landing. In contrast, "takeoff" is not adjacent to either "cruise" or "descent" because there is no permitted transition between "takeoff" and these other logical objectives.

[0382] A preferred approach is to begin with an initial version of the tactical control layer that handles the normal operating domain, and then incrementally add new logical targets that are adjacent to existing supporting logical targets. By adding neighboring logical targets, it is feasible to test the transitions between these logical targets. Furthermore, due to this adjacency, there must be overlapping input combinations between the new logical targets and those input combinations of existing logical targets. For example, “climbing” to a target altitude is the same as “cruising” when the difference between the target altitude and the current altitude is relatively small.

[0383] Therefore, to add a new logical objective, the first step is to identify the initial operating domain of that logical objective. For example, for "climb," the operating domain could be an airspeed equal to or higher than the takeoff speed, an altitude of 200 meters or higher and much lower than the aircraft's maximum altitude, plus near-horizontal pitch and roll. This is essentially the "normal" scenario for the "climb" logical objective.

[0384] Assuming this normal operating domain, the next step is to define a corresponding tile set for each actuator TCM. For example, the airspeed TCM might have a tile set for a logical "climb" target, mapped to a sequence that initially slightly increases airspeed to anticipate an increase in pitch and contribute to the climb. Then, after maintaining airspeed at the new pitch, it decreases to the target airspeed. The elevator tile set for this target maintains the elevator at 0 degrees until the airspeed exceeds the target airspeed. For the aileron TCM, its primary function is to keep the aircraft level.

[0385] Continuing this example, the "climb" logical objective can be expanded to include "turn," allowing the aircraft to climb and turn simultaneously. For a given TCM, this "climb and turn" logical objective can use the same sequence in the sequencer, treating a no-turn case as a 0-degree turn, and similarly for a no-climb case. The airspeed TCM tile set used for "climb and turn" can be aware that a non-zero turn is involved in the climb by having inputs corresponding to the degree of turn (e.g., perhaps discretized to four ranges). This allows it to anticipate that more airspeed will be needed as the aircraft will tilt and climb.

[0386] The initial operating domain corresponds to the normal state for the input logical objective, such that one or more sequences are those specified by the engineering design of the controlled system and are therefore known in advance. For example, for an autonomous aircraft, the takeoff sequence is specified as part of the aircraft engineering or otherwise adapted based on a known flight sequence. The initial operating domain does not include inputs that provide a reason for aborting the current sequence and / or switching to another sequence. For example, for the "takeoff" logical objective, the initial operating domain may not include inputs indicating external obstacles or system failures and assumes an infinite time for takeoff. In some cases, the initial operating domain may be handled by a single tile corresponding to the takeoff sequence, optionally plus boundary tiles defining the operating domain.

[0387] At this initial stage, the TCM can be extended as described above to handle the input logical objectives used in the normal operation of the controlled system. This is feasible because normal conditions are defined as having no faults and no externalities that complicate the selection of sequences. Furthermore, for the TCM layer, there exists a small, well-known set of logical objectives to support normal operation. For example, in normal operation, an aircraft cruises, climbs and turns, descends and turns, takes off and lands, taxis and stops. Therefore, it is generally feasible to incorporate the normal logical objectives handled by the TCM layer so that the controlled system can undergo its normal operational behavior. In particular, at this development stage, testing can be performed by executing the control system through the normal sequence of takeoff, climb, cruise, descent, landing, and stopping.

[0388] Tactical control is expanded by adding to the input range. The next step is to expand the range of each input so that it can handle more operational domains required by the application. Adding to the input range includes: adding new tiles to cover the additional range, expanding the range of one or more existing tiles to cover that range, and / or a combination of both. For example, for an autonomous aircraft, if the airspeed is expanded to a lower value that would then introduce a stall condition, additional tiles may need to be labeled to indicate the stall handling sequence. On the other hand, if the airspeed operating range is expanded to a higher value, it is possible that the same sequence is sufficient to handle it, so existing tiles can be expanded so that their airspeed range includes that higher value.

[0389] To extend the input dimension ID to a new maximum value MV, the first step is to determine, for a tile T with the current maximum value of the input dimension ID, whether it can be feasible to extend to this new maximum value MV, or whether one or more additional tiles are needed. If the label of vertex V on the smallest edge of the dimension ID of tile T has a different label than that of the potential vertex V', then one or more additional tiles are needed, where the potential vertex V' has the same coordinate values ​​as V, except that the value of the dimension ID is replaced by the newly proposed maximum value MV. Therefore, one technique is to use this vertex V' to evaluate the controlled system and determine the preferred tile label for that vertex. If this preferred tile label is different from the tile label of tile T, a new threshold may be required.

[0390] The new threshold can be determined (without restrictions, and as a method) by a binary search between the old maximum and the new maximum, which uses controlled system simulation / prediction to evaluate the tile labels for each candidate threshold to find the threshold where the tile labels change from one value to another.

[0391] In one embodiment, the technique for determining the new threshold includes the following steps:

[0392] 1. Set the candidate threshold CT to be halfway between the old maximum value and the newly proposed maximum value;

[0393] 2. Evaluate the controlled system for vertex V at the candidate threshold, i.e., use the value of vertex V, where only the values ​​in the extended dimension are replaced by CT;

[0394] 3. If the evaluation indicates that the candidate threshold places the system in a known, characterized singularity, then the CT threshold is corrected to be at the edge of that singularity, and the threshold for that "edge" is returned;

[0395] 4. If the selected tile label is the same as the original label, and if the CT plus margin evaluates to be the same tile label as the tile label for the proposed maximum value, or if the CT plus margin is greater than the proposed new maximum value, then use the CT and exit. Otherwise, if it evaluates to a label different from the original label and different from the label for the proposed maximum value, return the CT indication as the new lower bound. Otherwise, adjust the CT to half between the newly proposed maximum value and the CT, and return to step 2;

[0396] 5. If the selected tile label is different from the original label, the CT is corrected to half of the previous maximum value and the original CT, and the process returns to step 2.

[0397] If more than one threshold is required between the old maximum and the newly proposed maximum, the search returns a new lower bound for that search. The search is then repeated for the range from this new lower bound to the newly proposed maximum, but the result is processed as a replacement for the new maximum, not as a new threshold. That is, the result indicates the threshold at which a tile label changes from a new label to yet another new label. Therefore, it ensures that the final new maximum used requires only one threshold between the old maximum and the finally selected new maximum. In effect, the method finds a threshold that allows it to transition from one tile label to another, and if necessary, reduces the proposed maximum threshold, so that at most one threshold is required between the old and new maximums, or otherwise indicates that no threshold is needed.

[0398] If the input range is expanded to a new minimum value, a similar method is used to determine a new threshold, thus adapting the above steps to the minimum value.

[0399] Figures 14A and 14B are flowcharts illustrating embodiments of the process for expanding the input range. In one embodiment, the process of Figures 14A and 14B is performed by a control system, such as (322j) of Figure 4A, (372) of Figure 4B, or any system (e.g., the system of Figure 1).

[0400] In one embodiment, when expanding the input range for a given input dimension and TCM, the steps are illustrated in Figures 14A and 14B. Assume the minimum value of the input range to be expanded is being expanded:

[0401] 1. In step (1402), one or more neighboring tiles adjacent to the extended range are extended to include the new range, thereby marking each tile as a new extreme value with respect to the extended dimension as tentative.

[0402] 2. In steps (1404) to (1426), the process iterates through these provisional tiles; that is, for each provisional tile selected in step (1404), it is determined in step (1406) whether to process another tile. If not, the process in Figures 14A / 14B ends; otherwise, control is transferred to step (1408):

[0403] A. In step (1408), the next dimension is obtained, which may be a provisional tile for the new processing and the first dimension for said dimension. For the current dimension selected in step (1408), it is determined in step (1410) whether to process another dimension. If not, control is transferred to step (1412); otherwise, control is transferred to step (1414):

[0404] 1. In step (1414), the above technique is used to determine the thresholds for the minimum and maximum vertices of the current dimension, and if necessary, the new extended range minimum is reduced, so that there is at most one new threshold for each minimum and maximum value;

[0405] 2. In step (1416), determine whether there is a significant difference between these thresholds for the minimum and maximum values ​​of the current tile. If not, control is transferred to step (1408) for the next dimension; otherwise, control is transferred to step (1418):

[0406] i. In step (1418), a new threshold is selected between the minimum and maximum values ​​of the current tile in the current dimension;

[0407] ii. In step (1420), the current tile is split (divided into two parts) in the current dimension based on the new threshold.

[0408] iii. In step (1422), a new threshold is determined in the extended range dimension. For example, the new threshold is used for the selected dimension to evaluate a controlled system with inputs corresponding to a vertex, so as to select a new extended dimension threshold for that vertex;

[0409] iv. In step (1424), if the new extended dimension threshold is significantly different from the threshold for the minimum value, the segmentation is recursively repeated for the tile in the current dimension between the previous minimum value and the new threshold with respect to the selected threshold. The same operation is performed recursively, for example, comparing the threshold with the threshold for the maximum value, until the thresholds are no longer significantly different; and / or

[0410] v. In step (1426), a new extreme value in the extended dimension for the first segmented tile is defined as the extreme value of the threshold at the maximum value of that tile. That is, the segmentation of the original tile is extended to this new threshold, and the first segment is the current tile. A new extreme value in the second segmented tile is defined as the extreme value of the threshold in the extended dimension for this new threshold. The resulting tile is marked as no longer provisional for the current dimension;

[0411] B. In step (1412), mark the current tile as no longer provisional;

[0412] 3. When the process in Figure 14A / 14B is completed, output the resulting set of corrected tiles.

[0413] Informally, this technique can find a threshold in the range of extreme values ​​for each dimension other than the extended dimension for each tile adjacent to the extended dimension, and segment the tile if necessary, such that the resulting tile reasonably and accurately tracks the boundary between the current label on the tile and the newly labeled tile, thereby complying with the allowed restrictions on the input.

[0414] It's important to note that this means each new threshold is selected per expanded tile, and therefore specific to the input range associated with that tile. Consequently, the tile set can have tiles of various sizes based on the different ranges generated by these selected thresholds. It's not limited to the row / column structure of a lookup table, where each entry in a row can have the same range for the input dimension associated with that row.

[0415] The new threshold in the current dimension during the above iterations might simply lie between the minimum and maximum values ​​of the range in that current dimension. Alternatively, it could be selected by applying specific knowledge. As another alternative, it could be selected by searching for the furthest threshold for the minimum in the current dimension before a significant difference exists in the threshold. The latter may be more expensive in evaluation but can yield better results in minimizing the number of tiles. Because the difference between the extreme values ​​on the corrected tiles decreases in each iteration, because singularities are known and can be avoided, and / or because it has a finite number of vertices with new extreme values, a given set of original tiles might only require a finite number of iterations to terminate with the method described above.

[0416] This technique allows for situations where the segmentation is needed at points different from the previous limits of the input range. For example, for an autonomous aircraft, if the lower limit of airspeed is extended from 200 km / h to 50 km / h, and the stall speed is actually 80 km / h, then the new threshold should be at 80 km / h. Therefore, after extending the existing tile from the lower limit of 200 km / h to 50 km / h, the evaluation of the new extreme value of 50 km / h provides a different label than the upper limit of 500 km / h for that tile. Thus, the technique finds the new threshold at 80 km / h and segments the tile into: a tile extending from 80 km / h to 500 km / h with the previous tile label, and a tile with an airspeed range from 50 km / h to 80 km / h. It labels the latter tile as provisional regarding the airspeed with the new extreme value of 80 km / h. The technique then iterates on this new provisional tile.

[0417] Each of these new thresholds may be specific to the input dimension under consideration and does not require modification of any other tiles. Adding the new threshold will split the specific tile considered for this case into two tiles, one with one label and the other with another label. The new threshold does not necessarily imply that other tiles using the original range in that dimension also need to be segmented. Therefore, an additional threshold for this dimension may be specific to a particular range for other dimensions.

[0418] Figure 15 is an illustration of the expansion of the input dimension range. Similar to the example of boundary tiles shown in Figure 8 with a simplified example of an autonomous aircraft with the input logical objective "cruising", Figure 15 illustrates the boundary between the cruise operating domain and the outside of that operating domain, taking into account two dimensions: altitude and airspeed. Figure 15 also illustrates the expansion of the input dimension range for altitude.

[0419] In the example shown in Figure 15, the previous minimum limit for altitude is labeled as the darker shaded tile (1502), and since there is no difference in airspeed values, the "Cruise" label is applied to a single, larger darker shaded tile (1502). The altitude range is then extended downwards to the rightmost low value indicated by the top of the tile (1504). This stage establishes that the threshold for low airspeeds differs from the threshold for high airspeeds, and the difference between these speeds is significant. Therefore, a threshold (such as between half) is determined for airspeeds between these two values, which here corresponds to the top of the middle tile (1506) in Figure 15. The airspeed difference between this value and the lower minimum is determined. If the difference is not significant, the airspeed value at this threshold defines the dividing point of the provisional tile. If this is the case for this example, the first three shared tiles from the left (1508), (1510), and (1512) would be a single tile. However, in this example, the difference is significant, so another threshold is determined, and the tile is further segmented into three tiles on the left: (1508), (1510), and (1512), as illustrated in Figure 15. Similarly, the airspeed difference between the value at this threshold and the maximum value is determined, and if it is significant, the same segmentation is applied, resulting in the second tile (1510) from the left. In this simple example, there are only two dimensions. In general, in the case of more dimensions, each of these segmented K-dimensional tiles can be further segmented in additional dimensions.

[0420] The definition of a “significant difference” can be application-specific, and even deployment-specific. The larger the difference is before it is considered significant, the fewer tiles are required, and the less optimal the control. The portion above curve (1520) in Figure 15 indicates the region of less optimal control. The tighter the boundary regarding significant differences, the more optimal the control, but the more tiles are required. Therefore, applications may need to trade off tile overhead for control optimality.

[0421] The controlled system may not be expected to be in such boundary conditions frequently, or even if it is, it may not remain in such conditions for long periods; therefore, an approximation of the optimal control is usually sufficient. Furthermore, under dynamically changing conditions, some uncertainties in the actual state of the controlled system and the simulation fidelity mean that even the optimal decision based on simulation / prediction may not be significantly better than the approximation used. Finally, it may be unreasonable for a human driver or operator to meticulously track the exact trade-offs between multiple inputs, so there is no need to achieve an exact curve shape equivalent to or better than that of a human driver. Essentially, a K-dimensional tile (hyperrectangle) at the threshold approximates a K-dimensional continuous surface. However, this K-dimensional continuous surface is usually unknown. Instead, the points on this surface are determined by the controlled system simulation / prediction and engineering knowledge about the controlled system design and related technologies.

[0422] This approach, which maximizes the size of tiles in a tile set under sufficient control, offers improvements in reducing the space required for the tile set and lowering matching costs. This also means that the re-decision set can be limited to transitions between neighboring tiles in the tile set. This limitation reduces the amount of testing, as the control system does not need to be tested for transitions between tiles, since an edge of a tile is actually part of its neighboring tiles. This approach also has the improvement that transitions are less frequent compared to using smaller tiles, allowing pre-matching to avoid full decision tree matching most of the time. It further has the improvement that the frequency of preempting the current time series for different time series is reduced, thus providing efficient and stable control under normal conditions.

[0423] In some cases, the benefits of using simulation to support labeling tiles representing input values ​​outside the operational constraints of a controlled system are limited. This is because when outside operational constraints, the engineering design may not have an accurate understanding of the controlled system's behavior, and in such cases, simulation may not provide an accurate indication of the controlled system's behavior. In some situations, an understanding of the behavior exists when the input indication slightly exceeds the operational constraints, but no understanding exists when it exceeds them significantly. In such cases, additional thresholds can be added, creating tiles corresponding to both situations, and simulation can be used to determine the appropriate labeling for the slightly outside-constraint situation. As with the introduction of the new thresholds described above, these thresholds may be specific to a given set of tiles and the tiles under consideration, and do not require modification of any other set of tiles. Expanding the input range may mean modifying the boundary tiles that define the operational domain of the control system.

[0424] For some controlled systems, it may be important to extend the range of some inputs before others. For example, for autonomous aircraft, airspeed is crucial for aircraft control. Therefore, the airspeed operating domain of the control system can initially be extended from the normal range to speeds closer to stall, allowing the airspeed TCM behavior to be determined outside the normal range before extending other operating domains. For instance, when the pitch TCM is later extended for altitudes below the normal cruise range, the pitch control TCM selection can be evaluated, incorporating how the throttle would behave if the operating domain included low airspeed. In particular, if both airspeed and altitude are significantly below normal, the pitch TCM may need to delay increasing pitch until airspeed has increased. Similarly, if roll is excessive, it is best to correct it before attempting a significant climb.

[0425] Generally, a decoupled TCM recognizes that there are indeed couplings between them and the same controlled system through their connections. Expert knowledge of these couplings between different actuators can be used to provide tiles that function according to this coupling, as illustrated in these examples. It is important to note that because computer-based controls can often react to changes in the controlled system 10 times or even 100 times faster than human operators, relying on reactive control rather than anticipatory control is often feasible.

[0426] Furthermore, autonomous control can be sensitive to input changes that are much smaller than what a human operator might perceive. For example, an increase in pitch typically implies a decrease in airspeed unless the throttle is increased simultaneously. A human pilot might anticipate needing more throttle. However, in the case of autonomous control, in this example, the throttle TCM reacts to a decrease in airspeed and increasing the throttle is usually sufficient because it can react to small changes in airspeed, and within a tenth of a second or less, while human reaction time is typically considered to be at least ten times longer.

[0427] The limitations of expanding the input range may mean that, in some cases, the re-determiner needs to select a sequence different from the one used before expansion. For example, for an autonomous aircraft, if the logical objective is "cruising" but at an excessively low altitude, the tile label should correspond to an emergency climb sequence. Assuming this sequence is already implemented in the associated sequencer, using the corresponding tile label may suffice. Otherwise, guided by engineering and operational constraints, it may be necessary to expand the sequencer to provide this additional sequence.

[0428] Testing the tactical control layer. Prediction / simulation of the controlled system can be used to validate the selection of tile labels; that is, the simulation and prediction mechanisms of the controlled system, along with the objective function, are used to determine how well the controlled system performs under that control.

[0429] In one embodiment, the tile label is evaluated at each vertex of the tile. One form of testing is to start the simulation in the scenario corresponding to the input value for each vertex and then simulate backwards from there. This test can also initialize the state present in the sequencer. In particular, if the sequencer is using extrapolation to rely less on low-latency feedback about its process variables, the extrapolation logic can also be initialized. Because the extrapolation logic is expected and required to be fairly accurate, it is often sufficient to initialize it so that it predicts the initial conditions. For example, in the case of testing an autonomous aircraft, given a throttle setting and aircraft attitude, the airspeed of previous time steps can be initialized using a value that predicts the current airspeed as an approximation of the initial airspeed. The handling of scenarios that include events such as faults that significantly disrupt extrapolation can be tested by allowing faults to occur at the start of the simulation or after the simulation has started. These fault or anomalous tests may necessitate running evaluation simulations for an additional number of time steps.

[0430] In one embodiment using pre-matching, each vertex is extended with an additional margin or overlap that will be used in the pre-matching. For illustration, if the overlap is 10%, and if the airspeed range for the tile is 200 to 500, the label selection is validated to operate at airspeeds as low as 180 instead of 200 and as high as 550. Due to the continuous nature of the controlled system, successful evaluation at overlapping vertices may mean that the controlled system performs adequately anywhere within that extended tile. It should be noted that, as described earlier, if the margin leads the controlled system into regions corresponding to singularities, then the margin is not applied in the pre-matching.

[0431] In one embodiment, the test uses at least in part the same simulation / prediction mechanism as that used to determine the thresholds as described above, except for those determined from the engineering design. Therefore, the test primarily detects errors in the engineering specifications or simulation. However, the test may execute a longer sequence of steps than the sequence used to determine the thresholds. Therefore, when the simulation runs over multiple time steps, the test may sometimes detect problems with the thresholds or tile labels that manifest themselves at those time steps.

[0432] For example, for an autonomous aircraft, the peak of a near-stall condition might show that the airspeed TCM responds correctly in one or a few time steps, as does the pitch TCM. However, more time steps may be needed to identify when the aircraft cannot avoid a stall under normal airspeed TCM handling at throttle. Using the input range extension method described above, but with a longer simulation evaluation time, this situation can be re-evaluated to correct the tile set. Therefore, testing can be seen as a means of identifying when a longer simulation is needed for a given scenario to evaluate the behavior of the controlled system using candidate thresholds and tile labels.

[0433] When testing with longer simulations, and where the tests must be long enough for the controlled system to achieve its objective under appropriate conditions, the tests can be used to determine the expected time required to achieve the input logic objective. If this expected time indicates that the sequence takes too long to achieve its objective, this can serve as the basis for further refining one or more tile sets used by the sequence.

[0434] Additional inputs are used to extend tactical control. Figure 16 is a flowchart illustrating an embodiment of the process of adding inputs to a tile set. In one embodiment, the process of Figure 16 is pre-executed by a computing system, such as the computing system shown in Figure 1.

[0435] To add an input to the tile set TS, the first step (1602) determines whether the new input I is relevant to the tile set TS. If the new input I is irrelevant, the process is complete. Otherwise, control moves to step (1604) to determine the range of assumptions for I in the existing version of TS. This is partly because it is possible that a certain range of values ​​is assumed for the input under the current tile set TS. For example, for an autonomous aircraft, if the new input indicates the presence / absence of other traffic ahead of the aircraft, the initial TCM might assume that there is no traffic nearby ahead (otherwise, the initial TCM would have a provision to take evasive action).

[0436] In step (1606), the tile set TS is modified to include the input as a new dimension, having the same labels as previously assigned to tiles with this new input dimension within the assumed range. For example, continuing the example above, the assumed range for traffic ahead might be "more than five miles," meaning, in the distance. The existing tile set then has the added input dimension, where its range corresponds to traffic five miles or more ahead. Similarly, if the new input indicates a possible fault condition, the assumed input will correspond to "no fault." Tiles corresponding to this input that are outside the assumed range are labeled as boundary tiles, indicating that the control system may not be able to handle these situations and that they are outside the current operating domain. In effect, this step makes the operating domain of the tile set explicit in the case of the new input.

[0437] In step (1608), the operational domain of the input is expanded using the method described above. For example, if the input is a value indicating the presence of incoming traffic ahead of the aircraft, getting closer in traffic requires a sequence that changes course to avoid the traffic. Getting even closer in traffic requires a sequence that takes emergency action to avoid the traffic, such as suddenly driving to a lower altitude. The operational domain can be expanded incrementally until it has been expanded to correspond to the operational domain required by the application. Similarly, an input indicating the detection of an internal fault is expanded to include common fault conditions, where the associated tile indicates the optimal sequence to select when the common fault is detected.

[0438] It may not be necessary to expand the scope of a new input to its full final range before incorporating other inputs. In one embodiment, a normal, simple operating domain is preferred as a starting point, where all inputs are incrementally expanded to support a slightly more complete operating domain, which is expanded incrementally over time, and intervention is relied upon when the control system detects it outside the current operating domain. That is, starting with coverage of the normal operating domain, the range of inputs can be incrementally expanded to handle more abnormal / unusual scenarios, thereby reducing the frequency and urgency of intervention.

[0439] Additional actuators can be used to extend tactical control. The initial version of the tactical control layer does not need to include a TCM for every possible actuator. Furthermore, new instances of the controlled system may include additional actuators not present in previous versions used to develop the control system, for example, in two cases: 1) the new actuator is necessary / critical for the control of the controlled system; and 2) the new actuator is an optimization to improve the performance of the controlled system. As an example of the latter in the case of autonomous aircraft, the initial version may not include a TCM for flaps. Flaps may not be necessary for the operation of the aircraft, but they allow the aircraft to take off and land more efficiently and on shorter runways.

[0440] For the new actuators required to control the controlled system, the controlled system can be a new system with different simulations. Therefore, the development sequence of the control system described above can be repeated. This is because previously validated controls may be ineffective in the case of a new controlled system. In some cases, this situation can be transformed into one that can be handled as an optimization. For example, for an autonomous aircraft, if the initial version was developed for a single-engine aircraft and is therefore a throttle, the control system can be adapted to a twin-engine aircraft by replicating the throttle TCM for each engine and defining the operating domain as when both engines are performing correctly. Thus, if either engine fails to perform correctly, the engine failure input indication is set to true. Then, full adaptation to both engines can be handled as an optimization; that is, handling the case where one engine fails while the other does not is a better approach than treating it as a case of both engines failing.

[0441] When a new actuator is optimized, an existing tile set can be considered as having that actuator in a fixed or default setting. For example, using a flap example, the flap can initially be fixed in a non-deployed state. Then, after the initial version has been developed, a TCM can be added for that actuator using a tile set that keeps the flap non-deployed. The tile set can then be expanded to deploy the flap based on an input indicating a scenario where such deployment would improve the operation of the controlled system, determined similarly by engineering knowledge of the controlled system and / or by simulation of the controlled system. For example, in response to a takeoff input logic objective, the flap TCM can be expanded to deploy the flap halfway.

[0442] As an actuator other than the initial actuator, the absence of this TCM is equivalent to the presence of a TCM but with one tile always selecting a neutral or undeployed sequence. Furthermore, additional actuators, when active, typically only alter the timing of operation and, possibly, the operational efficiency. For example, in the case of aircraft flaps, when deployed, they provide additional lift during takeoff and additional drag during landing to reduce descent and airspeed. Other actuators (such as rocket boosters during takeoff and reverse thrusters during landing) have similar properties. For autonomous vehicles, a TCM used for explicit gear control can downshift to decelerate and thus reduce wear on the brakes.

[0443] To add more details, the first step could be to introduce a new TCM for the additional actuator, which has a tile set for each input logic target, and the tile set selects either non-expanded or default sequence in each case.

[0444] The second step could be identifying the input scenario, which can be active or deployed. For example, for flaps on an autonomous aircraft, the tile set could introduce tiles for the "landing" input logic objective, which matches when the aircraft is at the appropriate altitude and airspeed, causing the flaps to deploy at that point and remain deployed until the aircraft stops. Therefore, the tile set is defined using input dimensions corresponding to airspeed and altitude. Tiles covering the input space outside the area where the flaps can deploy remain non-deployed by default.

[0445] The third and / or subsequent steps can identify additional input scenarios, such as a range of input values, where the actuator and what it controls can provide improved operation. This process may continue beyond the initial version of the control system, thus providing refinement in subsequent versions.

[0446] The introduction of a new actuator introduces a new potential input, namely a process variable associated with the new actuator. This potential new input can then be evaluated using the methods described earlier for adding inputs to see if it is related to any other TCM.

[0447] A situation that can affect the behavior of a controlled system is when a new actuator behaves improperly and functions incorrectly. For example, in the case of an aircraft, if an actuator targeting reverse thrust activates reverse thrust during takeoff, this could have serious consequences for the aircraft. Instead of treating this type of fault as an input to all tile sets, a separate fault detection system can be used to identify the root cause of the fault and provide fault indications to the relevant TCM, as described above.

[0448] Improve input thresholds. Expanding actuators, logic input targets, and input ranges may mean opportunities to improve control system performance by introducing additional thresholds on the inputs in some cases, leveraging the refinement of tile label correspondences.

[0449] This refinement can be done by re-evaluating the system performance in the region of input values ​​of interest. For example, the tile set for the "takeoff" input logic objective in the TCM can be refined using an additional time threshold for takeoff, as the addition of flaps provides greater lift, allowing takeoff on shorter runways. To reliably utilize this refinement / optimization, if there is a possibility of takeoff without flaps deployed, these tile sets can be expanded to accept inputs indicating that flaps are deployed. It is important to note that this input may simply be a Boolean value indicating whether flaps are deployed, and not necessarily all values ​​associated with flap position.

[0450] In one embodiment, the additional actuator functions to alter timing for a given sequence. For example, flap deployment can reduce landing airspeed and thus reduce the time from landing to decelerating the aircraft to taxi speed. In one embodiment, it may be known in advance that a given actuator will not directly affect the decisions and activities of another TCM, so the entire step of reassessing the associated tile set is skipped. For example, flap deployment may have no practical effect on yaw, so there is no need to expand the tile set used to control yaw.

[0451] There may be incremental refinements to the granularity of existing inputs. For example, for an autonomous aircraft, an initial engine failure input might simply indicate true or false. However, this could be refined to no failure, power loss, and overheating, thus identifying a situation where the engine is still providing thrust but is experiencing a problem. To add this refinement, additional ranges corresponding to the new values ​​are added to the associated input dimension, and new tiles are added for each input combination, with different tile labels for each combination. For existing tiles where the same label exists for power loss or overheating, the tile range for that input dimension can be expanded to include values ​​for both possibilities, assuming they are numerically adjacent, such as "corresponding to 1 and 2," thus the range is [1, 3].

[0452] In practice, the benefits of refining the thresholds obtained from the overall process described in this section to improve efficiency may be limited. This is because the redeterminer determines the logical objective; the sequencer and the selected sequence are designed based on the system's engineering to achieve that objective as efficiently as possible. Furthermore, the efficiency of a controlled system is typically determined by its operation within the normal operating domain. For inputs in the domain, the tile set selects the corresponding "normal" operation sequence. The sequencer can be designed according to the engineering of normal operation.

[0453] Therefore, the primary areas for improvement are likely to be the boundary regions. For example, the set of tiles used to control flaps could be modified to deploy under stall conditions to provide additional lift at low speeds. Thus, these improvements are more likely to provide better behavior under harsh conditions in practice than strict general improvements.

[0454] Summary of the Tactical Layer. Generally, the development of the tile set for the tactical control layer can begin with a single input logical target and a restricted operating domain confined to a safe, normal scenario. It can then be incrementally expanded to handle more input logical targets, each initially restricted to its "normal" operating domain. When an input is outside this supported operating domain, boundary tiles defined around the normal operating domain cause intervention. The operating domain of each input logical target can then be expanded by extending the range of inputs that the logical target handles until it handles the entire operating domain required for the controlled system. Adding new inputs can often be done without interfering with existing control, as the existing control implementation assumes the range of that input. Therefore, the input can be added, and the range can then be expanded as described above. The addition of new actuators and associated TCMs can be handled similarly.

[0455] In practice, an initial version of a product may have a limited operating domain. It can then be incrementally expanded in subsequent versions of the control system. For example, the input space of non-boundary tiles can be incrementally expanded in terms of the range of inputs considered and the number of dimensions. A key aspect is that these incremental expansions do not require retesting the previous tile set, because the previous tile set remains unchanged except when new neighboring tiles have the same tile label, which might lead to tile expansion.

[0456] For autonomous operation, one improvement is a system that allows a human operator to take over after some notification, but without significant urgency, without the expectation of continued operator attention, and without significant malfunction (unless it is a catastrophic situation). That is, when the system eventually falls outside its operational domain, the control system can bring it into a safe state, allowing the operator time to be notified and take control. As described above, extending from normal to more unusual situations improves the level of autonomy, as measured by the frequency and urgency of intervention.

[0457] Strategy layer In one embodiment, the policy module is designed and / or implemented following a similar approach described for the tactical layer, including similar approaches when developing one or more associated tile sets. Unlike the tactical layer, a single policy module or subsystem is often sufficient for the entire controlled system. This is because the policy applies to the entire controlled system, rather than individual actuators.

[0458] In one embodiment, the policy module is constructed as the TSIR structure described herein, similar to the structure of modules in the tactical layer. The sequencer implements various policy sequences that a human driver might apply. For example, a strategy to avoid a storm might be to fly further left around the storm and then return to a flight path toward the next midpoint once the storm has passed. Essentially, each time series is a policy, constructed as a time sequence of steps. In one embodiment, there is a clear distinction between the steps of deciding a policy and executing it, and / or there are explicit provisions for rapidly and repeatedly re-evaluating previously decided policies / sequences and changing to different policies / sequences if changes in the input necessitate such a change.

[0459] As a first step, a sequencer can be implemented based on the sequence and control determined as part of the controlled system design. The initial version of the re-determiner is implemented with a single input logical objective for "normal" operation, and thus a single set of tiles mapped to that sequence. For example, for an autonomous aircraft, the basic sequence is a "cruise" logical objective of cruising at an assigned altitude and cruising airspeed, where the heading corresponds to the location of the next midway point, and there is no other traffic, no interfering weather conditions, and no aircraft malfunctions. Any other input logical objective can be mapped to boundary tiles. Therefore, the initial version of the policy re-determiner can select this sequence when its input logical objective is specified as "cruise to that next midway point".

[0460] This initial version is likely relatively simple to implement in practice, as it involves programming sequences that are designated as part of the engineering of a controlled system or as part of normal human operation (e.g., driver training). Furthermore, the redeterminer tile set is a single tile that selects the "cruising" sequence.

[0461] After completing this initial version of the strategy re-decisioner, it can be expanded with additional inputs indicating faults, time conditions, and environmental conditions, as well as additional input logical objectives. The additional input logical objectives can be processed in the same way as at the tactical level, i.e., by adding a tile set for each additional input logical objective.

[0462] One specific input to add is an indication of the controlled system's state to determine whether a "normal" input logical goal is feasible. For example, for an autonomous aircraft, if the aircraft is at a low altitude, the tile set with this additional dimension selects "climb" instead of "cruise" as the sequence to correct to the correct altitude. The tile set can also indicate erroneous conditions if the input logical goal is completely inconsistent with the current state of the controlled system.

[0463] For example, if the input logic objective is "cruising," but the aircraft is on the ground, the policy module could indicate an error instead of attempting to replicate the meta-policy module to get the aircraft airborne. In this particular example, the threshold for this input might be very coarse, as the issue is whether the altitude is significantly above ground level. In fact, the input could potentially be reduced to a Boolean input indicating whether the aircraft is "safely in the air," where "safely in the air" includes both altitude and airspeed.

[0464] Generally, adding new inputs can be handled similarly; a dimension is added to each tile set associated with that input, and then the original tiles correspond to those where the input is a default or normal value. For example, for an autonomous aircraft, the new input could indicate traffic in the same channel as the aircraft, where the default or normal value is "no traffic." The range of the input can then be expanded to include, for example, values ​​indicating traffic being overtaken ahead. Tiles with that input dimension matching that value are then labeled using a sequence to avoid that traffic. For example, it could choose a sequence that causes the aircraft to climb to a higher altitude to pass over the overtaken traffic and then descend to normal cruising altitude. As previously described, the tile set is replicated for each range of the new input dimension, where the original tile set is still labeled in the same way, with the value for each original tile in that input dimension being "no traffic." The remaining new tiles can be labeled according to standard flight procedures in response to traffic. This expansion may result in sequences being added to the corresponding sequencers. For example, there might be sequences corresponding to an emergency descent to avoid traffic.

[0465] The key difference in processing at the strategy layer is that decisions can be well-known and general, independent of the specific controlled system. This is because details are handled at the tactical layer. For example, a strategy to climb to a higher altitude to avoid traffic that an aircraft is catching can be used for virtually any aircraft, parameterized by aircraft constraints regarding altitude. Furthermore, attempting to evaluate sequence selection via simulation / prediction at the strategy layer is expensive because a strategy can take many time steps to implement to determine if it is a good choice. For example, a strategy to climb above oncoming traffic might take tens of seconds to implement.

[0466] In one embodiment, the strategy tile set is manually specified and then validated through testing. An alternative variation is to specify tiles for obvious situations and then use simulation-based evaluation to fill in the gaps between these points. For example, if the aircraft is moving away from the traffic it is catching, it may be preferable to climb to avoid it, while if the aircraft is moving close to the traffic it is catching, it may be safer to descend to avoid it. By explicitly specifying the situations where the aircraft can obviously climb and the situations where the closing time is obviously very short so that a descent is necessary, simulation can determine at which threshold it is better (or necessary) to descend rather than climb.

[0467] The input can also be refined to include additional thresholds / ranges. For example, a faulty system input might initially only indicate a fault or no fault. Thus, in the event of a "fault," the aircraft attempts to land immediately. However, this input can be refined to indicate faults that limit the aircraft's range or altitude, such as cabin pressure loss.

[0468] In some cases, new inputs / input values ​​may require additional inputs. For example, a strategy sequence for climbing above another aircraft that the aircraft is catching up with may require knowledge of the aircraft's current altitude or at least the difference between the current altitude and the maximum altitude.

[0469] In the autonomous aircraft use case, the input logic objectives corresponding to ground-based control indicate the benefits of having a separate set of tiles. This is because the inputs and sequences are significantly different from when in the air. On the ground, a midway point indicates the ground midway point from the runway landing point to the parking area (if just having landed). Therefore, the policy for each midway point on the ground indicates the sequence of taxiing to the next midway point. Thus, the policy layer can be incrementally extended to handle increasingly more relevant inputs as part of its decision-making.

[0470] Meta-strategy layer In one embodiment, the strategy and tactical layers delegate navigation to a separate meta-policy module / layer. The meta-policy layer takes a high-level objective that specifies the application's end goal and, where possible, generates "navigation" in the sequence of events leading to that end goal; otherwise, it reports a problem to the user / operator. This navigation merely identifies specific intermediate points to be achieved through a time series. For example, as mentioned above, a fully autonomous aircraft might have a top-level input objective: to fly from its current location to a designated airport. This example serves to further illustrate the following process.

[0471] Upon receiving the logical target, the meta-policy layer / module determines whether flying to that location is feasible based on weather conditions, available fuel, any sensor or mechanical problems with the aircraft, and other factors. If it decides to fly, its sequencer module may have a navigation submodule that determines the route to the destination, flight time, and any complication along the way. The re-decisioner may have a tile set with dimensions for flight time, fuel / flight time equivalents in the aircraft, aircraft malfunction conditions, environmental risks such as weather systems and traffic congestion, and other factors. After the sequencer determines the sequence, inputs to the re-decisioner can determine whether to maintain the sequence or make a re-decision based on its inputs. For example, if the flight time exceeds available fuel, it re-decisions not to proceed with takeoff.

[0472] If it decides to continue, its sequencer then continues stepwise through intermediate points generated as part of the route to that destination. These intermediate points may include taxi intermediate points / runway to which to proceed, which can affect the total travel time. The sequencer can process inputs about hold points along the way to the runway and obtain clearance from the tower to continue, including clearance for takeoff. These are part of the conditions for proceeding to the next step. Similarly, for the sequencer, the “decision” of determining whether the inputs / clearances are sufficient to allow proceeding to the next step is relatively straightforward. The sequence for reaching a specified destination is general in the sense that intermediate points are provided to the sequence as parameter values, so the sequence iterates on intermediate points, proceeding to the next intermediate point when the current target intermediate point is reached, and finally reaching the intermediate point on which the runway to proceed to landing and taxiing begins.

[0473] After deciding to continue to the destination, at each time step of the meta-policy layer, it re-determines whether there is enough fuel to reach the destination, whether a malfunction in the aircraft necessitates a change of target, and whether altered environmental conditions require a change of sequence. For example, in some cases, the re-determiner may decide to return to the departure airport. That is, it re-determines the sequence, thereby altering it so that the new destination becomes the original departure point.

[0474] When developing the tile set for this layer, the initial version could simply act on the input logic target, thereby invoking pathfinding to determine parameters / intermediate points, and then invoking intermediate point sequencing. That is, it did not have fault inputs or environmental inputs. This version is a practical improvement because it only implements the sequence required for normal flight, which has been specified as part of the engineering design.

[0475] The initial version is then incrementally expanded by adding new input dimensions one after another. For example, a fault report input indicating a specific type of fault can be added. This dimension is added by copying the current tile set for each value of the fault report input. The original tile set corresponds to the "fault-free" fault report input for this new input dimension. Tiles in this subset of the tile set are labeled in the same way as before. Each tile with another value for this new input dimension (the fault input) is then labeled as the correct sequence to follow for a given fault condition and its other input ranges. This correct sequence is specified as part of the training for a human pilot of the aircraft and can therefore be utilized without requiring significant judgment and / or analysis. For example, in the case of engine overheating, the tile could override the input logical target and reset the route to the nearest point where an emergency landing would be necessary, similar to how a human pilot would be trained.

[0476] Adding this fault input might require additional input to the tile set to make the correct decision. For example, if the aircraft is on the ground instead of in the air, the meta-policy re-determiner for the autonomous aircraft could choose a different sequence of actions. For instance, in the case of engine overheating on the ground, the re-determiner might simply decide to shut down the engine. However, if in the air, the best course of action is to land at the nearest opportunity. To accept this new input, the tile set can be expanded by an additional dimension corresponding to that input (elevation in this example). However, the threshold associated with this new dimension can be defined as the minimum number required to make the correct decision. For example, in the current example, there might only be two ranges for altitude, one corresponding to on the ground and one to in the air. Therefore, there are separate tiles corresponding to these different ranges of altitude input.

[0477] Once the tile set is expanded using this additional input, the redeterminer tile set can be expanded again using another additional input, thus increasing the dimensionality of the tile set again through duplication. For example, a fault input can be partitioned into two inputs corresponding to an engine problem and a flight surface control problem. The tile dimension increases again to handle this new input, and the number of tiles increases accordingly to the range of the new input. As another example, new inputs corresponding to the environment can be added. For example, the new input could indicate the weather conditions of the immediate vicinity.

[0478] As part of labeling these new tiles, if two adjacent tiles have the same label for a given fault input value, these two tiles can be merged into one tile, thus reducing the number of tiles. For example, in the case of both engine problems and flight surface control problems, the decision might be the same only for engine problems if the problem is in the air. Furthermore, as part of using new inputs for labeling, it can be recognized that these new inputs provide a basis for different decisions given existing tiles. For example, if an airspeed indicator is added, different decisions might be suggested based on high airspeeds compared to low or zero airspeeds. This identification might lead to splitting existing tiles into two or more, so that each tile can be labeled with an appropriate decision.

[0479] In a similar manner, one or more inputs corresponding to environmental conditions that may interfere with flight can be added. For example, a developing storm may prevent the use of the generated route.

[0480] The meta-policy layer is thus incrementally developed by adding a new input dimension one at a time, replicating the existing tile set for each range of the new input dimension, and preserving the labels on the original tiles for the new input dimension when their values ​​are default or neutral. Therefore, its development can follow the same sequence as that for the tactical layer.

[0481] Triple match In one embodiment, ternary matching, as described in U.S. Patent No. 10,761,921 entitled “AUTOMATIC ROOT CAUSE ANALYSIS USINGTERNARY FAULT SCENARIO REPRESENTATION” (which is incorporated herein by reference for all purposes), is used to match the input to tiles. Each “root cause row” described in U.S. Patent No. 10,761,921 corresponds to a tile as described herein, and each “symptom” or “column” described in U.S. Patent No. 10,761,921 corresponds to a threshold for a range or range of a particular input as described herein.

[0482] For example, if the flaps on the aircraft are up, partially down, or fully down, there are columns for each of these ranges. Input preprocessing then sets the corresponding entries in the “Actual Failure Scenario” vector described in U.S. Patent No. 10,761,921 based on the actual position of the flaps, and tile matching is performed in part based on that flap position. This encoding assumes that the ranges across the tile set used for this input do not overlap.

[0483] For example, if there are tiles that should match when the flap is partially down or fully down, then each of these subranges requires a separate row for that tile. It's worth noting that for a given row, if it requires input I to be within a given range corresponding to column C, then that row can be designated as "unconcerned" about other columns in that row corresponding to other ranges outside that range. This is because entries for these other columns in the actual fault scenario vector input are false, and therefore do not need to be matched against them. Designating these entries as "unconcerned" has a practical improvement in reducing the space requirements for the table. In the special case of a typically binary input, it can be matched using a single column indicating true, false, or unconcerned.

[0484] To avoid unnecessary oscillations, the same pre-matching as described above can be used, i.e., check whether the input is within the extended range of the previously matched input before using ternary matching, and if so, use the input; otherwise, perform ternary matching again.

[0485] For a given input, there may be different overlapping ranges. To handle multiple overlapping ranges on the same input, the encoding of specific inputs can be used, corresponding to associated columns of a threshold. Each input threshold has two columns, corresponding to: 1) less than the threshold, and 2) greater than or equal to the threshold. For example, column 42 could correspond to an airspeed threshold less than 150 km / h, column 43 to an airspeed threshold greater than or equal to 150 km / h, column 46 to an airspeed threshold less than 200 km / h, and column 47 to an airspeed threshold greater than or equal to 200 km / h. Therefore, by setting both columns 43 and 46, the row / tile only matches when the input is between 150 km / h and 200 km / h. If there are intermediate thresholds within this range, such as 175 km / h for columns 44 and 45, these thresholds can be set to "ignore" in the row.

[0486] The input vector indicates which range the input value is included in each row. Therefore, if the airspeed is measured as 167 km / h, it sets columns 43, 44, and 46. Thus, requiring the input to be in the range of 150 to 175 allows columns 43 and 44 to be set, thus matching the input vector for that input. If the input does not have a subrange of another range, the encoding into the table can use a single entry / column per threshold, saving space and columns. Subranges are allowed if a separate row exists for each superrange, and the input indicates each subrange and each subrange contains a superrange. Overlapping non-nested ranges are avoided by splitting the ranges and duplicating these rows.

[0487] In one embodiment, there are overlapping tiles and a prioritization mechanism for selecting which tile to use from the matched tiles to determine the output. In one embodiment, priority is assigned to one of a plurality of tiles by selecting the same or closest matching tile as a previously matched tile. Specifically, the matching algorithm checks whether a previously matched tile still matches, and if so, uses that tile; if not, it checks neighboring tiles for a match; otherwise, it performs a full search for a match. This approach has a practical improvement in matching cost by frequently avoiding full searches.

[0488] Using machine learning datasets If a labeled “training” dataset for the controlled system is available, similar to a dataset that might be used with a machine learning implementation, this dataset can be used to test the tile-based implementation enabled in this paper. Specifically, each input data sample is evaluated by matching one or more matching tiles, and the behavior associated with the tile label is compared to the behavior associated with that sample. If the behavior is inconsistent, the tile label can be modified to the tile label expected for that data sample. If another data sample maps to the same tile and expects a similar but still different label, the tile can be split such that the two values ​​are provided by their respective tiles, thus becoming consistent with the “training” dataset now used for testing. This process terminates at least when a separate tile exists for each data sample, if not previously.

[0489] In one embodiment, the tiles from the training dataset are used to refine the process as follows:

[0490] 1. Select the next data sample;

[0491] 2. Use this data sample as input to select tiles for each tile set. This assumes that the data sample includes all inputs used to determine the initial state of the controlled system, or that the data sample is part of a sequence of data samples that provide this information over time;

[0492] 3. If the training labels and discretized labels are not operationally consistent, add the tile to the set of "to be improved" tiles and record the sample; and / or

[0493] 4. After processing all data samples, refine each "to be improved" tile in the following way:

[0494] i. If the expected value of a sample corresponds to a value that the sequence selected by neighboring tiles may have already produced, and the input thresholds for that tile are defined to be adjustable without conflicting with other data samples, then adjust those input thresholds;

[0495] ii. Alternatively, adjust the value of the tile to satisfy the values ​​of all record samples associated with the tile, if possible; and / or

[0496] iii. Otherwise, segment the tile so that the resulting tile satisfies all samples, or ignore the data sample as erroneous.

[0497] The term "operationally consistent" as defined in this document means that a tile label will behave similarly to the behavior associated with the data sample. Specifically, if the effect of a control value differs from the sample value over a period of time, the behavior is significantly different from the behavior achieved by the data sample. For example, for an autonomous aircraft, if a data sample value causes the aircraft to climb, but a control system initialized using that data sample causes the aircraft to descend, the behavior is not operationally consistent. The term "operationally consistent" is used because it would be unwise to expect the generated control value to be exactly the same numerically or temporally as the value in the data sample.

[0498] After the process has generated a tile set consistent with the dataset, it performs as well as ML on the input samples. However, in contrast to ML-based implementations, it also exhibits predictable behavior on nearby data points. That is, if it's the same tile, the same control value, and if it's within a separate tile, the output is the label associated with that tile. Essentially, the ML training set used as described above is used to test the tile set and provides the basis for correcting or refining these tile sets, which is typically done manually.

[0499] During manual operation of a controlled system, training datasets can be generated by recording input and control variable values. For example, an aircraft can be manually operated, where its inputs and outputs, along with a high-level logical objective (e.g., "climb to cruise altitude"), are recorded and transmitted to the automatic control system. For instance, a climb instruction could be followed by a significant increase in throttle before the elevator is adjusted to increase the aircraft's pitch. In developing these initial tiles, some inputs can be pre-identified as being unrelated to the settings of the tile set's output.

[0500] In one embodiment, as an optimization, inputs are identified as unimportant or "unimportant" given a set of parameters or exceeding a given range. For example, if the altitude is greater than 500 feet, a particular altitude may be irrelevant to the control variable. In this case, tiles from across that dimension can be pre-combined into a smaller number of tiles instead of being evaluated individually as described above. Similarly, if the airspeed is zero, many inputs are meaningless. As another example, the longitudinal position of the joystick is irrelevant to the position of the ailerons because the longitudinal position of the joystick may not compensate for incorrect aileron positioning. That is, it must be assumed that they are set correctly. In general, not all inputs are needed for every control variable in all cases.

[0501] Recent work in machine learning aimed at reducing the amount of data required for machine learning has focused on so-called "soft labels" that are context-dependent. One example is Sucholutsky and Schonlau's "Less Than One-Shot Learning: Learning N Classes From M&N Samples." These soft labels are similar to tile labels, but lead to probabilistic control systems that are not interpretable or exhaustively testable, rather than what is presented.

[0502] Automatically generated The automatic generation of control systems described in this paper focuses on generating a set of tiles for each re-determiner. This is because, as described above, the engineering design process of the controlled system specifies the sequence required to operate the controlled system, and the actuators are characterized such that the sequence is known. Therefore, the sequencer is about converting these specifications into executable software. This step is practically simple. Furthermore, specifying these sequencers in software is feasible and more precise, so these specifications can be automatically converted into an executable form. For example, if these sequences are specified in a programming language such as Python, they can be compiled into a general systems language such as C for fast and efficient execution in production / operation.

[0503] In one embodiment, automatic generation focuses on the tactical layer, rather than the policy or meta-policy layer. This may be because:

[0504] 1. There are more Tactical Control Modules (TCMs) and correspondingly more tile sets at the tactical layer, thus offering greater advantages at the tactical layer;

[0505] 2. The meta-strategy and tactical layers are often common across a wider range of controlled systems, while the tactical layer can vary more significantly between controlled systems. For example, for autonomous aircraft, different aircraft may have different actuators, different capabilities, and different failure conditions; and / or

[0506] 3. The tactical layer may be more critical for the safe operation of a controlled system because it may react to sudden changes in input, in which case human intervention is much more difficult to achieve.

[0507] In one embodiment, automatic generation as described in U.S. Patent Application No. 17 / 156,378, entitled “AUTOMATIC GENERATION OF CONTROL DECISIONLOGIC FOR COMPLEX ENGINEERED SYSTEMS FROM DYNAMIC PHYSICAL MODEL” (which is incorporated herein by reference for all purposes), is applied to provide automatic generation of tile sets, including tile definitions and tags and / or TCMs. Specifically, automatic generation for a given actuator and input logic target includes:

[0508] 1. Provide a prediction function and an objective function for a controlled system, which has inputs plus an initial set of thresholds for each input and a set of sequences for the actuator. Then, define an initial / unlabeled tile set using these initial thresholds;

[0509] 2. For each unlabeled tile T in this tile set:

[0510] a. For each vertex of tile T:

[0511] For each candidate tile label;

[0512] a. When the control associated with the tile label is applied to the prediction function of the input associated with the vertex, score the tile label using the objective function; and / or

[0513] b. Record the tile labels for that vertex that have an acceptable score, if any;

[0514] b. Select a label that has an acceptable score for all vertices (if it exists) and use it as the tile label; and / or

[0515] c. If no tile label is found that has an acceptable score for all vertices of the tile, then the tile is split into multiple tiles by adding one or more new input thresholds, and these new tiles are added to the tile set as unlabeled tiles; and / or

[0516] 3. Output the set of tagged tiles.

[0517] Above, the inputs are process variables, environmental inputs, and fault conditions upon which the given actuator behavior depends. For example, aileron control depends on the current roll angle, airspeed, and altitude. A tile is defined with a threshold for each of these input dimensions. Iterations on all tiles correspond to iterations on the input combinations specified in U.S. Patent Application No. 17 / 156,378. Each vertex of a tile indicates a specific input combination. The tile label corresponds to the action selected in U.S. Patent Application No. 17 / 156,378. Finally, the generated rule corresponds implicitly to the labeled tile: if an input value considered as a K-dimensional location is contained within tile T, the action is to output a label associated with that tile.

[0518] The prediction function can be implemented using simulation of the controlled system. The objective function evaluates how well the controlled system is performing based on the candidate labels currently being evaluated. Because the tile set is tactical-level, the simulation may require relatively few time steps to provide an indication to the objective function. For example, for an autonomous aircraft, evaluating the effect of an elevator lift takes less than a second, so with a time step period of 100ms, fewer than 10 time steps may be needed.

[0519] Above, the evaluation at each vertex of tile T uses vertices computed with an additional margin, which will be used by the pre-matching step in the input map to avoid unacceptable oscillations.

[0520] The action of "adding one or more new input thresholds" described above is performed through a process that may be specific to the type of controlled system or may use the methods described above to select new thresholds as part of an expanded input range. Given the assumptions / requirements of engineering a controlled system for predictable and piecewise stable behavior, the input thresholds may ultimately become a sufficiently good approximation of the control actions required to achieve the desired performance, assuming the objective function is consistent with the system's engineering design requirements in terms of its scoring. That is, an objective function requiring performance that is more precise or efficient than the performance the controlled system is designed to achieve may not be met.

[0521] A similar approach can be used at the strategy level. However, evaluating a strategy requires a much larger number of time steps. For example, a strategy for an autonomous aircraft to maneuver around a major storm might take several minutes, if not hours. Therefore, the number of time steps required for evaluation could be 1,000 to 10,000 times that for the tactical level.

[0522] In summary, autonomous control, as described in this paper, supports ensuring the correctness of control systems through extensive, potentially exhaustive, testing, rather than relying on "control theory," the correctness of floating-point calculations, or statistical behavior as a result of extensive "training" similar to machine learning. It offers several benefits, including:

[0523] 1. Rapid Response to Change: The re-decision-maker selects potentially different sequences at each time step, and can therefore be designed to react immediately to changing circumstances. In contrast, planning-based methods, such as those used in traditional AI planning, require evaluation of the plan at each stage and dynamic generation of new plans before reacting, if necessary. Furthermore, traditional control methods assume continuous behavior, while change may produce or require discontinuous behavior in the controlled system;

[0524] 2. Handling Singularities with Nonlinear Mapping from Input to Output: In the case of using tile sets in the redeterminer, the choice of tile boundaries and labels for each tile is arbitrary from the perspective of tile specification and matching. In contrast, traditional rule-based methods combine the decision-making process (and sometimes basic control) with the redeterminer role. Moreover, traditional control theory for MIMO typically requires that the control be linear or at least differentiable, and often attempts to compute the output using some closed-form computation. This is inconsistent with the fact that many controlled systems exhibit singularities;

[0525] 3. Complete, deterministic, and predictable / interpretable: The described set of tiles provides a mapping from all possible input values ​​to one or more given tiles; that is, it is complete. Furthermore, based on the inputs and the specified set of tiles, the control system behavior is deterministic, predictable, or interpretable, and behaves according to engineering design / specifications in the absence of faults. This interpretability contrasts with traditional ML methods.

[0526] 4. Efficient Development: Based on the logical goals, constraints, and requirements of the controlled system, both the sequencer and the redecimalizer incorporate knowledge from the controlled system design and experts. Therefore, it avoids the costs of generating very large training sets and labeling them, as required by traditional ML / DL methods. Furthermore, the modular separation of key decision-making in the redecimalizer isolates the critical and most complex decision-making from the rest of the control module, while simultaneously offloading it from other aspects of the sequencer's control.

[0527] 5. Fewer and more identifiable test cases: Testing the re-determiner may only require covering the boundary conditions of the tiles, i.e., the vertices of each tile. While this may still be a large number of test cases, it is fewer than when using continuous input and continuous computation, where completeness is impossible. Boundary conditions effectively define the complete set of necessary test cases. In contrast, dynamic plan generation testing using traditional planning methods is much more difficult and expensive. Consistency with test-driven development and testability design is an improvement. There is no round-off effect or floating-point inaccuracy in the input-to-output mapping. Output control values ​​are determined based on integer comparisons;

[0528] 6. Incremental Expansion: An improvement is the ability to directly handle "normal" cases and then incrementally expand the control system's operating domain without modifying the portion handling the current operating domain, while always maintaining the ability to identify when the system is outside the supported operating domain. When inputs are contained within tile input dimensions and neighboring tiles, a change to one tile only affects the behavior of the control system. Therefore, the control system can evolve incrementally without retesting the entire range of inputs. Furthermore, in many cases, new dimensions / inputs can be added by treating existing tiles as projections of new tiles, where the new dimension is within the default or normal range. That is, existing tiles can be expanded to the new dimension by specifying a threshold for it, but without retesting all combinations of inputs where the new dimension is neutral, corresponding to the absence of that dimension. In contrast, using traditional control theory methods to change the control formula means retesting all cases. Similarly, in the case of traditional ML, new features / dimensions may have unknown effects on existing test cases, so they all need to be retested; and / or

[0529] 7. Adaptability to Different Controlled Systems: By changing the input preprocessing and sequencer parameters or logic, the control system can also be more easily adapted to new controlled systems of similar categories without requiring modification of the re-determiner. This is because the logical objectives are the same across different systems within the same category, such as across a single-engine aircraft. Therefore, the most challenging part of the control system—the tile set—does not need to be changed, or may only need slight modification.

[0530] Feasibility of using timed control The feasibility of using tiles / discrete tiles to provide sufficient control is based on the controlled system being engineered / designed to be controllable under manual control, and the piecewise continuous behavior of the physical system. These aspects imply three useful properties for the controlled system:

[0531] 1. Relatively stable;

[0532] 2. Relatively predictable; and / or

[0533] 3. There are a relatively small number of discontinuities to focus on, and they are known.

[0534] The first aspect—stability—means that the system can operate reasonably with the same control settings across certain significant changes in the input. In other words, it is designed to be stable within certain operating parameters. For example, a slight change in the aircraft's airspeed (such as due to a change in headwind) does not require an immediate change in control to avoid losing control of the aircraft. Otherwise, controlling the system would be infeasible for a human. That is, if a slight decrease in airspeed required an immediate and rapid change in the control variables to maintain control, a human operator would be unable to operate the controlled system.

[0535] Similarly, small changes in some control variables typically do not destabilize a controlled system. Otherwise, a human operator could destabilize a controlled system due to minor errors in setting the controls. This also means that, generally, if the logical objective determined by a tile is feasible at every vertex of the tile, it is feasible / acceptable at any point within that tile. This is because small incremental changes in any input typically do not require changes in the control values. Otherwise, in manual operation, a controlled system would require significantly rapid actions from the operator.

[0536] This inherent physical smoothness allows controlled systems to be efficiently controlled through appropriate discrete decision-making and discrete sequencing, relying on sub-component levels such as PID controllers for fine-grained tuning. Some smoothness is provided by the inherent delay between changing actuator settings and achieving the desired end result under the controlled system. Therefore, this separation of logical target decision-making and logical target sequencer does not introduce a latency exceeding the latency inherent in this indirect effort of the actuator. Furthermore, even with significant discrete changes to the logical target, the sequencing provided by the sequencer ensures smooth operation.

[0537] The second aspect—predictability—means that system behavior is predictable in the sense that changing some control variables in a specific way has known effects. Therefore, the control system can know in advance what changes to the control variables are necessary to achieve a particular end result. For example, increasing throttle and increasing aircraft pitch generally result in an increase in altitude. Similarly, an incremental increase in aileron angle results in an incremental increase in roll rate. Therefore, the control settings for a tile in advance for a given controlled system state as indicated by the input can be determined in advance. In other words, it is feasible t...

Claims

1. A control system for autonomous control of complex engineering systems, comprising: A timing sequencer is configured to execute a determined sequence of steps selected from a set of step sequences defined for the control system to achieve proper control of the actuator. And an immediate re-decisioner, configured to periodically: receive sensor inputs reflecting the current state of the engineering system, and autonomously re-determine, based on the sensor inputs, whether the time sequencer should execute the determined sequence of steps or an alternative sequence of steps, wherein the immediate re-decisioner is configured to make the re-decision at least in part based on a determined set of tiles, wherein the set of tiles has multiple dimensions matching multiple inputs, and wherein each tile is associated with a tag sequence such that executing the tag sequence with the associated tile inputs at the current time is sufficient to provide reasonable control of the control system at the current time.

2. The control system of claim 1, wherein the timing sequencer is configured to write control variables into the actuator.

3. The control system of claim 1, wherein the immediate re-decisioner is configured to make a re-decision at least in part based on new input data.

4. The control system of claim 1, wherein the immediate redecisioner is configured to make a redecision at least in part based on the determination of a discrete logic objective.

5. The control system of claim 1, wherein the control system is configured to control at least one of the following: an autonomous complex controlled system; an autonomous ground-based vehicle; an autonomous air-based vehicle; an autonomous space-based vehicle; and an autonomous water-based vehicle.

6. The control system of claim 1, wherein the duration between time steps is designed at least in part based on the ability to react quickly to abnormal or discontinuous situations.

7. The control system of claim 1, wherein the plurality of sequences in the set of step sequences are predetermined prior to operation of the control system.

8. The control system of claim 7, wherein the plurality of sequences in the set of step sequences are at least partially based on the engineering specifications of the control system.

9. The control system according to claim 7, wherein the plurality of sequences in the set of step sequences provide efficient, stable and reliable control under normal conditions.

10. The control system of claim 1, wherein the time sequencer is configured to: immediately preempt the determined sequence of steps and execute the alternative sequence of steps in response to receiving an instruction from the immediate re-determiner.

11. The control system of claim 1, further comprising a pre-matcher, wherein the pre-matcher is configured to: determine a match when the current tile input is mapped to a tile matched in a previous time step when the tile matched in the previous time step is extended by margin in one or more tile dimensions, in order to at least partially dampen oscillations.

12. The control system of claim 1, wherein the tile is associated with each combination of a range defined by a threshold relative to the input, one range from each input.

13. The control system of claim 1, wherein the immediate re-decisioner is configured to make a re-decision at least in part based on a predictive mechanism of the control system, the predictive mechanism being at least in part based on providing efficient, stable, and reliable control under normal conditions.

14. The control system of claim 13, wherein the prediction mechanism includes the use of dynamic simulation.

15. The control system of claim 1, wherein the immediate re-decisioner is configured to make a re-decision at least in part based on the training dataset.

16. The control system of claim 2, wherein the timing sequencer is further configured to execute a determined sequence of steps selected from the set of step sequences defined for the control system by implementing the writing of coupled control variables to coupled actuators.

17. The control system of claim 16, wherein writing the control variables to the actuators and writing the coupled control variables to the coupled actuators are at least partially based on a vector that maps decision results to decision values, one entry for each actuator.

18. The control system of claim 1, wherein the immediate redecisioner is further configured to provide a parameter range for a sequence of redecision steps to be performed by the time sequencer.

19. The control system of claim 18, wherein the time sequencer is further configured to refine the parameter values ​​within the parameter range based at least in part on dynamic simulation of the control system over relevant time periods.

20. The control system of claim 18, wherein the timing sequencer is further configured to refine the parameter values ​​within the parameter range based at least in part on the midpoint of the parameter range and the final parameter value from the last time step.

21. The control system of claim 1, wherein executing the determined sequence of steps includes using a dynamic adaptive time series.

22. The control system of claim 1, wherein performing the determined sequence of steps includes adjusting the actual output control value based on additional inputs.

23. The control system of claim 1, wherein the plurality of sequences in the set of step sequences are predetermined prior to operation of the control system, and wherein the plurality of sequences are dynamic adaptive time series.

24. A method for autonomous control of complex engineering systems, comprising: Execute the determined sequence of steps selected from the set of steps defined for the control system to achieve proper control of the actuator; And in a periodic time step: receiving sensor inputs reflecting the current state of the engineering system; and autonomously re-determining, based on the sensor inputs, whether to execute the determined sequence of steps or an alternative sequence of steps, wherein the re-determination is at least in part based on determining a set of tiles, wherein the set of tiles has multiple dimensions that match multiple inputs, and wherein each tile is associated with a tag sequence such that executing the tag sequence with the associated tile inputs at the current time is sufficient to provide reasonable control of the control system at the current time.

25. The method of claim 24, further comprising: In response to receiving a preemption instruction, the system immediately preempts the determined sequence of steps and executes the alternative sequence of steps.

26. A computer program product comprising instructions that, when executed by at least one processor of a computer, cause the computer to perform the method according to claim 24 or 25.

Images