Automated data-driven domain model synthesis

Abstract

A method (10) for automated data-driven domain model synthesis is provided. Logs (11) of data are received from a target system for which at least one goal is to be achieved. A set of predicates (44) that describe a domain of the target system is determined from the logs. The set of predicates represents an initial state and sequence state changes. Each predicate applies to one or more objects in the domain. Domain synthesis (13) is performed to create a model that describes the log data by adding the predicates and actions needed to achieve the goal. The model is revised by adding one or more missing predicates or actions when the model fails to cover all the logs.

Description

[0001]PCT Patent Application Attorney Docket No.254.2466.PC.UTL AUTOMATED DATA-DRIVEN DOMAIN MODEL SYNTHESIS TECHNICAL FIELD This application relates in general to building models, and in particular to a system and method for automated data-driven domain model synthesis. BACKGROUND ART Model synthesis involves developing multiple model designs and learning techniques. For example, applying various filters and transformations to a dataset ingested from a data connector can define a state space, which is used to learn a model for a system. When building a domain model, it is necessary to process offline data. Offline data sets, which are typically used for model learning, can be quite large and are not significantly constrained by processing time and results in longer processing times. While the connectors are critical components of the system, the implementation effort has already been invested in several open-source data connectors, such as Airbyte (2023). Therefore, a need remains for a data-driven approach that takes operational logs from a system and creates a planning domain model in a modeling language. Preferably, names of actions are made human-friendly by using a Large Language Model to improve readability of the generated domain model. DISCLOSURE OF THE INVENTION The automated data-driven domain synthesis (“auto synthesis”) process aids a user in designing a model, such as in the planning domain definition language (PDDL), to describe the processes that are captured in the form of operational logs from a system. The auto synthesis may not deliver completely accurate models in some cases due to some theoretical limits of the underlying computational problem and the properties of the input logs. However, in general, results of the auto synthesis are better if the provided logs are diverse, including containing many different events that are possible in the process. Also, to improve results, the auto synthesis process can be an interactive process. For example, a user can manually validate and adjust the model, if necessary. The auto synthesis system can then provide feedback on the user’s adjustments, including whether all the events observed in the logs are covered by the action model and whether the action models include redundant or unused action that never occurred in the logs. 2466.PC.UTL.ap1 - 1 - The auto synthesis system also supports incremental usage. This is useful when the processes get updated and new logs that include new types of events become available. In conjunction with the previous logs and the previous model, a new model can be synthesized more efficiently and will be more accurate. An embodiment provides a method for automated data-driven domain model synthesis. Logs of data are received from a target system for which at least one goal is to be achieved. A set of predicates that describe a domain of the target system is determined from the received logs. The set of predicates represents an initial state and sequence state changes, while each predicate applies to one or more objects in the domain. Actions that are low-level execution steps to be performed for achieving the goal are transformed to actions that are high-level plan steps regarding achieving the goal. Actions that are high level plan steps regarding achieving the goal are identified and transformed to actions that are low level execution steps to be performed for achieving the goal. Domain synthesis is performed to create a model that describes the log data by adding the predicates and actions needed to cover all the logs and achieve at least one goal. A determination is made as to whether the model covers all the logs and the model is revised by adding one or more missing predicates or actions when the model fails to cover all the logs. A further embodiment covers the semi-automatic generation of solvable test cases for evaluating the performance of automated planners in PDDL domains, with the ability to adjust the difficulty level of the problems to test scalability. The semi-automatic generation guarantees the generation of solvable instances, providing a reliable means to assess planner efficiency across various complexity levels. Still other embodiments will become readily apparent to those skilled in the art from the following detailed description, wherein are described embodiments by way of illustrating the best mode contemplated. As will be realized, other and different embodiments are possible and the embodiments’ several details are capable of modifications in various obvious respects, including time and clustering of events, all without departing from their spirit and the scope. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not as restrictive. DESCRIPTION OF THE DRAWINGS FIGURE 1 is a flow diagram showing a method for automated data-driven domain synthesis, in accordance with one embodiment. 2466.PC.UTL.ap1 - 2 - FIGURE 2 is a block diagram showing a system for automated data-driven domain synthesis, in accordance with one embodiment. FIGURE 3 is a flow diagram showing, by way of example, a process for generating the domain model of FIGURE 1. FIGURE 4 is a block diagram showing, by way of example, architecture of a target system. FIGURE 5 is a flow diagram showing, by way of example, a process for processing raw logs via the data management module. FIGURE 6 is a flow diagram showing, by way of example, a process for pushing actions to be executed by the target system. FIGURE 7 is a flow diagram showing, by way of example, a process of the diff stage for generating actions. FIGURE 8 is a flow diagram showing, by way of example, a process for performing the merge stage. FIGURE 9 is a flow diagram showing, by way of example, a process for performing the refine stage. FIGURE 10 is a flow diagram showing, by way of example, a process for performing match or spawn. FIGURE 11 is a flow diagram showing, by way of example, a process for converting logs prior to processing. FIGURE 12 is screenshot of the user interface of the system for automated data-driven domain synthesis of FIGURE 2. FIGURE 13 is a screenshot showing, by way of example, a log loaded via a user interface. FIGURE 14 is a screenshot showing, by way of example, a domain draft loaded via the user interface. FIGURE 15 is a screenshot showing, by way of example, options for controlling and displaying synthesis metadata via the user interface. FIGURE 16 is a screenshot showing, by way of example, a synthesized domain. FIGURE 17 is a screenshot showing, by way of example, additional synthesis options provided by the user interface. FIGURE 18 is a screenshot showing, by way of example, options for the synthesized domain displayed via the user interface. 2466.PC.UTL.ap1 - 3 - FIGURE 19 is a flow diagram showing, by way of example, a process for debugging models that overapproximate a problem. FIGURE 20 is a flow diagram showing, by way of example, a process for evaluating models. FIGURE 21 is a graph diagram showing, by way of example, a behavior tree for monitoring and adapting a plan in execution. FIGURE 22 is a graph diagram showing, by way of example, a composition framework. FIGURE 23 is a block diagram showing, by way of example, a control node architecture. FIGURE 24 is a graph diagram showing, by way of example, a directed acyclic graph of the composition framework of FIGURE 22. FIGURE 25 is a graph diagram showing, by way of example, a behavior tree for the composition framework of FIGURE 22. FIGURE 26 is a block diagram showing, by way of example, a composite AI system. FIGURE 27 is a screenshot showing, by way of example, an interface for composition development. FIGURE 28 is a flow diagram showing, by way of example, a process for creating and editing a composition. FIGURE 29 is a flow diagram showing, by way of example, a process for running a composition. FIGURE 30 is a block diagram showing, by way of example, a high level target system architecture. BEST MODE FOR CARRYING OUT THE INVENTION Generally, domain models are built by processing offline data. However, the offline data can exist in large amounts and are not significantly constrained by processing time, which can make building of domain models time consuming. A process that provides capabilities to learn individual models from provided datasets in sequential or time-ordered sequences can increase the efficiency of model building, as well as accuracy. The models can be built and maintained automatically with little to no human feedback. The models represent events and actions that can be used to plan in the operational space, solve domain problems, and automate systems on which the datasets were captured. FIGURE 1 is a flow diagram showing a method 10 for automated data-driven domain synthesis, in accordance with one embodiment. Operational logs from an external target 2466.PC.UTL.ap1 - 4 - system, such as via a data connector, is input (step 11). The logs can be interpreted and transformed (step 12) into a sequence of world states that are defined by true predicates. Incremental auto synthesis (step 13) is performed to generate a domain model, such as in PDDL format, that describes the log data. The auto synthesis can use an empty model or an existing model. When an existing model is used (step 14), attempts are made to interpret the logs with the given model. However, if not possible, the existing model can be extended by adding new types of objects that can appear in the logs, predicates, and actions. A user examines the model to determine (step 15) whether the model accurately describes the processes provided in the logs. If the model is not accurate, the user can modify (step 16) the model using an editor tool, such as a visual editor, including Blockly, or by editing the source file, such as PDDL, as text. Auto synthesis is then performed on the modified model to verify that the modified model covers all the changes that appear in the logs. However, if the model is accurate, the user can utilize (step 17) the model. For example, the model can be combined with a solver or a set of solvers that solve problems using the model. The model can also be used for documentation of processes or for reasoning about certain properties, model like complexity, and computability. The solver is a software tool or algorithm designed to solve specific problems and can be used in the context of constraint satisfaction problems, optimization problems, or logical reasoning tasks. Specifically, each solver can take a defined problem as input and applies various techniques and algorithms to search for a valid solution or to determine if a solution exists. The techniques and algorithms applied can be dependent on a nature of the problem being solved. For example, in constraint satisfaction problems, a solver can utilize backtracking, constraint propagation, or local search techniques to find a solution that satisfies a set of given constraints. However, in optimization problems, a solver can find a solution by evaluating and comparing different potential solutions based on an objective function. FIGURE 2 is a block diagram showing a system for automated data-driven domain synthesis, in accordance with one embodiment. A user, in front of a computing device 21, such as a desktop, laptop, tablet, or cellular telephone, can identify a goal to be achieved or a problem to be solved for or within a target system, and provide the target system to be described by a model via an Internetwork 22. In one embodiment, logs for synthesis and a goal or initial state of the target system can be provided in a request for the user. The request can be received by a server 23, which includes a module for performing auto synthesis 24 to generate a model to achieve the goal requested by the user. To generate the model 28, the server 23 accesses log data 402 stored in a database 401 associated with a server 400 for the target 2466.PC.UTL.ap1 - 5 - system or provided by the user. The logs 402 can include an initial state and state changes of the system. Once received, the logs 26, 402 are stored in the database 25. From the logs, predicates and actions 27 are identified and used in the model 28. The model is then fed to the target system for achieving the goal of the user. For example, the server 23, via a planner module (not shown), can generate a plan for solving the problem or achieving the goal, based on the model, initial state of the target system, and goal. In one embodiment, the model can be displayed to the user via the computing device 21 and revised based on any feedback of the user. Specifically, auto synthesis can utilize user feedback or alternatively, generate the model without any feedback. FIGURE 3 is a flow diagram showing, by way of example, a process 40 for generating the domain model of FIGURE 1. Domain synthesis data derives causal symbolic models from timestamped state change records or logs of a system, which are input 41 for processing during domain synthesis. Optionally, the logs can be preprocessed 42 to only include variable changes relevant to the model to be synthesized or to include only certain log events. Based on the logs 43, predicates and types, which are described in detail below, are discovered 44 and used with other log data, such as timestamp data 45 to determine actions 46. The actions can be used to generate the model using a PDDL model 47. Action names and descriptions can be determined and added to the model 48 (resulting in a beautified PDDL Model 49), which can be reviewed by the user, and if approved, published 500. A solution architect, such as the user, can also review 504 and provide feedback during preprocessing by adjusting preprocessing preferences 501, rescanning the logs 503 and reviewing the logs 502. Furthermore, the solution architect may provide feedback during action synthesis, in addition to the model, based on a preview of updated logs. Specifically, the solution architect can provide required input or adjust names and types of variables 504 discovered during scanning of the logs. Once preprocessed, the logs are submitted 43 to a multi-stage iterative synthesis process during which patterns in the system state changes are discovered and a model is generated. To aid the user in understanding the generated model, Large Language Models (LLMs) can be utilized to produce textual descriptions and meaningful names for actions 48. The domain model is presented to the user and the user can provide feedback, such as refusing a generation action to steer the synthesis of the model towards a different action atomicity level other than the minimized model with complex actions preferred by the synthesis. The user can also choose to augment the model with additional actions or adjust preconditions and effects beyond 2466.PC.UTL.ap1 - 6 - what was represented in the source time series data. Feedback from the user prompts a new synthesis iteration. Once the user accepts the model, the model is ready for use in the target system from which the logs were obtained. The target system 62 accepts actions and provides a perception of the system and its world’s state. FIGURE 4 is a block diagram showing, by way of example, architecture 60 of a target system. The user works with a back end system via a user interface 69 to define one or more domain models and a system of components that utilize the model running through tests and deployment to solve problems through action-space control. The backend system includes systems for data management 63, domain synthesis and predicate extraction 64, and deployed model runtime 68, as well as a domain problem knowledge base 65, predicate knowledge base 66, and action queue 67. In one example, the target system can include a system for solving the Towers of Hanoi game, which is a puzzle that involves moving discs from one place to another. For instance, the game includes n rods, where n is typically 3, and a stack of different-sized discs. The goal of the game is to move all the discs from one rod to another, following a few rules. The rules include moving only one disc at a time, never putting a bigger disc on top of a smaller disc, and using the other rods to help move the discs around, but only placing a disc on top of another disc or on an empty rod. The domain problem knowledge base 65 captures real world knowledge the model is designed to represent and consists of definitions of the target system’s predicates and goals. A predicate is a function of a set of parameters that returns a Boolean as an answer by taking raw logs from a system, an expert’s domain knowledge, existing domain knowledge, such as existing PDDL descriptions, and other sources that provide domain knowledge. The knowledge capture primarily involves preparing a set of predicates that describe the domain under interest following any method of knowledge capture from a consultant or domain expert. The knowledge base can include a description of predicates and their meaning, which are extracted from the target system’s raw logs. The predicates apply to a specific type of object or to all objects. The predicates are either true or false at any point and when not declared, they are assumed to be false. Returning to the game example, the knowledge base may contain information, such as the definition of useful predicates: “on(disc1, disc2) means disc1 is on disc 2, smaller(disc1, disc2) means disc 1 is smaller than disc 2, clear(disc1) means disc 1 is on the top and nothing lies on it. The goal to be achieved can also be defined, with the predicates. For example, the goal of the game is that “all discs lie on the last rod, with the predicates that 2466.PC.UTL.ap1 - 7 - are all true, such as disc_on_rod(disc1, rod3), disc_on_rod(disc2, rod3), and disc_on_rod(disc3, rod 3). Additionally, the initial state of the problem can be defined using predicates that describe all predicates describing the input scenario and all combinations of disc and their sizes and location. Information from the target system 62 flows into and out of the data management module 63, which takes perception data from the external world state of the target system or other systems that accurately report data form the environment of the target system. The perception data can include any external information relevant to the domain of interest, such as weather, traffic conditions, or general knowledge from the outside world. The target system can provide telemetry or operational logs, such as a raw stream or batch of data collected from components of the system. Returning to the game example, the data can include keys pressed in the game by a human player. The data management module also transmits action information to the target system, such as transformed planned actions that lead to actual system change. For example, in the Towers of Hanoi game, the planned action can include “move_disc_from_rod_to_rod,” which translated to the target system can mean “press arrow up, press arrow left, press arrow left, press arrow down” as a sequence of virtual keyboard key presses. Internal to the data management module, multiple process flows can exist. FIGURE 5 is a flow diagram showing, by way of example, a process for processing raw logs via the data management module. The external world state is recorded directly to the predicate knowledge base. Raw logs 71 from the target system are provided to a data connector 72. The data connector 72 receives the data, digests the data from the document source, such as a document, Google Drive, ftp, stream, websocket, Parquet or Amazon S3, and passes the data in a defined structure data format for processing by other entities, such as the user-based predicate extractor 73. With respect to the game example, Keboola or other data platform can be used to interchange the data from the user’s game, such as via websocket or batches in some storage, into a format defined by the predicate store, including format specifications, connectors, logins, and secrets. The user-based predicate extractor 73 receives predicate definition raw data and transforms the data given the predicate definition to create predicates from the input log stream. Returning to the game example, the data “Rod1=[disc3, disc2]” is converted to predicates, including disc_on_rod(disc3, rod1), disc_on_rod(disc2, rod1), disc_on_disc(disc2, disc3), smaller(disc2, disc3), disc_clear(disc2).” Once the raw logs are converted into predicate logs 2466.PC.UTL.ap1 - 8 - 74, another data connector 75 inserts the predicate logs into the predicate knowledge base 76, which res the predicates, including a history of predicates, from the target system. During synthesis, the functionality of the connectors are extended beyond their current functionality by adding preprocessing data transformations relevant to learning the models. Returning to the discussion with respect to Figure 4, the model testing and deployment user interface includes the front-end interactive tools that work with the user and includes model development, model-using system development, simulation testing, and deployment. Model development uses a text-based language augmented by a visual programming language. In one embodiment, the modeling language is PDDL text and Blockly provides a visual builder. The user iteratively refines the model through testing and deployment in a digital simulation of the target system or the target system itself. Model debugging tools provide insight and feedback for iterative refinement. As described above, the models can be synthesized from raw observational data. The interface tools can then be used to manipulate, edit, add, and remove actions within the synthesized model. The goal is a model containing information from domain synthesis on which action to use or not to use for control with the target system. Augmentation refines the models and improves understandability by renaming predicates, goals, and actions as needed. However, use of domain synthesis and LLM (Large Language Model) naming components actually minimizes a number of iterations by the user to create a domain model or the need for user editing, if needed at all. For instance, the use of LLM reduces the need to manually improve action and action parameter names. LLM can also reduce the need to manually write natural language descriptions of the action. In the puzzle example, the system can completely generate proper Towers of Hanoi PDDL from domain synthesis alone and accurate names from the LLM support. The model-using system development uses a text-based language augmented by a visual programming language. In one embodiment, the system language is text describing a behavior tree and Blockly provides a visual builder. A behavior tree visualizer also provides additional insights. In another embodiment, the user iteratively refines the model through testing and deployment in a digital simulation of the target system or the target system itself. Target application debugging tools can provide insight and feedback for iterative refinement, when applied. In the game example, the system invokes interfacing with the game to obtain the state of the puzzle, and subsequently, deploys a planner to use the PDDL model to determine action 2466.PC.UTL.ap1 - 9 - commands in a sequence that solves the provided puzzle. The system then executes the commands on the target game. Simulation testing via the user interface provides pre-deployment feedback on the system and underlying models in action, while deployment provides the ability to push control by the system and models to the target system. Deployment can also include live performance monitoring of the system in action and supports defining and observing performance through evaluation metrics. The deployment can target any type of cloud and container-based system. The user interface components rely on core algorithms, including domain synthesis, a Machine Learning (ML)-based predicate extractor, and a large language model (LLM) / ChatGPT-based predicate extractor. During domain synthesis, definitions, pre- processed predicates, user-defined input, existing models, and other information, such as raw logs, are input to synthesis a model that describes the actions that lead from one system state to another. In the game example, domain synthesis can take log data from a human player and develop a PDDL model that solves any Towers of Hanoi puzzle. The ML-based predicate extractor takes in predicate definitions and pre-processed predicates, and uses auto-ML techniques to extract additional, hard-to-code predicates, from pre-processed predicates, to produce a set of predicates to be fed into a PDDL solver or model. Although the ML-based predicate extractor is not applicable to the game example, the extractor can be applied to a game in which a bunny jumps on platforms higher and higher, where new platforms emerge as the bunny progresses upward. The game dynamics are usually fully unknown to the observer so the ML-based predicate extractor can be trained on historical data to create a predicate jumpable(platform1, platform2) that will be true if the bunny can jump from platform 1 to platform 2. Complex data can be input into the ML-based predicate extractor, such as platform width, mutual position of the platforms, and the bunny’s strength, stamina and speed. Other types of complex data can be used. The result of the input complex data is the predicate. The LLM / ChatGPT-based predicate extractor performs a similar process as the ML- based predicate extractor, but uses LLM models to extract semantic information from pre- processed predicates. For example, a collection of game-relevant terms which can be transformed into a recommended predicate configuration might include exact prompt wording or LLM model parameters, such as temperature or n_tokens. During deployed model runtime, deployment setup, description of container images, image registry, relevant services, external endpoints, API’s, data locations, and the input stream of target system predicates are input to deploy the model on a cloud-based platform. The 2466.PC.UTL.ap1 - 10 - deployed model runtime can run on any computer platform, and will run services and let them communicate according to a description given by the system, as well as ingest input data, such as pre-processed predicates, and output data, such as actions that lead to a goal of the target system. The system output is a stream of actions to be executed on the target system. The form of actions is some high-level plan. However, a user-based action transformer, which is part of the data management module, is responsible for making the plan happen in the real world. In the Towers of Hanoi example, a low-level Azure cloud configuration of individual system components, such as a planner, data store, and serving APIs, networking and interoperability settings, VPNs, container registry settings, and endpoint definitions needed to startup and execute a composed AI system to play the Towers of Hanoi. Actions are queued in an action queue and pushed to or read by a data management module. FIGURE 6 is a flow diagram showing, by way of example, a process 80 for pushing actions to be executed by the target system. A data connector 82 obtains information from an action queue 81 and converts the data to an action description with parameters 83, which describes actions to be executed in the target system to achieve a desired goal. For instance, in the game example, “move(disc1, disc2, disc3) means to move disc 1 from disc 2 to disc 3. The user-based action transformer 84 receives the set of actions provided by the deployed system and transforms the action from high-level plan steps to low-level execution steps having detailed descriptions of steps to feed into the target system so that all planned actions are performed and executed correctly. A different data connector 85 interfaces with the target platform to execute all action commands 86 on that platform, which affects the state through the actions. The incremental auto synthesis system utilizes input, such as an optional previous or partial PDDL model with definition of types and hierarchy, such as type names and their sub- type relations, definitions of predicates, including predicate names and types of parameters, and definition of actions. The types can refer to classes of objects that can be used as parameters to predicates and actions. For example, the objects disc1, disc2, and dis3 are associated with type “disc,” while rod1, rod2, and rod3 are associated with type “rod.” The parameters are variables of a function or predicate. For example, a predicate (at ?t – truck?l – location) represents that a truck ?t is in a location ?l(?t and ?l are the parameters of the “at” predicate. In a further example, a predicate titled “on” can have two parameters of type “disc,” representing the object of type “disc,” given as the first parameter, lies on the object of type “disc” provided as the second parameter. The input can also include a collection of change logs, such as lists of 2466.PC.UTL.ap1 - 11 - observed state transitions. In one embodiment, only the change logs are used as input without a previous PDDL model. Each change log is provided as a sequence of timestamped entries, where each entry is a list of (grounded) predicates, some of which are true and some which are false. Alternatively, an entry can include a single predicate. Each action can include one or more entries. A preprocessing task known as log segmentation can be applied to group the entries that belong to the same actions. The grouping can be based on information, such as a value of the timestamp associated with each entry, properties of the predicates, such as name and parameters, labeling on the entries, and action definitions. For example, the action definitions can be provided by the user or via a previous round of incremental synthesis. With respect to the entry labels, entries with different labels belong to different groups. The grouping during log segmentation is solved using the k-medoids algorithm where the distance metric between two entries is defined by the difference of the timestamp values, labels, and the similarity of the predicates in the entries. Overall, the goal of auto synthesis is to check whether the provided model consistently and completely covers all the provided change logs. If not, the model should be extended by adding missing types, predicates, and / or actions. If the input model is empty, then the synthesis will add all the required types, predicates, and actions needed to cover the provided change logs. When extending a model or generating a new model, multiple optimization goals are pursued. The goals can include: 1) the number of added actions should be minimal to improve the model’s readability and make easy any validations and adjustment by the user; 2) the number of preconditions of the added actions should be larger, rather than smaller. For example, it is better to overapproximate the set of preconditions than under approximate since it is easier for the user to identify and remove superfluous preconditions than add missing ones. Yet, the set of preconditions should not be too large to increase readability by the user and simplicity of the model. Change logs can differ based on the amounts of information included.. Each change log, after log segmentation has been performed, starts with a list of predicates representing an initial state (all predicates are true) followed by a sequence state change, such as what predicates become true and false in a given step or timestamp. An assumption is made that all state changes are fully observed and a state trajectory can be trivially reconstructed to capture the sequence of states that were observed in a dynamic system and recorded in the corresponding change log. 2466.PC.UTL.ap1 - 12 - Information about state changes can be enriched by information about actions that caused the state changes. Specifically, classes of changelogs can differ on how much information is provided about the actions. For example, L2 class includes full information about actions that made particular state changes, such as action names (labels) accompanied by an ordered list of parameters representing objects the given action manipulated, while L1 class include only information about action names (labels) of actions that made particular state changes. An L0 class may not contain any information about actions responsible for particular state changes, so the change logs include only information about state changes, and an L-1 class includes information about timestamps in which predicates are achieved or deleted by there is no distinction of atomicity of state changes, such as state changes caused by individual actions. Yet, no information about actions that are responsible for state changes may be present. Provided below is a sample changelog for the problem regarding solving the Towers of Hanoi. The changelog is an L2 class of logs. However, removing the action arguments in italics amends the changelog to an L1 class, while taking out action names, which are underlined, and the action arguments in italics, amends the changelog to the L0 type. Additionally, removing the “NEXT-STATE” separator makes the changelog an L-1 class of logs. Lines that include the phrase “PRECONDITION-HINT” are optional and can appear in all classes of logs. Changelog 2023-11-0414:23:41968; 100; (smaller d2 d3); init 2023-11-0414:23:41968; 100; (smaller d3 peg3); init 2023-11-0414:23:41968; 100; (on d2 d3); init 2023-11-0414:23:41968; 100; (smaller d1 d2); init 2023-11-0414:23:41968; 100; (smaller d2 peg2); init 2023-11-0414:23:41968; 100; (smaller d2 peg1); init 2023-11-0414:23:41968; 100; (smaller d3 peg2); init 2023-11-0414:23:41968; 100; (smaller d3 peg1); init 2023-11-0414:23:41968; 100; (on d1 d2); init 2023-11-0414:23:41968; 100; (smaller d1 peg3); init 2023-11-0414:23:41968; 100; (on d3 peg1); init 2023-11-0414:23:41968; 100; (clear peg3); init 2023-11-0414:23:41968; 100; (clear d1); init 2023-11-0414:23:41968; 100; (smaller d1 peg2); init 2466.PC.UTL.ap1 - 13 - 2023-11-0414:23:41968; 100; (smaller d1 peg1); init 2023-11-0414:23:41968; 100; (clear peg2); init 2023-11-0414:23:41968; 100; (smaller d1 d3); init 2023-11-0414:23:41968; 100; (smaller d2 peg3); init 2023-11-0414:23:41968; 100; NEXT-STATE; 2023-11-0414:23:41968; 101; PRECONDITION-HINT; (smaller d1 peg3) move peg3 d1 d2 2023-11-0414:23:41968; 101; PRECONDITION-HINT; (on d1 d2); move peg3 d1 d2 2023-11-0414:23:41968; 101; PRECONDITION-HINT; (clear d1); move peg3 d1 d2 2023-11-0414:23:41968; 101; PRECONDITION-HINT; (clear peg3); move peg3 d1 d2 2023-11-0414:23:41968; 101; (on d1 peg3); move peg3 d1 d2 2023-11-0414:23:41968; 101; (clear d2); move peg3 d1 d2 2023-11-0414:23:41968; 101; (not (on d1 d2)); move peg3 d1 d2 2023-11-0414:23:41968; 101; (not (clear peg3)); move peg3 d1 d2 2023-11-0414:23:41968; 101; NEXT-STATE; 2024-01-2612:00:00000; 102; PRECONDITION-HINT; (smaller d2 peg2) move peg2 d2 d3 2024-01-2612:00:00000; 102; PRECONDITION-HINT; (on d2 d3); move peg2 d2 d3 2024-01-2612:00:00000; 102; PRECONDITION-HINT; (clear d2); move peg2 d2 d3 2024-01-2612:00:00000; 102; PRECONDITION-HINT; (clear peg2); move peg2 d2 d3 2023-11-0414:23:41968; 102; (on d2 peg2); move peg2 d2 d3 2023-11-0414:23:41968; 102; (clear d3); move peg2 d2 d3 2023-11-0414:23:41968; 102; (not (on d2 d3)); move peg2 d2 d3 2023-11-0414:23:41968; 102; (not (clear peg2)); move peg2 d2 d3 2023-11-0414:23:41968; 102; NEXT-STATE; 2024-01-2612:00:00000; 103; PRECONDITION-HINT; (smaller d1 d2); move d2 d1 peg3 2024-01-2612:00:00000; 103; PRECONDITION-HINT; (on d1 peg3); move d2 d1 peg3 2024-01-2612:00:00000; 103; PRECONDITION-HINT; (clear d1); move d2 d1 peg3 2024-01-2612:00:00000; 103; PRECONDITION-HINT; (clear d2); move d2 d1 peg3 2023-11-0414:23:41968; 103; (on d1 d2); move d2 d1 peg3 2466.PC.UTL.ap1 - 14 - 2023-11-0414:23:41968; 103; (clear peg3); move d2 d1 peg3 2023-11-0414:23:41968; 103; (not (on d1 peg3)); move d2 d1 peg3 2023-11-0414:23:41968; 103; (not (clear d2)); move d2 d1 peg3 2023-11-0414:23:41968; 103; NEXT-STATE; 2024-01-2612:00:00000; 104; PRECONDITION-HINT; (smaller d3 peg3) move peg3 d3 peg1 2024-01-2612:00:00000; 104; PRECONDITION-HINT; (on d3 peg1); move peg3 d3 peg1 2024-01-2612:00:00000; 104; PRECONDITION-HINT; (clear d3); move peg3 d3 peg1 2024-01-2612:00:00000; 104; PRECONDITION-HINT; (clear peg3); move peg3 d3 peg1 2023-11-0414:23:41968; 104; (on d3 peg3); move peg3 d3 peg1 2023-11-0414:23:41968; 104; (clear peg1); move peg3 d3 peg1 2023-11-0414:23:41968; 104; (not (on d3 peg1)); move peg3 d3 peg1 2023-11-0414:23:41968; 104; (not (clear peg3)); move peg3 d3 peg1 2023-11-0414:23:41968; 104; NEXT-STATE; 2024-01-2612:00:00000; 105; PRECONDITION-HINT; (smaller d1 peg1); move peg1 d1 d2 2024-01-2612:00:00000; 105; PRECONDITION-HINT; (on d1 d2); move peg1 d1 d2 2024-01-2612:00:00000; 105; PRECONDITION-HINT; (clear d1); move peg1 d1 d2 2024-01-2612:00:00000; 105; PRECONDITION-HINT; (clear peg1); move peg1 d1 d2 2023-11-0414:23:41968; 105; (clear d2); move peg1 d1 d2 2023-11-0414:23:41968; 105; (on d1 peg1); move peg1 d1 d2 2023-11-0414:23:41968; 105; (not (on d1 d2)); move peg1 d1 d2 2023-11-0414:23:41968; 105; (not (clear peg1)); move peg1 d1 d2 2023-11-0414:23:41968; 105; NEXT-STATE; 2024-01-2612:00:00000; 106; PRECONDITION-HINT; (smaller d2 d3); move d3 d2 peg2 2024-01-2612:00:00000; 106; PRECONDITION-HINT; (on d2 peg2); move d3 d2 peg2 2024-01-2612:00:00000; 106; PRECONDITION-HINT; (clear d2); move d3 d2 peg2 2024-01-2612:00:00000; 106; PRECONDITION-HINT; (clear d3); move d3 d2 peg2 2466.PC.UTL.ap1 - 15 - 2023-11-0414:23:41968; 106; (on d2 d3); move d3 d2 peg2 2023-11-0414:23:41968; 106; (clear peg2); move d3 d2 peg2 2023-11-0414:23:41968; 106; (not (on d2 peg2)); move d3 d2 peg2 2023-11-0414:23:41968; 106; (not (clear d3)); move d3 d2 peg2 2023-11-0414:23:41968; 106; NEXT-STATE; 2024-01-2612:00:00000; 107; PRECONDITION-HINT; (smaller d1 d2); move d2 d1 peg1 2024-01-2612:00:00000; 107; PRECONDITION-HINT; (on d1 peg1); move d2 d1 peg1 2024-01-2612:00:00000; 107; PRECONDITION-HINT; (clear d1); move d2 d1 peg1 2024-01-2612:00:00000; 107; PRECONDITION-HINT; (clear d2); move d2 d1 peg1 2023-11-0414:23:41968; 107; (on d1 d2); move d2 d1 peg1 2023-11-0414:23:41968; 107; (clear peg1); move d2 d1 peg1 2023-11-0414:23:41968; 107; (not (clear d2)); move d2 d1 peg1 2023-11-0414:23:41968; 107; (not (on d1 peg1)); move d2 d1 peg1 2023-11-0414:23:41968; 107; NEXT-STATE; From the changelogs, all the existing predicate names and numbers of their parameters are known. However, the types of parameters remain unknown, unless the predicate is provided in the partial input model. To determine parameter types, the following algorithm can be used. A type A can be a subtype of B: A ⊑ B. Given the subtype relation, A is more specific than B, B is less specific than A, B is more general than A, and A is less general than B. A valid subtype relation should satisfy a set of conditions, including: 1) antisymmetric, where if A is a subtype of B and B is a subtype of A, then A and B are equal: A ⊑ B ∧ B ⊑ A ⇒ A = B; 2) transitive, where the subtype relation is transitive: A ⊑ B ∧ B ⊑ C A ⊑ C; 3) each type has a unique immediate super type with no multiple inheritance: ∀A: A = root ∨ ∃B : (A ⊑ B) ∧ (∀C : A ⊑ C ⇒ B ⊑ C); and 5) there is a most general type (unique root of the type hierarchy) named “object.” During an input transformation stage, a bipartite graph from the log that includes all information relevant to type synthesis is created. Each parameter position in an action or predicate is considered a slot. For example, the predicate can be (smaller ?d1 - disc ?d2 - disc), which is true if the value of ?d1 is smaller than the value of ?d2, has two slots. Generally, the predicate represents a relation between objects, such as disc ?d1 being smaller than disc ?d2. From the logs, a set of objects that occur is extracted, as well as slots 2466.PC.UTL.ap1 - 16 - in which each object is used. The determination of whether an object is used in a slot or not is determined, and the frequency of an object in use is not necessary. Thus, the relation between slots and objects forms a bipartite graph, or a list of tuples describing such a graph, with objects on the left and slots on the right, slots and objects together form the set of nodes V. There is an edge between an object and a slot if and only if that object is used in that slot at least once in the log. The goal is to identify a set of types and a valid subtype relationship of the types. Each slot and object can be assigned a type, such that if (object, slot) appears in the input, then the type of object is at least as specific as the type of slot. A type hierarchy can be built by starting from the bipartite input graph G. V represents nodes of the graph, while E represents edges of the graph. D is a set of subsets of V that can be constructed. For each C ∈ D, a type TC is created. C represents a subset of the vertices of V. There is a one-to-one correspondence between elements in D and their types, such that for two subsets C, C’ ∈ D, the following is true: For V ∈ D, type TVis the most general type of object. The other elements of D are constructed by finding all connected components of the graph G, using union find, and adding the set C of all vertices in the connected component to D. A connected component is a subgraph where every node is connected to every other node in the subgraph via a path, such as a sequence of edges. For each component, the slot with the most incident edges is removed from the graph G. For example, given a most used slot within a C ∈ D, an assumption is made that the slot must get the most general type of all types used in the group. The type can be TC and by construction, all other slots will get a more specific type. After removing nodes belonging to the most used slots, the algorithm can be repeated by finding all connected components in the remaining graph, adding the set of all vertices in the connected component to D, and removing the node belonging to the most used slot. Connected components found in one repetition of the algorithm are subsets of the connected components found in the previous repetition of the algorithm. Thus, the resulting type hierarchy satisfies the required conditions. Slots get assigned the type of the connected component they belonged to before being removed. Objects get assigned the most specific type of all slots they appear in. All slots in which an object appears, must end up in the same connected component and the types will be placed in a single chain. If a simple, flat type hierarchy is desired, the algorithm can be terminated after identifying the first set of connected components. 2466.PC.UTL.ap1 - 17 - As many types as there are slots are created with long chains of types that do not have a split. To simplify the type of hierarchy, all types that are on a chain are unified, such that all types that have multiple subtypes remain, while all types with only a single subtype get removed. The types assigned to objects and slots are updated accordingly. With respect to naming the types, an LLM can be used to give meaningful and human friendly names to the generated types. The LLM can be provided with the names and asked to summarize the names into a short phrase. The actions for the model can be generated using diff, merge, and refine stages. FIGURE 7 is a flow diagram showing, by way of example, a process 90 of the diff stage for generating actions. During the diff stage, an initial set of lifted actions is constructed. A list of ground actions is first generated based on neighboring states 92 from the logs 91. For example, there is a transition from state S to S’ and set of predicates A was added, while set of predicates B is removed. The ground action for this transition is generated with: 1) effects: add A, remove B; 2) preconditions: all the ground predicates in S that share all their parameters with any predicate in A or B (other heuristics are possible, which are offered to be implemented in the Refine stage discussed in detail below); and 3) parameters: all the parameters occurring in the preconditions and effects. In one embodiment, the logs can be received in the proper order or alternatively, the log files may require ordering to organize actions in proper sequence. For each change, a determination is made whether the change is explained with an existing action 93 and if not, create a new action and store the new action in memory 94. However, if the change can be explained, then continue to the next change. The existing action can be identified by descriptions 95 and the actions if they explain a change can be included in a set of learned actions 96 with any new actions. The ground actions are lifted 94 and normalized, and collected into a set of distinct lifted actions. Lifting focuses on generalization of the actions, such as by replacing each object with a variable. For example, a ground (or grounded) action can be “pickup disc3” and a corresponding lifted action can be “pickup an object of type disc.” When normalized, the parameters are renamed, while the preconditions, effects and parameters are reordered, which removes symmetries and allows recognition of cases when several ground actions belong to the same lifted action. Also, the normalized lifted actions are collected into a set data structure so duplicates are filtered automatically. In comparison to theoretically optimal learned action, there may be differences of missing effects, such as when the add (remove) effect was already true (false) in S’. 2466.PC.UTL.ap1 - 18 - Alternatively, or in addition, to missing effects, missing and extra conditions can exist due to the heuristic selection of preconditions. Negative preconditions and precondition predicates with no parameters can be missing at times. Finally, parameters can also be missing due to missed effects and preconditions. Learned actions from the diff stage can enter the merge stage, which aims to reduce a number of the learned actions and improved accuracy of the learned actions by adding missing effects and reducing a number of incorrect preconditions. Specifically, pairs of actions learned in the diff stage can be merged. For example, two actions A1 and A2 can be merged if there is an action A12 that each grounded occurrence of A1 and A2 in the input logs can be replaced by a grounded instance of A12, which is often satisfied when the set of effects of one action is a subset of the set of effects of the other action. FIGURE 8 is a flow diagram showing, by way of example, a process 100 for performing the merge stage. Pairs of learned actions are paired 101 and a determination is made as to whether the two actions in the pair can be merged with respect to the log 102. The actions can be merged when it is possible to replace the two actions with a single action. If no, then a different pair of learned actions is reviewed. However, if yes, then the pair of actions is merged and the current set of learned actions is updated 103 by removing the two actions and by inserting the merged action. When merging two actions, the new action’s set of effects is defined as the union of the effects of the two actions, and the set of preconditions is defined as the intersection of preconditions of the two actions. The parameters of the merged action are computed from the preconditions and effects by taking all the variables and constants that appear in them. Also, the merged action replaces the two actions from which the merged action was created. A particular order can exist for merging the actions by using the low-hanging fruit algorithm to improve performance. First, actions with identical effects can be merged. Effects are determined to be identical when they differ only in preconditions. The identical effects can be easy to detect, and quickly and drastically reduces a number of actions for the next stages. Next, actions where the effects of one are a subset of the effects of another can be merged. Finally, the remaining actions that can be merged are completed. The merged actions are provided to the refine stage, which improves the generated actions by adding predicates into their preconditions. FIGURE 9 is a flow diagram showing, by way of example, a process 110 for performing the refine stage. Using input from the merge stage 117 and information from the logs 111, a determination is made as to whether any new conditions can be added 112 to the actions from the merge stage. If so, the 2466.PC.UTL.ap1 - 19 - action is updated by adding the new preconditions 113 and saving the updated action in a set of learned actions 116. If not, another learned action 115 is reviewed to determine whether preconditions should be added. Once all the actions have been reviewed, the process terminates 114. The refine stage can be redundant depending on the heuristics used in the first stage when selecting an initial set of preconditions. The heuristics used at this stage include add all predicates that are true in each state that precedes the execution of the given action. Rigid predicates that are constant and unchanging must be handled carefully, otherwise the rigid predicates will end up in the preconditions of all actions. Predicates are added that share at least one parameter with the set of parameters in the effect predicates. Also, a probability of a predicate to be a preconditions of an action can be computed based on how often the given predicate appears in the observed world state. Predicates that are rare are more likely to be preconditions of actions for which they are always present in their preceding states. The diff, merge, and refine stages are performed when starting with an empty set of actions. For incremental usage, when some actions are already given in the input, modifications are made to each of the diff, merge, and refine stages. For instance, in the diff stage, the change is matched with a grounding of one of the given actions and if a match cannot be found, a new action is generated as described above with respect to FIGURE 7. In the merge stage, the input actions are not considered since they should be preserved in their original form, while in the refine stage, when preconditions are added to the input actions, the added predicates are marked when presenting the results so the user can easily recognize and validate the changes. Also, the user is notified if any of the input actions never matched any of the changes in the logs. In such a case, the given input action may be unnecessary and can be deleted. Returning to the Towers of Hanoi game, a sample changelog from playing the game with three pegs and three discs is provided below: 1966‑07‑3008:30:00 [(smaller d1 d2) (smaller d1 d3) (smaller d2 d3) (smaller d1 peg1)(smaller d2 peg1) (smaller d3 peg1) (smaller d1 peg2) (smaller d2 peg2)(smaller d3 peg2) (smaller d1 peg3) (smaller d2 peg3) (smaller d3 peg3) (on d3 peg1) (on d2 d3) (on d1 d2) (clear d1) (clear peg2) (clear peg3)] 1966‑07‑3008:30:20 [(clear d2) (on d1 peg3) (-on d1 d2) (-clear peg3)] 1966‑07‑3008:30:37 [(clear d3) (on d2 peg2) (-on d2 d3) (-clear peg2)] 1966‑07‑3008:30:55 [(clear peg3) (on d1 d2) (-on d1 peg3) (-clear d2)] 2466.PC.UTL.ap1 - 20 - The changelog is in a correct format and does not require segmentation. Specifically, the changelog includes four entries, each with a timestamp and list of predicates that change at the specified time. There are three types of predicates displayed in the changelog, including: (smaller ?argument1 ?argument2), (on ?argument1 ?argument2), and (clear ?argument1). The set of objects in the change log are: d1, d2, d3, peg1, peg2, peg3. Since all of the objects are used at least once as an argument of the predicate clear, the objects can be determined to have the same type, which can be labeled as type discOrPeg. The type declarations can be added to the predicate definitions. Once the predicates and types of objects in the problem are known, the actions can be synthesized. The problem identified in the game example is how to solve the Towers of Hanoi game. Based on the second line of the changelog, the changes are identified as (clear d2) (on d1 peg3) (-on d1 d2) (-clear peg3). The changes identify the effects of the learned action as follows: :effect(and not(clear(peg3)) not(on(d1, d2)) clear(d2) on(d1, peg3)) To compute the preconditions of this action, an action that moves disc d1 from being on top of disc d2 onto peg3, the predicates that include all the objects from the set of objects occurring in the effects, such as peg3, d1, and d2, are taken to yield: (smaller d1 d2) (smaller d1 peg3) (smaller d2 peg3) (on d1 d2) (clear d1) (clear peg3), and will become preconditions of the generated action: (:action a0 :precondition(and (smaller d1 d2) (smaller d1 peg3) (smaller d2 peg3) (on d1 d2) 2466.PC.UTL.ap1 - 21 - (clear d1) (clear peg3)) :effect(and not(clear(peg3)) not(on(d1, d2)) clear(d2) on(d1, peg3)) ) Next, the action should be lifted and normalized. During lifting, each object is replaced with a variable. During normalization, the predicates and variable names (for example, lexicographically) are sorted to remove symmetries. After lifting and normalizing, the following action is generated: (:action a0 :parameters(?disc-1 ?disc-2 ?disc-3 - discOrPeg) :precondition(and clear(?disc-1) clear(?disc-2) on(?disc-2, ?disc-3) smaller(?disc-2, ?disc-3) smaller(?disc-2, ?disc-1)) smaller(?disc-3, ?disc-1) :effect(and not(clear(?disc-1)) not(on(?disc-2, ?disc-3)) clear(?disc-3) on(?disc-2, ?disc-1)) ) The lifting and normalizing are performed on the remaining state transitions to get the following action for all remaining state transitions in which the action is involved: (:action a1 :parameters(?disc-1 ?disc-2 ?disc-3 - discOrPeg) 2466.PC.UTL.ap1 - 22 - :precondition(and clear(?disc-1) clear(?disc-2) on(?disc-2, ?disc-3) smaller(?disc-1, ?disc-3) smaller(?disc-2, ?disc-1) smaller(?disc-2, ?disc-3)) :effect(and not(clear(?disc-1)) not(on(?disc-2, ?disc-3)) clear(?disc-3) on(?disc-2, ?disc-1)) ) Actions a0 and a1 are similar, but they differ since a0 has smaller(?disc-3, ?disc-1), while a1 has smaller(?disc-1, ?disc-3). The two actions can enter the merge phase, and since both actions have the same effects, the actions can be merged. The merged action will inherit the set of effects from the two actions. The set of preconditions will be the intersection of the sets of preconditions of the two actions. The merged action appears below: (:action m01 :parameters(?disc-1 ?disc-2 ?disc-3 - discOrPeg) :precondition(and clear(?disc-1) clear(?disc-2) on(?disc-2, ?disc-3) smaller(?disc-2, ?disc-1) smaller(?disc-2, ?disc-3)) :effect(and not(clear(?disc-1)) not(on(?disc-2, ?disc-3)) clear(?disc-3) 2466.PC.UTL.ap1 - 23 - on(?disc-2, ?disc-1)) ) More human friendly names can be determined for the actions using an LLM model, which can be provided with PDDL definitions of the action and predicates. For example, the action “move” can be changed to “move-disc.” In this example, the refine stage would not add any new preconditions to the single learned action and thus, synthesis would terminate. Instead of using the diff, merge, and refine stages, a match or spawn process can be used. FIGURE 10 is a flow diagram showing, by way of example, a process 120 for performing match or spawn. The match or spawn process scans the logs 121 sequentially and for each pair of neighboring states 122, a change is compared to an existing lifted action in the model to see if a match exists. If the change cannot be matched to any existing actions, then a new actions is spawned 124 and added to the model. A check is performed to see if the newly spawned action can be merged with an existing action 125 and if so, the learned action is replaced with the new merged action 126. However, if the new action cannot be merged, the new action is added to the current set of learned actions 127, which is associated with previous action descriptions 128. If the change can be explained by an existing action, the next pair of neighboring states is processed. The match or spawn process addresses the issue of the diff, merge, and refine process regarding expensive merge and refine phases by preliminary merging the spawned action as the logs are scanned instead of doing it in a separate stage. Unlike the diff, merge, define process, the match or spawn process has only a single stage and is intrinsically incremental. Different classes of logs exist based on an amount of information the logs contain. When using the top-down synthesis (TDS) algorithm, the easiest log is identified and processed. For example, the L2 class is simple and includes more information than lower- level logs and allows for better and more accurate model synthesis. FIGURE 11 is a flow diagram showing, by way of example, a process 130 for converting logs prior to processing. A determination is made as to what kind of logs are available 131. When L2 logs are available, an L2 algorithm for synthesis is performed 134 on the log. If only L1 logs are available, the L1 logs can be converted or elevated 133 to L2 logs by synthesizing missing information, including the parameters of the grounded actions, and applying an L2 synthesis algorithm to obtain a domain model. 2466.PC.UTL.ap1 - 24 - Similarly, if the available logs are classified as L0 class, then the L0 logs can be converted or elevated 132 to L1 logs by computing the missing information, such as the action names for each transition, and then applying the method for handling L1 logs. In general, elevating logs, such as from L0 to L1 and L1 to L2, can be performed in different ways and involves heuristic decisions. An L2 log includes full information about the action that made a particular state change, including storing the action’s label and an ordered list of arguments, which are the objects that the action manipulated. The state changes, or transitions, are grouped by their respective action labels. Each action is synthesized in isolation based on transitions with a common label. Let TLbe the set of transitions with the same label L, which will be used together to synthesize a single action. Each transition carries with it the label L, an argument list A, and is from a before-state S to an after-state T, which are sets of ground predicates, such as predicate symbols applied to objects. To synthesize an action for the label L, a list of parameters P of length matching that of the argument lists for that label (all transitions for label L necessarily agree on this length) is fixed. Appropriate add (add) and delete (del) effects, as well as preconditions (pre) need to be determined. The add and delete effects and preconditions are going to be sets of predicate symbols applied to the parameters from P. Each of the observed transitions from TL must be explained in the following sense. Let σ be a substitution mapping the parameters from P to the arguments of A (in order). Then, for every f ∈ pre, fσ holds in S and that: T = (S ∖ {eσ ∣ e ∈ del}) ∪ {eσ ∣ e ∈ add}. The L2 synthesis algorithm includes three phases regarding synthesizing the add and delete effects. In the first phase, for every transition in TL and every observed state change, all the possible add / delete effect candidates that could explain the change are collected. In a second phase, the candidates are pruned and those which are incompatible with some other transition in TLare discarded. Finally, a small set of add and delete effects from the remaining candidates are selected so no observed state change remains unexplained. For every transition in TLand every fact that was observed as added f ∈ T ∖ S, a set of explanation candidates are generated, where the set of explanation candidates are explainf= {f′∣ f′σ = f}, andfandf′ share the predicate symbol, but the candidatesf′apply the symbol to parameters from P, andσis as above. All the explain sets are collected 2466.PC.UTL.ap1 - 25 - over all transitions and all observed added facts in set (of sets) calledmeasL(for must- explain-adds). The set medsL(must-explain-dels) is formed to explain all the observed deletions. In a second phase, the candidate explanations, which are disconfirmed by some other transition in TL are pruned. For additions, given any^^′ ∈ ^^ ^^ ^^ ^^ ^^ ^^ ^^^^∈ ^^ ^^ ^^ ^^^^if there is a transition for which ^^′^^ ∉ ^^, such a candidate shall be pruned. Deletions are analogous. Pruning is done in another pass through all the transitions. Disconfirmations are only sought for those add / delete effects which have been speculated as explanations, which reduces an amount of work needed. The third phase uses the prunedmeasLandmedsLsets to finally synthesize the add and delete effects for the action with label L. The third phase relies on a minimality principal, trying to introduce as few effects as possible while guaranteeing to explain all the observations, which is performed in a heuristic fashion, iteratively committing to one effect at a time. In an example, for adding since deleting in analogous, start withadd = ∅. Then, as long as there is an expl ∈ measLsuch that expl ∩ add = ∅ In, and an effect f ∈ ⋃ measL, which maximizes the size of{expl ∈ measl ∣ expl ∩ add = ∅ and f ∈ expl}, breaking ties arbitrarily, is selected. Then, add f to add to set add := add ∪ {f}.To determine the preconditions of an action, the set of potential preconditions for each transition is formed and an intersection of the set is determined. Also, to build the set of potential preconditions, the set of non-negated ground predicates in the before state that only use objects that are used as parameters of the action are considered. An inverse of the substitution mapping action parameters is applied to objects to get the potential preconditions. The substitution may not be injective and thus, there may be multiple possible inverse values. All possible combinations of the inverse values can be created. Elevating L1 logs to L2 logs In L1 logs, there are no argument lists supplied with a transition, so a transition only consists of the action label L, a before-state S, and an after-state T. The extra challenge, compared to working with L2 logs, therefore, comes from the need to additionally synthesize the exact form of σ, the substitution mapping in each transition of the action’s parameters to concrete objects. The number of parameters of each action should be determined. 2466.PC.UTL.ap1 - 26 - Like with the L2 logs, the L1 synthesis algorithm groups transitions according to the action label L into sets of transitions TL and works with each group separately. To determine the initial number of parameters for an action with label L, the L1 synthesis algorithm scans the transition in TL and counts a number of distinct objects that arearguments to predicates getting added or deleted in each, such as either the arguments ∈ T ∖ S or if f ∈ S ∖ T. A maximum count over all transitions can be taken as the initial number of parameters. If, during the follow-up steps, this number of parameters gets recognized as insufficiently small and can be increased by one and the whole process restarts. A heuristic bound on the number of such increases can be imposed upon the reaching of which the synthesis attempt is abandoned and the user gets notified. To speculate the exact form of separately for every transition in TL and to be able to backtrack over the main decisions regarding the action’s add and delete effects to synthesize, the L1 algorithm is built around an encoding of the associated problem into propositional logic. The L1 algorithm uses a dedicated tool, called a SAT solver, to search for the synthesis solution. The encoding process is incremental to maintain efficiency. The process alternates between a speculation phase, which initially focuses on a single transition only, and a verification phase, in which all other transitions are scanned and checked for compatibility with the speculated solution. If a transition is found incompatible, it is added to the set of transitions entering the speculation phase. A second important design choice, which is conceptually shared with the L2 synthesis algorithm, but manifests in its own way, is to first focus on explaining what is changing in a transition and only later, lazily, express what does not change and should therefore be excluded from the speculated solution space. The encoding works with two basic sets of propositional variables; however, other variables, such as auxiliary variables, can be used as well. The first set describes the shape ofσin the following way: for every parameter pi we have the variables bind(t, pi, oj), where ojranges over all the objects and encodes that σ(pi) = ojis for transition t ∈ TL. For every piand for every t ∈ TLexactly one bind(t, pi, oj) should be true. The second set contains variables of the form add(pred, pi, …) and del(pred, pi, …) for every predicate pred among the facts in T ∖ S and S ∖ T, respectively, and for every vector of formal parameters pi, ... of a length corresponding to the number of arguments pred expects. Unlike the bind variables, these variables are not indexed by transition t ∈ TL, s their meaning is for the whole action L to be synthesized and therefore, non-local. To 2466.PC.UTL.ap1 - 27 - explain an observed addition of one predicate, for example of pred(a, b) ∈ T ∖ S for some transition t, the encoding contains the formula: ⋁ iadd(pred, pi, pj) ∧ bind(t, pi, a) ∧ bind(t, pj, b), where the big disjunction is over all the possible vectors of available parameters of length two. To exclude a speculation solution which conflicts with an observed non-change, the following example is provided. A true variable add(pred, p1, p2) in the tentative solution assignment exists, along with, for example, bind(t, p1, a) and bind(t, p2, b) set true as well, while the corresponding ground fact of pred(a, b) is not in T formula to the encoding for some transition t ∈ TL. The following formula is then added to the encoding: ¬add(pred, p1, p2)∨ ¬bind(t, p1, a)∨ ¬bind(t, p2, b)A request is made to the SAT solver for a different solution. The process of lazily correcting the solution is performed until inconsistencies with the observations in transitions remain. Returning to the distinction between the speculation phase and the verification phase, the SAT solver is free to choose values for any variable to satisfy given formulas during the speculation phase. However, for the verification phase, a solution to the variables of the form add(pred, pi, …) and del(pred, pi, …) is fixed and supplied as unit clause assumptions. A determination is made as to given the currently encoded global solution to the action’s add and delete effects whether there is a location solution in terms of the substitution σ for another transition t ∈ TL. To further improve the efficiency of the algorithm, the speculation phase is ended before moving to the verification phase by explicitly minimizing the solution and aiming for a small number of variables in the form add(pred, pi, …) and del(pred, pi, …) to be set to true. Once the add and delete effects for an action are synthesized, along with the associated parameter to argument mappings σ, preconditions can be synthesized as described for the L2 logs. However, preconditions may need additional action parameters that are not used for any effect. Thus, additional parameters may need to be added. Whether additional parameters are needed is determined using the PRECONDITION-HINT; lines in the log file. An attempt is made to explain precondition predicates using the same method used for explaining addition predicates, with the only difference using pre(pred, pi, …) variables instead of add(pred, pi, …). During the process, additional parameters can be added. To 2466.PC.UTL.ap1 - 28 - account for the substitution σ that was found to explain the addition and deletion of predicates, the values of the bind(t, pi, oj ) variables are fixed accordingly and the formula is simplified. In contrast to the addition and deletion of predicates, the predicates are not considered all at once. Rather, the predicates are considered one at a time. If the process fails for one of the predicates, the precondition hint will be ignored and no new parameters will be added. Once the process is complete, preconditions can be synthesized as described for the L2 logs. Elevating L0 logs to L1 logs To obtain L1 logs from L0 logs, action labels need to be added to the observed transitions, which is the same as grouping them such that each group includes transitions that are explained by a single action. One way to group the transitions is by their signatures. The signature of a transition is defined by the names of the predicates and negated predicates in the transition. For an example, consider the line entry of the logs from the Towers of Hanoi example: (clear d2) (on d1 peg3) (-on d1 d2) (-clear peg3). The transition described from the log entry yields the following signature: {clear, on, -on, -clear}. Transitions with an identical signature can be grouped together and get the same label assigned. This is one example of one way to group and label the transitions, and other methods are possible. Also, in one embodiment, transitions with different signatures can be grouped together, for example, when one signature is a subset of another one. If a different embodiment, there may be an advantage to not grouping together the transitions with the same label, such as for actions that have identical effects, but distinct preconditions. An example is movement on a grid graph with actions, described below: (:action move-up :parameters(?from ?to - location) :precondition(and at(?from) above(?to ?from)) :effect(and not(at(?from) at(?to)) ) (:action move-right :parameters(?from ?to - location) :precondition(and 2466.PC.UTL.ap1 - 29 - at(?from) left-of(?from ?to)) :effect(and not(at(?from) at(?to)) ) ... analogously move-left and move-down The actions above have identical effects and could be merged into a single move action: (:action move :parameters(?from ?to - location) :precondition(and at(?from) :effect(and not(at(?from) at(?to)) ) For the new move action, any precondition that would restrict the ?from and ?to parameters cannot be learned to prevent conflicting preconditions. Although the logs may include the predicates (above ?l1 ?l2 - location) and (left-of?l1 ?l2 – location), the predicates can only be used if four separate move actions are synthesized for each direction instead of a single universal move action. To prevent groupings of observations with different log labels, a change can be added to the L0 synthesis. The log labels are a form of additional information provided in the logs that assign a label to each transition. If the log data allows for a 1-1 mapping between log labels and action, l1 synthesis can be used and the log labels can be treated as action labels. Workbench The user interface for automated domain synthesis can include a synthesis workbench, which includes input selection, controls and synthesis outputs, and can drive the domain synthesis user through the steps allowing user interaction. FIGURE 12 is 2466.PC.UTL.ap1 - 30 - screenshot of the user interface of the system for automated data-driven domain synthesis of FIGURE 2. The user can select a log 145 to synthesize the domain along with a domain draft 151, which can be partially complete or empty. A pre-selection of logs or domain 146 for open work can be loaded in a faster manner than selecting logs and domain from local directories. The loaded log can be displayed for examination and validation that all necessary data is included and that the log conforms to an L2, L1, or L0 type of log. FIGURE 13 is a screenshot showing, by way of example, a log 155 loaded via a user interface. The log 155 includes multiple entries, each identified by a date and timestamp. Further, the loaded domain draft can also be displayed via the user interface. FIGURE 14 is a screenshot showing, by way of example, a domain draft 160 loaded via the user interface. Once loaded, the domain draft 160 can be examined and edited according to the user. Returning to the discussion with respect to FIGURE 12, the user interface includes controls 152 to enable and disable actions 141, predicates 142 or complete types 143, which are inferred from the logs. The controls 152 can include toggle buttons, dropdown menus, or other types of controls. By disabling a control button for an option under one of the categories of actions, predicates or types, the option is filtered from the logs, which simplifies the domain and enables the iterative approach to synthesis. The slot types 143 of particular predicates are shown with hierarchy, which defines more or less general types. Disabling a particular type automatically disables more specific types. A set of all objects 144, present in the log, is listed for display via the user interface. One or more objects can be optionally disabled to remove that object from the logs, which may not be important for synthesis. The logs are visualized in the form of a changelog, in a table. The changelog can include predicates, which are shown as table columns. Each row can state each log object with a + or – sign to denote that the object in this predicate makes the predicate true or false for a given action. Alternatively, the changelog can include a simpler view where each row represents one action and the columns of the table state predicates and object values which are true (positive) as a result of the action or false (negative). The changelog representations can be selected by the user and allows the user to identify and view parts of the logs on which the user is focused. In addition to changing a view of the changelog, the automated domain synthesis can have multiple options 150 that simplify the approach to logs, including the option of log conversions from L0 to L1 and L1 to L2 formats and an amount of help with pre-loaded domain, including whether to take actions and predicates into account. 2466.PC.UTL.ap1 - 31 - The user interface can also include controls for synthesis. FIGURE 15 is a screenshot showing, by way of example, options for controlling and displaying synthesis metadata via the user interface. Once the user has selected options and representations for the predicate and action data, synthesis can begin via a button 181. As a result, a synthesized domain 182 and synthesis metadata 183 are provided. The synthesis metadata 183 provides detailed information about a frequency of synthesized actions, which can reveal under-used actions, derived from pre-loaded domain, or over-used actions which may reveal a too general synthesized action. The mapping from L1 action names to newly synthesized actions is possible as well. Learned action names apply to the case when the pre-loaded domain is empty and the synthesis algorithm extracts these from logs with the help of LLM, which provides better user-readable action names. A synthesized domain is shown to the user for validation and examination. As a domain-expert, the user can spot too general or too specific parts of actions that can be adjusted in a next step of the iterative approach to domain synthesis. FIGURE 16 is a screenshot showing, by way of example, a synthesized domain 170. The synthesized domain 170 includes predicates, actions, and effect. The user interface can also provide more options for auto synthesis. FIGURE 17 is a screenshot showing, by way of example, additional synthesis options provided by the user interface 190. The synthesis workbench of the user interface offers a way for debugging synthesized domain. Specifically, a planner 191 is connected to the workbench and allows for running the synthesized domain and solving a planning problem that is created from the logs by selecting starting and goal states. The user can select a particular planner 192, each having their specialties with respect to domain processed, performance options and as such, by running the synthesized domain and original log file-based problem, it can quickly partially validate possible missing or superfluous actions and preconditions. If found, the plan is displayed to the user. Otherwise, the synthesis workbench reports “plan does not exist” or “plan not found within the given time limit.” This can be used for a first quick check for the user that the domain model is viable with respect to all settings in previous numbered points. The initial and goal state is given by the slider which selects first line of changelog as the input state and last line of changelog as a goal state. This allows for delicate selection of transfers between logs and their interactive, incremental testing before moving to the next step. Subsequently, a button can be selected to open a new tab in a steps selection area 202. FIGURE 18 is a screenshot showing, by way of example, options for the synthesized 2466.PC.UTL.ap1 - 32 - domain displayed via the user interface 200. The next button 201 is displayed and guides the user to a step in which the log record remains the same, but the domain used is the result of synthesis. All the controls are refined with respect to the synthesized domain and the user can edit the synthesized domain, enable / disable action preconditions to make them more or less specific, to change the synthesis options, and to run the next round of synthesis and further examine its output. Once the button 201 is selected, the new tab appears where the domain can be changed, predicates added or deleted, testing performed until the user is satisfied with the final model or next step is pressed again to experiment more with the domain. When modeling a domain, mistakes can be made. The mistakes can include when the model underapproximates the modeled problem and allows for logs that should not be possible or when the model overapproximates the problem and there is no plan or some logs that should be possible, cannot be generated. Thus, debugging helps resolve such mistakes. Generally, debugging the model that underapproximates is easier since the user identify the mistake from invalid logs obtained from the planner. Debugging a model that overapproximates the problem is harder since the user is not receiving any useful information due to the non-existence of logs. FIGURE 19 is a flow diagram showing, by way of example, a process 210 for debugging models that overapproximate a problem. When an expected plan cannot be found by a planner, debugging can occur, during which an initial state, such as defined in the problem.pddl, is identified and set as the current state 216. The system displays all grounded actions that are applicable at the current state 215. The user inspects the displayed actions and if an action is identified that should be applied next 214, that action can be selected 213. Otherwise, if not applicable, the user can enter the action expected to be applicable 217 and the debugging tool can determine why that action is not applicable, such as the preconditions do not hold. The unsatisfied preconditions of the given action can be displayed 218. The selected action is applied to the current state 212 and a new state is created, which is designated as the current state. With the debugging process, the user can find out why a plan they believe should be possible under a given model is actually not possible. The user can continue selecting a next action they believe should follow until the action they wish to choose does not appear between the applicable actions. At this point, looking at the already executed actions and changes on the states, should give some hints regarding the modelling mistake. Furthermore, a determination as to whether the action expected to be applied is displayed, 2466.PC.UTL.ap1 - 33 - can be automated by having an algorithm heuristically choose an action or set of actions. In the setoff action view, the evaluator could quickly look at traces n-steps from the initial state and identify missing or biased vectors. Alternatively, an entire plan, such as a sequence of actions, that are expected to be valid can be provided. The tool can determine which is the first non-applicable action and why, such as those preconditions that do not hold. Evaluating models Given a PDDL domain model, test cases can be automatically generated to evaluate the performance of various automated planners on solving various problems of that domain. For such purposes, it is important to generate problem instances that are solvable and to generate sets of instances of various difficulty levels to test the scalability of the planning system. Both points can be addressed using a semi-automatic method of problem generation. FIGURE 20 is a flow diagram showing, by way of example, a process 230 for evaluating models. A semi-automatic method of problem generation can be used for evaluation. For example, in step 1, let k be a parameter defining the difficulty level of the desired problem (step 231). A fixed amount (depending on the properties of the problem domain and k) of objects of each type can be generated (step 232), and some fixed amount (depending on the properties of the problem domain and k) of predicates using the objects generated in step 2 can also be generated (step 233). The generated set of predicates define the initial state of the generated test-case. The initial state is set to be the current state in the algorithm. All applicable actions in the current state are calculated (step 236) and selected in a uniformly random fashion, such as by using a pseudo-random generator algorithm. If the set of applicable actions is empty, a different number of predicates can be generated. The selected action is applied to the current state and the resulting state is set to be the new current state (step 235). Calculating applicable actions (step 236) and applying the selected actions to a current state (step 235) are repeated for a fixed number of times (depending on the properties of the problem domain and k, which can be multiplied by a constant). If there is at least one state (step 235), then the application action can be selected and applied to the current state (step 234). A determination is made as to whether enough states have been identified based on the parameter k, for example have at least 10 times k states been identified (step 237). If so, the process is finished and the state arrived at defines the goal state of the generated test- case (step 238). 2466.PC.UTL.ap1 - 34 - Each problem generated using this method is guaranteed to be solvable, indeed one solution is the sequence of actions selected in the iterations of step 3. By increasing / decreasing the parameter k that can increase / decrease the difficulty of the generated problems. The models can be used in a composite AI system that integrates multiple models and techniques, such as AI planning, machine learning, scheduling, graph theory, constraint programming and large language models, can be used for solving problems across different domains, such as industrial automation and intelligent service creation. Learning a single end-to-end model is rarely computationally feasible. Instead, end- to-end models are typically composed of multiple heterogeneous models, some of which are machine-learned while others are human-built using AI and operations research modeling techniques. Accordingly, a system and method for the design, development and deployment of composite AI systems helps support and encourage widespread use of the composite systems. Key building blocks for constructing automation systems designed as a graph are provided, with the flexibility to transform into many graph forms for deployment. A data and model aware, logic-driven control node central to defining a composition that unifies a plurality of disparate models is introduced. The compositions can be used to generate code, be containerized, and then deployed autonomously. The underlying structure of a composite AI composition definition is a graph, which is a mathematical structure used to model pairwise relationships between objects. A graph includes a set of vertices, called nodes or points, and a set of edges, known as links or lines, that connect pairs of vertices. The vertices or nodes are the fundamental units or points in a graph. Each vertex can represent an entity, such as a process step in a procedure. The edges or links can each represent a relationship, interaction, or pathway between the entities represented by the vertices. Describing a system as a graph is beneficial for implementation because the graph offers a clear visualization and interpretation of relationships and interactions through nodes and edges. The visualization aids in understanding and communicating the system’s structure with little ambiguity. Graph-based representations allow the user of efficient algorithms for tasks like searching, finding shortest paths, and optimizing resource allocation. Additionally, graphs support modular design, where components can be independent developed and reused, enhancing maintainability and scalability, as well as 2466.PC.UTL.ap1 - 35 - accommodate dynamic changes, making them ideal for evolving systems that adapt over time. Further, graphs can effectively represent complex hierarchical and network structures, capturing interdependencies among various parts of the system. This approach aids in problem decomposition, which allows the system to be broken down into manageable subproblems. Various data structures, such as adjacency lists and matrices, efficiently store and manipulate graph data, ensuring scalability for large datasets. Graphs are also useful for simulation, modeling, and predictive analytics, providing valuable insights for planning and decision making. Overall, graphs used to describe systems can leverage their structured representation and the power of graph theory to make the system more efficient, flexible, and scalable. For many implementation instances, one types of graph can be used including Directed Acyclic Graphs (DAG) and Behavior Trees (BT). A DAG is a special type of graph that has two main characteristics, being directed and acyclic. With respect to the directed characteristic, the edges in the graph have a direction, meaning they go from one vertex to another specific vertex, whereas for acyclic characteristic graphs, the graph has no cycles, which means there is no way to start at a vertex and follow a directed path that loops back to the same vertex. A DAG is useful for representing systems by effectively managing order and dependencies and ensuring tasks are executed only after their prerequisites are met. The cycle free structure eliminates circular dependencies and simplifies dependency resolution, while avoiding deadlocks. The DAG supports efficient algorithms for optimizing performance and resource utilization. DAGs ensure correct order and efficient execution, and are valuable in predictive modeling and simulations to provide clarity and simplify the representation of complex systems. Overall, DAGs offer a robust and efficient framework for system management and optimization. A BT is a hierarchical graph that defines the behavior of autonomous agents, particularly in video game Artificial Intelligence (AI), robotics, and complex system control. A BT includes nodes representing different tasks or actions, structured in a tree- like manner where the root node is the starting point. Each node in the tree can be a composite, decorator, or leaf node, where composite nodes control the flow of execution by organizing other nodes, such as sequences or selectors. Decorator nodes modify the behavior of their child nodes by adding conditions or loops, while leaf nodes perform actions or checks. BT allow for clear and modular representations of complex behaviors, making them easy to design, understand and maintain. BT also support flexibility and 2466.PC.UTL.ap1 - 36 - reusability as different behaviors can be combined and nested to form sophisticated decision making processes to allow agents to reach dynamically to changing environments and achieve goals efficiently. FIGURE 21 is a graph diagram showing, by way of example, a behavior tree 10 for monitoring and adapting a plan in execution. A node, such as a top node 11 and a nested node 18, evaluates as: 1) a success if all its children 12-15 evaluate to success; 2) failure if at least one child node evaluates to failure; and 3) running if at least one node evaluates to running while none of the nodes evaluate to failure. A middle node 16 can evaluate to the following: 1) success if at least one child evaluates to success; 2) failure if all children evaluate to failure; and 3) running if at least one child evaluates to running and none evaluate to success. A benefit of behavior trees is that an original problem can be hierarchically decomposed by plugging in a behavior tree focused on a subproblem into another tree focused on a higher-level problem without needing to know details of the subproblem. Behavior trees can also incorporate planning and diagnosis. For instance, automated diagnosis can act as another node in the tree, and behaviors such as “if there is a single minimal diagnosis and the diagnosis rules are owned by a single service, restart the service” can be encoded into the tree as well. When a planner produces a plan of actions from the current state, these actions form a behavior tree where each action is represented by a single node. The node’s success, failure, or running evaluation directly reflects the success, failure, or current execution of the action it represents. For example, when diagnosis can fail, the whole behavior tree fails. Within a framework a behavior tree can be seen as a deterministic policy that observes the world with each tick of the tree and can send actions, such as “plan.” Planning can be a subproblem implemented as a policy through a solver that takes state, actions, and a goal to develop a plan for execution. As described in detail below, the graphs can be used to organize components for a composite AI system. Specifically, the composite AI system can include an automated assembly of data connectors, domain and goal-based problem models, computational services, such as solvers, reasoners and large-language models, and appropriately configured control nodes using a graph-based organization into a service supported by an externally interfacing API that can be plugged into a target system for automation. A composition framework for the composite AI system includes components for making compositions. FIGURE 22 is a graph diagram showing, by way of example, a composition framework 630. Online and / or offline data is input and adapted for distribution to one or 2466.PC.UTL.ap1 - 37 - more models organized into one or more stages. Novel control nodes 634, 636 can be flexibly configured to distribute data into and out of models 635, 640, 642, 644 in to, between, or out of stages. For instance, the output of control node 36 can feed into another stage of models defined by control node 634-1 with models 4x-1 and output control node 636-1, as well as other defined stages. If the data distribution has reached the last stage 637, online and offline data connectors 38 are utilized to provide composition output 639. Some models 640, 642 require computational pipelining 641, 643 to execute. For example, a planning model needs to be run through a solver with a problem specification. A Generative Pre-trained Transformer (GPT) model needs to have its weights loaded into the configured transformer architecture and be run with a problem prompt. At the end of all stages, the composition produces output, which is a solution for the problem or task that is being solved. The composite core is wrapped in a service API and designed for deployment in a container or set of containers to provide the configured service both in testing and productions environments. When building real-time intelligent services, both offline 33 and online data 31 can be processed. Offline data, which is typically used for model learning, can be large and is not significantly constrained by processing time. In contrast, online data is expected to arrive in real-time when the service is deployed and the results of processing the real-time data is also going to be produced and published by the service. Unlike conventional data connectors, the connectors 32 used here have extended functionality by adding preprocessing data transformations relevant to learning or deploying the models. The models are generally the core workers in a composite AI system and can be pretrained with data and augmented with new data. The models can also learn from zero using the new data or be algorithmic in nature. When the models need to be augmented or developed from the ground up, model specific techniques can be applied, such as automated domain driven model synthesis. By applying various filters and transformations to a dataset ingested from a data connector, the composite AI system can define the state space or other features, which are used to learn the model. The same dataset can be used to define state spaces or inputs for multiple different modeling approaches. For example, a predictive task can be modeled as a dependency of the engine temperature on speed, road incline, and total weight, while simultaneously modeling vehicle logistics using classical planning. Complex composite AI systems can be created by providing a control mechanism to combine, select, transform, deconstruct, and configure how the data flows into and out of 2466.PC.UTL.ap1 - 38 - the models in each stage until final output. Each control node is configured to marshal the data into or out of a set of models and configure the set of models as required using the following configuration elements: 1) permeability, which defines how data flows through the gate; 2) combination, which defines how inputs are combined before being passed to one or more models downstream; 3) selection, which defines how selected inputs are passed to one or more models downstream if at all; 4) transformation, which defines how inputs are altered before being passed to one or more models downstream; 5) downstream model configuration, which defines specific model configurations for each model as needed for input; and 6) orchestration, which defines conditions of received outputs and when or how to proceed in processing. Orchestration may only require a few responses instead of all responses from a set of upstream models before proceeding and may require specific properties to the responses before proceeding. With respect to downstream model configuration, each input into a model, for example, an LLM may specify some specific hyperparameters associated for processing the input. Also, there are different levels of permeability, including fully permeable, partially permeable, and impermeable. Fully permeable occurs when data is a complete pass-through to one or more models downstream and partially permeable occurs when data is a partial pass-through to one or more models downstream and can be combined, selected or transformed before being passed to the models. Finally, impermeable occurs when data is not passed through and will be combined, selected, or transformed before being passed to one or more models downstream, if at all. FIGURE 23 is a block diagram showing, by way of example, a control node architecture 650. Data 656 comes into a data combination, selection, and transformation logic component 652 of a control node 651. The control node 651 also includes a model configuration and orchestration logic component 653, a data to model component 654, and a model control component 655. The data combination, selection, and transformation logic component 652 applies logic rules to the incoming data to combine, select and transform the data. A model configuration and orchestration logic component 653 receives information about the models available for use and is configured through logic rules on how to set up, execute and orchestrate downstream models. The data combination, selection and transformation logic component 652 and the model configuration and orchestration logic component can share information that informs logic rules for processing. The data to model component 654 provides the data to the appropriate downstream components, which may be the system output. The model control component 655 configures the downstream models 2466.PC.UTL.ap1 - 39 - and kicks off their operation as defined by the logic rules. The control nodes 651 always have incoming data and some outgoing data, but may not have any model input or control. Any logical specification system can be used to implement a control node, such as, but not limited to, programming languages, logic gate circuitry, biological neural networks, photonic circuits, qubits, fluidics, analog circuits, DNA strands, and spiking neural networks. One example of how control nodes are used in a stage is to reduce LLM hallucinations, generated incorrect or fabricated information, by passing the same prompt into each of a set of LLM models, such as ChatGPT, Llama, and Gemini, in parallel. The outputs are then transformed into an intersecting set so that only outputs that are in fuzzy match are passed to the next stage. Information that is not the same in each response is filtered to reduce inconsistent aspects, which are often manifested as LLM hallucinations. Control nodes can output to other control nodes and be nested in model flows. Control nodes fit neatly into the system graph as nodes before and after models and model processing nodes. Most times system control is best defined as a DAG or BT, but in some embodiments just a directed graph may be used with cycles. These cycles typically specify a loop between control nodes so that information may be processed multiple times by the same set of models. FIGURE 24 is a graph diagram showing, by way of example, a directed acyclic graph 70 of the composition framework of FIGURE 22. Data is input 672 and received by a stage 1 control node 674, which corresponds with the control node 634 of FIGURE 22. The data 672 flows via the control node 674 to model 1675, which corresponds with the model 1635 of FIGURE 22, to model… + computation service 680, which corresponds to model…640 and computation service 641 of FIGURE 22, and to stage 1a control node 686, which corresponds to control node 646 of FIGURE 22. The data of model 1675 and model… + comp computation service 680 flows to stage 1 control node 676, which corresponds to control node 636. Data from the stage 1a control node 686 flows to a model n+comp service 682, which corresponds to model n 642 and computation service 643 of FIGURE 22, and to model n+1684, which corresponds to model n+1644 of FIGURE 22. Output of the models, model n+comp service 682 and model n+1684 flows to stage 1a control node 685, which corresponds to control node 645 of FIGURE 22. Subsequently, the data output from the stage 1a control node 685 flows to stage 1 control node. The process can be repeated if there are additional stages 681 to go through. However, if the last stage is reached, composition output 689 is produced. 2466.PC.UTL.ap1 - 40 - In a further embodiment, the composition framework can be displayed as a behavior tree. FIGURE 25 is a graph diagram showing, by way of example, a behavior tree 690 for the composition framework of FIGURE 22. Each node represented by a right facing arrow, such as the root node 691 and child node 703, is represented as a logical “AND” and has the properties of: 1) success if all its children evaluate to success; 2) failure if at least one node evaluates to failure; and 3) running if at least one node evaluates to running while no nodes evaluate to failure. Flowing downwards from the root node 91 are: 1) composition input 692, which corresponds to 632 of FIGURE 22; 2) stage 1 control node 693,which corresponds to control node 634 of FIGURE 22; 3) stage 1 control node 694, which corresponds to control node 636 of FIGURE 22: 4) stage n control node 695,which corresponds to the next stage ending with the equivalent of control node 636-n, where n is the stage; and 5) composition output 696, which corresponds to output 639 in FIGURE 22. A further node 703 also flows from the root node 691 and is associated with the following children:1) model 1697, which corresponds to model 1635 of FIGURE 22; 2) model--- +comp service 698, which corresponds to model 640 and computation service 641 of FIGURE 22; 3) stage 1a control node 699, which corresponds to control node 646 of FIGURE 22; 4) Stage 1a control node 700, which corresponds to control node 645 of FIGURE 22; and 5) a node identified with a question mark 704, which represents a logical “OR” and evaluates to: 1) success if at least one child evaluates to success; 2) failures if all children evaluate to failures; and 3) running if at least one child evaluates to running and no nodes evaluate to success. The node 704 has children, including model n+comp service 701 corresponding to 642 combined with 643 in FIGURE 22 and model n+1702 corresponding to645 in FIGURE 22. Compositions can be designed to operate under various modalities. A common set is train and test, which includes learning and performing, exploring and exploiting. Control nodes can be specified with different logic configurations to support each desired mode of composition usage. In most embodiments, the mode is specified through a global attribute read by each control node as a function parameter, configuration file, environmental variable, command line argument, user input, class attribute, decorator pattern, or dependency injection. During control node specification, each modality logic is clearly defined. Architectural Solution Composition Architectural Solution Composition involves building a graph, such as a behavior tree in this example, that combines multiple models with corresponding solvers, goals, and 2466.PC.UTL.ap1 - 41 - APIs into a service. Although the models share a single observation space, each model only considers its own local state space. Each model can be tied to computation service, such as a solver, which may continuously work on problem instances generated from the observations whenever triggered, such as by reaching the behavior tree associated with the model. For example, data flow through the graph initiates triggers, including performing work or evaluation when the data arrives. The system can use any type of software defined computation services, including planning solvers in the Unified Planning Framework and the wide range of AI techniques. The software defined computation services can also include transformers, classical planner, hierarchical planner, constraint satisfaction solvers, linear programming solvers, Boolean satisfiability solvers, satisfiability modulo theories solvers, and reinforcement learning. Reasoning and action execution are encapsulated into the nodes of a behavior tree providing the grounds to define a composed automated reasoning, planning, and execution system in terms of tree composition. The tree composition is an operationalization of multiple models together with the corresponding solvers, actors, goals, and APIs into a service. Any two or more models in a composition are related either by: 1) nesting (hierarchical), when one model is fully nested within another, and the subsystem operating with the outer model can invoke the subsystem operating the inner model; 2) precedence or chaining (sequential), when a subsystem for the earlier model can be fully executed before running the subsystem for the later model; or 3) concurrent (interleaving and / or parallel), when subsystems for both models can run concurrently and interact with each other, for example by exchanging their best solutions. These three model composition approaches are available to the solution architect through the control nodes and the architect can design a composition appropriate for the given real-world problem. While the system offers blueprint-compositions for different industries and use-cases within those industries, such as example solutions and templates, there are variations that fundamentally change the composition, such as having a customer who operates distribution centers both in cities and rural areas. The user may want to use a composition where a bin-packing problem precedes vehicle routing in the cities and vice versa in the rural areas, where the vehicles are rarely full. The models share a single observation space, the control nodes picking only the data relevant for them, such as vehicle routing with time windows precedes running a planner in 2466.PC.UTL.ap1 - 42 - a food delivery system example. First, optimal assignments of orders to drivers are produced, which is equivalent to adding control knowledge to the more general planning problem, which is solved consequently. We consider each model to be tied with a computational service subsystem as appropriate, such as plan with a planner, problem with a solver which keeps solving the problem instances generated from the observations every time it is triggered. The concept of model composition in the system is a tool given to the solution architect user. If desired, the user can build and compose models to guarantee optimal solutions. There can also be cases when optimality is infeasible or not desired, and the goal is to find solutions quickly with a reasonable distance from optimality, which are also supported. Service Code Generation The service code is generated by filling out template code of various components of the composition or by configuring the pre-build services, such as by using JSON configuration files, environment variables, or API calls. A code or service template exists for every type of composition node. The solution is composed of a main orchestration service that has knowledge about all the components in the composition. The individual nodes are directly part of the orchestration service, in the case of simple nodes, or the orchestration service has lightweight proxy wrappers for a dedicated microservice that are logically part of the composition. The orchestration service communicates with the microservice using the proxy and can directly fetch and push data to it via the proxy or can use it to configure the wrapped service to specify data inputs and outputs. Individual nodes are decoupled, language independent, and are flexible to be part of a wide range of compositions. Programming language of the nodes ranges from C++, such as gaming code instrumentation and integration, through Python, including machine learning and symbolic AI solutions, to scripting language, such as external API calls. The list of programming languages is non-exhaustive as the composition is agnostic to the language used by a specific component. Automatic code generation also creates API specs and documentation for the unique set up described for the composition and exposes it via an API so the customer can easily incorporate it to their own tools that are not yet supported by the connectors. All the parts of the solution templates, including the orchestration service and all prebuilt packages are packed into container technology by design. After code generation, all that is needed is to execute the already created build commands to create the containers 2466.PC.UTL.ap1 - 43 - and service manifest. These containers can be easily versioned, duplicated, and deployed locally or to a cloud solution. The containers can be used in various environments, such as testing, staging, demo, or production. Containerization allows exact specification of an environment where a service will run and serves as prevention from issues related to hardware or operating system differences. Depending on the composition complexity and requirements, the containerized services are deployed as a Docker Compose or Kubernetes deployment. Both Docker Compose and Kubernetes tools allow for direct interpretation of a composition in a versionable, replicable, re-deployable, and easily maintainable manifest file. This approach also supports making incremental, non-destructive changes to a composition, meaning if a user changes the composition in some way, only the parts that change will be redeployed and the rest will stay the same. Kubernetes deployment also allows scalability that automatically replicates a service if it is reaching resource limits or destroy replicas if they are underutilized to save computational costs. Deployment Deploying services produced by the system is a push-button action for the user, which involves a cloud agnostic or computational platform deployment of containerized microservices at the backend. Once deployed, users can monitor the deployment status and interact with it at the defined network address through the API defined in the composition. Final deployment also includes implicit and explicit tools and services not directly part of the data pipeline. These services allow incorporation of functional and non- functional requirements to the solution, such as High Availability, extensive monitoring and logging capacities, authorization, and authentication mechanisms. Tools, such as monitoring tools, are implicitly part of every deployment. An example of explicit supporting tools is the High Availability Service that will deploy the composition multiple times in multiple regions to serve as a failback if one instance fails, to allow faster and more scalable usage for customers from different regions, or for many customers in one place. The deployed services can be used as dedicated or distributed. Dedicated services are for a composition to which they belong, whereas distributed services are applied across multiple service environments, customer solutions, or even multiple customers. In further detail, dedicated services can be models dependent on the environment where they are used or if the customer wants dedicated services running. Distributed services can be shared data sources that are the same for all customers, for example, a weather service or resource-heavy service that is cost prohibitive to use for a single 2466.PC.UTL.ap1 - 44 - deployment, such as an LLM service. Customer isolation is guaranteed by the solution by using dedicated services or by not sharing context in the distributed services. Composition is a core part of a complete system. FIGURE 26 is a block diagram showing, by way of example, a composite AI system 710. A front end 711 of the system 710 includes a model editor 713, a composition editor 718, a simulation testing manager 719, and a deployment manager 720. The model editor 713 allows an expert user to define data inputs and how the inputs are transformed into an ingestible format, such as tabular data for predictors or machine learning models 716, or time series for symbolic models 717. Connector editors 714 or transformer editors 715 are then plugged into a machine learning or symbolic modeling approach. The expert user interactively adjusts and augments the proposed model until satisfied with the quality. The satisfaction may be based on accuracy for machine learning or a human check and test validation against input data for symbolic models. Models can be used in compositions and can have properties, such as defined input specifications, defined output specifications, defined model configuration parameters, defined model controls, defined computation and memory requirements, and characterization of performance on data. The composition editor 718 takes multiple models and offers diverse options to combine them using control nodes to implement a graph that can be executed as a DAG, BT, or other forms. Combination of the models can be performed by identifying and combining the best fitting models for each part or subproblem of the input problem or task to be performed. Further, model efficiency can be based on speed, CPU, memory, precision, and recall. Also, data connectors can be used to provide shared observation space. The composition editor 718 encodes both the decomposition of the mathematical structure representing the real-world problem and the operationalization of the model with the behavior of a general agent. This feature allows for quick iteration, learning, and versioning of agents that provide intelligent automation services. The editor also specifies input and output of the composition. The building blocks of a composition are explicitly versioned to support breaking changes. For example, if one service changes their API and another dependent service needs to adjust, the system can keep one composition that has services that have not adjusted on an old version and another composition that has the changes incorporated into a newer version. This supports failing back to older versions if needed, rolling out features gradually, and other cases where explicit versioning supports better system management. The building blocks can be different AI techniques, including a planner, solver, LLM, or 2466.PC.UTL.ap1 - 45 - classifier. Generally, the building blocks can include any type of symbolic or sub-symbolic AI component. The building blocks are used in embodiments to: 1) construct the behavior tree or other executable graph in Python or another OOP language based on customer needs, such as C++ for game testing integrations, and 2) list the deployment requirement of the components necessary to run the composition. The BT represented by the composition in an embodiment has shown to be a low latency (1-5 ms), minimal-computation tree structure that constantly runs when deployed and manages the other components or containers considered to be serviced in this interaction. The simulation testing manager 719 deploys compositions by mapping them into a simulation representative of the target system. Control of the simulation can run the composition through testing phases with the simulated system under any type of definable conditions. Simulations are optional and can test if the system works to precisely mimic the deployment environment to ensure there are no issues before connecting to a real world system. Outcomes of the simulation can include successful, which means the composition can be applied to the real world system or unsuccessful, which can trigger further processing and revision of the composition. The deployment manager 720 deploys compositions by mapping them 1:n into running instances n, each with its own configuration and connected to a target system. For example, a location, such as a DNS name or IP number, and an authentication token are assigned as minimal configurations. The deployment manager continually monitors, terminates, and provides other features expected from cloud-deployed services with minimal downtime. Compositions cannot tell the difference between the simulated and target systems when operating. Target system operators that also own reasonable fidelity simulators can plug their simulator in for testing or a generic simulator configured to simulate / emulate the target system can be used. Simulation is not required as some complex deployment systems do not have associated simulators, but if available should be used as part of a mature testing process prior to real system deployment. Support back-end infrastructure components support the front-end configuration systems. Model editing is supported by existing libraries for data extraction, transformation, and loading 721; machine learning (ML) is supported by an AutoML Frameworks 722 that makes it easier to create ML solutions through automating / assisting the process of selecting, training, and tuning machine learning models, making it easier and faster to build effective 2466.PC.UTL.ap1 - 46 - predictive models without extensive manual intervention; and symbolic reasoning is supported by an Auto Synthesis 723 process. Composition editing is supported by a recommendation system 724 that suggests local compositional structures based on learned prior examples and testing 725 that support running data through the composition, providing inspection details for each component, and providing visibility across the composition components and connections to support testing and evaluation prior to testing in a simulated or actual environment. This is an important part of being able to debug a composite AI system. Simulation Testing and Deployment management are supported by a complex system 730 that provides Deployment and Continuous Deployment 730 (updates from changes) to run the Composite AI system in a simulated (through an adapter 729) or real environment (through an adapter 728), monitor 731 all instrumented and inspectable operations, and perform multivariate testing 727 by adjusting the available parameters of the system. Iteration and Inspection Users can access an interactive development interface. FIGURE 27 is a screenshot showing, by way of example, an interface 330 for composition development. The interface 330 can support language development in parallel with a visual programming interface for creating a composition. The interface can also support Planning Domain Definition Language (PDDL) for editing planning models and a custom composition language used to describe a behavior tree graph with the Visual Programming Interface provided by Google Blockly. Iterative refinement through local testing and target deployment is supported by an inspection tool into each graph node and their edges. A left side of the interface 330 can show a title 331 for editing a graphical programming language version of the model 332. A language version, for example PDDL, of the model 333 can be provided on a right side of the interface 330. A solution architect can assemble one or more models into a composition. FIGURE 28 is a flow diagram showing, by way of example, a process 340 for creating and editing a composition. Static data and / or streaming data are input through a configuration of data connectors. A solution architect 342 can create a composition 341 by combining different models through chaining (sequential), concurrent (parallel or interleaved), and / or nested (hierarchical) connections through a control node. Once a model is placed in a composition, additional computational services, such as a solver configuration, can be provided depending on the model type. The additional computational services can be provided when the performance of the solver for a given model is not sufficient by observing the type of 2466.PC.UTL.ap1 - 47 - component and whether or not it needs additional computational support. Planning models can go into a planner, language prompts and information can go into an LLM, and a problem can go into a solver. The composition can be edited via an interactive visual editor, where it can be saved and revisited. During editing, the solution architect can identify one or more models for adding to the composition or removing from the composition. Once the composition is complete, it can be published to prepare for compilation and deployment as an automation service. Once generated, the composition can be deployed. FIGURE 29 is a flow diagram showing, by way of example, a process 350 for running a composition. A configured model composition can be transformed into a service, deployed, and run. Specifically, service code 352 from a configured composition 351 of models can be generated, along with telemetry instrumentation, such as distributed tracing, and API code and specification based on the contained models to output the composition with models as code 353. The composition is compiled into a set of container images 355 using Docker or Kubernetes for image generation 354. A deployment definition, including any optionally provided parameters is generated. The resulting artifacts, OS and file system snapshots with metadata 357 defining applications and their dependencies, to the platform’s compute infrastructure for user workloads is automatically deployed 356. The user is updated about the state of the deployed service(s) defined by the composition along with service endpoint location and API specification. Each service implements one or more parts of a composition and are connected through a communication mechanism (e.g., message passing). The deployment can operate in control of a target system or a simulated system for exhaustive testing. Target System Application An approach for building a general hierarchical autonomous control architecture using a composition is provided. Specifically, the approach combined well-known techniques from satisfiability theory, automated planning, automated diagnosis, constraint programming, reinforcement learning, and behavior trees. FIGURE 30 is a block diagram showing, by way of example, a high level target system architecture 360. A user 361 provides a target system 362 that accepts actions and provides a perception of it and its world’s state. The user works with the backend system, which includes a deployed composition runtime 367, via a user interface 364 for testing and deployment. The user can configure one or more domain models and a composite system of AI-based components organized by a graph, such as a behavior tree, running through tests and deployment to solve 2466.PC.UTL.ap1 - 48 - problems through action-space control via a user interface, which accessible through an internetwork, such as the Internet or a cellular network. As a running example, components set up for solving the Towers of Hanoi game as the target system is described. The Towers of Hanoi is a puzzle game that involves moving discs from one place or rod, to another. In one example, there are n rods, where n is typically three, and a stack of different sized discs. The goal of the puzzle is to move all the discs from one rod to another, following a few rules. The rules include only moving one disc at a time and never putting a bigger disc on top of a smaller disc. Also, the other rods can be used to help move the discs around, but each disc can only be placed on top of another disc or on an empty rod. In the deployed composition, the symbolically trained Towers of Hanoi Model captures the model in PDDL and includes definitions of the target systems’ predicates, actions, and goals. The model is paired with a planner in a layer book-ended with pass- through control nodes. A predicate is a function of a set of parameters that returns a Boolean as an answer. The model can be learned by taking raw logs from a system, an expert’s domain knowledge, existing domain knowledge, such as existing PDDL descriptions, and other sources that provide domain knowledge through capture collecting into a knowledge base. Knowledge capture primarily involves preparing a set of predicates that describe the domain under interest following any method of knowledge capture from a consultant or domain expert. The knowledge base may also include a description of predicates and their meaning extracted from the target system’s raw logs. Predicates apply to a specific type of object or to all objects. Predicates are either true or false at any point in a plan and when not declared, they are assumed to be false. In the Towers of Hanoi example, the knowledge base can include information, such as the definition of useful predicates: “disc_on_disc(disc1, disc2)” means disc1 is on disc2, “smaller(disc1, disc2)” means disc1 is smaller than disc2, “disc_clear(disc1)” means disc1 is on the top and nothing lies on it. It may also define the goal to be achieved, such as “all discs lie on the last rod,” composed of predicates that are all true, such as disc_on_rod(disc1, rod3), disc_on_rod(disc2, rod3), and disc_on_rod(disc3, rod3). The initial state of the problem can also be defined using predicates that describe the input scenario and all combinations of disc and their size and location. Information from the target system flows into and out of the deployed composition through the data connectors. This takes perception data from the external world state of the 2466.PC.UTL.ap1 - 49 - system or other systems that may accurately report data from the environment of the target system. The perception data can include any external information relevant to the domain of interest, such as weather, traffic conditions, and general knowledge from the outer world. The target system can provide telemetry operational logs, such as a raw stream / batch of data collected from system components. Returning to the Towers of Hanoi example, the operational logs can include keys pressed in the game by a human player. The composition output data connector transmits action information to the system. The action information can include transformed planned actions that lead to actual system change. In the example, the planned action can include “MoveDiscFromRodToRod,” which once translated to the target system can mean “press arrow up, press arrow left, press arrow left, press arrow left, press arrow down” as a sequence of virtual keyboard key presses. A data connector takes in data in some form, digests the data from the source, such as a document, Google Drive, SFTPL, stream, websocket, Parquet, or Amazon S3, and passes the data in a defined structured data format to be processed by other entities. In the puzzle example, Keboola or some other data platform can be used to interchange data from the user’s game (websocket, batches in some storage) into a format defined by the predicate store. This configuration can include format specifications, connectors, logins, or secrets. A data connector takes in predicate definition raw data, and transforms it given the predicate definition creating predicates from the input log stream. For example, for the Towers of Hanoi puzzle, “Rod1+[disc3, disc2]” is converted to “disc_on_rod(disc3, rod1), disc_on_rod(disc2, rod1), disc_on_disc(disc2, disc3), smaller(disc2, disc3), disc_clear(disc2). Within the deployed composition, the data connector passed input into a control node, which configures the trained Towers of Hanoi model with a computational service planner and passes in the input without editing. The model nodes execute and provide action output. A receiving control node accepts the output and without editing passes it on to the output data connector. Because the model was designed to take in raw system data and output, and provide appropriate action output, no transformations were necessary by the control nodes, but the capability is available. A data connector takes information from the composition output and converts it to an action description with parameters, which describes actions to be executed in the target system to achieve the desired goal. In the running example, “move(d1, d2, d3)” means to move disc1 from disc2 to disc3. The output data connector interfaces with the target 2466.PC.UTL.ap1 - 50 - platform to execute all action commands on that platform, thus affecting the state through action. The testing and deployment user interface is the front-end interactive tool that works with the user. The interface includes four main parts, which are described above in detail with respect to FIGURE 6. Model development, by the model editor, uses a text-based language augmented by a visual programming language. In one embodiment, the modeling language is PDDL text and Blockly provides a visual builder. The user can iteratively refine the model through testing and deployment in a digital simulation of the target system and / or the target system itself. Model debugging tools provide insight and feedback for iterative refinement. The models can be completely synthesized from raw observational data using a method, such as automated domain-driven model synthesis. The interface tools can then be used to manipulate, edit, add, and remove actions within the synthesized model. The goal is to obtain a model having information from domain synthesis on which action to use or not to use for control with the target system. User augmentation can be used to refine the model and improve understandability by renaming predicates, goals, and actions, as needed. Composition development is also included in the front end and uses a text-based language augmented by a visual programming language. In one embodiment, the composition language is text describing a behavior tree and Blockly provides a visual builder. A behavior tree visualizer also provides additional insights. The user can iteratively refine the composition and models through testing and deployment in a digital simulation of the target system. Composition debugging tools provide insight and feedback for iterative refinement. The goal given one or more models, data connectors and control nodes, is to structure the composition workflow in a way that all components cooperate and achieve the desired system performance. In the Towers of Hanoi example, the composition invokes interfacing with the game to obtain the puzzle’s state, such as which discs are on which rod, and deploys a planner to use the PDDL model to determine the action commands in a sequence that solves the provided puzzle. The composition then executes the commands on the target game. Subsequently, simulation testing provides pre-deployment feedback on the composition and underlying models in action. Finally, deployment provides the ability to push control by the composition and models to the target system. Deployment includes live performance monitoring of the 2466.PC.UTL.ap1 - 51 - system in action and supports defining and observing performance through evaluation metrics. The deployment can target any type of cloud and container-based system. As part of deployment, deployed model runtime takes the deployment setup, description of container images, image registry, relevant services, external endpoints, API’s, data locations, and the input stream of target system predicates to deploy the model on a cloud-based platform. Deployed model runtime can run on any compute platform and can run services to allow communication according to description given by composition and ingest input data, such as preprocessed predicates, and output data, such as actions that lead to the goal. The system output is a stream of actions to be executed on the target system and the target system can make the actions happen in the real world given action input. In the puzzle example, the system could be a low-level Azure cloud deployment configurations of individual composition components, such as a planner, data store or serving APIs, networking and interoperability settings, VPNs, container registry settings and endpoints definition, needed to startup and execute a composed AI system to play the Towers of Hanoi. While the invention has been particularly shown and described as referenced to the embodiments thereof, those skilled in the art will understand that the foregoing and other changes in form and detail may be made therein without departing from the spirit and scope of the invention. 2466.PC.UTL.ap1 - 52 -

Claims

CLAIMS:

1. A method (10) for automated data-driven domain model synthesis, comprising: receiving logs (11) of data from a target system for which at least one goal is to be achieved; determining from the received logs, a set of predicates (44) that describe a domain of the target system and represent an initial state and sequence state changes, wherein each predicate applies to one or more objects in the domain; identifying the actions that are high level plan steps regarding achieving the goal; transforming the identified actions that are high level plan steps to actions that are low level execution steps to be performed for achieving the goal; transforming the actions that are low-level execution steps to be performed for achieving the goal to actions that are high-level plan steps regarding achieving the goal; performing domain synthesis (13) to create a model that describes data from the logs by adding the predicates and actions needed to cover all the logs and achieve the at least one goal; determining whether the model covers all the logs; and revising the model by adding one or more missing predicates or actions when the model fails to cover all the logs.

2. A method (10) according to Claim 1, wherein each predicate resolves as true or false.

3. A method (10) according to Claim 1, wherein the logs of data are received via a data connector that converts the data to an action description with parameters that describe actions to be executed in the target system to solve the problem to be solved.

4. A method (10) according to Claim 1, further comprising: classifying each log as a type of log; and 2466.PC.UTL.ap1 - 53 -synthesizing the actions based on the type.

5. A method (10) according to Claim 1, further comprising: attempting to perform an interpretation of the logs with the model.

6. A method (10) according to Claim 5, further comprising: revising the model by adding new types, predicates, and actions when the interpretation of the logs with the model is unsuccessful.

7. A method (10) according to Claim 1, further comprising: receiving user feedback regarding one or more of the predicates, actions, and model via a user interface.

8. A method (10) according to Claim 1, wherein the model is based on an input model received that is blank.

9. A method (10) according to Claim 1, further comprising: identifying two or more of the actions to be merged; merging the actions; and in the model, replacing the two identified actions with the merged action.

10. A method (10) according to Claim 1, further comprising: performing debugging of the model; and determining whether a mistake exists in the model based on the debugging.

11. A method (10) according to Claim 1, comprising: generating at least one benchmark problem for a planning domain in a domain-specific language; establishing a difficulty parameter for complexity of the benchmark problem; creating objects and predicates as an initial state of the benchmark problem based on domain properties and the difficulty parameter; selecting actions randomly to transition through states to define a valid action sequence; and 2466.PC.UTL.ap1 - 54 -through repeated action applications, determining a goal state of the benchmark problem based on a final state achieved after applying the repeated action applications to the initial state.

12. A method (630) for deployment of composite artificial intelligence systems, comprising: obtaining (631) models each based on a technique of artificial intelligence and associated with a target function for which a goal is to be achieved; generating (718) a composition by combining two or more of the models, comprising: assembling components via a graph; and running input data through the assembly of components, wherein one of the components comprises two or more of the models; and obtaining output data based on the running of input data through the composition components; employing control nodes (651) to marshal the input data into or the output data out of the models and configuring the models and any associated computation services for the models; performing simulation testing (719) by mapping the composition into a simulation representative of the target system; deploying (720) the composition to the target system; and obtaining information regarding actions to achieve the goal of the target system based on the deployed composition, wherein the actions are executed by the target system to achieve the goal or to control systems to achieve the goal.

13. A method (630) according to Claim 1, wherein the graph comprises one of a directed acyclic graph, directed graph, or behavior tree.

14. A method (630) according to Claim 1, further comprising: revising the models prior to inclusion in the composition. 2466.PC.UTL.ap1 - 55 -15. A method (630) according to Claim 1, wherein the models are combined via chaining, parallel or interleaved configurations, or nested configurations.

16. A method (630) according to Claim 1, further comprising: receiving edits to the composition prior to deployment; and revising the composition based on the received edits.

17. A method (630) according to Claim 5, further comprising: running debugging tools on the composition to provide feedback for the revisions.

18. A method (630) according to Claim 1, further comprising: running information from the target system in and out of the deployed composition via data connectors.

19. A method (630) according to Claim 1, further comprising: transforming the information regarding the actions based on language specific to the target system.

20. A method (630) according to Claim 1, wherein the deployment is performed, comprising: generate service code for the composition; compile the composition into a set of container images; and generate a deployment definition for deploying. 2466.PC.UTL.ap1 - 56 -

Images