Method and apparatus for hierarchically planning execution of sequence of operations

Abstract

A method and apparatus for hierarchically planning execution of a sequence of operations. In order to hierarchically plan the execution of a sequence of target state operations (OPS1, OPS2) for implementing a technical system workflow and to check them by system simulation or hardware execution, whereby the permissible range of the target state before the start of the actual planning is precisely specified and is less likely to make mistakes, it is proposed to implement a target state specifying process (GSSP) for hierarchical planning and inspection.

Description

[0001] The present invention relates to a method for hierarchically planning the execution of operation sequences to achieve a target state of a technical system workflow and checking them through system simulation or system hardware execution according to preamble claim 1, and to a device for hierarchically planning the execution of operation sequences to achieve a target state of a technical system workflow and checking them through system simulation or system hardware execution according to preamble claim 7.

[0002] Technical systems, such as in particular robotic systems, contain components for flexible or even autonomous production, which depends on the automatic sequencing of production steps, known as orchestration. When a technical or robotic system includes, for example, a robot that performs manipulation, the task of orchestrating the automatic sequencing of production steps can be solved neither solely at the symbolic level nor solely at the sub-symbolic level, i.e., the sequential level.

[0003] For example, the continuous properties of these robotic manipulators, such as pose, torque, geometry, are important to ensure that the motion is feasible on the technical or robotic system. On the other hand, task sequences such as grasping or assembly change the configuration of the product and / or the technical or robotic system and can therefore only be effectively captured on the symbolic level for planning.

[0004] The more flexible the technical system workflow has to be, the more intertwined these two aspects become. Therefore, for sufficient flexibility, planning methods must consider both aspects in a combined manner, leading to complex planning tasks.

[0005] This complexity can be reduced through hierarchical planning, which is described in the following published patent documents EP3446185B1, W02018 / 121904A1, W02020 / 025103A1 and EP 3978202A1.

[0006] Thus, even for replanning and parallel processes due to planning deviations, hierarchical planning can compute such task sequences as if they were obtained by measurement. The hierarchy focuses on certain aspects, while ignoring others, and the hierarchy thus planned is used in the context of reducing complexity in exchange for feasibility with possible optimality. However, specifying tasks for any planner that can then be executed on real hardware is an error-prone process.

[0007] This is difficult, since one target object can be described at various levels of abstraction. Therefore, the user has to find the right level, since a too detailed model may impose too many constraints on the overall task, i.e., unnecessarily restrict the planner's degrees of freedom, so the execution may become suboptimal or even infeasible, and a too abstract model may lead to undesirable final configurations, such as the wrong location of the target product.

[0008] Furthermore, inexperienced users can easily define goals in a wrong way, which can lead to poor execution performance or the planner not being able to find a solution that matches the corresponding goal state. In both cases, the set of possible solutions is not necessarily smaller than desired, and user support may help to identify them. In the latter case, if the user has correctly specified the desired goal, then a solution will exist. If the goal would represent what the user wanted but was not correctly specified due to lack of support, then this may even be the case that a solution exists.

[0009] The general hierarchical planning approach is rather new and no interfaces exist so far that can be used by non-expert users.

[0010] A target in this context is an instance of a so-called concept. From a programming point of view, each concept can here correspond to a class in a suitable programming language such as Python or C++. A target is then an instance of that class.

[0011] The planner reaches a goal if an instance of this class is reached during planning that has attributes that match all the attributes specified in the goal. It should be noted in this context that a plan instance may have more attributes as targets, but never fewer attributes. It is possible to specify a tolerance when checking for matching of attributes, which is particularly important for floating point type attributes.

[0012] Currently, the goal is described by defining it manually in a text file. To do this, the user has to fill in the different properties of the desired instance at the desired level of abstraction. This approach is feasible at an expert level but lacks any intuition due to the lack of graphical support and is time-consuming.

[0013] In classic automation, engineers would manually program the entire sequence of an automated workflow. So while they didn’t have to specify the goals, they had to manually script the entire workflow.

[0014] Therefore, manually designed programs lack the flexibility of planning methods with respect to changes in setup, products and production orders.

[0015] The object of the invention is to propose a method and a device for hierarchically planning the execution of operation sequences to achieve target states of a technical system workflow and to check them by system simulation or hardware execution, whereby the permissible range of target states is precisely specified and less error-prone before starting the actual hierarchical planning.

[0016] This object is solved with respect to the method defined in the preamble of claim 1 by the features in the characterizing part of claim 1 .

[0017] Furthermore, the object is solved by a computer-implemented tool according to claim 6 for carrying out the method according to one of claims 1 to 5 .

[0018] This object is further solved by the features in the characterizing part of claim 7 with respect to the device defined in the preamble of claim 7 .

[0019] The main idea of ​​the invention according to claims 1 and 7 is to implement a target state specification process for hierarchical planning of the execution of sequences of operations that achieve target states of a technical system workflow and to check them by system simulation or hardware execution, wherein <1> For the workflow of the technical system, a workflow-related planning domain is given by deploying workflow-related facts, which is based on the workflow-related set of digital twins and planning operators (Note: operators are implementations of tasks that can be used for planning and / or execution, which will be defined in more detail further below.), <2> A goal state associated with a workflow comprising or consisting of a set of desired goal instances is specified by a goal state specification process given in successive steps based on a graphical user interface input, (i) a first step, wherein each time doing so, in particular by using a 3D editor, the user respectively selects a different initial goal state, wherein the selected different initial goal state is based on one of the graphical user interface inputs, (ii) a second step, wherein a finite number of "m" of initial goal states are generated depending on the selected initial goal state GSi ” is a list of different target state abstraction levels, where m GSi ∈N, and (iii) a third step, where each time this is done, the user selects the number “m GSi "The different target states in the abstract level have j GSi ∈{1 GSi ,…,m GSi} a different "j GSi" target state abstraction levels, wherein the different target state abstraction levels selected are based on one of the inputs of the graphical user interface, <3> defining planning tasks for a specified target state and according to a planning domain for a deployment of a planning operator selected based on facts and at least one of a workflow-related set for the planning operator, and <4> By means of a planning method <4a> according to a defined planning task, the execution of at least one selected planning operator is iteratively checked through a primary query loop in order to find a complete plan including a first sequence of operations for achieving a target state until a complete plan for achieving the target state is found or an error in the technical system workflow is reached, and <4b> when a complete plan is found in <4a>, the execution of the found complete plan for achieving the target state is iteratively checked through a secondary query loop until the target state is achieved.

[0020] Therefore, in more detail, according to the main query loop of the planning method in continuous loop iterations (1) at least one selected planning operator is executed, (2) main control commands are generated due to at least one executed selected planning operator, (3) the main control commands are applied accordingly to simulate the technical system workflow and (4) the system simulation is checked accordingly through the main measurement results of the measurements until a complete planning for achieving the target state is found or an error in reaching the technical system workflow is reached.

[0021] In further detail, according to a secondary query loop of a planning method, in which in successive loop iterations (1) a full planning is executed, (2) secondary control commands are generated as a result of the executed full planning, (3) the secondary control commands are applied accordingly to simulate or execute the technical system workflow in hardware, and (4) the system simulation or system hardware execution is checked accordingly through the measured secondary measurement results until a target state is reached.

[0022] In the event of an error in reaching the technical system workflow, the target state designation process for hierarchical planning of the execution of the sequence of operations for achieving the target state of the technical system workflow is preferably stopped or cancelled. However, there are other feasible measures, such as continuing the execution of the sequence of operations for hierarchical planning of the target state of the technical system workflow.

[0023] Manually specifying a goal or target state without the mentioned graphical user interface based target state specification process is cumbersome and requires a certain level of expertise. It often happens that the user does not specify the goal or target state correctly. By having visual feedback due to the graphical user interface based process, it is easy to handle and reduce the source of target specification errors.

[0024] Furthermore, running through the 3D editor in combination with the model from the planning domain simplifies the process and ensures that the user selects the goals or target states exactly at the desired level so that they can be processed. This ensures better planning performance, fewer iterations in specifying goals or target states, and less engineering overhead overall.

[0025] Thus, 3D editors are used to describe scenes for various tasks. Some can be used to specify goals for planning tasks such as motion planning. All of these planners are only at a single level of abstraction and therefore specify goals explicitly at that level of abstraction.

[0026] Advantageous further developments of the invention are given by the dependent claims.

[0027] According to claims 2 and 8, it is advantageous that after the third step, the workflow-related target state specified by the input of the target state specification process based on a graphical user interface is given by further consecutive steps, (iv) a fourth step, in which different feasible target states of the selected different target state abstraction levels are sampled and presented to the user, and (v) a fifth step, in which the user is asked whether all sampled, presented target states are feasible, and if the presented target states are feasible, both the selected different initial target states and the selected different target state abstraction levels are based on the input of the graphical user interface, otherwise if the presented target state is not feasible, then either ("Option I") the third step is performed for another selected different target state abstraction level, or ("Option II") the first step is performed for another selected different initial target state.

[0028] Therefore, according to claims 3 and 9, advantageously, one or more of the at least one selected planning operators can be selected several times to iteratively check executions in order to find a complete plan comprising the first sequence of operations. This is advantageous because in many applications, multiple executions of certain operators are required.

[0029] Furthermore, according to claims 4 and 10, the invention can be advantageously used when the technical system is a robotic system.

[0030] Finally, according to claim 5, it is advantageous if the technical system workflow is output via a data logger of the technical system.

[0031] This has the advantage that it allows the resulting workflow to be stored for offline use.

[0032] In the case of robotic systems and corresponding workflows, there can be discrete manufacturing with a robotic arm, where the control is, for example, torques, joint values ​​or reference trajectories.

[0033] In the case of technical systems and corresponding workflows, there can be, for example, the orchestration of continuous processes such as chemical, biological, etc. and the routing of liquid, gaseous product flows, where the controls are, for example, the temperature of a heater, the rotation speed of a mixer, the state of a valve, etc.

[0034] In the case of technical systems and corresponding workflows, there can be autonomous laboratory assistants, e.g. machine care, handling of materials, etc., e.g. in the pharmaceutical field, where control is a combination of the two above-mentioned use cases.

[0035] In the case of technical systems, there can be a flexible adaptation of the workflow depending on machine availability in the factory, processing times and / or current requirements, where the control is, for example, the start time of the machine, the identifier (ID) and the type of part or element to be processed.

[0036] In the case of robots or technical systems and corresponding workflows, there can be <agv>"Task allocation for a fleet of vehicles, including additional tasks such as anomaly detection, where the control is, for example, the location to be reached, the object identifier (ID) to be transported, the start time, the reference trajectory, etc.

[0037] In the context of the present invention with the terms used, the following explanations of well-known terms are given:

[0038] I. An operator is an instruction that maps a list of input instances to output instances. For example, this instruction can be implemented by programming a logical condition, evaluating a simulation, or reading sensor data. For hierarchical planning as a special case of master planning, operators can also be structured hierarchically - meaning that coarser operators use inputs and produce output instances that correspond to more abstract concepts than those in more detailed operators.

[0039] II. A concept is a structure that specifies a list of attributes, which are themselves concepts or primitive types. This corresponds to a class in object-oriented programming. Depending on the attributes selected for each concept, the software automatically calculates a hierarchy similar to an inheritance structure in programming.

[0040] III. An instance is a realization of such a concept. It specifies for each attribute a value of a given primitive type or an instance of a concept specified for that attribute. This recursive definition results in a tree structure filled with primitive type values ​​representing facts. This corresponds to an instance in object-oriented programming.

[0041] IV. Facts are instances that are assumed to be part of the current state. The facts are either specified directly by the user or imported from sensors or databases.

[0042] V. A goal is an instance of a desired target state that fully or partially describes the workflow of a technical system - this is a partially ordered set of operators. A representation of such a set is a precedence graph, where directed edges go from one node to another if one operator (represented by the first node) must complete successfully before another operator (represented by the second node) can start. The target state is specified by the user.

[0043] A planning task is a combination of facts that describe a starting configuration, a goal state that specifies what must be accomplished, and operators (eg, a hierarchy) that capture all the actions that can be accomplished.

[0044] VI. Planning methods (e.g., hierarchical) are software that searches for an at least partially ordered set of operators, called a plan or work plan, that transforms facts as an initial instance into a target state. The set of operators is a precedence graph that captures which operator must be successfully executed before some other operator can begin.

[0045] The planning method can be run in offline mode, where the plan is a dedicated output, which can then be used by the execution engine to control the actual hardware. But it can also be run in online mode, where the execution is the most detailed planning level, so that the hardware is controlled adaptively. This means that the planning method iteratively generates plans on all levels of the hierarchy, but the most detailed plan is already the execution level on the hardware. This means that a successful planning on the most detailed level corresponds to a successful execution of the task on the hardware.

[0046] VII. The execution engine executes the plan on hardware.

[0047] The context based target state specification process, which is an important part of the present invention, can be summarized as follows in principle:

[0048] Using the preferred 3D editor and based on the world model of the technical system and the corresponding workflow, a scenario is created, which is assumed as a given entity in the context. The scenario describes all physical properties and geometric relationships. During execution, it can be viewed as a digital twin of the technical system and the corresponding workflow.

[0049] The idea behind the target state specification process is now to use this existing 3D representation to enable intuitive target specification, as it allows to specify possible configurations of the target object.

[0050] The goal state specification process provides a conceptual structure with all available attributes, where the task in the goal state specification is now to choose the right choice of these attributes, i.e., by using sufficient combinations.

[0051] This will then lead to hierarchical planning and the data structures required for the planning task.

[0052] To select these properties, the user may select and deselect properties of objects created by the target state specification process and in particular by using a 3D editor.

[0053] Another aspect of the target state specification process is to display the set of instances that match the currently selected set of attributes. The fewer attributes selected, the more configurations that meet the target requirements.

[0054] These sets of objects, or sampled versions of them, are then computed for display to the user.

[0055] The 3D environment allows the user to easily check whether all displayed objects will be accepted as targets.

[0056] If the set is too large, the user needs to select additional desired properties.

[0057] If the set is too small, the user must deselect some of the currently required properties.

[0058] When a suitable set is displayed, the user can generate a target state description in a data format suitable for hierarchical planning and planning tasks.

[0059] Based on the specified target state, the planned execution of the operation sequences that realize the target state of the technical system workflow is calculated and checked by system simulation or system hardware execution, which leads to the abstracted target and sends the corresponding control inputs to the system simulation or system hardware.

[0060] Example scenario for target : The goal state is to have a specific product in a certain work area, such as a box with a lid on it. Therefore, the only thing that matters is that the lid is on top, no exact position or orientation is required. In the 3D World Editor, the user places the product on a table in the work area. The 3D Editor now has exact information about the pose of the object.

[0061] However, if the object must be on the table, the user can consider many more configurations than the currently displayed gestures.

[0062] The user can relax the actual values ​​in the x and y directions and stick only to the z value, otherwise the object will hoover above the table or get stuck in the table.

[0063] The visualization will then display multiple translated objects on the table, i.e. sampled versions of the full set.

[0064] However, the orientation of the object may also be free.

[0065] If the user now removes the property to rotate around the z-axis from the target description, the instances are now not only distributed over the entire table, but they can also be rotated.

[0066] This is again visualized in a 3D environment in a sampling manner.

[0067] The scenario outlined has the following program structure:

[0068] #Describe the planning domain of facts and operators.

[0069] -The selection of these elements together with the goal describes the planning task to be solved.

[0070] - Planning domains as components can be accessed via an additional "graphical user interface" <gui>” or a text editor.

[0071] -The planning domain is also part of hierarchical planning.

[0072] #The planning method takes a planning task from the planning domain component and tries to find a plan that transforms the initial instance (fact) into the desired instance (goal).

[0073] -Planning methods are also part of hierarchical planning.

[0074] -If a complete plan is found, it can be forwarded to the execution engine.

[0075] #"Graphical User Interface <gui>In addition to the two elements, a 3D world model editor for modeling the target scene and a target selection section are combined as described above, wherein the targets are displayed at different abstraction levels via three additional elements, namely, a visualization of the planned hierarchy, a selection menu for the operator, and a visualization, for example, a 3D visualization.

[0076] -Automatically specify goals for hierarchical planning by selecting the desired level of abstraction for the target state according to the target state specification process.

[0077] - By selecting the planning steps and some operators of interest, a step-by-step invocation of the operators can be implemented and executed via the execution engine.

[0078] -The execution engine is also part of the scenario.

[0079] -The execution engine triggers the invocation of the operator.

[0080] - Depending on the level of abstraction and the implementation of the operator, the operator is executed in software or in hardware.

[0081] - The execution results, such as joint trajectories, are then sent to the "GUI <gui>"Visual component.

[0082] - Fixed task specifications and general setup are not directly part of the invention disclosed in this application, but it closes the loop and can be part of the planning domain with a GUI.

[0083] Furthermore, an additional advantageous development of the invention results from the Figure 1 and 2 The following description of preferred embodiments of the present invention. They show:

[0084] Figure 1 Equipment for hierarchical planning of the execution of sequences of operations that achieve target states of the workflow of a technical system and for checking them by system simulation or hardware execution,

[0085] Figure 2 In accordance with Figure 1 The functionality and working method of the functional units implemented in the device.

[0086] Figure 1 The target state GS for hierarchical planning of the workflow WF for the technical system TS is shown. WF The execution of a sequence of operations of a device DEV, the technical system TS being, for example, a robotic system, and wherein the achievement of a target state GS is checked by system simulation or hardware execution WF Execution of a sequence of operations.

[0087] For this purpose, the device DEV comprises a processor PRC, a graphical user interface GUI, a communication interface CI and a non-transitory processor-readable storage medium STM having processor-readable program instructions of a program module PGM for hierarchical planning and execution of a checking sequence of operations, which are stored in the non-transitory processor-readable storage medium STM.

[0088] When the two interfaces GUI, CI are connected to the processor PRC, the processor PRC executing the processor-readable program instructions of the program module PGM forms, on the one hand, a computer-implemented tool CIT for hierarchical planning of the target states GS of the workflow WF of the technical system TS using a non-transitory processor-readable storage medium STM comprising the processor-readable program instructions of the program module PGM. WF and then checking them by system simulation or hardware execution, and on the other hand forming a functional unit FTU for hierarchical planning of a target state GS of a workflow WF of a technical system TS by means of a graphical user interface GUI, a communication interface CI and a non-transitory processor-readable storage medium STM with processor-readable program instructions of a program module PGM. WF The execution of a sequence of operations is then checked through system simulation or hardware execution.

[0089] The computer implemented tool CIT is preferably a computer program product which is designed, for example, as application software, called APP, and which is loaded into the device DEV so that it can be executed by the processor PRC.

[0090] The functional unit FTU thus formed is designed in such a way that the target state GS of the technical system workflow WF considered WF The spf is specified according to the target state definition GSD and input into the processor PRC of the device DEV via the graphical user interface GUI, and the technical system workflow WF is based on the workflow-related planning domain PD WF , which is part of the processor PRC that executes processor-readable program instructions of a program module PGM for planning and checking the execution of a sequence of operations.

[0091] How do users use the target state GS for hierarchical planning of the workflow WF of the technical system TS? WF The execution of the sequence of operations and checking their execution by system simulation or hardware execution of the device DEV completes the target state definition GSD and how to specify the target state GS of the technical system workflow WF considered by the spf according to this target state definition GSD WF Will be described in Figure 2 The details are explained below.

[0092] Planning Domain PD WF Given by a set of operator SOPs, which include planning operator POPs 1,WF ,…,POP n,WF Workflow related collection. Planning operator POP 1,WF ,…,POP n,WF The workflow-related set is deployed dpl and input into the processor PRC of the device DEV via the graphical user interface GUI. This deployment is usually and preferably also used by the user for hierarchical planning of the target state GS of the workflow WF of the technical system TS WF The execution of the sequence of operations is accomplished by the device DEV which checks them through system simulation or hardware execution.

[0093] Moreover, the planning domain PD WF Given by a digital twin DT of the technical system TS, e.g. a 3D representation, which contains workflow-related facts FC WF . Workflow related facts FC WF Also deployed dpl and also input into the processor PRC of the device DEV via the graphical user interface GUI. This deployment is usually and preferably also completed by the user.

[0094] Part of the processor PRC which executes processor-readable program instructions of a program module PGM for hierarchical planning and checking the execution of a sequence of operations is also a planning task PT which is a task for a specified target state GS WF Based on the facts FC WF And for planning operators POP 1,WF ,…,POP n,WF A planning domain PD deployed by at least one selected planning operator in the workflow-related set WF Definition of dfn.

[0095] Furthermore, the part of the processor PRC that executes processor-readable program instructions of the program module PGM for hierarchical planning and checking the execution of a sequence of operations is a planning method PA according to the defined planning tasks PT and the execution engine ECE. Figure 2 Describe together the hierarchical planning to achieve the target state of the technical system workflow WF GS WF execution of sequences of operations and checking the functionality of these parts in their context through system simulation or hardware execution, Figure 2 Shown in accordance with Figure 1 The functionality and working method of the functional unit FTU implemented in the device DEV.

[0096] To define the dfn planning task PT, the target state GS WF - As mentioned above - the spf must be specified based on the target state definition GSD. For this purpose, the target state definition GSD is composed of Figure 2 The target state specifying process GSSP depicted in FIG. 1 is executed. According to the target state specifying process GSSP, the target state GS related to the workflow inputted via the graphical user interface GUI is WF spf is specified by a graphical user interface based input. Thus, the target state specification process GSSP is given by a series of steps, (i) a first step S1, in which the user preferably selects an initial target state GS by using a 3D editor i , where the selected initial target state GS i is one of the inputs based on the graphical user interface, (ii) a second step S2, in which, depending on the selected initial target state GS i Generate a finite number of GSi Different target state abstraction levels of GSAL1 GSi ,…,GSALm GSi A list of m GSi ∈N, and (iii) a third step S3, in which the user selects a quantity "m" from the generated list GSi Different target state abstraction levels of GSAL1 GSi ,…,GSALm GSi In the GSi ∈{1 GSi ,…,m GSi } GSi ” Target State Abstraction Level GSALj GSi , where the selected target state abstraction level GSALj GSi It is one of the input based on graphical user interface.

[0097] However, if the user does not ensure that the selected target state abstraction level GSALj GSi is feasible, then a preferred development of the embodiment given according to steps S1 to S3 is possible in order to specify the workflow-related target state GS by inputting based on a graphical user interface WF , the target state specification process GSSP is extensible. Therefore, after the third step S3, the extended target state specification process GSSP is given by further consecutive steps, (iv) a fourth step S4, in which for the selected different target state abstraction levels GSALj GSi The different feasible target states are sampled and presented to the user, and (v) a fifth step S5, in which the user is asked whether all the sampled and presented target states are feasible, and if the presented target states are feasible, the selected initial target state G is i and the selected target state abstraction level GSALj GSi Both are based on graphical user interface input.

[0098] However, if not all presented target states are feasible, then according to "Option 1", it returns from the fifth step S5 to the third step S3 and each time it does so, executes the third step S3 to select the number "m" respectively. GSi Different target state abstraction levels of GSAL1 GSi ,…,GSALm GSi In the GSi ∈{1 GSi ,…,m GSi another different (compared to the previously selected one or some of the previously selected) GSi ” Target State Abstraction Level GSALj GSi , where the different target state abstraction levels GSALj are chosen GSi It is one of the input based on graphical user interface.

[0099] However, if in the case that the presented target state is not feasible, instead, according to "Option II", it returns from the fifth step S5 to the first step S1 and each time it does so, respectively executes the first step S1 to select another different (compared to the previously selected one or some of the previously selected) initial target state G i , where the different initial target states GS are selected i It is one of the input based on graphical user interface.

[0100] Therefore, when planning operators POP accordingly 1,WF ,…,POP n,WF When a planning operator defines a planning task PT by selecting at least one of the workflow-related sets, the following steps occur in planning to achieve the target state GS of the technical system workflow WF WF The execution of a sequence of operations and the examination of their context through system simulation or hardware execution:

[0101] A. Iteratively check the execution of the selected planning operators to find the target state GS achieved through the main query loop PQL WF The complete plan PL including the first operation sequence OPS1 f The main query loop PQL runs through in successive loop iterations until the target state GS is achieved by finding WF The complete program PL f or reaching an error ERM of the technical system workflow (WF), (1) executing exc at least one selected planning operator by the execution engine ECE, (2) generating grt a main control command CC attributed to at least one executed selected planning operator by the execution engine ECE p , (3) correspondingly apply the main control command CC output by apl via the communication interface CI p to simulate the sml technical system workflow WF, and (4) by measuring the main measurement results MMR of MM p To check the chk system simulation accordingly, it will be fed back to the execution engine ECE.

[0102] In the event of an error in reaching the technical system workflow, the target state specifying process for hierarchically planning the execution of the sequence of operations for achieving the target state of the technical system workflow is stopped or canceled.

[0103] In the context of step A. It is also possible that one or more of the at least one selected planning operator may be selected several times to iteratively check the execution in order to find a complete plan PL comprising the first sequence of operations OPS1. f .

[0104] B. When in step A. Find the complete program PL f , check the target state GS iteratively through the secondary query loop SQL WF The complete program PL found f The secondary query loop SQL runs in successive loop iterations until the target state GS is achieved by WF :(1) Execute exc through the execution engine ECE to complete the plan PL f , (2) Generate the complete plan PL attributed to the execution of grt through the execution engine ECE f Secondary control command CC s , (3) correspondingly apply the secondary control command CC output by apl via the communication interface CI s To simulate SML or execute the EHW technology system workflow WF in hardware and (4) accordingly measure the secondary measurement results MMR of MM s Check chk system simulation or hardware execution, which is fed back to the execution engine ECE.

[0105] As an option, it is preferably possible to convert the secondary measurement result MMR s Feedback is given to the Graphical User Interface GUI for outputting them to the user for control purposes.

[0106] In the technical system TS there is a data logger DL which records the control commands CC p 、CC s and the measurement results MMRp, MMRs. Due to this recording of the reference data, the workflow WF of the technical system TS is characterized and can be output, for example, on a display for a user or for storage in a file.< / gui> < / gui> < / gui> < / agv>

Claims

1. A method for hierarchical planning of the execution of operation sequences (OPS1, OPS2) for achieving a target state of a technical system workflow and checking them by system simulation or system hardware execution, wherein a) for a workflow (WF) of a technical system (TS), by deploying (dpl) a digital twin (DT) of said technical system (TS), for example a 3D representation, based on a graphical user interface for inputting workflow-related facts (FC WF ), Planning Operator (POP 1,WF ,…,POP n,WF ) of the workflow-related set, giving the workflow-related planning domain (PD WF ), b) specifying (spf) workflow-related target states (GSSP) through graphical user interface-based input of a target state specification process (GSSP) given by sequential steps WF ), b1) A first step (S1) in which the user selects a different initial target state (GS) each time he does so, in particular by using a 3D editor i ), where the different initial target states (GS i ) is one of the graphical user interface based inputs, b2) The second step (S2), in which the initial target state (GS) is selected i ) generates a finite number of GSi "Different target state abstraction levels (GSAL1 GSi ,…,GSALm GSi ) list, where m GSi ∈N, and b3) A third step (S3), wherein each time this is done the user selects the number "m" from the generated list respectively GSi "Different target state abstraction levels (GSAL1 GSi ,…,GSALm GSi ) has j GSi ∈{1 GSi ,…,m GSi } a different GSi "Target State Abstraction Level (GSALj GSi ), where the different target state abstraction levels (GSALj GSi ) is one of the inputs based on the graphical user interface, c) for the specified target state (GS WF ) and based on the facts (FC WF ) and for planning operators (POP 1,WF ,…,POP n,WF ) of at least one selected planning operator's deployment planning domain (PD WF ) defines (dfn) a planning task (PT), and d) by a planning approach (PA) based on a defined planning task (PT) d1) iteratively checking the execution of the at least one selected planning operator in order to find a plan that achieves the target state (GS) by means of a primary query loop (PQL) WF ) includes the complete plan (PL f ), where in successive loop iterations, executing (exc) said at least one selected planning operator <ii>Generate (grt) a main control command (CC) attributed to at least one selected planning operator for execution p ), <iii>The main control command (CC) is applied accordingly. p ) to simulate (sml) the technical system workflow (WF) and <iv>The main outcome measure (MMR) p ) accordingly checks (chk) the system simulation until a state that achieves the target state (GS WF ) of the complete program (PL f ) or an error (ERR) reaching the workflow (WF) of said technical system, and< / iv> < / iii> < / ii> d2) When the complete plan (PL) is found in d1) f ), the target state (GS) is iteratively checked through the secondary query loop (SQL) WF ) of the found complete program (PL f ), where in successive loop iterations, the Execute (exc) the complete plan (PL f ), <ii>Generate (grt) the complete plan (PL) attributed to the execution f )'s secondary control command (CC s ), <iii>The secondary control command (CC s ) to simulate (sml) or execute (ehw) in hardware the technical system workflow (WF) and <iv>Secondary outcome measures (MMR) s ) accordingly checks (chk) the system simulation or system hardware execution until the target state (GS) is achieved WF ).< / iv> < / iii> < / ii> 2. The method according to claim 1, characterized in that After the third step (S3), the workflow-related target state (GSSP) specified by the graphical user interface-based input (spf) of the target state specification process (GSSP) is given by further consecutive steps. WF ), b4) A fourth step (S4) in which different target state abstraction levels (GSALj) are selected. GSi ) and present them to the user, and b5) A fifth step (S5), in which the user is asked whether all sampled presented target states are feasible, and if the presented target states are feasible, the different initial target states (G i ) and different target state abstraction levels (GSALj GSi ) are both inputs to the graphical user interface, or if the presented target state is not feasible, then either ("Option I") select a different target state abstraction level (GSAL) for another choice GSi ) execute the third step (S3), or ("Option II") for another selected different initial target state (G i ) executes the first step (S1).

3. The method according to claim 1 or 2, characterized in that One or more of the at least one selected planning operator may be selected several times for iteratively checking execution in order to find the complete plan (PL) comprising the first sequence of operations (OPS1). f ).

4. The method according to claim 1, 2 or 3, characterized in that The technical system (TS) is a robotic system.

5. The method according to any one of claims 1 to 4, characterized in that The technical system workflow (WF) is output via a data logger (DL) of the technical system (TS).

6. A computer-implemented tool (CIT), in particular a computer program product, designed, for example, as an application software, referred to as APP, for hierarchical planning of the execution of operation sequences (OPS1, OPS2) for achieving a target state of a technical system workflow and for checking them by system simulation or hardware execution, having - a non-transitory processor-readable storage medium (STM) having processor-readable program instructions of a program module (PGM) for planning and checking said execution of said sequence of operations (OPS1, OPS2), stored in said non-transitory processor-readable storage medium (STM), and - a processor (PRC), connected to the storage medium (STM), executing the processor-readable program instructions of the program module (PGM) to perform the method according to one of claims 1 to 5.

7. A device (DEV) for hierarchically planning the execution of operation sequences (OPS1, OPS2) for achieving a target state of a technical system workflow and checking them by system simulation or hardware execution, having a processor (PRC), a graphical user interface (GUI) and a communication interface (CI), each of which is connected to the processor (PRC) and a non-transitory processor-readable storage medium (STM), the non-transitory processor-readable storage medium (STM) having processor-readable program instructions of a program module (PGM) for planning and checking the execution of the operation sequences, which are stored in the non-transitory processor-readable storage medium (STM), wherein the processor (PRC) executing the processor-readable program instructions of the program module (PGM), the graphical user interface (GUI), the communication interface (CI) and the non-transitory processor-readable storage medium (STM) having the processor-readable program instructions of the program module (PGM) form a functional unit (FTU), which is designed in such a way that a) For a workflow (WF) of a technical system (TS), by deploying (dpl) a digital twin (DT) of the technical system (TS), e.g. a 3D representation of workflow-related facts (FC) WF ), each inputted via the graphical user interface (GUI) by a planning operator (POP) 1,WF ,…,POP n,WF ) gives the workflow-related planning domain (PD WF ), b) specifying (spf) a workflow-related target state (GSSP) input via the graphical user interface (GUI) by means of a graphical user interface-based input of a target state specification process (GSSP) given by successive steps; WF ), b1) A first step (S1) in which the user selects a different initial target state (GS) each time he does so, in particular by using a 3D editor i ), where the different initial target states (GS i ) is one of the graphical user interface based inputs, b2) The second step (S2), in which the initial target state (GS) is selected i ) generates a finite number of GSi "Different target state abstraction levels (GSAL1 GSi ,…,GSALm GSi ) list, where m GSi ∈N, and b3) A third step (S3), wherein each time this is done the user selects the number "m" from the generated list respectively GSi "Different target state abstraction levels (GSAL1 GSi ,…,GSALm GSi ) has j GSi ∈{1 GSi ,…,m GSi } a different GSi "Target State Abstraction Level (GSALj GSi ), where the different target state abstraction levels (GSALj GSi ) is one of the graphical user interface based inputs, c) For the specified target state (GS WF ) and based on the facts (FC WF ) and for planning operators (POP 1,WF ,…,POP n,WF ) of at least one selected planning operator's deployment planning domain (PD WF ) defines (dfn) a planning task (PT), and d) Through the planning method (PA) and execution engine (ECE) according to the defined planning task (PT) d1) Iteratively checking the execution of the selected planning operator in order to find the target state (GS) that is achieved through the main query loop (PQL) WF ) includes the complete plan (PL f ), where in successive loop iterations, executing (exc) said at least one selected planning operator by said execution engine (ECE) <ii>Generates (grt) by the execution engine (ECE) a primary control command (CC) attributed to a selected planning operator of at least one execution p ), <iii>The main control command (CC) output via the communication interface (CI) is applied accordingly. p ) to simulate (sml) the technical system workflow (WF) and <iv>The main measurement result (MMR) of the measurement (MM) is fed back to the execution engine (ECE) p ) checks (chk) the system simulation accordingly until a state that achieves the target state (GS WF ) of the complete program (PL f ) or an error (ERR) reaching the workflow (WF) of said technical system, and< / iv> < / iii> < / ii> d2) When the complete plan (PL) is found in d1) f ), the target state (GS) is iteratively checked through the secondary query loop (SQL) WF ) of the found complete program (PL f ), where in successive loop iterations, the The complete plan (PL) is executed (exc) by the execution engine (ECE) f ), <ii>A complete plan (PL) attributed to the execution is generated (grt) by the execution engine (ECE) f )'s secondary control command (CC s ), <iii>The secondary control command (CC) output via the communication interface (CI) is applied accordingly. s ) to simulate (sml) or execute (ehw) in hardware the technical system workflow (WF) and <iv>The secondary measurement result (MMR) of the measurement (MM) fed back to the execution engine (ECE) s ) accordingly checks (chk) the system simulation or system hardware execution until the target state (GS) is achieved WF ).< / iv> < / iii> < / ii> 8. Device (DEV) according to claim 7, characterized in that After the third step (S3), the workflow-related target state (GSSP) specified (spf) by the graphical user interface-based input of the target state specification process (GSSP) is given by further consecutive steps. WF ), b4) A fourth step (S4) in which different target state abstraction levels (GSALj) are selected. GSi ) and present them to the user, and b5) A fifth step (S5), in which the user is asked whether all sampled presented target states are feasible, and if the presented target states are feasible, the different initial target states (G i ) and different target state abstraction levels (GSALj GSi ) are both inputs to the graphical user interface, or if the presented target state is not feasible, then either ("Option I") select a different target state abstraction level (GSAL) for another choice GSi ) execute the third step (S3), or ("Option II") for another selected different initial target state (G i ) executes the first step (S1).

9. Device (DEV) according to claim 7 or 8, characterized in that One or more of the at least one selected planning operator may be selected several times for iteratively checking execution in order to find the complete plan (PL) comprising the first sequence of operations (OPS1). f ).

10. Device (DEV) according to claim 7, 8 or 9, characterized in that The technical system (TS) is a robotic system.

11. Device (DEV) according to any one of claims 7 to 10, characterized in that A computer into which the computer implemented tool (CIT) according to claim 6 is loadable.

Images