Method for automatically generating a behavior tree program for controlling a machine
By observing user-machine interactions and using statistical inference to automatically generate behavior tree programs, the complexity of autonomous machine programming is solved, programming efficiency and accuracy are improved, and the rapid implementation of autonomous machines is supported.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SIEMENS AG
- Filing Date
- 2022-04-29
- Publication Date
- 2026-05-29
Smart Images

Figure CN115268855B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to a computer-implemented method for automatically generating, for example, a behavior tree program for controlling machines in an industrial automation system, and an auxiliary device configured to perform the method. Background Technology
[0002] There is a need for systems that control machines or machines across various technological fields. For example, in the context of industrial automation, autonomous machines are becoming increasingly important. There is ongoing effort in the processing and manufacturing industries to efficiently achieve mass production based on factories equipped with modular and autonomous machines operating on the shop floor. However, modular, and especially autonomous, machines are challenging to program and coordinate.
[0003] In the case of structured text, the programming languages for programmable logic controllers (PLCs) according to the IEC 61131 standard follow an imperative programming paradigm. They define the behavior of a concrete machine as a rigid sequence with well-defined decision points. In the case of function block diagrams, analog logic circuits are used. This is well-suited for classic factories where precisely defined repetitive movements are expected, responses depend on simple, typically binary, sensor inputs, and machines have a finite number of states, such as running, idle, and off.
[0004] Behavior trees are a powerful programming paradigm used to overcome the limitations of classic PLC programming methods. Behavior trees describe the transitions between finite sets of tasks in a modular way. Their strength stems from their ability to create very complex tasks composed of simple tasks without worrying about how the simple tasks are implemented. Tasks describe individual behavioral steps, and several tasks are combined into a composition, which defines the order and conditions in which individual tasks should be executed. Behavior trees exhibit some similarities to hierarchical state machines, with the key difference being that the main building blocks of behavior are tasks rather than states. It is easily understood by humans, making behavior trees less prone to errors. The terms "expression of behavior trees" and "behavior tree program" are used synonymously throughout this document.
[0005] Today, behavior trees are manually created by automation engineers and input into the controllers of machines or machine systems. While behavior trees are easier to handle than classic PLC programming languages, they still require some amount of training for automation engineers. Summary of the Invention
[0006] Therefore, one objective of this invention is to support automation engineers in generating behavior trees.
[0007] These objectives are addressed by the subject matter of the independent claims. Advantageous embodiments are proposed in the dependent claims.
[0008] According to the first aspect, a computer-implemented method for automatically generating behavior tree programs for controlling machines is proposed, the method comprising the following steps:
[0009] - Transmit a sequence of machine commands input by the user from the user interface to the controller.
[0010] - When machine commands are executed in the controller that controls the machine, supervisory data is received from the controller in the user interface.
[0011] - Observe and replicate machine commands and supervisory data transmitted between the controller and the user interface.
[0012] Machine commands and monitoring data are stored in the log recording unit.
[0013] - A program that generates derived behavior trees from stored machine commands and supervisory data through statistical inference, and
[0014] - The generated behavior tree program is sent to the controller unit to control the machine.
[0015] The machine is controlled by user-input machine commands. These commands are executed in the controller, invoking one or more broadly defined, enclosed program segments to perform actions on the machine or its components. Machine commands are structured according to programming languages such as those in the IEC 61131 standard. Instead of programming a behavior tree, it is automatically generated based on user-machine interaction. The user's intent regarding machine operations (which is not explicitly available) is not directly queried but rather collected through observation, prediction, and extrapolation of user behavior. Statistical inference is the process of using data analysis to infer the properties of the distribution of potential probabilities. The structure of the behavior tree is derived using statistical inference. Methods applied to statistical inference can include, for example, association rule mining, prior algorithms, or methods for sequence pattern mining. Machine commands are transmitted via communication protocols (e.g., OPC UA, Profinet) over a communication connection between the user interface and the controller.
[0016] According to an embodiment of the present invention, machine commands activate machine behaviors, functions, or operating states within the machine.
[0017] Machine commands define machine behavior, functions, or operational states at varying levels of detail, ranging from high-level descriptions of complete actions to low-level descriptions of highly precise and detailed components of those actions. This allows for detailed analysis considering the substructures of machine commands and enables the prediction of the behavior tree structure closely approximating the user's expected machine operations. In this specification, the terms "machine command" and "high-level machine command" are used as synonyms.
[0018] According to an embodiment of the invention, the monitoring data includes machine status information and / or sensor data when controlled by a controller executing higher machine commands.
[0019] Status information includes whether the execution of machine commands and / or actions activated by machine commands was successful or failed. Status information also includes sensor data values collected by sensors in the machine or its components, providing information about the machine's condition or the status of the automated process being performed. Supervisory data can be provided independently of the execution of machine commands. Therefore, status information provides a comprehensive overview of the machine's operation.
[0020] According to an embodiment of the invention, at least one task of the behavior tree is derived from at least one machine command at the inference unit.
[0021] The task provides the basic components of the behavior tree and executes the machine's actions when it is executed. The inference unit provides the basic components of the behavior tree and thus generates the behavior tree from scratch, i.e., without any input of the programming elements of the behavior tree.
[0022] According to an embodiment of the present invention, the structure of the behavior tree is deduced by analyzing the following:
[0023] - Which exported tasks are executed according to their time relationships?
[0024] - The return value of the supervision data,
[0025] - Parameter settings for task execution.
[0026] - Co-occurrence of supervisory data values and changes in supervisory data values with task execution, task cancellation, and task parameterization.
[0027] The inference unit derives further structure of the behavior tree through the described analysis steps, particularly defining the composition and decorator of rules regarding the conditions and order of task execution. Specifically, decorators are inferred by analyzing the co-occurrence of supervised data values and their changes with task execution, task cancellation, and task parameterization. Co-occurrence means analyzing which supervised data values or value opportunities occur simultaneously or temporally related to the execution, cancellation, or parameterization of one or more tasks.
[0028] According to an embodiment of the present invention, the synthesis of several tasks in the combined behavior tree is inferred by analyzing the sequences of successfully executed tasks and failed tasks, as well as the transition probabilities of each pair of tasks in the sequences.
[0029] According to an embodiment of the invention, the inference unit provides suggestions to the user interface for further machine commands to be input by the user.
[0030] This allows users to automate machine operations. Users require minimal knowledge of machine command programming. The suggestions from the inference unit can be reviewed and examined, and can be used to continuously train the inference unit.
[0031] According to an embodiment of the present invention, the inference unit provides the user interface with suggestions for further behavior tree structure elements to be input by the user.
[0032] The structure of a behavior tree includes at least structural elements, which comprise the composition of behavior tree components. Behavior tree components include, for example, tasks, compositions, and decorators. This advice assists users in improving their programming of behavior trees and enhancing the quality of the resulting behavior trees.
[0033] According to an embodiment of the present invention, the reasoning unit includes a semantic model that provides domain knowledge about at least one of a sequence of tasks related to a specific machine operation scenario, the conditions of the tasks, and the transition probabilities of each pair of tasks in the sequence.
[0034] The semantic model comprises a set of behavior tree components related to specific machine operation scenarios. These components define typical sequences of tasks and the conditions under which tasks should be performed. The semantic model can provide information about tasks that may fail and which fallback solutions are possible in such cases. This accelerates the generation of behavior trees for new operation scenarios and variations thereof. Furthermore, it allows the inference unit to provide a wide range of suggestions.
[0035] According to an embodiment of the invention, the inference unit provides recommendations for behavior tree structure elements to the user interface based on assumptions about further expected machine operations, in order to verify or modify the recommendations.
[0036] This provides the inference unit with the possibility of receiving immediate feedback from the user and early alignment of the behavior tree program with the machine's expected actions.
[0037] According to an embodiment of the present invention, the generated behavior tree program is provided from the inference unit to the controller for controlling the machine.
[0038] The generated behavior tree program can be provided directly to the controller from the inference unit, or it can be passed from the inference unit to the user interface, which, upon receiving user confirmation, forwards it to the controller. Direct provision allows for rapid implementation of the behavior tree. Indirect provision via the user interface allows users to approve, correct, or disable the implementation, i.e., the execution of the behavior tree automatically generated in the controller.
[0039] According to the second aspect, an auxiliary system for assisting in the generation of machine behavior tree programs is proposed, comprising a programming interface, an observer unit, a logging unit, and an inference unit, wherein...
[0040] - The interface unit is configured to transmit a sequence of machine commands input by the user from the user interface to the controller, and to receive supervisory data from the controller while the machine commands are executed in the controller that controls the machine.
[0041] - The observer unit is configured to observe and replicate machine commands and supervisory data transmitted between the controller and the user interface.
[0042] - The log recording unit is configured to store machine commands and monitoring data.
[0043] - The inference unit is configured to generate a derived behavior tree program from stored machine commands and supervisory data through statistical inference, and send the generated behavior tree program at the controller unit to operate the machine.
[0044] According to an embodiment of the present invention, the user interface includes a graphical user interface configured to display supervisory data, suggestions or recommendations to the user, and / or a sequence of input machine commands or feedback on the suggestions and / or recommendations to the user.
[0045] This provides flexible communication between users and controllers.
[0046] According to an embodiment of the present invention, the log recording unit is configured as a knowledge base.
[0047] A knowledge base is a technology used to store complex structured and unstructured information used by computer systems that provide expert systems. A knowledge base represents facts and the ways in which inferences are made about those facts to deduce new facts or highlight inconsistencies. Preferably, the knowledge base is structured as a knowledge graph.
[0048] According to a third aspect, a computer program product that can be directly loaded into the internal memory of a digital computer is proposed, including a software code portion for performing the steps according to any one of the preceding claims when the product is run on the digital computer. Attached Figure Description
[0049] In the following description, embodiments of the present disclosure will be described in more detail and with reference to the accompanying drawings.
[0050] Figure 1 A classic programming system for programming a machine is illustrated schematically.
[0051] Figure 2 An auxiliary system for generating a behavior tree for controlling a machine is schematically illustrated according to an embodiment of the present disclosure.
[0052] Figure 3 The reasoning of the structural elements of a behavior tree according to an embodiment of the present disclosure is illustrated schematically.
[0053] Figure 4A -B schematically illustrates the synthesis of a behavior tree inferred according to embodiments of the present disclosure.
[0054] Figure 5A -C schematically illustrates a decorator for inferring a behavior tree according to embodiments of the present disclosure.
[0055] Figure 6 A behavior tree program generated according to an embodiment of the present disclosure is illustrated schematically.
[0056] Figure 7 A flowchart illustrating a method according to an embodiment of the present disclosure is shown. Detailed Implementation
[0057] Note that in the following detailed description of the embodiments, the drawings are merely schematic, and the elements illustrated are not necessarily shown to scale. Rather, the drawings are intended to illustrate the function and cooperation of the components. It is to be understood here that any connection or coupling of functional blocks, devices, components, or other physical or functional elements may also be achieved through indirect connections or couplings (e.g., via one or more intermediate elements). Connections or couplings of elements or components may be achieved, for example, through wired connections, wireless connections, and / or a combination of wired and wireless connections. Functional blocks may be implemented by dedicated hardware, by firmware and / or software mounted on programmable hardware, and / or by a combination of dedicated hardware and firmware or software.
[0058] Figure 1 A programming system 10 is schematically illustrated for programming machine 14, for example, in a classic automated industrial setting.
[0059] The programming system includes an operation interface 11, a programming interface 12, automation, a controller 13, and a machine 14. The machine 14 can be any type of machine manufactured, such as an automation unit, factory unit, production line, processing line, assembly line, or autonomous guided vehicle (AGV), robot, and others.
[0060] Programming interface 12 serves as the interface between user 16 (e.g., an automation engineer) and controller 13. The automation engineer uses programming interface 12 to define automation program 15. Automation program 15 can be structured using any kind of programming language capable of controlling the machine. Examples of programming languages using the structured text programming language of the IEC 61131 standard for programmable logic controllers (PLCs), such as Step7, include contact plan (KOP), function plan (FUB), and Graph7. Programming interface 12 transmits automation program 15 to controller 13. Automation program 15 is executed on controller 13. Controller 13 can be an automation system, personal computer, edge device, PLC, or any other type of logic machine. During machine operation, user 16 (e.g., a machine operator) uses operation interface 11 to monitor and control machine 14.
[0061] Figure 2 The illustration shows an auxiliary system 20 for generating a behavior tree program 28 for controlling machine 24 according to an embodiment of the present disclosure. Machine 24 can be any type of machine in manufacturing, such as an automation unit, factory unit, production line, processing line, assembly line, or autonomous guided vehicle (AGV), robot, and others, such as machine 14. Controller unit 23 controls machine 24 by executing behavior tree program 28. The auxiliary system 20 assists in generating behavior tree program for machine 24 and includes a programming interface 21, an observer unit 25, a logging unit 26, and an inference unit 27.
[0062] User interface 21 is the interface between user 22 (e.g., machine operator or automation engineer) and controller unit 23. User interface 21 is configured as a graphical user interface, display, keyboard, mouse, control unit, and the like. User interface 21 is configured to receive input from the user in the role of machine operator, i.e., inputting machine commands during machine operation. During the training phase of auxiliary system 20, particularly during the training phase of inference unit 27, user 22 controls machine 24 by inputting machine commands into user interface 21. Machine commands, especially high-level machine commands, define production steps, transport steps, material flow, material handling, material processing, assembly steps, and so on.
[0063] User 22 uses user interface 21 to monitor and control the machine. To control the machine, such as a conveyor belt, the user enters a first machine command, which, when executed at the controller, instructs the machine to place a block on the conveyor belt. With a second machine command, user 22 starts the conveyor belt. With a third machine command, user 22 stops the conveyor belt when the block encounters a light barrier.
[0064] Observer unit 25 intercepts communication on communication connection 29 between user interface 21 and controller unit 23. Observer unit 25 collects all user machine commands input to user interface 21, which are intended for use by controller unit 23. Observer unit 25 is a support tool for understanding user intent. Observer unit 25 collects user interactions and feedback from 22, i.e., supervisory data from the machine or from components of the machine, such as sensors and light barriers configured for a conveyor belt machine, where several activities are performed during the transport of objects.
[0065] Log recording unit 26 stores communication, namely machine commands and supervisory data observed and copied over the connection between user interface 21 and controller unit 23. Log recording unit 26 can be a database, list, file, knowledge graph, knowledge base, or the like. Log recording unit 26 is a central storage system and contains all relevant information input by user 22 into user interface 21 to derive automation programs for controller unit 23.
[0066] Inference unit 27 derives a behavior tree from information in logging unit 26. Inference unit 27 derives at least one task from at least one machine command into the behavior tree. Tasks in the behavior tree typically correspond to machine commands. Inference unit 27 includes statistical inference functionality, also known as a statistical inference engine, which learns from stored machine commands and supervisory data which tasks are typically executed in parallel or sequentially, what the typical parameter settings for tasks are under what conditions, and which tasks typically follow certain other tasks, events (e.g., proximity alarms), or conditions (e.g., light barrier interruptions). Methods applied to statistical inference can include, for example, association rule mining, prior algorithms, or methods for sequence pattern mining.
[0067] In doing so, preconditions, fallback scenarios, and typical task sequences can be learned, as well as the composition of tasks and the logical interconnections of several tasks, such as sequence composition or fallback composition. The inference unit 27 sends suggestions to the user interface 21 to assist the user 22 in automating the operation.
[0068] Figure 3An example of statistical inference performed in inference unit 27 is shown. Auxiliary system 20 observes and stores two log files, L1 and L2, which contain sequences of executed machine commands. Based on the machine commands, tasks T11, ..., T15, T21, ..., T26 are derived. Log files L1 and L2 further include the success status and variable status for each task T11, ..., T15, T21, ..., T26. Figure 3 In the sequence, tasks T11, T14, T15, T21-T23, T25, and T26 have been successfully completed, while tasks T12, T13, and T24 have failed. Based on this information, the inference unit 27 infers the success and failure transition probabilities TP. The inference unit 27 can now provide user 22 with suggestions S for sequence synthesis and backtracking synthesis, which user 22 can accept or reject.
[0069] Figure 4A Sequence 30 of tasks T31, ..., T33 is shown. Tasks are executed with high transition probabilities, see the values indicated above the arrows between two tasks. Sequence 30 of tasks T31, ..., T33 is inferred as a candidate for sequence synthesis in the case of successful task execution.
[0070] The task that follows a failed task with a high probability is a candidate for rollback synthesis. Figure 4B This shows a backoff composition 40 for tasks T41, ..., T45. Task 42 (which is the successfully executed task) follows task T41 with a high transition probability.
[0071] Decorator identifiers can also be supported through log file analysis. Decorators are attached to compositions or tasks and define whether a branch in the behavior tree or even a single node can be executed. Decorators can also modify the success or failure status of the decorated task or composition.
[0072] exist Figure 5A In this context, task T51 is followed by either task T52 or task T53. The probability of the sequence T51-T52-T53 is related to the value of variable A, and the probability of the sequence T51-T53 is related to the value of variable B. This means that variables A and B are candidates for the "if" decorator.
[0073] Candidates for the "guard" decorator can be identified by inferring whether the value of a specific variable changes for a failed task. Figure 5B In the example, task T56 failed, and the value of variable A has changed. Variable A is a candidate for the "protect" decorator.
[0074] If the value of a specific variable changes at the start of a new task, then candidates for the "wait" decorator can be identified by observation. Figure 5C The example in the text shows the change in the value of variable A in relation to the transition between tasks T54 and T55.
[0075] Figure 6 A behavior tree program 50 automatically generated by the auxiliary system 20 is shown. Behavior tree 50 controls the system's movement to a light barrier, see 51. Behavior tree component 52 represents "sequence" composition. Behavior tree component 53 represents "wait for light-barrier=false", behavior tree component 54 represents "<> stop", component 55 represents "wait for sensor_2=true", and component 56 represents "<> right". The generated program understands that:
[0076] - Sensor 2 triggers the start of the conveyor belt (waiting for a change in the value of sensor 2).
[0077] -light_barrier stops the conveyor belt (waiting for the value of light_barrier to change).
[0078] The reasoning unit 27 may include a semantic model that provides domain knowledge. For example, it may provide information such as "the task can only be performed at a certain safety level, the robotic arm's pickup is successful only in 90% of cases, and therefore 'backoff' is necessary." The semantic model can provide information about tasks that may fail and which backoff solutions are possible. The reasoning unit 27 may make recommendations to the user 22 based on assumptions. The user 22 may verify or correct the assumptions. Whenever the reasoning unit 27 is uncertain about something when creating the behavior tree program, it may request user input, such as parallel tasks, backoffs, preconditions, wait conditions, loops, if-else statements, or stopping conditions.
[0079] The example question given to users is, "We've noticed that task A typically executes after the light barrier is interrupted. Would you like to save this as a prerequisite?" "
[0080] The auxiliary system 20 is configured to execute a computer-implemented method for automatically generating a behavior tree program for controlling the machine 24, such as... Figure 7 As shown schematically in the diagram.
[0081] In the first step S1, a sequence of machine commands input by user 22 at user interface 21 is transmitted to controller unit 23. When the machine commands are executed in controller unit 23, monitoring data is received from controller unit 23 in user interface 21, see step S2. Machine commands and monitoring data are transmitted over the communication connection between user interface 21 and controller unit 23, for example via protocol messages according to OPV UA, Profinet, and the like. Machine commands activate machine behavior, functions, or operating states in machine 24. Monitoring data includes machine status information and / or sensor data when controlled by the controller executing the machine commands.
[0082] The machine commands and monitoring data are observed and copied at the communication connection between the controller 23 and the user interface 21, as described in step S3. Subsequently, in step S4, the machine commands and monitoring data are stored in the log recording unit 26.
[0083] In step S5, a behavior tree program is derived from stored machine commands and supervision data through machine learning, particularly through statistical inference. At least one task of the behavior tree is derived from at least one machine command at inference unit 27. The structure and components (i.e., synthesizers and decorators) of the behavior tree are inferred by analyzing: which derived tasks are executed in relation to each other in time, the return values of the supervision data, the parameter settings of the tasks regarding task execution, the supervision data values, and the co-occurrence of changes in supervision data values with task execution, task cancellation, and task parameterization. Inference unit 27 provides suggestions to user interface 21 for further machine commands to be input by the user.
[0084] The inference unit 27 provides suggestions for further machine commands to the user interface 21, which can be accepted and entered by the user. The suggestions can also be ignored, and / or different machine commands or behavioral task structure elements, i.e., tasks, synthesizers, or decorators, can be entered. Based on assumptions about further expected machine operations, the inference unit 27 provides recommendations for behavior tree structure elements to the user interface 21 to verify or modify the recommendations. Finally, the generated behavior tree program is sent to the controller unit 23 to control the machine 24, see step S6.
[0085] It should be understood that the above description of the examples is intended to be illustrative, and the concepts illustrated are readily adaptable to various modifications. For example, the illustrated concepts can be applied to monitor various types of machines or machine systems and various types of data that can be monitored on machines or machine systems, and can utilize detailed labeling information in various ways.
Claims
1. A computer-implemented method for automatically generating a behavior tree program for controlling a machine, the method comprising the following steps: - Transmit a sequence of machine commands input by the user from the user interface to the controller. - When machine commands are executed in the controller that controls the machine, supervisory data is received from the controller in the user interface. - Observe and replicate machine commands and supervisory data transmitted between the controller and the user interface. Machine commands and monitoring data are stored in the log recording unit. - A program that generates derived behavior tree procedures from stored machine commands and supervisory data through statistical inference, wherein the inference unit includes a semantic model that provides information about tasks that may fail and which fallback solutions are possible in such cases. - The generated behavior tree program is sent to the controller unit to control the machine.
2. The computer-implemented method according to claim 1, wherein the machine command activates machine behavior, function, or operating state in the machine.
3. The computer-implemented method according to claim 1, wherein, Supervisory data includes machine status information and / or sensor data when controlled by a controller that executes machine commands.
4. The computer-implemented method according to any one of claims 1-3, wherein, At least one task of the behavior tree program is derived from at least one machine command at the inference unit.
5. The computer-implemented method according to claim 4, wherein, The structure of the behavior tree program can be deduced by analyzing the following: - Which exported tasks are executed according to their time relationships? - The return value of the supervision data, - Parameter settings for task execution. - The co-occurrence of supervisory data values and changes in supervisory data values with task execution, task cancellation, and task parameterization.
6. The computer-implemented method according to claim 5, wherein, By analyzing the sequences of successfully executed and failed tasks, as well as the transition probabilities of each pair of tasks in the sequence, the composition of several tasks in the combined behavior tree program can be deduced.
7. The computer-implemented method according to claim 4, wherein, The inference unit provides suggestions to the user interface for further machine commands that the user needs to input.
8. The computer-implemented method according to claim 4, wherein, The inference unit provides suggestions to the user interface for further behavior tree structure elements to be input by the user.
9. The computer-implemented method of claim 4, wherein the semantic model provides domain knowledge about at least one of a sequence of tasks related to a specific machine operation scenario, the conditions of the tasks, and the transition probabilities of each pair of tasks in the sequence.
10. The computer-implemented method according to claim 4, wherein, The inference unit provides recommendations for behavior tree structure elements to the user interface based on assumptions about further expected machine operations, in order to verify or modify the recommendations.
11. The computer-implemented method according to claim 4, wherein, The generated behavior tree program is provided from the inference unit to the controller for controlling the machine.
12. An auxiliary system for assisting in the generation of machine behavior tree programs, comprising a programming interface, an observer unit, a logging unit, and an inference unit, wherein... - The interface unit is configured to transmit a sequence of machine commands input by the user from the user interface to the controller, and to receive supervisory data from the controller while the machine commands are executed in the controller that controls the machine. - The observer unit is configured to observe and replicate machine commands and supervisory data transmitted between the controller and the user interface. - The log recording unit is configured to store machine commands and monitoring data. - The inference unit is configured to generate a derived behavior tree program from stored machine commands and supervisory data through statistical inference, and to send the generated behavior tree program to the controller unit to operate the machine, wherein... The reasoning unit includes a semantic model that provides information about tasks that may fail and which fallback solutions are possible in such cases.
13. The auxiliary system according to claim 12, wherein, The user interface includes a graphical user interface configured to display supervisory data, suggestions or recommendations to the user, and / or sequences of machine commands or feedback on the user's suggestions and / or recommendations.
14. The auxiliary system according to claim 12 or 13, wherein, The log recording unit is configured as a knowledge graph or knowledge base.
15. A computer program product that can be directly loaded into the internal memory of a digital computer, comprising software code portions for performing the steps according to any one of claims 1-11 when the product is run on the digital computer.