Method for generating behavior tree for controlling robot device

Abstract

To provide a device and method for generating a behavior tree for controlling a robot device.SOLUTION: According to various embodiments, a method for generating a behavior tree for controlling a robot device comprises: combining a plurality of predetermined behavior trees and background knowledge into a behavior tree knowledge graph; representing the behavior tree knowledge graph in a latent space; extracting, from a prompt describing a desired behavior of the robot device, a prompt representation graph; supplementing the prompt representation graph according to relations specified by the behavior tree knowledge graph; selecting a sub graph of the behavior tree knowledge graph depending on a similarity to the supplemented prompt representation graph; and generating the behavior tree for controlling the robot device by adjusting the selected sub graph according to knowledge from the prompt and the behavior tree knowledge graph.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to devices and methods for generating behavior trees for controlling robotic devices. [Background technology]

[0002] Robotic devices can be controlled using behavior trees, which indicate the actions a robot should take in various situations and the constraints the robotic device should respect. Behavior trees in robotics aim to provide a structured, modular framework for defining and executing autonomous behaviors. They decompose complex tasks into manageable modules, allowing for easier design, modification, and reusability. This promotes clear decision-making logic and facilitates efficient validation and testing of robot behaviors, contributing to their effective operation in diverse environments.

[0003] Behavior trees can be manually generated by experts, but this has several drawbacks: Time-consuming. Manually creating a behavior tree can be a time-intensive process, especially for complex tasks or scenarios. It requires careful planning, design, and implementation and can be resource-intensive. Depends on expertise. Effective manual behavior tree generation requires a deep understanding of the domain and specific system requirements. This expertise is not always readily available, leading to potential design shortcomings. Prone to human error. Because behavior trees are created by humans, there is a greater chance of making mistakes in design or implementation. These errors can lead to unexpected or incorrect behavior in the system. Lack of flexibility. When behavior trees are designed and implemented manually, they can be less flexible in adapting to unexpected or dynamic situations. Making major changes may require reevaluating the entire design. Summary of the Invention [Problem to be solved by the invention]

[0004] Therefore, an efficient approach for the automatic generation of behavior trees for controlling robotic devices is desirable. [Means for solving the problem]

[0005] According to various embodiments, a method for generating a behavior tree for controlling a robotic device is provided, the method comprising: combining a plurality of predetermined behavior trees and background knowledge into a behavior tree knowledge graph; representing the behavior tree knowledge graph in a latent space by a knowledge graph embedding method in the form of a plurality of multiple clusters of latent space elements, each cluster representing a subgraph of the behavior tree knowledge graph; extracting from a prompt describing a desired behavior of the robotic device a prompt representation graph, the prompt representation graph representing terms in the prompt as nodes and relationships between the terms indicated in the prompt as edges; and interfacing a plurality of terms indicated in the prompt and / or one or more terms indicated in the prompt with the behavior tree knowledge graph. supplementing the prompt representation graph with one or more nodes and / or one or more edges according to relationships specified by the behavior tree knowledge graph between the prompt representation graph and other terms represented in the prompt; representing the supplemented prompt representation graph as an additional cluster in the latent space by the knowledge graph embedding method; determining a similarity of the additional cluster with the cluster among the plurality of clusters; selecting one of the subgraphs of the behavior tree knowledge graph according to the similarity of the cluster representing the subgraph with the additional cluster; and generating the behavior tree for controlling the robotic device by adjusting the selected subgraph according to knowledge from the prompt and the behavior tree knowledge graph.

[0006] The above-described method enables the automatic generation of a behavior tree appropriate for a given task with low user overhead, since the desired behavior can be specified by text prompts augmented by domain knowledge and expert knowledge contained in the behavior tree knowledge graph.

[0007] The automatic generation of behavior trees (as provided by the methods described above) allows for streamlining the human-robot interaction process, so that humans can easily extend or tune the robot's behavior for new or specific scenarios.

[0008] Leveraging domain knowledge by utilizing a behavior tree knowledge graph in the generation of a behavior tree, i.e., domain-specific knowledge represented in a knowledge graph, allows for: Leveraging domain expertise: Domain-specific knowledge graphs provide detailed information about the entities, relationships, and concepts specific to the domain. This allows for a deeper understanding of the domain and the generation of more accurate and contextually relevant behavior trees. Improved context understanding: Integrating domain-specific information allows for a better understanding of the context in which the information is presented. This is especially important for tasks that require a nuanced understanding of domain-specific concepts and terminology. Enhanced fact verification and information retrieval: Through domain-specific knowledge graphs, it is possible to verify facts and find accurate information relevant to a specific domain. This is important for applications where information accuracy and reliability are paramount. Tailored response generation: The method described above can generate responses, i.e., behavior trees, tailored to text input. This is especially important in domains such as robotics, where accuracy is crucial. Ambiguity Reduction: Domain-specific knowledge graphs can help disambiguate terms or concepts that may have multiple meanings in different contexts. This reduces the chances of generating ambiguous or inaccurate behavior trees. Better integration with existing systems: Structured information in the form of knowledge graphs can be seamlessly integrated into existing systems and workflows in the domain, providing enhanced capabilities and efficiency.

[0009] In the following, various examples are given.

[0010] Example 1 is a method for generating a behavior tree for controlling a robotic device as described above.

[0011] Example 2 includes extracting the prompt expression graph in the method of Example 1 by a machine learning model configured to operate on natural language text (i.e., a natural language processing model, e.g., a large-scale language model (LLM)) to which the prompt is provided as input.

[0012] Machine learning models such as LLMs enable effective extraction of semantics, and therefore terms and their relationships. Furthermore, machine learning models may be trained as part of training the behavior tree generation process in an end-to-end manner (e.g., a user provides feedback on the behavior tree generated from prompts, and the generation process is adjusted (trained) accordingly, particularly the machine learning model and possibly the KGE).

[0013] Example 3 is a method of Example 1 or 2, in which selecting a subgraph includes selecting a subgraph of a behavior tree knowledge graph, in which a cluster of the subgraph in which the subgraph of the behavior tree knowledge graph is represented has the highest similarity to an additional cluster.

[0014] This ensures the best fit criteria is used for behavior tree generation, however other criteria may also be considered in the selection.

[0015] Example 4 includes, in the method of any one of Examples 1 to 3, generating a prompt such that the prompt expression graph includes one or more nodes and / or one or more edges for terms and / or relationships that are not included in the behavior tree knowledge graph.

[0016] This means that behavior can be specified beyond what is contained in the behavior tree knowledge graph.

[0017] Example 5 is the method of any one of Examples 1 to 4, further comprising combining the generated behavior tree with a behavior tree knowledge graph.

[0018] Thus, the behavior tree knowledge graph grows in terms of the behaviors and knowledge it specifies during the generation of a "new" behavior tree, and thus the generation process improves over time. For example, if a behavior tree for a prompt that specifies a particular sub-behavior is generated (and combined with the behavior tree knowledge graph), it may not be necessary to specify this sub-behavior for subsequent prompts for the generation of subsequent behavior trees.

[0019] Example 6 is a method for controlling a robotic device, comprising: describing a desired behavior of the robotic device in a text prompt; generating a behavior tree for controlling the robotic device according to any one of Examples 1 to 5; and controlling the robotic device according to the generated behavior tree.

[0020] A seventh embodiment is a data processing device (particularly a robotic device controller) configured to perform the method of any one of the first to sixth embodiments.

[0021] An eighth embodiment is a computer program comprising instructions for causing a computer to carry out the method according to any one of the first to sixth embodiments when executed by the computer.

[0022] Example 9 is a computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of Examples 1 to 6.

[0023] In the drawings, like reference characters generally refer to equivalent parts throughout the different views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various aspects are described with reference to the following drawings: [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a schematic diagram of a vehicle. [Figure 2] FIG. 10 is an illustration of the creation of a "new" behavior tree according to one embodiment. [Figure 3] FIG. 1 is a flow diagram illustrating a method for generating a behavior tree for controlling a robotic device, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0025] The following detailed description refers to the accompanying drawings, which show, by way of example, specific details and aspects of the present disclosure in which the invention may be practiced. Other aspects may be utilized, and structural, logical, and electrical changes may be made, without departing from the scope of the present invention. Various aspects of the present disclosure are not necessarily mutually exclusive, as some aspects of the present disclosure may be combined with one or more other aspects of the present disclosure to form new aspects.

[0026] Various examples are described in more detail below.

[0027] FIG. 1 shows a vehicle 101 .

[0028] In the example of FIG. 1, a vehicle 101, eg a car or truck, is equipped with a vehicle control device (electronic control unit, eg also referred to as a control unit, eg an electronic control unit (ECU)) 102.

[0029] The vehicle control device 102 has data processing components, such as a processor (e.g., a CPU (Central Processing Unit)) 103, and memory 104 for storing control software 107 according to which the vehicle control device 102 operates and data processed by the processor 103. The processor 103 executes the control software 107.

[0030] For example, the stored control software (computer program) has instructions that, when executed by the processor, cause the processor 103 to perform driver assistance functions (i.e., functions of an advanced driver assistance system (ADAS)) or to autonomously control the vehicle (autonomous driving (AD)).

[0031] The control software 107 may be transferred, for example, from the computer system 105 to the vehicle 101, for example, via the network 106 (or using a storage medium such as a memory card). This may also be done during operation (or at least while the vehicle 101 is with the user), for example, as the control software 107 is updated with new versions over time.

[0032] The control software 107 determines control actions (steering actions, braking actions, etc.) for the vehicle from available input data that contain information about the environment or that derive information about the environment (such as by detecting other road users, e.g. other vehicles), such as sensor data from one or more sensor devices 109, such as cameras, RADAR sensors, LiDAR sensors, ultrasonic sensors, etc. of the vehicle 101, connected to the vehicle control device 102 via a communication system 110 (e.g. a vehicle bus system such as CAN (Controller Area Network)).

[0033] The control software 107 may include, for example, a behavior tree (eg, generated by the computer system 105).

[0034] According to various embodiments, an approach is provided for automatically generating behavior trees (BTs) for robotic device control (e.g., vehicle control or robotic device control for controlling a robot (e.g., a robotic arm) to perform a robotic manipulation task), possibly in an uncertain and changing environment. This approach can be integrated, for example, within a task and motion planning framework that calculates a sequence of actions and associated motions to be performed by each robotic device (e.g., to perform a manipulation task). For example, the generated BTs may be automatically written to one or more XML files describing the BTs that are used as a basis for controlling the robotic device.

[0035] To this end, according to various embodiments, behavior trees for robot control (e.g., robot manipulation) are represented together with domain and expert knowledge to provide a comprehensive representation of the relevant knowledge and enable the generation of new behavior trees in a (semi-)automated manner.

[0036] According to various embodiments, the approach first represents all relevant knowledge, such as information about robot sensors, configurations, behaviors, etc., in a structured format known as a behavior tree knowledge graph. This enables learning and understanding of relationships, hierarchies, and contextual information about entities appearing in a control situation (or scene). A user can describe new behaviors for each robotic device in a natural language format. This description is taken as input for generating a behavior tree based on the similarity of the given input with previous (i.e., predetermined) behavior tree descriptions.

[0037] Behavior trees can become complex and difficult to manage for large or complex systems, potentially leading to design challenges and poor maintainability. In addition, they may struggle to handle continuous or dynamic environments where behavior needs to be continually adapted, requiring additional mechanisms for real-time responsiveness.

[0038] Using a behavior tree knowledge graph that conforms to an underlying ontology, high-level domain knowledge information can be encoded by conceptualizing entities and their relationships as well as sensory information and configuration. According to various embodiments, this knowledge is then used to automate the generation of new behavior trees on demand that conform to predefined constraints and reasonable behaviors. Thus, the robot is empowered with understanding and can respond to human commands or queries related to the manipulation task. This enhances human-robot interaction and collaboration.

[0039] FIG. 2 illustrates the generation of a (“new”) behavior tree 215 according to one embodiment.

[0040] In a first preparation step, all available behavior trees (BTs) 201 (hence also referred to as "predetermined" BTs), e.g., previously manually generated by experts, are represented in the form of a structured graph, i.e., a general behavior tree knowledge graph (BTKG) 202, which includes domain knowledge 203 and background knowledge 204. The BTKG 202 is denoted as G = [V, E], where V represents vertices (e.g., states or actions, such as grasping, and domain entities), and E represents edges (e.g., to represent the tree structure and relationships between vertices and their environment). In addition to information from the predetermined BTs 201, the BTKG 202 includes background and contextual knowledge about other relevant agents, behaviors, and sensors, as well as their hierarchical and interrelationships. Each predetermined BT 201 is included in the form of a subgraph G' = [V', E'], where V' ⊆ V and E' ⊆ E are the BTKG 202.

[0041] In a second preliminary step, feature and relation extraction from these subgraphs is performed to characterize their properties. These features include node labels (e.g., behavior tree name, validity, etc.), edge weights (to represent the strength of the connection between two nodes, if relevant), or any other relevant information. A knowledge graph embedding (KGE) method 205 is applied to convert this information into a lower dimensional space, i.e., an embedding hBTKG in the latent space 207, e.g., for each subgraph, the hBTKG of the node embeddings 207 is G’ We create a compact and informative embedding for each subgraph in the form of a cluster 206 denoted as

[0042] These two preparatory steps help to set up the automatic generation of the (new) behavior tree 215.

[0043] Applications of this automatic generation of behavior trees include, for example:

[0044] 1. A user provides a request to create a new behavior tree in the form of a natural language prompt 208, for example, "Drive more carefully when there are many people around." The given prompt 208 is validated to check the user's intent and to check whether it contains contradictions or has incomplete information.

[0045] 2. Terminology extraction 209 is applied to the prompt 208 using NLP (Natural Language Processing) techniques, such as LLMs (Large Scale Language Models). The extracted terms are further enriched with domain knowledge from the BTKG 202, thus forming a graph 210 with vertices V′ and edges E′, denoted as the prompt representation graph G′=[V′,E′], where V′′\V ≠ φ and E′′\E ≠ φ, which means that new concepts and / or relationships are present and may not be present in the original BTKG 202. For example, from the above example prompt 208, the following concepts (or terms) are extracted: “drive, carefully, people.” Furthermore, more nuanced concepts such as “there are many people around” and “driving carefully” are also extracted. Enrichment is performed by matching the extracted concepts (terms or expressions) to the BTKG 202. For example, in the BTKG 202, in addition to the definition of “people is a subclass of person,” there is a relationship to the “behavior” concept, which is how a robot should treat such kinds of entities. Furthermore, complex expressions such as "drive carefully" can also be defined in BTKG 202. This information is fetched from BTKG 202 and inserted into subgraph G". Also, for a concept (in the form of a term or expression of a term) in prompt 208, e.g., "drive", BTKG 202 can specify a similarity relationship with another concept, e.g., "run", which is then also added to G" (e.g., as an additional node for "run" and an edge between the nodes of "run" and "drive", with the edge specifying the similarity relationship (e.g., an edge node)). This kind of enrichment is performed for each concept in prompt 208, and thus additional relevant information is retrieved from BTKG 202. However, there may be concepts mentioned in prompt 208 that do not have a match in BTKG 202; these continue to be considered (without considering the graph structure, simply the textual description of a given BT201 in BTKG 202, i.e., any G' with G"). i to use them for similarity calculations (see steps below) based on

[0046] 3. A knowledge graph embedding method 211 (equivalent to or at least similar to the knowledge graph embedding method 205 to give corresponding / equivalent results) is then applied to the prompt representation graph 210, i.e., G″, to obtain h G” This graph embedding h G” is the embedding h of a given BT201 in the embedding space hBTKG. G’ This comparison is performed based on similarity, e.g., cosine similarity, and G’ vector of h G” For example, given the graph embedding h for one of the BT201, G” and the graph embedding h G’ The distance between G” vector and h G’ the minimum distance between vectors in h, i.e., the minimum distance among all such pairs. G” and h G’ The similarity score between G″ and each given BT201 is, for example, the inverse of this minimum distance.

[0047] 4. The resulting top N elements (closest embeddings h) are ranked according to their similarity score. G’ ~h G” A list L is generated containing the given BT201 with h G’ (Note again that ∇ is the embedding of the subgraph in which each given BT201 is included in BTKG202, not the given BT201 itself.) For simplicity, we assume these are equivalent, but there may be adaptations, for example, to fit given BT201s together when BTKG202 is formed.

[0048] 5. Then, one of the given BTs 201 in the list (or, more precisely, its corresponding subgraph in the BTKG 202) is selected, e.g., the one with the highest similarity score. From this selected behavior tree 213, a new BT 215 is generated in a BT generation step 214. During the BT generation step 214, new concepts (behaviors, agents) present in the prompt 208 are used to replace the old concepts in the selected BT 214.

[0049] 6. The new BT 215 can then be verified according to common predefined constraints and reasonable behavior.

[0050] 7. Optionally, the BTKG 202 can be updated by including a new BT 215.

[0051] In summary, according to various embodiments, a method is provided as shown in FIG.

[0052] FIG. 3 shows a flow diagram 300 illustrating a method for generating a behavior tree for controlling a robotic device, according to one embodiment.

[0053] At 301, a plurality of predefined behavior trees and background knowledge are combined into a behavior tree knowledge graph (ie, an "aggregated" (behavior tree knowledge) graph).

[0054] At 302, the behavior tree knowledge graph is represented in latent space by a knowledge graph embedding method (KGE, also referred to as knowledge representation learning (KRL)) in the form of multiple compound clusters of latent space elements (i.e., vectors in the latent space, each vector being, for example, an embedding of a respective node of the behavior tree knowledge graph), each cluster representing a subgraph of the behavior tree knowledge graph (e.g., corresponding to (or being a part of) a respective one of the given behavior trees).

[0055] At 303, a prompt representation graph is extracted from a (e.g., user) prompt (i.e., a text string) describing a desired behavior of the robotic device, where the terms in the prompt are represented as nodes and the relationships between the terms shown in the prompt are represented as edges. The terms (which may also form a representation) can describe entities ("person," "child," ...), actions ("drive," "stop," ...), and attributes and / or constraints ("carefully," "slowly," ...). The edges specify relationships (e.g., equivalence, order of actions, applied constraints, etc.) between the terms represented by the connected nodes. Note that a graph node may correspond to a single term, but in some cases may also correspond to multiple terms (i.e., a representation of multiple terms). Relationships between nodes that represent relationships to other nodes can be considered to represent relationships of the terms of the expression with one or more terms (e.g., expressions) represented by the other nodes.

[0056] At 304, the prompt representation graph is supplemented with one or more nodes and / or one or more edges in accordance with relationships specified by the behavior tree knowledge graph between the terms indicated in the prompt and / or one or more terms indicated in the prompt and other terms represented (e.g., as nodes) in the behavior tree knowledge graph (e.g., the prompt may specify "person" and the behavior tree knowledge graph (BTKG) may specify that "person" and "people" are equivalent, so a "people" node may be added to the prompt representation graph with an edge to the "person" node indicating the equivalence). Another example is when the prompt specifies both "person" and "people" and the BTKG specifies that these are equivalent, so an edge is added between the corresponding nodes in the prompt representation graph.

[0057] At 305, the enriched prompt representation graph is represented as an additional cluster (i.e., an additional “enriched graph cluster”) in the latent space by a knowledge graph embedding method.

[0058] At 306, the similarity of the additional cluster to the cluster of the plurality of clusters is determined.

[0059] At 307, one of the subgraphs of the behavior tree knowledge graph is selected according to the similarity between the cluster representing the subgraph and the additional cluster.

[0060] At 308, a behavior tree for controlling the robotic device is generated by adjusting (i.e., modifying) the selected subgraph according to the prompt and knowledge from the behavior tree knowledge graph. For example, this may involve examining the selected subgraph to see what is missing with respect to the prompt (e.g., which terms are missing), and then filling in the missing information from the behavior tree knowledge graph or replacing information in the subgraph with information from the prompt or the behavior tree knowledge graph (or even deleting nodes). For example, the prompt may say, "Drive more carefully when children are nearby," the selected subgraph may specify the behavior "Drive more carefully when people are nearby," and the BTKG may specify that a greater distance should be maintained for children than for the average person. The safety distance included in the subgraph (for people) may then be replaced with the safety distance included in the BTKG.

[0061] The method may further include validating the generated behavior tree against one or more predefined constraints to ensure that the generated behavior tree is consistent and does not violate any rules. For example, it is used for robot control only if it passes the validation. If not, it is, for example, modified or regenerated.

[0062] 3 can be used to determine a behavior tree, which can then be used to calculate control signals for controlling a technical system, such as a computer-controlled machine, such as a robot, a vehicle, a home appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system. Thus, according to various embodiments, a behavior tree for controlling a technical system can be determined from a text prompt, and the technical system can then be operated accordingly.

[0063] The generation of appropriate behavior trees based on user input (prompts) lowers the barrier to manual programming of new behaviors. In a behavior tree knowledge graph, information such as relationships, hierarchies, and axioms are represented in a well-defined structure. By leveraging this structure, time and effort can be saved in automatically creating behavior trees.

[0064] Various embodiments may receive and use image data (i.e., digital images) from various visual sensors (cameras), such as video, radar, LiDAR, ultrasonic, thermal imaging, motion, sonar, etc., as a basis for controlling a technical system according to the generated behavior tree (e.g., by determining the situation and / or state the technical system is in and deriving the action to be taken that the behavior tree specifies for this situation and / or state). The sensor data may be one-dimensional or multi-dimensional data, such as that generated in a manufacturing process.

[0065] When generating a behavior tree, only the embeddings generated by the neural network are needed, which turn out to encode high-level semantic concepts.

[0066] It should be noted that the method may be performed at least in part by a machine learning model (e.g., for extraction of the prompt expression graph), which may be trained, for example, using reinforcement learning (e.g., by providing feedback from the user on the generated behavior tree).

[0067] The method of FIG. 3 may be performed by one or more data processing devices (e.g., computers or microcontrollers) having one or more data processing units. The term “data processing unit” may be understood to mean any type of entity that enables processing of data or signals. For example, data or signals may be processed according to at least one (i.e., one or more) specific functions performed by the data processing unit. The data processing unit may include or be formed from an analog circuit, a digital circuit, a logic circuit, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or any combination thereof. Any other means for implementing the respective functions described in more detail herein may also be understood to include a data processing unit or a logic circuit. One or more of the method steps described in more detail herein may be performed (e.g., implemented) by a data processing unit through one or more specific functions performed by the data processing unit.

[0068] Thus, according to one embodiment, the method is computer-implemented.

Claims

1. A method for generating a behavior tree (215) for controlling a robotic device (101), comprising: Combining (301) a plurality of predetermined behavior trees (201) and background knowledge to form a behavior tree knowledge graph (202); Representing (302) the behavior tree knowledge graph (202) in a latent space (207) in the form of a plurality of multiple clusters (206) of latent space elements by a knowledge graph embedding method (205, 211), each cluster (206) representing a subgraph of the behavior tree knowledge graph (202); extracting (303) from a prompt (208) describing a desired behavior of the robotic device (101) a prompt representation graph (210) that represents terms in the prompt (208) as nodes and relationships between the terms shown in the prompt (208) as edges; supplementing (304) the prompt representation graph (210) with one or more nodes and / or one or more edges according to a plurality of terms indicated in the prompt (208) and / or relationships specified by the behavior tree knowledge graph (202) between one or more terms indicated in the prompt (208) and other terms represented in the behavior tree knowledge graph (202); Representing (305) the supplemented prompt representation graph (210) as an additional cluster (212) in the latent space (207) by the knowledge graph embedding method (205, 211); determining (306) a similarity of the additional cluster (212) to the cluster (206) of the plurality of clusters; selecting (307) one of the subgraphs of the behavior tree knowledge graph (202) according to the similarity of the cluster (206) representing the subgraph with the additional cluster (212); generating (308) the behavior tree (215) for controlling the robotic device (101) by adjusting the selected subgraph according to the prompt (208) and knowledge from the behavior tree knowledge graph (202); A method comprising:

2. 2. The method of claim 1, comprising extracting the prompt expression graph (210) by a machine learning model (i.e., a natural language processing model, e.g., a large-scale language model (LLM)) configured to operate on natural language text to which the prompt (208) is provided as input.

3. 3. The method of claim 1, wherein selecting the subgraph comprises selecting the subgraph of the behavior tree knowledge graph (202) in which a cluster (206) of the subgraph, in which the subgraph of the behavior tree knowledge graph (202) is represented, has the highest similarity to the additional cluster (212).

4. 4. The method of claim 1, further comprising generating the prompts (208) such that the prompt representation graph (210) includes one or more nodes and / or one or more edges for terms and / or relationships not included in the behavior tree knowledge graph (202).

5. The method of claim 1 , further comprising combining the generated behavior tree with the behavior tree knowledge graph (202).

6. A method for controlling a robotic device (101), comprising: Describing a desired behavior of the robotic device (101) in a text prompt (208); Generating a behavior tree (215) for controlling a robotic device (101) according to any one of claims 1 to 5; Controlling the robotic device (101) according to the generated behavior tree; A method comprising:

7. A data processing device (105) configured to perform the method of any one of claims 1 to 6.

8. A computer program comprising instructions that, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 6.

9. A computer-readable medium containing instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 6.

Images