Method for controlling a robot device

Abstract

According to various embodiments, a method for controlling a robot device is provided, comprising collecting state information about an agent located in an environment of the robot device, converting the state information about the agent into a textual state description, feeding the textual state description to a large-language model to generate a prediction of the agent's behavior, a future state of the agent and / or the environment, and generating a task plan for the robot device taking into account the prediction and control of the robot device according to the task plan.

Description

[0001] The present disclosure relates to methods for controlling a robotic device.

[0002] DE 10 2022 208 089 A1 discloses a method for training a control strategy for a robot device.

[0003] DE 11 2022 002 891 T5 discloses an information processing device comprising: a prediction unit that predicts, based on an action performed by an operation unit and step information that can identify a work step, a next action to be performed at a next step by the operation unit in response to a remote control, and a motion control unit that controls the operation unit to support remote control of the operation unit based on a relationship between the predicted next action, remote control operation information, and an object to be operated by the operation unit.

[0004] DE 10 2020 204 388 A1 discloses a method for controlling an agent.

[0005] DE 20 2019 103 862 U1 discloses a device for creating a strategy for an agent, so that when the agent, in particular a robot, carries out actions situationally based on the strategy, a predeterminable goal is achieved.

[0006] DE 20 2017 106 132 U1 discloses neural networks for selecting actions to be performed by a robot agent.

[0007] US 2023 / 0 359 789 A1 discloses varying embeddings and / or action models for automatically generating an action sentence in response to a natural language request.

[0008] In many applications, it is desirable for robots to be able to operate autonomously in environments where humans are present or (other) obstacles are present. To do this, they must be able to consider the possible behavior of dynamic objects or "agents" in their environment, such as humans, during control. Therefore, approaches are desirable that enable safe, automatic control of robotic devices in environments where agents, such as humans, are present, who move within and / or influence the state of the respective environment, such as moving objects.

[0009] It is therefore an object of the invention to provide a safe method for controlling a robot device. This object is achieved according to the claims.

[0010] According to various embodiments, a method for controlling a robot device is provided, comprising collecting state information about an agent located in an environment of the robot device, converting the state information about the agent into a textual state description, feeding the textual state description to a large language model for generating a prediction of a behavior of the agent, a future state of the agent and / or the environment, generating a task plan for the robot device taking into account the prediction and controlling the robot device according to the task plan.

[0011] The method described above enables safe and successful control of a robot device in a dynamic environment, i.e. an environment in which one or more agents act or interact with it, i.e. in which, for example, people are present. This enables, in particular, cooperation between one or more people and a robot. According to various embodiments, the control (in particular predictions of the state of the environment and task planning) for the robot device (e.g. in a confusing and dynamic environment) is carried out using one or more large language models (LLMs). LLMs have been shown to be able to react in the manner of a human (e.g. as a chatbot). They are therefore also suitable for predicting human behavior.

[0012] Various examples of implementation are given below.

[0013] Embodiment 1 is a method for controlling a robot device as described above.

[0014] Embodiment 2 is the method according to embodiment 1, comprising training the large language model to predict agent behavior from textual state descriptions.

[0015] The LLM can be trained, for example, using deep learning (e.g., by minimizing the difference between a target output from a training dataset and an actual output from the model using an optimizer). This adapts it to its use in a control system. The LLM can also be pre-trained.

[0016] The task plan is generated by supplying a textual target description, a textual environment description and a textual description of the state of the robot device to the or another LLM (“LLM Planner”).

[0017] This can be another LLM, or the same LLM can be prompted further (so that it takes the prediction into account). This allows a task schedule to be generated efficiently.

[0018] Embodiment 3 is a method according to embodiment 1 or 2, comprising generating the task plan by supplying the prediction (textual as output of the LLM) and a textual target description, a textual environment description and a textual description of the state of the robot device to a further LLM ("LLM Planner").

[0019] The separation into two LLMs enables fine-tuning or training of both LLMs for the respective task (e.g. through reinforcement learning, e.g. reinforcement learning from human feedback).

[0020] Embodiment 4 is the method according to embodiment 3, comprising training the further large language model to generate task plans.

[0021] The further LLM can be trained, for example, using reinforcement learning (e.g., using rewards received for controlling the robot device using the task plans generated for it). This adapts it to its use in control.

[0022] Embodiment 5 is the method according to one of the embodiments 1 to 4, comprising generating a scene graph from the collected state information and from further information about the environment of the robot device and generating the textual state description from the scene graph.

[0023] This enables an efficient generation of the textual state description, since, for example, relationships between agents and other objects (or spaces, etc.) represented by the scene graph (by edges) can be directly converted into text.

[0024] Embodiment 6 is a (e.g. robot) control device which is configured to carry out a method according to one of the embodiments 1 to 5.

[0025] Embodiment 7 is a computer program including instructions that, when executed by a processor, cause the processor to perform a method according to any one of embodiments 1 to 5.

[0026] Embodiment 8 is a computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method according to any one of embodiments 1 to 5.

[0027] In the drawings, like reference characters generally refer to the same parts throughout the several views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various aspects are described with reference to the following drawings. Fig. 1 shows a control scenario. Fig. 2 illustrates a data flow for controlling a robot device according to an embodiment in a respective control device. Fig. Figure 3 shows an example of a scene graph for a control scenario in which a mobile robot is to be navigated through a floor of a building. Fig. 4 illustrates the functionality of a prediction and task scheduling module according to one embodiment. Fig. 5 shows a flowchart illustrating a method for controlling a robot device according to an embodiment.

[0028] The following detailed description refers to the accompanying drawings, which, by way of illustration, show specific details and aspects of this disclosure in which the invention may be practiced. Other aspects may be utilized, and structural, logical, and electrical changes may be made without departing from the scope of the invention. The various aspects of this disclosure are not necessarily mutually exclusive, as some aspects of this disclosure may be combined with one or more other aspects of this disclosure to form new aspects.

[0029] Various examples are described in more detail below.

[0030] Fig. 1 shows a control scenario.

[0031] A mobile robot 100 is located in an environment 101 (e.g., in a factory hall or on a construction site). The robot 100 has a starting position 102 and is to reach a target position 103. The environment 101 contains obstacles 104 that the robot 100 is to avoid. For example, they may not be passed by the robot 100 (e.g., machines, walls, or trees) or should be avoided because the robot would damage or injure them (e.g., people, e.g., in a factory hall where the robot 100 is used to transport objects).

[0032] The robot 100 has a control device 105 (which can also be spatially separated from the robot 100, ie the robot 100 can be remotely controlled). In the example scenario of Fig. 1, the goal is for the controller 105 to control the robot 100 to navigate through the environment 101 from the starting position 102 to the target position 103. The robot 100 is, for example, an autonomous vehicle, but may also be a robot with legs or tracks or another type of propulsion system (e.g., a deep-sea or Mars rover).

[0033] Furthermore, the embodiments are not limited to the scenario that a robot (as a whole) is to be moved between the positions 102, 103, but can also be used for controlling a robot arm whose end effector is to be moved between the positions 102, 103 (without encountering obstacles 104), etc.

[0034] Accordingly, terms such as robot, vehicle, machine, etc. are used below as examples of the "object" or agent to be controlled, i.e., the computer-controlled technical system (e.g., the machine). The approaches described here can be applied to various types of computer-controlled machines such as robots or vehicles, and others (and possibly their environment). The general term "robot device" is also used below for all types of technical systems (which are mobile and / or have one or more movable components) that can be controlled using the approaches described below.

[0035] Ideally, the controller 105 has learned a control strategy that enables it to successfully control the robot 100 for any scenario (i.e., environments, start and finish positions) in specific scenarios that the controller 105 has not yet encountered (from the start position 102 to the finish position 103 without encountering obstacles 104).

[0036] For this purpose, the control device 105 is suitably trained. For such training, the scenario (in particular, the environment 101) can also be simulated, but in use, it is usually real. However, the robot 100 and the environment can also be simulated in use (e.g., if a simulation of vehicles (according to the control strategy) is used to test a (different) control strategy of an autonomous vehicle).

[0037] According to various embodiments, a controller for a robot device (e.g., controller 105) is provided that uses a large language model (LLM) to predict the behavior of an agent (or a result of the behavior) in the environment of the robot device. These predictions can then be generated by a task scheduling module of the controller for task scheduling. For example, human behavior in an indoor environment is predicted, e.g., where a human moves to avoid a collision between a human and a robot.

[0038] In recent years, with the tremendous advances in natural language processing (NLP) and generative artificial intelligence (AI) research, various powerful LLMs have been developed that are capable of producing human-like text.

[0039] According to various embodiments, one or more LLMs are used to predict future activities or behaviors of an agent (e.g., a human). This allows, for example, the robot 100 to navigate through a confusing environment with (possibly many) people, i.e., human "obstacles" 104, since it has been shown that LLMs can learn "common sense" and are thus also suitable for predicting human behavior. One or more people are used as an example below, but the behavior of other "agents," such as animals (e.g., for the navigation of a robot in a cowshed), could also be predicted in a similar way.

[0040] According to various embodiments, for example, a prediction and planning module is provided for a control device for a mobile robot in a human-crowded indoor environment, which receives the following inputs (information): • A description of the type of environment • a description of the task • Sensor data about the current position of the robot • a scene graph with all objects in the environment and their relationships, e.g. including one or more agents (e.g. people).

[0041] The prediction and task planning module uses one or more LLMs to predict human behavior and create a (high-level) task plan to fulfill the user-defined goal (i.e., task) while minimizing certain costs (e.g., travel distance, human-robot interaction costs, etc.).

[0042] Fig. 2 illustrates a data flow for controlling a robot device (e.g., an autonomous system such as a self-driving robot) according to an embodiment in a respective control device.

[0043] Perception 201 provides information about the environment of the robot device (according to various embodiments in the form of a scene graph). From this information, a prediction 202 is made about a future state of the environment. This prediction 202 is used for task planning 203. If the execution of the task is planned (e.g., first move to point A, wait for 5 seconds, then move to point B), motion planning 204 is carried out (i.e., the trajectory(s) for executing the respective movement(s) according to the task plan are determined). Control 205 of the robot device is then carried out in accordance with the motion planning.

[0044] The functionality of task planning 203 and motion planning 204 can be combined into the functionality of "planning," which can be the functionality of a planning module 206. Task planning 203 (which can be viewed as "high-level" planning) can therefore be considered (at least primarily) part of the planning level of the respective autonomous system. In the present description, the functionality of prediction 202 and task planning 203 are combined into the functionality of a "prediction and task planning module" 207.

[0045] According to various embodiments, the perception 201 provides information in the form of a scene graph that contains all objects (including static and dynamic objects, ie, robots and humans) in the environment as nodes and the relationships between the objects as edges.

[0046] Fig. 3 shows an example of a scene graph 300 for a control scenario in which a mobile robot is to be navigated through a floor of a building.

[0047] In this example, the scene graph 300 has the form of a tree. The root 301 of the scene graph is assigned to the floor. For each room 302 of the floor, the scene graph 300 contains an inner node 303 of the scene graph 300 assigned to the respective room (i.e., neither root nor leaf). Each leaf (i.e., each leaf node) 304 of the scene graph 300 is assigned to a respective object 305 (or a respective location in the room) and is connected to the node 300 assigned to the room in which the respective object 305 is located.

[0048] Each node (root, inner nodes, and leaves) can have one or more properties (also called attributes). These can be binary properties, e.g., for an inner node, "object_in_room=true," "robot_in_room=true," or for a leaf, "is_to_open=false," "is_pickable=true," and other non-binary attributes, such as for a leaf, "category=bed," "weight=10kg," or for an inner node, "room_type=storage room," the position and / or speed of the object, etc., or for the root, "floor_type=basement." One or more of the objects are agents, for example, humans in this example (they are identified, for example, by the binary property "is_human=true" or more generally, "is_dynamic_object=true"). The robot itself can also be represented by a leaf.Nodes that are assigned to objects (object nodes) can be human nodes, robot nodes or environment nodes, depending on what they represent (people, robots or other objects in the environment).

[0049] The scene graph 300 can also have additional levels. For example, rooms can first be divided into locations (each of which is assigned to a node that is connected to the respective room node) and objects can then be specified for these locations. Relations can also exist between objects (e.g., a box is on a shelf). An edge can then be created to connect these two object nodes and describe the relationship. If an agent is near (at Euclidean distance) an object or has a direct relationship to it (e.g., a person is sitting on a chair), the two corresponding nodes can also be connected with an edge (however, there are no edges between two agent nodes, for example). The scene graph may then no longer have the shape of a tree.

[0050] According to various embodiments, an LLM is used for prediction 202 to predict human behavior. The predictions are provided as input to task planner 303, which then outputs a task plan using another LLM for motion planner 204 to calculate a motion trajectory for the robot. Controller 205 invokes the appropriate actuators (e.g., for moving arms, wheels, etc.) to follow the motion trajectory.

[0051] For example, this is used in a factory scenario for the assembly of an e-bike, where the robot is required to collect 100 different parts of the e-bike, e.g. handlebar, rear wheel, frame, battery and saddle, from different locations and bring them one after the other to the assembly area without driving through human-occupied areas (i.e. without colliding with or obstructing people).

[0052] A task scheduling problem is given by a tuple (O, P, A, T, C, I, G). O is the set of all basic objects of the problem. P is a set of properties (or attributes), each defined over one or more objects (binary properties are subclasses of attributes that have Boolean values). A is a finite set of actions that operate on object tuples. T is a state transition model, and C denotes the costs of state transitions. I is an initial state, and G denotes one or more goal states. A state is an assignment of values ​​to all possible properties of the objects. It is described, for example, by a scene graph, as described above (at least partially; e.g., information about the robot state (such as joint positions, etc.) can also be present as additional information).

[0053] A language model (LM) can be viewed as a distribution over sequences of word tokens. One approach is to model the conditional distribution p(w k lw 1:k ) about w k , the next token at the k-th position, given a sequence of previous input tokens w 1:k . Newer LMs are based on the Transformer architecture, significantly enlarging the LMs and resulting in LLMs (large language models), i.e., large language models that typically have tens or hundreds of billions of parameters and are capable of generating human-like text or programming code.

[0054] Fig. 4 illustrates the functionality of a prediction and task scheduling module 400 according to one embodiment.

[0055] The prediction and task scheduling module 400 includes an “SG2NL” module 401 that converts a scene graph (SG) 402 into a natural language description of the scene that the scene graph describes (i.e., the state) (e.g., a description in words of the properties or states of the objects associated with the leaves), an LLM predictor “LLM Präd “ 403, which predicts human behavior, and an LLM planner “LLM Plan “ 404, which creates a task plan for the robot.

[0056] The inputs of the prediction and task scheduling module 400 are the scene graph 402 and a user-defined natural language goal description G NL , and its output is a generated task plan π. As explained above, the scene graph 402 consists of several nodes and edges, as in Fig. 3: Some of the nodes (e.g. the leaves) represent the objects and the agents (e.g. a human and also the robot itself) in the environment, nodes (including the leaves) can have several attributes, e.g. position, orientation, actions of the human or the robot, etc.

[0057] Algorithm 1 gives an example of creating a task schedule from the scene graph for a goal G NL in pseudocode (using the usual English keywords such as while, do and end).

[0058] In line 2 of Algorithm 1, the human states S H (of one or more people), the environmental conditions S E and the robot state S R determined from the scene graph SG, where S H and S E two sets of attributes of human nodes and environment nodes (for other objects and also rooms) and S Ris the set of attributes of the robot node.

[0059] In line 3, the scene graph 402 is parsed by the SG2NL module 401. The SG2NL module 401 converts the scene graph 402 into an NL description of the human states S H NL, the environmental conditions S E NL of the robot state S R NL where S H NL , S E NL and S R NL textual sentences in natural language.

[0060] Then S H NLan handed over the LLM predictor 403, which predicts future human behavior in natural language Ŝ H NL generated (line 4), which appends the predicted human behavior to the current human state. An example output prediction could be: "The person is standing in front of a refrigerator; he will open the refrigerator soon."

[0061] The description of the environmental conditions S ENL is used together with the target description G NL , the robot state in natural language S R NL and predicting the human condition in natural language Ŝ H NL passed to the LLM planner 404, which generates a task plan π (e.g. a sequence of steps) that leads the robot to the given task goal (line 5).

[0062] For example, a task plan for placing an apple in a refrigerator might look like this: "Go to the refrigerator, wait until the person leaves the refrigerator area, open the refrigerator, place the apple in the refrigerator, close the refrigerator."

[0063] The LLM predictor 404 and the LLM planner 403 may be the same pre-trained LLM, but may be given different prompts depending on the situation.

[0064] The task plan π is then executed by the robot and interacts with the environment 405, and the next human states S' H , environmental conditions S' E and the next robot state S' R are observed (line 6). The scene graph is then updated accordingly based on the state changes (line 7). This process is repeated until the user-defined goal is reached.

[0065] Algorithm 2 gives an example of the conversion of the scene graph SG into natural language by the SG2NL module (here, too, the usual English keywords such as for, do, end, if, else, then are used).

[0066] First, the NL descriptions of human states S H NL , the environmental conditions S E NL and the robot state S R NLinitialized as empty (line 1). From lines 2 to 10, all nodes in the scene graph are traversed in a loop. For each node, all edges (i.e., relationships) and neighboring nodes (i.e., neighboring objects and people) are accessed (lines 3-4). If a node is a human node, the current state of the human (position, orientation, and action) and its relationships to all neighboring objects and people are converted to text and stored in S H NL added (lines 5-7). If the node is an environment node that represents another object, the position of the object and its relationships to all neighboring objects (including people) are stored in S E NL (lines 8-10). Otherwise, the relationships to all neighboring objects and people of the robot node are set to S R NL added (line 12).

[0067] In summary, according to various embodiments, a method is provided as described in Fig. 5 shown.

[0068] Fig. 5 shows a flowchart 500 illustrating a method for controlling a robotic device according to an embodiment.

[0069] In 501, state information about an agent (e.g., human or animal, ie, a “dynamic object” that moves in the environment and possibly influences the environment, e.g., a human can remove another object from the environment or carry it somewhere else) that is located in an environment of the robot device is collected.

[0070] In 502, the state information about the agent is converted into a textual state description.

[0071] At 503, the textual state description is fed to an LLM ("LLM predictor") to generate a prediction of a behavior of the agent, a future state of the agent, and / or the environment (the LLM can, for example, output the agent's behavior or the result of the behavior with respect to the state of the environment (including the agent itself)). In other words, the textual state description is fed to a (trained) LLM to generate the prediction.

[0072] In 504, a task plan for the robot device is generated taking the prediction into account.

[0073] In 505, the robot device is controlled according to the task plan.

[0074] The procedure of Fig.5 may be performed by one or more computers having one or more data processing units. The term "data processing unit" may be understood as any type of entity that enables the processing of data or signals. The data or signals may, for example, be handled according to at least one (i.e., one or more than one) specific function performed by the data processing unit. A data processing unit may include or be formed from an analog circuit, a digital circuit, a logic circuit, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable gate array (FPGA) integrated circuit, or any combination thereof.Any other way of implementing the respective functions described in more detail herein may also be understood as a data processing unit or logic circuit arrangement. One or more of the method steps described in detail herein may be executed (e.g., implemented) by a data processing unit through one or more specific functions performed by the data processing unit.

[0075] According to various embodiments, the method is therefore particularly computer-implemented.

[0076] Controlling the robot device according to the task plan involves generating one or more control signals for the robot device. The term "robot device" can be understood as referring to any technical system (with a mechanical part whose movement is controlled), such as a computer-controlled machine, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system.

[0077] Various embodiments may receive and use sensor signals from various sensors, such as video, radar, LiDAR, ultrasound, motion, thermal imaging, etc., for example, to obtain sensor data for state information. The sensor data may be processed to obtain the state information (i.e., for perception). This may include classifying the sensor data or performing semantic segmentation on the sensor data, for example, to detect the presence of objects (in the environment in which the sensor data was obtained).

Claims

[1] A method for controlling a robot device (100), comprising: Collecting (501) state information about an agent located in an environment (101) of the robot device (100); Converting (502) the state information about the agent into a textual state description; Feeding (503) the textual state description to a large language model (403) to generate a prediction (202) of a behavior of the agent, a future state of the agent and / or the environment (101); Generating (504) a task plan for the robot device (100) taking into account the prediction (202), wherein the task plan is generated (504) by supplying a textual target description, a textual environment description, and a textual description of the state of the robot device (100) to the or a further large language model (404); and controlling (505) the robot device (100) according to the task plan. [2] The method of claim 1, comprising training the large language model (403) to predict agent behavior from textual state descriptions. [3] Method according to claim 1 or 2, comprising generating the task plan by supplying the prediction and a textual target description, a textual environment description and a textual description of the state of the robot device (100) to a further large language model (404). [4] The method of claim 3, comprising training the further large language model (404) to generate task plans. [5] Method according to one of claims 1 to 4, comprising generating a scene graph (402) from the collected state information and from further information about the environment (101) of the robot device (100) and generating the textual state description from the scene graph. [6] Control device which is arranged to carry out a method according to one of claims 1 to 5. [7] A computer program comprising instructions which, when executed by a processor, cause the processor to perform a method according to any one of claims 1 to 5. [8] A computer-readable medium storing instructions which, when executed by a processor, cause the processor to perform a method according to any one of claims 1 to 5.

Images