Methods for robots to learn skills from user demonstrations and robots, data structure and computer program for autonomous task execution

Abstract

Method for robot learning skills (120, 122, 124, 126, 128, 130) from user demonstration to autonomously perform tasks using a robot (100), characterized in that - a movement sequence is demonstrated, - the movement sequence is discretized and states are determined, - the states are transferred to a feature space, - the movement sequence is segmented into skills, taking into account the states and the feature space.

Description

[0001] The invention relates to a method for robots learning skills from user demonstrations in order to autonomously perform tasks using a robot. Furthermore, the invention relates to a robot for autonomously performing tasks. Furthermore, the invention relates to a data structure for autonomously performing tasks using a robot. Furthermore, the invention relates to a computer program for autonomously performing tasks using a robot.

[0002] The publication "C. Willibald, T. Eiband, and D. Lee, Collaborative programming of conditional robot tasks, in IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 5402-5409." concerns the collaborative programming of conditional robot tasks. This publication proposes an active learning system in which the robot and the user collaborate to program a complex task incrementally. Starting from a basic model, to expand the robot's task knowledge over time as new situations require additional skills, an online anomaly detection algorithm automatically identifies new situations during task execution by monitoring the deviation between measured and commanded sensor values. The robot then triggers a learning phase in which the user decides whether to refine an existing skill or demonstrate a new skill.The different skills of a task are encoded in separate probabilistic models and structured in a high-level graph, which ensures robust execution and successful transitions between skills. The online algorithm scales the sensitivity to sensor deviations based on the current time step; time-invariant anomaly detection is therefore not possible. Furthermore, absolute sensor value trajectories are used as a reference value, which does not allow for generalization or anomaly detection in generalized execution. To ensure that a subsequent skill is suitable for resolving an existing anomaly, a task graph is used that restricts subsequent skills and the time of a transition. This means, however, that skills cannot be reused and decisions orForks in the task graph are only possible in so-called decision states at a specific time. In the event of an anomaly, only a decision state is inserted into the task graph, which can be viewed as programming an if statement at a specific time step.

[0003] From the document DE 10 2021 118 885 A1 a system is known comprising one or more processors coupled to computer-readable media, wherein the computer-readable media store executable instructions which, as a result of execution by the one or more processors, cause the system to perform the following steps: determining, from a three-dimensional image of a limb holding an object, a pose of the limb and an object pose; determining a set of grasping poses that enable a robot gripper to grasp the object; selecting, from the set of grasping poses, a grasping pose that does not interfere with the limb; and executing the grasping pose.

[0004] From the document DE 10 2020 207 085 A1, a method is known for controlling a robot, comprising: providing demonstrations for performing each of a plurality of capabilities; training a robot trajectory model for each capability from the demonstrations, each trajectory model being a hidden semi-Markov model with one or more initial states and one or more final states; training a precondition model for each capability from the demonstrations, which comprises, for each initial state of the robot trajectory model of the capability, a probability distribution of robot configurations before executing the capability, and a final condition model for each capability, which comprises, for each final state of the robot trajectory model of the capability, a probability distribution of robot configurations after executing the capability; receiving a description of a task,wherein the task comprises performing the capabilities of the multiple capabilities in a sequence and / or in branches; generating a composite robot trajectory model. If two capabilities are to be performed in a sequence in the task,The composite robot trajectory model is generated by cascading the robot trajectory models of the skills by incorporating the states of the trajectory models of the two skills into the composite robot trajectory model and calculating a transition probability between each final state of the trajectory model of the first skill of the two skills and each initial state of the trajectory model of the second skill of the two skills as a function of the similarity between the probability distribution of the final condition model of the first skill for the first state of the first skill and the probability distribution of the initial model of the second skill for the initial state of the second skill. If two skills are executed in branches in the task,The composite robot trajectory model is generated by combining the robot trajectory models of the capabilities by incorporating the states of the trajectory models of the two capabilities into the composite robot trajectory model and setting the transition probability between states of the first capability and states of the second capability to zero; and controlling the robot according to the composite robot trajectory model to perform the task.

[0005] From the document DE 10 2020 127 508 B4 a computer-implemented method is known, comprising: obtaining touch sensor information from a robot limb manipulating an object in the real world; generating a plurality of simulations of the robot limb manipulating the object, wherein the individual simulations have different poses for the object; determining a plurality of costs, wherein each cost of the plurality of costs corresponds to a respective simulation of the plurality of simulations and each cost of the plurality of costs is based at least in part on differences between the touch sensor information and simulated touch sensor information generated by the respective simulation of the plurality of simulations; identifying an individual simulation of the plurality of simulations based at least in part on the cost;Determining a pose of the object in the real world based at least in part on a pose of the object in the identified individual simulation; and providing the pose of the object to a robot control system that controls a robot to perform a task based at least in part on the pose of the object.

[0006] From the document DE 11 2020 005 156 T5 a system is known, comprising: one or more circuits for enabling a robot gripping mechanism to grip objects having a first volume or less than the first volume, based, at least in part, on one or more neural networks trained only with one or more objects having a volume corresponding to the first volume; and one or more memories for storing the one or more neural networks.

[0007] From the document DE 10 2014 108 287 B4 a method is known for training a robot to autonomously perform a predetermined robot task which requires the application of a linear force and a torque to an object by an end effector of the robot, the method comprising: moving the end effector through a plurality of states of the predetermined robot task to thereby demonstrate the predetermined robot task to the robot in a set of n training demonstrations; measuring a set of training data, which comprises measuring at least the linear force and the torque with the aid of a force-torque sensor while moving the end effector through the plurality of states of the predetermined robot task;Key features are extracted from the measured set of training data using a controller, which comprises segmenting the measured set of training data into a time sequence of control primitives and identifying transitions between adjacent segments of the time sequence; measuring at least the linear force and torque using the force-torque sensor as online / real-time data during subsequent autonomous execution of the demonstrated robot task by the robot; detecting transitions using the controller during subsequent autonomous execution of the demonstrated robot task by the robot; and automatically switching between a plurality of different control modes in response to the detected transitions.

[0008] From document US 2021 / 0387330 A1, a robot is known that includes an RL agent configured to learn a strategy for maximizing the cumulative reward of a task by determining one or more features that are minimally correlated with each other. The features are then used as pseudo-rewards called feature rewards, where each feature reward corresponds to an option policy or capability that the RL agent learns to maximize. In one example, the RL agent is configured to select the most relevant features to learn corresponding option strategies. The RL agent is configured to learn, for each of the selected features, the respective option policy that maximizes the respective feature reward.Using the learned option strategies, the RL agent is configured to learn a new (second) strategy for a new (second) task, which can choose from any of the learned option strategies or actions available to the RL agent.

[0009] The invention is based on the object of structurally and / or functionally improving a method mentioned above. Furthermore, the invention is based on the object of structurally and / or functionally improving a robot mentioned above. Furthermore, the invention is based on the object of structurally and / or functionally improving a data structure mentioned above. Furthermore, the invention is based on the object of structurally and / or functionally improving a computer program mentioned above.

[0010] The problem is solved by a method having the features of claim 1. Furthermore, the problem is solved by a robot having the features of claim 16. Furthermore, the problem is solved by a data structure having the features of claim 17. Furthermore, the problem is solved by a computer program having the features of claim 18. Advantageous embodiments and / or further developments are the subject of the subclaims.

[0011] The method can be designed to transfer the skills required to perform tasks from a user to a robot. The method can include learning from demonstration (LfD). The method can be designed to enable a robot to perform tasks autonomously. The method can be implemented as an algorithm.

[0012] A task can also be referred to as a "task." "Skill" in this context refers in particular to an ability required to perform a task and can also be referred to as a "skill." Skills can be executed sequentially when performing a task. The execution of a task can be directed toward achieving a task goal. A task goal can be specified. "Robot learning" in this context refers in particular to a robot learning how to perform a task. "User demonstration" in this context refers in particular to a user demonstrating how to perform a task.

[0013] A movement sequence can begin from a start configuration. A movement sequence can begin after a start configuration. A movement sequence can be directed towards achieving a task goal. A movement sequence can comprise a process until a task goal is achieved. A movement sequence can end with the achievement of a task goal. A movement sequence can end before a task goal is achieved. A movement sequence can be a movement sequence demonstrated by a user. A demonstration of a movement sequence can be observed and / or recorded. A movement sequence can be an observed and / or recorded movement sequence.

[0014] A movement sequence can be recorded as a trajectory. The trajectory can be a trajectory of a robot's end effector. The trajectory can be recorded in Euclidean space. The trajectory can be converted into a sequence of observations. Each observation can be described by a state-action tuple. The movement sequence can be discretized after receiving a user demonstration. "State" can also be referred to as "condition" and, in this case, particularly refers to a coordinate-related state of a robot. "Feature" in this case particularly refers to an externally effective, quantifiable, coordinate-free property of a process or a state in a system that comprises a robot and a robot environment. "Feature space" in this case particularly refers to a structured set of features. Segmentation can include identifying and parameterizing individual skills."Task graph" can also be referred to as a "task graph" and, in this context, refers specifically to a topological structure that represents a set of skills along with the connections between these skills. To make a user demonstration accessible to a robot, it may be necessary to encode a demonstrated movement sequence. Skills can encode a demonstrated movement sequence at a low level. A task graph can encode a demonstrated movement sequence at a high level.

[0015] Skills can be described by individual subgoals and feature constraints. Each skill can be described by an individual subgoal and individual feature constraints. "Subgoal" can also be referred to as a "sub-goal" or "partial goal." A subgoal can encode an intention of a skill. Subgoals can be derived, determined, or calculated from a demonstrated movement sequence. Subgoals can be saved. "Feature constraints" here specifically refers to limits for feature values. Feature constraints can be derived, determined, or calculated from a demonstrated movement sequence. Feature constraints can be saved. Feature constraints can be used as expected limits for feature values. Feature constraints can define ranges for feature values. Feature constraints can be multimodal or multidimensional.

[0016] Deriving the coding of a task from the user demonstration. Derivation of the individual feature constraints of each skill, which enables the detection of multimodal anomalies even when generalizing the skill to new constellations.

[0017] Subgoals can be derived using intention recognition. "Intention recognition" can also be referred to as "intention recognition." The intention of a skill can be to achieve its specific subgoal. Subgoals can be derived based on inverse reinforcement learning, specifically Bayesian nonparametric inverse reinforcement learning. "Inverse reinforcement learning" can also be referred to as "inverse reinforcement learning." "Bayesian nonparametric inverse reinforcement learning" can also be referred to as "Bayesian nonparametric inverse reinforcement learning."

[0018] Feature constraints can be derived using model-based clustering. For this purpose, states obtained after discretizing a demonstrated motion sequence can be clustered in the feature space. Feature constraints can be derived taking into account a mixture distribution, in particular using a Gaussian mixture model, in particular using a nonparametric Bayesian Gaussian mixture model. A "Gaussian mixture model" can also be referred to as a "Gaussian mixture model." A "Bayesian nonparametric Gaussian mixture model" can also be referred to as a "Bayesian nonparametric Gaussian mixture model."

[0019] The method can combine intent recognition and feature clustering. This allows a low-level encoding of a movement sequence to be derived from a user demonstration. The method can combine aspects of subgoal-driven action generation and feature clustering. Subgoals and feature constraints can be derived using a joint model. The joint model can combine aspects of a Bayesian nonparametric Gaussian Mixture Model (BNGMM) and aspects of Bayesian nonparametric inverse reinforcement learning.

[0020] When a robot autonomously executes a task, skills from the task graph can be executed. During autonomous execution of a task, the skill can be checked to see whether the subgoal of the executed skill has been achieved and / or the feature constraints can be used to detect anomalies.

[0021] To detect anomalies during the execution of a skill, it can be checked whether current feature values ​​lie within an expected feature constraint range. Current feature values ​​can be recorded during autonomous execution of a task. An expected feature constraint range can be determined by taking into account feature constraints previously derived and stored from a demonstrated movement sequence.

[0022] To check whether current feature values ​​lie within an expected range of feature constraints, a classification probability can be determined for a current feature value combination and compared with a skill-specific threshold. The threshold can be predetermined based on a user demonstration and / or updated based on the successful execution of skills.

[0023] If, during autonomous execution of a task, the subgoal of an executed skill is achieved and / or no anomaly is detected, a transition can be made from the executed skill to the next skill provided in the task graph. If, during autonomous execution of a task, the subgoal of an executed skill is not achieved and / or an anomaly is detected, a suitable skill for achieving the subgoal and / or circumventing the anomaly can be searched for in the task graph. If a suitable skill for achieving the subgoal and / or circumventing the anomaly is found in the task graph, a transition to the suitable skill can be made. If no suitable skill for achieving the subgoal and / or circumventing the anomaly is found in the task graph, the user can be given a demonstration of a further movement sequence for achieving the subgoal and / or circumventing the anomaly.

[0024] The further movement sequence can be processed in the same way as the initially demonstrated movement sequence. In particular, the demonstrated further movement sequence can be discretized and further states can be determined. The further states can be transferred to the feature space in order to expand the feature space. The further movement sequence can be segmented into further skills, taking into account the further states and the expanded feature space. The task graph can be expanded with the obtained further skills. Furthermore, with regard to the further movement sequence and the processing of the further movement sequence, reference is also made to the above description of the initially demonstrated movement sequence. All other features of the method according to the invention as described above can also be applied to the further movement sequence.

[0025] A task graph generated based on a demonstrated movement sequence can also be referred to as an initial task graph. An initial task graph can be incrementally expanded based on further demonstrated movement sequences, if a task requires it.

[0026] The method can be executed using the robot, the data structure, and / or the computer program. The method can be implemented in the robot, the data structure, and / or the computer program.

[0027] The robot can be designed to autonomously perform tasks. The robot can be designed to assist a user in performing tasks. The robot can be designed for learning through user demonstration. The robot can be an autonomous mobile robot. The robot can be an assistance robot, a humanoid robot, a personal robot, or a service robot. The robot can have kinematics. The robot can have joints and limbs. The robot can have actuators and sensors. The robot can have a user interface. The user interface can be designed as an input and / or output device. The user interface can be embodied as a touchscreen. The robot can have a control device. The control device can have at least one processor, at least one main memory, at least one data memory, and / or at least one signal interface.The control device and the actuators, the sensors, and / or the input and / or output device can be connected to one another for signal transmission. The robot can be a real robot. The robot can be a simulated robot or a robot simulation. The robot can be designed to execute the method and / or steps of the method. The robot, in particular the control device, can be designed to execute the computer program and / or steps of the computer program. The robot, in particular the control device, can be designed to represent and / or use the data structure.

[0028] The data structure, in particular the task graph, can be designed to store and / or organize data. The data structure, in particular the task graph, can be designed to arrange and / or link data in a predetermined manner. The data structure, in particular the task graph, can be designed to overcome unidirectional linking. The data structure can be characterized by operations on this data that enable and implement access and management. The data structure, in particular the task graph, can be characterized by the operations "insert," "delete," and "find" skills. The data structure, in particular the task graph, can be or comprise an abstract structure that represents a set of skills together with the connections between these skills.The data structure, in particular the task graph, can be designed to execute the method and / or steps of the method. The data structure, in particular the task graph, can be represented and / or used in the robot, in particular in the control device, and / or in the computer program.

[0029] The computer program can be installed and / or executed on a control device of a robot. The computer program can be present as installed and / or executed program code. The computer program can be present on a data carrier as an installable and / or executable program file. The computer program can be designed to be loaded into a working memory of a control device of a robot and / or executed with the aid of at least one processor. The computer program can be designed to carry out the method and / or steps of the method. The computer program can be designed to support the execution of the method and / or steps of the method. The computer program can be designed to represent and / or use the data structure.

[0030] In summary and in other words, the invention thus results, among other things, in multi-stage task learning from demonstrations for autonomous robot manipulation.

[0031] Individual skills can be learned from a continuous demonstration of a complete task. The individual skills can be topologically structured at a higher level of abstraction. The individual skills can be structured in a task graph (TG). The TG can be incrementally expanded if the robot requires additional skills to complete the task.

[0032] The logical steps, also known as skills, can be identified from a single, coherent demonstration. To do this, the demonstration can be divided into discrete states and their feature values ​​calculated. Using a segmentation algorithm, the demonstration can then be divided into individual skills, where each skill can be characterized by a subgoal and feature constraints. The individual skills can be stored in the TG according to the observed sequence. If a deviation in the measured feature values ​​is detected during autonomous execution on a robot, the system can attempt to find a suitable skill in the TG to resolve the anomaly. If the system does not yet know of a suitable skill, the user can be prompted to demonstrate a solution that can be used as a resolution strategy if the situation occurs repeatedly.This allows the system to incrementally learn new decisions and resolution strategies for anomalies. The newly demonstrated strategy can be segmented, and the learned skills can be linked in the TG to the skill during whose execution the anomaly was discovered.

[0033] To segment a demonstration into individual skills, a combination of feature constraints and the skill's intention can be used. The segmentation algorithm can combine intention detection with feature clustering to learn an inherent structure in the acquired data, requiring no prior knowledge (training data) other than the user's demonstration of the task. The intention of a skill can be described by a subgoal, a specific state during the demonstration. It can be assumed that a skill serves to achieve a specific subgoal, e.g., an end-effector pose for grasping an object or a covered area during a surface treatment. To infer the subgoal from the observed actions of the demonstration, an approach based on inverse reinforcement learning can be used.This can be combined in a joint model with a Bayesian-nonparametric Gaussian Mixture Model for feature clustering. Feature clustering can use commonalities between the states during the demonstration in feature space to generate clusters (mixtures). Each cluster in this approach can correspond to a skill. From the demonstration, the number of skills, the subgoal of each skill, and the expected feature region for each skill can be determined.

[0034] The individual feature constraints of a skill can be used for unsupervised online anomaly detection during task execution under changing environmental conditions. The feature commonalities used for clustering can be used during the execution of the learned task with a robot to individually check for each skill whether the measured feature values ​​match the expected ones. Features can be defined relative to objects in the environment and thus can also be used for anomaly detection during generalized execution, with changing configurations of objects in the environment. The expected feature region of a skill can be modeled as a multidimensional Gaussian distribution, whereby the mean and covariance of each feature can be learned individually. Any sensor modality, such as force, position, orientation, sound, etc., can be used as a feature., can be used. If the classification probability for a combination of measured features falls below a certain threshold during execution, an anomaly can be detected. The threshold can be chosen as the lowest observed probability during the demonstration. To make the system more robust, the threshold can also be updated with classification probabilities observed during the successful execution of a skill with the robot.

[0035] Flexible skill transitions allow for successful progress to be monitored, and in the event of an anomaly, a decision to solve the task can be made autonomously. Two different transitions between skills can exist: A) The transition to a subsequent skill occurs when the subgoal of the previous skill has been achieved. This ensures that all subgoals are met to complete the task. B) If an anomaly is detected, the task switches to a skill that is linked to the current skill in the task graph and serves to resolve this anomaly. This allows the anomaly to be resolved and the intended execution to continue. During execution, flexible transitions between skills can be enabled at any time step. The intention of each skill can be determined by the subgoal, thus, a trajectory for achieving the subgoal of the next skill can be generated from any robot configuration.

[0036] The invention provides an integrated approach for low- and high-level learning of skills for task execution. This approach can efficiently segment a demonstrated movement sequence into skills and extract an individual feature constraint range and a state subgoal for each skill. The skills can be encoded as dynamic movement primitives that can be generalized to new subgoal configurations and enable flexible transitions during autonomous execution. A task graph can structure these skills at a higher level and be incrementally expanded as new situations require. The derived feature constraints can be used during autonomous task execution for unsupervised time-invariant anomaly detection based on the predictive posterior distribution of the feature cluster.Anomalies can be automatically resolved through task graph recovery behavior. The invention enables a robot to apply learned skills to different contexts and make autonomous decisions while autonomously executing tasks. Learning can occur with reduced computational complexity and increased performance.

[0037] Manual design and demonstration of individual, isolated skills by a user can be eliminated. A collaborative approach to incremental learning of a task is enabled, with the robot able to autonomously detect anomalies and decide whether the user needs to be consulted.

[0038] In the following, embodiments of the invention are described in more detail with reference to figures, which show schematically and by way of example: Fig. 1 an arrangement with a robot for autonomously performing tasks that can learn from user demonstration, Fig. 2 a segmented end-effector trajectory of the user demonstration of a task, Fig. 3 a procedure for learning an alternative skill, Fig. 4 a task graph with recovery behavior and Fig. 5 an anomaly detection.

[0039] Fig. 1 shows an arrangement with a robot 100, a camera 102, a first object 104, a second object 106, and an input device 108. The robot 100 can learn skills from user demonstrations to autonomously perform tasks, wherein a movement sequence is demonstrated, the movement sequence is discretized and states are determined, the states are transferred into a feature space, the movement sequence is segmented into skills taking into account the states and the feature space, and the obtained skills are structured in a task graph.

[0040] The robot 100 is mounted on a linear axis 110 and has a gripper as an end effector 112 and a sensor for measuring forces and torques acting on the end effector 112. The camera 102 is used to track the 6D object poses. The input device 108 is used to start / stop a user demonstration and to actuate the gripper. When starting the demonstration mode, the robot 100 compensates for its own weight, enabling a user demonstration by simply moving the robot 100. The task that the robot 100 must learn from a user demonstration is to pick up the first object 104 and place it on the second object 106 such that the object edges are aligned. Both objects 104, 106 can have any initial 6D poses.Furthermore, the robot 100 should be able to independently resolve an anomaly if the robot 100 loses the first object 104 during transport. The monitored features are: the Euclidean distance between the end effector 112 of the robot 100 and the objects 104, 106; the relative orientation between the end effector and the objects 104, 106; the force acting on the end effector 112; the gripper finger distance, and the gripping state of the gripper ∈ {-1,0,1}.

[0041] Fig. Figure 2 shows a segmented end-effector trajectory 114 of the user demonstration of the task. The end-effector trajectory 114 is converted into a task graph representation. Fig. Figure 3 shows a procedure for learning an alternative skill in the task graph. Here, the anomaly occurs when robot 100 loses the first object 104 during a transport skill. Fig. Figure 4 shows a skill sequence or task graph 116 with recovery behavior. The anomaly detection is in Fig. 5 shown.

[0042] As from Fig. 4, the skill sequence 118 intended to perform the task includes the skills “approach the first object” 120, “close the gripper” 122, “move to the second object” 124, “align with the second object” 126, “open the gripper” 128 and “return to the starting position” 130.

[0043] Fig. 3 illustrates in an upper branch how, upon the first occurrence of an anomaly 132, the user 134 demonstrates a strategy for resolving the situation, 136, 138. The demonstrated strategy is then transferred to the task graph 116 with the additional skills "Approaching the first object" 140 and "Closing the gripper" 142 and linked as an alternative strategy 144 to the skill "Moving to the second object" 124. In the event of a repeated occurrence of the anomaly 146 in subsequent executions of the task, the robot 100 can, as illustrated in a lower branch, resolve the situation autonomously by switching to the alternative strategy 144. The anomaly can occur at any time step during the skill "Moving to the second object" 124, and the robot 100 can automatically generalize the movement for new object configurations using the information about the subgoals.

[0044] Fig. Figure 5 shows anomaly detection using distance-based features. Values ​​of a distance-based feature Dist EEF-Obj 2 relative to the second object are plotted in the x-direction, while values ​​of a distance-based feature Dist EEF-Obj 1 relative to the first object are plotted in the γ-direction. The feature values ​​recorded during the user demonstration are labeled 150. The feature values ​​150 recorded during the user demonstration constitute training data and are used to learn an expected feature region or feature constraint range 152. The expected feature region 152 is represented by a multidimensional Gaussian distribution. As soon as the feature values ​​recorded during execution leave the expected feature region 152, the robot 100 registers an anomaly 154.Feature values ​​recorded during execution that lie within the expected feature region 152 are labeled 156, while feature values ​​recorded during execution that lie outside the expected feature region 152 are labeled 158. Figures 1 and 2 in . Fig. 5 show the user demonstration ( Fig. 3: 136, 138). The images 3 and 4 in Fig. 5 show the autonomous execution of the task ( Fig. 3: 160, 162) by the robot 100.

[0045] "May" refers in particular to optional features of the invention. Accordingly, there are also further developments and / or embodiments of the invention that additionally or alternatively comprise the respective feature(s). Reference symbol 100 robots 102 Camera 104 first object 106 second object 108 Input device 110 Linear axis 112 End effector 114 End-effector trajectory 116 Task Graph 118 intended skill sequence 120 Skill “Approaching the first object” 122 Skill “Close Gripper” 124 Skill “Move to second object” 126 Skill “Align to Second Object” 128 Skill “Open Grabber” 130 Skill “Return to starting position” 132 first occurrence of the anomaly 134 users 136 User demonstration 138 User demonstration 140 Skill “Approaching the first object” 142 Skill “Close Gripper” 144 alternative strategy 146 repeated occurrence of the anomaly 148 alternative strategy 150 feature values ​​from user demonstration 152 Feature region, feature constraints area 154 Anomaly 156 feature values ​​within the expected region 158 feature values ​​within the expected region 160 autonomous task execution 162 autonomous task execution

Claims

[1] Method for robot learning skills (120, 122, 124, 126, 128, 130) from user demonstration to perform tasks autonomously using a robot (100), characterized by , that - a movement sequence is demonstrated, - the movement sequence is discretized and states are determined, - the states are transferred into a feature space, - the movement sequence is segmented into skills taking into account the states and the feature space. [2] Method according to claim 1, characterized by that the skills (120, 122, 124, 126, 128, 130) are described by individual subgoals and feature constraints. [3] Method according to claim 2, characterized by that the subgoals are derived using intention recognition based on inverse reinforcement learning. [4] Method according to at least one of claims 2 to 3, characterized bythat the feature constraints are derived using a Gaussian mixture model. [5] Method according to at least one of the preceding claims, characterized by that the obtained skills (120, 122, 124, 126, 128, 130) are structured in a task graph (116). [6] Method according to claim 5, characterized by that during an autonomous execution of a task, wherein skills (120, 122, 124, 126, 128, 130) of the task graph (116) are executed, when executing a skill (120, 122, 124, 126, 128, 130) it is checked whether the subgoal of the executed skill has been achieved. [7] Method according to at least one of claims 2 to 6, characterized by that during an autonomous execution of a task, wherein skills (120, 122, 124, 126, 128, 130) of the task graph (116) are executed, the feature constraints are used to detect anomalies (154). [8] Method according to at least one of claims 2 to 7, characterized bythat in order to detect anomalies (154) when executing a skill (120, 122, 124, 126, 128, 130) it is checked whether current feature values (154, 156) are within an expected feature constraints range (152). [9] Method according to at least one of claims 2 to 8, characterized by that in order to check whether current feature values lie within an expected feature constraints range (152), a classification probability is determined for a current feature value combination and compared with a skill-specific limit value. [10] Method according to claim 9, characterized by that the threshold is predetermined taking into account a user demonstration and / or updated taking into account a successful execution of skills (120, 122, 124, 126, 128, 130). [11] Method according to at least one of claims 2 to 10, characterized bythat, when, during an autonomous execution of a task, wherein skills (120, 122, 124, 126, 128, 130) of the task graph (116) are executed, the subgoal of an executed skill is reached and / or no anomaly (154) has been detected, a transition takes place from the executed skill (120, 122, 124, 126, 128, 130) to the following skill (120, 122, 124, 126, 128, 130) provided in the task graph (116). [12] Method according to at least one of claims 2 to 11, characterized by that, when, during an autonomous execution of a task, wherein skills (120, 122, 124, 126, 128, 130) of the task graph (116) are executed, the subgoal of an executed skill (120, 122, 124, 126, 128, 130) is not achieved and / or an anomaly (154) is detected, a skill (140, 142) suitable for achieving the subgoal and / or for circumventing the anomaly (154) is searched for in the task graph (116). [13] Method according to claim 12, characterized bythat, if a skill (140, 142) suitable for achieving the subgoal and / or for circumventing the anomaly has been found in the task graph (116), a transition to the suitable skill (140, 142) takes place. [14] Method according to claim 12, characterized by that, if no skill suitable for achieving the subgoal and / or for circumventing the anomaly (154) was found in the task graph (116), the user is given the opportunity to demonstrate a further movement sequence for achieving the subgoal and / or for circumventing the anomaly (154). [15] Method according to claim 14, characterized by , that - the demonstrated further movement sequence is discretized and further states are determined, - the additional states are transferred into the feature space to expand the feature space, - the further movement sequence is segmented into further skills, taking into account the further states and the extended feature space, and - the task graph is extended with the additional skills received (140, 142). [16] Robot (100) for autonomously performing tasks, characterized by that the robot (100) is designed to carry out a method according to at least one of claims 1 to 15. [17] Data structure for autonomously performing tasks using a robot (100), characterized by that the data structure comprises an expandable task graph (116) in which skills (120, 122, 124, 126, 128, 130, 140, 142) learned can be structured using a method according to at least one of claims 1 to 14. [18] Computer program for autonomously performing tasks using a robot (100), characterized by that the computer program comprises program code sections with which a method according to at least one of claims 1 to 15 can be carried out when the computer program is executed on a control device of the robot (100).

Images