Method and apparatus for training robotic system operation skills - Patents.com

Abstract

To provide a method and a system capable of realizing the learning of new skills and tasks by an efficient and simple method.SOLUTION: A method of training operation skill of a robot system includes: a step (110) for receiving physical input from a human trainer, the step having a set of kinesthetic sense demonstrations on a robot system; a step (120) for deciding a task parameter set for a skill model, the step including deciding whether each task parameter is an additional task parameter related to an article that is a part of the kinesthetic sense demonstrations or a free task parameter that is not related to a physical article; a step (130) for obtaining data about each task parameter; and a step (140) for training the skill model by using the data obtained about each task parameter.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a method and apparatus for training a robotic system. [Background technology]

[0002] Robotic systems are used for the manipulation of objects.

[0003] Robot manipulation skills such as picking, inserting, and dropping are often hard-coded for specific tasks, resulting in little flexibility and reuse when new situations arise, such as different workspaces or changing conditions.

[0004] Furthermore, while it is known to train robotic systems by demonstration, simply recording and replaying the demonstrated trajectory is often insufficient, as any attempt will fail due to changes in the environment, such as fluctuating poses of the robot and / or object. Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present disclosure is to provide a method and system that enables learning of new skills and tasks in an efficient and simple manner. [Means for solving the problem]

[0006] Disclosure of the Invention This object is achieved by an apparatus and a method according to the independent claims.

[0007] 1. A method for training a robotic system, in particular for manipulating an object, the robot being capable of implementing a set of skills, each skill being learned as a skill model, in particular a TP-HSMM, Task Parameterized Hidden semi-Markov Model, combined with a TP-GMM, Task Parameterized Gaussian Mixture Model, comprising: receiving physical input, particularly from a human trainer, regarding a skill to be learned by the robot, the physical input comprising a set of kinesthetic demonstrations on the robotic system; determining a task parameter set for the skill model, the determining the task parameter set including, for each task parameter of the task parameter set, determining whether a task parameter is an adjunct task parameter related to an object that is part of a kinesthetic demonstration or a free task parameter not related to a physical object; acquiring data for each of the task parameters of the task parameter set from a set of kinesthetic demonstrations; training a skill model using the set of task parameters and the acquired data for each of the task parameters; This is achieved by a method comprising:

[0008] Preferably, one skill is learned as a TP-HSMM, a task-parameterized hidden semi-Markov model, combined with a TP-GMM, a task-parameterized Gaussian mixture model.

[0009] The basic idea of ​​demonstration-based learning is to fit a prescribed skill model, such as a GMM or Gaussian mixture model, to a small number of demonstrations. For each skill, a trainer, in particular a human trainer, performs multiple trials, in particular M a A set of kinesthetic demonstrations are performed on the robot. Each demonstration consists of:

number

number

number

number

[0010] TP-GMM uses the parameters

number

number

[0011] Once learned, TP-GMM can be used at run-time to reconstruct trajectories for the learned skill, i.e., the observed frames are

number

number

number

number

[0012] Hidden Semi-Markov Models (HSMMs) extend the standard Hidden Markov Models (HMMs) by embedding information about the underlying stochastic process over time: whereas in the case of HMMs the underlying hidden process is considered to be Markovian, i.e. the probability of a transition to the next state depends only on the current state, in the case of HSMMs the state process is considered to be semi-Markovian, meaning that a transition to the next state depends both on the current state and on the time elapsed since entering that state.

[0013] Regarding robot skill coding for learning temporal and spatial features of a demonstration, a specific example of applying a hidden semi-Markov model in combination with a task-parameterized Gaussian mixture model (TP-GMM) is disclosed in detail in, for example, the following literature, namely, "Learning Robot Manipulation Tasks with Task-Parameterized Hidden Semi-Markov Model" by TANWANI, Ajay Kumar, and CALINON Sylvain (IEEE Robotics and Automation Letters, pp. 1-8, published in 2016).

[0014] More specifically, the task parameterized HSMM model has the following parameters:

number

number

number

[0015] Preferably, the structure of the semi-Markov model is linear, meaning that the sequence of states is deterministic and only the duration of each state is probabilistic. We define this linear structure as δ for i≠j. ij = 0 and when i = j, δ ij = 1, π k = δ 1k and a kh = δ (k+1)h This can be achieved by setting: Assuming a linear structure, each skill follows the same segment and is trained in a separate model. Preferably, no branching points are required.

[0016] Preferably, the aforementioned formalism of the TP-HSMM is adapted to Riemannian manifolds, for a detailed theory of which see ZEESTRATEN, Martijn, "Programming by demonstration on Riemannian manifolds" (PhD thesis 2017, Department of Informatics, Bioengineering, Robotics, System Engineering, DIBRIS University of Genova, Department of Advanced Robotics Istituto Italiano di Tecnologia).

[0017] For each skill, a trainer, in particular a human trainer, performs multiple kinesthetic demonstrations on the robot, each having a state sequence consisting of multiple states of the end effector of the robotic system and multiple states of the object that is part of the kinesthetic demonstration.

[0018] In particular, for a skill a∈A, where A is the set of operational skills, and the set of objects involved is O a ∈O, and the demonstration set is D a ={D1,...D Ma}, where each demonstration is

number

[0019] According to the present disclosure, a skill allows the robotic system to manipulate an object. In the following, we distinguish between core skills, which have a desired final state of the object, and transition skills, which allow the final state of the object to be arbitrarily selected during the demonstration.

[0020] The task parameter set of a skill model is essential for the flexibility and generalization of the skill, which allows the features of the demonstration to be learned based on different perspectives. According to this disclosure, two different types of task parameters are introduced:

[0021] On the one hand, task parameters usually belong to the objects involved in the skill and are constructed from their poses. These are called attached task parameters. We define the attached task parameters as the object pose p ∈ M p For example,

number

[0022] However, on the other hand, in the case of translational skills, there are further relevant task parameters that are not related to physical objects. These are called free task parameters. The free task parameters are imperceptible and are preferably explicitly set by the user or the planning algorithm. The choice of determining which one parameter is the free task parameter directly affects the outcome of the execution of the translational skill.

[0023] The method of the present disclosure advantageously allows a robotic system to be trained with new skills in a generic manner since it can recognize and code the intent behind the demonstration.

[0024] This improves the success rate of new skills.

[0025] Moreover, the methods of the present disclosure advantageously allow for learning new skills without implementing a simulation model.

[0026] According to one embodiment, the step of determining a set of task parameters comprises selecting task parameters from the following: task parameters associated with a pose of the object that is part of the kinesthetic demonstration, free task parameters, or task parameters associated with a pose, in particular an initial pose, of an end effector of a robotic system. In particular, for skill a, TP a The task parameter set represented by a Set of F a In general, the task parameter set can be selected from among a set of free task parameters represented by a and the initial posture of the robot arm represented by r, and the free task parameters F aBy using TP a =O a ∪F a ∪{r}.

[0027] Advantageously, choosing an appropriate set of task parameters may increase computational cost and / or decrease reproduction performance.

[0028] According to one embodiment, the step of selecting task parameters is based on verification of the reproduction error of said selection. For this, a ground truth is required, which is given as a demonstration. Usually, the demonstration set is rather small, so that it is difficult to obtain the same demonstration set D a is used for training and validation. This allows validation, i.e.

number

number

[0029] When the number of objects involved for a skill is small, the model can be trained on all combinations of task parameters and each choice can be validated using the validation described above. When the number of objects is larger, the trainer should preferably pre-select some likely choices of task parameters to reduce computation time.

[0030] According to one embodiment, the method further comprises a step of learning a precondition model of a skill, which precondition model refers to the relative relationship between the robotic system, in particular the end effector of the robotic system, and the object at an initial time. Preferably, the precondition is fulfilled when the skill is executed to ensure successful execution. The skill model learned as described above includes information about where the robotic arm, in particular the end effector, will start from, but does not incorporate information about the rest of the environment of the robotic system, in particular the object. Such information preferably includes how the object and the robotic arm, in particular the end effector, are positioned relative to each other when the execution of the particular skill starts. Thus, according to one embodiment, a task parameterized Gaussian, TP-G, is learned for each object based on the demonstrations in order to adapt the pose of the object. More specifically, the precondition model of a skill is:

number

number

[0031] Advantageously, the prerequisite models can be learned solely from kinesthetic demonstrations performed on the robotic system.

[0032] According to one embodiment, the method further comprises a step of learning a model of the effect of the skill, which model of effect refers to the relative relationship between the robotic system, in particular the end effector of the robotic system, and the object at the final time. The effect of a skill is the resulting system state after the execution of the skill, which includes both the pose of the robotic arm and the pose of the object involved. The skill model learned as described above includes information about where the robotic arm, in particular the end effector, will stop, but does not incorporate information about the rest of the environment of the robotic system, in particular the object. Such information preferably includes how the object and the robotic arm, in particular the end effector, are positioned relative to each other when the execution of a particular skill is finished. Thus, according to one embodiment, a task parameterized Gaussian, TP-G, is learned for each object based on demonstrations in order to adapt the pose of the object. More specifically, the effect model of a skill is:

number

number

[0033] Advantageously, the skill model can be learned solely from kinesthetic demonstrations performed on the robotic system.

[0034] In practice, to incorporate absolute as well as relative terms, γ 1,a and γ T,a For each of the object poses, different task parameters can be selected. Preferably, the skill model and / or the prerequisite model are based on a manifold of object poses M p It is calculated within

[0035] According to a further embodiment, the method comprises a step of evaluating a measure of the prerequisites of the skill using the prerequisite model. In particular, the product of the observation probabilities for the robotic system, in particular the robot arm and each object, or equivalently the sum of their logarithmic values, can be calculated. The measure provides a continuous value that evaluates how similar the current situation is to the situation demonstrated during the kinesthetic demonstration. This measure can already be useful in the following manner: it can be compared to a predefined threshold to determine whether the prerequisites of skill a are fulfilled or not, or it can be compared to the prerequisites of different situations and p. F The measurements can be used to compare various free task parameters in to determine which one better meets the prerequisites.

[0036] According to a further embodiment, the method further comprises predicting the effect of the skill using the effect model, which prediction may serve to predict the effect of the skill, which is the resulting system state after the execution of the skill, including both the pose of the robot arm and the pose of the objects involved.

[0037] The present disclosure also relates to a robotic system configured to be able to train via physical input, in particular from a human trainer, on a skill to be learned by the robot, where the physical input comprises a set of kinesthetic demonstrations on the robot according to a method according to any of the previously described embodiments.

[0038] According to one embodiment, the method further includes optimizing a confidence measure of the skill sequence, the confidence measure being used to measure how likely it is to successfully perform the skill in a new situation and / or under new conditions.

[0039] According to one embodiment, the robotic system is configured to track a single reference trajectory with an end effector of the robotic system.

[0040] According to one embodiment, a robotic system is configured to detect a target object of interest.

[0041] The present disclosure also relates to a computer for controlling a robotic system according to the previously described embodiments, wherein the computer is configured to perform a method according to the previously described embodiments.

[0042] The present disclosure also relates to a computer program having a computer program code which, when executed in a computer, enables the computer to carry out the method according to the above described embodiments.

[0043] The present disclosure also relates to a machine-readable storage medium having a computer program according to the above-described embodiments.

[0044] The present disclosure also relates to the use of a robotic system, in particular for manipulating an object, the robot being capable of implementing a skillset to execute a skill sequence having the skillset learned by the robotic system according to a method according to any of the previously described embodiments and / or using a computer according to the previously described embodiments and / or using a computer program according to the previously described embodiments.

[0045] After determining the optimal selection of free task parameters according to the method described above, the skill sequence is executed as follows: Starting from h=0, the current system state s, including all relevant objects and the poses of the robot arm, is calculated. h is obtained from a state estimation system, such as a perception and robot state estimator. The associated task parameters are then denoted as s according to the optimal selection of task parameters used during training of the skill model. hIn the case where the skills are transition skills, the free task parameters associated with the l objects cannot be obtained from the state estimation system, but are instead preferably set according to the optimal choices mentioned above. Thus, given a complete set of task parameters, the GMM components can be calculated. The above process can be summarized as shown in Algorithm 1 below. The training process in lines 2-5 is performed offline for each skill a in skill set A. During execution, after observing the initial state s0, the optimal task parameters p are selected in lines 6-7. F Then, each skill is executed as described in lines 9-14, thereby using the learned model and optimization results.

[0046] Algorithm 1: Optimize transition skills for manipulation tasks. Input: A set of skills, A, for each a∈A a demonstration D a , skill sequence a 1 During training, for each a∈A. 2 Task parameters TP a Select . 3 Given TP a and D. a In the TP-HSMM model θ a Train. 4 Given TP 1,a and D. a In,we learn the precondition distribution,θ,1,a,. 5 Given TP T,a and D. a In,we learn the system state evolution,θ,T,a,. 6 During execution, observe the initial system state s0. 7 Reliability C a (s0,p * F ) to maximize p * F Calculate. 8 each a h For ∈a, do the following: 9 Current System Statush is observed. 10a h If is a transition skill, 11 p * F Set the free TP of a according to 12θ ah The most likely sequence of states in k * Calculate. 13k * Based on the reference trajectory ξ * Generate. 14 Until the End * Perform motion control to track.

[0047] The present disclosure also relates to the use of a robotic system, particularly for manipulating an object, where the robot is capable of implementing a skillset to determine a specific sequence having at least one transition skill of the skillset and at least one core skill of the skillset, and further to generating a reference trajectory based on the specific sequence.

[0048] Further advantageous embodiments emerge from the following description and the drawings. [Brief description of the drawings]

[0049] [Figure 1] FIG. 1 illustrates a first aspect of a method for training a robotic system according to one embodiment. [Figure 2a] 1A to 1C are diagrams illustrating various situations of a robot system and an object of interest. [Figure 2b] 1A to 1C are diagrams illustrating various situations of a robot system and an object of interest. [Figure 2c] 1A to 1C are diagrams illustrating various situations of a robot system and an object of interest. [Figure 2d] 1A to 1C are diagrams illustrating various situations of a robot system and an object of interest. [Diagram 3] FIG. 1 illustrates a robotic system in a training situation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0050] FIG. 1 shows a schematic diagram of a method 100 for training a robotic system 200 according to a first embodiment.

[0051] This robotic system is shown, for example, in schematic form in FIGS. 2a-2d and 3.

[0052] The robotic system 200 can be used for the manipulation of objects. The robotic system 200 is configured to perform a skill set. According to this embodiment, the robotic system 200 has a gripper and is configured to perform, for example, the skill "grasp top" as shown in Fig. 2a, which allows the gripper of the robotic system to grasp the object 300 at the top of the object 300.

[0053] In the situation shown in Fig. 2a, the learned skill "grab top" results in poor execution because the object 300 is too far away. Therefore, as shown in Fig. 2b and Fig. 2c, this situation can be improved by using the translation skill "translate" to bring the object 300 closer, in order to facilitate the skill "grab top". In Fig. 2c, the object 300 has reached a position where the skill "grab top" can be executed. For example, a method 100 for training a robotic system to enable the robotic system to determine whether and how a translation skill is required and should be executed in a situation where it is desirable to position the object using the translation skill "translate" is described with reference to Fig. 1.

[0054] According to one embodiment, the method 100 includes the following steps.

[0055] Step 110 of receiving physical input, particularly from a human trainer, regarding the skill to be learned by the robot, the physical input comprising a set of kinesthetic demonstrations on the robotic system.

[0056] determining 120 a task parameter set for the skill model, the determining the task parameter set including, for each task parameter in the task parameter set, determining whether a task parameter is an adjunct task parameter that is related to an object that is part of the kinesthetic demonstration or a free task parameter that is not related to a physical object;

[0057] Data is obtained 130 from a set of kinesthetic demonstrations for each of the task parameters in the set of task parameters.

[0058] The set of task parameters and the data obtained for each task parameter are used to train 140 the skill model.

[0059] According to this embodiment, the kinesthetic demonstration involves the movement of the robotic system 200, in particular the robotic arm 210, and in particular the end effector 220, of the robotic system 200. An exemplary training scenario is shown in Figure 3. The end effector 220 of the robotic system 200 is moved by the kinesthetic demonstration guided by the hand 240 of a human trainer.

[0060] According to this embodiment, each skill is trained as a skill model, where the skill model is preferably trained as a TP-HSMM, a task-parameterized hidden semi-Markov model, combined with a TP-GMM, a task-parameterized Gaussian mixture model.

[0061] The basic idea of ​​demonstration-based learning is to fit a prescribed skill model, such as a GMM or Gaussian mixture model, to a small number of demonstrations. For each skill, a trainer, in particular a human trainer, performs multiple trials, in particular M a A set of kinesthetic demonstrations are performed on the robot. Each demonstration consists of:

number

number

number

number

[0062] TP-GMM uses the parameters

number

number

[0063] Once learned, TP-GMM can be used to reconstruct trajectories for the learned skill on the fly, i.e., if the observed frames are

number

number

number

number

[0064] Hidden Semi-Markov Models (HSMMs) extend the standard Hidden Markov Models (HMMs) by embedding information about the underlying stochastic process over time: whereas in the case of HMMs the underlying hidden process is considered to be Markovian, i.e. the probability of a transition to the next state depends only on the current state, in the case of HSMMs the state process is considered to be semi-Markovian, meaning that a transition to the next state depends both on the current state and on the time elapsed since entering that state.

[0065] Regarding robot skill coding for learning temporal and spatial features of demonstrations, a specific example of applying a hidden semi-Markov model in combination with a task-parameterized Gaussian mixture model (TP-GMM) is disclosed in detail in, for example, the following literature, namely, TANWANI, Ajay Kumar, CALINON Sylvain, "Learning Robot Manipulation Tasks with Task-Parameterized Hidden Semi-Markov Model" (IEEE Robotics and Automation Letters, pp. 1-8, published in 2016).

[0066] More specifically, the task parameterized HSMM model has the following parameters:

number

number

number

[0067] Preferably, the structure of the semi-Markov model is linear, meaning that the sequence of states is deterministic and only the duration in each state is probabilistic. We define this linear structure as δ for i≠j. ij = 0 and when i = j, δ ij = 1, π k = δ 1k and a kh = δ (k+1)h This can be achieved by setting: Assuming a linear structure, each skill follows the same segment and is trained in a separate model. Preferably, no branching points are required.

[0068] Preferably, the aforementioned formalism of the TP-HSMM is adapted to Riemannian manifolds, the detailed theory of which is described in ZIESTRATEN, Martijn, "Programming by demonstration on Riemannian manifolds" (PhD thesis, 2017).

[0069] For each skill, a trainer, in particular a human trainer, performs multiple kinesthetic demonstrations on the robot. Each kinesthetic demonstration has a state sequence consisting of multiple states of the end effector of the robotic system and multiple states of the object that is part of the kinesthetic demonstration. During the demonstration, the states of the end effector and the object are recorded.

[0070] In particular, for a skill a∈A, where A is the set of operational skills, and the set of objects involved is O a ∈O, and the demonstration set is D a ={D1,...D Ma}, where each demonstration is

number

[0071] According to the present disclosure, a skill allows the robotic system to manipulate an object. In the following, we distinguish between core skills, which have a desired final state of the object, and transition skills, which allow the object's final state to be arbitrarily selected during the demonstration.

[0072] The task parameter set of a skill model is essential for the flexibility and generalization of the skill, which allows the features of the demonstration to be learned based on different perspectives. According to this disclosure, two different types of task parameters are introduced:

[0073] On the one hand, task parameters usually belong to the objects related to the skill and are constructed from their poses. These are called attached task parameters. We define the attached task parameters as the object pose p ∈ M p For example,

number

[0074] However, on the other hand, in the case of translational skills, there are further relevant task parameters that are not related to physical objects. These are called free task parameters. The free task parameters are imperceptible and are preferably explicitly set by the user or the planning algorithm. The choice of determining one parameter that is a free task parameter directly affects the outcome of the execution of the translational skill.

[0075] According to one embodiment, the step 120 of determining a set of task parameters comprises selecting task parameters from among the following: task parameters associated with a pose of the object that is part of the kinesthetic demonstration, free task parameters, or task parameters associated with a pose, in particular an initial pose, of an end effector of a robotic system. In particular, for skill a, TP a The task parameter set represented by a Set of F a In general, the task parameter set can be selected from among a set of free task parameters represented by a and the initial posture of the robot arm represented by r, and the free task parameters F a By using TP a =O a ∪F a ∪{r}.

[0076] The object frame should not be selected as a task parameter if the object may generate irrelevant task parameters, which may increase computational cost and / or reduce the reproduction performance. One problem that may be caused by time-varying task parameters such as object pose is that TP-HSMM only codes how the task parameters affect the robot arm motion, but not how the robot arm motion acts on the object pose. For example, while performing a skill to move an object, the trajectory of the robot arm in the frame belonging to the object is only a single fixed point, because the object follows every movement of the robot arm while being grasped. Thus, during reproduction, the robot arm will follow the object, i.e., stay where the object is, because when generating the trajectory, it is not aware that the robot arm may move freely without leaving a single point component in the object frame. In this case, it is better not to use the object frame as a task parameter.

[0077] According to one embodiment, the step of selecting task parameters is based on verification of the reproduction error of said selection. For this, a ground truth is required, which is given as a demonstration. Usually, the demonstration set is rather small, so that the same demonstration set D a is used for training and validation. This allows validation, i.e.

number

number

[0078] When the number of objects involved for a skill is small, the model can be trained on all combinations of task parameters and each choice can be validated using the validation described above. When the number of objects is larger, the trainer should preferably pre-select some likely choices of task parameters to reduce computation time.

[0079] According to one embodiment, the method further comprises a step 150 of learning a precondition model of a skill, which precondition model refers to the relative relationship between the robotic system, in particular the end effector of the robotic system, and the object at an initial time. Preferably, the preconditions are met when the skill is executed to ensure successful execution. The skill model learned as described above includes information about where the robotic arm, in particular the end effector, will start from, but does not incorporate information about the rest of the environment of the robotic system, in particular the object. Such information preferably includes how the object and the robotic arm, in particular the end effector, are positioned relative to each other when the execution of a particular skill starts. Thus, according to one embodiment, a task parameterized Gaussian, TP-G, is learned for each object based on demonstrations in order to adapt the pose of the object. More specifically, the precondition model of a skill is:

number

number

[0080] Advantageously, the prerequisite models can be learned solely from kinesthetic demonstrations performed on the robotic system.

[0081] According to one embodiment, the method further comprises a step 160 of learning an effect model of the skill, which effect model refers to the relative relationship between the robotic system, in particular the end effector of the robotic system, and the object at the final time. The skill model learned as described above includes information about where the robotic arm, in particular the end effector, will stop, but does not incorporate information about the rest of the environment of the robotic system, in particular the object. Such information preferably includes how the object and the robotic arm, in particular the end effector, are positioned relative to each other when the execution of a particular skill is finished. Thus, according to one embodiment, a task parameterized Gaussian, TP-G, is learned for each object based on demonstrations in order to adapt the pose of the object. More specifically, the effect model of a skill is:

number

number

[0082] Advantageously, the skill model can be learned solely from kinesthetic demonstrations performed on the robotic system.

[0083] In practice, to incorporate absolute as well as relative terms, γ 1,a and γ T,a Various task parameters can be selected for . Preferably, the skill model and / or the prerequisite model are based on a manifold of object poses M p It is calculated within

[0084] The present disclosure also relates to a robotic system 200 configured to be trained via physical input, particularly from a human trainer, with respect to a skill to be learned by the robotic system 200, where the physical input comprises a set of kinesthetic demonstrations on the robot according to the method 100 according to any of the previously described embodiments.

[0085] According to one embodiment, the robotic system 200 is configured to track a trajectory by an end effector of the robotic system. Preferably, during a demonstration, the trajectory of the end effector of the robotic system can be retrieved, for example, by a control module of the robotic system.

[0086] According to one embodiment, the robotic system 200 is configured to detect an object of interest 300. For example, the robotic system includes a camera module to detect the object 300. Preferably, during the demonstration, task parameters associated with the object are detected by the camera module of the robotic system 200.

[0087] The present disclosure also relates to a computer for controlling the robotic system 200 according to the previously described embodiment, wherein the computer is configured to execute the method 100 according to the previously described embodiment.

[0088] The present disclosure also relates to a computer program having a computer program code which, when executed in a computer, enables the computer to perform the method 100 according to the previously described embodiments.

[0089] The present disclosure also relates to a machine-readable storage medium having a computer program according to the above-described embodiments.

[0090] The present disclosure also relates to the use of the robotic system 200, in particular for manipulating an object 300, which is capable of implementing a skillset to execute a skill sequence having the skillset learned by the robotic system according to a method according to any of the previously described embodiments and / or using a computer according to the previously described embodiments and / or using a computer program according to the previously described embodiments.

[0091] After determining the optimal selection of free task parameters according to the method described above, the skill sequence is executed as follows: Starting from h=0, the current system state s, including all relevant objects and the poses of the robot arm, is calculated. h is obtained from a state estimation system, such as a perception and robot state estimator. The associated task parameters are then denoted as s according to the optimal selection of task parameters used during training of the skill model. h In the case where a skill is a transition skill, the free task parameters associated with the l objects cannot be obtained from the state estimation system, but are instead preferably set according to the optimal selection mentioned above. Thus, given a complete set of task parameters, the GMM components can be calculated. The above process can be summarized as shown in Algorithm 1 below. The training process in lines 2-5 is performed offline for each skill a in skill set A. During execution, after observing the initial state s0, the optimal task parameters p are selected in lines 6-7. F Then, each skill is executed as described in lines 9-14, thereby using the learned model and optimization results.

[0092] Algorithm 1: Optimize transition skills for manipulation tasks. Input: A set of skills, A, for each a∈A a demonstration D a , skill sequence a 1 During training, for each a∈A. 2 Task parameters TP a Select . 3 Given TP a and D. a In the TP-HSMM model θ a Train. 4 Given TP 1,a and D. a In,we learn the precondition distribution,θ,1,a,. 5 Given TP T,a and D. a In,we learn the system state evolution,θ,T,a,. 6 During execution, observe the initial system state s0. 7 Reliability C a (s0,p * F ) to maximize p * F Calculate. 8 each a h For ∈a, do the following: 9 Current System Status h is observed. 10a h If is a transition skill, 11 p * F Set the free TP of a according to 12θ ah The most likely sequence of states in k * Calculate. 13k * Based on the reference trajectory ξ * Generate. 14 Until the End * Perform motor control to track.

[0093] The present disclosure also relates to the use of the robotic system 200, particularly for manipulating an object, which is capable of implementing a skillset to determine a particular sequence having at least one transition skill of the skillset and at least one core skill of the skillset, and further to generating a reference trajectory based on the particular sequence.

Claims

1. A method (100) for training a robotic system (200), in particular for manipulating an object (300), comprising: The robotic system (200) is capable of implementing a skillset, each skill being learned as a skill model, in particular a TP-GMM, a TP-HSMM combined with a task-parameterized Gaussian mixture model, a task-parameterized hidden semi-Markov model, The method comprises: receiving (110) physical input, in particular from a human trainer, regarding a skill to be learned by the robot, said physical input comprising a set of kinesthetic demonstrations on the robotic system (200); determining (120) a task parameter set for the skill model, the determining (120) including, for each task parameter in the task parameter set, determining whether a task parameter is an adjunct task parameter related to an object (300) that is part of the kinesthetic demonstration or whether a task parameter is a free task parameter not related to a physical object; acquiring (130) data from the set of kinesthetic demonstrations for each task parameter in the set of task parameters; training (140) the skill model using the set of task parameters and the data obtained for each task parameter; Including, A method (100) for training a robotic system (200).

2. The step of determining (120) the set of task parameters comprises selecting said task parameters from the following: task parameters associated with a pose of the object that is part of the kinesthetic demonstration, free task parameters or task parameters associated with a pose, in particular an initial pose, of an end effector of the robotic system; 2. The method of claim 1 (100).

3. selecting the task parameters is based on verification of a reproduction error of the selection; 3. The method (100) of claim 2.

4. One skill is trained as a TP-HSMM, a task-parameterized hidden semi-Markov model, combined with a TP-GMM, a task-parameterized Gaussian mixture model. The method (100) of any one of claims 1 to 3.

5. The method further comprises a step of learning (150) a prerequisite model of a skill, the prerequisite model referring to a relative relationship between the robot system (200), in particular an end effector (220) of the robot system (200) and an object (300) at an initial time point. The method (100) of any one of claims 1 to 4.

6. The method further comprises a step of learning (160) a skill effect model, the effect model referring to a relative relationship between the robotic system (200), in particular an end effector (220) of the robotic system (200) and an object (300) at a final time point. The method (100) of any one of claims 1 to 5.

7. A robotic system (200) configured to be trained via physical input, particularly from a human trainer, with respect to a skill to be learned by the robotic system, comprising: The physical input comprises a set of kinesthetic demonstrations on a robot according to the method of any one of claims 1 to 6. A robot system (200).

8. The robotic system (200) is configured to track a reference trajectory by an end effector of the robotic system. The robotic system (200) of claim 7.

9. The robotic system (200) is configured to detect an object of interest. The robotic system (200) according to claim 7 or 8.

10. A computer for controlling a robotic system (200) according to any one of claims 7 to 9, comprising: A computer configured to carry out the method (100) of any one of claims 1 to 6.

11. 1. A computer program having computer program code, comprising: The computer program code, when executed on a computer, enables the computer to carry out the method (100) of any one of claims 1 to 6. Computer program.

12. A machine-readable storage medium containing the computer program of claim 11.

13. Use of a robotic system (200), particularly for manipulating an object (300), comprising: The robotic system (200) is capable of implementing a skillset according to any one of claims 7 to 9 in order to implement a skill sequence comprising a learned skillset by the robotic system (200) in accordance with a method (100) according to any one of claims 1 to 6 and / or by using a computer according to claim 10 and / or by using a computer program according to claim 11. Use of the robotic system (200).

Images