Method and apparatus for training manipulation skills of a robotic system
By combining the TP-GMM and TP-HSMM models and using kinesthetic demonstration to train the robot system, the flexibility problem of the robot's manipulation skills in different environments is solved, and efficient new skill learning and reproduction is achieved.
Patent Information
- Application Number
- CN202011164203.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-10-28
- Filing Date
- 2020-10-27
- Publication Date
- 2025-09-12
- Estimated Expiration
- 2040-10-27
AI Technical Summary
Existing robotic manipulation skills lack flexibility and reusability in different workspaces or changing conditions, and simple trajectory replay is difficult to succeed when the environment changes.
The TP-GMM task-parameterized Gaussian mixture model is combined with the TP-HSMM task-parameterized hidden semi-Markov model. By receiving kinesthetic demonstrations of physical inputs, the task parameter set is determined and the skill model is trained, including the distinction between additional and free task parameters, to learn object manipulation skills.
It improves the success rate and flexibility of the robot system in learning new skills, and can achieve efficient learning and reproduction of new skills without the need for simulation models.
Smart Images
Figure CN112720453B_ABST
Abstract
Description
Background Art
[0001] The present disclosure relates to methods and apparatus for training robotic systems.
[0002] Robotic systems are used for manipulation of objects.
[0003] Robotic manipulation skills such as picking, inserting, and lowering are often hard-coded for specific tasks. However, this leads to poor flexibility and reusability in new scenarios such as different workspaces or changing conditions.
[0004] Furthermore, it is known to train robotic systems using demonstrations. Simply recording and replaying a demonstration's trajectory is often insufficient, as changes in the environment, such as changing robot and / or object poses, will render any attempt unsuccessful.
[0005] The object of the present disclosure is to provide a method and system with which learning of new skills and tasks can be achieved in an efficient and simple manner. Summary of the Invention
[0006] This is achieved by a device and a method according to the independent claims.
[0007] A method for training a robotic system, in particular for object manipulation, wherein the robot is capable of performing a set of skills, wherein each skill is learned as a skill model, in particular as a TP-HSMM task-parameterized hidden semi-Markov model combined with a TP-GMM task-parameterized Gaussian mixture model, the method comprising the following steps:
[0008] receiving physical input regarding a skill to be learned by the robot, particularly from a human trainer, wherein the physical input comprises a set of kinesthetic demonstrations on the robotic system;
[0009] determining a set of task parameters for the skill model, wherein the step of determining the set of task parameters comprises determining, for each task parameter of the set of task parameters, whether the task parameter is an additional task parameter that is related to an object that is part of the kinesthetic demonstration, or whether the task parameter is a free task parameter that is not related to a physical object;
[0010] obtaining data from a set of kinesthetic demonstrations for each task parameter of the set of task parameters, and
[0011] The skill model is trained using the set of task parameters and the data obtained for each task parameter.
[0012] Preferably, a skill is learned as a TP-HSMM task-parameterized hidden semi-Markov model combined with a TP-GMM task-parameterized Gaussian mixture model.
[0013] The basic idea of learning from demonstrations is to fit a prescribed skill model, such as a GMM Gaussian mixture model, to a small set of demonstrations. For each skill, a trainer, typically a human trainer, performs several, typically multiple, Kinesthetic demonstrations. Each demonstration contains Total observations The dataset data points, of which Preferably, the demonstration is recorded from the perspective of P different coordinate systems TP, which are also called task parameters. In order to obtain data for the task parameters, the demonstration is transformed from the global frame to the frame ,
[0014] ,
[0015] in are the translation and rotation of frame p relative to a reference frame, specifically the world frame at time t.
[0016] TP-GMM is described by the following parameters:
[0017]
[0018] Where K represents the number of Gaussian components in the mixture model, is the prior probability of each component, and 、 are the mean and covariance of the kth component in frame p.
[0019] Once learned, TP-GMM can be used during execution to reproduce the trajectory of the learned skill. That is, given the observed frames By multiplying the Gaussian components of the affine transformation across different frames, the learned TP-GMM is transformed into a model with parameters A single GMM is as follows
[0020] ,
[0021] The parameters of the updated Gaussian distribution at each frame p are calculated and For more details, refer to the following paper: CALINON Sylvain: A tutorial on task-parameterized movement learning and retrieval. Intelligent Service Robotics, 9(1):1-29, 2016.
[0022] Hidden Semi-Markov Models (HSMMs) extend the standard Hidden Markov Model (HMM) by embedding temporal information about the underlying stochastic process. That is, while in HMMs, the underlying process is assumed to be a Markov process, meaning that the probability of transitioning to the next state depends only on the current state, in HSMMs, the state process is assumed to be a semi-Markov process. This means that the transition to the next state depends on both the current state and the time elapsed since entering that state.
[0023] For example, exemplary details of applying a hidden semi-Markov model combined with a task-parameterized Gaussian mixture model (TP-GMM) for robotic skill encoding to learn spatiotemporal features of demonstrations are disclosed in the following reference: TANWANI, Ajay Kumar, CALINON Sylvain: Learning Robot Manipulation Tasks with Task-Parameterized Hidden Semi-Markov Model. IEEE Robotics and Automation Letters, pages 1-8, 2016.
[0024] More specifically, the task parameterized HSMM model consists of the following parameters
[0025]
[0026] in is the transition probability from state k to h; describes the Gaussian distribution over the duration of a state k, i.e., the probability of remaining in state k for a certain number of consecutive steps; and is a TP-GMM as described above, and for each k, describes the probability of emission corresponding to state k, i.e. the probability of observation. However, a priori In the HSMM, only the probability distribution of the initial component at t = 1 is described. The probability distribution of the components at subsequent time steps is determined via the underlying semi-Markov model. Note that in the HSMM, each state corresponds to a Gaussian component in the associated TP-GMM.
[0027] Preferably, the structure of the semi-Markov model is linear, which means that the sequence of states is deterministic and only the duration in each state is probabilistic. This linear structure can be achieved by setting and (For , And for , By assuming a linear structure, each skill follows the same steps and is trained in a separate model. Preferably, no forking is required.
[0028] Preferably, the aforementioned TP-HSMM formalism is applicable to Riemannian manifolds. For detailed theoretical information, refer to ZEESTRATEN, Martijn: Programming by demonstration on Riemannian manifolds. 2017. This is a doctoral thesis from the Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa DIBRIS and the Department of Advanced Robotics, Italian Institute of Technology.
[0029] For each skill, a trainer, typically a human trainer, performs several kinesthetic demonstrations on the robot. Each kinesthetic demonstration comprises a state sequence consisting of states of an end effector of the robotic system and of states of an object that is part of the kinesthetic demonstration.
[0030] In particular, for skills , where A is the set of manipulation skills and the set of objects involved is represented by Given, and the demonstration set is given by Given, where each demonstration is represented by Given, where It is a manifold The robot end effector status within and manifolds Each object state within Through the combination of these skills, these objects can be manipulated to reach the desired final state.
[0031] According to the present disclosure, skills enable a robotic system to manipulate an object. In the following, a distinction is made between core skills, which include a desired final state of an object, and transition skills, which allow the final state of an object to be freely selected during demonstration.
[0032] The set of task parameters in a skill model is crucial for the flexibility and generalization of skills. Thus, the characteristics of a demonstration can be learned from different perspectives. According to the present disclosure, two different types of task parameters are introduced.
[0033] On the one hand, task parameters are usually attached to objects related to the skill and are constructed from their poses. These are called attached task parameters. Attached task parameters can be derived from object poses. For example, for , the associated task parameters are given by and is given by , where R is the rotation matrix corresponding to the orientation in p.
[0034] On the other hand, for transition skills, there are additional task parameters that are not related to physical objects. These are called free task parameters. Free task parameters cannot be perceived and must be explicitly set by the user or the planning algorithm. The choice of parameters to be considered free task parameters directly impacts the outcome of executing the transition skill.
[0035] Methods according to the present disclosure advantageously enable training a robotic system with new skills in a general manner because the robotic system is able to recognize and encode the intention behind the demonstration.
[0036] Therefore, the success rate of new skills is improved.
[0037] Furthermore, methods according to the present disclosure advantageously enable learning new skills without implementing a simulation model.
[0038] According to an embodiment, the step of determining a set of task parameters comprises selecting task parameters from among: task parameters associated with the pose of the object as part of the kinesthetic demonstration, free task parameters, or task parameters associated with the pose of the end effector of the robotic system, and in particular the initial pose of the end effector of the robotic system. In particular, for skill a, the task parameters are selected from The set of task parameters marked can be selected from the following: Related objects A collection of The set of free task parameters denoted by and the initial pose of the robot arm denoted by r. Typically, this can be done by attaching the frame to all involved objects and attached to the initial pose of the robot arm indexed by r and using the free task parameters Used to transform skills to select a set of task parameters, i.e. .
[0039] Advantageously, by selecting an appropriate set of task parameters, the computational cost can be increased and / or the reproduction performance can be reduced.
[0040] According to an embodiment, the step of selecting task parameters is based on the verification of the reproduction error of said selection. For this purpose, we need a ground truth, which is given as a demonstration. Usually the set of demonstrations is small, so that the same set of demonstrations is used. Perform training and validation. This produces validation:
[0041]
[0042] in is to retrieve the task parameters from the demonstration from the skill model The trajectory of is the number of demos, where each demo contains Total observations in the dataset data points.
[0043] If the number of subjects involved in the skill is small, the model can be trained for all combinations of task parameters and each choice can be verified using the above validation. If the number of subjects is large, the trainer can preferably pre-select some promising task parameter choices to reduce computation time.
[0044] According to an embodiment, the method further comprises the step of learning a prerequisite model of the skill, the prerequisite model of the skill referring to the relative relationship between the robotic system, and in particular the end effector of the robotic system, and the object at an initial time. Preferably, the prerequisites are met when executing the skill to ensure successful execution. The learned skill model as described above contains information about where the robotic arm (in particular the end effector) will start from, but does not contain information about the rest of the environment of the robotic system (in particular the object). Such information should include how the object and the robotic arm (in particular the end effector) are positioned relative to each other when the execution of a particular skill starts. Therefore, according to an embodiment, a task parameterized Gaussian distribution TP-G is learned for each object to fit its posture from the demonstration. More specifically, the prerequisite model of the skill is defined by
[0045]
[0046] in, is the Gaussian distribution of the initial pose of object o at time 1 from the perspective of the initial pose of object p at the initial time 1.
[0047] Advantageously, the precondition model can be learned entirely from kinesthetic demonstrations performed on the robotic system.
[0048] According to an embodiment, the method further comprises the step of learning a skill effect model, which refers to the relative relationship between the robotic system, and in particular the end effector of the robotic system, and the object at a final time. The effect of a skill is the resulting system state after the execution of the skill, which includes both the posture of the robotic arm and the posture of the object involved. The learned skill model as described above contains information about where the robotic arm (in particular the end effector) will stop, but does not contain information about the rest of the environment of the robotic system (in particular the object). Such information should include how the object and the robotic arm (in particular the end effector) are positioned relative to each other when the execution of a particular skill ends. Therefore, according to an embodiment, a task parameterized Gaussian distribution TP-G is learned for each object to fit its posture from the demonstration. More specifically, the effect model of the skill is defined by
[0049]
[0050] in is the Gaussian distribution of the final pose of object o at time T from the perspective of the initial pose of object p at initial time 1.
[0051] Advantageously, the skill model can be learned entirely from kinesthetic demonstrations performed on the robotic system.
[0052] In fact, it can be and Different task parameters are selected to include not only relative conditions but also absolute conditions. Preferably, the skill model and / or the prerequisite model are on the object pose manifold. Calculated within.
[0053] According to a further embodiment, the method comprises the step of evaluating a measure of the preconditions of the skill using the precondition model. In particular, the product of the observation probabilities of the robotic system (in particular the robotic arm) and each object, or equivalently the sum of their logarithmic values, can be calculated. This measure provides a continuous value that evaluates how similar the current situation will be to the situation demonstrated during the kinesthetic demonstration. This measure can already be useful in the following ways: it can be compared with a given threshold to decide whether the preconditions of the skill a are met, or it can be used to evaluate different scenarios and Compare different free task parameters in and decide which one better meets the prerequisites.
[0054] According to another embodiment, the method further comprises the step of predicting the skill effect using the effect model. The prediction may be useful for predicting the effect of the skill, which is the resulting system state after performing the skill, including both the pose of the robot arm and the pose of the involved object.
[0055] The present disclosure also relates to a robotic system configured to be trainable via physical input, in particular physical input from a human trainer, regarding a skill to be learned by the robot, wherein the physical input comprises a set of kinesthetic demonstrations on the robot according to a method according to any of the described embodiments.
[0056] According to an embodiment, the method further comprises optimizing the confidence of the skill sequence. The confidence measure is used to measure how likely it is to successfully perform the skill in a new scenario and / or under new conditions.
[0057] According to an embodiment, the robotic system is configured to track a reference trajectory using an end effector of the robotic system.
[0058] According to an embodiment, a robotic system is configured to detect an object of interest.
[0059] The present disclosure also relates to a computer for controlling a robotic system according to an embodiment, wherein the computer is configured to perform a method according to an embodiment.
[0060] The present disclosure also relates to a computer program comprising a computer program code which, when executed on a computer, enables said computer to perform the method according to an embodiment.
[0061] The present disclosure also relates to a machine-readable storage medium comprising a computer program according to the embodiments.
[0062] The present disclosure also relates to the use of a robotic system, in particular for manipulation of objects, the robot being capable of performing a skill set according to an embodiment, for performing a skill sequence comprising a skill set learned by the robotic system according to a method according to any one of the embodiments and / or using a computer according to an embodiment and / or using a computer program according to an embodiment.
[0063] After determining the best choice of free task parameters according to the above method, the skill sequence is executed as follows. Starting from h = 0, the actual system state The poses of all relevant objects and the robot arm are obtained from state estimation systems such as perception and robot state estimators. Then, based on the optimal choice of task parameters used during the training of the skill model, the poses of all relevant objects and the robot arm are obtained from the state estimation system such as perception and robot state estimator. Construct the associated task parameters. If the skill is a transition skill, the free task parameters associated with the I objects cannot be obtained from the state estimation system, but should be set according to the best choice as described above. Therefore, given the complete set of task parameters, the GMM components can be calculated.
[0064] The above process is summarized in the following Algorithm 1. The training process between lines 2 and 5 is completed offline for each skill a in the skill set A. During execution, when observing the initial state Afterwards, the optimal task parameters are calculated in lines 6 and 7 Each skill is then executed as described in lines 9 to 14, which utilizes the learned model and optimization results.
[0065] Algorithm 1: Optimizing the transformation skills of the manipulation task
[0066] Input: Skill set A; for each Demo ;Skill sequence a.
[0067] 1For each / / During training
[0068] 2Select task parameters .
[0069] 3 training given and TP-HSMM model in .
[0070] 4. Learn the given and Precondition distribution in .
[0071] 5Learn the given and System state evolution in .
[0072] 6 Observe the initial system state . / / During execution
[0073] 7 Calculate confidence Maximized .
[0074] 8 for each do
[0075] 9Observe the current system status .
[0076] 10 if It is a transformation skill then
[0077] 11 According to Freedom to set a .
[0078] 12 Calculation The most likely state sequence .
[0079] 13 based on Generate reference trajectory .
[0080] 14 motion control to track Until the end.
[0081] The present disclosure also relates to the use of a robotic system, in particular for manipulation of an object, the robot being capable of performing a skill set, for determining a specific sequence comprising at least one transition skill of the skill set and at least one core skill of the skill set, and generating a reference trajectory based on the specific sequence.
[0082] Further advantageous embodiments are apparent from the following description and the accompanying drawings.
[0083] Figure 1 A first aspect of a method of training a robotic system according to an embodiment is described;
[0084] Figures 2a to 2d Different scenarios of robotic systems and objects of interest are depicted;
[0085] Figure 3 Depicts the robotic system in a training scenario.
[0086] Figure 1 A schematic diagram depicts a method 100 of training a robotic system 200 according to a first embodiment.
[0087] The robot system is shown in the schematic diagram, e.g. Figures 2a to 2d and Figure 3 Depicted in.
[0088] The robotic system 200 can be used for manipulation of objects. The robotic system 200 is configured to perform a skill set. According to an embodiment, the robotic system 200 includes a gripper and is configured to perform, for example, Figure 2a The skill “Grab Top” depicted in FIG. 4 enables the gripper of the robotic system to grasp the object 300 at the top of the object 300 .
[0089] Figure 2a A situation is described in which the learned skill "grab the top" results in poor performance because the object 300 is too far away. Figure 2b and 2c As shown in , this situation can be improved by using the transition skill "Pan" which pulls the object 300 closer to facilitate the skill "Grab the Top". Figure 2c , the subject 300 has reached a position from which it can perform the skill "Grab the Top". Figure 1A method 100 for training a robotic system is described to enable the robotic system to decide whether a transition skill is needed and how the transition skill should be performed, such as where an object should be placed using the transition skill "translation".
[0090] According to an embodiment, the method 100 comprises the following steps:
[0091] A step 110 of receiving physical input regarding a skill to be learned by the robot, in particular from a human trainer, wherein the physical input comprises a set of kinesthetic demonstrations on the robotic system.
[0092] a step 120 of determining a set of task parameters for the skill model, wherein the step of determining the set of task parameters comprises determining, for each task parameter of the set of task parameters, whether the task parameter is an additional task parameter, which is related to an object that is part of the kinesthetic demonstration, or whether the task parameter is a free task parameter, which is not related to a physical object;
[0093] a step 130 of obtaining data from a set of kinesthetic demonstrations for each task parameter of said set of task parameters, and
[0094] Step 140 of training the skill model using the set of task parameters and the data obtained for each task parameter.
[0095] According to an embodiment, the kinesthetic demonstration relates to the movement of the robotic system 200 , in particular the movement of the robotic arm 210 of the robotic system 200 , in particular the movement of the end effector 220 of the robotic system 200 . Figure 3 An exemplary training scenario is depicted. The end effector 220 of the robotic system 200 is moved by kinesthetic demonstration guided by the hand 240 of a human trainer.
[0096] According to an embodiment, each skill is learned as a skill model, wherein the skill model is preferably learned as a TP-HSMM task-parameterized hidden semi-Markov model combined with a TP-GMM task-parameterized Gaussian mixture model.
[0097] The basic idea of learning from demonstrations is to fit a prescribed skill model, such as a GMM Gaussian mixture model, to a small set of demonstrations. For each skill, a trainer, typically a human trainer, performs several, typically multiple, Kinesthetic demonstrations. Each demonstration contains Total observations The dataset data points, of which Preferably, the demonstration is recorded from the perspective of P different coordinate systems TP, which are also called task parameters. In order to obtain data for the task parameters, the demonstration is transformed from the global frame to the frame ,
[0098]
[0099] in are the translation and rotation of frame p relative to a reference frame, specifically the world frame at time t.
[0100] TP-GMM is described by the following parameters:
[0101]
[0102] Where K represents the number of Gaussian components in the mixture model, is the prior probability of each component, and 、 are the mean and covariance of the kth component in frame p.
[0103] Once learned, TP-GMM can be used during execution to reproduce the trajectory of the learned skill. That is, given the observed frames By multiplying the Gaussian components of the affine transformation across different frames, the learned TP-GMM is transformed into a model with parameters A single GMM is as follows
[0104] ,
[0105] The parameters of the updated Gaussian distribution at each frame p are calculated and For more details, refer to the following paper: CALINON Sylvain: A tutorial on task-parameterized movement learning and retrieval. Intelligent Service Robotics, 9(1):1-29, 2016.
[0106] Hidden Semi-Markov Models (HSMMs) extend the standard Hidden Markov Model (HMM) by embedding temporal information about the underlying stochastic process. That is, while in HMMs, the underlying process is assumed to be a Markov process, meaning that the probability of transitioning to the next state depends only on the current state, in HSMMs, the state process is assumed to be a semi-Markov process. This means that the transition to the next state depends on both the current state and the time elapsed since entering that state.
[0107] For example, exemplary details of applying a hidden semi-Markov model combined with a task-parameterized Gaussian mixture model (TP-GMM) for robotic skill encoding to learn spatiotemporal features of demonstrations are disclosed in the following reference: TANWANI, Ajay Kumar, CALINON Sylvain: Learning Robot Manipulation Tasks with Task-Parameterized Hidden Semi-Markov Model. IEEE Robotics and Automation Letters, pages 1-8, 2016.
[0108] More specifically, the task parameterized HSMM model consists of the following parameters
[0109]
[0110] in is the transition probability from state k to h; describes the Gaussian distribution over the duration of a state k, i.e., the probability of remaining in state k for a certain number of consecutive steps; and is a TP-GMM as described above, and for each k, describes the probability of emission corresponding to state k, i.e. the probability of observation. However, a priori In the HSMM, only the probability distribution of the initial component at t = 1 is described. The probability distribution of the components at subsequent time steps is determined via the underlying semi-Markov model. Note that in the HSMM, each state corresponds to a Gaussian component in the associated TP-GMM.
[0111] Preferably, the structure of the semi-Markov model is linear, which means that the sequence of states is deterministic and only the duration in each state is probabilistic. This linear structure can be achieved by setting and (For , And for , By assuming a linear structure, each skill follows the same steps and is trained in a separate model. Preferably, no forking is required.
[0112] Preferably, the aforementioned TP-HSMM formalism is applicable to Riemannian manifolds. For detailed theoretical information, refer to the doctoral thesis ZEESTRATEN, Martijn: Programming by demonstration on Riemannian manifolds. 2017.
[0113] For each skill, a trainer, typically a human trainer, performs several kinesthetic demonstrations on the robot. Each kinesthetic demonstration includes a sequence of states consisting of the end effector state of the robot system and the state of an object that is part of the kinesthetic demonstration. During the demonstration, the end effector state and the object state are recorded.
[0114] In particular, for skills , where A is the set of manipulation skills and the set of objects involved is represented by Given, and the demonstration set is given by Given, where each demonstration is represented by Given, where It is a manifold The robot end effector status within and manifolds Each object state within Through the combination of these skills, these objects can be manipulated to reach the desired final state.
[0115] According to the present disclosure, skills enable a robotic system to manipulate an object. In the following, a distinction is made between core skills, which include a desired final state of an object, and transition skills, which allow the final state of an object to be freely selected during demonstration.
[0116] The set of task parameters in a skill model is crucial for the flexibility and generalization of skills. Thus, the characteristics of a demonstration can be learned from different perspectives. According to the present disclosure, two different types of task parameters are introduced.
[0117] On the one hand, task parameters are usually attached to objects related to the skill and are constructed from their poses. These are called attached task parameters. Attached task parameters can be derived from object poses. For example, for , the associated task parameters are given by and is given by , where R is the rotation matrix corresponding to the orientation in p.
[0118] On the other hand, for transition skills, there are additional task parameters that are not related to physical objects. These are called free task parameters. Free task parameters cannot be perceived and must be explicitly set by the user or the planning algorithm. The choice of parameters to be considered free task parameters directly impacts the outcome of executing the transition skill.
[0119] According to an embodiment, the step 120 of determining a set of task parameters comprises selecting task parameters from among: task parameters associated with the pose of the object as part of the kinesthetic demonstration, free task parameters, or task parameters associated with the pose of the end effector of the robotic system, and in particular the initial pose of the end effector of the robotic system. In particular, for skill a, the task parameters are selected from The set of task parameters marked can be selected from the following: Related objects A collection of The set of free task parameters denoted by and the initial pose of the robot arm denoted by r. Typically, this can be done by attaching the frame to all involved objects and attached to the initial pose of the robot arm indexed by r and using the free task parameters Used to transform skills to select a set of task parameters, i.e. .
[0120] In cases where the object generates irrelevant task parameters, which may increase computational cost and / or reduce reproduction performance, the object frame should not be selected as a task parameter. One problem that may arise with time-varying task parameters like object pose is that the TP-HSMM only encodes how the task parameters affect the robot arm motion, but does not encode how the robot arm motion affects the object pose. For example, when performing a skill that moves an object, the trajectory of the robot arm attached to the object in the frame is only a single constant point, because the object follows every movement of the robot arm as it is grasped. Therefore, the robot arm will follow the object during reproduction, i.e. stay where the object is, because the trajectory generation is not aware that the robot arm can move freely without leaving a single point component in the object frame. In this case, it is better not to use the object frame as a task parameter.
[0121] According to an embodiment, the step of selecting task parameters is based on the verification of the reproduction error of said selection. For this purpose, we need a ground truth, which is given as a demonstration. Usually the set of demonstrations is small, so that the same set of demonstrations is used. Perform training and validation. This produces validation:
[0122]
[0123] in is to retrieve the task parameters from the demonstration from the skill model The trajectory of is the number of demos, where each demo contains Total observations in the dataset data points.
[0124] If the number of subjects involved in the skill is small, the model can be trained for all combinations of task parameters and each choice can be verified using the above validation. If the number of subjects is large, the trainer can preferably pre-select some promising task parameter choices to reduce computation time.
[0125] According to an embodiment, the method further comprises the step of learning 150 a prerequisite model of the skill, the prerequisite model of the skill referring to the relative relationship between the robotic system, and in particular the end effector of the robotic system, and the object at an initial time. Preferably, the prerequisites are met when executing the skill to ensure successful execution. The learned skill model as described above contains information about where the robotic arm (in particular the end effector) will start from, but does not contain information about the rest of the environment of the robotic system (in particular the object). Such information should include how the object and the robotic arm (in particular the end effector) are positioned relative to each other when the execution of the specific skill starts. Therefore, according to an embodiment, a task parameterized Gaussian distribution TP-G is learned for each object to fit its posture from the demonstration. More specifically, the prerequisite model of the skill is defined by
[0126]
[0127] in, is the Gaussian distribution of the initial pose of object o at time 1 from the perspective of the initial pose of object p at the initial time 1.
[0128] Advantageously, the precondition model can be learned entirely from kinesthetic demonstrations performed on the robotic system.
[0129] According to an embodiment, the method further comprises the step of learning 160 a skill effect model, which refers to the relative relationship between the robotic system, and in particular the end effector of the robotic system, and the object at a final time. The learned skill model as described above contains information about where the robotic arm (in particular the end effector) will stop, but does not contain information about the rest of the environment of the robotic system (in particular the object). Such information should include how the object and the robotic arm (in particular the end effector) are positioned relative to each other when the execution of a particular skill ends. Therefore, according to an embodiment, a task parameterized Gaussian distribution TP-G is learned for each object to fit its pose from the demonstration. More specifically, the effect model of the skill is defined by
[0130]
[0131] in is the Gaussian distribution of the final pose of object o at time T from the perspective of the initial pose of object p at initial time 1.
[0132] Advantageously, the skill model can be learned entirely from kinesthetic demonstrations performed on the robotic system.
[0133] In fact, it can be and Different task parameters are selected to include not only relative conditions but also absolute conditions. Preferably, the skill model and / or the prerequisite model are on the object pose manifold. Calculated within.
[0134] The present disclosure also relates to a robotic system 200 configured to be trainable via physical input, in particular physical input from a human trainer, regarding a skill to be learned by the robotic system 200, wherein the physical input comprises a set of kinesthetic demonstrations on the robot according to the method 100 according to any of the described embodiments.
[0135] According to an embodiment, the robotic system 200 is configured to track the trajectory using an end effector of the robotic system. Preferably, during the demonstration, the trajectory of the end effector of the robotic system can be acquired, for example, using a control module of the robotic system.
[0136] According to an embodiment, the robotic system 200 is configured to detect an object of interest 300. For example, the robotic system includes a camera module for detecting the object 300. Preferably, during the demonstration, the camera module of the robotic system 200 is used to detect task parameters associated with the object.
[0137] The present disclosure also relates to a computer for controlling a robotic system 200 according to an embodiment, wherein the computer is configured to perform the method 100 according to an embodiment.
[0138] The present disclosure also relates to a computer program comprising a computer program code which, when executed on a computer, enables said computer to perform the method 100 according to an embodiment.
[0139] The present disclosure also relates to a machine-readable storage medium comprising a computer program according to the embodiments.
[0140] The present disclosure also relates to the use of a robotic system 200, in particular for manipulation of an object 300, the robotic system 200 being capable of executing a skill set according to an embodiment for executing a skill sequence comprising a skill set learned by the robotic system according to a method according to any one of the embodiments and / or using a computer according to an embodiment and / or using a computer program according to an embodiment.
[0141] After determining the best choice of free task parameters according to the above method, the skill sequence is executed as follows. Starting from h = 0, the actual system state The poses of all relevant objects and the robot arm are obtained from state estimation systems such as perception and robot state estimators. Then, based on the optimal choice of task parameters used during the training of the skill model, the poses of all relevant objects and the robot arm are obtained from the state estimation system such as perception and robot state estimator. Construct the associated task parameters. If the skill is a transition skill, the free task parameters associated with the I objects cannot be obtained from the state estimation system, but should be set according to the best choice as described above. Therefore, given the complete set of task parameters, the GMM components can be calculated.
[0142] The above process is summarized in the following Algorithm 1. The training process between lines 2 and 5 is completed offline for each skill a in the skill set A. During execution, when observing the initial state Afterwards, the optimal task parameters are calculated in lines 6 and 7 Each skill is then executed as described in lines 9 to 14, which utilizes the learned model and optimization results.
[0143] Algorithm 1: Optimizing the transformation skills of the manipulation task
[0144] Input: Skill set A; for each Demo ;Skill sequence a.
[0145] 1For each / / During training
[0146] 2Select task parameters .
[0147] 3 training given and TP-HSMM model in .
[0148] 4. Learn the given and Precondition distribution in .
[0149] 5Learn the given and System state evolution in .
[0150] 6 Observe the initial system state . / / During execution
[0151] 7 Calculate confidence Maximized .
[0152] 8 for each do
[0153] 9Observe the current system status .
[0154] 10 if It is a transformation skill then
[0155] 11 According to Freedom to set a .
[0156] 12 Calculation The most likely state sequence .
[0157] 13 based on Generate reference trajectory .
[0158] 14 motion control to track Until the end.
[0159] The present disclosure also relates to the use of a robotic system 200, in particular for manipulation of an object, the robotic system 200 being capable of executing a skill set, for determining a specific sequence comprising at least one transition skill of the skill set and at least one core skill of the skill set, and generating a reference trajectory based on the specific sequence.
Claims
1. A method (100) of training a robotic system (200) for manipulation of an object (300), the robotic system (200) being capable of performing a set of skills to manipulate the object (300), wherein each skill is learned as a skill model, the skill model being a TP-HSMM task-parameterized hidden semi-Markov model combined with a TP-GMM task-parameterized Gaussian mixture model, the method comprising the following steps: For each skill in the skill set: receiving (110) physical input from a human trainer regarding a skill to be learned by the robot, wherein the physical input comprises a set of kinesthetic demonstrations on the robotic system (200); determining (120) a set of task parameters for the skill model of the skill, wherein the step of determining (120) the set of task parameters comprises determining, for each task parameter of the set of task parameters, whether the task parameter is an additional task parameter, which is related to an object (300) that is part of the kinesthetic presentation, or whether the task parameter is a free task parameter, which is not related to a physical object, wherein, in the determining, when the skill is a transition skill, at least one of the set of task parameters of the skill model is determined to be a free task parameter, the free task parameter being explicitly set by a user and directly influencing an outcome of the transition skill; obtaining (130) data from a set of kinesthetic demonstrations for each task parameter of the set of task parameters, and training (140) the skill model using the set of task parameters and the data obtained for each task parameter; wherein the skill set includes at least one core skill and at least one transition skill, each core skill comprises a desired end state of an object, and each transition skill is a skill for freely selecting the end state of an object during demonstration; and The method further comprises: determining a sequence of skills, the sequence comprising the at least one core skill and at least one transition skill, the sequence for manipulating an object; and generating a reference trajectory based on the sequence; and The method further comprises: Learn the prerequisite model for each skill, where the prerequisite model is defined by the following formula: Among them, (μ 1,o (p) ,Σ 1,o (p) ) is a Gaussian distribution at initial time 1; where α is the skill, O a is the task parameter of the object related to the skill, F a is a free parameter of the skill, TP a are all task parameters, s is the current state of the robot system when starting the skill, and p F is the value of a free task parameter, o is a task parameter of an object related to the skill or a free parameter of the skill, and p is one of the task parameters other than o.
2. The method (100) according to claim 1, wherein: The step of determining (120) a set of task parameters includes selecting task parameters from among: task parameters associated with the pose of the object as part of the kinesthetic demonstration, free task parameters, or task parameters associated with an initial pose of an end effector of the robotic system.
3. The method (100) according to any one of the preceding claims 1-2, wherein: The step of selecting task parameters is based on a verification of the reproduction error of said selections.
4. The method (100) according to any one of the preceding claims 1-2, wherein: The skills are learned as a TP-HSMM task-parameterized hidden semi-Markov model combined with a TP-GMM task-parameterized Gaussian mixture model.
5. The method (100) according to any one of the preceding claims 1-2, wherein: The method further includes the step of learning (160) an effect model of the skill, the effect model of the skill referring to the relative relationship between the end effector (220) of the robotic system (200) and the object (300) at a final time.
6. A robotic system (200) configured to be trainable via physical input from a human trainer, the physical input relating to a skill to be learned by the robotic system, wherein the physical input comprises a set of kinesthetic demonstrations on the robot according to the method of any one of claims 1 to 5.
7. The robotic system (200) of claim 6, configured to track the reference trajectory using an end effector of the robotic system.
8. The robotic system (200) according to any one of claims 6 or 7, configured to detect an object of interest.
9. A computer for controlling a robotic system (200) according to any one of claims 7 to 8, wherein: The computer is configured to perform the method (100) according to any one of claims 1 to 5.
10. A computer program product comprising computer program code which, when executed on a computer, enables the computer to perform the method (100) according to any one of claims 1 to 5.
11. A machine-readable storage medium comprising the computer program product according to claim 10.
12. Use of a robotic system (200) for manipulating an object (300), the robotic system (200) being capable of executing a skill set according to any one of claims 7 to 8, for executing a skill sequence, the skill sequence comprising a skill set learned by the robotic system (200) according to a method (100) according to any one of claims 1 to 5 and / or using a computer according to claim 9 and / or using a computer program according to claim 10.