Method for controlling a robot and robot controller

Abstract

According to various embodiments, a method for controlling a robot is provided, comprising: providing (1001) demonstrations for performing each of a plurality of skills, training (1002) a robot trajectory model for each skill from the demonstrations, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states, training (1003) a precondition model for each skill and a final condition model for each skill from the demonstrations, the precondition model including, for each initial state of the robot trajectory model for the skill, a probability distribution of the robot configuration before performing the skill, the final condition model including, for each final state of the robot trajectory model for the skill, a probability distribution of the robot configuration after performing the skill, receiving (1004) a description of a task, wherein the task includes performing skills from a plurality of skills in sequence and / or in branches, generating (1005) a synthetic robot trajectory model, and controlling (1006) the robot to perform the task according to the synthetic robot trajectory model.

Description

[0001] The present disclosure relates to a method for controlling a robot and a robot controller.

[0002] Robotic skills can be programmed using a learning-from-demonstration approach, where the robot learns the nominal plan of the skill from demonstrations. However, the skill to be applied to perform a certain task may vary depending on the initial system state. For example, when a task involves picking up an object, the skill to be applied may be "pick up the object from the side" or "pick up the object from the top," depending on how the object is oriented. Furthermore, if the object is picked up from the side, it may be necessary to reorient the object to progress further in the task. Therefore, a method that allows the robot to sequence several skills and apply them alternately in changing situations is desirable.

[0003] At the 2019 Conference on Robot Learning, L. Schwenkel, M. Guo, and M. Bürger, in “Optimizing sequences of probabilistic manipulation skills learned from demonstration” (hereinafter referred to as reference [1]), described a skill-centric approach in which each skill is learned independently in a variety of scenarios but is not attached to any specific task.

[0004] According to various embodiments, a method for controlling a robot is provided, comprising: providing demonstrations for performing each of a plurality of skills, training a robot trajectory model for each skill from the demonstrations, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states, training a precondition model for each skill and a final condition model for each skill from the demonstrations, wherein the precondition model includes, for each initial state of the robot trajectory model for the skill, a probability distribution of the robot configuration before performing the skill, and the final condition model includes, for each final state of the robot trajectory model for the skill, a probability distribution of the robot configuration after performing the skill, receiving a description of a task, wherein the task includes performing the skills from the plurality of skills in sequence and / or in branches, generating a synthetic robot trajectory model by:

[0005] - When two skills are to be executed sequentially in a mission, the robot trajectory model for cascading skills is as follows

[0006] ○ Include the states of the trajectory models of both skills in the composite robot trajectory model, and

[0007] Calculate the transition probability between each final state of the trajectory model of the first skill and each initial state of the trajectory model of the second skill based on the similarity between the probability distribution of the final conditional model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state of the second skill

[0008] - When executing two skills in a branch in a mission, the robot trajectory model of the skills is combined by the following

[0009] ○ Include the states of the trajectory models of both skills in the composite robot trajectory model, and

[0010] ○ Set the transition probability between the first skill's state and the second skill's state to zero; and

[0011] Control the robot to perform tasks according to the synthesized robot trajectory model.

[0012] According to further embodiments, a robot controller configured to perform the above method is provided.

[0013] The aforementioned method for controlling a robot and robot controller allow for the automatic synthesis of manipulation skills to perform a specified manipulation task. Therefore, the user or operator does not need to manually define branching conditions to select between different skills to perform the task. In other words, after specifying a task graph, the operator can directly run it, even if the task graph includes skills in branches that require selecting between different skills depending on the current system state (e.g., robot and / or object configuration).

[0014] This saves a lot of effort in executing the manipulation task, because evaluating branch conditions requires direct access to various sensors, such as the robot joint states and the pose of the manipulated object. Reading and recording the correct data can be quite time-consuming. Moreover, such data needs to be written directly to the source code, which requires direct access to the source code. In addition, the provided control method and robot controller increase the robustness of task execution, because it is often the case that the combination of branch conditions fails to cover the entire state space, and therefore there are situations where no condition is satisfied, resulting in failure.

[0015] In the following, various examples are given.

[0016] Example 1 is a method for controlling a robot as described above.

[0017] Example 2 is the method of Example 1, wherein each skill includes manipulation of one or more objects by the robot, and the hidden semi-Markov model for each skill is a task-parameterized hidden semi-Markov model that includes task parameters corresponding to the robot and object configuration in which the skill is applied.

[0018] In particular, various embodiments allow for efficient training and control of robots for treating one or more objects.

[0019] Example 3 is the method of Example 2, further comprising training an effect model for each skill, the effect model comprising for each final state of the robot trajectory model for the skill a probability distribution of how the robot and / or object configuration changes after performing the skill relative to the initial robot and / or object configuration to which the skill was applied, wherein, when two skills are to be performed sequentially in a task, the robot trajectory model of the cascaded skill comprises transforming the task parameterized hidden semi-Markov model of the second skill so that its task parameters are task parameters corresponding to the robot and / or object configuration given by the effect model of the first skill.

[0020] Therefore, it is possible to ensure that the control trajectory is correctly calculated over the skill sequence. For example, the transformation of the model ensures that the associated global Gaussian of the second skill can be directly calculated based on the initial state of the first skill. The global Gaussian can then be used to calculate observation probabilities, such as for application of the Viterbi algorithm.

[0021] Example 4 is the method of Example 3, further comprising affine transforming the effect model of the second skill so that its task parameters are task parameters corresponding to the robot and / or object configuration given by the effect model of the first skill.

[0022] This allows for the correct calculation of control trajectories over skill sequences in scenes with, for example, one or more objects manipulated by a robot.

[0023] Example 5 is the method of any one of Examples 1 to 4, further comprising, when two skills are to be performed sequentially in a task, using the prerequisite model of the first skill as the prerequisite model of the cascade robot trajectory model for the skills of the two skills, and using the final condition model of the second skill as the final condition model of the cascade robot trajectory model for the skills.

[0024] By computing precondition models for skill cascades, additional skills can be added, for example, a composite model of more than two skills in sequence or branching can be determined. This allows for the training and control of complex tasks.

[0025] Example 6 is a method of any one of Examples 1 to 5, further comprising, when two skills are to be performed in a branch in a task, including both a prerequisite model for the first skill and a prerequisite model for the second skill in the prerequisite model of the combined robot trajectory model of the skills, and including both a final condition model for the first skill and a final condition model for the second skill in the prerequisite model of the combined robot trajectory model of the skills.

[0026] By computing the prerequisite models for skill branches, additional skills can be added, for example, a composite model of more than two skills in a branching sequence can be determined. This allows for the training and control of complex tasks.

[0027] Example 7 is the method of any one of Examples 1 to 6, wherein the similarity between the probability distribution of the final conditional model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state of the second skill is the KL-divergence of the probability distribution of the final conditional model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state of the second skill.

[0028] The use of KL-divergence allows for efficient cascading of trajectory models of skill sequences.

[0029] Example 8 is the method of any one of Examples 1 to 7, wherein the precondition model and the final condition model are task-parameterized Gaussian mixture models.

[0030] For example, a Gaussian is determined for each task parameter value. Task parameterization allows the application of the trained model in different scenarios (i.e., varying initial system states (configurations)).

[0031] Example 9 is the method of any one of Examples 1 to 8, wherein generating a synthetic robot trajectory model includes repeatedly cascading robot trajectory models, cascaded robot trajectory models, and combined robot trajectory models according to a task, and combining the robot trajectory models, the cascaded robot trajectory models, and the combined robot trajectory so that the robot trajectory model is a trajectory model for the entire task.

[0032] Therefore, a synthetic model for a complex task can be determined iteratively, thereby ensuring robustness to the complex task and freeing the operator from having to define a complex hierarchy of branch conditions.

[0033] Example 10 is a robot controller configured to perform the method of any one of Examples 1 to 9.

[0034] Example 11 is a computer program comprising instructions which, when executed by a processor, cause the processor to perform the method according to any one of Examples 1 to 9.

[0035] Example 12 is a computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform the method according to any one of Examples 1 to 9.

[0036] In the drawings, like reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily to scale, but emphasis is generally placed upon illustrating the principles of the invention. In the following description, various aspects are described with reference to the following drawings, in which:

[0037] Figure 1 A robot is shown.

[0038] Figure 2 A flow chart is shown illustrating a manipulation task including skills in sequence and branching.

[0039] Figure 3 A flow chart illustrating a method for controlling a robot according to an embodiment is shown.

[0040] Figure 4 Illustrated is a cascading operation for concatenating robot trajectory models of skills to be performed sequentially into a composite robot trajectory model.

[0041] Figure 5 Illustrated is a combining operation for combining robot trajectory models of skills to be performed in branches (ie, alternations) into a composite robot trajectory model.

[0042] Figure 6 shows that applying the cascade operation to Figure 2 The task graph obtained after the task graph.

[0043] Figure 7 shows that applying the combination operation to Figure 6 The task graph obtained after the task graph.

[0044] Figure 8 shows that applying the cascade operation to Figure 7 The task graph obtained after the task graph.

[0045] Figure 9 shows that applying the combination operation to Figure 8 The task graph obtained after the task graph.

[0046] Figure 10 A flow chart illustrating a method for controlling a robot according to an embodiment is shown.

[0047] The following detailed description refers to the accompanying drawings, which illustrate, by way of illustration, specific details and aspects of the present disclosure in which the present invention may be practiced. Other aspects may be utilized, and structural, logical, and electrical changes may be made, without departing from the scope of the present invention. The various aspects of the present disclosure are not necessarily mutually exclusive, as some aspects of the present disclosure may be combined with one or more other aspects of the present disclosure to form new aspects.

[0048] Hereinafter, various examples will be described in more detail.

[0049] Figure 1 A robot 100 is shown.

[0050] Robot 100 includes a robotic arm 101, such as an industrial robot arm for handling or assembling a workpiece (or one or more other objects). Robot arm 101 includes manipulators 102, 103, and 104 and a base (or support) 105 that supports manipulators 102, 103, and 104. The term "manipulator" refers to the movable component of robot arm 101, whose actuation enables physical interaction with the environment, such as performing a task. For control, robot 100 includes a (robot) controller 106, which is configured to implement interaction with the environment according to a control program. The last component 104 of manipulators 102, 103, and 104 (furthest from support 105) is also referred to as the end effector 104 and may include one or more tools, such as a welding torch, a gripping instrument, or spray equipment.

[0051] Other manipulators 102, 103 (closer to the support 105) may form a positioning device such that, together with the end effector 104, a robotic arm 101 is provided having the end effector 104 at its end. The robotic arm 101 is a mechanical arm that can provide similar functions to a human arm (possibly with a tool at its end).

[0052] The robotic arm 101 may include joint elements 107, 108, 109 that interconnect the manipulators 102, 103, 104 with each other and with the support 105. The joint elements 107, 108, 109 may have one or more joints, each of which may provide the associated manipulators with rotational motion (i.e., rotational motion) and / or translational motion (i.e., displacement) relative to each other. Movement of the manipulators 102, 103, 104 may be initiated by actuators controlled by a controller 106.

[0053] The term "actuator" is understood to mean a component adapted to affect a mechanism or process in response to being actuated. An actuator can implement a command (so-called activation) issued by controller 106 into mechanical movement. An actuator (e.g., an electromechanical converter) can be configured to convert electrical energy into mechanical energy in response to actuation.

[0054] The term "controller" may be understood to refer to any type of logically implemented entity, which may include, for example, circuitry and / or a processor capable of executing software stored in a storage medium, firmware, or a combination thereof, and which may issue instructions, such as, in this example, to an actuator. The controller may be configured, for example, via program code (e.g., software), to control the operation of the system (in this example, the robot).

[0055] In this example, the controller 106 includes one or more processors 110 and a memory 111 storing code and data based on which the processors 110 control the robotic arm 101. According to various embodiments, the controller 106 controls the robotic arm 101 based on a statistical model 112 stored in the memory 111.

[0056] The robot 100 can learn to perform a task or collaborate with a human partner using a learning from demonstration (LfD) approach. Human demonstrations can be encoded by a probabilistic model (also known as a statistical model) that represents a nominal plan for the robot's task. The controller 106 can then use the statistical model (also known as a robot trajectory model) to generate desired robot movements, which may depend on the state of both the human partner and the environment.

[0057] The basic idea of ​​LfD is to match a prescribed skill model (such as GMM) with a small number of demonstrations. Let there be M demonstrations, each containing Total observations The dataset data points, of which Furthermore, assume that the same demonstration is recorded from the perspective of P different coordinate systems (given by the task parameters, such as the local coordinate system or the frame of the object of interest). A common way to obtain such data is through Transform the presentation from the static global frame to frame p. Here, is the translation and rotation of the (local) frame relative to the world (i.e. global) frame. Then, the TP-GMM is composed of the model parameters Description, where K represents the number of Gaussian components in the mixture model, is the prior probability of each component, and is the parameter of the kth Gaussian component in frame p.

[0058] Unlike the standard GMM, the above hybrid model cannot be learned independently for each frame. In fact, the hybrid coefficient is shared by all frames, and the kth component in frame p must be mapped to the corresponding kth component in the global frame. Expectation-maximization (EM) is a well-established method for learning such models.

[0059] Once learned, the TP-GMM can be used during execution to reproduce the trajectory for the learned skill. That is, given the observed frame , by multiplying the Gaussian components of the affine transformation across different frames, the learned TP-GMM is transformed into a A single GMM is as follows

[0060]

[0061] where the updated Gaussian parameters at each frame p are calculated as as well as Although task parameters may vary over time, the time index is discarded for ease of notation.

[0062] Hidden Semi-Markov Models (HSMMs) extend standard Hidden Markov Models (HMMs) by embedding temporal information about the underlying stochastic process. That is, while in HMMs the underlying hidden process is assumed to be Markov, i.e. the probability of transitioning to the next state depends only on the current state, in HSMMs the state process is assumed to be semi-Markov. This means that the transition to the next state depends on the current state as well as on the time that has passed since entering that state. They can be combined with TP-GMMs for robotic skill encoding to learn the spatiotemporal characteristics of demonstrations. More specifically, the Task Parameterized HSMM (TP-HSMM) model is defined as:

[0063] ,

[0064] in is the transition probability from state h to k; ( ) a Gaussian distribution describing the duration of state k, i.e., the probability of staying in state k for a certain number of consecutive steps; Equivalent to the TP-GMM introduced earlier, it represents the probability of observation corresponding to state k. Note that the number of states in this paper corresponds to the number of Gaussian components in the “attached” TP-GMM.

[0065] Given a (part of) sequence of observation data points , assuming The associated state sequence in is given by Given. Data points Belongs to state k (ie s t =k) ​​is determined by the forward variable gives:

[0066]

[0067] in is the launch probability and is derived from (1) given the mission parameters Furthermore, the same forward variables can also be used during reproduction to predict future steps, until .

[0068] However, in this case, since future observations are not available, only the transition and duration information is used, i.e., by generating set up Finally, by selecting ( ), determine the sequence of most likely states .

[0069] Now let the expected final observation of the robot state be given as , where T is the skill time horizon (e.g., the average length over the demonstration). In addition, the initial robot state is observed as For a given learning model The execution of skills (i.e., skill reproduction) under the given and The most likely state sequence .

[0070] In this case, the forward variables cannot be directly used for reproduction, because the forward variables in Equation (2) calculate the most likely sequence of marginal states, while what is desired is the sequence of states that are most likely to be found given and As a result, when using (2), there is no guarantee that the returned sequence Matching both the spatiotemporal patterns of the demonstration and the final observations. For the example of picking up an object, it might return the most likely sequence corresponding to "picking from the side" even if the desired final configuration is the end effector on top of the object.

[0071] To overcome this problem, according to one embodiment, a modification of the Viterbi algorithm is used. The classic Viterbi algorithm can be used to find the most likely sequence of states (also called the Viterbi path) in an HMM that produces a given stream of observed events. According to one embodiment, a method is used that differs from this in two main respects: (a) it works on HSMMs instead of HMMs; and more importantly (b) most observations are lost except for the first and last. Specifically, in the absence of observations, the Viterbi algorithm becomes

[0072]

[0073] in is the duration probability of state j, is the likelihood that the system is in state j at time t and is not in state j at t+1; and

[0074]

[0075] in is given In the case of (1) That is, at each time t and for each state j, the record maximizes the equation two independent variables and use a simple backtracking procedure to find the most likely sequence of states In other words, the above algorithm derives from Start generating final observations The most likely sequence of skill a .

[0076] Since the robot task space is represented by the time-varying pose (with position and orientation) of the end effector, classical Euclidean-based methods are insufficient to process such data. Therefore, according to various embodiments, the robot task space is given a Riemannian manifold In short, for the manifold Each point in , there exists a tangent space This allows us to perform Euclidean operations locally while being geometrically consistent with the manifold constraints.

[0077] Exponential and logarithmic graphs can be used to and Mapping points between. Point The points in the tangent space of are mapped to points on the manifold while maintaining geodesic distances. The inverse operation is called the logarithmic map Another useful operation is translation. , which moves elements between tangent spaces without introducing distortion. The exact form of the above operation depends on the Riemannian metric associated with the manifold. According to various embodiments, the Riemannian manifold is used to appropriately compute Based on statistics, the Riemann normal distribution encodes the observed movement patterns, and a Riemann optimal controller is used to retrieve the control actions corresponding to the task plan (i.e., sequence skills).

[0078] For the following explanation, a multi-DoF robotic arm 101 is considered as an example, whose end effector 104 has a state of operating within a static and known workspace. (Describes the Cartesian position, orientation quaternion and gripper state). In addition, within the reach of the arm 101, there are Identifies objects of interest, each with a state For simplicity, the overall system state is given by Mark.

[0079] In this setup, a human user performs several kinesthetic demonstrations on the arm to manipulate one or several objects for certain manipulation skills. Let the set of demonstrated skills be In addition, for skills , the set of objects involved is represented by Given, and the available demo collection is given by It should be noted that all demonstrations follow the object-centric structure introduced above, i.e. they are recorded from multiple frames, usually with The skill "insert a stake into a cylinder" involves the objects "stake" and "cylinder", and the associated demonstrations are recorded from both the robot, the "stake", and the "cylinder" frames.

[0080] The (manipulation) tasks considered below consist of skills from demonstration A Skills selected in For example, the insertion task involves "picking up the lid, reorienting the lid, picking up the lid again and inserting it". At the end of the task, the target configuration G is reached as the desired final state of the system (including the robot and the object).

[0081] A common way to organize control tasks in a factory is through diagrams or flow charts. These are usually defined via drag-and-drop in a graphical user interface (GUI). This approach is sufficient if two conditions are met: (1) the task is simple and specified as a linear sequence of skills; and (2) each skill is simple and has no branches. In this way, each skill can be triggered and executed in the specified order.

[0082] However, in many cases, none of the above conditions hold true, for example, a desired task has multiple execution options in various workspace situations, or some internal skills have multiple execution options in various situations.

[0083] Figure 2 A flow chart (or task diagram) 200 is shown illustrating a manipulation task including skills in a sequence and in branches.

[0084] For example, a manipulation task includes, as a first operation 201, picking up an object. This may mean that the robot must perform the skill "Pick up object from top" in 202, the skill "Pick up object from left" in 203, or the skill "Pick up object from right" in 204, depending on the object's initial configuration (i.e., state). Therefore, the task involves executing these skills in a branched manner, meaning they are executed alternately, or only one of them is executed. The first operation 201 (i.e., the execution of one of the skills in 202, 203, or 204) is followed by one or more skills in sequence. For example, if the skill "Pick up object from top" is executed in 202, this is then followed (in sequence) by the skill "Attach object" in 205.

[0085] If the corresponding skill in 203 or 204 is executed, this must be followed by a reorientation operation 206, i.e., in each case, the execution of the reorientation skill in 207 or 208. The reorientation skills may differ in the reorientation direction. The reorientation operation 206 is then followed by the execution of the "pick up object from top" skill 209 and finally the execution of the "attach object" skill in 210.

[0086] Branching can be handled by manually specifying branch conditions 211, 212, 213, typically as "if" conditions, such as "if the subject is standing" 211, "if the subject is lying to the left" 212, and "if the subject is lying to the right" 213. To design such conditions, one can manually measure regions of system state as validity regions for which the conditions hold.

[0087] This means that the robot may have a pre-installed set of manipulation skills (either pre-programmed from the factory or taught via demonstration), and for a specific assembly task, the operator manually constructs a graph that specifies the task (e.g. Figure 2 ), where a building block is a collection of learned skills. Due to branching (possibly at both the task level and the skill level), the operator needs to manually define the branching conditions for each branch.

[0088] According to various embodiments, methods are provided that specifically allow avoiding the necessity of manually defining branch conditions.

[0089] Figure 3 A flow chart illustrating a method for controlling a robot according to an embodiment is shown.

[0090] At 301 , a demonstration of the skill is performed.

[0091] The skills include at least those skills required to perform the tasks given by the task graph 303 .

[0092] As mentioned above, for a demonstrated skill , the collection of available demos is provided by P Recorded in a framework It should be noted that such a frame is directly attached to The objects in .

[0093] At 302 , a robot trajectory model (also labeled “robot behavior model”) is learned for each skill.

[0094] For example, as described above, given an appropriately chosen number of components K , an EM-like (expectation maximization) algorithm can be used to learn a TP-HSMM model that abstracts the spatiotemporal features of the trajectory associated with skill a .

[0095] At 304 , a synthetic robot trajectory model is generated from the robot trajectory model learned at 302 .

[0096] To this end, the learning of skill models further includes the learning of precondition models, final conditions and effect models for each skill. In 304, using these models, a synthetic model of the specified task is constructed, and then choices at the task level and skill level can be made automatically depending on the workspace situation. In simple terms, the precondition model encapsulates how the system should be before the skill is executed, while the effect model and final condition model encapsulate how the system should change after the skill is executed. These models are an important part for calculating the synthetic model because they measure the compatibility between skills and track the evolution of the system state. It should be noted that the term "skill model" can be understood to include all of the robot trajectory model, precondition model, final condition model and effect model for the skill.

[0097] As reference Figure 2 As described, tasks can include branching (i.e., alternatives) and the performance of skills in sequence.

[0098] Therefore, the generation of the composition model includes recursively applying a combination operation for combining skills in sequence and an operation for combining skills in parallel.

[0099] Figure 4 A cascading operation for cascading the robot trajectory models of skills 401 , 402 , 403 to be performed sequentially into a composite robot trajectory model 404 is illustrated.

[0100] Figure 5 Illustrated is a combining operation for combining the robot trajectory models of skills 501 , 502 , 503 , 504 to be performed in branches (ie, alternating) into a composite robot trajectory model 505 .

[0101] like Figure 4The combined operation of the cascaded skill sequence illustrated in comprises one or more applications of the operations of the two skills to be executed sequentially. Figure 5 The combined operation of the combined skill sequence illustrated in

[0065] includes one or more applications of an operation that combines two skills to be performed in a branch.

[0102] For a combination of two skills to be executed sequentially, the trajectory models of the two skills are concatenated into a composite trajectory model as follows.

[0103] Consider two TP-HSMMs of two skills in sequence and , cascade them into The operations are summarized in Algorithm 1.

[0104]

[0105] It should be noted that the calculation and update of lines 3 and 4 of Algorithm 1 can be performed according to equations (4) and (5) given below, respectively.

[0106] It can be seen that the key insight is that the same model depending on Cascaded to The final components (i.e., HSMM states) of Afterwards Different transformations of the task parameters are encoded, which in turn lead to updating Therefore, the synthetic model With size ,in and They are and The quantity of the component, and yes More specifically, Algorithm 2 consists of two main operations: (a) Calculate the final number of components from Each final component in The transition probability of each initial component in ; (b) is Cascaded to Each final component in the modification All the ingredients.

[0107] According to one embodiment, the precondition model and effect model described in reference [1] are used. In particular, The premise model for the labeled learning includes a TP-GMM for the initial robot state (i.e., the initial configuration (e.g., position and / or pose) of the robot), i.e., ,in is a set of selected task parameters derived from the initial system state, i.e., the initial configuration (e.g., position and / or pose) of the robot and / or object. In addition, The final conditional model denoted by Learning in a similar way, but targeting the final robot state, i.e., ,in is a selected set of frames derived from the final system state. In simple terms, Skills in execution a Previously, the initial configuration was modeled, and The final configuration is modeled later. In addition, The learning effect model denoted by TP-GMM contains the final system state for prediction, i.e., ,in exist It is worth noting the differences between these three models: The task parameters are derived from the final system state (execution a After that) is calculated, and and The task parameters are from the initial system state (execution a For the convenience of notation .

[0108] So, from A final component of arrive An initial component of The transition probability is:

[0109]

[0110] in is the KL (Kullback-Leibler)-divergence, Is used for framework p The amount The associated GMM, Is used for framework p The amount associated GMM; is a common set of frames shared by both models, which can be forced to be non-empty by always adding the global frame. The final component and Repeat this process for all pairs of initial components in . It should be noted that The exit probability of any final component in should be normalized.

[0111] Second, given A final component of , Each component k All should be affine transformed as follows,

[0112]

[0113] The operation is defined as the same operation as (1); is based on The task parameters are calculated by averaging the values ​​of o is with Old framework in p associated objects, and yes It should be noted that in the given In the case of the initial system state, the framework changes for direct calculation All components of are important. The same process is performed based on Changing its frame also applies to Each component of.

[0114] Finally, as stated in Algorithm 1, such as duration probability, initial and final distributions Other model parameters are and A common setup with slight changes. For example, The duration probability is copied to Multiple replicas; initial distribution is set to zero because The initial state corresponds to the first model The initial state of The final component of is removed because The final state is now Updates to the final component of its multiple instances.

[0115] For a combination of two skills to be executed in parallel, the trajectory models of the two skills are combined into a composite trajectory model as follows.

[0116] Consider two TP-HSMMs with two skills in parallel and , which is summarized in Algorithm 2 by combining them into operation.

[0117]

[0118] Algorithm 2 consists of two parts: one part is used to calculate the synthesized TP-HSMM model , and the other part is used to calculate the synthesized TPGMM model The first and most important step is to Update the total number of components in This is to avoid multiple components forming different skills with the same index. Afterwards, all associated TPGMM models, duration models, precondition and effect models are updated accordingly. Last but not least, when calculating the composite transfer matrix When we need to of and of Append to the diagonal , while filling the remaining entries with zeros. This means that arrive No additional transfers were added because they were synthesized in parallel (i.e., not sequentially).

[0119] In summary, generating the composite model in 304 includes repeated application of the following operations of the two skills:

[0120] 1) Operation 1 (Cascade Sequential Skills): If "Skill #1" and "Skill #2" are connected in sequence (e.g. Figure 4 (Figure 2), the synthetic skill model is calculated according to Algorithm 1

[0121] 2) Operation 2 (Combining skills in the branch): If "Skill #1" and "Skill #2" are connected in parallel (e.g. Figure 5 (see figure in the figure), the synthetic skill model is calculated according to Algorithm 2.

[0122] Specifically, these two operations are repeated as follows:

[0123] A) For each branch or sub-branch within the task graph 303, recursively apply operation 1 to derive the composite skill for each branch.

[0124] B) For all parallel branches, recursively apply operation 2 to derive the composite skills of all branches. Note that after A, each branch should have only one composite skill.

[0125] C) Recursively, apply A to all skill sequences, and apply B to all skills in parallel.

[0126] For example, for Figure 4 The task illustrated in FIG, the process for generating the synthetic model includes the application of A, then B, then A again, and then B again, as Figures 6 to 9 As shown in the picture.

[0127] Figure 6It shows that when A is first applied to Figure 2 The result after the task given by the task graph.

[0128] Skill models 201 and 205 have been combined into 601 , skill models 203 , 207 have been combined into 602 , and skill models 204 , 208 have been combined into skill model 603 .

[0129] Figure 7 The result is shown after the first application of B. Skill models 602 and 603 have been combined into skill model 701 .

[0130] Figure 8 The result after the second application of A is shown. Skill models 701 , 209 and 210 have been combined into skill model 801 .

[0131] Figure 9 The result is shown after the second application of B. Skill models 601 and 801 have been combined into skill model 901 .

[0132] As can be seen, in Figure 9 In [ 15 ], the entire task is represented by a single composite skill model 901. No "if" conditions are required.

[0133] When the synthetic robot trajectory model has been generated in 304 , the task can be performed under the given circumstances.

[0134] To this end, an initial system state (configuration) for a given situation is observed in 305 and the most likely sequence of components within the synthetic robot trajectory model is determined, for example by applying equation (3) to the synthetic robot trajectory model, which drives the system (including the robot and the object) to the target state with the highest probability.

[0135] The determination of the component sequence also outputs the actual sequence of skills that need to be performed in a given situation. This is important because the skill sequence is different when the algorithm chooses different branches.

[0136] During execution, the optimal skill sequence contained in the output is executed by following the optimal sequence of components in 306. Given a state sequence, Linear Quadratic Tracking (LQT) can, for example, be used to retrieve the optimal trajectory.

[0137] In summary, according to various embodiments, there is provided Figure 10 The method shown in the figure.

[0138] Figure 10 Shown is a flow chart 1000 illustrating a method for controlling a robot according to an embodiment.

[0139] At 1001 , a demonstration for performing each of a plurality of skills is provided.

[0140] At 1002 , a robot trajectory model is trained for each skill from a demonstration, where each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states.

[0141] In 1003, a precondition model and a final condition model are trained from the demonstration of each skill, wherein the precondition model includes a probability distribution of the robot configuration before executing the skill for each initial state of the robot trajectory model of the skill, and the final condition model includes a probability distribution of the robot configuration after executing the skill for each final state of the robot trajectory model of the skill.

[0142] At 1004 , a description of a task is received, where the task includes performing a skill in a plurality of skills in sequence and / or in branches.

[0143] In 1005, a synthetic robot trajectory model is generated by:

[0144] - When two skills are to be executed sequentially in a mission, the robot trajectory model for cascading skills is as follows

[0145] ○ Include the states of the trajectory models of both skills in the composite robot trajectory model, and

[0146] Calculate the transition probability between each final state of the trajectory model of the first skill and each initial state of the trajectory model of the second skill based on the similarity between the probability distribution of the final conditional model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state of the second skill

[0147] - When executing two skills in a branch in a mission, the robot trajectory model of the skills is combined by the following

[0148] ○ Include the states of the trajectory models of both skills in the composite robot trajectory model, and

[0149] Set the transition probability between the first skill's state and the second skill's state to zero.

[0150] At 1006 , the robot is controlled to perform the task according to the synthesized robot trajectory model.

[0151] In other words, according to various embodiments, models for a robot are trained for multiple skills, and when a task involving performing those skills multiple times in a branched or sequential manner is to be performed, the models are concatenated and / or combined into a composite model. The composite model can then be used to control the robot as if it were a model for a single skill, i.e., by determining the optimal state sequence for the task (and the initial configuration of the robot and objects in which the task is to be performed) and controlling the robot accordingly.

[0152] Figure 10 The method can be performed by one or more computers including one or more data processing units. The term "data processing unit" can be understood to mean any type of entity that allows for the processing of data or signals. For example, data or signals can be processed according to at least one (i.e., one or more) specific functions performed by the data processing unit. A data processing unit can include or be formed from analog circuits, digital circuits, mixed signal circuits, logic circuits, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA) integrated circuit, or any combination thereof. Any other means of implementing the corresponding functions (which will be described in more detail below) can also be understood as a data processing unit or logic circuit. It will be understood that one or more method steps described in detail herein can be performed (e.g., implemented) by a data processing unit through one or more specific functions performed by the data processing unit.

[0153] The term "robot" may be understood to refer to any physical system having mechanical parts whose movement is controlled, such as a computer-controlled machine, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system.

[0154] Various embodiments can receive and use sensor signals from various sensors, such as video, radar, LiDAR, ultrasound, motion, and thermal imaging, to obtain, for example, sensor data regarding the state, configuration, and context of a display or system (robot and object). The sensor data can be processed. This can include classifying the sensor data or performing semantic segmentation on the sensor data, for example, to detect the presence of objects in the environment in which the sensor data was obtained. Embodiments can be used to train machine learning systems and autonomously control robots (e.g., robotic manipulators) to perform various manipulation tasks in different scenarios. In particular, embodiments are suitable for controlling and monitoring the execution of manipulation tasks, such as in assembly lines. For example, they can be seamlessly integrated with traditional GUIs used to control processes.

[0155] Although specific embodiments have been illustrated and described herein, those skilled in the art will appreciate that various alternative and / or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present invention. This application is intended to cover any adaptation or variation of the specific embodiments discussed herein. Accordingly, the present invention is intended to be limited only by the claims and their equivalents.

Claims

1. A method for controlling a robot, comprising: providing a demonstration for performing each of the plurality of skills; training a robot trajectory model for each skill from the demonstrations, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states; training a precondition model for each skill and a final condition model for each skill from the demonstrations, wherein the precondition model includes, for each initial state of the robot trajectory model of the skill, a probability distribution of the robot configuration before executing the skill, and the final condition model includes, for each final state of the robot trajectory model of the skill, a probability distribution of the robot configuration after executing the skill; receiving a description of a task, wherein the task includes performing skills from the plurality of skills in sequence and / or in branches; Generate a synthetic robot trajectory model by When two skills are to be executed sequentially in a task, the robot trajectory model of the cascaded skills is as follows: The states of the trajectory models of the two skills are included in the composite robot trajectory model, and Based on the similarity between the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state of the second skill, the transition probability between each final state of the trajectory model of the first skill of the two skills and each initial state of the trajectory model of the second skill of the two skills is calculated. When the two skills are executed in a branch in the task, the robot trajectory models of the skills are combined by The states of the trajectory models of the two skills are included in the composite robot trajectory model, and Setting the transition probability between the state of the first skill and the state of the second skill to zero; and Control the robot to perform tasks according to the synthesized robot trajectory model.

2. The method according to claim 1, wherein Each skill includes manipulation of one or more objects by the robot, and the hidden semi-Markov model for each skill is a task-parameterized hidden semi-Markov model that includes task parameters corresponding to the robot and object configuration in which the skill is applied.

3. The method of claim 2 further comprising training an effect model for each skill, the effect model comprising, for each final state of the robot trajectory model for the skill, a probability distribution of how the robot and / or object configuration changes after performing the skill relative to an initial robot and / or object configuration to which the skill was applied, wherein When two skills are to be performed sequentially in a task, the robot trajectory model for the cascaded skill includes transforming the task parameterized hidden semi-Markov model of the second skill so that its task parameters are task parameters corresponding to the robot and / or object configuration given by the effect model of the first skill.

4. The method according to claim 3 further includes affine transforming the effect model of the second skill so that its task parameters are task parameters corresponding to the robot and / or object configuration given by the effect model of the first skill.

5. The method according to any one of claims 1 to 4 further includes, when two skills are to be performed sequentially in a task, using the prerequisite model of the first skill as the prerequisite model of the cascade robot trajectory model of the skill, and using the final condition model of the second skill as the final condition model of the cascade robot trajectory model of the skill.

6. The method according to any one of claims 1 to 4 further includes, when two skills are to be performed in a branch in a task, including both a prerequisite model for the first skill and a prerequisite model for the second skill in the prerequisite model of the combined robot trajectory model of the skills, and including both a final condition model for the first skill and a final condition model for the second skill in the prerequisite model of the combined robot trajectory model of the skills.

7. The method according to any one of claims 1 to 4, wherein The similarity between the probability distribution of the final conditional model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state of the second skill is the KL-divergence of the probability distribution of the final conditional model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state of the second skill.

8. The method according to any one of claims 1 to 4, wherein The precondition model and the final condition model are task-parameterized Gaussian mixture models.

9. The method according to any one of claims 1 to 4, wherein Generating a synthetic robot trajectory model includes repeatedly concatenating robot trajectory models, concatenated robot trajectory models, and combined robot trajectory models according to the task, and combining the robot trajectory models, concatenated robot trajectory models, and combined robot trajectory so that the robot trajectory model is a trajectory model for the entire task.

10. A robot controller configured to perform the method of any one of claims 1 to 9.

11. A computer program product comprising instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 9.

12. A computer readable medium storing instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 9.

Images