Device and method for controlling robot device
Patent Information
- Application Number
- JP2022177175
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-11-05
- Filing Date
- 2022-11-04
- Publication Date
- 2025-09-12
AI Technical Summary
Existing robotic systems face challenges in selecting the correct sequence of skill primitives and parameters for complex manipulation tasks, which often involve multiple skill sequences and branching, making manual specification cumbersome and difficult.
A method for controlling robotic devices that includes providing a control model and a selection model, allowing on-the-fly training through user input to select and execute tasks, using machine learning models like TP-GMM and TP-HSMM to learn and refine skill sequences and transitions, and employing GTN for task networks to automate skill selection.
Enables robotic devices to perform complex manipulation tasks autonomously with increased reliability and reduced user effort by progressively learning and refining skill sequences and transitions, minimizing errors and damage through user intervention when necessary.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical Field]
[0001] prior art The present disclosure relates to an apparatus and method for controlling a robotic device. [Background technology]
[0002] Robot skills can be programmed through a demonstration-based learning (LfD) approach, in which a nominal plan for the skill is learned by the robot based on demonstrations. The key idea of LfD is to parameterize a Gaussian distribution by the pose of a camera monitoring the robot's workspace and the target object to be processed by the robot. Demonstration-based learning provides a fast, intuitive, and efficient framework for programming robot skills, for example, for industrial applications. However, instead of a single movement, complex manipulation tasks often involve multiple branches of skill sequences that share some common skills. Therefore, a planning process is required to generate the correct sequence of skills and their parameters under different scenarios. For example, a bin-picking task involves picking an object from a bin (depending on its location in the bin), clearing it from a corner if necessary, orienting the object so that its barcode is visible, and presenting the barcode to a scanner. Selecting the correct skill sequence is essential for flexible robotic systems across different applications. Manually specifying such transitions between skills and the associated conditions is often difficult and tedious. Summary of the Invention [Problem to be solved by the invention]
[0003] Therefore, a reliable approach for selecting the correct sequence of skill primitives and the correct parameters for each skill primitive under different scenarios is desired. [Means for solving the problem]
[0004] Disclosure of the Invention According to various embodiments, a method for controlling a robotic device is provided, the method including: for each function of a plurality of functions, providing a control model for controlling the robotic device to perform the function; providing a selection model for selecting from among the plurality of functions; performing multiple instances of a task with the robotic device, the execution including, in each execution when a function of the plurality of functions needs to be selected to perform the task instance, checking whether the selection model provides a function selection; if the selection model provides a function selection, controlling the robotic device to perform the function selected by the selection model using the control model for the selected function; if the selection model does not provide a function selection, receiving user input indicating a function selection, selecting a function according to the selection indicated by the user input, controlling the robotic device to perform the selected function according to the selection indicated by the user input using the control model for the selected function; and training the selection model according to the selection indicated by the user input.
[0005] The above-described method thus allows for on-the-fly training of a selection model during control of a robotic device by user (i.e., human) input. The provided selection model may be untrained or may only be pre-trained, and thus the selection model does not provide functional choices for at least some configurations of the robotic device (or control system, including configurations of the environment, e.g., objects). The selection model becomes increasingly reliable during the execution of a task instance, which ultimately enables the robotic device to perform complex manipulation tasks involving multiple skills and / or sequences of skill branches.
[0006] Various examples are described below.
[0007] Example 1 is a method for controlling the robot described above.
[0008] Example 2 is the method of Example 1, wherein the selection model outputs an indication of confidence for a selection of a function from a plurality of functions, and training the selection model according to the selection indicated by the user input includes adjusting the selection model to increase the confidence output by the selection model to select a function to be selected according to the selection indicated by the user input.
[0009] Thus, the robotic device gains increasing reliability in its selection for received user input until it is able to behave autonomously.
[0010] Example 3 is a method according to Example 1 or 2, wherein the selection model outputs an indication of reliability for a selection of functions from a plurality of functions, and checking whether the selection model provides a selection of functions includes checking whether the selection model outputs an indication of reliability for the selection of functions that is above a predetermined lower confidence limit.
[0011] Thus, the selection model is trained to become increasingly reliable regarding function selection until it achieves sufficient confidence in a particular choice (e.g., skill branch selection in a particular state), at which point user input is no longer necessary (e.g., no longer requested). Thus, the effort for the user decreases over time, and the robotic device may even eventually perform the task autonomously.
[0012] On the other hand, in unencountered (and therefore less reliable) situations, user input is used as the basis for selection, thus avoiding erroneous decisions that could lead to, for example, damage to the robotic device or the workpiece.
[0013] Example 4 is a method according to any one of Examples 1 to 3, wherein the function includes skills and skill branches, and the selection function is trained to provide skill selections for alternative sets of skills and branch selections for alternative sets of skill branches.
[0014] Thus, a hierarchical approach is used in which skill selection is performed and then branch selection is performed on the selected skill. Thus, the selection model may include an edge selector (e.g., to select among skills in a task network) and a branch selector (to select among branches of a skill). This makes the selection more understandable and therefore intuitive to a human user, reducing user effort and errors in selection.
[0015] Example 5 is the method of any one of Examples 1 to 4, wherein providing a control model for each function includes performing a demonstration of the function and training the control model using the demonstration.
[0016] In other words, functions (e.g., primitive skills) are trained from learning-by-demonstration, which provides an efficient approach to learning primitive skills.
[0017] Example 6 is the method of any one of Examples 1 to 5, wherein the selection model is a logistic regression model.
[0018] This allows for reliable and fast training (and retraining) from small amounts of data.
[0019] Example 7 is the method of any one of Examples 1 to 6, wherein if the selection model does not provide a selection for the function, the method includes pausing operation of the robotic device until receiving user input indicating a selection for the function.
[0020] Thus, the robot operates until its controller can no longer determine which function to proceed with, and then pauses until the user guides the robot, thereby avoiding erroneous actions that could lead to damage. Furthermore, pausing the robot indicates to the user that user input is required.
[0021] Example 8 is a robot controller configured to perform the method according to any one of Examples 1 to 7.
[0022] Example 9 is a computer program comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of Examples 1 to 7.
[0023] Example 10 is a computer-readable medium containing instructions that, when executed by a computer, cause the computer to perform the method of any one of Examples 1 to 7.
[0024] In the drawings, like reference numbers generally refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various aspects are described with reference to the following drawings: [Brief explanation of the drawings]
[0025] [Figure 1] FIG. 1 is a diagram illustrating a robot. [Figure 2] FIG. 1 illustrates robot control for a task goal according to one embodiment. [Figure 3] FIG. 10 illustrates the determination of feature vectors for edge selectors and branch selectors. [Figure 4] 1 is a flowchart illustrating a method for controlling a robotic device. DETAILED DESCRIPTION OF THE INVENTION
[0026] The following detailed description refers to the accompanying drawings, which show, by way of illustration, specific details and aspects of the present disclosure in which the invention may be practiced. Other aspects may be utilized, and structural, logical, or electrical changes may be made without departing from the scope of the present disclosure. Various aspects of the present disclosure are not necessarily mutually exclusive, as some aspects of the present disclosure may be combined with one or more other aspects of the present disclosure to form new aspects.
[0027] Various embodiments are described in more detail below.
[0028] FIG. 1 shows a robot 100 .
[0029] The robot 100 includes a robot arm 101, such as an industrial robot arm for handling or assembling workpieces (or one or more other objects). The robot arm 101 includes manipulators 102, 103, and 104 and a base (or support) 105 that supports the manipulators 102, 103, and 104. The term "manipulator" refers to a movable member of the robot arm 101, which can be moved to physically interact with the environment, for example, to perform a task. For control, the robot 100 includes a (robot) controller 106, which is configured to interact with the environment according to a control program. The last member 104 of the manipulators 102, 103, and 104 (farthest from the support 105) is also referred to as an end effector 104 and may include one or more tools, such as a welding torch, a gripping tool, or a painting device.
[0030] The other manipulators 102, 103 (closer to the support 105) can form a positioning device, which together with the end effector 104 provides a robotic arm 101 having the end effector 104 at its end. The robotic arm 101 is a mechanical arm that can provide functions similar to a human arm (possibly with a tool at its end).
[0031] The robotic arm 101 may include joint members 107, 108, 109 that interconnect the manipulators 102, 103, 104 with each other and with the support 105. The joint members 107, 108, 109 may have one or more joints, each capable of providing rotatable (i.e., rotational) and / or translational (i.e., displacement) movement for the associated manipulator relative to each other. Movement of the manipulators 102, 103, 104 may be initiated using actuators controlled by the controller 106.
[0032] The term "actuator" may be understood to be a component adapted to affect a mechanism or process in response to being actuated. An actuator may carry out commands (so-called activations) issued by the controller 106 to cause mechanical movement. An actuator, for example an electromechanical converter, may be configured to convert electrical energy into mechanical energy in response to actuation.
[0033] The term "controller" may be understood to refer to any type of logic-implementing entity, which may include, for example, circuitry and / or a processor capable of executing software, firmware, or a combination thereof stored on a storage medium, and which may, for example, in this example, issue instructions to an actuator. For example, program code (e.g., software) may configure the controller to control the operation of a system, in this example, a robot.
[0034] According to this example, the controller 106 includes one or more processors 110 and a memory 111 that stores code and data, and the processor 110 controls the robotic arm 101 based on the code and data. According to various embodiments, the controller 106 controls the robotic arm 101 based on machine learning models 112 stored in the memory 111. The machine learning models 112 include control models for skills and skill branches and selection models for selecting among the skills and skill branches.
[0035] The robot 100 can learn to perform a skill or collaborate with a human partner using a demonstration-based learning (LfD) approach. The human demonstration can be encoded by a probabilistic model (also called a control model). The controller 106 can then use the control model, also called a robot trajectory model, to generate desired robot motion as a function of the state (configuration) of both the human partner and the robot, and the robot's environment, as the case may be.
[0036] The basic idea of LfD is to fit a specified skill model, such as a GMM, to a handful of demonstrations. Suppose there are M demonstrations, each of which has a size N=Σ m T m A dataset of all observations
number
number
number
number
number
number
[0037] Unlike standard GMMs, the mixture model described above cannot be trained independently for each frame. In fact, the mixture coefficients π k is shared by all frames, and the kth component in frame p must be mapped to the corresponding kth component in the global frame. Expectation-Maximization (EM) is a well-established method for learning such models.
[0038] Once learned, TP-GMM can be used during execution to reproduce trajectories for the learned skill.
[0039] Hidden Semi-Markov Models (HSMMs) extend standard Hidden Markov Models (HMMs) by embedding the temporal information of the underlying stochastic process. That is, while in the case of HMMs the underlying hidden process is considered to be Markovian, i.e. the probability of transitioning to the next state depends only on the current state, in the case of HSMMs the state process is considered to be semi-Markovian. This means that the transition to the next state depends on both the current state and the time elapsed since entering that state. These parameters can be combined with TP-GMMs and applied to robot skill encoding to learn the spatiotemporal features of demonstrations, resulting in a task-parameterized HSMM (TP-HSMM) model. The task-parameterized HSMM (TP-HSMM) model is:
number
number
number
[0040] Consider a multi-DoF (degree of freedom) robot arm 101 in a static and known workspace. The end effector 104 of the robot arm 101 has a state r, such as its 6D pose and gripper state. Also, O={o1,...,o J There are multiple objects of interest 113, represented by {}.
[0041] A={a1,a2,...,a H We assume that there exists a set of primitive skills, denoted by}, that allow the robot to manipulate these objects. For each skill, the human user performs several kinesthetic demonstrations on the robot. In particular, for a skill a ∈ A, the set of associated objects is O a ⊆O, and the set of demonstrations is
number
number
[0042] Therefore, for each (primitive) skill a∈A, TP-HSMM
number
[0043] Through a combination of these skills, the object 113 can be manipulated by the robot arm 101 to reach different states. The robot 100 (specifically, the controller 106) is preferably trained for generic manipulation tasks, i.e., it is capable of performing multiple different instances of a single generic task. Each task instance has an initial state s0 and (at least one) desired goal state s G The system states (also called configurations) are specified by a set of s0 through s G When the status changes to , the task (instance) is resolved.
[0044] Therefore, the new task
number
[0045] FIG. 2 illustrates robot control for a task goal 201 according to one embodiment.
[0046] According to various embodiments, the controller implements an (extended) primitive skill model 202 and a GTN (geometric task network), which are trained interactively during online execution with human input.
[0047] Learning Primitive Skills As illustrated for skills in skill model 202, there are often multiple ways to execute the same skill under different scenarios (called branches). For example, there may be five different ways to pick an object from a bin, each approaching at a different angle depending on the distance to the respective boundary. To handle the branches, the controller uses a TP-HSMM model for the skill.
number
[0048] The controller 106 trains the branch selector 207 from the demonstrations 205 and online instructions 204 to select a branch 206 for each skill 202 to be executed to achieve the task goal 201. The branches 206 are, in other words, variations of a skill. For example, the skill may be picking an object from a bin, and the branch may be picking the object from the left or right depending on where the object is located in the bin. For example, if the right side of the object is located near the bin wall, the branch selector 207 selects the branch to pick the object from the left.
[0049] Consider a skill primitive a with M demonstrations (from the demonstrations provided for all skills 205) and B distinct branches. Each execution trajectory or demonstration of a skill is associated with exactly one branch.
number
number
number
number
number
number
number
number
number
number
number
number
number
number
number
number
number
number
[0050] Building a task network As mentioned above, complex manipulation tasks often involve a sequence of different skills 202 to account for different scenarios. For example, if the barcode to be scanned is on top of an object, the robot 100 needs to rotate the object (i.e., perform the rotate skill) before picking it up and presenting it to the camera (i.e., perform the present skill). This may not be necessary if the barcode is already on the bottom. A high-level abstraction of these relationships between skills is called a task network. Effective plans evolve by transitioning from one skill 202 to another until the task is solved. Manually specifying the conditions for these transitions can be particularly difficult and tedious. Therefore, according to various embodiments, the controller 106 uses a coordination structure called a geometric task network (GTN) 203, in which conditions are learned from task execution.
[0051] GTN203 has a structure defined by a triple G = (V, E, f). The set of nodes V is a subset A of primitive skills, the set of edges E ⊆ V × V contains the allowed transitions from one skill to another, and the function f:v → C maps each node to an edge selector associated with all its outgoing edges. Intuitively, (V, E) specifies how skills can be sequentially executed for a given task, while the function f(v) models the various geometric constraints between the object and the robot with respect to the outgoing edges of node v. Note that f(·) is explicitly conditioned on both the current system state and the goal state.
[0052] The complete plan for the task is the following sequence:
number
number
number
number
number
number
number
number
number
number
number
number
number
number
number
number
number
number
[0053] Figure 3 shows the skill frame.
number
number
number
[0054] The controller 106
number
number
number
number
number
number
number
number
number
number
number
number
number
[0055] In the previous two sections, we described an approach for training the extended skill model (including the branch selector 207) and the task network 203 (including the edge selector 208). The required training data are the execution trajectories of the skill and the complete plan of the task. According to various embodiments, the controller 106 generates training data for the branch selector 207 and the edge selector 208 from human instructions 204 provided during execution. This allows for on-the-fly improvement of both the skill model and the task network.
[0056] The GTN G is initialized as empty (i.e., not initially trained). The problem instance of the task, i.e.,
number
number
number
number
number
number
number
number
number
number
number
number
number
number
number
[0057] Upon execution and update of the branch selector 207, a is selected as the next skill (according to the edge selector 208). n+1 Then, the controller 106 selects
number
number
number
number
number
number
number
number
[0058] The controller 106 selects the desired next skill.
number
number
number
number
number
[0059] The system state then becomes:
number
[0060] Below is an example of the overall algorithm in pseudocode (using normal English keywords like "while", "do", etc.): [Table 1]
[0061] During online execution to solve new tasks, the algorithm runs as described above and updates the GTN 203, possibly with human input as needed.
number
number
[0062] In summary, according to various embodiments, a method is provided as shown in FIG.
[0063] FIG. 4 shows a flow chart 400 illustrating a method for controlling a robotic device.
[0064] In 401, for each of the plurality of functions, a control model (skill model or skill branching model) is provided for controlling the robotic device to execute the function.
[0065] At 402, a selection model is provided for selecting among a plurality of functions.
[0066] At 403, multiple instances of a task are executed by the robotic device (i.e., the robotic device is controlled to execute multiple instances of a task), and in each execution, when it is necessary to select one of multiple functions to execute the task instance, the execution includes checking whether a selection model provides a function selection; if the selection model provides a function selection, at 404, controlling the robotic device to execute the function selected by the selection model using a control model for the selected function; if the selection model does not provide a function selection, at 405, receiving user input indicating a function selection, selecting a function according to the selection indicated by the user input, controlling the robotic device to execute the selected function according to the selection indicated by the user input using the control model for the selected function, and training the selection model according to the selection indicated by the user input.
[0067] 4 can be used to compute control signals for controlling physical systems, collectively referred to as "robotic devices," such as, for example, robots, vehicles, home appliances, power tools, manufacturing machines, computer-controlled machines such as personal assistants, or access control systems. According to various embodiments, policies for controlling physical systems can be learned, and the physical systems can then behave accordingly.
[0068] Various embodiments may receive and use image data (i.e., digital images) from various visual sensors (cameras), such as video, radar, LiDAR, ultrasound, thermal imaging, motion, sonar, etc., as the basis for, for example, a descriptor image.
[0069] According to one embodiment, the method is computer-implemented.
[0070] While specific embodiments have been illustrated and described above, it will be apparent to those skilled in the art that various alternative and / or equivalent embodiments may be substituted for the specific embodiments shown and described herein without departing from the scope of the invention. This application is intended to cover any adaptations or variations of the specific embodiments discussed herein. Accordingly, it is intended that the present invention be limited only by the claims and equivalents thereof.
Claims
1. 1. A method for controlling a robotic device, comprising: providing, for each of the plurality of functions, a control model for controlling the robotic device to execute the function; providing a selection model for selecting among said plurality of functions; executing, by the robotic device, multiple instances of a task, the execution including, in each execution, when it is necessary to select one of the multiple functions to execute the task instance, checking whether the selection model provides a function selection; if the selection model provides a selection of functions, using a control model for the selected function to control the robotic device to perform the function selected by the selection model; if the selection model does not provide a selection for a function, receiving a user input indicating a selection for a function, selecting a function according to the selection indicated by the user input, controlling the robotic device using a control model for the selected function to perform the selected function according to the selection indicated by the user input, and training the selection model according to the selection indicated by the user input; A method comprising:
2. the selection model outputs an indication of confidence in a selection of a function from the plurality of functions; training the selection model according to the selections indicated by the user input includes adjusting the selection model so that a confidence output by the selection model increases for selecting functions selected according to the selections indicated by the user input; The method of claim 1.
3. the selection model outputs an indication of confidence in a selection of a function from the plurality of functions; checking whether the selection model provides a function selection includes checking whether the selection model outputs a confidence indication for the function selection that is above a predetermined lower confidence limit; The method of claim 1.
4. The function includes a skill and a branch of the skill; the selection function is trained to provide skill choices for alternative skill sets and branch choices for alternative skill branch sets; The method of claim 1.
5. Providing a control model for each function is performing a demonstration of said function; training the control model using the demonstrations; The method of claim 1 , comprising:
6. The selection model is a logistic regression model. The method of claim 1.
7. If the selection model does not provide a function selection, pausing operation of the robotic device until receiving user input indicating a function selection. The method of claim 1 , comprising:
8. A robot controller configured to carry out the method of any one of claims 1 to 7.
9. A computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 7.
10. A computer-readable medium containing instructions which, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7.