Method of training machine learning model for controlling robot to manipulate object

Abstract

According to various embodiments, a method for training a machine learning model for controlling a robot is described, the method comprising: for each training data element in a set of training data elements, where each training data element has training input information, the training input information relates to a position of a surface point of a corresponding object and one or more possible approaching directions of the robot for manipulating the object, determining one or more contact points by means of a machine learning model, determining weighted parameter values of a mixed distribution of a spherical distribution of the approaching directions by means of the machine learning model, and training the machine learning model to reduce losses including each training data element and a proximity direction loss component for each possible proximity direction.

Description

Technical Field

[0001] The present disclosure relates to methods for training a machine learning model used to control a robot to manipulate an object. Background Art

[0002] Picking (i.e. grasping) objects is an important problem in robotics. Recent research uses machine learning methods to allow model-free grasping of unknown objects in unstructured environments. Many solutions focus on solving the three-degree-of-freedom (DoF) grasping problem for parallel grippers and single-suction cup grippers, and output the grasp position together with a success metric. While this simplifies the grasping problem, it requires assumptions about the orientation of the gripper, which, for example, limits the grasping design (i.e. approach direction) of parallel grippers to top-down, and additionally limits the applicability of these solutions to specific gripper types.

[0003] Therefore, what are desired are approaches to automatically control a robot for robotic picking (or general manipulation) that allow flexibility in the approach direction of the gripper.

[0004] The public document “The power spherical distribution” (De Cao, N. and Aziz, W., 2020, arXiv preprint arXiv:2006.04437) described below as reference 1 describes the power spherical distribution. Summary of the invention

[0005] According to various embodiments, there is provided a method for training a machine learning model for controlling a robot (for manipulating objects, in particular for picking up objects), comprising:

[0006] for each training data element in the set of training data elements, wherein each training data element has training input information (e.g., a 3D point cloud or a depth image (including associated intrinsic camera parameters) or information derived therefrom, such as a 3D voxel normal grid) relating to a location of a surface point of a corresponding object and one or more possible approach directions for the robot to manipulate the object,

[0007] o determining one or more contact points on the surface of the object by a machine learning model for use in manipulating the object by an end effector (e.g., a gripper) of the robot,

[0008] o determining, by a machine learning model, a weighted parameter value (i.e., a mixing parameter value ω) of a mixture of spherical distributions (i.e., for example, a mixture of power spherical distributions) of approach directions for manipulating an object, wherein each of the spherical distributions is assigned a corresponding end-effector orientation angle (and the weighted parameter of the spherical distribution is an estimate of the approach direction corresponding to the end-effector orientation angle); and

[0009] Training a machine learning model to reduce a loss that includes an approach direction loss component for each training data element and each possible approach direction, wherein the approach direction loss component decreases as the probability of the mixture distribution providing the possible approach direction increases, wherein a direction parameter (i.e., a mean direction vector) for each of the spherical distributions is set according to the end effector orientation angle assigned to the spherical distribution (i.e., the direction vector is set so that it corresponds to the assigned end effector orientation angle and lies in a plane whose surface normal is consistent with the direction of a basis vector that describes the direction between two contact points (especially during grasping) and can be determined as a ground-truth from the assigned basis vectors from the training data elements).

[0010] The set of training data elements is typically a batch consisting of multiple training data elements (but in extreme cases, a single training data element).

[0011] The method described above allows training a machine learning model to densely predict the grasping quality for pairs of grasping contact points (e.g. together with the value of the gripper opening width) and for multiple grasping orientations (and thus approach directions) for each contact point pair. The dense prediction of grasping points with multiple orientations (if necessary orientations and / or contact points that are close to each other, hence "dense") allows for more successful grasping of objects even in the presence of additional constraints (e.g. the reachability of the robot): the dense prediction of multiple orientations for each contact point pair of a parallel gripper makes the scheme very flexible in situations where it is restricted to the determined gripper orientations (e.g. when removing objects from containers with a narrow grasping space and when there are possible limitations on the robot's kinematic reachability). Here, the additional gripper orientations make it possible to grasp objects even in the presence of these constraints and can increase the number of graspable objects in these scenarios.

[0012] The processing described above also allows grasping to be predicted for objects that are not (completely) visible, which is very important in scenarios where only one camera is used for parallel grasping (e.g. when taking objects out of a container): the prediction of contact point pairs is possible even if they are not (two) visible pixels, thus allowing grasping at points that are not visible in the camera image. This is particularly important in grasping scenarios with only one viewpoint, such as when taking objects out of a container, where the camera viewpoint is typically limited to viewing from above due to the geometry of the container, and thus only a single-view camera image is available as input for the machine learning model, in which case the grasped object is only partially captured from one or more sides, depending on the geometry of the object. However, for parallel grippers, contact points on opposite sides of the object geometry must be predicted, which are mostly not captured from the single camera viewpoint. Therefore, the prediction of grasping points on non-visible parts of the object is also decisive for the successful grasping of a large number of objects in such "single-view" scenarios with limited spatial accessibility (e.g. when taking objects out of a container).

[0013] The prediction of a collision-free grasp (grasp pose (including approach direction) plus gripper opening width) allows the predicted grasp to be executed directly without additional collision testing.

[0014] Various examples are given below.

[0015] Embodiment 1 is a method for training a machine learning model for controlling a robot as described above.

[0016] Embodiment 2 is a method according to embodiment 1, wherein each training data element contains at least one direction vector between contact points (as reference facts) (for example, two points corresponding to two points at which the end effector grasps an object during corresponding grasping), and wherein the loss also includes a basis vector loss component for at least one of the determined contact points for each training data element, and the basis vector loss component decreases as the probability that the spherical distribution determined by the machine learning model for the basis vectors of the determined contact points is consistent with the spherical distribution of the basis vectors assigned to the determined contact points and contained in the training data elements increases.

[0017] Therefore, the machine learning model is also trained to correctly predict the direction vectors (also referred to as basis vectors in this article) between the contact points.

[0018] Embodiment 3 is a method according to embodiment 1 or 2, wherein each training data element has one or more contact points (as reference facts), and wherein the method includes determining associated paired contact points for each training data element and each determined contact point (i.e., in particular, all pairs of contact points grasped using a parallel gripper, wherein the paired contact points of a contact point are derived from the opening width of the contact point in the direction of the basis vector), and wherein the loss also includes a width loss component for each training data element and each determined contact point, and the width loss component decreases as the distance from the determined associated paired contact point to the one or more paired contact points of the training data element decreases.

[0019] Therefore, the machine learning model is trained to correctly predict the contact point pairs.

[0020] Embodiment 4 is a method according to any one of embodiments 1 to 3, wherein each training data element has at least one contact point quality assessment (as a benchmark fact), and the method includes determining a quality assessment for each determined contact point through a machine learning model, and wherein the loss also includes a quality loss component for each training data element and each determined contact point, and the quality loss component increases as the difference between the quality assessment determined for the determined contact point and the contact point quality assessment that the training data element has for the associated contact point (for example, the quality assessment of one or more of the following contact points adjacent to the determined contact point: for which the training data element has a contact point quality assessment).

[0021] Thus, the machine learning model is trained to correctly predict the quality of a contact point (and thereby, for example, a grasp).

[0022] Embodiment 5 is a method according to any one of Embodiments 1 to 4, wherein each training data element includes at least one (benchmark fact) position of a contact point on the surface of the object to be manipulated and at least one position of a reference point (e.g., a grasping center) of the end effector (as a benchmark fact), and the method includes classifying spatial regions into spatial regions with contact points and without contact points and with reference points and without reference points through a machine learning model, and wherein the loss also includes a classification loss for the classification of each training data element as a contact point-reference point loss component.

[0023] The classification may include soft value outputs (as is common for classification tasks), based on which a classification loss is determined. Thus, a machine learning model is trained to predict the appropriate position and orientation of the end effector.

[0024] Embodiment 6 is a method for controlling a robot to manipulate an object to be manipulated, comprising:

[0025] feeding information about the positions of surface points of the object to be manipulated into a machine learning model, the machine learning model being trained to output, in response to the feeding of the information about the positions of the surface points of the object, a weighted parameter value of a mixture of spherical distributions of contact points on the surface of the object for manipulating the object by an end effector of the robot and approach directions for manipulating the object, wherein each of the spherical distributions is assigned a corresponding end effector azimuth;

[0026] by comparing with a predetermined threshold value (e.g. 0.9 in a weighting parameter value range of 0 to 1) (and, if necessary, by comparing a quality value that may also be output by the machine learning model with a predetermined quality threshold value), selecting a weighting parameter value that is above the predetermined threshold value from the weighting parameter values ​​(and, if necessary, the quality values) output by the machine learning model in response to the supplied information; and

[0027] Controlling the robot to manipulate the object to be manipulated by moving the robot's end effector toward the object along an approach direction given by the end effector azimuth angles assigned to the following spherical distribution, which is weighted by the selected weighting parameter value in the mixed distribution. (For example, by moving the robot's end effector toward the object at an end effector position and orientation given by the contact points output by the machine learning model, the associated output basis vectors and the approach direction (and, if necessary, also the gripper opening width output by the machine learning model (to avoid collisions between gripper fingers and the object)).

[0028] In this way, the robot can be controlled without collision.

[0029] Embodiment 7 is the method according to embodiment 6, wherein the machine learning model is trained according to any one of embodiments 1 to 5.

[0030] Embodiment 8 is a robot control device, which is configured to execute the method according to any one of Embodiments 1 to 7.

[0031] Embodiment 9 is a computer program having instructions, which, when executed by a processor, cause the processor to perform the method according to any one of embodiments 1 to 7.

[0032] Embodiment 10 is a computer-readable medium storing instructions, which, when executed by a processor, cause the processor to perform the method according to any one of embodiments 1 to 7. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In the accompanying drawings, similar reference numerals generally refer to the same components in all the different views. The accompanying drawings are not necessarily drawn to scale, with emphasis instead generally placed on demonstrating the principles of the invention. In the following description, various aspects will be described with reference to the attached drawings.

[0034] Figure 1 A robot with an end effector (eg, a gripper), a camera, and a control device is shown, as well as an object (eg, for grasping) and a container.

[0035] Figure 2 Figure 2 shows a machine learning model for predicting grasp pose, associated gripper opening width, collision probability, and grasp quality.

[0036] Figure 3 A gripper is shown grasping an object.

[0037] Figure 4 The figure shows (for example, reference Figure 2 Training of machine learning models described in [ 14 ].

[0038] Figure 5 The figure shows that by means of (for example, reference Figure 2 Inference performed by a trained machine learning model (described in [ 17 ] ).

[0039] Figure 6 A flow chart is shown illustrating a method for training a machine learning model used to control a robot to manipulate an object, according to one embodiment. DETAILED DESCRIPTION

[0040] The following detailed description relates to the attached drawings, which show that the disclosure can implement the specific details and various aspects of the present invention for the purpose of explanation. Other aspects can also be used and structural, logical and electrical changes can be made without departing from the scope of protection of the present invention. Various aspects of the present disclosure are not necessarily mutually exclusive, because some aspects of the present disclosure can be combined with one or more other aspects of the present disclosure to form new aspects.

[0041] Each example is described in more detail below.

[0042] Figure 1 A robot 100 is shown.

[0043] The robot 100 comprises a robot arm 101, for example an industrial robot arm for manipulating or installing a workpiece (or one or more other objects). This is only used as an example here, and the processing method described below is not limited to performing a gripping process with a robot arm, but can also be used for other robot kinematics (for example a parallel kinematic robot). The robot arm 101 comprises movable arm elements 102, 103, 104 and a base (or support) 105 carrying the arm elements 102, 103, 104. The term "movable arm element" refers to the movable components of the robot arm 101, the operation of which allows physical interaction with the environment, for example to perform a task. For control, the robot 100 comprises a (robot) control device 106, which is designed to realize the interaction with the environment according to a control program. The last arm element 104 of the arm elements 102, 103, 104 (the farthest from the support 105) is also called the end effector 104 and can include one or more tools, such as a welding gun, a gripping tool, a paint sprayer or the like.

[0044] The other arm elements 102, 103 (closer to the support 105) may form positioning means, thereby providing together with the end effector 104 a robot arm 101 having an end effector 104 at its end. The robot arm 101 is a mechanical arm (possibly with a tool at its end).

[0045] The robot arm 101 may include joint elements 107, 108, 109, which connect the arm elements 102, 103, 104 to each other and to the support 105. The joint elements 107, 108, 109 may have one or more joints, which can provide the associated arm elements with a rotational movement (i.e., rotational movement) and / or a translational movement (i.e., displacement) relative to each other. The movement of the arm elements 102, 103, 104 can be initiated by means of actuators controlled by the control device 106.

[0046] The term "actuator" may be understood as a component designed to implement a mechanism or process in response to its actuation. An actuator may implement a command (so-called activation) created by the control device 106 in a mechanical movement. An actuator (e.g., an electromechanical converter) may be designed to convert electrical energy into mechanical energy in response to the activation of the actuator.

[0047] The term "control device" may be understood as any type of logical implementation entity (including one or more computers), which may include, for example, circuits and / or processors capable of executing software, firmware, or a combination thereof stored in a storage medium and issuing commands (e.g., to actuators in this example). For example, the control device may be configured by program code (e.g., software) to control the operation of the system (in this example, the robot).

[0048] In this example, the control device 106 includes one or more processors 110 and a memory 111 storing codes and data, based on which the processor 110 controls the robotic arm 101. According to different embodiments, the control device 106 controls the robotic arm 101 based on a machine learning model 112 stored in the memory 111.

[0049] According to various embodiments, the machine learning model 112 is designed and trained to enable the robot 100 to recognize manipulation gestures on one or more objects 113, with which the robot 100 can pick up one or more objects 113 (or otherwise interact with them, such as painting them).

[0050] For example, the robot 100 may be equipped with one or more cameras 114 that allow the robot to record images of its workspace. For example, the camera 114 is fastened to the robot arm 101 so that the robot can take images of the object 113 from different angles by moving its robot arm 101 back and forth. However, the camera 114 may also be Figure 1 As shown in FIG, it is fixedly mounted in a robot cell to capture the object to be grasped.

[0051] According to various embodiments, the machine learning model 112 is a neural network, and the control device 106 transmits input data to the neural network based on one or more digital images of the object 113 (depth images with optional color images and intrinsic camera parameter values ​​or point clouds with optional color images), and the neural network determines the appropriate posture of the end effector 104 (hereinafter, exemplarily assumed to be a gripper, and the "grasp" determined by the machine learning model is discussed accordingly). The machine learning model can also determine the quality value of such a grasp. For example, the quality value output by the neural network for the grasp is a probability value, which describes the expected probability of successfully grasping (or manipulating in general) using the corresponding grasp. In addition, the machine learning model can also determine the collision value of such a grasp. For example, the collision value output by the neural network for the grasp is a probability value, which describes the expected probability of grasping (or manipulating in general) without colliding with other objects, boxes or the environment when grasping (or manipulating in general) using the corresponding gripper. These probability values ​​can be considered in subsequent processing to ultimately control the robot to perform grasping (or manipulating in general).

[0052] According to various embodiments, a process (particularly an architecture) is provided for end-to-end training of a machine learning model (e.g., machine learning model 112) (and performing associated inference, i.e., prediction) such that the machine learning model is capable of predicting the grasping (or its collision probability value and quality value) of an end effector 104 (e.g., a gripper with six degrees of freedom (for grasping posture), particularly a parallel gripper), wherein the machine learning model takes into account multiple possible orientations.

[0053] For example, a machine learning model is trained so that it can densely predict contact point pairs (on the surface of the object to be grasped) and gripper opening widths, collision probability values, and grasping quality values ​​for multiple grasping positions for each contact point pair based on a point cloud of a grasping scenario (e.g., taking an object out of a container). In particular, the prediction of dense contact points and multiple grasping positions for each contact point pair provides a large solution space that allows finding a successful grasp (grasping posture plus gripper opening width) even when additional constraints (such as the reachability of the robot) prevent multiple grasps from being performed.

[0054] The process described herein is not limited to a particular design of parallel gripper. The machine learning model is, for example, a neural network (e.g., a convolutional network), for example having a Unet architecture, and may also be a 3D convolutional network (3D CNN) and / or having or comprising a 3D-Unet architecture. The input to the machine learning model is, for example, image data (for the grasping scene, i.e., the control status of the object to be grasped), which is represented in the form of a 3D voxel grid (e.g., by the representation of the surface normal vector of each voxel).

[0055] For example, the machine learning model includes a 3D convolutional network (e.g., having a 3D-Unet architecture) that maps each voxel feature of a 3D voxel grid (e.g., each normal vector of each voxel) to an output of a 3D voxel grid having the same resolution, where in this case, for each voxel, a probability of whether the voxel contains a contact point, a reference point, or does not contain a contact point / reference point is output.

[0056] According to various embodiments, a machine learning model is trained with the aid of a loss function based on a power spherical distribution or a mixed power spherical distribution. However, loss functions based on other spherical distributions (e.g., Mises-Fisher distribution) may also be used. Accordingly, the mixed distribution is not necessarily a mixed power spherical distribution, but may also be a mixture of other spherical distributions.

[0057] Figure 2The diagram shows a machine learning model 200 for predicting a gripping pose, an associated gripper opening width, and a gripping quality. The machine learning model 200 corresponds to the machine learning model 112, for example, and is, for example, a neural network.

[0058] The input 201 of the machine learning model 200 is a normal grid G, i.e., a 3D voxel grid (i.e., the 3D space considered is divided into voxels), in which each voxel contains the surface normal of the point on the surface of an object (e.g., the object to be grasped 113) that is closest to the center of the voxel. The normal grid is obtained, for example, from a 3D point cloud by preprocessing (e.g., determined from a camera image with depth information and intrinsic camera parameter values).

[0059] The normal grid G ​​is first processed by a 3D convolutional neural network (3D-CNN) 202 of the machine learning model 200, which has multiple encoder layers, each of which (in addition to the convolutional layers) can contain a gate function f. The gate function is optional and can be used to improve performance in cases where the contact point prediction is sparse, such as in a grasping scenario of taking an object out of a container, where the object to be grasped only occupies a small part of the camera's field of view. Here, the gate function is used as a weighting function for the output features of each layer to invalidate unimportant features.

[0060] The result of 3D-CNN is a voxel grid G C Each voxel of this grid is represented by a 3D vector (l1, l2, l3), where l1, l2, l3 are the confidence values ​​of the voxel being a "null point", "contact point", and "grasp center" (or corresponding to or containing such points, since the voxel covers the entire spatial region), respectively. Each voxel is thus identified as a "null point", "contact point", or "grasp center" (depending on which confidence value is the highest).

[0061] Figure 3 A gripper 300 is shown gripping an object 301 .

[0062] The gripper 300 grasps the object 301 at two contact points (i.e., the contact point pair (c, c')) that are spaced apart from each other by the gripper opening width w (the distance between the gripper's gripping fingers 302, 303) in the direction of the "base" vector b. The orientation of the gripper 300 in the gripping posture in which the gripper 301 is grasped is determined by the vector a (which points in the direction in which the gripper fingers 302, 303 extend; this vector can be understood as the approach direction, since the gripper typically moves in this direction at least shortly before grasping).

[0063] The gripper has a gripper center t, and the point t'=c-la is called the gripper center (or also called the reference point on the gripper) (where l is the position of the contact point c along the gripper finger (here the left side) touching the contact point).

[0064] Defining the grasp center t′ as being near the contact point allows the grasp center t′ and the contact point to be in the same receptive field, which supports the training of 3D-CNN.

[0065] Then, the machine learning model 200 pairs of data from G C The contact point P of the voxel marked as "contact point" in C Sampling is performed, that is, for each voxel in a plurality of (possibly all) voxels identified as “contact points”, one or more contact points in the voxel are sampled respectively (for example, randomly sampling in the voxel or by subdividing the voxel).

[0066] Then, MLP 203 obtains as input the multi-resolution features of the contact points obtained by concatenating the features provided by the layers (for different resolutions) of the 3D-CNN network (i.e., the feature maps output by the encoder layers of the 3D-CNN network). Since these encoder layers provide features for each voxel rather than for each contact point, the feature values ​​of the contact points are determined using, for example, (e.g., trilinear) interpolation.

[0067] The MLP 203 has multiple layers, and the last layer of the MLP 203 outputs the parameters ν, κ, and ω of a mixed power spherical (PS) distribution, the gripper opening width w, and the mass q for each contact point sampled. In this case, ν (direction parameter) and κ (divergence parameter) are parameters of the power spherical distribution (as described in reference 1, but labeled as μ and κ there). The power spherical distribution given by ν and κ (for the corresponding contact point and the corresponding approach direction a) is the prediction (or estimate) of the machine learning model for the distribution of the basis vector b. Here, the power spherical distribution can be understood as a distribution of the basis vector b on a hypersphere in 3D space (as a counterpart to the Gaussian distribution). The parameter ν then describes the average (or best predicted) basis vector, and κ is a measure of the divergence around the average basis vector (the lower the κ, the greater the uncertainty in the prediction).

[0068] The vector ω has n r entries, where each entry ω i is the coefficient of the power spherical distribution in the mixed power spherical distribution. The mixed power spherical distribution is a distribution close to the vector a, which can be divided into n r One of the angles (e.g., sampled from [0,π] or set in [0,π]) lies in the plane perpendicular to ν. A mixed power spherical distribution is a power spherical distribution (for n rangles) with a power spherical distribution for each of the corresponding ω i Weighted, that is, the sum weighted in this way is a mixed power spherical distribution of approach vectors. Each of the power spherical distributions in the sum is parameterized by the approach direction corresponding to the respective angle and κ (see also the loss term L below). a formula).

[0069] Each weight ω i Evaluate the admissibility of the corresponding approach vector (in the range [0; 1]: for a safe collision ω i = 0, for safe collision-free ω i =1). Thus, ω is both a parameter of the mixed power spherical distribution (in training) and a measure or assessment of the admissibility (collision-free or non-collision-free; in inference, this value lies between 0 and 1 and describes the probability of a collision-free grasp, i.e., by comparing with a threshold, it can be determined whether a grasp is evaluated as a collision-free grasp or a collision-prone grasp). In the 3D case, the mixed power spherical distribution can be viewed as the counterpart of the Gaussian mixture distribution.

[0070] The parameter q is an evaluation of the quality of the corresponding grasp contact point, that is, a measure of the probability of a stable grasp at this contact point. Since the stability of the grasp does not depend on the grasp direction or the approach vector, all n r Each crawl has the same quality assessment q.

[0071] This approach can also be used for other spherical distributions (such as the Mises-Fisher distribution).

[0072] In order to merge the information of MLP 203 with 3D-CNN 202 again and improve the prediction of the corresponding contact points, the output values ​​of mass q and collision probability ω are combined in the downstream post-processing (PP) step. i The output value of is adapted to the position of the contact point within the corresponding voxel. To this end, a contact point interpolation score is determined for the sampled contact point by trilinear interpolation, which is multiplied by the corresponding quality value q of the associated contact point. In addition, the actual position of the grasp center is determined for the corresponding contact point, and the corresponding grasp center interpolation score is determined by trilinear interpolation of the grasp center grid, which is multiplied by the corresponding weighting factor ω of the power spherical distribution. i , these weighting factors represent the collision degrees of freedom for different approach angles of the contact point.

[0073] The output 204 of the machine learning model 200 is the sampled contact points P C , parameters ν, κ and ω, the gripper opening width w and the mass q at each contact point, and theC Information about which voxels are identified as “grasp centers” and “contact points”.

[0074] Figure 4 The diagram illustrates the training of machine learning model 400 (corresponding to machine learning model 200).

[0075] Each training data element 401 of the training data set contains a training input for a control situation and ground truth data (or “ground truth” data, i.e., “labels” or target outputs), from which a normal mesh G can be generated by preprocessing (e.g., from a depth image with intrinsic camera parameter values ​​or from a 3D point cloud). The ground truth data contains k possible grasps for the control situation scene, which are represented as contact points Basis Vectors Approach vector Grip Width And the corresponding crawl quality The training data may also include gripper parameters such as finger length l and gripper height d (eg, from the top of the gripper to the ends of the fingers).

[0076] The training dataset can be generated, for example, with the aid of a simulation environment. This requires simulating the grasping object, the camera and additional components (e.g., containers in the case of sorting containers), as well as selecting a number of grasping candidates and evaluating their grasping quality (e.g., using a physics simulator).

[0077] The machine learning model 400 to be trained is now as shown in the reference Figure 2 During training, G is calculated C The grab center ω i To do this, we use the n values ​​of each sampled contact point. r n close to the vector calculation r G C In order to determine the confidence estimate ω of the grasp center by interpolation (from l3) i The interpolated confidence estimate of the grasp quality q of the contact point is determined in the same way by inserting the sampled contact points and performing interpolation (l2).

[0078] In order to calculate each sampled contact point P C The loss contribution of P C Contact points within the surrounding radius r The set N(P) of contact points is given in the ground truth of the corresponding training data element. C ):

[0079]

[0080] Based on the corresponding N(P C ), the sampled contact points are divided into two groups: points in set C have at least one benchmark fact neighboring point (i.e., N(P C )>1). Collection The points in have no neighbors in the set of contact points of the ground truth for the corresponding training data element:

[0081]

[0082] As reference Figure 2 As explained, the machine learning model 400 outputs parameters ν and κ for each sampled contact point and approach vector, which are the parameters of the power spherical (PS) distribution of the basis vector b of the contact point. To form the distribution of the approach vector a and calculate the loss of the gripper width w, the following equation is used to derive the ground truth: Make an estimate:

[0083]

[0084] The approach direction distribution at each contact point is modeled as a mixed power spherical (PS) distribution, which is expressed as κ and ω are parameterized. T is a transformation that Generate n r Approaching direction (corresponding to the pre-given angular step size).

[0085] Belongs to the contact point pair (attached to the contact point P C ) of the contact point P C′ (also called a paired contact point belonging to a contact point) can be based on and P C And calculated using the following equation:

[0086]

[0087] As reference Figure 2 As described above, the machine learning model 400 also predicts the voxel grid G ​​during training. C The grasp center and contact points are also used as input to the following loss function.

[0088] According to various implementations, the loss for a training data element is the sum of five loss terms:

[0089] L=βL b +αL a +L w +θL q +γLG C

[0090] where α, β, γ and θ are the weights of the individual loss components.

[0091] These loss terms are described in detail below.

[0092] 1. Loss of basis vector L b : The negative log-probability of sampling the basis vectors of the ground truth from the power spherical distribution (parameterized by (ν,κ)) provided by the machine learning model 400 for the training data elements.

[0093]

[0094] 2. Approach loss L a : From a mixture of power spherical distributions (using ( The negative log-probability of a close vector of the ground truth sampled in (κ, ω) is parameterized.

[0095]

[0096] 3. Width loss L W :P C′ and The distance between

[0097]

[0098] Here, D can be any geometric distance function, such as Euclidean distance.

[0099] 4. Quality loss L q :q and The difference between

[0100]

[0101] Here, D can be any distance function, such as squared Euclidean distance.

[0102] 5. Loss of the grasp center and contact point grid : The loss can be any classification loss (e.g. focal loss).

[0103] To train the machine learning model 400, the parameters of the machine learning model 400 (including the parameters of the 3D-CNN 202 and the MLP 203) are updated by back-propagation (in terms of the loss L, e.g., summing or averaging a batch of training data elements and separately sampled contact points).

[0104] Figure 5 The figure shows a trained machine learning model 500 (e.g., reference Figure 2 described).

[0105] The input 501 for inference consists of a 3D point cloud (eg, obtained from a depth image from a unique viewpoint with intrinsic camera parameters) transformed into a normal mesh G.

[0106] Machine Learning Model 400 is now available as a reference Figure 2 During inference, G is computed C The grab center ω i To do this, we use the n values ​​of each sampled contact point. r n close to the vector calculation r G C In order to determine the confidence estimate ω of the grasp center by interpolation (from l3) i The interpolated confidence estimate of the grasp quality q of the contact point is determined in the same way by inserting the sampled contact points and performing interpolation (l2).

[0107] The output 502 of the machine learning model 500 is the contact point P C and mixed power spherical distribution parameters ν, κ, and ω, as well as the gripper width w and grip quality q at each contact point.

[0108] The output 502 can be transformed into a 6-DoF robot gripping pose representation by means of a transformation 503 (which can be performed directly by the robot control, for example). For this purpose, the gripper center t and the gripper orientation matrix R are required. The relationship between the gripper center t and the contact point c is given by the following equation:

[0109] t=c-ad+0,5wb

[0110] The orientation matrix R can be calculated based on the approach vector a and the basis vector b as follows:

[0111] R = (b, a × b, a)

[0112] Here, d is the gripper height and b corresponds to ν, which is part of the output 502 of the machine learning model. As in training, during inference, the transformation T(.) is applied to each ν to produce n for each contact point. r The entries of ω (i.e., the admissibility evaluation of each approach vector) can be transformed into binary evaluations by setting a threshold to filter out collision-free grasps.

[0113] The final result 504 is a list of collision-free grasps, each described by a translation vector t, a rotation matrix R, a gripper width w, and a grip mass q.

[0114] Alternatively, in addition to outputting the captures in 504, the admissibility assessment ω for each capture may also be output together. i(without converting to a binary evaluation) to use the probability of a collision-free grasp represented by the evaluation, for example in downstream robotic motion planning.

[0115] In summary, according to various embodiments, there is provided Figure 6 The method shown in .

[0116] Figure 6 A flowchart 600 is shown illustrating a method for training a machine learning model used to control a robot to manipulate an object, according to one embodiment.

[0117] At 601, for each training data element in a set of training data elements, wherein each training data element has training input information (e.g., a point cloud or a depth image with intrinsic camera parameter values ​​or information derived therefrom, such as a 3D voxel normal grid) relating to a surface point position of a corresponding object and one or more possible approach directions for a robot to manipulate the object, e.g., including an associated grasp quality (or probability of successful grasp),

[0118] At 602, determining, by a machine learning model, one or more contact points on a surface of an object for use in manipulating the object by an end effector (e.g., a gripper) of a robot,

[0119] In 603, a weighted parameter value (i.e., a mixed parameter value ω) of a mixture distribution of spherical distributions (i.e., for example, a mixed power spherical distribution) for the approach direction of the manipulated object is determined by the machine learning model. Here, each spherical distribution is assigned a corresponding end effector azimuth. The weighted parameter value of the spherical distribution can be regarded as an evaluation of the approach direction corresponding to the end effector azimuth. According to various embodiments, the contact point mesh and the grasp center mesh are also output by the machine learning model.

[0120] In 604, a machine learning model is used to reduce a loss, the loss including an approach direction loss component for each training data element and each possible approach direction, the approach direction loss component decreasing as the probability of the mixed distribution providing the possible approach direction increases, wherein the direction parameter (i.e., the average direction vector) of each spherical distribution is set according to the end effector orientation angle assigned to the spherical distribution (that is, the direction vector is set so that it corresponds to the assigned end effector orientation angle and lies in the following plane: the surface normal of the plane is consistent with the direction of the basis vectors that describe the direction between two contact points (especially when grasping) and can be determined as the ground truth from the assigned basis vectors from the training data elements). Training can be performed according to a process for training a machine learning model (i.e., for example, a neural network or comprising one (or more) such neural networks), typically back propagation.

[0121] refer to Figure 6 The described process can be used in conjunction with software for controlling a robotic gripper, e.g. for a robotic application for retrieving objects from one or more containers. Here, the process allows prediction of collision-free grasping candidates using information about gripper position, gripper orientation and gripper opening width and the corresponding grasp quality (e.g. in the form of an expected probability of a successful grasp). The output information can be used in further post-processing steps, e.g. for planning a robot trajectory to ultimately execute a grasping process in a robotic application.

[0122] A grasp containing training data elements is, for example, represented by the following items: contact points (between the object and the grasping fingers), basis vectors (the basis vectors describe the direction between the two contact points of the grasp), the approach vector of the grasp (the approach vector describes the approach direction perpendicular to the basis vector), the gripper opening width (the gripper opening width describes the distance between the two fingers of the parallel gripper) and the associated grasping quality (the grasping quality describes the probability of a successful grasp).

[0123] For example, a spherical distribution at a contact point (e.g., for a basis vector describing the direction between two contact points or for a proximity vector describing the direction perpendicular to the basis vector when grasping) is determined with the aid of a set of contact points and associated directions assigned to the contact point (e.g., basis vectors and proximity vectors of training data elements from a benchmark factual data set).

[0124] For example, the machine learning model also determines spherical distribution parameters (e.g., distribution parameter values ​​ν and κ of a power spherical distribution) for basis vectors used to manipulate objects (the basis vectors describe the directions between two pairs of contact points of the gripper).

[0125] The loss may also include additional loss components (loss for basis vectors, approach vectors, gripper opening width, grasp quality, grasp contact points, and grasp center). Thus, a machine learning model may, for example, be trained to reduce the loss as a weighted sum of the loss for basis vectors (described by the parameter values ​​ν and κ of a power spherical distribution), the loss for approach vectors (described by the mixture distribution value ω of different power spherical distributions for different approach vectors for each contact point), the loss for gripper opening width (described by the distance between contact points for a contact point pair), the loss for grasp quality (described by the probability of a successful grasp), and the losses for contact points and grasp center.

[0126] Figure 6The method can be performed by one or more computers having one or more data processing units. The term "data processing unit" can be understood as any type of entity that enables processing of data or signals. Data or signals can be processed, for example, according to at least one (i.e., one or more) specific functions performed by the data processing unit. The data processing unit may include or be constructed from the following: an analog circuit, a digital circuit, a logic circuit, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an integrated circuit of a field programmable gate array (FPGA), or any combination thereof. Any other means for implementing the corresponding functions described in more detail herein may also be understood as a data processing unit or a logic circuit device. One or more method steps described in detail here may be performed (e.g., implemented) by a data processing unit through one or more specific functions performed by the data processing unit.

[0127] Therefore, according to various embodiments, the method is particularly computer-implemented.

[0128] The machine learning model can be applied to sensor data determined by at least one sensor after training. For example, the machine learning model can be used to generate control signals for the robot by transmitting sensor data related to the robot and / or its environment to the robot after training.

[0129] Various embodiments may receive and use time series of sensor data from various sensors such as video, radar, lidar, ultrasound, motion, thermal imaging, etc. The sensor data may be measured or simulated over a time period. The embodiments may be used to train machine learning systems and control robots (e.g., autonomously by robotic manipulators) to achieve different manipulation tasks in different scenarios. In particular, the embodiments may be applied to control and monitor the execution of manipulation tasks, such as on an assembly line. The embodiments may be seamlessly integrated, for example, with traditional graphical user interfaces for controlling processes.

Claims

1. A method for training a machine learning model (112, 200, 400, 500) for controlling a robot (100), comprising: For each training data element (401) in the set of training data elements, wherein each training data element (401) has training input information relating to a position of a surface point of a corresponding object (113, 301) and one or more possible approach directions for the robot (100) to manipulate the object (113, 301), determining one or more contact points on the surface of the object (113, 301) by the machine learning model (112, 200, 400, 500) for manipulating the object (113, 301) by an end effector (104, 300) of the robot (100), Determining, by means of the machine learning model (112, 200, 400, 500), a weighted parameter value of a mixture distribution of spherical distributions for manipulating the approach direction of the object (113, 301), wherein each of the spherical distributions is assigned a corresponding end effector azimuth angle; and The machine learning model (112, 200, 400, 500) is trained to reduce a loss, wherein the loss includes an approach direction loss component for each training data element (401) and each possible approach direction, wherein the approach direction loss component decreases as the probability that the mixed distribution provides the possible approach direction increases, wherein the direction parameter of each of the spherical distributions is set according to the end effector azimuth angle assigned to the spherical distribution.

2. The method according to claim 1, wherein each training data element (401) comprises at least one direction vector between contact points, and wherein, The loss also includes a basis vector loss component for each training data element (401) for at least one of the determined contact points, and the basis vector loss component decreases as the probability that the spherical distribution determined by the machine learning model for the basis vectors of the determined contact points is consistent with the spherical distribution of the basis vectors assigned to the determined contact points and contained in the training data elements increases.

3. A method according to claim 1 or 2, wherein each training data element (401) has one or more contact points, including an associated gripper opening width and an associated basis vector (the basis vector describing, for each contact point, the direction of the associated contact point pair), and wherein, The method includes determining an associated paired contact point for each training data element (401) and each determined contact point, and wherein the loss also includes a width loss component for each training data element (401) and each determined contact point, and the width loss component decreases as the distance between the determined associated paired contact point and one or more associated paired contact points of the training data element (401) decreases.

4. The method according to any one of claims 1 to 3, wherein each training data element (401) comprises at least one touch point quality assessment, and the method comprises determining a quality assessment of each determined touch point by means of the machine learning model (112, 200, 400, 500), and wherein, The loss also includes a quality loss component for each training data element (401) and each determined contact point, the quality loss component increasing as the difference between the quality assessment determined for the determined contact point and the contact point quality assessment that the training data element has for the associated contact point increases.

5. A method according to any one of claims 1 to 4, wherein each training data element (401) comprises at least one (ground truth) position of a contact point on the surface of the object to be manipulated and at least one position of a reference point (e.g. a grasping center) of the end effector (104, 300), and the method comprises classifying spatial regions into spatial regions with and without contact points and into spatial regions with and without reference points by means of the machine learning model (112, 200, 400, 500), and wherein, The loss also includes a classification loss for the classification of each training data element (401) as a contact point-reference point loss component.

6. A method for controlling a robot (100) to manipulate an object to be manipulated (113, 301), comprising: Transmitting information about the position of the surface point of the object (113, 301) to be manipulated to a machine learning model (500), wherein the machine learning model is trained to output weighted parameter values ​​of a mixed distribution of spherical distributions on the surface of the object (113, 301) for manipulating the object (113, 301) by the end effector (104, 300) of the robot (100) and for manipulating the approach direction of the object (113, 301), in response to the transmission of information about the position of the surface point of the object (113, 301), wherein each of the spherical distributions is assigned a corresponding end effector azimuth; By comparing with a predetermined threshold, selecting a weighted parameter value that is higher than the predetermined threshold among the weighted parameter values ​​output by the machine learning model (500) in response to the transmitted information; and The robot (100) is controlled to manipulate the object (113, 301) to be manipulated by moving an end effector (104, 300) of the robot (100) toward the object (113, 301) along an approach direction given by an end effector azimuth angle assigned to a spherical distribution that is weighted in the mixed distribution by a selected weighting parameter value.

7. The method according to claim 6, wherein the machine learning model (500) is trained according to any one of claims 1 to 5.

8. A robot control device arranged to carry out the method according to any one of claims 1 to 7.

9. A computer program having instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A computer readable medium storing instructions, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

Images