Collision handling method for grip generation

Abstract

To realize a robot gripping generation technique that calculates gripping candidate with high quality without manual teaching, and does not unnecessarily exclude gripping.SOLUTION: Shapes of components and a gripping part are provided as input from CAD file. Also, kinematics of the gripping part is defined as input. A candidate gripping set is provided by use of any known preliminary gripping generation tool. A point model of component and a model of a gripping part contact surface comprising a clearance margin are used for optimization calculation that is applied to each gripping candidate, and an adjusted gripping database is obtained. The adjusted gripping optimizes gripping quality by use of a virtual gripping part surface, and an actual gripping part surface is so positioned as to be slightly away from the component. Calculation of a signed distance field is performed for each adjusted gripping, and any collision between the gripping part and the component is discarded. The gripping database includes gripping with high quality and non-collision to be used for robot component pick-and-place operations.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] This disclosure relates, in general terms, to a method for generating high-quality gripping data for robotic gripping of parts, and more specifically, to a method for robotic gripping generation that starts with a set of gripping candidates for a specific part and gripping part, optimizes each gripping candidate using a virtual gripping surface that allows for fine-tuning of the grip, and then analyzes the adjusted grips using signed-distance-field calculations to eliminate any collisions. [Background technology]

[0002] Industrial robots are commonly used to perform a variety of manufacturing, assembly, and material handling operations. One such application is pick-and-place operations, in which a robot picks up individual parts, places each part in a designated location, and then processes or packages them further. An example of a pick-and-place operation is to remove a part from a container and place it on a conveyor belt for transporting parts to the next operation; to remove a part from a container and place it in a fixture on a machine tool; and then to remove the part from the machine tool and place the processed part on a conveyor belt.

[0003] In the applications described above, a vision system (one or more cameras) is typically used to identify the position and orientation of individual parts within the container. Next, a specific robotic grip is identified for the selected part, which can then be selected from a pre-calculated database of grips generated for specific part shapes and grip shapes.

[0004] Any robotic gripping operation requires a database of high-quality grips. Gripping generation techniques are known that can automatically generate a large number of gripping candidates. However, these existing techniques have drawbacks, such as very high computational costs, slow processing speeds, or lower calculated gripping quality due to simplification. Furthermore, some existing gripping generation techniques exclude gripping candidates where a very slight collision between the gripper and the part is predicted, which would actually work perfectly well in the real world because the part is likely to move slightly during the gripping process, thus leading to a high-quality grip.

[0005] In light of the above circumstances, there is a need for robotic grip generation technology that can calculate high-quality grip candidates without manual teaching, is computationally efficient, and does not unnecessarily exclude grips based on predicted collisions that would be easily compensated for by real-world gripping parts and components. [Overview of the project]

[0006] A robotic grip generation technique is presented in accordance with the teachings of this disclosure. The geometry of the part and the gripper are typically provided as input from CAD files. In addition, the kinematics of the gripper are defined as input. A set of candidate grips is provided using a known preliminary grip generation tool. A refined grip database is obtained by using a point model of the part and a model of the gripper contact surface with clearance margins in optimization calculations applied to each candidate grip. The refined grips optimize grip quality using a virtual gripper surface and position the actual gripper surface slightly away from the part. Next, a signed distance field calculation is performed for each refined grip, and any grips with any collision between the gripper and the part are discarded. The resulting grip database contains high-quality collision-free grips for use in pick-and-place operations of robotic parts.

[0007] Additional features of the currently disclosed method will become apparent from the following description and the attached claims, in conjunction with the attached drawings.

Brief Description of the Drawings

[0008] [Figure 1] FIG. 1 is a diagram of a pick-and-place system by a robot in which an industrial robot extracts parts from a container, places the parts on a conveyor or a machining station, and further processes them. [Figure 2] FIG. 2 is an exemplary flowchart diagram of steps included in a grasping generation process used to create a collision-free grasping database for use in the pick-and-place system by the robot of FIG. 1 according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a diagram showing details of some steps of the grasping generation process of FIG. 2, including a grasping adjustment optimization step using a virtual grasping surface to achieve a desired grasping part positioning for a part according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a diagram of a general signed distance field calculation technique showing interference between a first object and a second object, which is known in the art. [Figure 5] FIG. 5 is a diagram showing details of a signed distance field collision inspection technique used in the grasping generation processes of FIGS. 2 and 3 to eliminate a grasp having part grasping interference after grasping adjustment according to an embodiment of the present disclosure.

Modes for Carrying Out the Invention

[0009] The following considerations of embodiments of the present disclosure directed to a method for collision handling in robot-based grasping generation are illustrative in nature and are not intended to limit the disclosed technology or its use or application in any way.

[0010] It is well known to use industrial robots to extract parts from a supply source and place them at a destination. In a typical pick-and-place operation, a feeder for the parts is provided in a container, such as a container containing a random pile of parts that need to be cast, molded, or partially assembled, which need to be removed from their random orientation within the container and placed in a specific orientation at the destination location.

[0011] FIG. 1 is a diagram of a robotic pick-and-place system in which an industrial robot 100 uses a gripping portion 120 to extract a part from a container 110 and place the part on either a machining station 130 or a conveyor 140 for further processing or packaging. The gripping portion 120 may be designed with a finger shape selected based on the shape of the part to be processed by the robot 100. The shape of the gripping portion fingers will be further considered below.

[0012] The movement of the robot 100 is typically controlled by a controller 150 that communicates with the robot 100 via a cable. The controller 150, as is known in the art, provides joint motion commands to the robot 100 and receives joint position data from encoders within the joints of the robot 100. The controller 150 also provides commands (grip / ungrip commands and width) for controlling the operation of the gripping portion 120.

[0013] A computer 160 communicates with the controller 150. The computer 160 includes a processor and memory / storage and implements an algorithm configured to calculate high-quality gripping of the parts within the container 110 based on images and data from at least one camera 170. The camera 170 is typically a three-dimensional (3D) camera that provides both color image data and pixel depth map data, but may be other types of 3D sensors that provide data suitable for determining the pose (position and orientation) of the parts within the container 110. The camera 170 may also be a multi-camera system that includes two or more 2D cameras or 3D cameras.

[0014] In some applications, the container 110 may contain only one part, which may be in any position and orientation. This would be the case if the part is processed by the machining station 130 at the same speed as it arrives in the container 110. Apart from this, the container 110 may contain several parts or a pile of parts (as shown in Figure 1). In any case, the best quality grip candidate is determined by matching image data from the camera 170 with a pre-calculated database of stable grips.

[0015] The computer 160 provides target gripping data to the controller 150, which controls the robot 100 and the gripping unit 120 to grip a part from the container 110. The target gripping data includes the gripping target point (x / y / z coordinates), the tracking angle of the gripping unit 120, and the rotation angle and width of the gripping unit. Using the gripping data, the controller 150 can calculate robot motion commands to cause the gripping unit 120 to grip the part and place the part on the machining station 130 or conveyor 140. Separately, the camera 170 may provide images to the controller 150, which will directly calculate part gripping. In any case, grip generation (providing a database of many different stable grips) is performed in advance. The generation of the gripping database is the subject of this disclosure and will be discussed in detail below.

[0016] Teaching a robot to recognize and grasp individual parts in a container full of parts in real time has always been challenging. To improve the speed and reliability of robotic part extraction operations, it is known to pre-calculate grips of a designated gripping part that grasps a particular part in various orientations. This pre-calculation of grips is known as grip generation, and as discussed above with reference to Figure 1, the pre-calculated (generated) grips are then used to make real-time decisions during the robotic part extraction operation.

[0017] Historically, grip generation methods have involved manually gripping extraction points on known 3D features of an object. Such methods require considerable time spent on heuristic design to determine the optimal gripping orientation, and such manually designed heuristic approaches may not yield a sufficient number of high-quality grips. Due to the difficulties of using heuristic teaching, automated grip generation methods have been developed.

[0018] However, existing automated grasp generation methods also have drawbacks. One known learning-based technique uses mathematically rigorous grasp quality to search for grasp candidates and then feeds such candidates into a convolutional neural network (CNN) classifier, but it is computationally expensive, and the simplifications used in the computation may mean that the solution is not optimal in real-world situations. Another grasp generation method uses empirical trials to collect data and generate realistic grasps, but this method can require tens of thousands of robot hours to implement complex force control, and any changes to the grasp would require repeating the entire process. Yet another computer-based grasp generation method discards many grasps due to "collisions," which are actually just contacts between the grasp and a normal part at the contact surface of the grasp's fingers. In general, existing automated grasp generation methods have not yet provided a sufficient number of high-quality grasps that work well under real-world conditions.

[0019] This disclosure describes a grip generation technique that can be automatically applied to any combination of gripper and component designs and efficiently generates a large number of realistic and high-quality grips. The disclosed grip generation method uses optimization calculations with a virtual gripper surface that adjusts candidate grips to compensate for slight contact during gripping, followed by signed-distance-field collision inspection of the adjusted grips. The resulting grip database includes grips that would be unnecessarily excluded by the prior art, but does not include grips with actual interference or collision conditions. By pre-generating a database of good grips, the computer 160 or controller 150 can perform real-time grip calculations quickly and efficiently during actual robot pick-and-place operations.

[0020] Figure 2 is an illustrated flowchart 200 of the steps included in a grip generation process used to create a collision-free grip database for use in the robotic pick-and-place system of Figure 1, according to one embodiment of the present disclosure.

[0021] First, a specific part and gripping part are selected. The gripping part model shown in Figures 2 and later is a two-fingered parallel gripping part, but other types of gripping parts may be used, such as gripping parts that grip an object with the fingers spread apart. The gripping fingers are equipped with fingertips that have a concave shape selected to correspond to the part (water pipe valve) that the robot is gripping. That is, the fingertips can stably grip many different parts of a water supply valve, such as the handle, coupling body, and valve stem throat. Different gripping parts with different fingertips may be adopted based on the shape of the part to be gripped.

[0022] In Box 210, for a given part (workpiece) and grip design, a number of grip candidates are provided using a preliminary grip generation tool. The preliminary grip generation tool may be, for example, a sampling-based tool or a learning-based tool. Each grip in the set of grip candidates is a grip in the coordinate frame of the part / workpiece (T gripperDefines the posture of ). In a preferred embodiment, each grip i is given a position vector p i , rotation vector R i and gripping part width value w i Includes.

[0023] The gripping within the set of gripping candidates includes numerous different positions and orientations of the gripping portion on the part. For example, the gripping portion shown in the figure and considered in this application is a parallel finger gripping portion comprising a housing body including an actuator and two gripping fingers that open and close in opposite directions, with a gripping width value w i The component shown in the figure and considered in this application is a water supply valve having a body having opposing cylindrical openings extending from the body, and a handle coupled to the body via a neck and shaft. The grips of the set of gripping candidates preferably include gripping fingers that grip the edges of the handle from many different angles, grip the body in many different orientations, grip each cylindrical end, or grip the neck of the valve. The number of candidate grips is identified as N.

[0024] The grips within a set of grip candidates are typically not analyzed for grip quality or collision testing, and the candidate grips are merely theoretical attitudes of the gripper towards the part. If the robot controller directly uses the set of grip candidates during robot operation, some of the attempted grips will fail due to poor grip quality or interference from the part gripper. Therefore, additional steps are required to improve the grip and eliminate collisions, as shown in the rest of Figure 2.

[0025] Box 220 employs a grip optimization technique that realistically simulates real-world gripping using a "clearance margin" or virtual grip surface, while avoiding contact between the part and the grip that could be interpreted as interference. The grip optimization technique adjusts each grip using a virtual grip surface model. This optimizes each grip for grip quality while leaving a part grip clearance margin that would be easily adapted by grip closure, allowing for slight part centering movements in real-world gripping. The grip optimization technique in Box 220 adjusts the grip.

number

[0026] In box 240, a signed distance field collision check is performed on each grip in the adjusted grip database 230. The signed distance field collision check quickly identifies any collisions in the adjusted grips in database 230. As described above, the grip optimization in box 220 adjusts the grip to optimize the contact between the gripper contact surface and the part, and the virtual gripper surface used in the optimization calculation ensures that the resulting grip includes a small gap between the gripper and the part. However, some of the adjusted grips in database 230 still contain collisions or interferences away from the gripper contact surface, and collisions or interferences may prevent such grips from being used in actual robot part gripping operations. The signed distance field collision check eliminates grips with such interferences, resulting in a collision-free grip database 250. The collision-free grip database 250 contains grips

number

[0027] The grasping optimization technique for box 220 and the signed distance field collision inspection for box 240 are discussed in detail below in relation to FIGS. 3 and 5, respectively.

[0028] FIG. 3 shows details of some steps of the grasping generation process of FIG. 2, including a grasping adjustment optimization step that uses a virtual grasping surface to achieve a desired grasping position for a part according to an embodiment of the present disclosure. In box 300, CAD models of the gripper 302 and the part (workpiece) 304 are provided. The CAD model is typically a solid model, but as will be understood by those skilled in the art, it may also be a surface model including a complete 3D surface shape. In box 300, the mechanized kinematics of the gripper 302 are also provided. In the case of a parallel finger gripper, this includes defining the position / shape of the finger elements of the gripper 302 with respect to the gripper width w.

[0029] In box 310, the grasping candidates {p i , R i , w i} i=1,N are provided. This is the set of grasping candidates provided in box 210 of FIG. 2 discussed above. The set of grasping candidates typically includes hundreds or thousands of grasping candidates. For each candidate grasp i (i = 1, N), in addition to the gripper width w i , the position vector p i (three positions; x, y, z) of the gripper 302 with respect to the part 304 and the rotation vector R i (three rotations such as yaw, pitch, roll) are included. In box 310, the part 304 is shown as a point cloud for each grasp. The point cloud representation of the part 304 can be easily created from the CAD solid model as will be understood by those skilled in the art.

[0030] In box 320, a "clearance margin" is created with respect to the contact surface of the gripping portion 302. The purpose of the clearance margin is to adjust each grip so that the gripping portion opens slightly wider than the width required for actual contact with the part surface. This is done so that the resulting grip is not identified as interfering with the part and gripping portion (and therefore discarded) in a later step. The clearance margin is achieved by defining a virtual gripping surface that is "padded" by a thickness of about 1-2 mm with respect to the actual gripping contact surface, as will be discussed below.

[0031] Box 320 shows a 2D cross-sectional view of part 330 and gripping fingers 340A / 340B. Part 330 and gripping finger contact surfaces 340A / 340B represent the part and gripping part within box 300. For clarity, simplified 2D cross-sectional shapes are used from box 320 onward in Figure 3. The gripping finger contact surfaces 340A / 340B are the contact surfaces of the two opposing fingers of the parallel finger gripping part in box 300. In addition to the gripping finger contact surfaces 340A / 340B (which are the actual gripping finger contact surfaces), virtual gripping contact surfaces 342A / 342B are shown. The virtual gripping contact surfaces 342A / 342B are formed using a clearance margin that may have a value of 1-2 mm from the gripping finger contact surfaces 340A / 340B, as discussed above. The clearance margin is in the direction in which the gripping fingers move when gripping an object. In addition, the virtual contact surfaces 342A / 342B of the gripping portion may be formed to be slightly longer than the finger contact surfaces 340A / 340B in order to create a clearance margin in the lateral direction across the gripping fingers (not just in the opening and closing direction).

[0032] Box 350 performs grip optimization calculations using the virtual contact surfaces 342A / 342B of the gripping portion. The result of the optimization calculation for each grip is an adjusted grip with optimized grip quality while providing a small clearance margin between the gripping portion and the part, so that the grip is not incorrectly excluded as having a collision in subsequent signed distance field calculations.

[0033] Figure 360 ​​shows a cross-section of part 330, along with the gripping finger contact surface 340A and the gripping virtual contact surface 342A. For simplicity, Figure 360 ​​shows the actual and virtual contact surfaces for only one of the gripping fingers (the left side). The same concept applies to the gripping finger on the opposite side. The gripping finger contact surface 340A and the gripping virtual contact surface 342A are shown in Figure 360 ​​in their "initial" position or configuration. That is, the gripping finger contact surface 340A is in a gripping position relative to part 330, as defined by the candidate grip from the box 310 to be optimized.

[0034] The initial configuration can be one of the examples shown in box 310 or one of hundreds of other candidate grips. Starting with the initial configuration shown in diagram 360, an optimization calculation is performed to adjust the grip of the gripping part on component 330. The optimization calculation (discussed below) adjusts the grip.

[0035] In the optimization calculation, the gripping portion is represented using the virtual contact surfaces 342A / 342B of the gripping portion, thereby optimizing the gripping quality and "pushing" the gripping portion finger contact surfaces 340A / 340B away from the part 330. The clearance margin of the virtual contact surfaces 342A / 342B of the gripping portion ensures that there is no intrusion or contact between the gripping portion finger contact surfaces 340A / 340B and the part 330, and as a result, the adjusted grip will not be flagged in later steps by false detection of interference between the part and the gripping portion.

[0036] The following is a consideration of an example of a grip optimization calculation that may be used in box 350. As will be understood by those skilled in the art, other grip optimization formulations that can calculate a stable grip between the gripping contact surface and the part may also be used. The grip optimization calculation involves the orientation (T) of the gripping part in the coordinate frame of the part / workpiece 330. gripperIt starts with one of the candidate grips. The coordinate frame includes the grip width w, along with the position p and orientation R of the grip finger contact surfaces 340A / 340B relative to the part 330. The optimization calculation uses the virtual grip contact surfaces 342A / 342B instead of the grip finger contact surfaces 340A / 340B. As shown in diagram 360 in box 350, the virtual grip contact surfaces 342A / 342B will initially penetrate (interfere with) the part 330.

[0037] On the surface of the gripping part virtual contact surface 342A / 342B, points pt are drawn using any appropriate sampling density. i Defines a set of sampling points pt on the virtual contact surface 342A of the gripping portion. i One of these is shown in Figure 360, section 366. In addition, sampling points are defined on the surface of part 330. Figure 3 is shown as a 2D cross-section for clarity. However, in the actual optimization calculation, both the virtual contact surfaces 342A / 342B of the gripping part and the surface of part 330 are defined as sampling points pt i It is three-dimensional, just like the coordinates of [the object].

[0038] Point pt on the virtual contact surface 342A / 342B of the gripping portion i For each of these, the nearest neighbor point on the surface of part 330 is determined. The nearest neighbor point is determined by a method known in the art, point pt i This is determined by calculating the shortest 3D distance from the point pt on the surface of part 330 to the sampling point. i Then, using the corresponding nearest neighbor, the optimization loop is used to update the movement, rotation, and opening width of the gripping part, resulting in improved gripping quality and reduced interference. Since the gripping part virtual contact surfaces 342A / 342B will be the first to penetrate (interfere with) the part 330, the first iteration of the optimization calculation imposes a "pushing" constraint penalty by moving the gripping part virtual contact surfaces 342A / 342B away from the part 330, in particular by widening the gripping part width w.

[0039] To calculate stable gripping, (point pt) iSurface contact (using its nearest neighbors) and strict mathematical quality are employed in the optimization model. Collisions between the gripping part and the part are penalized in the optimization to avoid penetration, as mentioned above. Throughout the consideration of the optimization calculation, point pt i The virtual contact surfaces 342A / 342B of the gripping portion, as defined by S f This is called S, and the surface of the part (or "object") (all sampling points, specifically the nearest neighbors) is called S o It is called that.

[0040] Each iteration of the optimization calculation results in a movement of the gripping virtual contact surfaces 342A / 342B, defined with respect to a transformation T which includes a translational transformation Δp, a rotational transformation ΔR, and a gripping width transformation Δw. For example, the first iteration moves the gripping virtual contact surfaces 342A / 342B from the initial configuration shown in 360 to a subsequent (not shown) intermediate configuration in which the nearest neighbor on part 330 is determined again and another optimization iteration is performed.

[0041] The optimization formulation is provided below as equations (1a) to (1d), and will be discussed in the following paragraph.

number

number

number

number

[0042] The optimization formulation includes gripping quality Q. g This includes an objective function (Equation 1a) defined to maximize the gripping quality Q. g The contact surface S f and S o It is a function of . Grasping Quality Q g Q may be defined in any suitable way. In a typical embodiment, gripping quality Q g The gripping portion virtual contact surface S f The dot above (pt)i and the object surface S o This is the negative value of the total distance between the above matching nearest neighbor point NN. In practice, a good grip is stable, and any small movement of the part 330 within the gripping section is quickly stopped by friction and / or normal forces, and does not lead to loss of grip.

[0043] The optimization formulation is as follows: the virtual contact surface S of the gripping part with respect to the transformation (Δp, ΔR, Δw). f The equation includes a constraint function (Equation 1b) that defines the following: The constraint function (Equation 1c) indicates that the transformed virtual contact surface (T(∂F; Δp, ΔR, Δw)) must not penetrate object O; that is, the distance must be greater than or equal to zero. This constraint causes the gripping virtual contact surfaces 342A / 342B to be "pushed" away from part 330, as described above. Finally, Equation (1d) indicates that the transformation T(Δp, ΔR, Δw) is an element of a set of feasible transformations X (in one embodiment, Δp and ΔR may have any values ​​within a predefined range, and Δw is limited by the gripping range defined in the gripping model from box 300).

[0044] The constraint functions in equations (1b) to (1c) above are considered by a penalty method in which constraint violations are treated as a cost function in the gripping quality calculation. That is, the larger the constraint violation, the larger the cost function becomes, and the lower the gripping quality. By imposing a constraint on the cost to gripping quality, the optimization formulation can be solved using the least squares method.

[0045] A one-step least-squares linear algebra calculation is performed on the optimization formulation of equations (1a) to (1d) to identify a transformation T(Δp, ΔR, Δw) that moves the gripping fingers in a direction that improves gripping quality while reducing / eliminating penetration of the virtual contact surfaces 342A / 342B of the gripping portion with the part 330. This process is repeated until the gripping position converges to a maximum gripping quality that minimizes or eliminates penetration of the virtual contact surfaces 342A / 342B of the gripping portion with the part 330.

[0046] In Figure 362, the cross-section of part 330 is shown together with the gripping finger contact surface 340A and the gripping virtual contact surface 342A. Here, the gripping finger contact surface 340A and the gripping virtual contact surface 342A are shown in the "adjusted" position after optimization convergence in Figure 362. That is, the gripping finger contact surface 340A is in the gripping position relative to part 330 after an optimization calculation that optimizes the gripping quality and "pushes" the gripping finger contact surface 340A away from part 330. The clearance margin of the gripping virtual contact surface 342A ensures that there is no intrusion or point-to-point contact between the gripping finger contact surface 340A and part 330, and as a result, the adjusted grip can be determined to be collision-free (at least in the area of ​​the gripping contact surface) in a later step.

[0047] After converging to a quality-adjusted grip for one of the grip candidates, the process is restarted from the beginning with a new initial configuration (i.e., another of the grip candidates). This grip optimization calculation is performed for each of the candidate grips from box 310, resulting in a set of adjusted grips in box 370. The set of adjusted grips in box 370 is as previously discussed with respect to Figure 2,

number

[0048] Box 380 performs a signed distance field collision test on each of the N adjusted grips in Box 370. The signed distance field collision test determines whether there is any collision between the part and the grip after the grip has been adjusted. The signed distance field collision test checks for collisions not only between the finger contact surface of the grip but also between the entire part 330 and any part of the grip. Each adjusted grip is evaluated using a signed distance field calculation. Grips without collisions (indicated with a checkmark in Box 390) are retained, and grips with collisions between the part and the grip (indicated with an X) are discarded.

[0049] In box 390, one adjusted grip (indicated by ×) that is discarded has interference away from the gripping finger surface. Specifically, the valve handle interferes with a portion of the gripping finger near the gripping actuator body. This interference is not considered in the optimization calculation in box 350 because the optimization only considers the gripping finger contact surface. This is precisely why signed distance field collision testing is used in box 380. As a result, the optimization calculation can optimize grip quality while providing a small clearance margin between the gripping finger contact surface and the part, and then the signed distance field calculation can detect interference caused by the overall gripping posture.

[0050] Box 390 provides the final set of collision-free grips. Details of the signed distance field collision inspection in Box 380 will be discussed later with reference to Figure 5. The final set of collision-free grips in Box 390 is as discussed previously with respect to Figure 2.

number

[0051] The advantages of the gripping method shown in Figure 3 are as follows: • In the optimization calculation for Box 350, only the finger contact surface of the gripping part is considered. This allows the adjusted grip to be calculated much faster than when the entire gripping part and body are considered, as in the conventional technique. • The optimization calculation uses the virtual gripping contact surfaces 342A / 342B. This results in a adjusted grip with a small clearance margin between the actual gripping finger contact surfaces 340A / 340B and the part 330. • The signed distance field collision test of box 380 (details discussed below) is an extremely efficient calculation for identifying and discarding any adjusted grips involving interference between the gripping portion (in whole) and part 330. • In signed distance field collision testing, the adjusted grip includes a small clearance margin, so collisions will not be falsely detected in the area of ​​the grip's finger contact surface.

[0052] Figure 4 is a diagram of a common signed distance field calculation technique known in the art to show interference between a first object and a second object. In the diagram 400 on the left, the signed distance field grids 410A and 410B represent the left and right gripping fingers in the context of this study. Object 420 represents the part gripped by the gripping fingers, where object 420 is defined by a surface point mesh. The signed distance field grids 410A and 410B and object 420 are shown in a configuration in which the signed distance field grids 410A and 410B are positioned relative to object 420 according to a specific grip. Although Figure 4 is a 2D diagram for clarity, it should be understood that in actual signed distance field collision inspection calculations, both the surface point mesh of object 420 and the signed distance field grids 410A and 410B are 3D.

[0053] In Figure 430 on the right, the signed distance field grids 410A and 410B are shown without object 420. The signed distance field grids 410A and 410B are shaded (with hatch lines) to indicate the occupancy of grid cells by object 420. Reference numeral 432 shows a group of six (shaded) grid cells occupied by object 420. It can be easily seen in Figure 400 on the left that some parts of object 420 (one or more points of the surface point mesh) are located within the six grid cells shown in 432. Similarly, reference numeral 434 shows a group of eleven (shaded) grid cells occupied by object 420 (part of the rabbit's nose), and reference numeral 436 shows a group of seven (shaded) grid cells occupied by object 420 (one of the rabbit's ears).

[0054] To determine (yes / no) the occupancy of each grid cell, the distance from each grid cell to the nearest point on object 420 is calculated. Grid cells with a distance greater than zero are not occupying (not shaded in Figure 430), and grid cells with a distance less than or equal to zero are occupying (shaded). A simple visual example in Figure 4 illustrates the concept of signed distance field grid occupancy, which is used in box 380 in Figure 3 and will be explained in detail below.

[0055] Figure 5 shows details of the signed distance field collision inspection technique used in the grip generation process of Figures 2 and 3 to eliminate grips having interference between the part and the grip after grip adjustment, according to one embodiment of the present disclosure. As discussed above, the signed distance field collision inspection is performed in box 380 for each of the adjusted grips from box 370. This process is illustrated in the discussion of Figure 5 for one specific grip.

[0056] Box 510 provides one of the adjusted grips from Box 370. The gripping part 512 is shown in CAD model format, and the object or part 514 is shown in point cloud format. The grip is shown as gripping part 512(T gripper This is defined as the posture of ). Box 520 generates a signed distance field grid 522 for the gripping part 512. The generation of the signed distance field grid 522 is performed only once in an offline process for the entire set of adjusted grips to be evaluated. In other words, generating the signed distance field grid 522 does not affect the computation time required to evaluate any adjusted grip from Box 370. The signed distance field grid 522 is a 3D grid of volume elements such as hexahedrons, and may be brick-shaped or cubic. The size of the grid elements may vary in location to suit accuracy requirements, for example, the grid tangent size of the gripping part body / actuator is 3 mm and the grid tangent size of the gripping fingers is 0.3 mm.

[0057] Box 530 performs the first step of a signed distance field collision test for a specific grip. Part 534 is shown in CAD model format. In box 530, part 534 is a grip T gripper By using the reciprocal of , it is transposed to the gripping part coordinate frame. That is, part 534 is

number

[0058] In box 550, the second step of signed distance field collision testing for a specific grip is performed. Here, part 544 in the grip coordinate frame is used together with the signed distance field grid 522 of the grip 512. The signed distance field grid 522 is defined with respect to the 3D grid space (i,j,k). Each point 544 in the point cloud of part 544 (which may contain thousands of points, especially on the surface of the part) n Regarding this, the Cartesian coordinates of the point are transformed into the grid space (i,j,k). Point 544 n If point 544 has a grid index (i, j, k) that is completely outside the signed distance field grid 522, then point 544 n The next point 544 in the point cloud of part 544 is far from the gripping portion 512, and the process is as follows: m Move to inspect it.

[0059] point 544 n However, if point 544 has a grid index (i,j,k) that lies inside the entire boundary of the signed distance field grid 522, n It is known that point 544 is either near the gripping portion 512 or interferes with the gripping portion 512. In this case, the sign of the distance to the nearest grid (SDF(i,j,k)) is checked using a known 3D distance calculation. If SDF(i,j,k)>0, point 544 n There is no collision with the gripping portion 512, and the process proceeds as follows: 544 m We proceed to the inspection. If SDF(i,j,k)<0, point 544 n This collides with the gripping portion 512. This means that part 514 interferes with the gripping portion 512.

[0060] For a particular grip (from the adjusted grips in box 370), if every point in the point cloud of part 544 has a signed distance field value greater than zero, the grip is determined to be collision-free, and the grip is placed in the final collision-free grip set in box 390.

[0061] For a particular grip, if any point in the point cloud of part 544 has a signed distance field value less than 0, the grip is determined to have interference between the part and the grip, and the grip will not be placed in the final collision-free grip set in box 390. In this case, the minimum distance value (maximum penetration) is listed, and the corresponding object point is visually displayed so that the user understands why the particular grip was discarded due to interference between the part and the grip.

[0062] As shown in Figure 5, the process described above is performed for each of the adjusted grips in box 370, and the final set of grips in box 390 includes only those adjusted grips that are determined to be free of collisions. The final set of grips in box 390 is then ready for the robot to use for pick-and-place operations, as shown in Figure 1.

[0063] The grip generation technology shown in Figures 2, 3, and 5 has been demonstrated on various real-world parts and generates a high-quality grip database. This database includes various gripping positions and gripping approach directions on the part, while eliminating interference between the gripping unit and the part. The calculated grip database has also been demonstrated to enable rapid and effective grip identification in robotic gripping systems of the type shown in Figure 1.

[0064] The grip generation technique discussed above offers several advantages over existing methods. The disclosed method provides high-quality, full-degree-of-freedom grips via surface contact that are robust against uncertainty and disturbances. Because the grip generation method involves grip optimization modeled only on the grip contact surface, it provides significantly faster calculations than conventional optimization methods of the overall geometry. The disclosed method uses a virtual grip surface to perform grip optimization and provide a clearance margin, thereby eliminating false collision warnings in subsequent signed-distance-field calculations. Furthermore, signed-distance-field collision inspection allows for rapid and efficient detection of actual part-to-part interference conditions occurring at locations away from the intended interaction between the grip contact surface and the part.

[0065] Throughout the discussion so far, various computers and controllers have been described and implied. It should be understood that the software applications and modules for such computers and controllers run on one or more computing devices having processors and memory modules. In particular, this includes a processor in a robot controller 150 that controls the robot 100 that performs object grasping, and a processor in a computer 160 that performs grasp generation calculations. As discussed so far, either the controller 150 or the computer 160 may be configured to identify the object to be grasped in real time.

[0066] Numerous exemplary aspects and embodiments of collision handling methods in robotic gripping generation have been discussed above, and those skilled in the art will recognize modifications, substitutions, additions, and partial combinations thereof. Therefore, the following appended claims, and any claims introduced thereafter, are intended to be interpreted as including any modifications, substitutions, additions, and partial combinations that fall within their true spirit and scope. (Aspect 1) A method for generating a collision-free gripping database for use by a robotic gripping system, wherein the method is: The steps include providing an object model including three-dimensional (3D) shape data of the object to be grasped, and a grasping part model including three-dimensional shape data of the grasping part and the kinematics of the fingers, The steps include providing a database of candidate gripping positions for the gripping portion on the object, A step of defining a virtual contact surface of the gripping portion that is separated from the finger contact surface of the gripping portion on each finger of the gripping portion, based on a predetermined clearance margin, A step of performing grip optimization for each of the candidate grips in order to generate a database of adjusted grips using a computer having a processor and memory, wherein the grip optimization maximizes the grip quality value using the virtual contact surface of the grip and the point cloud representation of the object model. The steps include performing a signed distance field collision inspection on each of the adjusted grips using the point cloud representation of the object model and the signed distance field grid of the gripping portion, A method comprising the step of providing a collision-free grip database, which includes each of the adjusted grips that have been determined to be collision-free in the signed-distance-field collision test. (Aspect 2) The method according to embodiment 1, wherein the virtual contact surface of the gripping portion is offset from the finger contact surface of the gripping portion in the direction in which each gripping portion finger moves when gripping an object. (Aspect 3) The method according to embodiment 1, further comprising the steps of: identifying a target object to be grasped from a container of objects using a collision-free grasping database during operation of a robot in operation, the steps of: identifying the target object by mapping the grasps from the collision-free grasping database to the orientation of the object estimated from camera or sensor data; and providing target object grasping data to a robot controller that instructs a robot to which the grasping unit is attached to grasp the target object. (Aspect 4) The method according to embodiment 1, wherein candidate grip, adjusted grip, and collision-free grip each define the orientation of the gripping portion in the coordinate frame of the object. (Appendix 5) The method according to embodiment 4, wherein the orientation of the gripping portion of the object in the coordinate frame includes three positions and three rotations, as well as the width of the gripping portion. (Aspect 6) The method according to embodiment 1, wherein the gripping optimization is formulated by defining sampling points on the virtual contact surface of the gripping portion and the point cloud representation of the object model including the nearest neighbor points on the object corresponding to each of the sampling points on the virtual contact surface of the gripping portion. (Aspect 7) The method according to embodiment 1, wherein the gripping optimization includes an objective function that converges to a locally optimal gripping quality defined based on the total distance between each sampling point on the virtual contact surface of the gripping portion and its corresponding nearest neighbor on the object, and includes constraint equations that penalize the penetration of the virtual contact surface of the gripping portion into the object. (Pattern 8) The grip optimization using the virtual contact surface of the gripping portion results in an adjusted grip in which the finger contact surface of the gripping portion is separated from the object by the clearance margin, according to the method of embodiment 7. (Aspect 9) The method according to embodiment 1, wherein the step of performing the signed distance field collision test includes converting the point cloud representation of the object model into a coordinate frame of the signed distance field grid of the gripping portion using the reciprocal of the orientation defining the adjusted grip. (Aspect 10) The method according to embodiment 9, wherein the step of performing the signed distance field collision test includes the step of determining that the adjusted grip is collision-free if a point in the point cloud representation of the object model is not in a grid cell of the signed distance field grid of the grip. (Aspect 11) The method according to Embodiment 1, wherein the database of candidate grips includes a large number of grips, each grip defining a different position and orientation of the gripping portion relative to the object, and is not evaluated for gripping quality or interference between the object and the gripping portion. (Aspect 12) A method for generating a collision-free gripping database, wherein the method is The steps include providing a three-dimensional (3D) model of the object to be grasped and a 3D model of the grasping part, The steps include providing a database of candidate gripping positions for the gripping portion on the object, A step of performing grip optimization for each of the candidate grips in order to generate a database of adjusted grips using a computer having a processor and memory, wherein the grip optimization maximizes the grip quality value using the gripping finger contact surfaces on each finger of the gripping part and the virtual gripping contact surfaces that are outside the point cloud representation of the object model. A method comprising the step of performing a signed distance field collision test on each of the adjusted grips in order to provide a collision-free grip database, which includes each of the adjusted grips that has been determined to be collision-free in the signed distance field collision test. (Aspect 13) A robotic gripping and generating system comprising a computer having a processor and memory, wherein the computer is A step to generate a collision-free gripping database, The steps include providing an object model that includes three-dimensional (3D) shape data of the object to be grasped, and a grasping part model that includes three-dimensional shape data of the grasping part and the kinematics of the fingers, The steps include providing a database of candidate gripping positions for the gripping portion on the object, A step of defining a virtual contact surface of the gripping portion that is separated from the finger contact surface of the gripping portion on each finger of the gripping portion, based on a predetermined clearance margin, A step of performing grip optimization on each of the candidate grips to generate a database of adjusted grips, wherein the grip optimization maximizes the grip quality value using the virtual contact surface of the gripping part and the point cloud representation of the object model. The steps include performing a signed distance field collision inspection on each of the adjusted grips using the point cloud representation of the object model and the signed distance field grid of the gripping portion, A system configured to perform a step including: providing a collision-free grip database, which includes each of the adjusted grips that have been determined to be collision-free in the signed-distance-field collision test. (Aspect 14) A 3D camera that provides a depth image of an object container to a computer during operation of a robot, wherein the computer identifies a target object to be grasped from the object container by mapping a grasp from the collision-free grasping database to the object's orientation estimated from the depth image, A robot controller that communicates with the computer and receives gripping data relating to the target object, The system according to embodiment 13, further comprising: a robot equipped with a gripping unit that grips and moves the target object based on a command from the controller. (Aspect 15) The system according to embodiment 13, wherein the virtual contact surface of the gripping portion is offset from the finger contact surface of the gripping portion in the direction in which each gripping portion finger moves when gripping an object. (Aspect 16) The system according to embodiment 13, wherein candidate grip, adjusted grip, and collision-free grip each define the orientation of the grip in the coordinate frame of the object, including three positions and three rotations, as well as the grip width. (Aspect 17) The grip optimization is formulated by defining sampling points on the virtual contact surface of the gripping portion and the point cloud representation of the object model, which includes the nearest neighbor points on the object corresponding to each of the sampling points on the virtual contact surface of the gripping portion, according to the system in embodiment 13. (Aspect 18) The grip optimization system according to embodiment 13, comprising an objective function that converges to a locally optimal grip quality defined based on the total distance between each sampling point on the virtual contact surface of the gripping portion and its corresponding nearest neighbor on the object, and comprising constraint equations that penalize the penetration of the virtual contact surface of the gripping portion into the object. (Aspect 19) The grip optimization using the virtual contact surface of the gripping portion results in an adjusted grip in which the finger contact surface of the gripping portion is separated from the object by the clearance margin, according to the system of embodiment 17. (Aspect 20) The system according to embodiment 13, wherein the step of performing the signed distance field collision test includes the step of determining that the adjusted grip is collision-free if a point in the point cloud representation of the object model is not within a grid cell of the signed distance field grid of the grip.

Claims

1. A method for generating a collision-free gripping database for use by a robotic gripping system, wherein the method is: The steps include providing an object model including three-dimensional (3D) shape data of the object to be grasped, and a grasping part model including three-dimensional shape data of the grasping part and the kinematics of the fingers, The steps include providing a database of candidate gripping positions for the gripping portion on the object, A step of defining a virtual contact surface of the gripping portion that is separated from the finger contact surface of the gripping portion on each finger of the gripping portion, based on a predetermined clearance margin, A step of performing grip optimization for each of the candidate grips in order to generate a database of adjusted grips using a computer having a processor and memory, wherein the grip optimization maximizes the grip quality value using the virtual contact surface of the grip and the point cloud representation of the object model. The steps include performing a signed distance field collision inspection on each of the adjusted grips using the point cloud representation of the object model and the signed distance field grid of the gripping portion, A method comprising the step of providing a collision-free grip database, which includes each of the adjusted grips that have been determined to be collision-free in the signed-distance-field collision test.

2. The method according to claim 1, wherein the virtual contact surface of the gripping portion is offset from the finger contact surface of the gripping portion in the direction in which each gripping portion finger moves when gripping an object.

3. The method according to claim 1, further comprising the steps of: identifying a target object to be grasped from a container of objects using a collision-free grasping database during operation of a robot in operation, the steps of: identifying the target object by mapping the grasps from the collision-free grasping database to the orientation of the object estimated from camera or sensor data; and providing target object grasping data to a robot controller that instructs a robot to which the grasping unit is attached to grasp the target object.

4. The method according to claim 1, wherein candidate grip, adjusted grip, and collision-free grip each define the orientation of the gripping portion in the coordinate frame of the object.

5. The method according to claim 4, wherein the orientation of the gripping portion of the object in the coordinate frame includes three positions and three rotations, as well as the width of the gripping portion.

6. The method according to claim 1, wherein the gripping optimization is formulated by defining sampling points on the virtual contact surface of the gripping portion and the point cloud representation of the object model including nearest neighbor points on the object corresponding to each of the sampling points on the virtual contact surface of the gripping portion.

7. The method according to claim 1, wherein the gripping optimization includes an objective function that converges to a locally optimal gripping quality defined based on the total distance between each sampling point on the virtual contact surface of the gripping portion and its corresponding nearest neighbor on the object, and includes constraint equations that penalize the penetration of the virtual contact surface of the gripping portion into the object.

8. The method according to claim 7, wherein the gripping optimization using the virtual contact surface of the gripping portion results in an adjusted grip in which the finger contact surface of the gripping portion is separated from the object by the clearance margin.

9. The method according to claim 1, wherein the step of performing the signed distance field collision test includes the step of converting the point cloud representation of the object model into a coordinate frame of the signed distance field grid of the gripping portion using the reciprocal of the orientation defining the adjusted grip.

10. The method according to claim 9, wherein the step of performing the signed distance field collision test includes the step of determining that the adjusted grip is collision-free if a point in the point cloud representation of the object model is not in a grid cell of the signed distance field grid of the grip.

11. The method according to claim 1, wherein the database of candidate grips includes a large number of grips, each grip defining a different position and orientation of the gripping portion relative to the object, and is not evaluated for gripping quality or interference between the object and the gripping portion.

12. A method for generating a collision-free gripping database, wherein the method is The steps include providing a three-dimensional (3D) model of the object to be grasped and a 3D model of the grasping part, The steps include providing a database of candidate gripping positions for the gripping portion on the object, A step of performing grip optimization for each of the candidate grips in order to generate a database of adjusted grips using a computer having a processor and memory, wherein the grip optimization maximizes the grip quality value using the gripping finger contact surfaces on each finger of the gripping part and the virtual gripping contact surfaces that are outside the point cloud representation of the three-dimensional (3D) model of the object. A method comprising the step of performing the signed distance field collision test on each of the adjusted grips in order to provide a collision-free grip database, which includes each of the adjusted grips that has been determined to be collision-free by the signed distance field collision test.

13. A robotic gripping and generating system comprising a computer having a processor and memory, wherein the computer is A step to generate a collision-free gripping database, The steps include providing an object model including three-dimensional (3D) shape data of the object to be grasped, and a grasping part model including three-dimensional shape data of the grasping part and the kinematics of the fingers, The steps include providing a database of candidate gripping positions for the gripping portion on the object, A step of defining a virtual contact surface of the gripping portion that is separated from the finger contact surface of the gripping portion on each finger of the gripping portion, based on a predetermined clearance margin, A step of performing grip optimization on each of the candidate grips to generate a database of adjusted grips, wherein the grip optimization maximizes the grip quality value using the virtual contact surface of the gripping part and the point cloud representation of the object model. The steps include performing a signed distance field collision inspection on each of the adjusted grips using the point cloud representation of the object model and the signed distance field grid of the gripping portion, A system configured to perform a step including: providing a collision-free grip database, which includes each of the adjusted grips that have been determined to be collision-free in the signed-distance-field collision test.

14. A 3D camera that provides a depth image of an object container to a computer during operation of a robot, wherein the computer identifies a target object to be grasped from the object container by mapping a grasp from the collision-free grasping database to the object's orientation estimated from the depth image, A robot controller that communicates with the computer and receives gripping data relating to the target object, The system according to claim 13, further comprising: a robot equipped with a gripping unit that grips and moves the target object based on a command from the robot controller.

15. The system according to claim 13, wherein the virtual contact surface of the gripping portion is offset from the finger contact surface of the gripping portion in the direction in which each gripping portion finger moves when gripping an object.

16. The system according to claim 13, wherein candidate grip, adjusted grip and collision-free grip each define the orientation of the grip in a coordinate frame of the object, including three positions and three rotations, as well as the grip width.

17. The grip optimization is formulated by defining sampling points on the virtual contact surface of the gripping portion and the point cloud representation of the object model including the nearest neighbor points on the object corresponding to each of the sampling points on the virtual contact surface of the gripping portion, according to claim 13.

18. The grip optimization includes an objective function that converges to a locally optimal grip quality defined based on the total distance between each sampling point on the virtual contact surface of the gripping portion and its corresponding nearest neighbor on the object, and includes constraint equations that penalize penetration of the virtual contact surface of the gripping portion into the object, according to claim 13.

19. The grip optimization using the virtual contact surface of the gripping portion results in an adjusted grip in which the finger contact surface of the gripping portion is separated from the object by the clearance margin, according to claim 17.

20. The system according to claim 13, wherein the step of performing the signed distance field collision test includes the step of determining that the adjusted grip is collision-free if a point in the point cloud representation of the object model is not in a grid cell of the signed distance field grid of the grip.

Images