Grasp planning method for industrial robot and grasp planning system for industrial robot

Abstract

To provide robot grasp planning adaptable for bin picking.SOLUTION: Workpieces in a bin, which have random positions and poses, are to be grasped by a robot and placed in a goal position and pose. A shape of the workpiece is analyzed to identify a plurality of robust grasp options, each grasp option having a position and orientation. The workpiece shape is also analyzed to determine a plurality of stable intermediate poses. Each individual workpiece in the bin is evaluated to identity a set of feasible grasps, and the workpiece is moved to the goal pose if such direct movement to the goal pose is possible. If direct movement is not possible, a search problem is formulated, where each stable intermediate pose is a node. The search problem is solved by evaluating the feasibility and optimality of each link between nodes. Feasibility of each link is evaluated in terms of collision avoidance constraints and robot joint motion constraints.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to the field of motion control of industrial robots, and in particular to techniques for planning a gripping posture of a robot's gripper based on a predefined placement posture, where it is seen that parts are present in a picking bin in random postures, and the motion of the robot is such that the robot removes the part from the bin and moves it to a defined placement location and posture in a single motion, preferably without re-gripping, or in successive motions with minimal re-gripping, while avoiding collisions between the robot and the bin and staying within the motion limits of the robot's joints. [Background technology]

[0002] The use of industrial robots to perform a variety of manufacturing, assembly, and material transfer tasks is well known. One such application is a pick-and-place operation, where a robot removes individual parts from a bin and places each part on a conveyor. One example of this application is where molded or machined parts are dropped into a bin and placed in random locations and orientations, and the robot is tasked with removing each part and placing it in a predefined orientation (pose) on a conveyor that transports the parts for packaging or further processing. Depending on the degree to which the parts in the bin are overlapped, finger-type graspers or suction-type grippers can be used as robot tools. Typically, a vision system (one or more cameras) is used to identify the location and pose of the individual parts in the bin.

[0003] Systems exist for planning the motion of a robot for the above-mentioned pick-and-place applications while adhering to the physical limits of motion of the robot's joints and avoiding collisions of the robot or the part with containers or other obstacles. However, prior art systems are typically hard-coded to look for a particularly preferred face or surface of the part that can be reached by a suction gripper. If the preferred face is accessible, the robot will grip the part and place it on the conveyor in one motion. If the preferred face of the part is not accessible, the robot will find another predefined gripping surface, move the part to a predefined intermediate pose, re-grip the part using the preferred face, and then place the part on the conveyor.

[0004] Prior art systems of the type described above may miss opportunities to pick and place a part in a single motion, and may also select motion sequences that include intermediate poses that cause the robot travel distance and joint motions to be larger than necessary.

[0005] Given the above circumstances, a robotic grasp planning method is needed that adaptively identifies the most efficient part grip and motion sequence. Summary of the Invention

[0006] In accordance with the teachings of the present disclosure, an adaptive robotic grasp and motion planning method is disclosed. A workpiece in a container placed in a random position and pose is grasped by a robot and placed in a final position and pose. The shape of the workpiece is analyzed to identify multiple robust grasp options, each grasp option having a position and orientation. The shape of the workpiece is also analyzed to determine multiple stable intermediate poses. Each individual workpiece in the container is evaluated to identify a set of feasible grasps, and the workpiece is moved to the final position and pose if a direct move is possible. If a direct move is not possible, a search problem is formulated. Each feasible grasp and each stable intermediate pose is a node. The search problem is solved by evaluating the feasibility and optimality of each link between the nodes. The feasibility of each robot move is evaluated in terms of collision avoidance constraints and motion constraints of the robot joints. When an optimal solution to the search problem is found, a robot motion command is calculated and provided to the robot to evaluate another part in the container. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram of a workpiece in a container that is required to be placed in a predefined target position and orientation, such as on a conveyor. [Diagram 2] 1 is a diagram of a workpiece in a container being moved to a predetermined target position and orientation via direct transitions or via predetermined intermediate states, according to techniques known in the art. [Diagram 3] FIG. 1 is an illustration of a workpiece showing multiple sample grips, each of which is evaluated for robustness or robustness, in accordance with an embodiment of the present disclosure. [Figure 4] 1 is a diagram of a workpiece illustrating multiple stable workpiece orientations according to an embodiment of the present disclosure. [Figure 5A] FIG. 1 illustrates an illustration of a search problem formulated to evaluate the feasibility and optimality of multiple paths from an initial workpiece pose through one intermediate pose to a goal pose, in accordance with an embodiment of the present disclosure. [Figure 5B]FIG. 1 is an illustration of a search problem formulated to evaluate the feasibility and optimality of multiple paths from an initial workpiece pose through two or more intermediate poses to a goal pose, in accordance with an embodiment of the present disclosure. [Figure 6] 1 is a flowchart of a method for adaptive robotic grasp and motion planning including evaluation of multiple grasps and intermediate poses to optimize robot motion, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The following description of embodiments of the present disclosure relating to adaptive robotic grasping and motion planning is merely preferred in nature and is not intended to limit the disclosed apparatus and techniques or their applications or uses.

[0009] The use of industrial robots in a variety of manufacturing, assembly and material transfer tasks is well known. In some types of such tasks, the robot must be programmed to move along paths that have different start and / or destination points from one task to the next. For example, in a pick, move and place task, parts in a bin with random locations and orientations must be picked by the robot and moved onto a conveyor in a predefined pose. Parts may arrive on an incoming conveyor in random locations and orientations, again requiring the calculation of a unique pick / move / place sequence for each individual part. There are many other examples of applications where a new path must be calculated for each individual task or task of the robot.

[0010] Furthermore, in many robot working environments, obstacles are present that may be in the path of the robot's motion, i.e., one or more obstacles may be between the starting point and the goal point, or more generally, between the robot's current position and the robot's destination position. If the parts are provided in a container, the container itself becomes an obstacle for the robot when picking up the parts.

[0011] FIG. 1 is a diagram of workpieces in a bin that need to be placed at a predefined target location and orientation, such as on a conveyor. FIG. 1 shows an industrial robot performing pick, move and place operations in a situation of the type described above. In this situation, each individual workpiece in the bin requires a unique grip solution, which in turn requires a unique move, reorient and place solution. The robot must perform the pick / move / place operation reliably (all parts are placed on the conveyor in the proper orientation, no parts are dropped) and the operation must be 100% collision-free (e.g., between the robot arm and the bin). Furthermore, efficiency of the pick / move / place operation is important, since maximum part throughput is important.

[0012] A robot 100 having a gripper 102 operates within a workspace 104. The operation of the robot 100 is controlled by a controller 110, which typically includes: cable The controller 110 communicates with the robot 100 via a controller 110 controller provides joint movement commands to the robot 100 and receives joint position data from encoders in the joints of the robot 100 as known in the art. The controller 110 also provides commands to control the movement of the gripper 102. The pick, move and place scenarios of Figure 1 are merely illustrative and exemplary.

[0013] The adaptive gripping and motion planning methods of the present disclosure are applicable to any type of robotic task where parts with irregular poses must be grasped and moved, and the suction cup gripper 102 may be replaced with another type of gripper (e.g., finger type).

[0014] The camera 120 communicates with the control device 110. Work 130In particular, the images from the camera 120 are used to identify the position and orientation of a workpiece being manipulated by the robot 100. In some embodiments, the images from the camera 120 can be used to identify the position where a workpiece is to be placed by the robot 100 or to identify moving or temporary obstacles in the workspace 104. The camera 120 can be replaced with some other means for identifying the position and orientation of a workpiece being moved by the robot 100 (e.g., other types of object sensors - radar, LiDAR, ultrasonic, etc.).

[0015] A number of workpieces 130 are placed in the container 140. The task of the robot 100 is to remove each individual workpiece 130 from the container 140, move the workpiece 130 to place it on the conveyor 150 in a predefined pose (orientation), and then repeat the operation with the next workpiece. The camera 120 provides images of the workpieces 130 in the container 140. An algorithm is used to identify each individual workpiece 130 for the next pick operation based, for example, on the height of the workpiece center above the bottom of the container 140, the centralized location within the container 140, or the accessibility of a preferred pick surface.

[0016] The workpieces 130 all face the same orientation when placed on the conveyor 150. Some of the workpieces 130 on the conveyor 150 appear to have slightly different orientations in Figure 1, but this is simply due to perspective effects. The orientation of the workpieces 130 on the conveyor 150 (known as the target orientation) is predetermined for optimal efficiency of the subsequent processing step, which could be packaging, painting, machining, or other operations.

[0017] For each workpiece 130 in the container 140, a new path must be calculated by the controller 110 (or calculated by another computer and then provided to the controller 110) to cause the robot 100 to move the gripper 102 from a home or approach position to pick the workpiece 130 using the appropriate gripping point and orientation, move the workpiece 130 along the path to the target position and orientation while avoiding obstacles such as the sides of the container 140, and then move the gripper 102 back to the home or approach position to prepare for the next workpiece 130. In some cases, an intermediate stop may be required, in which case the robot 100 places the workpiece 130 on a surface, re-gripping another portion of the workpiece 130, and moves the workpiece 130 to the target. The path calculation must be performed in real time so that the robot 100 can move one workpiece 130 and pick the next workpiece as quickly as possible, so the gripping point and path for each workpiece must be calculated very quickly by the controller 110.

[0018] Figure 2 is an illustration of a workpiece in a container being moved to a predefined target location and orientation via direct modification or via predefined intermediate states according to techniques known in the art. Systems have been developed to perform the pick, move and place operations shown in Figure 1. These existing systems typically use hard-coded, fixed logic to determine where to grip the workpiece and whether intermediate stops are required to place the workpiece at the predefined target orientation.

[0019] In Figure 2, workpieces 230 are again in random positions and orientations within container 240. A camera or other sensor (not shown) is again used to select individual workpieces 230 and identify their positions and orientations. In the ideal case, a selected workpiece 230 can be grasped (or gripped) in a preferred position by a rigid grip, moved to a target point and reoriented in a single motion by the robot. This is the case when workpiece 230A is moved directly along path 250 to the target position and orientation.

[0020] It is well known that articulated robots have joint motion limitations that limit the flexibility of the robot. Therefore, in many cases, it is not possible to move the workpiece 230 directly to the target position and orientation in a single motion. At least one intermediate step is required, for example, when the workpiece 230 is placed in the container 240 such that only one face is accessible, and gripping that face makes it impossible for the robot to move the workpiece 230 to the target orientation. This is the case when the workpiece 230B is moved along the path 260 to an intermediate state. In the intermediate state, the workpiece 230B is placed on the face and the robot releases it. The workpiece 230B is then re-gripped by the robot at a different point and moved along the path 262 to the target position and orientation.

[0021] In prior art systems, the possible initial gripping points on the workpiece 230, the workpiece orientations in intermediate states, and the possible re-gripping points of the workpiece 230 are all hard-coded using fixed logic. This type of hard-coded logic requires time-consuming analysis and programming for each application (workpiece design, container size / shape, target orientation, etc.). Furthermore, this type of fixed logic is difficult to define for complex workpiece geometries, resulting in only one preferred initial gripping location being defined on the workpiece, which may not be accessible for many of the workpieces in the container. Furthermore, the limited options for gripping points and one intermediate state pose may result in excessive robot joint motions, requiring a second intermediate state and a second re-grip to reach the target orientation.

[0022] The grasp and motion planning method of the present disclosure is designed to optimize the efficiency of pick-transfer-place operations by selecting any feasible initial grasp point that moves the workpiece directly to the target point, or by selecting an initial grasp point and intermediate pose that minimizes robot motion when a direct single-step move is not possible.

[0023] FIG. 3 is a diagram of a workpiece 310 with multiple sample grips identified, each of which is evaluated as being robust or not, according to an embodiment of the present disclosure. The first step of the method of the present disclosure is to generate a set G of robust grip locations for the workpiece 310. Given that the workpiece 310 is a solid or a 3D surface model, a set N of random sample grip points on the surface of the workpiece 310 is defined using any suitable technique. Set N can include hundreds of points, although only a few dozen are shown in FIG. 3 for clarity. Each of the sample grip points in set N is analyzed to determine its suitability as a grip point based on the surface flatness of the local area. If an area of ​​suitable size and flatness (to be reliably gripped by a suction cup gripper) is found to exist around a particular point, the point is included in the set G of robust grip locations. The points in the set G of robust grip locations are shown as small circles in FIG. 3. If it becomes apparent that there is no area of ​​suitable size and flatness around a particular point, then that point is not included in the set G of robust grasp locations, indicated in the figure with a small x. Set N contains all sample points, whether robust for grasping or not, and set G contains only robust grasp points.

[0024] For each point in the set of robust gripping locations G, a gripping orientation is also defined as normal (orthogonal) to the local surface tangent plane. The gripping orientation is used to define the approach direction of the robot's gripper tool when gripping the workpiece 310. Other factors can also be considered in determining which points are included in the set of robust gripping locations G. For example, for a particular gripping location and orientation, the offset distance of the center of mass of the workpiece 310 from the centerline of the robot's gripper tool can be calculated. If a particular gripping point is at a peripheral location of the workpiece 310 where the gripper tool centerline passes away from the center of mass of the workpiece, this gripping point will be rejected from the set of robust gripping locations G because it will create a large bending moment in the suction cup gripper. An analysis of the workpiece 310 (for a particular part design) to generate the set of robust gripping locations G is first performed and then used to calculate the gripping and motion planning for each individual workpiece in the container.

[0025] 4 is an illustration of a workpiece 310 according to an embodiment of the present disclosure, where multiple stable workpiece poses are identified. In the adaptive grasp planning method of the present disclosure, when a straightforward transition to the target pose of the workpiece in the container is not possible, various combinations of initial grasps and intermediate poses are analyzed for feasibility and optimality. To perform this analysis, all stable poses of the workpiece 310 must be known in advance. A stable pose is an orientation in which the workpiece 310 can come to rest on a flat surface (or on a combination of points, lines and surfaces) without tilting.

[0026] One technique for identifying stable orientations of the workpiece is to mathematically generate a convex hull (such as a tetrahedron or octahedron) around the workpiece, whose faces all have different sizes, and mathematically generate a bounding ball around the convex hull with the center of mass of the workpiece as its center. Each face of the convex hull is projected onto the bounding ball to obtain an area S. The likelihood of a stable placement of the workpiece on the surface represented by each face of the convex hull is proportional to the area S of this face divided by the surface area of ​​the bounding ball. Intuitively, this means that the likelihood of a stable placement is greater for larger faces of the convex hull.

[0027] Another technique for identifying stable poses of a workpiece is to find various poses that allow stable placement of the workpiece in human experience, and then replicate these poses in the CAD solid modeling system.

[0028] 4, a set P of four different stable poses of the workpiece 310 is shown in box 420. Each of the four stable poses is unique; that is, in each stable pose, the workpiece 310 rests on a different surface relative to the others. An analysis of the workpiece 310 to identify the set P of stable poses is first performed, which are then used in the grasp and motion planning calculations for each individual workpiece in the bin.

[0029] As discussed with reference to Figures 3 and 4, the disclosed method first calculates a set of robust grips G and a set of stable poses P for the workpiece. These calculations are only a function of the shape of the workpiece, and are independent of how the individual workpieces are placed in the bin. A final (target) pose B is also provided, which includes the desired location and orientation of the workpiece after it is placed by the robot. A bin containing a number of actual workpieces is then provided, and images of the workpieces are analyzed. A particular workpiece is selected for movement, and an initial pose A of the selected workpiece is identified from the camera images.

[0030] Then, for each workpiece in the container, an analysis is performed to determine which grip points in the set of robust grips G are possible based on the part's orientation and exposed surfaces in the container. It is then determined whether any of the possible robust grips would move the workpiece directly to the target pose B in a single motion. If any of the possible robust grips would move the workpiece directly to the target pose B in a single motion, the robot controller plans robot motion using inverse kinematics and checks for possible collisions (such as collisions between the robot arm or tool and the side walls of the container) and robot motion limitations. If any of the possible robust grips would move the workpiece directly to the final pose B in a single motion without collisions and within the limitations of the robot motion, the robot controller then executes the planned robot motion commands to move the workpiece. If multiple possible robust grips would move the workpiece directly to the target pose B in a single motion, the grips and motions can be selected based on grip robustness, motion efficiency, or other factors.

[0031] For each workpiece being evaluated, an analysis is performed to find the best grasp and action, including one or more intermediate states in which the workpiece may be temporarily placed and then re-gripped if no possible robust grasp can directly move the workpiece to the target pose B in a single motion, as discussed below.

[0032] 5A is an illustration of a formulated search problem for evaluating the feasibility and optimality of multiple paths from an initial workpiece pose through one intermediate pose to a goal pose, according to an embodiment of the present disclosure. Workpiece 510 is one of many in a bin to be picked and placed by a robot as described above. Workpiece 510 has already been analyzed to determine a set of robust grasps G and a set of stable poses P, as previously discussed with respect to FIGS. 3 and 4. Additionally, a final (goal) pose B has also been defined, including a target location and orientation.

[0033] 5A, a particular workpiece 510 has already been evaluated and it has been determined that no robust grip points are accessible that would move the workpiece 510 from an initial pose A to a target pose B in a single motion. In this situation, a search problem is formulated to consider all possible combinations of grip points and stable intermediate poses. A set P of stable poses of the workpiece 510 is shown in box 520. Each stable pose in set P is a node in the search problem. The workpiece 510 in a target pose B is shown in 530. The search problem is formulated to consider paths 540 from the initial pose of the workpiece 510 to each of the stable poses in box 520 and paths 550 from each of the stable poses in box 520 to the target pose B shown in 530.

[0034] The search problem is evaluated as follows: For intermediate pose 522, it is determined whether at least one gripping point on the workpiece 510 is accessible for an initial workpiece pose that would cause the robot to move the workpiece 510 to intermediate pose 522. If such a move is possible, it is determined whether at least one point on the workpiece 510 is accessible for an intermediate workpiece pose that would cause the robot to move the workpiece 510 to the target pose B shown at 530. This analysis is also performed for intermediate poses 524 (paths 544 and 554), 526 (paths 546 and 556), etc. If only one path is feasible from the initial pose to the target pose, this feasible path is selected and the robot motion is planned. If multiple paths are feasible, the optimal path is selected using the minimum robot joint motion and / or gripper tool travel distance as an objective function. The selected path includes moving the workpiece 510 from the initial pose A to the intermediate pose M, regrip of the workpiece by the robot, and moving the workpiece 510 from the intermediate pose M to the target pose B.

[0035] As mentioned above, before issuing commands to the robot to physically move the workpiece 510, the robot controller must plan the robot's motion using inverse kinematics and check for possible collisions (clash of the robot arm or tool with the side wall of the container) and robot motion limitations. Planning of the robot motion can be performed before the optimal path is selected so that it can be taken into account in selecting the robot's joint motions. Furthermore, multiple robust grasps may be feasible to move the workpiece 510 from the initial pose to a particular intermediate state, such as the intermediate pose 522. If the multiple feasible grasps are substantially different from each other (e.g., the grasp points on different faces of the workpiece 510 have different orientation angles of more than 45°), each different feasible grasp is considered as a path in the search. That is, the path 542 has multiple transformations.

[0036] In summary, FIG. 5A shows how the workpiece 510 is stably moved from its initial position to its final position. Intermediate posture M The feasibility of a path 540 to each of the reachable Intermediate posture M The feasibility of a path 550 from the initial pose to the target pose 530 is evaluated. If multiple complete paths are feasible, the optimality of the feasible complete paths is evaluated. The optimal path is selected and a command is issued to the robot to move the workpiece 510 from the initial pose to the selected intermediate pose M, and to re-grip the workpiece 510 and move it to the target pose 530. In the analysis performed as shown in FIG. 5A, it is possible that there is no feasible complete path at all. This occurs when the workpiece is placed on a corner of the container or in the case of an irregular design where the workpiece has few strong gripping points. If it is not possible to move the workpiece from the initial pose to the target pose using a single intermediate pose M, the search problem is reformulated to include a second stage of intermediate poses. This is discussed below.

[0037] FIG. 5B is an illustration of a search problem formulated to evaluate the feasibility and optimality of multiple paths from an initial pose of a workpiece through two or more intermediate pose stages to a target pose, according to an embodiment of the present disclosure. In FIG. 5B, the workpiece 510 is again shown on the left in an initial pose. The stable poses are again shown in box 520 along with paths 540 from the initial pose to each of the stable poses. However, in this case the stable poses in box 520 represent the first stage intermediate poses, and the second stage intermediate poses are shown in box 560. The second stage intermediate poses in box 560 include a set P of stable poses just like in box 520. At each intermediate pose stage, the workpiece 510 is placed on the work surface, released from the robot's gripper, and re-gripped using another one of the grips in the set G of robust grips.

[0038] The formulation of the search problem continues as follows: A set of paths 570 is defined such that one path is evaluated from each reachable intermediate pose of the frame 520 (first stage) to each intermediate pose of the frame 560 (second stage) using any robust grasp of the set G. Then, a set of paths 580 is evaluated from each pose of the frame 560 (second stage) reachable by one or more of the paths 570 to the target pose 530 using any robust grasp of the set G. It should be appreciated that the search problem shown in FIG. 5B has many more paths and combinations than the search problem of FIG. 5A. A complete end-to-end path is defined as including one path from each of the sets 540, 570, and 580 together with two intermediate poses M1 and M2, for example, from the initial pose of the workpiece 510 to the first stage intermediate pose 522, to the second stage intermediate pose 562, to the target pose 530. The feasibility of the complete end-to-end path is evaluated, and then the optimality of the feasible end-to-end path is evaluated.

[0039] From the above analysis, an optimal path is selected and the robot is commanded to move the workpiece 510 from the initial pose A to the selected first-stage mid-pose M1, re-grip the workpiece 510, move to the selected second-stage mid-pose M2, re-grip the workpiece 510, and move to the target pose 530. In the unusual case where no feasible end-to-end path can be found using the two-stage mid-poses of Figure 5B, a search problem using three-stage mid-poses is formulated and evaluated, and so on.

[0040] 6 is a flowchart 600 of a method for adaptive robotic grasp and motion planning including multiple grasp and intermediate poses to optimize robot motion, in accordance with an embodiment of the present disclosure. In box 602, a set of robust grasps G and a set of stable poses P are calculated for each workpiece, as shown in FIGS. 3 and 4 and discussed above. These calculations are only a function of the workpiece geometry, and are independent of how the individual workpieces are placed in the container.

[0041] In box 604, a bin containing a number of actual workpieces is provided, and camera images (or sensor data) of the workpieces are provided. In box 606, an individual workpiece is selected for movement, and an initial pose A of the selected workpiece is determined from the camera image. In box 608, a target pose B is also provided, which includes a desired position and orientation of the workpiece after it is placed by the robot. The final pose B may be the same for all workpieces in the bin, or the final pose B may be different from one workpiece to the next.

[0042] In box 610, infeasible grasps are removed from the set of robust grasps G. Infeasible grasps are grasps that are not possible to grasp or move the workpiece directly to the target pose B in a single motion. First, grasps from set G that are not possible based on the selected initial pose A of the workpiece are eliminated. That is, only robust grasp points on the faces of the workpiece that are exposed and facing the open space can remain in set G. Then, of the remaining grasps, those that do not allow direct movement from the initial pose A to the target pose B are eliminated.

[0043] In decision diamond 612, it is determined whether set G is empty after eliminating infeasible grasps. If set G is empty, in box 614, an intermediate state M is selected from the set P of stable poses, and in box 616, grasping and motion of the workpiece is planned from the initial pose A to intermediate state M and from intermediate state M to target pose B. The selection of intermediate state M and planning of grasping and motion from A to M and B are described in detail above in connection with FIG. 5A. In the event that there is no feasible path from A to a single intermediate state M and then to B, two or more stages of intermediate poses (e.g., M1 and M2) can be selected as described above.

[0044] If set G is not empty in decision diamond 612, then in box 618 the grasping and movement of the workpiece is planned from an initial pose A to a successive target pose B. In box 620, robot motion commands are provided from the controller to the robot to cause the robot to physically grasp the workpiece and move from the initial pose A to the target pose B. The grasping and movement of the workpiece from the initial pose A to the target pose B is either direct if possible (from box 618) or uses one or more intermediate states M if not (from boxes 614 and 616).

[0045] At decision diamond 622, it is determined from the camera image whether the current container is empty. If the current container is not empty, the process loops back to box 606 to identify the next workpiece and determine its initial pose A. If the current container is empty at decision diamond 622, then at box 624 the process waits for the next container of workpieces to be loaded (back to box 604), and if there is no next container of workpieces coming, the process ends.

[0046] In the above discussion, various computers and controllers are described and suggested in connection with the steps of the disclosed methods. It should be understood that the software applications and modules of these computers and controllers are executed on one or more computing devices having a processor and memory modules. In particular, this includes the processor in the robot controller 110 of FIG. 1 as discussed above. In particular, the processor in the controller 110 is configured to adaptively plan the gripping and motion of the workpiece as described throughout the disclosure above, and in particular in the flowchart 600. It is also possible that another computer performs some or all of the gripping and motion planning and provides the calculated path to the controller 110. In this case, the controller 110 simply controls the robot 100, and the other computer is configured to perform the gripping and motion planning calculations.

[0047] As discussed above, the disclosed techniques of adaptive robotic grasp and motion planning improve the speed and reliability of robotic grasp and path planning. The disclosed techniques provide more options than prior art for grasping and moving a workpiece to a target pose in a single motion, and the disclosed techniques also find optimal grasp and motion planning solutions when intermediate workpiece configurations are required en route to a target pose. The disclosed method is applicable to complex workpieces, eliminates pre-programming of static grips and poses, and selects a grasp that maximizes grasp strength and minimizes robot travel distance given the most appropriate workpiece temporary configuration.

[0048] Although numerous preferred forms and embodiments of the adaptive robotic grasping and motion planning method have been discussed above, those skilled in the art will recognize modifications, permutations, additions, and subcombinations thereof, and therefore, the following claims are intended to be interpreted as including all such modifications, permutations, additions, and subcombinations that fall within their true spirit and scope.

Claims

1. A grasp planning method for an industrial robot, comprising: Determining a set of firm grips and a set of stable postures for the shape of the workpiece; Determining an initial orientation of a workpiece; Providing a target posture of the workpiece; filtering out infeasible grasps from the set of robust grasps, selecting feasible grasps, and planning robot motions to grasp the workpiece and move directly from the initial pose to the target pose; if no viable grasp exists, selecting an intermediate pose from the set of stable poses, planning a robot motion to grasp the workpiece and move from the initial pose to the selected intermediate pose, and re-grasp the workpiece and move to the target pose; sending commands from a robot controller to the robot for the robot to move the workpiece using a planned motion of the robot.

2. The method of claim 1 , wherein the initial pose, the target pose, and the intermediate poses each comprise positions and orientations defined in a single common coordinate system or multiple coordinate systems having known relative position and orientation changes.

3. The method of claim 1 , wherein determining an initial orientation of the workpiece comprises analyzing camera images or sensor data of a plurality of workpieces in a container, the workpieces having a shape of the workpiece.

4. The method of claim 3 , including returning to determine an initial pose of another workpiece in the container after moving the workpiece using a planned motion of the robot.

5. 2. The method of claim 1, wherein determining the set of strong grips comprises: defining a plurality of sample points on a three-dimensional model of the shape of the workpiece; selecting each sample point having a surrounding area with a size and surface flatness that meets predetermined criteria; and including in the set of strong grips the selected sample points, each of which has an orientation vector perpendicular to a tangent plane of the surface.

6. The method of claim 1 , wherein determining the set of stable poses comprises identifying each unique pose in which the workpiece feature can be stably placed on a flat surface.

7. 2. The method of claim 1 , wherein eliminating infeasible grasps from the set of robust grasps comprises: eliminating grasps that are not accessible by the robot based on an initial pose of the workpiece; and eliminating grasps that do not allow the robot to directly move the workpiece from the initial pose to the target pose.

8. 2. The method of claim 1 , wherein planning robot motion to grasp and move the workpiece includes using inverse kinematics calculations to calculate robot joint motions to grasp and move the workpiece with a gripper tool of the robot, and further includes performing collision avoidance calculations for the motion of the robot and the workpiece.

9. 2. The method of claim 1 , wherein selecting the intermediate postures from the set of stable postures comprises formulating a search problem that includes a path from the initial posture to each of the stable postures and a path from each of the stable postures to the target posture.

10. 10. The method of claim 9, further comprising selecting the intermediate pose by solving the search problem to find a feasible and optimal path from the initial pose to one of the stable poses and to the target pose, where optimality is determined by an objective function including workpiece travel distance, workpiece change in orientation, robot joint travel angle, or a combination thereof.

11. The method of claim 10 , further comprising, if no intermediate configurations with feasible paths are found, reformulating the search problem to include two or more intermediate configurations.

12. A grasp planning system for an industrial robot, comprising: A means for determining an initial orientation of the workpiece; A robot control device that communicates with the robot, the robot controller eliminates infeasible grasps from a set of robust grasps of the workpiece, selects feasible grasps, and plans a motion for the robot to grasp the workpiece and move directly from the initial pose to the target pose; if no viable grasp exists, selecting an intermediate pose from a set of stable poses of the workpiece, planning a robot motion to grasp the workpiece and move it from the initial pose to the selected intermediate pose, and re-grasp the workpiece and move it to the target pose; A grasp planning system having a processor and memory configured to send commands to the robot to move the workpiece using planned movements of the robot.

13. The system of claim 12 , wherein the initial pose, the target pose, and the intermediate pose each include a position and an orientation.

14. The system of claim 12 , wherein the means for determining an initial pose of a workpiece includes a camera or sensor that provides data about a plurality of workpieces in a container to the robot controller.

15. 13. The system of claim 12, wherein the set of robust grips is determined by defining a plurality of sample points on a three-dimensional model of the shape of the workpiece, selecting each sample point having a surrounding area with a size and surface flatness that meets predetermined criteria, and including the selected sample points, each having an orientation vector perpendicular to a tangent plane of the surface, in the set of robust grips.

16. The system of claim 12 , wherein the set of stable poses is determined by identifying each unique pose in which the workpiece feature can be stably placed on a flat surface.

17. 13. The system of claim 12, wherein eliminating infeasible grasps from the set of robust grasps comprises: eliminating grasps that are not accessible by the robot based on the initial pose of the workpiece; and eliminating grasps that do not allow the robot to directly move the workpiece from the initial pose to the target pose.

18. 13. The system of claim 12, wherein planning robot motion to grasp and move the workpiece includes using inverse kinematics calculations to calculate robot joint motions to grasp and move the workpiece with a gripper tool of the robot, and further includes performing collision avoidance calculations for movement of the robot and the workpiece.

19. 13. The system of claim 12, wherein selecting an intermediate posture from the set of stable postures includes formulating a search problem that includes a path from the initial posture to each of the stable postures and a path from each of the stable postures to the target posture.

20. 20. The system of claim 19, further comprising: selecting the intermediate pose by solving the search problem to find a feasible and optimal path from the initial pose to one of the stable poses and to the goal pose, where optimality is determined by an objective function that includes a weighted combination of workpiece travel distance, workpiece change in orientation, and robot joint travel angles; and further comprising reformulating the search problem to include two or more intermediate poses if an intermediate pose with a feasible path is not found.

Images