Framework for online robot operation plan

Abstract

To provide a technique for planning robot operation, that uses an external computer communicating with a robot controller.SOLUTION: A camera or a sensor system provides a computer with input scene information containing start and goal points, and obstacle data. The computer plans an operation of a robot tool on the basis of the start and goal points and an obstacle environment, and robot operation is planned by using in-series or in-parallel combinations of sampling base and optimization base planning algorithms. With the in-series combinations, an implementable path is found out using a sampling method, and an optimization method improves the quality of the path. With the in-parallel combinations, both sampling and optimization are used, and a path is selected according to factors such as calculation time and path quality. The computer converts a congested passing point planned for transfer to a robot controller, to a sparse command point, and the controller calculates robot kinematics and interpolation points in order to control operation of the robot.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates generally to the field of industrial robot motion control, and more particularly to a robot motion control technique that uses a computer separate from the robot controller to plan robot motion based on start and finish points and an obstacle environment, where robot motion planning is performed using a serial or parallel combination of sampling-based and optimization-based planning algorithms to convert densely planned waypoints into sparse command points for transfer to the robot controller. [Background technology]

[0002] Industrial robots are widely used to perform a variety of manufacturing, assembly, and material handling tasks. In some robotic applications, the start and / or finish points vary for every robot task. For example, a robot's task may be to pick up parts from an inbound conveyor and place them in an open location in a shipping container. In such applications, a new robot motion path must be calculated in real time for each task. Furthermore, in many robot workspace environments, obstacles may exist and be located in the robot's motion path. The obstacles may be permanent structures, such as machinery or fixtures, or they may be temporary or mobile. Collisions between the robot and any obstacles must be avoided at all costs.

[0003] Various techniques are known in the art for planning robotic tool path motion based on start and finish points and the obstacle environment. One commonly used technique is a sampling-based method, in which a random sample of waypoints is defined and a collision-free tree of waypoints is grown until the start and finish points are connected. Another known technique is an optimization-based method, in which waypoints along the path from the start to the finish point are iteratively modified using an optimization objective and constraint function until a collision-free path is obtained. Other techniques exist, including variations on sampling-based methods.

[0004] A recent trend in robot motion planning is to perform motion plan calculations on a computer separate from the robot controller. Systems are known that perform sampling-based motion planning to find collision-free paths, then calculate interpolation points to generate a robot tool trajectory, and then transfer the trajectory to the robot controller for execution by the robot. However, these systems require intensive communication bandwidth, fail to take advantage of the robot motion plan calculation capabilities of the robot controller, and may not provide path plans of the desired quality.

[0005] In view of the foregoing, there is a need for a robot motion planning technique that provides high-quality path planning, is fast enough to execute in real time while the robot is moving, and can take advantage of the inherent computational performance of the robot controller. Summary of the Invention

[0006] In accordance with the teachings of the present disclosure, a robot motion planning technique is disclosed that uses an external computer in communication with a robot controller. A camera or sensor system provides input scene information, including start and finish points and obstacle data, to the external computer. The computer plans the robot's tool motion based on the start and finish points and the obstacle environment, where the robot motion is planned using a serial or parallel combination of sampling-based and optimization-based planning algorithms. In serial combination, a sampling method first finds a feasible path, and an optimization method improves the path quality. In parallel combination, both sampling and optimization methods are used to select a path based on computation time, path quality, and other factors. The computer converts the planned dense waypoints into sparse pathpoints for transfer to the robot controller, and the controller calculates the robot kinematics and interpolated points, which are used to control the robot's motion.

[0007] Additional features of the disclosed apparatus and methods will become apparent from the following description and appended claims, considered in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] Figure 1 shows an industrial robot performing pick, move, and place operations in an environment where a new path must be calculated for each workpiece move and the workspace contains one or more obstacles that must be avoided. [Figure 2] FIG. 2 is a block diagram of a robot motion path planning system using an external computer in communication with a robot controller in the known prior art. [Figure 3] FIG. 3 is a block diagram of a robot motion path planning system using an external computer in communication with a robot controller according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a block diagram of a motion planning module of a system using a combination of optimization and sampling motion planning methods in the embodiment of the present disclosure of FIG. [Figure 5] FIG. 5 is a flowchart of a path selection method using a serial combination of sampling and optimization motion plans in the motion planning module of FIG. 4 according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a flowchart of a path selection method using a parallel combination of sampling and optimization motion planning in the motion planning module of FIG. 4 according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a block diagram of a path function fitting module used in the motion planning module of FIG. 4 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] The following discussion directed to robot motion planning techniques using combined optimization and sampling methods and path function fitting in accordance with embodiments of the present disclosure is merely exemplary in nature and is not intended to limit in any way the disclosed apparatus and techniques or their applications or uses.

[0010] The use of industrial robots for a variety of manufacturing, assembly, and material handling operations is widely known. For some of these operations, the robot must be programmed to move along a path that has a different start and / or goal point from one operation to the next. For example, in a move, pick, and place operation, an incoming part on a conveyor may be picked up by the robot from a different location each time, and a part placed by the robot in a shipping container may be placed in a different designated location within the shipping container. Or, a part may be picked by the robot from a bin full of parts and placed on an outgoing conveyor. Many other applications exist where a new path must be calculated for each independent task or operation of the robot.

[0011] Furthermore, in many robot workspace environments, obstacles exist that may be in the robot's motion path. One or more obstacles may be located between the start and finish points, or more generally, between the robot's current position and the robot's desired position. Obstacles may be permanent structures such as machinery or furniture, or they may be temporary or mobile.

[0012] FIG. 1 illustrates an industrial robot performing pick, move, and place operations in a scenario of the type described above, where the workspace environment includes one or more obstacles to avoid and a new path must be calculated for each workpiece movement. The robot 100 has a gripper 102 and operates within a workspace 104. The movement of the robot 100 is typically controlled by a controller 110, which communicates with the robot 100 via a cable 112. As is known in the art, the controller 110 provides joint movement commands to the robot 100 and receives joint position data from encoders on the robot 100's joints. The controller 110 also provides commands to control the movement of the gripper 102. The pick, move, and place scenario of FIG. 1 is for purposes of explanation and example only. The optimization motion planning techniques of the present disclosure are applicable to any type of robot operation, and the gripper 102 may be replaced by any type of robot tool.

[0013] A computer 120 is used to calculate the robot's tool path in a manner discussed below. A camera 122 is in communication with the computer 120 and provides images of the workspace 104. Images from the camera 122 may be used to identify the position and orientation of a workpiece being manipulated by the robot 100 and / or where the workpiece will be placed by the robot 100, as well as any moving or temporary obstacles within the workspace 104. The camera 122 may be replaced by or used to augment any other type of sensor suitable for the start and finish points (position and orientation) of the workpiece.

[0014] A workpiece 130 enters the workspace 104 on a conveyor 140. The task of the robot 100 is to pick up the workpiece 130 from the conveyor 140, move the workpiece 130, and place it inside a container 150 (e.g., a shipping container). The position and orientation of a start point 160 are defined based on information about the position of the workpiece 130 on the conveyor 140 and the speed of the conveyor 140. The orientation of the workpiece 130 at the start point 160 is required to determine the orientation of the gripper 102. Similarly, the position and orientation of a goal point 162 are defined based on information about the next available compartment or location within the container 150. FIG. 1 shows one waypoint 164 located between the start point 160 and the goal point 162. In a real robot motion planning calculation, multiple intermediate path points are calculated to satisfy all path constraints, and a smooth tool path is calculated through the multiple path points. Again, this is just one example of various start and finish points, and many other examples are possible. For example, the operations may be performed in reverse, such as the workpiece 130 being picked up from a bin full of parts and placed on the conveyor 140 in a particular orientation.

[0015] There is an obstacle 170, shown as a wall, between the start point 160 and the finish point 162. It will be appreciated that the sides of the container 150 represent obstacles that must be avoided by the robot 100 and taken into account in path planning. Other obstacles may also be present.

[0016] For each workpiece 130 arriving on the conveyor 140, the controller 110 and computer 120 must calculate a new path for the robot 100 to move the gripper 102 along path segment 180 from a home or approach position to pick up the workpiece 130 at the start point 160, move the workpiece 130 along path 182 toward the finish point 162 while avoiding obstacles 170, and return the gripper 102 to the home or approach position for the next workpiece 130. The new path must be calculated very quickly by the controller 110 and computer 120 because the path calculation must be done in real time, as fast as the robot 110 can move one workpiece 130 and return to pick up the next workpiece 130.

[0017] 2 is a block diagram of a known prior art system for robot motion path planning using a computer 120 in communication with a robot controller 110. To provide an input scene description, a sensor processing and task management module 210 receives images from a camera 122 or data from different types of sensors. Briefly, module 210 determines start and finish points for upcoming robot tasks by identifying parts coming in on a conveyor or specific parts to be removed from a container, and the associated destination positions and poses (e.g., on a conveyor or within a container). The locations of obstacles in the workspace may also be provided by module 210, especially if moving obstacles are present or likely to be present.

[0018] The motion planning module 220 calculates the tool path based on the data from module 210. Various techniques for planning a robot's tool path motion based on the start and finish points and the obstacle environment are known in the art. One commonly used technique is a sampling-based method, in which random sample waypoints are defined and evaluated, and a collision-free tree of waypoints is extended until the start and finish points are connected. One particular sampling-based technique is called the rapidly-exploring random tree (RRT) method. RRT rapidly builds a tree of nodes that randomly fills the space from the initial configuration to the final configuration until a collision-free path is found. Given surface / space or point cloud data representing obstacles and the initial and final workpiece configurations (start and finish points and poses), RRT can be used to determine the motion sequence, if any. Each node in an RRT motion plan must be evaluated and determined to be collision-free to be included as a valid or feasible waypoint. Furthermore, because each candidate waypoint is randomly placed, RRT may find a different solution each time, even if the initial and boundary conditions are the same. Furthermore, the paths defined by RRT are inherently curved (zigzag), and the "best" candidate waypoints at each step are usually off-axis from the ideal direction. Therefore, RRT-based motion planning techniques have been found to be ineffective in some applications.

[0019] The trajectory generation module 230 receives the tool path motion plan from the motion planning module 220 and calculates the entire trajectory. In the prior art system of Figure 2, the trajectory generation module 230 calculates all of the trajectory interpolation points and generates a dense point cloud for the robot tool motion. The trajectory interpolation points are transferred from the computer 120 to the robot controller 110 and received by the interpolation module 260. Because the trajectory interpolation points have already been calculated by the trajectory generation module 230 on the computer 120, the interpolation module 260 on the controller 110 simply receives the trajectory interpolation points and passes them to the servo-motor control module 270, which drives the joint motors of the robot 100 in a known manner.

[0020] Systems of the type shown in FIG. 2 have proven effective. However, these systems have several disadvantages. The system of FIG. 2 bypasses the software functions of the robot controller 110, underutilizing the controller 110 and placing additional computational burden on the computer 120. Bypassing the computational functions of the controller 110 can result in slower and less smooth robot motion than if the controller 110 were able to perform more of the kinematic and interpolation calculations itself. Additionally, data exchange (real-time data streaming for multiple points during each interpolation period) adds additional overhead, and data packet loss can cause discontinuities in the robot's motion. Finally, conventional systems such as those shown in FIG. 2 typically rely on sampling-based motion planning, which can result in undesirably jagged motion and random motion patterns.

[0021] This disclosure describes systems and methods that overcome some of the shortcomings of prior art systems, and these techniques are discussed in detail below.

[0022] Figure 3 is a block diagram of a robot motion path planning system using an external computer in communication with a robot controller according to an embodiment of the present disclosure. The system of Figure 3 uses a computer 120 in communication with a controller 110 as shown in Figure 1. However, the system of Figure 3 includes different programming configurations in both the computer 120 and the controller 110 than the system of Figure 2. Specifically, the computer 120 is configured to plan the best quality path possible within the allotted cycle time, and the controller 110 uses all of its capabilities to calculate robot kinematics and interpolation points to provide smooth and rational tool motion.

[0023] In Figure 3, sensor processing and task management module 210 operates as described above for Figure 2. In other words, module 210 receives camera images or sensor data and provides start and finish points along with obstacle data for upcoming robot tasks. The start and finish points and obstacle data are provided to motion planning module 320. Motion planning module 320 in Figure 3 differs significantly from motion planning module 220 in Figure 2. Details of motion planning module 320 are discussed below with reference to Figures 4, 5, and 6.

[0024] The motion planning module 320 provides a sparse sequence of planned path points to the top-level program execution module 340 of the robot controller 110. The program execution module 340 handles high-level functions such as interaction with the operating system, the user interface, the data interface, and control of other software modules. The program execution module 340 provides the sparse planned path points to the planning module 350. The planning module 350 performs several functions, including interpreting the sparse planned path points and determining which inverse kinematics calculation to use (e.g., whether to calculate all joints simultaneously or to calculate the first three joints first independently of the last three joints). The planning module 350 also defines trigger points for tools attached to the robot, such as grab or release commands for a gripper, or to turn a torch on or off in a welding operation.

[0025] The planning module 350 provides robot motion and control instructions to an interpolation module 260, similar to the similarly numbered modules in Figure 2. However, in this case, the interpolation module 260 adds interpolation points to define a complete set of motion instructions for the servo motor control module 270 that drives the joint motors of the robot 100, as previously described.

[0026] By providing a sparse sequence of points from the computer 120 to the program execution module 340, all of the essential capabilities of the robot controller 110 can be utilized, resulting in the robot kinematic calculations, interpolated points, and motion instructions that will cause the robot 100 to move smoothly and without wavering along the path calculated by the motion planning module 320.

[0027] Figure 4 is a block diagram of a motion planning module 320 that uses a combination of optimization and sampling motion planning methods in an embodiment of the present disclosure in the system of Figure 3. As described above, the motion planning module 320 is software that runs on the computer 120 and receives start and finish point and obstacle environment data from the sensor module 210.

[0028] The motion planning module 320 includes both an optimization method calculation module 410 and a sampling method calculation module 420. Both the optimization module 410 and the sampling module 420 receive start and finish point and obstacle data, and modules 410 and 420 may operate independently. The optimization method calculation module 410 starts with an initial reference path, which may be a straight line from the start point to the finish point, or other initial reference (described below), and executes an optimization routine that includes an objective function and a constraint function. The optimization routine iterates until the constraints are satisfied and the objective function is minimized, resulting in a collision-free path with minimum distance and maximum path quality (smoothness, curvature, etc.). The sampling method calculation module 420 executes a random tree growing program using the method described above until a collision-free path from the start point to the finish point is obtained. The outputs from the optimization module 410 and the sampling module 420 may be used and combined in several different ways, as indicated by the dotted arrows in FIG. 4 and discussed below.

[0029] While the diagram shown in FIG. 4 is two-dimensional, it will be understood that all calculations in the presently disclosed technology are three-dimensional and involve six degrees of freedom (DOF) for applicable items. Start and finish points include position (three coordinates) and orientation (three angular directions). Obstacles are defined as three-dimensional objects, either as CAD surface or solid models, or as point clouds or surface data from the sensor module 120. The calculated tool path exists in three-dimensional space and includes not only the position but also the orientation (6 DOF) of the tool center point. In addition, collision avoidance calculations are performed not only between the robot tool and obstacles, but also between all parts / arms of the robot and obstacles.

[0030] The motion planning module 320 includes a path selection module 430 that selects the best path to use from the candidate planned paths from the optimization module 410 and / or sampling module 420 for each cycle of the robot. The path selection module 430 may be configured in one of two different ways: a serial or parallel combination of optimization and sampling paths. Two configurations of the path selection module 430 (identified as 430A and 430B, respectively) are discussed below with reference to Figures 5 and 6. By using a combination of optimization and sampling algorithms, the path selection module 430 can take advantage of the advantages of both methods and avoid the disadvantages of each method to obtain the best possible planned tool path.

[0031] The path selection module 430 provides the planned tool path to the path function fitting module 440. The planned tool path from the path selection module 430 includes a fairly dense sequence of way points necessary to ensure that no collisions occur between the way points. For transfer to the robot controller 110, the path function fitting module 440 converts the dense sequence of way points into a sparse sequence of command points. The sparse sequence of points provided by the path function fitting module 440 is calculated by the robot controller so that it can calculate interpolated points to match the original dense sequence of way points provided by the path selection module 430. The path function fitting module 440 (described below) provides the sparse sequence of command points to the tool motion interface module 450, which transfers the sparse command points to the top-level program execution module 340 on the robot controller 110 as described above.

[0032] 5 is a flowchart of a path selection method 500 using serial combination of sampling and optimization motion plans in the motion planning module of FIG. 4 in accordance with an embodiment of the present disclosure. A block diagram of the motion planning module 320 (of FIG. 4) is shown on the left side of FIG. 5 for reference. Flowchart diagram 500 illustrates one embodiment of a path selector (serial combination) encoded in path selection module 430A of the motion planning module 320.

[0033] A robot motion cycle involves the robot picking up a workpiece at a start point, pausing, moving the workpiece to a finish point, placing it there, pausing, and returning to a ready position to pick up another workpiece. The calculations in motion planning module 320, and thus the calculations in flowchart 500, are performed for each robot motion cycle.

[0034] Data for the current robot cycle (or task) is received from the camera or sensor module 210 on input line 502. The data includes start and finish points, part motion poses, and obstacle data. Processing then branches depending on whether the current robot task is expected to have a similar motion pattern to the previous task (with only a slight difference in the start and / or finish points). Decision diamond 510 determines whether a process reset has occurred. A process reset could be an event such as providing a new bin of parts to pick (which will result in a significantly different start point than the previous part) or providing a new shipping container to place parts in (which will result in a significantly different finish point than the previous part). Any change in obstacle data, such as the entry of a mobile obstacle into the workspace that could affect possible robot tool motion, will result in a process reset. The answer to reset the process is also YES on the first pass through flowchart 500.

[0035] If the answer to resetting the process at decision diamond 510 is YES, i.e., if the current robot task may not perform similarly to the previous task, the process proceeds to sampling method box 420. In sampling method box 420, a collision-free path from the start point to the goal point is calculated using a sampling method such as RRT. As mentioned above, sampling methods may calculate paths with undesirable characteristics, such as zigzags. Therefore, the serial combination method of FIG. 5 is followed by an optimization path planning step that uses the results of the sampling path plan as input to improve the shape of the planned path.

[0036] The optimization method box 410 receives the initial path calculated by the sampling method box 420 along with the start and finish point and obstacle data, and uses the initial path as an initial reference for the optimization calculation. The optimization method performs an iterative optimization calculation involving an objective function and a constraint function to find a collision-free path from the start point to the finish point with optimal path quality characteristics. The iterative calculation requires an initial reference path to be used in the first iteration. The initial reference path can be a primitive straight line from the start point to the finish point, but this may require excessive optimization calculations and may not reach a solution. Because the sampling-based initial path is collision-free and connects the start and finish points, the sampling-based path serves as a very good initial reference for the optimization method calculation. The optimization method calculation in box 410 improves path quality by removing undesirable movements and zigzags.

[0037] In box 520, the quality of the path from optimization method box 410 is evaluated. Quality may be determined by a combination of tool path travel distance, path curvature, path execution time, path smoothness, and / or other factors. The objective function typically includes these parameters and therefore inherently provides a smooth, well-formed path, and path quality should usually be very good because it is the result of the optimization calculation. Insufficient path quality is likely only if the optimization calculation times out and does not converge, and a sampling-based initial path is provided in box 520. If the path quality in box 520 is insufficient (lack of smoothness, jerky sections), a system failure is declared and the part is not lifted by the robot.

[0038] If the path quality is sufficient in box 520, the path from optimization method box 410 is provided on output line 530. When the planned path is provided on output line 530, the planned path is sent to path function fitting module 440 (mentioned above and discussed further below), and a final set of sparse command points is provided to tool path motion interface module 450, which forwards the sparse command points to robot controller 110.

[0039] When the planned path is output on line 530 from box 520, the process also loops back at 532 to begin calculations for the next robot cycle. At box 540, the path just output on line 530 is stored in a data repository for later use in generating an initial reference path, as described below. The process then returns to the top of the flowchart, to input line 502.

[0040] After the robot executes a pick-up operation along the just-planned path, new scene data is provided on input line 502 from camera / sensor module 210. If there is no significant process reset (a new bin of input parts, a new output shipping container, or a new obstacle), the answer to decision diamond 510 is NO. Processing then moves to box 550 for initial reference generation. Initial reference generation involves providing an initial reference path for the optimized motion calculation, where the initial reference path is based on the shape of the previously planned path and is fine-tuned (scaled and moved) so that the start and finish points align with the path currently being calculated. An initial reference generation technique suitable for use in box 550 is described in U.S. patent application Ser. No. 16 / 839,720, filed April 3, 2020, and assigned to the assignee of the present invention, entitled "Initial Reference Generation for Optimal Motion Planning of a Robot," the contents of which are incorporated herein by reference.

[0041] Because the path being calculated is very similar to that of previously calculated paths (similar obstacle environment, similar start and finish locations), the initial reference generation in box 550 can quickly provide an initial reference path that is a very good approximation of the path currently being calculated. Thus, the optimization method in box 410 quickly converges on an optimized path and provides a planned path to box 520. Thus, each planned path is output on line 530 and executed by the robot, after which there is a loopback to store the planned path and receive new input scene data to calculate a new path, and the process continues.

[0042] The serial combination of sampling-based and optimization-based motion planning described above offers a number of unique advantages in robot motion computation. Other approaches using parallel combinations of sampling-based and optimization-based motion planning are discussed below.

[0043] Figure 6 is a flowchart of a path selection method using a parallel combination of sampling and optimized motion plans in the motion planning module of Figure 4 in accordance with an embodiment of the present disclosure. A block diagram of motion planning module 320 (of Figure 4) is again shown on the left of Figure 6 for reference. Flowchart diagram 600 illustrates one embodiment of a path selector (parallel combination) encoded in path selection module 430B of motion planning module 320. As before, the calculations in motion planning module 320, and thus the calculations in flowchart diagram 600, are performed for each robot motion cycle or task.

[0044] Candidate planned paths are received from the optimization module 410 and the sampling module 420 at input lines 602 and 604. The optimization module 410 and the sampling module 420 independently calculate path candidates based on data (start and finish points and obstacle data) from the camera or sensor module 210. The optimization module 410 may provide input at line 602, and the sampling module 420 may provide input at line 604, or vice versa. Decision diamonds 610 and 620 determine whether a candidate planned path has been received at their respective input lines. If not, the decision diamonds continue to wait / loop until a path candidate is received. If a path candidate is received at either decision diamond 610 or 620, the path is passed to box 630. Box 630 periodically evaluates the number of path candidates received. If two candidate paths have been received (one each from the optimization module 410 and the sampling module 420 ), the two candidate paths are passed from box 630 to box 650 .

[0045] In box 650, the quality of the candidate paths is evaluated to select one to use. Quality may be determined as a combination of robot joint path travel distance, path curvature, path execution time, path smoothness, and / or other factors. The path with the highest quality score is selected and output on line 660, provided to the path function fitting module 440, and ultimately forwarded to the robot controller 110 for execution by the robot 100.

[0046] If only one path candidate has been received in box 630 (either from optimization module 410 or sampling module 420), the path candidate is passed from box 630 to decision diamond 640. Decision diamond 640 determines whether a latency threshold has been exceeded. The latency threshold may be predefined based on the robot's cycle time, or in other words, determined based on the robot's ideal cycle time, which is the maximum time allowed to wait in decision diamond 640. If the latency threshold has not been exceeded, processing loops back from decision diamond 640 to box 630 to again periodically check how many path candidates have been received. Either a second path candidate is received and both paths are provided to box 650, or the latency threshold is eventually reached in decision diamond 640 and one path candidate is provided to box 650.

[0047] If a decision diamond reaches its latency threshold and only one candidate path is provided to box 650, box 650 does not compare the quality of one candidate path against the other. In this case, box 650 evaluates the quality of the one candidate path based on the same criteria as discussed above. If the path quality is adequate, the path is provided to output line 660. If the path quality is not adequate (e.g., excessive zigzag or too long a path length), no path is provided to output line 660 and a system failure is declared (no parts picked up this cycle).

[0048] When the planned path is provided on output line 660, the path is sent to the path function fitting module 440 (mentioned above and discussed further below), and a final set of sparse command points is provided to the tool path motion interface module 450, which forwards the sparse command points to the robot controller 110. Processing starts again from the beginning of flowchart 600 as the robot's next cycle, and when ready, sensor data is provided on input lines 602 and 604 to the optimization module 410 and sampling module 420, which provide calculated path candidates.

[0049] Returning to the motion planning module 320 of Figure 4, two embodiments have been discussed above for the path selection module 430: a serial combination module 430A and a parallel combination module 430B. In either case, a path of appropriate quality is provided to the path function fitting module 440, described below.

[0050] 7 is a block diagram 700 of the path function fitting module 440 used in the motion planning module 320 of FIG. 4 according to an embodiment of the present disclosure. The purpose of the path function fitting module 440 is to provide a sparse set of command points for the planned path to provide to the robot controller 110.

[0051] A path is planned for the current robot task, such as moving a part from a start point to a goal point while avoiding collisions with obstacles, in path selection module 430. Path planning may be performed using path selection module 430A (a serial combination of sampling and optimization methods) or path selection module 430B (a parallel combination of sampling and optimization methods), as described in detail above.

[0052] The path selection module 430 provides a dense sequence of waypoints because it is necessary to ensure that there are no collisions along the entire path. If the planned waypoints are sparse and widely spaced, collisions between waypoints may exist and go undetected. For reasons previously discussed, including communication bandwidth requirements and the requirement that the robot controller 110 perform robot kinematics and interpolation point calculations, it is not required to send a dense sequence of waypoints to the robot controller 110.

[0053] The path function fitting module 440 shown at 442 first fits the planned path points (q,...,q T ) to calculate the spline function s. The spline function s may be calculated using any suitable technique, such as fitting a piecewise cubic polynomial to a series of planned path points. The spline function s may be represented as a continuous entity with an arc length parameter a indicating the distance along the spline s. It has a = 0 at the start point q0 and a = 0 at the goal point q T where a=1.

[0054] The next computation of the path function fitting module 440, shown at 444, is to compute a sparse set of path points from the spline s. Based on the robot task and the nature of the path, the number of command points N to be transferred to the robot controller 110 is selected. A typical example of a pick, move, and place operation defined by a three-dimensional curve shape may require sending 10 command points to the robot controller 110 (whereas the planned path included a much larger number of way points, such as 30). Thus, in this example, the value of N is 10. The 10 path points P[1], ..., P

[10] are then computed by inputting the appropriate value of the arc length parameter a into P[1] (a = 0) through P

[10] (a = 1).

[0055] The path function fitting module 440 then provides a sparse sequence of tool path command points P[i] to the tool path motion interface module 450, which forwards the sparse command points to the top-level program execution module 340 on the robot controller 110 as described above. The robot controller 110 then uses its inherent capabilities of joint kinematics calculation and interpolation point calculation to calculate smooth and efficient robot motion to perform the task. Knowledge of the robot controller's spline function fitting and interpolation point calculation algorithms can help ensure that the spline calculations in the path function fitting module 440 produce desirable results when the controller 110 converts the command points P[i] into robot motion commands.

[0056] Various computers and controllers have been described and alluded to throughout the preceding discussion. It will be understood that the software applications and modules of these computers and controllers execute on one or more computing devices having a processor and memory modules. This includes, among other things, the processors within the robot controller 110 and computer 120 of FIGS. 1 and 3 described above. In particular, the processor of computer 120 is configured to perform the calculations of the motion planning module 320, including path planning and selection using a combination of sampling-based and optimization-based methods, and path function fitting. The processor of robot controller 110 is configured to perform the program execution, motion management and planning, interpolation / fitting, and servo motor control functions previously discussed.

[0057] As outlined above, the disclosed technique for robot motion planning using a combination of optimization and sampling methods and path function fitting improves the speed and quality of robot path planning. The combination of optimization and sampling path planning methods provides the best quality path within the allotted cycle time, and path function fitting allows the robot controller to utilize inherent performance in calculating robot joint movements and interpolation points, enabling smooth and fast execution of the robot's tool movements.

[0058] While several example aspects and embodiments of robotic motion planning techniques have been discussed above, those skilled in the art will recognize modifications, permutations, additions, and sub-combinations, and it is therefore intended that the following appended and hereafter introduced claims be interpreted as including all such modifications, permutations, additions, and sub-combinations as are within their true spirit and scope.

Claims

1. providing input information for planning the path of the tool on the robot, including data on start and finish points of the tool's path and obstacles to avoid; calculating and selecting a planned path comprising a set of waypoints based on the input information by a computer having a processor and a memory, the planned path including calculating and selecting the planned path using a parallel combination of a sampling-based motion planning method and an optimization-based motion planning method; calculating, by a robot controller in communication with the computer, robot joint movements such that the tool on the robot follows the planned path; Including, In calculating and selecting the planned path, the sampling-based motion planning method and the optimization-based motion planning method each calculate a proposed path based on the input information, and when proposed paths are provided by both the sampling-based motion planning method and the optimization-based motion planning method, the planned path is selected as the proposed path with higher quality, where the quality is determined as at least a path length and a path smoothness. A motion planning method for industrial robots.

2. 2. The method of claim 1, wherein providing input information includes providing camera images or sensor data depicting a workspace, and determining the start and finish points from the camera images or sensor data, the start and finish points each including a position and orientation in three dimensions (3D).

3. The method of claim 2 , wherein the data defining the obstacle is determined from either the camera image or sensor data, or is provided as 3D model data from another source.

4. 2. The method of claim 1, wherein the sampling-based motion planning method determines the waypoints on the planned route by building a random tree structure of proposed waypoints and ultimately selecting waypoints that connect the start point and the goal point without collision.

5. 2. The method of claim 1, wherein the optimization-based motion planning method determines the waypoints on the planned path by iteratively calculating the waypoints from the start point to the goal point using an objective function for path quality and a constraint function for collision avoidance.

6. 2. The method of claim 1, wherein only one of the sampling-based method or the optimization-based method provides a suggested route when a latency threshold is reached, and the suggested route provided is designated as the planned route unless the quality of the suggested route falls below a predefined quality index value.

7. calculating, by the computer, a reduced set of command points representing the planned route, wherein the number of waypoints in the planned route is greater than the number of command points; transferring the set of command points to the robot controller and calculating joint movements of the robot based on the command points, wherein calculating the reduced set of command points includes calculating a spline function to fit the waypoints of the planned path, wherein an arc length parameter having a value ranging from 0 to 1 defines a position on the spline function; The method of claim 1 , further comprising: evaluating the spline curve function at each of the command points based on the arc length parameter and the number of command points.

8. 8. The method of claim 7, wherein calculating the joint movements of the robot includes selecting a type of inverse kinematics calculation, performing the inverse kinematics calculation, and calculating interpolation points between the command points.

9. The method of claim 8 , further comprising: defining a robot tool movement command associated with the interpolation point; and providing the joint movement command and the robot tool movement command to the robot.

10. calculating and selecting a planned path using a parallel combination of a sampling-based motion planning method and an optimization-based motion planning method, wherein a computer having a processor and a memory calculates and selects the planned path including a set of waypoints based on input information including data defining a start point and a finish point of the planned path and obstacles to be avoided; calculating, by the computer, a reduced set of command points representing the planned route; transferring the reduced set of command points to a robot controller; calculating, by the robot controller, robot joint movements based on the command points so that a tool on the robot follows the planned path; Including, In calculating and selecting the planned path, the sampling-based motion planning method and the optimization-based motion planning method each calculate a proposed path based on the input information, and when proposed paths are provided by both the sampling-based motion planning method and the optimization-based motion planning method, the planned path is selected as the proposed path with higher quality, where the quality is determined as at least a path length and a path smoothness. A motion planning method for industrial robots.

11. means for providing input information describing a workspace including start and finish points of the tool's path and data on obstacles to be avoided; a computer in communication with the means for providing input information, the computer having a processor and memory configured to calculate and select a planned route comprising a set of waypoints based on the input information, the planned route comprising using a combination of a sampling-based motion planning method and an optimization-based motion planning method, and to calculate a reduced set of command points representing the planned route, the number of waypoints in the planned route being greater than the number of command points; a robot controller in communication with the computer and the robot configured to calculate robot joint movements such that the tool on the robot follows the planned path based on the reduced set of command points received from the computer; Equipped with calculating and selecting the planned path uses a parallel combination of the sampling-based and optimization-based motion planning methods, wherein the sampling-based motion planning method and the optimization-based motion planning method each calculate a proposed path based on the input information, and when proposed paths are provided by both the sampling-based motion planning method and the optimization-based motion planning method, one of the proposed paths having a higher quality is selected as the planned path, the quality being determined based on at least a path length and a path smoothness. A path planning system for industrial robots.

12. 12. The system of claim 11, wherein when a latency threshold is reached and only one of the sampling-based method or the optimization-based method provides a suggested route, the suggested route is designated as the planned route unless the quality of the suggested route falls below a predefined quality index value.

Images