Framework for Online Robot Operation Plan

Abstract

To provide a technique for planning robot operation, that uses an external computer communicating with a robot controller.SOLUTION: A camera or a sensor system provides a computer with input scene information containing start and goal points, and obstacle data. The computer plans an operation of a robot tool on the basis of the start and goal points and an obstacle environment, and robot operation is planned by using in-series or in-parallel combinations of sampling base and optimization base planning algorithms. With the in-series combinations, an implementable path is found out using a sampling method, and an optimization method improves the quality of the path. With the in-parallel combinations, both sampling and optimization are used, and a path is selected according to factors such as calculation time and path quality. The computer converts a congested passing point planned for transfer to a robot controller, to a sparse command point, and the controller calculates robot kinematics and interpolation points in order to control operation of the robot.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure generally relates to the field of operation control of industrial robots, and more specifically, to an operation control technique of a robot that uses a computer separated from a robot controller to plan robot operations based on start and goal points and an obstacle environment. Here, the robot operation plan uses a serial or parallel combination of sampling-based and optimization-based planning algorithms, and is performed by converting the dense waypoints planned for transfer to the robot controller into sparse command points.

Background Art

[0002] As applications of industrial robots, the execution of various manufacturing, assembly, and material handling operations is widely known. In certain robot applications, the start and / or goal points change according to all robot tasks. For example, this applies when the task of the robot is to pick up parts from an incoming conveyor and place the parts in an empty space in a transport container. In such applications, a new robot operation path must be calculated in real time for each task. Further, in the working space environment of many robots, obstacles exist and may be located on the operation path of the robot. The obstacles may be permanent structures such as machines and equipment, or the obstacles may be temporary or mobile. Collisions between the robot and any obstacle must be absolutely avoided.

[0003] Various techniques for planning the tool path operation of a robot based on start and goal points and an obstacle environment are known in the art. One commonly used technique is a sampling-based method in which random sample waypoints are defined and a tree of collision-free waypoints is extended until the start point is connected to the goal point. Another known technique is an optimization-based method in which waypoints along the path from the start point to the goal point are repeatedly corrected using optimization objectives and constraint functions until a non-colliding path is obtained. There are also other techniques including variants of the sampling-based method.

[0004] It is a recent trend in robot motion planning to perform motion planning calculations on a computer separated from the robot controller. There are known systems that execute sampling-based motion planning to find a non-colliding path, then calculate interpolation points to create a robot tool path, and transfer the path to the robot controller for the robot to execute. However, these systems can result in intensive communication bandwidth requirements, fail to take advantage of the robot controller's robot motion planning computing capabilities, and may not be able to provide a path plan of the desired quality.

[0005] In view of the foregoing situation, there is a need for a robot motion planning technique that can provide a high-quality path plan, is fast enough to execute in real time during robot operation, and can take advantage of the computing performance inherent in the robot controller. SUMMARY OF THE INVENTION

[0006] In accordance with the teachings of the present disclosure, a robot motion planning technique using an external computer that communicates with a robot controller is disclosed. A camera or sensor system provides input scene information including start and goal locations, and obstacle data, to the external computer. The computer plans the motion of the robot's tool based on the start and goal locations, and the obstacle environment, where the robot motion is planned using a serial or parallel combination of sampling-based and optimization-based planning algorithms. In the serial combination, first a feasible path is found in the sampling method, and the optimization method improves the quality of the path. In the parallel combination, both the sampling and optimization methods are used, and the path is selected based on calculation time, path quality, and other factors. The computer converts the planned dense waypoints to sparse path points for transfer to the robot controller, and the controller calculates the robot kinematics and interpolation points and uses these to control the motion of the robot.

[0007] Additional features of the disclosed apparatus and method will become apparent from the following description taken in conjunction with the accompanying drawings and the appended claims.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0009] The following discussion regarding the combination of optimization and sampling methods and robot motion planning techniques using path function fitting, related to embodiments of the present disclosure, is merely exemplary in nature and is in no way intended to limit the disclosed apparatus and techniques, or applications or uses.

[0010] It is widely known to use industrial robots for various manufacturing, assembly, and material handling operations. For some of these operations, the robot must be programmed to move along a path that has different start and / or end (goal) points for one operation compared to the next. For example, in pick-and-place operations, parts being loaded onto a conveyor may be picked up by the robot from a different location each time, and parts being placed by the robot into a shipping container will each be placed in a different designated location within the shipping container. Alternatively, parts may be picked up by the robot from a full container and placed onto an outbound conveyor. There are also many other applications where a new path must be calculated for each independent task or operation of the robot.

[0011] Furthermore, in many robot work space environments, obstacles exist, and they can be present on the robot's path of motion. They may be located between the start and goal points, or more generally, between the robot's current position and the robot's target position. The obstacles may be permanent structures such as machinery or equipment, or the obstacles may be temporary or mobile.

[0012] FIG. 1 is a diagram of an industrial robot performing pick, move, and place operations in the above-described type of scenario where the environment of the work space includes one or more obstacles that must be avoided and a new path must be calculated each time an object to be processed is moved. The robot 100 has a gripper 102 and operates within a work space 104. The operation of the robot 100 is typically controlled by a controller 110 that communicates with the robot 100 via a cable 112. As is known in the art, the controller 110 provides joint movement commands to the robot 100 and receives joint position data from the encoders of the joints of the robot 100. The controller 110 also provides commands to control the operation of the gripper 102. The pick, move, and place scenario of FIG. 1 is for illustrative and exemplary purposes only. The optimization motion planning techniques of the present disclosure are applicable to any type of robot operation, and the gripper 102 may be replaced with any type of robot tool.

[0013] A computer 120 is used to calculate the tool path of the robot in the manner discussed below. A camera 122 communicates with the computer 120 and provides an image of the work space 104. The image from the camera 122 may be used to identify the position and orientation of the object to be processed manipulated by the robot 100, and / or the location where the object to be processed is placed by the robot 100, and moving or temporary obstacles within the work space 104. The camera 122 may be replaced with any other type of sensor suitable for the start and goal points (position and orientation) of the object to be processed, or may be used to augment other types of sensors.

[0014] The object to be processed 130 enters the working space 104 on the conveyor 140. The task of the robot 100 is to pick up the object to be processed 130 from the conveyor 140, move the object to be processed 130, and place it inside the container 150 (such as a transport container). The position and direction of the starting point 160 are defined based on the information about the position of the object to be processed 130 on the conveyor 140 and the speed of the conveyor 140. The direction of the object to be processed 130 at the starting point 160 is required to determine the direction of the gripper 102. Similarly, the position and direction of the goal point 162 are defined based on the information about the next available compartment or position in the container 150. In FIG. 1, one intermediate point 164 located between the starting point 160 and the goal point 162 is shown. In the actual calculation of the robot's motion plan, a plurality of intermediate path points are calculated to satisfy all the path constraints, and a smooth tool path passing through the plurality of path points is calculated. Although it is repetitive, this is just an example of various start and goal points, and many other examples can be assumed. For example, the operations may be performed in the reverse order, such as the object to be processed 130 being picked up from a container full of parts and placed on the conveyor 140 in a specific posture.

[0015] There is an obstacle 170 shown as a wall between the starting point 160 and the goal point 162. It should be understood that the side surface of the container 150 represents an obstacle to be avoided by the robot 100 and considered in the path planning. Other obstacles may also exist.

[0016] For each workpiece 130 arriving on conveyor 140, the controller 110 and computer 120 must calculate a new path for the robot 100 to move the gripper 102 from the home or approach position along the path segment 180 to pick up the workpiece 130 at the start point 160, move the workpiece 130 along path 182 towards the goal point 162 while avoiding obstacles 170, and then return the gripper 102 to the home or approach position in preparation for the next workpiece 130. The new path must be calculated very quickly by the controller 110 and computer 120 because the path calculation must be performed in real time, as quickly as the robot 110 moves one workpiece 130 and returns to pick up the next workpiece.

[0017] Figure 2 is a block diagram of a known prior art system for robot motion path planning using a computer 120 that communicates with the robot controller 110. To provide a depiction of the input scene, the sensor processing and task management module 210 receives images from the camera 122 or data from different types of sensors. Briefly, the module 210 determines the start and goal points for future robot tasks by identifying specific parts to be removed from parts or containers entering on the conveyor, as well as the associated target positions and poses (e.g., on the conveyor or within the container). The positions of obstacles in the work space may also be provided by the module 210, particularly if moving obstacles are present or may be present.

[0018] The motion planning module 220 calculates the tool path based on the data from module 210. Various techniques for planning the tool path motion of a robot based on the start and goal points, as well as the obstacle environment, are known in the art. One commonly used technique is a sampling-based method, in which waypoints of random samples are defined and evaluated, and a tree of collision-free waypoints is extended from the start point until the goal point is connected. A particular sampling-based technique is called the rapidly-exploring random tree (RRT) method. In RRT, a tree of nodes that rapidly fills the space randomly is constructed from the initial configuration to the final configuration until a collision-free path is found. Given the data of surfaces / spaces or point clouds representing obstacles, as well as the initial and final workpiece configurations (start and goal points and postures), RRT can be used to find the motion sequence if it exists. Each node of the RRT motion plan must be evaluated and determined to be collision-free in order to be included as a valid or achievable waypoint. Also, since each candidate waypoint is randomly placed, RRT can obtain different solutions each time even if the initial conditions and boundary conditions are the same. Furthermore, the path defined by RRT is essentially tortuous (zigzag), and the "best" candidate waypoint at each step usually deviates from the ideal direction. Therefore, the motion planning technique based on RRT has been recognized as not useful in certain applications.

[0019] The trajectory generation module 230 receives the tool path motion plan from the motion planning module 220 and calculates the overall trajectory. In the conventional technology system of FIG. 2, the trajectory generation module 230 calculates all the trajectory interpolation points and generates a dense point cloud for the robot tool motion. The trajectory interpolation points are transferred from the computer 120 to the robot controller 110 and received by the interpolation module 260. Since the trajectory interpolation points have already been calculated by the trajectory generation module 230 on the computer 120, the interpolation module 260 on the controller 110 only receives the trajectory interpolation points and passes them to the servo-motor control module 270 that drives the joint motors of the robot 100 in a known manner.

[0020] It has been demonstrated that a system of the type shown in FIG. 2 is effective. However, these systems have several disadvantages. The system of FIG. 2 bypasses the software functions of the robot controller 110, not only fails to fully utilize the controller 110 but also imposes an extra computational load on the computer 120. By bypassing the computational functions of the controller 110, the robot operation may become slower and less smooth compared to the case where the controller 110 itself could perform more kinematic and interpolation calculations. Also, data exchange (real-time data streaming of a large number of points during each interpolation period) further increases the load, and packet loss may cause the robot operation to become discontinuous. And the conventional system of FIG. 2 typically relies on a sampling-based motion plan and has the disadvantage of generating unnecessary zigzag and random motion patterns.

[0021] The present disclosure describes systems and methods that overcome some of the disadvantages of prior art systems. These techniques are discussed in detail below.

[0022] FIG. 3 is a block diagram of a robot motion path planning system using an external computer communicating with a robot controller according to an embodiment of the present disclosure. The system of FIG. 3 uses a computer 120 that communicates with the controller 110 as shown in FIG. 1. However, the system of FIG. 3 includes a programming configuration different from that of the system of FIG. 2 in both the computer 120 and the controller 110. Specifically, the computer 120 is configured to plan the best quality path possible within the assigned cycle time, and the controller 110 uses all of its functions for robot kinematics and interpolation point calculations to provide a smooth and reasonable tool motion.

[0023] In FIG. 3, the sensor processing and task management module 210 operates as described above with respect to FIG. 2. In other words, the module 210 receives camera images or sensor data and provides start and goal points along with obstacle data for future robot tasks. The start and goal points and the obstacle data are provided to the motion planning module 320. The motion planning module 320 in FIG. 3 is significantly different from the motion planning module 220 in FIG. 2. Details of the motion planning module 320 will be discussed below with reference to FIGS. 4, 5, and 6.

[0024] The motion planning module 320 provides a sparse sequence of planned path points to the top-level program execution module 340 of the robot controller 110. The program execution module 340 handles high-level functions such as interaction with the operating system, user interface, data interface, and control of other software modules. The program execution module 340 provides the sparse planned path points to the planning module 350. The planning module 350 performs several functions including interpretation of the sparse planned path points, determination of the inverse kinematic calculations to be used (whether to calculate all joints simultaneously, or first calculate the first three joints independently of the last three joints, etc.). The planning module 350 also defines trigger points for commands to grasp or release a tool attached to the robot, such as a gripper, or to turn the torch on / off in a welding operation.

[0025] The planning module 350 provides robot motion and control commands to the same-numbered interpolation module 260 as in FIG. 2. However, in this case, the interpolation module 260 adds interpolation points to define a complete set of operating commands for the servo motor control module 270 that drives the joint motors of the robot 100 as described above.

[0026] By providing a sparse sequence from the computer 120 to the program execution module 340, all the essential performance of the robot controller 110 can be utilized. As a result, calculations of robot kinematics, interpolation points, and motion commands are provided such that the robot 100 moves smoothly along the path calculated by the motion planning module 320 without getting lost.

[0027] FIG. 4 is a block diagram of the motion planning module 320 using a combination of the optimization and sampling motion planning methods in an embodiment of the present disclosure in the system of FIG. 3. As described above, the motion planning module 320 is software executed on the computer 120 and receives start and goal points and obstacle environment data from the sensor module 210.

[0028] The motion planning module 320 includes both an optimization method calculation module 410 and a sampling method calculation module 420. Both the optimization module 410 and the sampling module 420 receive start and goal points and obstacle data, and the modules 410 and 420 may operate independently. The optimization method calculation module 410 starts from an initial reference path that may be a straight line from the start point to the goal point or other initial reference (described later) and executes an optimization routine including an objective function and a constraint function. The optimization routine repeats the calculation until the constraints are satisfied and the objective function is minimized, resulting in a collision-free path with the minimum distance and the highest path quality (smoothness, curvature, etc.). The sampling method calculation module 420 executes a random tree growth planning method until a collision-free path from the start point to the goal point is obtained by the above-described method. The outputs from the optimization module 410 and the sampling module 420 are indicated by the dotted arrows in FIG. 4 and may be used and combined in several different ways as discussed below.

[0029] Although the figure shown in FIG. 4 is two-dimensional, it will be understood that all calculations in the technology disclosed this time are three-dimensional for applicable items and include six degrees of freedom (DOF). The start and goal points include position (three coordinates) and orientation (three angular directions). Obstacles are defined as three-dimensional objects as CAD planes or solid models, or point clouds or surface data from the sensor module 120. The calculated tool path exists in three-dimensional space and includes not only the position of the tool center point but also the direction (6DOF). In addition, collision avoidance calculations are performed not only between the robot tool and the obstacle but also between all parts / arms of the robot and the obstacle.

[0030] The motion planning module 320 includes a path selection module 430 that selects the best path to use from candidate planned paths from the optimization module 410 and / or the sampling module 420 for each cycle of the robot. The path selection module 430 may be configured in one of two different ways, a serial combination or a parallel combination of the optimization and sampling paths. Two configurations of the path selection module 430 (identified as 430A and 430B respectively) are discussed below with reference to FIGS. 5 and 6. By using a combination of optimization and sampling algorithms, the path selection module 430 can utilize the advantages of both methods and avoid the disadvantages of each method in order to obtain the highest possible planned tool path.

[0031] The path selection module 430 provides the planned tool path to the path function fitting module 440. The planned tool path from the path selection module 430 includes a fairly dense sequence of waypoints necessary to compensate for no collisions occurring between the waypoints. For transfer to the robot controller 110, the path function fitting module 440 converts the dense sequence of waypoints into a sparse sequence of command points. The sparse sequence of points provided by the path function fitting module 440 is calculated such that the robot controller can calculate interpolation points to match the original dense sequence of waypoints by the path selection module 430. The path function fitting module 440 (described later) provides the sparse sequence of command points to the tool motion interface module 450, and the tool path motion interface module 450 transfers the sparse command points to the top-level program execution module 340 on the robot controller 110 as described above.

[0032] FIG. 5 is a flowchart of a path selection method 500 using a serial combination of sampling and optimization motion planning in the motion planning module of FIG. 4 in an embodiment of the present disclosure. A block diagram of the motion planning module 320 (of FIG. 4) is shown on the left side of FIG. 5 for reference. The flowchart diagram 500 shows an embodiment (serial combination) of a path selector encoded in the path selection module 430A of the motion planning module 320.

[0033] A cycle of robot operation includes the robot picking up a workpiece at a start point and pausing, moving the workpiece to a goal point, placing it and pausing, and returning to a preparation position to pick up another workpiece. The calculations in the motion planning module 320, and thus in the flowchart diagram 500, are performed for each robot operation cycle.

[0034] At input line 502, data for the current robot cycle (or task) is received from the camera or sensor module 210. The data includes start and goal points, poses in part operations, and obstacle data. Then, the process branches depending on whether the current robot task is expected to have an operation pattern similar to the previous task (only the start point and / or goal point are slightly different). In diamond 510 that makes the determination, it is determined whether a reset of the process has occurred. A reset of the process can be an event such as, for example, the provision of a new container containing the part to be picked up (which becomes a start point quite different from the previous part), or the provision of a new shipping container for the part to be placed (which becomes a goal point quite different from the previous part). Any change in obstacle data, such as when a mobile obstacle that can affect possible robot tool operations enters the work space, is a reset of the process. Even when passing through flowchart 500 for the first time, the answer to the reset of the process is YES.

[0035] If the answer to the reset of the process in diamond 510 that makes the determination is YES, that is, if the current robot task may not have an operation similar to the previous task, the process proceeds to sampling method box 420. In sampling method box 420, a collision-free path from the start point to the goal point is calculated using a sampling method such as RRT. As described above, the sampling method can calculate a path with undesirable characteristics such as a zigzag. Therefore, in the series combination method of FIG. 5, an optimization path planning step that uses the result of the sampling path planning as an input to improve the shape of the planned path is added later.

[0036] The optimization method box 410 receives the initial path calculated by the sampling method box 420 along with the start and goal points and obstacle data, and uses the initial path as the initial reference for the optimization calculation. The optimization method performs iterative optimization calculations including an objective function and a constraint function to obtain a non-colliding path from the start point to the goal point with optimal path quality characteristics. The iterative calculation requires an initial reference path to be used in the first iteration. The initial reference path may be a primitive straight line from the start point to the goal point, but it may require an excessive number of times for the optimization calculation and may not reach the solution. Since the sampling-based initial path avoids collisions and connects the start and goal points, the sampling-based path is a very good initial reference for the optimization method calculation. The optimization method calculation in box 410 removes undesirable movements and zigzag shapes and improves the quality of the path.

[0037] The quality of the path of the optimization method box 410 is evaluated in box 520. The quality may be determined by a combination of the travel distance of the tool path, the curvature of the path, the path execution time, the smoothness of the path, and / or other factors. Since the objective function typically includes these parameters, it provides an essentially smooth and well-shaped path, and the quality of the path should usually be very good as it is the result of the optimization calculation. There is a high possibility of insufficient path quality only when the optimization calculation times out and does not converge and the sampling-based initial path is provided to box 520. If the path quality is insufficient (lacking smoothness, having jerky operating parts) in box 520, a system failure is declared and the parts cannot be lifted by the robot.

[0038] If the path quality is sufficient in box 520, the path from the optimization method box 410 is provided at the output line 530. When the planned path is provided at the output line 530, the planned path is sent to the path function fitting module 440 (described above and discussed further later), and a set of the final sparse command points is provided to the tool path motion interface module 450, and the sparse command points are transferred to the robot controller 110.

[0039] When the planned path is output at line 530 from box 520, the process also loop - backs at 532 and starts the calculation for the next robot cycle. At box 540, the path just output at line 530 is stored in the data repository for later use in generating the initial reference path as described below. Then the process returns to the beginning of the flowchart, to input line 502.

[0040] After the robot executes the picking operation along the just - planned path, new scene data is provided from the camera / sensor module 210 to input line 502. If there is no reset of important processing (new container of input parts, or new output shipping container, or new obstacle), the answer in diamond 510 for judgment is NO. Then the process moves to box 550 for initial reference generation. Initial reference generation includes providing an initial reference path for the optimization operation calculation, where the initial reference path is finely adjusted (scaled and moved) based on the form of past planned paths so that the start and goal points match the path being calculated this time. The initial reference generation technique suitable for use in box 550 is described in U.S. Patent Application No. 16 / 839720, filed on April 3, 2020, titled "Initial Reference Generation in Optimal Motion Planning of Robots", and assigned to the assignee of the present invention, the content of which is incorporated herein by reference.

[0041] Since the path being calculated is very similar to those of past - calculated paths (similar obstacle environment, similar start and goal points), the initial reference generation in box 550 can quickly provide an initial reference path that is a very good approximation of the path currently being calculated. Thus, the optimization method box 410 quickly converges to the optimized path and provides the planned path to box 520. In this way, each planned path is output at line 530, executed by the robot, then there is a loop - back to store the planned path, and the process continues in a form where new input scene data is received to calculate a new path.

[0042] The serial combination of the above-described sampling-based and optimization-based motion plans offers numerous unique advantages in robot motion calculation. Other approaches using a parallel combination of sampling-based and optimization-based will be discussed below.

[0043] Figure 6 is a flowchart of a path selection method using a parallel combination of sampling and optimization motion plans in the motion planning module of FIG. 4 in an embodiment of the present disclosure. A block diagram of the motion planning module 320 (of FIG. 4) is shown to the left of FIG. 6 for reference again. Flowchart 600 shows an embodiment (parallel combination) of a path selector encoded in the path selection module 430B of the motion planning module 320. As before, the calculations in the motion planning module 320, and thus in flowchart 600, are performed for each robot motion cycle or task.

[0044] At input lines 602 and 604, candidate planned paths are received from the optimization module 410 and the sampling module 420. The optimization module 410 and the sampling module 420 independently calculate path candidates based on data (start and goal locations, as well as obstacle data) from the camera or sensor module 210. The optimization module 410 may provide an input at line 602, the sampling module 420 may provide an input at line 604, or vice versa. In diamonds 610 and 620 for making a determination, it is determined whether the candidate planned paths have been received at their respective input lines. If not received, the diamonds for making a determination continue to wait / loop until a path candidate is received. If a path candidate is received at either of diamonds 610 and 620 for making a determination, the path is passed to box 630. The number of path candidates received at box 630 is periodically evaluated. If two path candidates have been received (one each from the optimization module 410 and the sampling module 420), the two path candidates are passed from box 630 to box 650.

[0045] In box 650, the quality of the path candidates is evaluated to select one path to be used. The quality may be determined as a combination of the moving distance of the robot joint path, the curvature of the path, the path execution time, the smoothness of the path, and / or other factors. The path with the highest quality score is selected and output at line 660, provided to the path function fitting module 440, and ultimately transferred to the robot controller 110 to be executed by the robot 100.

[0046] In box 630, if only one path candidate is received (from the optimization module 410 or the sampling module 420), the path candidate is passed from box 630 to the diamond 640 that makes a decision. In the diamond 640 that makes a decision, it is determined whether the waiting time exceeds the threshold. The waiting time threshold is predefined based on the cycle time of the robot. In other words, it may be determined based on the ideal cycle time of the robot, which is the maximum time allowed to wait in the diamond 640 that makes a decision. If the waiting time threshold is not exceeded, the process loops back from the diamond 640 that makes a decision to box 630 to periodically check again how many path candidates have been received. When the second path candidate is received and both paths are provided to box 650, or when the diamond 640 that makes a decision eventually reaches the waiting time threshold and one path candidate is provided to box 650.

[0047] If the diamond that makes a decision reaches the waiting time threshold and only one path candidate is provided to box 650, box 650 does not compare the quality of one path candidate with another. In this case, box 650 evaluates the quality of one path candidate based on the same criteria as the above discussion. If the quality of the path is appropriate, the path is provided to the output line 660. If the quality of the path is not appropriate (e.g., an extreme zigzag shape or an overly long path length), the path is not provided to the output line 660 and a system failure is declared (the part cannot be picked up in this cycle).

[0048] When a planned path is provided at output line 660, the path is sent to a path function fitting module 440 (described above and further discussed later), a set of final sparse command points is provided to a tool path motion interface module 450, and the sparse command points are transferred to a robot controller 110. The processing for the next cycle of the robot starts again from the beginning of flowchart 600, and when ready, sensor data is provided to an optimization module 410 and a sampling module 420 at input lines 602 and 604, and a calculated path candidate is provided.

[0049] Returning to the motion planning module 320 of FIG. 4, two embodiments, a series combination module 430A and a parallel combination module 430B, were discussed above for the path selection module 430. In either case, a path of appropriate quality is provided to the path function fitting module 440 described below.

[0050] FIG. 7 is a block diagram 700 of a path function fitting module 440 used in the motion planning module 320 of FIG. 4 according to an embodiment of the present disclosure. The purpose of the path function fitting module 440 is to provide a set of sparse command points of a planned path for providing to the robot controller 110.

[0051] In the path selection module 430, a path is planned for the current robot task, such as moving a part from a start point to a goal point while avoiding collisions with obstacles. The path planning may be performed using the path selection module 430A (a series combination of sampling and optimization methods) or the path selection module 430B (a parallel combination of sampling and optimization methods) as described in detail above.

[0052] The route selection module 430 provides a dense sequence of waypoints because it is necessary to ensure that no collisions occur throughout the route. This is because if the planned waypoints are sparse and there are gaps between them, there may be collisions between the waypoints that go undetected. Due to the previously discussed reasons including communication bandwidth requirements and the requirements for the robot controller 110 to perform robot kinematics and interpolation point calculations, it is not required to send a dense sequence of waypoints to the robot controller 110.

[0053] The path function fitting module 440 shown in 442 first calculates a spline function s based on the planned path points (q0,..., q T ). The spline function s may be calculated using any suitable technique such as fitting a series of planned path points with piecewise cubic polynomials. The spline function s may be represented as a continuous entity with an arc length parameter a that indicates the distance along the spline s. It has a = 0 at the start point q0 and a = 1 at the goal point q T .

[0054] The next calculation of the path function fitting module 440 shown in 444 is to calculate a sparse set of path points from the spline s. Based on the nature of the robot task and the path, the number N of command points to transfer to the robot controller 110 is selected. In a representative example of a pick, move, and place operation defined by a three-dimensional curve shape, it may be required to send 10 command points to the robot controller 110 (while the planned path may include a much larger number of waypoints such as 30). Thus, in this example, the value of N is 10. Next, the 10 path points P[1], ···, P

[10] are calculated by inputting appropriate values of the arc length parameter a from P[1] (a = 0) to P

[10] (a = 1).

[0055] The path function fitting module 440 then provides a sparse sequence of tool path instruction points P[i] to the tool path motion interface module 450, and the tool path motion interface module 450 transfers the sparse instruction points to the top-level program execution module 340 on the robot controller 110 as described above. The robot controller 110 then uses its essential capabilities such as joint kinematics calculation and interpolation point calculation to calculate a smooth and effective robot motion for task execution. Knowledge of the spline function fitting and interpolation point calculation algorithms of the robot controller can help ensure that the controller 110 obtains desirable results through the spline calculation in the path function fitting module 440 when converting the instruction point P[i] into a robot motion instruction.

[0056] Through the foregoing discussion, various computers and controllers have been described and implied. It will be understood that the software applications and modules of these computers and controllers are executed on one or more computer devices having a processor and a memory module. In particular, this includes the processors within the robot controller 110 and the computer 120 of FIGS. 1 and 3 described above. In particular, the processor of the computer 120 is configured to perform the calculations of the motion planning module 320, which includes planning and selecting a path using a combination of sampling-based and optimization-based methods and performing path function fitting. The processor of the robot controller 110 is configured to execute the program execution, motion management and planning, interpolation / fitting, and servo motor control functions discussed previously.

[0057] As outlined above, the combination of the optimization and sampling methods for the disclosed robot motion plan, and the technique using path function fitting improve the speed and quality of the robot's path planning. The combination of the optimization and sampling path planning methods provides the best quality path within the allotted cycle time, and with path function fitting, the robot controller can use the essential performance for calculating the joint motions and interpolation points of the robot that enable smooth and fast execution of the robot's tool motion.

[0058] Although several example aspects and embodiments of the robot motion planning technique have been discussed above, those skilled in the art will recognize modifications, substitutions, additions, and partial combinations. Accordingly, the appended claims and the claims introduced next are intended to be construed as including all such modifications, substitutions, additions, and partial combinations as being within the true spirit and scope.

Claims

Claim 1 providing input information for planning the path of the tool on the robot, including start and goal points of the path of the tool and data of obstacles to be avoided; calculating and selecting a planned path with a set of waypoints based on the input information by a computer having a processor and a memory, including using a parallel combination of a sampling-based motion planning method and an optimization-based motion planning method to calculate and select the planned path; calculating the motions of the joints of the robot by a robot controller communicating with the computer so that the tool on the robot follows the planned path; comprising in the calculation and selection of the planned path, the sampling-based motion planning method and the optimization-based motion planning method each calculate a proposed path based on the input information, and when proposed paths are provided by both the sampling-based motion planning method and the optimization-based motion planning method, the planned path is selected as the proposed path with higher quality, and the quality is determined as at least path length and smoothness of the path; A method for motion planning of an industrial robot. Claim 2 The providing of the input information includes providing a camera image or sensor data depicting the work space and determining the start point and the goal point from the camera image or sensor data, and the start point and the goal point each include a position and a direction in three dimensions (3D). The method according to claim 1. Claim 3 The data defining the obstacles is determined by either the camera image or the sensor data, or provided as 3D model data from another information source. The method according to claim 2. Claim 4 The sampling-based motion planning method determines the waypoints on the planned path by constructing a random tree structure of the proposed waypoints and finally selecting a waypoint that connects the start point and the goal point without collision. The method according to claim 1. Claim 5 The optimization-based motion planning method determines the waypoints on the planned path by iteratively calculating using an objective function for the quality of the path and a constraint function for collision avoidance from the start point to the goal point. The method according to claim 1. Claim 6 When the waiting time threshold is reached, only one of the sampling-based method or the optimization-based method provides a proposed path, and the proposed path provided is designated as the planned path as long as the quality of the proposed path does not fall below a predefined quality metric value, the method according to claim 1.

7. Calculating, by the computer, a reduced set of instruction points representing the planned path, where the number of waypoints in the planned path is greater than the number of instruction points, Transferring the set of instruction points to the robot controller and calculating joint movements of the robot based on the instruction points, where the calculation of the reduced set of instruction points includes calculating a spline curve function to fit the waypoints of the planned path, where an arc length parameter having a value in the range from 0 to 1 defines a position on the spline curve function, Further comprising evaluating the spline curve function at each of the instruction points based on the arc length parameter and the number of instruction points, the method according to claim 1.

8. Calculating the joint movements of the robot includes selecting a type of inverse kinematics calculation, performing the inverse kinematics calculation, and calculating interpolation points between the instruction points, the method according to claim 7.

9. Further comprising defining robot tool movement instructions related to the interpolation points and providing the joint movement instructions and the robot tool movement instructions to the robot, the method according to claim 8.

10. Including calculating and selecting a planned path using a parallel combination of a sampling-based motion planning method and an optimization-based motion planning method, by a computer having a processor and a memory, calculating and selecting the planned path including a set of waypoints based on input information including data defining a start point and a goal point of the planned path and obstacles to be avoided, Calculating, by the computer, a reduced set of instruction points indicating the planned path, Transferring the reduced set of instruction points to a robot controller, Calculating, by the robot controller, joint movements of the robot based on the instruction points such that a tool on the robot follows the planned path, Including In the calculation and selection of the planned path, the sampling-based motion planning method and the optimization-based motion planning method each calculate a proposed path based on the input information. When proposed paths are provided by both the sampling-based motion planning method and the optimization-based motion planning method, the planned path is selected as the proposed path with higher quality, and the quality is determined as at least the path length and the smoothness of the path. A method for motion planning of an industrial robot.

11. Means for providing input information depicting a work space including start and goal points of a path of a tool and data of obstacles to be avoided, A computer having a processor and a memory configured to calculate and select a planned path including a set of waypoints based on the input information, including using a combination of a sampling-based motion planning method and an optimization-based motion planning method, and the number of waypoints in the planned path is greater than the number of commanded points, and to calculate a reduced set of commanded points representing the planned path, and communicating with the means for providing the input information, A robot controller communicating with the computer and the robot, configured to calculate robot joint motions such that the tool on the robot follows the planned path based on the reduced set of commanded points received from the computer. Comprising Calculating and selecting the planned path uses a parallel combination of the sampling-based and optimization-based motion planning methods. The sampling-based motion planning method and the optimization-based motion planning method each calculate a proposed path based on the input information. When proposed paths are provided by both the sampling-based motion planning method and the optimization-based motion planning method, the one with higher quality among the proposed paths is selected as the planned path, and the quality is determined based on at least the path length and the smoothness of the path. A path planning system for an industrial robot.

12. The system according to claim 11, wherein when only one of the sampling-based method or the optimization-based method provides a proposed path when a waiting time threshold is reached, the proposed path is designated as the planned path as long as the quality of the proposed path does not fall below a predefined quality index value.

Images