HUMAN SKILLS-BASED RAILWAY GENERATING

Abstract

A method for robot path planning using skills extracted from human-taught motion programs and applied to a new obstacle environment. A three-dimensional convolutional neural network is used to extract features describing an obstacle environment, with the feature vector representation of the obstacles overcoming problems encountered when using point cloud obstacle data. The obstacle feature data and the robot path start and end points are provided to a neural encoder-decoder network system trained to extract skills from a database of human-created motion programs. The neural encoder-decoder network system generates a distribution of waypoints for the current obstacle environment and the current start / end points.The distribution of waypoints is used to generate a final collision-free path using either a Rapidly Exploring Random Tree (RRT) method or an optimization-based method.

Description

STATE OF THE ARTTechnical field

[0001] The present disclosure relates generally to the field of motion programming for industrial robots, and more particularly to a method for generating a trajectory for a robot that uses a convolutional neural network to extract features describing an obstacle environment and an encoder-decoder neural network system to extract capabilities from a database of human-created motion programs and to generate a distribution of waypoints for a current obstacle environment and generates a final collision-free trajectory based on the distribution of waypoints. Discussion of related technology

[0002] The use of industrial robots to repeatedly perform a wide range of manufacturing, assembly, and material handling tasks is well known. There are a variety of methods for teaching a robot to move from a starting point to a destination. However, when obstacles exist between the starting point and the destination, existing trajectory generation techniques all suffer from certain shortcomings.

[0003] A common method for path generation is the use of a teaching device. The teaching device communicates with the robot controller and is operated by a human operator. The user uses the teaching device to instruct the robot to perform incremental movements—such as "step in the x-direction" or "rotate the gripper around the local z-axis." The robot movements are recorded by the robot controller and stored as a motion program. Such teaching device commands and robot movements are well suited for simple paths with straight-line movements and few or no obstacles. However, using a teaching device to program a robot in a complex obstacle environment often proves difficult, error-prone, and time-consuming.

[0004] Another well-known method for trajectory generation involves using a collaborative robot in a "lead-through" approach. In lead-through, a human operator grasps the tool or workpiece at the end of the robot arm and moves the tool or workpiece from the start point to the destination point. Lead-through has the advantage of leveraging human expertise in selecting a trajectory and also allows for collision avoidance evaluation for all parts of the robot during movement, which is critical in applications where any part of the robot (not just the tool or workpiece) could come into contact with an obstacle. Unfortunately, when the obstacle environment is complex, it may be difficult or impossible for the human operator to manipulate the entire robot (including all intermediate joint positions) to avoid collisions.

[0005] Teaching a robot through human demonstration is also known, in which a human demonstrator grasps a workpiece by hand and moves it from the start position to the target position. However, trajectory generation through human demonstration may lack the positioning accuracy required for precise workpiece movement, and trajectory generation through human demonstration does not take into account the avoidance of collision of the robot arm itself with obstacles in the workspace.

[0006] Automatic trajectory generation techniques are also known, in which the start and destination points, along with the geometric definition of the obstacles in the environment, are provided and a trajectory generation calculation is attempted. These automatic trajectory generation techniques also have certain shortcomings, which are discussed below.

[0007] In view of the circumstances described above, there is a need for an improved method for generating a robot trajectory that captures the core content of the skills imparted by human-generated trajectories and applies these skills to generate collision-free trajectories in a novel obstacle environment. SUMMARY OF THE INVENTION

[0008] The present disclosure describes a method for robot trajectory planning using skills extracted from human-taught motion programs and applied to a new obstacle environment. A three-dimensional convolutional neural network is used to extract features describing an obstacle environment, where the feature vector representation of the obstacles overcomes problems associated with using point cloud obstacle data. The obstacle feature data and the robot trajectory start and finish points are provided to an encoder-decoder neural network system trained to extract skills from a database of human-created motion programs. The encoder-decoder neural network generates a distribution of waypoints for the current obstacle environment and the current start / finish points.The distribution of waypoints is used to generate a final collision-free trajectory using either a Rapidly Exploring Random Tree (RRT) method or an optimization-based method.

[0009] Further features of the devices and methods disclosed herein will become apparent from the following description and the appended claims taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1A is an illustration of automatic trajectory generation using a RRT (Rapidly-Exploring Random Tree) method and Fig. 1B is an illustration of automatic path generation using an optimization-based method as known in the art; Fig. 2 is a block diagram representation of a system for human skill-based robot trajectory generation according to an embodiment of the present disclosure; Fig. Figure 3 is a diagram showing how the mathematical representation of a point cloud changes when the order of numbering of the points in the cloud changes; Fig. 4 is a block diagram representation illustrating the extraction of features from obstacle point cloud data and the use of the feature vector data as input for human skill-based trajectory generation according to an embodiment of the present disclosure; Fig. 5 is a block diagram representation showing how encoder and decoder neural networks are used to generate a distribution of waypoints for a robot trajectory with human skill extraction using start and finish points and obstacle feature data as input along with a database of human-generated trajectories for training, according to one embodiment of the present disclosure; Fig. 6 is a block diagram representation showing how, according to embodiments of the present disclosure, the distribution of waypoints of Fig. 5 is used to generate a collision-free trajectory using an RRT or an optimization-based method; Fig. 7 is a schematic diagram of a system for human-skill-based robot trajectory generation showing robot and obstacle geometries from an exemplary implementation according to an embodiment of the present disclosure; and Fig. 8 is a flowchart of a method for human skill-based robot trajectory generation including obstacle feature extraction and human skill extraction in a neural network system and generating a collision-free trajectory from a distribution of waypoints, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0010] The following discussion of embodiments of the disclosure relating to a method for human-skill-based robot trajectory generation is merely exemplary in nature and is not intended to limit the disclosed devices and techniques or their applications and uses in any way.

[0011] The use of industrial robots for a variety of manufacturing, assembly, and material handling activities is well known. It has long been a goal to develop simple methods for creating robot motion programs that are efficient and avoid collisions with any obstacles present in the work environment. However, existing path generation techniques all suffer from certain shortcomings.

[0012] It is widely recognized that human intuition and visualization are powerful tools that can be used in robot trajectory generation. Therefore, various techniques for trajectory generation using human input have been developed. These techniques—including manipulation with a teaching device, lead-through of a collaborative robot, and human demonstration of the actions of picking and placing a workpiece—can be very effective in trajectory generation for certain types of tasks. However, in an environment with complex obstacles—for example, when a robot is mounted on one side of a workpiece and needs to reach through an opening in the workpiece to perform an action on the other side—the capabilities of these existing trajectory generation techniques are often insufficient to generate an efficient and collision-free trajectory.

[0013] Automatic trajectory generation techniques are also known, in which the start and destination points are provided along with the geometric definition of the obstacles in the environment, and an automatic calculation of a collision-free trajectory is attempted. Two such techniques, along with their limitations, are discussed below.

[0014] Fig. 1A is an illustration of automatic trajectory generation using a RRT (Rapidly-Exploring Random Tree) method and Fig. Figure 1B is an illustration of automatic path generation using an optimization-based method, as known in the art. Fig. 1A illustrates an RRT trajectory generation scenario 100. In scenario 100, a collision-free trajectory from a starting point 110 to a destination point 112 is desired. Numerous obstacles 120 exist in the workspace between the starting point 110 and the destination point 112. A path 130 generated with RRT is shown halfway through its growth process.

[0015] As is known to those skilled in the art, the RRT method proposes a new waypoint within an incremental distance of a previous path point (or the starting point 110) and evaluates the feasibility of a path segment from the previous path point to the new waypoint. If the path segment is collision-free, then the new waypoint is added to the path, and another new waypoint is evaluated. Many branches develop in the RRT path, and eventually, a complete path from the starting point 110 to the destination point 112 can be found. However, paths generated with RRT typically have an unnatural shape, comprising many short path segments that zigzag back and forth. For this and other reasons, paths generated with a pure RRT method are often perceived as less desirable than paths generated by other methods.

[0016] In Fig. 1B, a scenario 140 includes the same starting point 110 and the same destination point 112, as well as the same obstacle environment 120, as in scenario 100 discussed above. In scenario 140, an optimization-based method attempts to find a collision-free path from the starting point 110 to the destination point 112. A path 150 runs from the starting point 110 partially through the obstacle environment of scenario 140. Optimization-based path generation methods can generate paths using an objective function that supports smoothness and efficiency in path guidance and constraints that ensure collision avoidance throughout the path. However, in a complex obstacle environment, optimization-based methods may not be able to find a feasible solution.This situation is illustrated in scenario 140, in which the path 150 runs to a point 152 where it stops behind two of the obstacles 120 and is obstructed by an outer boundary of the work area.

[0017] The techniques of the present disclosure were developed to overcome the limitations and inadequacies of the existing trajectory generation methods discussed above. The techniques discussed below leverage the capabilities embodied in human-generated trajectories while using advanced computation to apply human trajectory generation capabilities to find a collision-free trajectory through a new obstacle environment.

[0018] Fig. Figure 2 is a block diagram representation of a system 200 for human-based robot trajectory generation according to an embodiment of the present disclosure. At block 210, obstacle data for a workspace environment is provided, including start and end point positions for a new trajectory to be calculated. These inputs correspond to scenarios 100 and 140 described in Figures 10-14. Fig. 1A / 1B and have already been discussed. Instead of directly calculating a trajectory, the inputs from block 210 are provided to an encoder / decoder block 220. The encoder / decoder block 220 includes encoder and decoder neural networks configured to extract human capabilities from existing robot motion programs and use those capabilities to calculate a trajectory in a new obstacle environment. The encoder / decoder block 220 is discussed in detail below.

[0019] A database 230 of existing motion programs is also provided to the encoder / decoder block 220. The database 230 contains robot motion programs created using any technique that incorporates human skills. These techniques include, for example, the use of a teaching device to define a robot motion program as a sequence of incremental movements, the use of a collaborative robot by a human in a lead-through motion capture method, and the human demonstration of a workpiece pick-and-place operation. The database 230 contains a plurality of human-taught motion programs (robot trajectories) and a definition of the obstacle environment corresponding to the robot trajectory for each motion program.In this way, database 230 describes the manner in which a human expert solves a motion programming problem for various obstacle environments. That is, database 230 captures human skills, such as those related to path curvature and obstacle avoidance distances, when plotting a path from a starting point to a destination point. Database 230 is used to train encoder / decoder block 220. The human skills contained in database 230 are extracted by encoder / decoder block 220 and ultimately used to generate a path for a new obstacle environment, as discussed below.

[0020] The encoder / decoder block 220 generates a waypoint distribution 240 containing a distribution (a type of cloud path) of waypoints in a workspace environment containing the obstacles defined in the input block 210. The waypoint distribution 240 is then used in one of two ways to generate a collision-free trajectory 250. The system 200 captures the trajectory generation skills of a human expert, applies those skills to draw a trajectory through a new obstacle environment, and leverages the power of automated trajectory generation techniques in a manner that overcomes the limitations of existing trajectory generation methods. Each of the elements of the system 200 and their interactions with each other are discussed in more detail below.

[0021] Fig. Figure 3 is an illustration showing how the mathematical representation of a point cloud changes when the numbering order of the points in the cloud changes. Many existing trajectory generation techniques use a point cloud representation of the obstacle environment. As is known to those skilled in the art, a point cloud is a set of points representing one or more objects, where the points are typically defined in a three-dimensional (3D) coordinate system. The points in the point cloud can be located only on external surfaces of the object(s), or the points can be located on both the outside and inside of the object(s).

[0022] Although point clouds are a convenient form of 3D representation of obstacles, they have some properties that pose problems in the subsequent calculations in which they are used. For example, consider a simple point cloud 310 containing five points in a spatial environment, where the points are numbered 1 through 5. A point cloud 320 includes five points in the spatial environment whose positions are identical to those of the points in point cloud 310, but the points in point cloud 320 have been numbered 1 through 5 in a different order. Point cloud 310 corresponds to a coordinate matrix 330, where point 1 of coordinate matrix 330 has the coordinates (x1, y1, z1), and so on. The point cloud 320 corresponds to a coordinate matrix 340, where point 1 of the coordinate matrix 340 has the coordinates (x1, y1, z1) which differ from the coordinates (x1, y1, z1) of the coordinate matrix 330.In other words, although point cloud 310 and point cloud 320 define five identically positioned points in space, coordinate matrix 330 is completely different from coordinate matrix 340. Therefore, the point clouds do not exhibit permutation invariance; that is, the definition of the point cloud varies depending on the order in which the elements are listed.

[0023] Another property of point clouds is that they lack transformation invariance. This means that a point cloud representing an object oriented one way (e.g., "right side up") is completely different from a point cloud representing the same object with a different orientation (e.g., "upside down"). Both permutation variance and transformation variance lead to problems and inefficiencies when using point clouds to define obstacles, because these two phenomena make it difficult or impossible for objects to be detected and processed efficiently, especially by machine learning systems.

[0024] Fig. Figure 4 is a block diagram illustrating the extraction of features from obstacle point cloud data and the use of the feature vector data as input for human-based trajectory generation according to one embodiment of the present disclosure. Fig. 2, block 210 was shown, which provides obstacle data input including a definition of the start and finish point positions. Fig. 4 illustrates how the obstacle data is provided to the system by Fig. 2 be provided as input in a way that overcomes the problems with point cloud data discussed above.

[0025] A point cloud 410 represents an object (in this case, a chair, which can be considered an obstacle) in a known manner and as discussed above. A 3D convolutional neural network (CNN) 420 is used to extract features from the obstacle point cloud 410. Feature extractors—such as the CNN 420—take a point cloud or other data (such as a solid model or an image) as input and provide as output a set of feature vectors characterizing the input. The feature extractor CNN 420 drastically reduces the amount of data required for subsequent processing by replacing a point cloud containing thousands of point coordinates with a set of numerically defined feature vectors, which may number in the several hundred.The feature extractor CNN 420 also overcomes the problems discussed above associated with defining an obstacle from a point cloud by using a feature vector representation of the obstacle instead of the raw point cloud data.

[0026] As will be apparent to one skilled in the art, a CNN is a common architecture used for feature extraction, and indeed, pre-trained feature extractor CNNs are available in which the network structure, including the number of layers and the size of each layer output, is fixed, and the network parameters are trained. The feature extractor CNN 420 may be of this type of pre-trained feature extractor CNN. The final output of the CNN 420, which is used as input to the encoder / decoder block 220 (of Fig. 2) is a set of feature vectors, shown in block 430. The output shown in block 430 is an array of feature vectors, where the array has a size of "X" by "Y" and each feature vector has a dimension of {NNx1}. The number of feature vectors (X·Y) and the dimension of each feature vector can be chosen in a manner appropriate for a particular application and according to the design of the CNN 420.

[0027] The feature vectors from block 430, which describe the obstacles from the point cloud 410, are added to the Fig. 2. The start and end point positions for the robot path to be generated in a block 440 are also provided as input to the encoder / decoder block 220. The encoder / decoder block 220, after being trained with the database 230 of existing motion programs, uses the obstacle and start / end inputs from blocks 430 and 440 to calculate the waypoint distribution 240, which has already been calculated with respect to Fig. 2. The training and calculation of the waypoint distribution are discussed below.

[0028] Fig. Figure 5 is a block diagram illustrating how encoder and decoder neural networks are used to generate a distribution of waypoints for a robot trajectory with human skill extraction according to one embodiment of the present disclosure. The encoder / decoder block 220 is shown in Fig. 5 and receives the obstacle feature vectors 430 and the start and finish points as input in block 440. The database 230 of human-generated motion programs, including corresponding obstacle data, is also provided as input. The obstacle and motion data from the database of human-generated trajectories is used to train encoder and decoder neural networks that capture human capabilities, where the encoder defines a Gaussian distribution of probabilities associated with a set of state data (start and finish point positions, obstacle) and action data (motion), and the decoder determines actions (waypoints) associated with a set of state data (start and finish point positions, obstacle) and the corresponding probability.

[0029] The encoder / decoder block 220 includes an encoder neural network 510. The data from the human-generated movement program database 230 provides a sequence of corresponding "state" and "action" data—(s0, a1), (s1, a2), (s2, a3), and so on—that is used to train the encoder neural network 510. Each of the "states" s i is an obstacle property from one of the movement programs in the database 230 including start and finish point positions, and the corresponding “action” a i+1is the motion property (waypoint from the motion program) that resulted from the state. The encoder neural network 510 generates a distribution q (shown at 520) of probabilities z associated with a state s and an action a. The distribution q(z|s, a), shown at 520, captures the human capabilities from the human-generated motion programs in the database 230. The human-generated motion program database 230, which includes both obstacle environment data and corresponding human-generated motion programs, serves as labeled data for training the encoder neural network 510 and a decoder neural network 530 to exhibit the desired trajectory generation behavior.

[0030] After training, the decoder neural network 530 is then used to determine an action a corresponding to a state s and a probability z. This is done using a probability function π(a|s, z), as shown on line 540. The decoder neural network 530 receives as input the feature vectors 430 that define the obstacle environment for the trajectory to be generated, along with the start and finish point positions of the trajectory to be generated, from block 440. The decoder neural network 530 then calculates a set of actions a (waypoints) corresponding to states s (start and finish point positions and obstacles) and probabilities z using the probability function π.This results in the waypoint distribution 240, which is not a complete and definitive trajectory (motion program), but rather a distribution of waypoints based on the obstacle environment of the generated trajectory and the probabilities encoded in the human capabilities by the database 230.

[0031] Training of the encoder / decoder block 220 is accomplished using the database 230 of human-created motion programs, which contains the obstacle environment for each of the motion programs. Using the database 230 as a labeled dataset for training, the encoder neural network 510 learns the distribution q that captures human capabilities, while the decoder learns the probability function π that generates a waypoint distribution along the motion program used as input. Training of the encoder neural network 220 can be accomplished using a known loss function approach or any other technique determined to be most appropriate.

[0032] Summarizing the above, the human-generated motion program database 230 (containing the obstacle environment for each of the motion programs) is used to train the encoder neural network 510 and the decoder neural network 530 to capture human trajectory generation capabilities applied to many different obstacle scenarios. After training, the decoder neural network 530 then receives start / finish positions and the obstacle environment (obstacle feature vector data) for a new trajectory to be generated and, using the distribution q, calculates the probability function π that generates the waypoint distribution 240. The waypoint distribution 240 is a set of points along a trajectory traveling from the start point to the finish point, generally avoiding obstacles along the path.

[0033] In the experimental evaluation of the methods disclosed herein, the training dataset (in database 230) comprised 300 motion programs (each with its corresponding obstacle environment). This training dataset demonstrated the ability to adequately train encoder / decoder block 220. After training with the dataset of 300 motion programs, a test dataset comprising 50 obstacle environments was provided for evaluation, with a waypoint distribution being calculated for each of the 50 examples in the test dataset. More than half of the test examples resulted in a waypoint distribution that was collision-free throughout the entire obstacle environment of the respective example. However, the remaining test examples resulted in a waypoint distribution that included some waypoint overlays with obstacles.Thus, although the waypoint distribution 240 does a very good job of capturing the general movements representative of human capabilities in the training database, an additional procedural step is required to generate a reliably collision-free trajectory for each start / finish condition and obstacle environment.

[0034] Fig. 6 is a block diagram representation showing how, according to embodiments of the present disclosure, the distribution of waypoints of Fig. 5 is used to generate a collision-free trajectory using an RRT or an optimization-based method. As explained above, there is no guarantee that the waypoint distribution 240 along the trajectory from the starting point to the destination point is collision-free. Therefore, the technique disclosed here adds a method step of generating a collision-free trajectory based on the waypoint distribution 240.

[0035] A first technique for generating a collision-free path based on the waypoint distribution 240 uses the RRT method. This is shown on the upper path (①) in Fig. 6. In block 610, the waypoint distribution 240 is shown together with the beginning of an RRT path shown at 612. Unlike the one described with reference to Fig. 1, in the trajectory generation technique of the present disclosure, proposed new waypoints are not randomly selected from anywhere in the workspace. Instead, in the trajectory generation technique disclosed, only proposed new waypoints are selected from the waypoint distribution 240. This ensures that the path generated with RRT follows the trajectory specified by the encoder / decoder block 220, which captures human trajectory generation skills. This also makes trajectory generation with RRT very efficient because most proposed new waypoints will be collision-free, and therefore, an RRT path is quickly constructed from the starting point to the destination point following the waypoint distribution based on human skills while avoiding all obstacles along the way (because each RRT path segment is checked for collision avoidance).The resulting collision-free path 250 - after removing all unproductive branches - is shown on the right side of . Fig. 6 of the waypoint distribution 240 is shown superimposed.

[0036] A second technique for generating a collision-free path based on the waypoint distribution 240 uses an optimization calculation. This is shown on the lower path (②) in Fig. 6. In block 620, the waypoint distribution 240 is used in the optimization calculation as follows. The optimization model is defined using an objective function that penalizes deviation from the waypoint distribution, along with an inequality constraint that ensures that the trajectory remains collision-free. A preferred, but non-limiting, embodiment of the objective function is defined below in equation (1): min[λ1(wprobot−wphuman)+λ2∑di+λ3∑ti]

[0037] Where: wp robotis a waypoint of the robot on the calculated path, wp human is the waypoint distribution 240 provided by the encoder / decoder block 220, which incorporates the human trajectory generation capabilities, Σd i is the total distance of the generated path (a summation of the distances for each path point i), Σt i is the total cycle time of the robot movement using the generated trajectory, and λ1-λ3 are weighting factors. The first term of the objective function in equation (1) serves to minimize the deviation of the calculated robot trajectory from the waypoint distribution 240, preferably using the center or average position of the waypoint distribution in this calculation. The weighting factors λ1-λ3 can be chosen to achieve the desired trajectory generation performance for any given application.

[0038] In addition to the objective function of Equation (1), the optimization model includes at least one constraint, such as an inequality constraint, which calculates a minimum distance between the robot and any obstacle for each iteration of the waypoint calculation. The constraint stipulates that the minimum distance must be greater than a predefined threshold (the threshold can be zero or any positive distance value, such as 15 mm). This requires that each robot trajectory point i be collision-free before being added to the trajectory. In addition to the inequality constraint for collision avoidance, other constraints can be included in the optimization model, such as constraints on robot joint positions, which ensure that the robot configuration is feasible at each trajectory point i.

[0039] The optimization model defined above ensures that the generated trajectory follows the waypoint distribution specified by the encoder / decoder block 220, which captures human trajectory generation capabilities. The optimization model also ensures that the generated trajectory is highly efficient while avoiding all obstacles along the way. The collision-free trajectory resulting from the optimization-based trajectory generation is also Fig. 6, superimposed on the waypoint distribution 240.

[0040] Fig. 7 is a schematic diagram of a human-skill-based robot trajectory generation system 700 showing robot and obstacle geometries from an exemplary implementation according to an embodiment of the present disclosure. Fig. 7 is provided as a more realistic illustration of how the procedures and system described above (with reference to the Fig. 2 and 4 to 6) can be applied to an actual robot in a complex 3D obstacle environment, and not just a matter of drawing a curve through a simple group of two-dimensional obstacles.

[0041] In a first step (shown at ①), a database 710 with human-created robot motion programs is used to train a neural encoder-decoder network system 720. This step has been described in detail with reference to Fig. 5, wherein the database 710 (corresponding to the database 230) contains many (e.g., hundreds) motion programs created using a human teaching method, along with the corresponding obstacle data for each motion program, and wherein the data is used to train the encoder-decoder system 720 (the distribution q(z|s, a) and the probability function π(a|s, z) to emulate human capabilities in defining trajectory points that avoid obstacles in a spatial environment).

[0042] In a second step (shown at 2), the start and destination positions for a new path to be generated, along with the obstacle environment through which the path must be found, are provided to the trained neural encoder-decoder network system 720. Obstacle geometry data is provided to a 3D CNN 730, which extracts obstacle feature data; the encoder-decoder system 720 is provided with obstacle feature data instead of raw obstacle geometry data or point cloud data. This step was described in detail with reference to Fig. 4. Box 702 contains an illustration of a robot and a door frame. This represents an example of the type of actual robot trajectory generation scenario to which the methods and system disclosed herein are applied. In this example, the start point has a tool center point located on the "near" side of the door frame (on the same side on which the robot base is mounted), while the destination point has the tool center point on the "far" side of the door frame. This would be representative of a trajectory type that needs to be generated for, for example, a spot welding robot, where welding operations are required on both sides of the workpiece (door frame). In addition, the door frame itself is an obstacle that the robot must avoid. There may also be other obstacles in the workspace environment.

[0043] In a third step (shown at 3), the start and destination positions and the obstacle feature data are used by the neural decoder network to generate a waypoint distribution 740. This step has also already been described with reference to Fig. 5. The waypoint distribution 740 includes a point distribution calculated by the probability function π based on the start / finish positions and the obstacle environment. In the illustrated example, the waypoint distribution 740 follows a trajectory through an opening or hole in the door frame. The waypoint distribution 740 embodies the human ability to generate a trajectory that avoids collisions, which are detected in the encoder-decoder system 720. The waypoint distribution 740 is not guaranteed to be collision-free, which is why a final trajectory generation step is included.

[0044] In a fourth step (shown at 4), a collision-free path 750 is generated based on the waypoint distribution 740. This step has already been described with reference to Fig. 6. The collision-free trajectory 750 can be calculated from the waypoint distribution 740 using either the RRT method or the optimization-based method, as previously discussed. In both the RRT method and the optimization-based method, the waypoint distribution 740 is used to select the points along the final collision-free trajectory.

[0045] Referring to the Fig. A few important points should be noted from the illustrations provided in Figure 7. First, the obstacles involved in programming robot motion are generally 3D objects that have complex shapes (such as a car door frame) and can also include fixtures and platforms with simpler prismatic shapes. Furthermore, each waypoint, or robot trajectory point, is not simply a point in 2D or 3D space; rather, it is defined by a complete robot configuration that includes the positions of all the robot arm's joints. Therefore, collision avoidance calculations (either in the RRT method or the optimization-based trajectory generation method) calculate minimum distances in three dimensions between all parts of the robot arm (including the tool and any workpiece that the tool might grip) and all obstacles.The calculations of the minimum distances may be performed in any suitable manner, such as using a method based on signed distance fields or other suitable methods. Fig. Figure 7 illustrates the complexities actually involved in all data and calculations, whereas previous figures presented simplified examples for the purpose of conceptual explanation.

[0046] Fig.8 is a flowchart 800 of a method for human-skill-based robot trajectory generation, including obstacle feature extraction and human skill extraction in a neural network system and generating a collision-free trajectory from a distribution of waypoints, according to an embodiment of the present disclosure. At box 802, a training database of human-generated motion programs is provided.This is the previously discussed database 230 (or 710), which contains robot motion programs created using any technique that incorporates human skills, including, for example, the use of a teaching device to define a robot motion program as a sequence of incremental movements, the use of a collaborative robot by a human in a lead-through motion capture technique, and the human demonstration of a workpiece pick-and-place operation. The database 230 contains a plurality of human-taught motion programs (robot trajectories) and a definition of the obstacle environment corresponding to the robot trajectory for each motion program.

[0047] At block 804, the training database is used to train an encoder-decoder neural network system. As previously discussed, the encoder-decoder neural network system (block 220) learns to encode or extract human abilities for constructing a movement program in the face of an obstacle environment. Through this training, the encoder-decoder neural network system is taught (through the distribution q(z|s, a) and the probability function π(a|s, z)) to emulate human abilities in defining trajectory points that avoid obstacles in a spatial environment.

[0048] At box 806, start and destination positions and obstacle feature data are provided for a new trajectory to be generated. In a preferred embodiment, a 3D convolutional neural network (3D CNN) is used to extract feature vector data from 3D obstacle geometry data (e.g., point cloud or solid model).

[0049] At box 808, the encoder-decoder neural network system calculates a waypoint distribution for the new trajectory. Specifically, the trained decoder neural network uses the likelihood function π(a|s, z) to calculate the waypoint distribution based on the start and destination positions and obstacle feature data for the new trajectory. Because the decoder neural network is co-trained with the encoder neural network, which encodes human skill in the distribution q, the resulting waypoint distribution emulates human skill in defining trajectory points that avoid obstacles in a spatial environment.

[0050] At box 810, a collision-free trajectory is generated based on the waypoint distribution. During the collision-free trajectory generation, the waypoint distribution is used for guidance, and a complete perturbation check analysis (for the entire robot) is performed at each trajectory point. The collision-free trajectory can be generated using the RRT method at box 812 or the optimization-based method at box 814. The resulting collision-free trajectory embodies the human ability to find a trajectory through an obstacle environment, as taught using the training database. The collision-free trajectory is a complete robot motion program (defining all joint positions at each trajectory point) or can be readily converted into a complete motion program using inverse kinematics calculations, as will be apparent to one skilled in the art of robotics.

[0051] Throughout the above discussion, various computers and controllers are described and assumed. It is understood that the software applications and modules of these computers and controllers execute on one or more computing devices having a processor and a memory module. In particular, this includes the processors in a computer or robot controller, where the controller / computer is configured to perform the computations for human-based trajectory generation in the manner discussed above. The computing device(s) may include special-purpose computing devices configured specifically for executing the 3D CNN and / or the encoder-decoder neural network system.

[0052] As explained above, the disclosed techniques for human-skill-based robot trajectory generation offer several advantages over existing robot trajectory generation methods. The disclosed techniques capture the intuitiveness of human trajectory generation in a trained neural network, utilize a feature vector representation of obstacles to improve consistency, apply human skill in generating a waypoint distribution for a new trajectory, and perform trajectory generation based on the waypoint distribution to ensure that the final trajectory is collision-free.

[0053] While several exemplary aspects and embodiments of human-based trajectory generation have been discussed above, those skilled in the art will recognize modifications, permutations, additions, and subcombinations thereof. Therefore, it is intended that the following appended claims and claims introduced hereafter be construed to encompass all modifications, permutations, additions, and subcombinations consistent with their true spirit and scope.

Claims

[1] A method for human-based robot trajectory generation, the method comprising: Providing a training database containing a plurality of human-created robot motion programs including data defining obstacles to be avoided during the execution of each motion program by a robot; Training an encoder-decoder neural network system comprising an encoder neural network and a corresponding decoder neural network executing on a computing device using the training database; Providing obstacle feature data and start and finish positions for a new trajectory to the encoder-decoder neural network system; Calculating a waypoint distribution for the new trajectory by the encoder-decoder neural network system, wherein the waypoint distribution is a set of waypoint candidates for the new trajectory; and Generating the new trajectory based on the waypoint distribution, the new trajectory defining movements for a robot to move a tool center point from the start position to the target position, comprising performing a robot-obstacle collision avoidance calculation for each trajectory point. [2] The method of claim 1, wherein the training database includes robot motion programs created by one or more methods selected from a group comprising programming with a human-operated teaching device, demonstrating a path by lead-through teaching of a collaborative robot by a human, and demonstrating a robot action by a human grasping and manipulating a tool or workpiece. [3] The method of claim 1, wherein the neural encoder network determines a distribution of probabilities associated with states and actions, the states and actions being represented by the obstacles and corresponding steps in the movement programs in the training database. [4] The method of claim 3, wherein the decoder neural network determines a probability function of actions corresponding to states and probabilities, the actions being defined as the waypoint candidates in the waypoint distribution. [5] The method of claim 4, wherein training the encoder-decoder neural network system comprises using the training database as a labeled training dataset that causes the encoder neural network to learn the distribution and the decoder neural network to learn the likelihood function that captures human trajectory generation capabilities from the human-created motion programs in the training database. [6] The method of claim 1, wherein providing obstacle feature data comprises providing a point cloud or solid model representation of obstacles in a working space of the new trajectory to a three-dimensional convolutional neural network (3D-CNN), extracting feature vectors describing the obstacles using the 3D-CNN, and providing the feature vectors to the encoder-decoder neural network system. [7] The method of claim 1, wherein generating the new trajectory based on the waypoint distribution comprises using a Rapidly-exploring Random Tree (RRT) computation, wherein proposed waypoints evaluated in the RRT computation are selected from the candidate waypoints in the waypoint distribution. [8] The method of claim 1, wherein generating the new trajectory based on the waypoint distribution comprises using an optimization calculation, the optimization calculation minimizing an objective function including a term calculating a distance between the trajectory points on the new trajectory and an average position of the candidate waypoints in the waypoint distribution. [9] The method of claim 8, wherein the objective function further comprises terms that calculate a total distance of the new path and a total time for executing the new path, and the optimization calculation further comprises an inequality constraint that requires a minimum robot-obstacle distance at each path point to be greater than a predefined non-negative threshold. [10] The method of claim 1, wherein a robot configuration is calculated for each path point on the new path using inverse kinematics calculations, and the robot configuration is used in the calculation to avoid collisions between the robot and the obstacle. [11] The method of claim 1, further comprising providing the new path to a controller of the robot and executing the new path with the robot based on motion commands from the controller. [12] A method for human-based robot trajectory generation, the method comprising: Providing a training database containing a plurality of human-created robot motion programs including data defining obstacles to be avoided during the execution of each motion program by a robot; Training an encoder-decoder neural network system comprising an encoder neural network and a corresponding decoder neural network executing on a computing device using the training database, wherein the training causes the encoder neural network to learn a distribution of probabilities associated with states and actions, and the decoder neural network to determine a probability function for actions corresponding to states and probabilities; Providing obstacle feature data and start and destination positions for a new trajectory to the encoder-decoder neural network system, comprising providing a point cloud or solid model representation of obstacles in a working space of the new trajectory to a three-dimensional convolutional neural network (3D-CNN), extracting feature vectors describing the obstacles using the 3D-CNN, and providing the feature vectors to the encoder-decoder neural network system; Calculating a waypoint distribution for the new trajectory by the encoder-decoder neural network system, wherein the waypoint distribution is a set of waypoint candidates for the new trajectory; and Generating the new trajectory based on the waypoint distribution, wherein the new trajectory defines movements for a robot to move a tool center point from the start position to the target position, wherein generating the new trajectory comprises using either a rapidly-exploring random tree (RRT) calculation or an optimization-based calculation and comprises performing a robot-obstacle collision avoidance calculation for each trajectory point. [13] A system for human-based robot trajectory generation, the system comprising: a robot connected to a robot controller; and a computing device having a processor and a memory, the computing device being configured with one or more algorithms to perform steps comprising Accessing a training database containing a plurality of human-created robot movement programs that include data defining obstacles to be avoided during the execution of each movement program by a robot; Training an encoder-decoder neural network system comprising an encoder neural network and a corresponding decoder neural network using the training database; Providing obstacle feature data and start and finish positions for a new trajectory to the encoder-decoder neural network system; Calculating a waypoint distribution for the new trajectory by the encoder-decoder neural network system, wherein the waypoint distribution is a set of waypoint candidates for the new trajectory; and Generating the new path based on the waypoint distribution, the new path defining movements for a robot to move a tool center point from the start position to the target position, comprising performing a calculation for collision avoidance between the robot and the obstacle for each path point, where the new path is provided to the controller and the robot executes the new path based on motion commands from the controller. [14] The system of claim 13, wherein the training database includes robot motion programs created by one or more methods selected from a group comprising programming with a human-operated teaching device, demonstrating a trajectory by lead-through teaching of a collaborative robot by a human, and demonstrating a robot action by a human grasping and manipulating a tool or workpiece. [15] The system of claim 13, wherein the neural encoder network determines a distribution of probabilities associated with states and actions, the states and actions being represented by the obstacles and corresponding steps in the movement programs in the training database. [16] The system of claim 15, wherein the decoder neural network determines a probability function of actions corresponding to states and probabilities, the actions being defined as the waypoint candidates in the waypoint distribution. [17] The system of claim 16, wherein training the encoder-decoder neural network system comprises using the training database as a labeled training dataset, causing the encoder neural network to learn the distribution and the decoder neural network to learn the likelihood function that capture human trajectory generation capabilities from the human-created motion programs in the training database. [18] The system of claim 13, wherein providing obstacle feature data comprises providing a point cloud or solid model representation of obstacles in a working space of the new trajectory to a three-dimensional convolutional neural network (3D-CNN), extracting feature vectors describing the obstacles using the 3D-CNN, and providing the feature vectors to the encoder-decoder neural network system. [19] The system of claim 13, wherein generating the new trajectory based on the waypoint distribution comprises using a Rapidly-Exploring Random Tree (RRT) computation, wherein proposed waypoints evaluated in the RRT computation are selected from the candidate waypoints in the waypoint distribution. [20] The system of claim 13, wherein generating the new trajectory based on the waypoint distribution comprises using an optimization calculation, the optimization calculation minimizing an objective function including a term calculating a distance between the trajectory points on the new trajectory and an average position of the candidate waypoints in the waypoint distribution, the objective function further including terms calculating a total distance of the new trajectory and a total time for executing the new trajectory, and the optimization calculation further including an inequality constraint requiring a minimum robot-obstacle distance at each trajectory point to be greater than a predefined non-negative threshold.

Images