Track path planning and management of robotic arm motion

Abstract

The present invention relates to a trajectory generation method for planning a motion in advance and creating a database of trajectory paths simultaneously by implementing a planning manager, a sequence manager and a trajectory path manager, the robot arm is moved to a predetermined end point in a fastest and smoother path by avoiding obstacle and jitter motion within the shortest possible time, and a database is then used to store and retrieve repetitive motion of the robot arm, resulting in time saving and smooth motion. In a dynamically changing operating environment, endpoints are calculated by implementing an intelligent module that captures a three-dimensional image of a processing area and predicts a 3D pose that in turn is converted into position coordinates for the robot to utilize to move to a target endpoint. A planning manager working in synchronization with the intelligent modules and sequence managers enables a very advantageous environment to allow the object to move seamlessly and efficiently smooth at a controlled rate, thereby mimicking human motion.

Description

Technical Field

[0001] The present invention relates to a trajectory generation and storage method for smooth and shortest path motion of a robot arm, the method uses a planning manager, a sequence manager and a trajectory path manager in combination with an intelligent module. The present invention also provides parallel processing of trajectory paths for future robot arm motions to minimize operation time, thereby resulting in increased speed of the manufacturing process. Background Art

[0002] In industrial scenarios, trajectories are taught by human experts and later reproduced in a typical teach-and-reproduce manufacturing environment. A trajectory or path connects two or more waypoints, resulting in one or more trajectories for each robot arm. Each waypoint is typically taught by a human operator based on an object pick and place pose or position.

[0003] According to conventional methods, an operator prepares the robot arm by teaching waypoints in a trial-and-error manner, so as to satisfy the condition that the robot arm does not enter an area where it becomes inoperable or does not interfere with peripheral devices or other waypoints. In some cases, there is a possibility that the robot trajectory path that enables the robot arm to operate at an optimal motion rate is not generated quickly enough, thus delaying the cycle time. In such a case, manual intervention is required to reconsider and reset the taught waypoints of the robot arm to achieve the optimal cycle time.

[0004] In another conventional method, an operator may prepare the robotic arm by teaching two or more waypoints that may require sharp turns or changes in direction, which may cause the arm to generate a trajectory with sudden acceleration or deceleration motions that may affect the object pick-up or place operation due to vibrations generated by such motions of the robotic arm. The likelihood of affecting the manufacturing environment is high, resulting in inefficient operations, increased cycle times, and low-quality end products.

[0005] A reproducible robotic arm is typically designed so that the robotic controller teaches the robotic arm a number of waypoints at one end of the robotic arm (manipulator); these waypoints may include (but are not limited to) a start point, an end point, and intermediate points between the start point and the end point. A robotic hand is integrated into one end of the manipulator in an appropriate manner. The robotic controller is programmed to perform interpolation on the taught command points to determine a trajectory through the waypoints. Next, the robotic controller is programmed to move the robotic arm smoothly along the determined trajectory. While this may work in most environments, the robotic arm may encounter collisions with certain obstacles or dynamically changing environments where waypoints may move depending on the task at hand and the objects to be processed.

[0006] Trajectory path generation aims to create a motion control path for the robot arm, which ensures that the planned trajectory is executed smoothly without any unstable jerking motion and covers the distance in the fastest possible time without colliding with any waypoints around the operating area.

[0007] Advanced trajectory path generation is required along with a planning manager to implement forward-looking strategies to execute motions for a specific trajectory path without adding time overhead, and can include motions that have been well planned in advance to accelerate the trajectory path of the robot arm. More efficient, safe and fast operations are ensured by implementing intelligent modules to manage the robot arm trajectory taking into account obstacle avoidance, traversing complex mazes, and dynamically changing waypoints in some complex situations. During the management and advanced planning of the trajectory, when electric / hydraulic or pneumatic drive modes are used in actual implementation to achieve a smooth trajectory path, there are many mechanical motion laws that must be considered, such as the speed and acceleration saturation limits of the motor drive device, and the resonance points of the actuator structure. The trajectory path involves determining the path using multiple waypoints, planning the route through the path determination to avoid collisions and determine the shortest route. The path is then used by the robot controller to move the robot arm in the planned trajectory path. The trajectory path uses the geometric path planned for a specific task to determine the speed and acceleration around each waypoint according to the lateral angle to minimize the effects of resonance and smooth the movement of the robot arm. In simple industrial machines, the trajectory path is usually trained by human operators through trial and error. However, as machines become more complex and increase their performance capability levels, it becomes necessary to incorporate automated methods of trajectory path planning where a separate trajectory path planning manager may not be sufficient. In such cases, imaging-based devices and methods need to be combined with intelligent modules to dynamically plan trajectory paths by using forward-looking algorithms and pre-stored trajectory paths, or in cases where the robot environment has changed, it is important to quickly calculate and transmit the new trajectory path to the robot controller to ensure fast performance. In addition, by combining artificial intelligence and machine learning techniques, all frequently used trajectory paths are stored in a database related to specific product types in order to minimize machine configuration settings during product changeovers. Therefore, these methods have become a necessity rather than an option in industrial scenarios for robot control.

[0008] In some cases, the trajectory path planning algorithm has limitations, which may sometimes result in the inability to plan the motion of the robot arm from one waypoint to another in one trajectory by avoiding obstacles. In such cases, when the robot arm needs to carefully maneuver around obstacles, it is necessary to plan multiple trajectories when handling objects, where two or more intermediate points may need to be added between two waypoints, and thereby plan and combine multiple trajectories to reach the target location. This results in a start / stop motion characteristic of the robot motion executing multiple trajectories, where there are multiple deceleration motions and acceleration motions for each trajectory, thereby causing vibrations due to the resonance characteristics involved. Summary of the invention

[0009] In view of the background, it is an object of one aspect of the present invention to provide a trajectory path planning manager for a motion control device using a planning manager in conjunction with a sequence manager for standard real-world applications where waypoints do not change.

[0010] Another object of one aspect of the present invention is to provide a trajectory path planning manager for a motion control device for dynamically changing environments where waypoints can change by complementing the planning manager and sequence manager with an intelligent module to capture images and provide mapping to the trajectory path management module. The intelligent module helps to handle complex situations that usually occur in multiple robot arm applications or obstacles. Using the work layout map provided by the intelligent module, the trajectory path management module utilizes a class of polynomial interpolation or spline interpolation to generate the desired path waypoints to achieve a time-based control sequence so that the robot arm moves smoothly from its current position to the destination. The trajectory path manager module handles the exponential complexity of the degrees of freedom (controllable joints) of a given robot by applying artificial intelligence and then machine learning techniques. The trajectory path manager also includes a device for calculating according to a spline interpolation implementation that produces smooth motion without position, velocity and acceleration discontinuities between multiple path segments and avoids the Runge phenomenon that causes oscillations or ringing at the beginning or end of the robot arm movement; the Runge phenomenon is typical in the polynomial interpolation implementation when working with equidistant waypoints.

[0011] According to another aspect of the present invention, a method for determining a single motion by implementing a chained motion pattern is provided, wherein multiple trajectories are planned together and then joined into one trajectory. This will reduce the pauses between two trajectories, as the robot arm does not have to wait for the next trajectory to be planned for each segment of the multi-part path. When merging multiple trajectories, some deceleration is typically experienced during the transition of multiple trajectories due to deceleration and acceleration at each intermediate waypoint, especially when the planned trajectory requires a sudden change in direction. In order to overcome this problem, a blending concept is implemented for the trajectory with a smooth curve, wherein the intermediate waypoints are not reached but the blending radius is calculated to merge with the path of the next trajectory to effectively reduce or minimize the effects of sudden deceleration and acceleration. The blending is controlled by a blending radius parameter, which determines the amount of curvature required to smooth the path near the transition. This makes the movement as smooth as human movement.

[0012] According to another aspect of the present invention, a method is provided wherein trajectories between all known waypoints are planned in advance by a planning manager, so that when a trajectory sequence needs to be executed, the robot controller obtains the trajectory from the planning manager without any waiting time because the trajectory is pre-planned and pre-stored. In the case where a pre-planned and stored trajectory path is not available for a set of waypoints, the planning manager together with the sequence manager determines a trajectory path from a starting state to the nearest intermediate point where a pre-planned trajectory path exists, resulting in a significant reduction in time compared to the case where a pre-planned trajectory path is not available. In the case where a pre-stored trajectory path to a final state does not exist, the same logic is executed, in which case a new trajectory path is determined from the nearest waypoint to the final state. The flexibility of the database of pre-stored trajectory paths is enhanced, making it more efficient and fast.

[0013] According to another aspect of the invention, a method is provided in which a 3D (three-dimensional) image of the working environment is captured by an intelligent module that predicts a 3D pose or map to determine the coordinates of all waypoints and obstacles to guide the robot arm to a specific endpoint according to instructions received from a robot controller. In addition, parameters such as rotation coordinates for roll, pitch and yaw are additionally determined based on the 3D map to derive a 6D (six-dimensional) calculation of the trajectory path for moving the robot arm from a starting state to a final state.

[0014] According to another aspect of the present invention, a method is provided in which an intelligent manager processes a pipeline of imaging server actions to determine object positions and transmits non-linear trajectory sequences to a robotic controller in real time.

[0015] According to another aspect of the present invention, a method for quickly calculating motion trajectories for automated systems, such as robots, cranes, and vehicles, is provided. Given a set of waypoints and associated obstacles and constraints, one or more starting states of the robot, and one or more desired end states for the robot, the method begins with a novel iterative process to calculate sample motion trajectories that are fast and meet the goals of the robot's target state. Each iteration improves the trajectory by implementing a PID (proportional-integral-differential) algorithm, an MFC (model predictive control) algorithm, and an adaptive control algorithm to make the motion of the robot arm faster and smoother until the motion of the robot arm meets all criteria for the desired tolerance of the acceleration and velocity of the payload. The process, which is offline (before execution), is used to generate multiple trajectories in several starting and ending states of the robot, and then artificial intelligence and machine learning techniques are applied to create a database of all frequently used trajectories. The process enables rapid response to commands for moving the robot from one state to another, because the database has most of the trajectories pre-calculated and stored in advance. With the further implementation of deep learning techniques, robot processes and motions are stored and optionally fine-tuned by frequent calculations involving changes in the input and output states of the robot.

[0016] With the Planning Manager and Sequence Manager working together with the Intelligent Module, most trajectories are ready to be executed with minimal computation time. This process of handling a vision-guided robotic arm with complex balancing paths is fast, seamless and smooth, mimicking human motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present disclosure will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings:

[0018] Figure 1 A block diagram of a computing system including the various subsystems involved in determining an optimized trajectory path to command a robotic controller for corresponding robotic arm motions is shown.

[0019] Figure 2 A flow chart illustrating trajectory motion planning for a robotic arm with predetermined waypoints according to an embodiment is shown.

[0020] Figure 3 A flow chart illustrating trajectory motion planning for a robotic arm with predetermined and dynamic waypoints according to an embodiment is shown.

[0021] Figure 4 A flow chart of sequence A according to an embodiment is shown, wherein the intelligent manager predicts 3D waypoints, calculates rotation coordinates roll, pitch and yaw, and thereby generates a six-dimensional trajectory path of the working area.

[0022] Figure 5 An example of a trajectory path between six waypoints using spline interpolation is shown. DETAILED DESCRIPTION

[0023] The following description relates to calculating motion trajectory paths for an automated system that takes into account the payload and balance of the robot while avoiding obstacles, reducing stress to minimize wear on the machinery involved, and producing fast and jerk-free motion from an initial position to a final position.

[0024] In one embodiment of the present disclosure, the systems and methods provided herein are used to accelerate and improve the motion of a robot for loading and unloading contact lens blister packages from an input loader to an inspection position in an automated production line. In this case, the setting of the start state is defined by the desired position of a suction cup or a pick-up end effector integrated into the robot to lift the blister package. The setting of the end state is defined by the position of the pick-up end effector at the inspection station, wherein precise fine positioning occurs through intelligent imaging and subsequent deactivation of the end effector suction cup that holds the contact lens blister package, resulting in unloading at the inspection station. In the end state of unloading the contact lens blister package, there is a high degree of positional accuracy in the x-axis, y-axis, and z-axis and orientation. In order to increase system throughput and reduce wear on the robot, it is desirable to use a chain mode concept to quickly calculate a motion trajectory path from one of the start states to one of the end states, which avoids both fixed obstacles and dynamically moving obstacles. The chain mode is implemented by a planning manager by cascading multiple trajectories using smooth motions of velocity and acceleration to ensure vibration-free and jerk-free motion through all given waypoints.

[0025] Reference Figure 1, a block diagram 50 is illustrated as an example of a computing system 52 for jitter-free and vibration-free motion planning according to an embodiment. The computing system 52 includes a sequence planning subsystem 106, a path planning subsystem 108, and a trajectory planning subsystem 110, which are linked to a database in an appropriate manner and communicatively coupled to a robot controller 104 to control a robot arm 102 within an automated machine 112 or another system suitable for autonomous motion. Although the computing system 52 is depicted as a separate system from the robot controller 104, it should be understood that in some examples, one or more components of the computing system 52 can be integrated into the robot controller 104 or as a stand-alone module to operate the robot arm 102. The computing components can include, but are not limited to, artificial intelligence algorithms, machine learning algorithms, reinforcement learning algorithms, and deep learning algorithms to build a powerful and efficient database of trajectory paths that can be readily obtained when a waypoint map is received from the computing system, rather than calculating the trajectory paths in real time. The database serves as a trajectory path storage subsystem that can be easily used, suitable for different configurations of industrial machines used in processing and manufacturing different products.

[0026] Reference Figure 2 , flowchart 100 schematically illustrates the operations to be performed by the robot controller. Flowchart 100 shows Figure 1 The flow chart particularly focuses on automated machines where the planned trajectory path is based on a predetermined set of waypoints. This can be applicable to automated machines such as packaging, sorting or labeling of processed products where most of the processing points for the products are fixed.

[0027] The process generally begins with an input command 116 to the robot controller. The robot controller then transmits the waypoints (initial waypoint and final waypoint) to the planned trajectory path database 110. If a pre-planned path or a predetermined path is available for the waypoint, the database in turn sends the desired trajectory path to the robot controller, after which the robot controller 104 will drive the robot arm 102 to the final destination waypoint at the specified movement rate, speed and trajectory path. However, in the event that a predetermined trajectory path does not exist or is unavailable, the robot controller 104 obtains information from the planned trajectory path database 110 that a predetermined trajectory path does not exist, which is then followed by the robot controller 104 sending a command to the sequence manager 106 to start the process of planning a new trajectory path. The sequence manager 106 also checks the waypoint positions and determines the sequence of trajectory paths to be executed after implementing the blending of the trajectory paths with all intermediate waypoints, taking into account the blending radius parameter, which determines the amount of curvature required to smooth the trajectory path near the transition from one waypoint to the next waypoint. The calculated sequence is then transmitted to the path manager 108 which determines the fastest trajectory path to the final destination waypoint and uploads the information to the planned trajectory path database 110 for storage and further transmission to the robotic controller 104 which in turn executes the motion of the robotic arm 102. The executed motion results in smooth motion like human motion.

[0028] exist Figure 3 , flowchart 130 schematically illustrates the operations to be performed by the robot controller in the event of encountering a dynamic waypoint. Flowchart 130 shows Figure 1 The invention relates to the interaction between various modules within the computing system 52 of the present invention, wherein the planned trajectory path is based on both a changing set of waypoints and a fixed set of waypoints. This can be applicable to automated machines, where interactions between several processing points are involved, resulting in unpredictable environments and changing conditions. The complexity of such operations requires a series of pre-planned motion sequences, on-the-fly calculation of trajectories, and a path manager that can quickly rearrange the sequence and trajectory path to achieve smooth and precise operation of the robotic arm.

[0029] Figure 3The flowchart in begins with an input command 116 to the robot controller. The robot controller 104 then transmits the waypoints (initial waypoints, intermediate waypoints, and final waypoints) to the planned trajectory path database 110. If a pre-planned trajectory path or a predetermined trajectory path is available for the waypoint, the database in turn sends the desired trajectory path to the robot controller, after which the robot controller 104 will drive the robot arm 102 to the final destination waypoint at the specified motion rate, speed, and curvature. However, in the event that the predetermined trajectory path does not exist or is unavailable, or if there are obstacles to the pre-planned trajectory path, the robot controller 104 sends a command to the sequence manager 106 to start the process of planning a new trajectory. The sequence manager 106 also checks the waypoint locations including obstacles and intermediate waypoints. This information is passed to the planning manager 108. The planning manager, aware of the new changed environment, sends a query to branch 112 (A). The trajectory path manager determines the trajectory path by implementing a PID (proportional-integral-derivative) algorithm, an MFC (model predictive control) algorithm, and an adaptive control algorithm to ensure jitter-free and resonance-free motion from the robot arm's starting state or starting waypoint through all intermediate waypoints to the end state or end waypoint. In addition, the calculated trajectory path is uploaded to 116 to further characterize the trajectory path using artificial intelligence, machine learning, and reinforcement learning, and then the calculated trajectory path is updated to reside in Figure 1 The database 110 is in the computing system 52 or resides in the cloud. The database 110 is iteratively fine-tuned and updated to ensure that the robotic arm can operate with minimal delay for specific commands by utilizing pre-planned trajectory paths.

[0030] exist Figure 4 , flowchart A begins with the intelligent manager 195 sending a command to the imaging system to capture a 3D pose of the work environment at step 200. At step 205, the software algorithm identifies and calculates the x-coordinate, y-coordinate, and z-coordinate, and determines the rotation values ​​associated with roll, pitch, and yaw, and then returns a total of six values ​​at step 210, namely the x-coordinate, y-coordinate, and z-coordinate of the current state of waypoint positions that may include obstacles and intermediate waypoint positions, and the determined values ​​for roll, pitch, and yaw. At step 210, a trajectory path is determined. In addition, all waypoints, starting states, and final states are considered at 205 to determine parameters, such as rotation coordinates, such as pitch, roll, and yaw, to determine the trajectory path at 210. In the next step, the six-dimensional trajectory path determined at 210 is relayed to Figure 3 , and then ends the process in 220.

[0031] exist Figure 5, an example of a trajectory path calculated using spline interpolation between six waypoints P0 to P6 is shown. Spline interpolation fits a low-degree polynomial between two adjacent waypoints, for example, five cubic polynomials or piecewise polynomials between each of the six waypoint pairs, instead of fitting a single sextic polynomial. This minimizes interpolation errors and also eliminates problems associated with oscillations (vibrations), resulting in smooth motion of the robot arm. Figure 5 In the line graph 300 of FIG. 3 , six waypoints, namely P0, P1, P2, P3, ... P6, are shown as examples. C1 to C5 are the generated curves, which are blended to form a spline curve between each pair of waypoints in the waypoints. Obviously, the robot controller follows the trajectory path of C1 to C5 to move from its starting state P0 to the ending state P6. The mathematical formulas or analysis between the waypoints, acceleration, deceleration and peak speed are not discussed because they fall outside the scope of the present invention.

[0032] It will be understood that various modifications not described herein may be made, and all such modifications falling within the scope of the invention are intended to be covered in the appended claims. The written description uses typical scenarios to disclose the invention, and also enables a person skilled in the relevant art to practice the invention, including making and using any device or system and implementing any included method. The patentable scope of the invention is defined by the claims, and may include other scenarios that may occur to a person skilled in the art.

Claims

1. A method comprising: calculating a trajectory from a start state through one or more intermediate waypoints to an end state using a plan manager, a sequence manager, and a trajectory manager in conjunction with an intelligent module, the intelligent module utilizing an imaging system, using spline interpolation and fitting a plurality of low-degree polynomials between two adjacent waypoints; updating the trajectory to a database in a computing system or on the cloud for rapid retrieval of trajectory paths and rapid processing of objects in an environment in which the robotic arm is driven to obtain a final trajectory, according to kinematic and dynamic constraints of the automated system; as well as The robotic arm is operated from a starting state through one or more intermediate waypoints to an ending state according to the final trajectory transmitted by a robotic controller.

2. The method according to claim 1, wherein: Calculating the trajectory includes: calculating multiple trajectories using a PID (proportional-integral-derivative) algorithm, an MFC (model predictive control) algorithm, and an adaptive control algorithm for an input command, wherein the input command specifies one or more starting states of the robot arm and one or more ending states of the robot arm, and each trajectory path in the multiple trajectories has a different duration and a different distance.

3. The method according to claim 2, wherein: The planning manager and the sequence manager calculate the plurality of trajectories based on degrees of freedom associated with the robotic arm in one or more of the starting states and one or more of the ending states.

4. The method according to claim 2, wherein: The planning manager and the sequence manager in conjunction with the intelligence module calculate the plurality of trajectories based on degrees of freedom associated with the robotic arm in one or more of the starting states and one or more of the ending states.

5. The method of claim 2 further comprises updating the trajectory path to achieve optimal rate and accuracy over time by repeated fine-tuning using applied artificial intelligence and machine learning in combination with reinforcement learning techniques to minimize jitter in the robotic arm. 6 . The method according to claim 2 , further comprising updating the trajectory path to a database for storage and retrieval according to corresponding input states and end states.

7. The method according to claim 1, wherein: The work environment includes a robotic arm mounted on an industrial machine, the robotic arm being configured for repetitive and non-repetitive motions.

8. The method according to claim 7, wherein: The repeated motion trajectory path is obtained from the pre-stored trajectory path existing in the database in the computing system or the cloud.

9. The method according to claim 7, wherein: A non-repetitive motion trajectory path is calculated by utilizing the 3D positions of all waypoints of the working environment acquired by the imaging system and determining the roll, pitch and yaw required for the robotic arm to move from a starting state to the ending state.

10. The method according to claim 7, wherein: In the event that there is no non-repetitive motion trajectory path from a given starting state to an ending state in the database, the computing system first identifies a nearest waypoint to the starting state and a nearest waypoint to the ending state, and then obtains a trajectory path between the two repetitive waypoints pre-stored in the database.

11. The method according to claim 10, wherein: The computing system computes a trajectory path from the starting state to the nearest waypoint, and then computes another trajectory path from the nearest waypoint to the ending state to arrive at a complete trajectory path from the starting state to the ending state, thereby effectively combining a predetermined trajectory path obtained from the database and a set of computed trajectory paths.

12. A system comprising: Automated machines utilizing robots and operated by a robotic controller system configured for automated, repetitive, and non-repetitive motion; The robotic controller is configured with a software application which, when executed, causes the controller to: Utilizing a PID (proportional-integral-derivative) algorithm, an MFC (model predictive control) algorithm, and an adaptive control algorithm to calculate a trajectory path from one or more starting states to one or more ending states of a working environment; updating the trajectory path with associated constraints and restrictions to a database residing on a computing system or in the cloud for subsequent storage and retrieval; as well as The automated machine is automatically controlled from one of the one or more starting states to one of the one or more ending states according to the trajectory path.

13. The system according to claim 8, wherein: In order to calculate the trajectory, the controller is also configured with an algorithm that enables the controller to: A plurality of trajectory paths are calculated for inputs specifying the one or more starting states of the automated machine and the one or more ending states of the automated machine, each trajectory path of the plurality of trajectory paths having a different duration, speed, acceleration, and curvature.

14. A system comprising: Automated machines utilizing robots and operated by a robotic controller system incorporating an imaging system and configured for automated, repetitive, and non-repetitive motion; The robotic controller is configured with a software application which, when executed, causes the controller to: using the imaging system to acquire images of the work environment and identify waypoints; Using an intelligent module and applying a PID (proportional-integral-derivative) algorithm, an MFC (model predictive control) algorithm, and an adaptive control algorithm to calculate a trajectory path that navigates from one or more starting states of the operating environment through intermediate waypoints to one or more ending states; updating the trajectory path with associated constraints and restrictions to a database residing on a computing system or in the cloud for subsequent storage and retrieval; as well as The automated machine is automatically controlled from one of the one or more starting states to one of the one or more ending states according to the trajectory path.

Images