Method and system for orbital path planning

Abstract

To provide a trajectory generating method for moving a robot arm in the fastest and smoothest path.SOLUTION: The method of the present invention achieves time saving and smooth movements by implementing a planning, sequence and trajectory path manager to plan movements ahead of a predetermined time, and by concurrently creating a database of trajectory paths that is subsequently used for storing and retrieving of repetitive movements of a robot arm. In a dynamically changing operating environment, endpoints are calculated through implementation of an intelligence module. The intelligence module captures a three-dimensional image of the processing area and predicts a three-dimensional pose. The three-dimensional pose, in turn, is translated to positional coordinates for a robot to utilize, to move to the target endpoint. The planning manager working in sync with the intelligence module and the sequence manager enables a very favorable environment to seamlessly and efficiently move objects smoothly at controlled speeds, mimicking human movement.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a trajectory generation and storage method for smooth and shortest path movement of a robot arm using a planning, sequence and trajectory path manager in conjunction with an intelligent module. The present invention also provides for parallel processing of trajectory paths for future robot arm movements to minimize operation time and result in faster manufacturing processes. [Background technology]

[0002] In industrial environments, trajectories are taught by a human expert and then played back in a typical teach-and-playback manufacturing environment. One or more trajectories are generated for each robotic arm by connecting two or more waypoints, where each waypoint is typically taught by a human operator based on the pick and place pose or position of the object.

[0003] According to conventional methods, an operator prepares a robot arm by teaching it way points in a trial-and-error manner, while ensuring that the way points do not enter areas where the robot arm is unable to operate or do not interfere with peripheral devices or other way points. In certain situations, there is a possibility that a robot trajectory path that allows the robot arm to operate at an optimal motion speed may not be generated quickly enough, resulting in a delayed cycle time. In such cases, manual intervention is required to reconsider and reset the way points of the robot arm to achieve the optimal cycle time.

[0004] In other conventional methods, an operator may prepare a robotic arm by teaching it one or more waypoints that may require abrupt turns or changes of direction. This causes the arm to generate a trajectory with sudden acceleration or deceleration movements, which can affect object pickup or placement due to vibrations caused by such movements of the robotic arm. This can potentially impact the manufacturing environment, resulting in inefficient operations, as well as increased cycle times and lower quality of the final product.

[0005] A playback robot arm is typically designed so that a robot controller teaches the robot arm multiple waypoints at one end of the robot arm (manipulator). These waypoints may include (but are not limited to) a start point, an end point, and intermediate points between them. A robot hand is suitably integrated at one end of the manipulator. The robot controller is programmed to perform interpolation on the taught command points to determine a trajectory that passes through the waypoints. The robot controller is then programmed to move the robot arm smoothly along the determined trajectory. While this may work in most environments, the robot arm is likely to encounter collisions with predetermined obstacles or dynamically changing environments where the waypoints may move depending on the task at hand and the object being machined.

[0006] Trajectory path generation aims to create a motion-controlled path for the robot arm that will allow it to smoothly execute the planned trajectory without unstable jerks and cover the distance at the highest possible speed without colliding with any of the waypoints around the working area.

[0007] Advanced trajectory path generation, along with a planning manager, is required to implement forward-looking strategies for specific trajectory path movements without adding overhead to timing, and for movements that may include trajectory paths that have been planned well in advance to speed up the robot arm. Managing the robot arm trajectory by implementing an intelligence module to avoid obstacles, navigate complex mazes, and, in some complex cases, take into account dynamically changing waypoints, ensures more efficient, safe, and rapid operation. There are many machine motion laws that must be considered during trajectory management and advanced planning, such as the saturation limits of motor drive speed and acceleration, and the resonance points of actuator structures when arriving at a smooth trajectory path in practical implementations using electro-hydraulic or pneumatic drive modes. Trajectory paths involve determining a path using multiple waypoints, and routing through those waypoints is planned to avoid collisions and determine the shortest path. The path is then used by the robot controller to move the robot arm along the planned trajectory path. The trajectory path uses a geometric path planned for a specific task, determining the velocity and acceleration around each waypoint according to the turning angle, minimizing the effects of resonance and ensuring smooth robot arm movement. For simple industrial machines, trajectory paths are typically trained by a human operator through trial and error. However, as machines become more complex and performance levels increase, an automated method for trajectory path planning must be incorporated. Here, a trajectory path planning manager alone may not be sufficient. In such cases, an image-based device and method in conjunction with an intelligence module is required to dynamically plan trajectory paths by using a forward lookup algorithm and pre-stored trajectory paths, or, when the robot environment changes, it is essential to quickly calculate and communicate new trajectory paths to the robot controller to ensure high-speed performance. Furthermore, by incorporating artificial intelligence and machine learning techniques, all frequently used trajectory paths are stored in a database associated with a specific product type to minimize machine configuration settings during product changeover.Therefore, these methods have become a necessity rather than an option in robotic controlled industrial environments.

[0008] In certain cases, trajectory path planning algorithms have limitations that may result in the planning of the robot arm's movement from one waypoint to another in a trajectory failing due to obstacle avoidance. In such cases, when the robot arm needs to carefully maneuver around an obstacle, it is essential to plan multiple trajectories while handling the object, in which case two or more intermediate points must be added between two waypoints, and multiple trajectories must be planned and combined to reach the target position. This results in start / stop operations for the robot motion executing multiple trajectories, and multiple deceleration / acceleration operations for each trajectory will generate vibrations due to the resonance characteristics involved. Summary of the Invention

[0009] In view of the background, it is an object of one aspect of the present invention to provide a trajectory path planning manager in a motion controller that uses a planning manager in conjunction with a sequence manager for typical practical applications where waypoints do not change.

[0010] Another object of one aspect of the present invention is to provide a motion controller with a trajectory path planning manager by providing a planning and sequence manager with an intelligence module that captures images and provides mapping to the trajectory path managing module for dynamically changing environments where waypoints may change. The intelligence module helps handle complex situations typically encountered in multiple robot arm applications or obstacles. Using the task layout map provided by the intelligence module, the trajectory path managing module generates desired path waypoints using a class of polynomial or spline interpolation to arrive at a time-based control sequence that smoothly moves the robot arm from its current position to its destination. The trajectory path manager module handles exponential complexity for a given robot's degrees of freedom (controllable joints) by applying artificial intelligence following machine learning techniques. The trajectory path manager further includes means for computing from a spline interpolation implementation to generate smooth motion between multiple path segments without discontinuities in position, velocity, and acceleration, avoiding the Runge phenomenon. The Runge phenomenon causes oscillations or ringing at the beginning or end of robot arm movements, which is typical of polynomial interpolation implementations when manipulating equidistant waypoints.

[0011] According to a further aspect of the present invention, a method for determining a single motion is provided by implementing a chain motion mode in which multiple trajectories are planned together and then assembled into one. This reduces pauses between two trajectories because the robot arm does not need to wait for the next trajectory to be planned for each segment of the other partial path. While merging multiple trajectories, deceleration typically occurs during multiple trajectory transitions due to deceleration and acceleration at each waypoint. This is especially true when the planned trajectory requires abrupt direction changes. To overcome this problem, a blending concept is implemented for trajectories with smooth curves. Here, rather than reaching a waypoint, a blending radius is calculated to merge with the path of the next trajectory to effectively reduce or minimize the impact of sudden decelerations and accelerations. The blending is controlled by a blending radius, which determines the amount of curvature needed to smooth the path near the transition. This makes the motion as smooth as a human's movements.

[0012] According to a further aspect of the present invention, a method is provided in which trajectories between all known waypoints are planned in advance by a planning manager. When a trajectory sequence needs to be executed, the robot arm retrieves the trajectory from the planning manager without waiting because the trajectory has been pre-planned and pre-stored. In the event that a pre-planned and stored trajectory path is unavailable for a set of waypoints, the planning manager, together with the sequence manager, determines a trajectory path from the start state to the nearest waypoint for which a pre-planned trajectory path exists. This significantly reduces the time compared to when a pre-planned trajectory path is unavailable. The same logic is also performed in the event that a pre-stored trajectory path does not exist until the final state. In this case, a new trajectory path is determined from the nearest waypoint to the final state. This enhances the flexibility of the pre-stored trajectory path database, making it efficient and fast.

[0013] According to a further aspect of the present invention, a method is provided in which a 3D (three dimensional) image of the working environment is captured by an intelligence module which predicts a 3D pose or map, and the coordinates of all way points and obstacles are determined to guide the robot arm to a specific end point according to instructions received from a robot controller. Furthermore, parameters such as rotational coordinates of column pitch and yaw are additionally determined according to the 3D map, resulting in a 6D (six dimensional) calculation of a trajectory path that will move the robot arm from a start state to a final state.

[0014] According to a further aspect of the invention, a method is provided for handling imaging server operations in a pipeline where an intelligence manager determines object positions and communicates non-linear trajectory sequences to a robot controller in real time.

[0015] According to a further aspect of the present invention, a method for rapidly calculating motion trajectories for automated systems such as robots, cranes, and vehicles is provided. Given a set of waypoints and associated obstacles and constraints, one or more starting states of the robot, and one or more desired ending states of the robot, the method initiates a novel iterative procedure to calculate rapid sample motion trajectories to meet the objective of the robot's goal state. Each iteration refines the trajectory by implementing PID (proportional-integral-derivative), MFC (model predictive control), and adaptive control algorithms until all criteria are met, up to the desired acceleration and velocity tolerances for the payload, resulting in faster and smoother robot arm movements. Offline (prior to execution), this procedure is used to generate multiple trajectories for several starting and ending states of the robot, followed by applying artificial intelligence and machine learning techniques to create a database of all frequently used trajectories. This process enables rapid response to commands to move the robot from one state to another, since the database has most trajectories pre-calculated and stored ahead of time. Further, by implementing deep learning techniques, the robot's processes and movements are stored and optionally fine-tuned through frequent calculations involving various input and output states of the robot.

[0016] The planning manager and sequence manager working in conjunction with the intelligence module facilitate the execution of most trajectories in minimal computational time. This process of maneuvering a visually guided robot arm through complex equilibrium paths is fast, seamless, and smooth, mimicking human movement. [Brief explanation of the drawings]

[0017] The present disclosure will be better understood from reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0018] [Figure 1]Figure 1 shows a block diagram of a computational system including the various subsystems involved, which determine the optimized trajectory path and provide commands to the robot controller for each robot arm movement. [Figure 2] FIG. 1 shows a flow diagram illustrating trajectory motion planning for a robotic arm with pre-determined waypoints, according to one embodiment. [Figure 3] FIG. 1 shows a flow diagram illustrating trajectory motion planning for a robotic arm with predetermined and dynamic way-points, according to one embodiment. [Figure 4] 10 shows a flowchart of Sequence A in which an intelligence manager predicts 3D waypoints, calculates rotational coordinates for roll, pitch, and yaw, resulting in a six-dimensional orbital path for a working area, according to one embodiment. [Figure 5] An example of a trajectory path between six waypoints using spline interpolation is shown. DETAILED DESCRIPTION OF THE INVENTION

[0019] The following description relates to the calculation of motion trajectory paths for automated systems that consider the robot's payload and balance while avoiding obstacles, reduce stress to minimize wear on the mechanical mechanisms involved, and generate rapid, jerky-free movement from initial to final positions.

[0020] In one embodiment of the present disclosure, the systems and methods provided herein are used to speed up and improve robot motion for loading and unloading contact lens blister packs from an input loader in an automated line to an inspection station. In this case, a set of starting states is defined by the desired position of a suction cup or pickup end effector integrated into the robot to pick up the blister packs. A set of ending states is defined by the position of the pickup end effector at the inspection station. Here, accurate and fine positioning is achieved through intelligent imaging and subsequent deactivation of the end effector suction cup holding the contact lens blister pack, resulting in unloading at the inspection station. At the end state of unloading the contact lens blister pack, there is a high degree of positional accuracy in the X, Y, and Z axes and orientation. To increase system throughput and reduce wear on the robot, it is desirable to quickly calculate a motion trajectory path from one of the starting states to one of the ending states. This motion trajectory path uses a chain mode concept to avoid both fixed and dynamically moving obstacles. Chain mode is implemented by a plan manager that links multiple trajectories together by employing smooth motion in terms of velocity and acceleration to ensure vibration- and jerky-free movement through all given waypoints.

[0021] Referring to FIG. 1 , a block diagram 50 is shown as an example of a computing system 52 for jerky-free and vibration-free motion planning, according to one embodiment. The computing system 52 includes a sequence planning subsystem 106, a path planning subsystem 108, and a trajectory planning subsystem 110 suitably linked to a database, and is communicatively coupled to a robot controller 104 to control a robot arm 102 or other system adapted for autonomous movement within an automated machine 112. Although the computing system 52 is depicted as a separate system from the robot controller 104, it should be understood that in some examples, one or more components of the computing system 52 may be integrated into the robot controller 104 or may serve as stand-alone modules to operate the robot arm 102. The computing components may include, but are not limited to, artificial intelligence, machine learning, reinforcement learning, and deep learning algorithms that build a powerful and efficient database of readily available trajectory paths instead of calculating trajectory paths in real time as waypoint maps are received from the computing system. The database serves as a readily available trajectory path storage subsystem that can adapt to different configurations of industrial machines used to process and manufacture different products.

[0022] Referring to Figure 2, a flow chart 100 illustrates schematically the operations performed by the robot controller. The flow chart 100 illustrates the interactions between various modules in the computing system 52 of Figure 1. The flow chart is particularly focused on automated machines, where the planned trajectory path is based on a predetermined set of waypoints. This is applicable to automated machines such as those used for packaging, sorting, or labeling processed products, where most of the processing points for the product are fixed.

[0023] This flow typically begins with an input command 116 to the robot controller. Subsequently, the robot controller communicates waypoints (initial waypoint and final waypoint) to the planned trajectory path database 110. In return, the database sends the desired trajectory path to the robot controller if a pre-planned or predetermined path is available for that waypoint. The robot controller 104 drives the robot arm 102 to the final destination waypoint at the specified motion speed, velocity, and trajectory path. However, in the event that a pre-determined trajectory path does not exist or is unavailable, the robot controller 104 obtains information about the absence of a pre-determined trajectory path from the planned trajectory path database 110, which then initiates the process of planning a new trajectory path, followed up by a command from the robot controller 104 to the sequence manager 106. The sequence manager 106 further checks the waypoint locations and determines the sequence of trajectory paths to be executed after implementing the blending of the trajectory path with all intermediate waypoints, taking into account a blend radius parameter that determines the amount of curvature required to smooth the trajectory path near the transition from one waypoint to the next. The calculated sequence is subsequently communicated to the path manager 108, which determines the fastest trajectory path to the final destination waypoint and uploads that information to the planned trajectory path database 110 for storage and further communication to the robot controller 104. The robot controller 104 then executes the movement of the robot arm 102. The executed movement results in a motion as smooth as a human's.

[0024] In FIG. 3, a flow chart 130 schematically illustrates the operations performed by the robot controller, where a dynamic waypoint is encountered. Flow chart 130 illustrates the interactions between various modules in the computational system 52 of FIG. 1, where the planned trajectory path is based on both a variable set of waypoints and a fixed set of waypoints. This is adaptable to automated machinery where interactions between several machining points are involved, resulting in unpredictable environments and changing conditions. The complexity of such operations requires a series of pre-planned sequence operations, on-the-fly calculation of trajectories, and a path manager that can quickly rearrange the sequences and trajectory paths to achieve smooth and accurate movement of the robot arm.

[0025] The flowchart in Figure 3 begins with an input command 116 to the robot controller. Subsequently, the robot controller 104 communicates waypoints (initial waypoint, intermediate waypoint, and final waypoint) to the planned trajectory path database 110. In return, the database sends the desired trajectory path to the robot controller if a pre-planned or pre-determined trajectory path is available for that waypoint. The robot controller 104 drives the robot arm 102 to the final destination waypoint with the specified motion speed, velocity, and curvature. However, in the event that a pre-determined trajectory path does not exist or is unavailable, or if an obstacle exists in the pre-planned trajectory path, the robot controller 104 sends a command to the sequence manager 106 to begin the process of planning a new trajectory. The sequence manager 106 further checks the waypoint locations, including obstacles and intermediate waypoints. This information is communicated to the planning manager 108. The planning manager, recognizing the new environment after the change, sends a query to branch 112(A). The trajectory path manager determines the trajectory path by implementing PID (Proportional Integral Derivative), MFC (Model Predictive Control), and adaptive control algorithms to ensure jerky and resonance-free movement of the robot arm from the starting state or waypoint to the ending state or waypoint, through all intermediate waypoints. Furthermore, the calculated trajectory path is uploaded to 116 for further characterization of the trajectory path using artificial intelligence, machine learning, and reinforcement learning, and subsequently updated to database 110 residing in computing system 52 of FIG. 1 or in the cloud. Database 110 is iteratively fine-tuned and updated to ensure that the robot arm can operate with minimal delay for specific commands by utilizing the pre-planned trajectory path.

[0026] In FIG. 4, Flowchart A begins in step 200 with the intelligence manager 195 sending a command to the imaging system to capture a 3D pose of the working environment. In step 205, a software algorithm identifies and calculates X, Y, and Z coordinates and determines rotational values ​​related to roll, pitch, and yaw, followed by returning a total of six values ​​in step 210: the X, Y, and Z coordinates of the current state of waypoint locations, which may include obstacles and intermediate waypoint locations, and the determined values ​​of roll, pitch, and yaw. In step 210, a trajectory path is determined. Further, parameters such as rotational coordinates like pitch, roll, and yaw are determined in 205, taking into account all waypoints, the starting state, and the final state, and the trajectory path is determined in 210. In the next step, the six-dimensional trajectory path determined in 210 is relayed to the path manager 107 in FIG. 3, followed by ending the flow in 220.

[0027] FIG. 5 shows an example of a trajectory path calculated using spline interpolation between waypoints P0-P6. Spline interpolation fits a low-order polynomial between two adjacent waypoints. For example, instead of fitting a single sixth-order polynomial between each pair of six waypoints, five third-order or piecewise polynomials are fitted. This minimizes interpolation errors and eliminates vibration-related problems, resulting in smooth robot arm movement. In the diagram 300 of FIG. 5, six waypoints, namely P0, P1, P2, P3...P6, are shown as an example. C1-C5 are the resulting curves blended to form a spline curve between each pair of waypoints. Obviously, the robot controller moves from the start state P0 to the end state P6 by following the trajectory path of C1-C5. The mathematical formulas or mathematical analysis between waypoints, acceleration, deceleration, and peak velocity are beyond the scope of this invention and will not be described here.

[0028] It will be understood that various modifications not yet described may be made, and it is intended by the appended claims to cover all such modifications that are within the scope of the present invention. The written specification uses exemplary examples to disclose the invention and enable those skilled in the art to practice the invention, including making and using any device or system, and implementing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art.

Claims

1. 1. A method comprising: calculating a trajectory by a plan manager, a sequence manager, and a trajectory manager in combination with an intelligence module utilizing an imaging system, using spline interpolation, and fitting a plurality of low-order polynomials between two adjacent waypoints from a start state to an end state through one or more intermediate waypoints; According to the kinematic and dynamic limitations of the automation system to obtain the final trajectory, a fast search for a trajectory path of an object in the working environment for the driving environment of the robot arm and updating said trajectory to a database in a computing system or cloud for the purpose of fast processing; moving the robot arm from the start state through one or more waypoints to the end state according to a final trajectory communicated by a robot controller; A method comprising:

2. 2. The method of claim 1, wherein calculating the trajectories comprises calculating a plurality of trajectories for input commands specifying one or more starting states of the robot arm and one or more ending states of the robot arm by PID (Proportional Integral Derivative), MFC (Model Predictive Control), and adaptive control algorithms, wherein each trajectory path in the plurality of trajectories has a different duration and a different distance.

3. The method of claim 2 , wherein the plan manager and sequence manager calculate the plurality of trajectories according to degrees of freedom associated with the robot arm at the one or more starting states and the one or more ending states.

4. 3. The method of claim 2, wherein the planning manager and sequence manager, in cooperation with the intelligence module, calculate the plurality of trajectories according to degrees of freedom associated with the robot arm at the one or more starting states and the one or more ending states.

5. 3. The method of claim 2, further comprising updating the trajectory path to minimize jerks of the robotic arm with iterative fine-tuning via application of artificial intelligence and machine learning combined with reinforcement learning techniques to achieve optimal speed and accuracy over time.

6. 3. The method of claim 2, further comprising updating the orbital path to the database for storage and retrieval purposes in response to respective input and output states.

7. The method of claim 1 , wherein the robotic arm is attached to an industrial machine configured for repetitive and non-repetitive movements in the work environment.

8. The method of claim 7 , wherein the trajectory path of the repetitive motion is obtained from a pre-stored trajectory path present in the database located on the computing system or the cloud.

9. 8. The method of claim 7, wherein a trajectory path of a non-repetitive motion is calculated by utilizing the 3D positions of all way points in the work environment acquired by the imaging system and determining the roll, pitch, and yaw required for the robot arm to move from the start state to the end state.

10. 8. The method of claim 7, wherein if a trajectory path of a non-repeating motion from a given start state to an end state does not exist in the database, the computing system first identifies a waypoint closest to the start state and a waypoint closest to the end state, and subsequently obtains a trajectory path between these two repetitive waypoints, which may be pre-stored in the database.

11. 11. The method of claim 10, wherein the computing system calculates a trajectory path from the start state to a nearest waypoint, and then calculates another trajectory path from the nearest waypoint to the end state to arrive at a complete trajectory path from the start state to the end state, effectively combining the set of calculated trajectory paths with a predetermined trajectory path obtained from the database.

12. 1. A system comprising: Automated machines utilizing robots, including automated machines operated by a robot controller configured for automated repetitive and non-repetitive motion; When executed, the robot controller calculating trajectory paths from one or more start states to one or more end states of the working environment by PID (proportional integral derivative), MFC (model predictive control) and adaptive control algorithms; updating the orbital path with associated constraints and restrictions to a database residing on a computing system or cloud; automatically controlling the automated machine from a start state of the one or more start states to an end state of the one or more end states according to the trajectory path; A system configured to have a software application that causes

13. In order to calculate the trajectory path, the robot controller further an algorithm that causes the robot controller to calculate a plurality of trajectory paths in response to inputs specifying the one or more starting states of the automated machine and the one or more ending states of the automated machine; The system of claim 12 , wherein each orbital path of the plurality of orbital paths has a different duration, velocity, acceleration, and curvature.

14. 1. A system comprising: An automated machine utilizing a robot, including an automated machine operated by a robot controller incorporating an imaging system and configured for automated repetitive and non-repetitive motion; When executed, the robot controller acquiring an image of a work environment using the imaging system to identify waypoints; calculating a trajectory path from one or more start states to one or more end states of the working environment, guiding the path through waypoints using PID (proportional integral derivative), MFC (model predictive control) and adaptive control algorithms; updating the orbital path with associated constraints and restrictions to a database residing on a computing system or cloud; automatically controlling the automated machine from a start state of the one or more start states to an end state of the one or more end states according to the trajectory path; A system configured to have a software application that causes

Images