A mechanical arm trajectory planning method, device, system and storage medium

Abstract

The application relates to a mechanical arm trajectory planning method, which comprises the following steps: acquiring the current position and working environment of a machine arm; obtaining two planning motion trajectories of the machine arm according to the current position and working environment of the machine arm, which are respectively a fast and smooth trajectory segment and a disturbance-free trajectory segment; and respectively performing trajectory planning on the fast and smooth trajectory segment based on a particle swarm algorithm and a deep neural network and on the disturbance-free trajectory segment based on a Cartesian space trajectory planning. The application can make the calculation amount of the trajectory planning algorithm lower and the calculation speed faster.

Description

Technical Field

[0001] This invention belongs to the field of edge computing for robots, and specifically relates to a method, device, system and storage medium for planning the trajectory of a robotic arm. Background Technology

[0002] Pharmaceutical workshops have high cleanliness requirements, and operators must wear protective suits when entering and exiting, which are expensive. Furthermore, bottle failure is a rare, accidental event, and the inconvenience of manually wearing protective suits is a significant factor. In a clean pharmaceutical workshop, ten hours might only yield less than one hour of effective working time, resulting in low efficiency. Therefore, a cleanroom robotic maintenance system is proposed to replace human operators and achieve intelligent and automated identification and handling of production line failures. Existing robotic gripping methods include using grippers and suction cups. However, the filling production line in clean spaces is relatively long, and the daily working frequency is low. Based on these characteristics, the gripping motion system adopts a single robotic arm plus a motion platform.

[0003] Currently, a method for planning robotic arm trajectories based on genetic algorithms has been proposed. This method includes determining the constraints of the robotic arm, including spatial position, velocity, and acceleration. Based on the genetic algorithm, and according to the constraints and a preset optimization objective, the robotic arm's motion trajectory is planned.

[0004] However, genetic algorithms are essentially heuristic algorithms. When used for trajectory planning in high-dimensional, redundant robotic systems, this method cannot provide real-time computational capabilities. Furthermore, genetic algorithm-based robotic arm trajectory planning methods do not perform in-depth analysis of trajectory characteristics. Summary of the Invention

[0005] To address this, the present invention provides a robotic arm trajectory planning method, apparatus, system, and storage medium, which segments the trajectory according to different trajectory characteristics. Different trajectory planning strategies are employed for each trajectory segment, resulting in a lower computational load and faster computation speed for the system's motion planning algorithm while meeting usage requirements.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a robotic arm trajectory planning method, the method comprising:

[0008] Obtain the current position and working environment of the robotic arm; based on the current position and working environment of the robotic arm, obtain two planned motion trajectories for the robotic arm, namely a fast and smooth trajectory segment and a undisturbed trajectory segment; perform trajectory planning for the fast and smooth trajectory segment and the undisturbed trajectory segment respectively.

[0009] Furthermore, the trajectory planning for the fast and smooth trajectory segment includes:

[0010] The particle swarm optimization algorithm is used to determine the number and position of interpolation points in the fast and smooth trajectory segment. The interpolation points include the start and end points of the robotic arm. Inverse kinematics is performed on the interpolation points to obtain a preset number of joint angle vectors. Joint space trajectory planning is performed between the joint angle vectors.

[0011] Furthermore, the robotic arm moves away from sparse obstacles during the rapid and smooth trajectory segment, and the movement is continuous and smooth, with stable start and stop, and the trajectory error is within a preset range.

[0012] Furthermore, trajectory planning is performed on the undisturbed trajectory segment using trajectory planning in Cartesian space.

[0013] Furthermore, the step of using the particle swarm optimization algorithm to determine the number of interpolation points and the positions of the interpolation points in the fast smooth trajectory segment includes:

[0014] The optimal solution is obtained using the particle swarm optimization algorithm, and a dataset is created. Based on the dataset, two neural networks are trained. The number and location of the interpolation points are determined using the two neural networks.

[0015] Furthermore, determining the number and location of the interpolation points using the two neural networks includes:

[0016] The two neural networks include a first neural network and a second neural network. The poses of the start and end points and the error constraints are input into the first neural network, which outputs the number of interpolation points. The poses of the start and end points are input into the second neural network, which outputs the poses of the interpolation points. The first projection point on the straight path segment of the first interpolation point is taken as the start point. The second neural network is run again to calculate the position of the second interpolation point. The second projection point on the straight path segment of the second interpolation point is taken as the start point. The second neural network is run again to calculate the position of the next interpolation point. This process continues until the number of interpolation points of the second neural network is equal to the number of interpolation points output by the first neural network.

[0017] In a second aspect, the present invention provides a robotic arm trajectory planning device, the device comprising:

[0018] The acquisition unit is used to acquire the current position and working environment of the robotic arm;

[0019] The trajectory planning unit is used to obtain two planned motion trajectories of the robotic arm based on the current position and working environment of the robotic arm, namely a quick and smooth trajectory segment and a undisturbed trajectory segment; and to perform trajectory planning for the quick and smooth trajectory segment and the undisturbed trajectory segment respectively.

[0020] Thirdly, the present invention provides a robotic arm trajectory planning system, the system comprising:

[0021] The host computer is used to receive the position and orientation of the object provided by the vision system, convert the reference coordinate system from the vision system's world coordinate system to the robot's base coordinate system, plan the motion trajectory according to the robot's base coordinate system and the working environment, send the trajectory planning results to the motion platform and the robotic arm, control the linkage between the motion platform and the robotic arm, and control the grasping system based on the communication interface to perform grasping or sucking and releasing actions.

[0022] A motion platform includes a moving component and a supporting component. A robotic arm is mounted at the end of the moving component, which is used to perform horizontal movement and rotational movement with its axis perpendicular to the ground.

[0023] A robotic arm is used to adjust the actuator of the gripping system to approach the object in any posture. The robotic arm is connected to the end of the moving part of the motion platform. The actuator of the gripping system is installed at the end of the robotic arm, and the camera of the vision system is installed at the end of the robotic arm.

[0024] The vision system includes an RGBD camera at the end of the robotic arm and an RGB camera array placed around the environment, the RGB camera array being used to monitor the item in real time if it meets preset conditions;

[0025] The gripping system includes a PLC, a relay, a vacuum pump, and a suction cup. The vacuum pump is connected to the suction cup, and the PLC is used to control the start and stop of the vacuum pump to complete the suction and release of the item.

[0026] Fourthly, the present invention provides a robotic arm trajectory planning device, comprising:

[0027] processor;

[0028] Memory used to store processor-executable instructions;

[0029] The processor is configured to execute the method described in the first aspect or any embodiment of the first aspect.

[0030] Fifthly, the present invention provides a storage medium storing instructions that, when executed by a processor of a terminal, enable the terminal to perform the method described in the first aspect or any embodiment of the first aspect.

[0031] The present invention, by adopting the above technical solution, has at least the following beneficial effects:

[0032] This invention obtains the current position and working environment of a robotic arm; based on the current position and working environment, it obtains two planned motion trajectories for the robotic arm: a quick and smooth trajectory segment and a undisturbed trajectory segment; and it performs trajectory planning for both the quick and smooth trajectory segment and the undisturbed trajectory segment. This invention segments the trajectory according to different trajectory characteristics. Different trajectory planning strategies are adopted for the characteristics of each trajectory segment, resulting in a lower computational load and faster calculation speed for the system's motion planning algorithm while meeting usage requirements.

[0033] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a robotic arm motion trajectory planning diagram provided in an embodiment of this disclosure.

[0036] Figure 2 This is a flowchart illustrating a robotic arm trajectory planning method according to an exemplary embodiment.

[0037] Figure 3 This is a flowchart illustrating a robotic arm trajectory planning method according to an exemplary embodiment.

[0038] Figure 4 This is a flowchart illustrating a robotic arm trajectory planning method according to an exemplary embodiment.

[0039] Figure 5 This is a schematic diagram of the network 1 structure provided in an embodiment of this disclosure.

[0040] Figure 6 This is a schematic diagram of the network 2 structure provided in an embodiment of this disclosure.

[0041] Figure 7 This is a schematic diagram of trajectory interpolation points provided in an embodiment of this disclosure.

[0042] Figure 8 This is a flowchart illustrating the process of interpolation points provided in the embodiments of this disclosure.

[0043] Figure 9 This is a schematic diagram of a robotic arm trajectory planning system according to an exemplary embodiment.

[0044] Figure 10 This is a block diagram 100 of a robotic arm trajectory planning device according to an exemplary embodiment.

[0045] Figure 11 This is a block diagram 200 of a robotic arm trajectory planning system according to an exemplary embodiment.

[0046] Figure 12 This is a schematic diagram of the structure of an application embodiment of the electronic device of the present invention.

[0047] Figure 13 A schematic diagram of the structure of an embodiment of the electronic device disclosed herein is shown. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0049] The cleanroom for pharmaceutical manufacturing meets the cleanliness standards of GMP2010 Good Manufacturing Practice for Pharmaceuticals and GB50457 Cleanroom Design Standards for Pharmaceutical Industry. The current automation level of the high-speed filling line basically meets the requirements. However, there is still a certain probability of bottle failure during filling. The main failure modes include bottle tipping and breakage, which still require manual handling, necessitating 24-hour on-duty personnel. Pharmaceutical workshops have high cleanliness requirements, and operators need to wear protective clothing when entering and exiting, which is expensive. Furthermore, bottle failure is a rare, accidental event; wearing protective clothing manually is inconvenient, and in a ten-hour workday in a cleanroom, the effective working time may be less than one hour, resulting in low efficiency. Therefore, a cleanroom robotic maintenance system is proposed, which can replace human workers and achieve intelligent and automated identification and handling of failures on the production line.

[0050] Existing robotic grasping methods include using grippers and suction cups. Cleanroom filling lines are often long, with very low daily operating frequencies. Based on these characteristics, the grasping motion system adopts a single robotic arm plus a motion platform.

[0051] In existing technologies, a robotic arm trajectory planning method based on genetic algorithms involves determining the constraints of the robotic arm, including spatial position, velocity, and acceleration. Based on the genetic algorithm, the robotic arm's motion trajectory is planned according to the constraints and a preset optimization objective. This improves the robotic arm's working efficiency, and during operation, the entire robotic arm's trajectory, velocity, and acceleration curves are smooth, with smooth and continuous joint angles, angular velocities, and angular accelerations. This ensures stable, vibration-free rotation of the robotic arm's motors, preventing abrupt changes and extending the robotic arm's lifespan.

[0052] However, genetic algorithms are essentially heuristic algorithms. When used for trajectory planning in high-dimensional, redundant robotic systems, this method cannot provide real-time computational capabilities. Secondly, genetic algorithm-based robotic arm trajectory planning methods do not conduct in-depth analysis of trajectory characteristics. The working trajectory of a high-speed filling line intelligent processing system exhibits different characteristics at different stages. Depending on the environment and requirements, the trajectory characteristics of the robot's end effector vary.

[0053] This invention can be applied to scenarios including using robotic arms to grasp and pick up various objects. In unstructured environments, this invention can reduce the computing power requirements of edge computing systems, and its applications are not limited to pharmaceutical production lines.

[0054] The present invention uses particle swarm optimization and deep neural networks to plan and improve the trajectory of fast and smooth trajectory segments.

[0055] Figure 1 This is a robotic arm motion trajectory planning diagram provided in an embodiment of this disclosure. For example... Figure 1As shown, the section from point A to point B is a fast and smooth segment. This segment involves relatively unobstructed movement, high speed, and low precision requirements. The trajectory error of the robotic arm's end effector only needs to be within a certain range (±50mm), i.e., within the densely dotted lines in the diagram. An ideal trajectory is not required, i.e., the sparsely dotted lines between A and B in the diagram. The section from point B to point C is a undisturbed segment. The robotic arm approaches and picks up the object, requiring undisturbed trajectory planning. Precise control of the speed and posture of the robotic arm's end effector as it approaches the object is necessary here. Therefore, this segment is planned in Cartesian space to achieve precise control. For the fast and smooth trajectory segment from point A to point B, this segment involves a long-distance movement through a sparse obstacle space. While trajectory planning in joint space could significantly reduce computation, it would be impossible to predict the end effector's movement. As shown, the end effector trajectory may exceed the predicted range. Only the starting point, ending point, and intermediate interpolation points can be determined; it cannot guarantee that the robotic arm's end effector will move within the specified trajectory error range. If trajectory planning is performed in Cartesian space, it would require inverse kinematics (IK) for every point on the planned trajectory, approximately every 5 ms, resulting in a very high computational load. Although the motion accuracy is very high, with the actual trajectory differing from the planned trajectory by within ±0.05 mm, the accuracy required for the computational task far exceeds the needs of the engineering task, making it unnecessary. In summary, the requirements for trajectory planning in segment AB are: smooth and continuous motion, stable start and stop, and trajectory error within a certain range (±50 mm).

[0056] Please see Figure 2 , Figure 2 This is a flowchart illustrating a robotic arm trajectory planning method according to an exemplary embodiment, the method comprising the following steps:

[0057] Step S11: Obtain the current position and working environment of the robotic arm;

[0058] Step S12: Based on the current position of the robotic arm and the working environment, obtain two planned motion trajectories for the robotic arm, namely a fast and smooth trajectory segment and a undisturbed trajectory segment.

[0059] Step S13: Perform trajectory planning for the quick and smooth trajectory segment and the undisturbed trajectory segment respectively.

[0060] In this embodiment, a simulation model of the robotic arm and motion platform is established in the Robot Operating System (ROS). The physical model of the robotic arm is described using the DH matrix method, and a Universal Robot Description Format (URDF) file is written based on the actual situation. The URDF file can be used to display the model in rviz and can then be used for simulation in gazebo.

[0061] In this embodiment, joint space planning is used for the quick and smooth trajectory segment. In one embodiment, there are 5 interpolation points in the quick and smooth trajectory segment, including the start and end points of the robotic arm. Inverse kinematics is performed at these points to obtain the joint angles, and then a fifth-order polynomial is used to simulate the change curves of the joint angles. For the undisturbed trajectory planning segment, Cartesian space trajectory planning is used. This requires providing relatively dense positions and corresponding velocities, and inverse kinematics is needed for each point to calculate how the joint angles should change. The entire motion planning is implemented based on the ROS system.

[0062] In this embodiment, to meet the operational requirements of the first fast and smooth trajectory segment, the robotic arm moves away from sparse obstacles during motion. The minimum number of critical path points is planned on an ideal trajectory in Cartesian space. Trajectory planning is performed in joint space between every two critical path points. A particle swarm optimization algorithm is used to determine the number and insertion positions of interpolation points.

[0063] In this embodiment, trajectory planning can be divided into two types: trajectory planning in joint space and trajectory planning in Cartesian space. When using joint space trajectory planning, if the robot arm's pose is known to be in Cartesian space, inverse kinematics is required to convert the end effector path points into joint path points for each joint. Then, a suitable function is fitted based on constraints such as joint angles, velocities, and acceleration constraints. This function describes the trajectory of the robot arm joints from the starting point through intermediate path points to the target point. Cartesian space trajectory planning uses the Cartesian spatial positions of the starting and ending points to interpolate in space, thus obtaining the trajectory of the robot arm's end effector. The inverse kinematics is then used to obtain the angles corresponding to each joint, completing the trajectory planning. The advantages of joint space trajectory planning are low computational cost and wide applicability, but the resulting trajectory is not intuitive. Cartesian space trajectory planning offers intuitive trajectory, high accuracy, and good portability, but requires a high number of inverse kinematics calculations, resulting in excessive computation and limited applicability. Cartesian space trajectory planning is used when there are specific requirements for the accuracy of the robot arm's end effector trajectory, such as in painting robots and arc welding robots. Joint space planning only requires solving for the polynomial trajectory coefficients. Cartesian space planning uses inverse kinematics operations for the robot.

[0064] Please see Figure 3 , Figure 3 This is a flowchart illustrating a robotic arm trajectory planning method according to an exemplary embodiment, the method comprising the following steps:

[0065] Step S21: Use the particle swarm optimization algorithm to determine the number and location of interpolation points in the fast smooth trajectory segment. The interpolation points include the start and end points of the robotic arm.

[0066] Step S22: Perform inverse kinematics operation on the interpolation points to obtain a preset number of joint angle vectors;

[0067] Step S23: Perform joint space trajectory planning between joint angle vectors.

[0068] In this embodiment, a small number of interpolation points, such as five, are appropriately selected on the fast and smooth trajectory segment using a particle swarm optimization algorithm. Inverse kinematics is performed on the five Cartesian space critical path points to obtain five sets of joint angle vectors. Then, joint space trajectory planning is performed between the five sets of joint vectors. In summary, trajectory planning is performed in joint space after inserting critical path points in Cartesian space for inverse kinematics. The trajectory function is determined from the identified critical path points. A fifth-order polynomial not only ensures the continuity of position, velocity, and acceleration but also minimizes jitter.

[0069] Furthermore, in fifth-degree polynomial form: q(t) represents the value of the polynomial at a distance of (t-t0) from time t0, q(t) = a0 + a1(t-t0) + a2(t-t0). 2 +a3(t-t0) 3 +a4(t-t0) 4 +a5(t-t0) 5

[0070] Conditions for the starting and ending points:

[0071] q(t0)=q0, q(t1)=q1

[0072]

[0073]

[0074] Here we define T = (t - t0) and h = q1 - q0, and obtain the polynomial coefficients as follows:

[0075] a0 = q0

[0076] a1 = v0

[0077]

[0078] This method solves for a fifth-degree polynomial using 14 additions and 33 multiplications, yielding 6 coefficients. Compared to inverse kinematics, this method significantly reduces the computational load.

[0079] Inverse kinematics is used to determine the joint angles given the end effector position and orientation of a robotic arm and its link parameters. Its purpose is to transform the motion assigned to the end effector in Cartesian space into the corresponding joint space motion, enabling the desired motion to be executed. Inverse kinematics is complex because the equations to be solved are usually nonlinear, thus not always finding a valid solution; multiple solutions may exist; there may be infinitely many solutions (in cases where the robotic arm has kinematic redundancy); and from the perspective of the robotic arm's kinematic structure, there may be no feasible solution.

[0080] In one embodiment, the particle swarm optimization algorithm is implemented as follows: each particle can be considered as a search entity in a 24-dimensional search space (corresponding to angles, angular velocities, and angular accelerations along eight axes). Obtaining joint angles and link parameters to calculate the end effector position and orientation of the robotic arm constitutes the forward kinematics. The forward kinematics equations are represented by the homogeneous transformation matrix as follows:

[0081]

[0082] Where q is an (n×1) joint variable vector, n e s e a e p is a unit vector in the coordinate system fixed to the end effector. e The origin of this coordinate system is relative to the base coordinate system O. b -x b ybz b The position vector of the origin.

[0083] In one embodiment, the 8-axis robot system has a total of 10 homogeneous transformation matrices, including one end effector transformation matrix, eight motion axis transformation matrices, and one base transformation matrix relative to the world coordinate system. Solving each DH matrix requires 6 multiplication operations and 4 trigonometric function calculations. The 8 motion axes have 8 varying DH matrices, requiring 48 multiplication operations and 32 trigonometric function calculations.

[0084] In one embodiment, the pose of the end effector relative to the base coordinate system is obtained by matrix multiplication of nine DH matrices. ee =A0·A1·A2·A3·A4·A5·A6·A7·A8·A9 requires a total of 39 additions, 84 multiplications, and 14 trigonometric function calculations. This yields the final pose of an example. Substituting this result into the optimization objective function, we obtain the alternative solution for this particle. When minimizing the pose error is the optimization objective, it requires 39 additions, 132 multiplications, and 71 trigonometric function calculations. Each time, the particle swarm optimization algorithm is used to solve for a single target point; the number of particles, denoted as k, is generally no less than 2000.

[0085] In one embodiment, each iteration requires updating the particle velocity of all particles (this explanation only pertains to the robot's pose variables). Each particle requires 4 additions, 5 multiplications, and 2 random number calculations. (See reference)

[0086]

[0087] In one embodiment, k particles require 4k additions, 5k multiplications, and 2k random number calculations. For a single target point, the particle swarm optimization algorithm typically requires at least 30 iterations to select the optimal solution. A total of 120k additions, 150k multiplications, and 60k random number calculations are needed (K>=2000). When k is 2000, a total of 240,000 additions, 300,000 multiplications, and 120,000 random number calculations are required.

[0088] In one embodiment, the coordinate transformation formula for the position and orientation of coordinate system n relative to coordinate system 0 is as follows:

[0089]

[0090] The forward kinematics equations are calculated recursively, and a systematic approach is used through homogeneous transformation matrices. It is obtained by simple multiplication. Each homogeneous transformation matrix is ​​a function of a single joint variable.

[0091] The current position of a particle represents a candidate solution to the corresponding optimization problem, and the particle's movement is its search process. The particle's speed can be dynamically adjusted based on its historical best position and the population's historical best position. Each particle has only two attributes: speed and position. Speed ​​represents the rate of movement, and position represents the direction of movement. The optimal solution searched by each particle individually is called its individual extreme value, and the optimal individual extreme value in the particle swarm is taken as the current global optimal solution. This process iterates continuously, updating speed and position. Finally, the optimal solution satisfying the termination condition is obtained, which is the optimal angles, angular velocities, and angular accelerations along the eight axes.

[0092] The velocity update formula for the i-th particle at position d is as follows:

[0093]

[0094] Where i represents the i-th particle in the particle swarm, t represents the t-th iteration of the algorithm, and w is the inertia weight.

[0095] The formula for updating the position at position d is: previous position + next speed.

[0096]

[0097] In this embodiment of the disclosure, the robotic arm moves away from sparse obstacles during the movement of the fast and smooth trajectory segment, and the movement is continuous and smooth, the start and stop of the movement are stable, and the trajectory error is within a preset range.

[0098] In this embodiment of the disclosure, trajectory planning is performed on the undisturbed trajectory segment using trajectory planning in Cartesian space.

[0099] Please see Figure 4 , Figure 4 This is a flowchart illustrating a robotic arm trajectory planning method according to an exemplary embodiment, the method comprising the following steps:

[0100] Step S31: Use the particle swarm optimization algorithm to obtain the optimal solution and create a dataset;

[0101] In this embodiment, any fast and smooth trajectory segment can be decomposed into multiple straight-line trajectories, thus the training of straight-line trajectories is universal. A particle swarm optimization algorithm is used to provide the optimal solution, a dataset is created, and two neural networks are trained. The neural networks are then used to quickly calculate the number and position of interpolation points.

[0102] Step S32: Train two neural networks based on the dataset;

[0103] In this embodiment of the disclosure, Figure 5 This is a schematic diagram of the network 1 structure provided in an embodiment of this disclosure. For example... Figure 5 As shown, in Network 1, the number of interpolation points required in the first segment is calculated. Too many interpolation points increase the computational load, while too few interpolation points cause the trajectory at the end to deviate from the straight line segment. Therefore, an "interpolation point count network" is designed and trained. The input includes the poses of the start and end points, as well as error constraints. The network has three hidden layers, each containing 25 neurons. The output is the number of interpolation points, and the last layer is a softmax layer.

[0104] In this embodiment of the disclosure, Figure 6 This is a schematic diagram of the network 2 structure provided in an embodiment of this disclosure. For example... Figure 6 As shown, in Network 2, after determining the number of interpolation points, it is necessary to determine the poses of the interpolation points. Therefore, the downstream network "Interpolation Point Pose Network" is designed and trained. The input includes the poses of the current point and the endpoint. The network has three hidden layers, each containing 30 neurons. The output is the pose of the interpolation point closest to the current point on the trajectory from the current point to the endpoint. Figure 7 This is a schematic diagram of trajectory interpolation points provided in an embodiment of this disclosure. For example... Figure 7As shown, starting from the projection point A' on the straight path segment from the first interpolation point A, the network is run again to calculate the position of the next interpolation point B. Then, starting from the projection point B' on the straight path segment from interpolation point B, the network is run again to calculate the position of the next interpolation point. This process continues until the total number of interpolation points equals the number of insertion points output by the "interpolation point count network".

[0105] Step S33: Use the two neural networks to determine the number and location of the interpolation points.

[0106] In this embodiment, Network 1 requires 1775 additions and 1775 multiplications. Network 2 requires 2340 additions and 2340 multiplications. In one embodiment, a trajectory has 5 key points. Using the particle swarm optimization algorithm, 1,200,000 additions, 1,500,000 multiplications, and 600,000 random number calculations are required. Using a deep neural network method, Network 1 needs to perform the calculation once, and Network 2 needs to perform the calculation five times, requiring a total of 13,475 additions and 13,475 multiplications. It is evident that the method proposed in this invention has a significantly lower computational load than existing methods.

[0107] In this embodiment, the network training method employs supervised learning. The number of interpolation points and their poses obtained through particle swarm optimization are used as samples to train the "interpolation point count network" and the "interpolation point pose network." Since the sample labels are automatically calculated, the sample size is set to 3000 trajectories.

[0108] Figure 8 This is a flowchart illustrating the interpolation point process provided in the embodiments of this disclosure, such as... Figure 8 The diagram illustrates the workflow of using a network to solve for interpolation points, where P represents the existing number of interpolation points and C represents the number of interpolation points obtained by network 1. The network obtains the number of interpolation points C from network 1 and the interpolation point poses from network 2. If the existing number of interpolation points is less than the obtained number, the interpolation point poses are obtained again from network 2; if the existing number of interpolation points is greater than or equal to the obtained number, the network solves for the interpolation points.

[0109] Through the solution of the present invention, the current position and working environment of a robotic arm are obtained; according to the current position and working environment of the robotic arm, planned motion trajectories of the robotic arm divided into two segments are obtained, which are a fast and smooth trajectory segment and a disturbance-free trajectory segment respectively; trajectory planning is performed on the fast and smooth trajectory segment and the disturbance-free trajectory segment respectively. According to different trajectory characteristics, the present invention divides the trajectory into segments. According to the characteristics of each trajectory segment, different trajectory planning strategies are adopted, so that under the condition of meeting usage requirements, the calculation amount of the system motion planning algorithm is lower and the calculation speed is faster. The method proposed by the present invention inserts a small number of key trajectory points in the Cartesian space, and planning is performed in the joint space between interpolation points. Through the present invention, rapid calculation of a trajectory with controllable trajectory error is realized, and the calculation speed is improved by at least 89 times.

[0110] In the present invention, the particle swarm optimization algorithm is used to independently find the optimal solution, that is, the optimal scheme for the number and positions of intermediate interpolation points. Then a deep neural network is used to replace the particle swarm optimization algorithm, so as to realize the function of rapidly calculating the number and positions of interpolation points.

[0111] Figure 9 is a schematic diagram of a robotic arm trajectory planning system shown according to an exemplary embodiment. As Figure 9 shown:

[0112] The functions of the upper computer are as follows: first, receiving the position and posture of an article provided by a vision system, and converting the reference coordinate system from the world coordinate system of the vision system to the robot base coordinate system; second, planning a motion trajectory according to the current position and working environment of the robot; third, sending the trajectory planning result to the motion platform and the robotic arm for controlling the linkage of the motion platform and the robotic arm; fourth, controlling the grasping system based on a communication interface to execute grasping / sucking and releasing actions at an appropriate time.

[0113] The motion platform comprises a moving component and a supporting component. The robotic arm is mounted at the end of the moving component, and the moving component can complete horizontal movement and rotation motion with a rotation axis perpendicular to the ground.

[0114] The robotic arm has 6 degrees of motion freedom in Cartesian space, and can adjust the actuator of the grasping system to grasp articles in any posture. The robotic arm is connected to the end of the moving component of the motion platform, the actuator of the grasping system is mounted at the end of the robotic arm, and the camera of the vision system is mounted at the end of the robotic arm.

[0115] The vision system comprises two types: 1. An RGBD camera at the end effector of the robotic arm; 2. An array of RGB cameras placed around the environment. The RGB camera array monitors high-movement objects in real time, while the environmental RGB camera array detects the approximate position and orientation of the objects, transmitting this information to the robotic arm. The robotic arm receives the orientation information of the objects and moves to a position near them. The camera at the end effector of the robotic arm then precisely identifies the position and orientation of the objects.

[0116] The gripping system includes a PLC, relays, a vacuum pump, and suction cups. The vacuum pump is connected to the suction cups. The PLC receives action commands from the host computer and controls the vacuum pump to start and stop, completing the gripping and release of items.

[0117] Figure 10 This is a block diagram 100 illustrating a robotic arm trajectory planning device according to an exemplary embodiment. (Refer to...) Figure 10 The device includes an acquisition unit 101 and a trajectory planning unit 102.

[0118] Acquisition unit 101 is used to acquire the current position and working environment of the robotic arm;

[0119] The trajectory planning unit 102 is used to obtain two planned motion trajectories of the robotic arm based on the current position of the robotic arm and the working environment, namely a quick and smooth trajectory segment and a undisturbed trajectory segment; and to perform trajectory planning for the quick and smooth trajectory segment and the undisturbed trajectory segment respectively.

[0120] Figure 11 This is a block diagram 200 illustrating a robotic arm trajectory planning system according to an exemplary embodiment. (Refer to...) Figure 11 The device includes a host computer 201, a motion platform 202, a robotic arm 203, a vision system 204, and a gripping system 205.

[0121] The host computer 201 is used to receive the position and orientation of the object provided by the vision system, convert the reference coordinate system from the vision system world coordinate system to the robot base coordinate system, plan the motion trajectory according to the robot base coordinate system and the working environment, send the trajectory planning result to the motion platform and the robotic arm, control the linkage of the motion platform and the robotic arm, and control the grasping system based on the communication interface to perform grasping or sucking and releasing actions.

[0122] The motion platform 202 includes a motion component and a support component. A robotic arm is mounted at the end of the motion component, which is used to perform horizontal movement and rotational movement with its axis perpendicular to the ground.

[0123] The robotic arm 203 is used to adjust the actuator of the gripping system to approach the object in any posture. The robotic arm is connected to the end of the moving part of the motion platform. The actuator of the gripping system is installed at the end of the robotic arm, and the camera of the vision system is installed at the end of the robotic arm.

[0124] The vision system 204 includes an RGBD camera at the end of the robotic arm and an RGB camera array placed around the environment, the RGB camera array being used to monitor the item in real time if it meets preset conditions;

[0125] The gripping system 205 includes a PLC, a relay, a vacuum pump, and a suction cup. The vacuum pump is connected to the suction cup, and the PLC is used to control the start and stop of the vacuum pump to complete the suction and release of the item.

[0126] Figure 12 This is a schematic diagram of the structure of an application embodiment of the electronic device of the present invention. Refer to the following... Figure 12 It illustrates a structural schematic diagram of an electronic device suitable for implementing embodiments of the present invention, such as a terminal device or server. Figure 12 As shown, the electronic device includes a memory for storing computer programs and one or more processors for executing the computer programs stored in the memory. In one example, the memory may be read-only memory (ROM) and / or random access memory (RAM).

[0127] In one example, one or more processors may be one or more central processing units (CPUs) and / or one or more graphics processing units (GPUs), etc. The processors can perform various appropriate actions and processes according to executable instructions stored in ROM or executable instructions loaded from storage into RAM. In one example, the electronic device may also include a communication unit, which may include, but is not limited to, a network interface card (NIC), which may include, but is not limited to, an Infiniband (IB) NIC. The processor can communicate with ROM and / or RAM to execute executable instructions, connect to the communication unit via a bus, and communicate with other target devices through the communication unit, thereby completing the operation corresponding to any method provided in the embodiments of the present invention.

[0128] In addition, the RAM can store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. With RAM present, the ROM is an optional module. The RAM stores executable instructions, or executable instructions are written to the ROM during runtime. These executable instructions cause the processor to perform operations corresponding to any of the methods described above. Input / output (I / O) interfaces are also connected to the bus. The communication unit can be integrated or configured with multiple sub-modules (e.g., multiple IB network cards) linked on the bus.

[0129] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard disks; and communication sections including network interface cards such as LAN cards and modems. The communication sections perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0130] It needs to be explained, such as Figure 12 The architecture shown is only one optional implementation. In practice, the above can be modified according to actual needs. Figure 12 The number and type of components can be selected, deleted, added, or replaced; different functional components can also be implemented by separate or integrated configurations. For example, the GPU and CPU can be set separately or the GPU can be integrated on the CPU; the communication unit can be set separately or integrated on the CPU or GPU, and so on. All these alternative implementation methods fall within the protection scope disclosed in this invention.

[0131] Figure 13 A schematic diagram of the structure of an embodiment of the electronic device of this disclosure is shown. Reference is made below. Figure 13 It illustrates a structural schematic diagram of an electronic device suitable for implementing embodiments of the present invention, such as a terminal device or server. Figure 13 As shown, the electronic device includes a processor and a memory. The electronic device may also include input / output devices. Both the memory and the input / output devices are connected to the processor via a bus. The memory stores instructions executed by the processor; the processor calls the instructions stored in the memory and executes the robotic arm trajectory planning method described in the above embodiment.

[0132] This disclosure also provides a computer-readable storage medium storing computer-executable instructions, which, when executed on a computer, perform a robotic arm trajectory planning method described in the above embodiments.

[0133] This disclosure also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute a robotic arm trajectory planning method involved in the above embodiments.

[0134] In one or more alternative embodiments, this disclosure also provides a computer-readable storage medium for storing computer-readable instructions that, when executed, cause a computer to perform a robotic arm trajectory planning method in any of the possible implementations described above. In another alternative example, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0135] Although the operations are described in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the operations shown to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0136] The methods and apparatus disclosed herein can be implemented using standard programming techniques, utilizing rule-based logic or other logic to implement various method steps. It should also be noted that the terms "apparatus" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0137] Any step, operation, or procedure described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product comprising a computer-readable medium containing computer program code, which is executable by a computer processor to perform any or all of the described steps, operations, or procedures.

[0138] The foregoing description of embodiments of this disclosure has been provided for purposes of illustration and description. The foregoing description is not exhaustive and is not intended to limit this disclosure to the exact form disclosed; various modifications and variations may be made in accordance with the foregoing teachings, or may be derived from practice of this disclosure. These embodiments were chosen and described to illustrate the principles of this disclosure and its practical application, enabling those skilled in the art to utilize this disclosure in various implementations and modifications suitable for the particular purpose conceived.

[0139] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0140] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0141] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0142] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0143] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0144] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0145] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0146] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" or "more" means at least two.

[0147] It should be understood that when an element is referred to as "fixed to" or "set on" another element, it may be directly on the other element or may have an intervening element present at the same time; when an element is referred to as "connected to" another element, it may be directly connected to the other element or may have an intervening element present at the same time. In addition, the term "connected" as used herein may include wireless connections; the word "and / or" as used includes any unit and all combinations of one or more of the associated listed items.

[0148] Any process or method description in the flowchart or otherwise herein can be understood as: representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0149] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0150] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0151] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0152] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0153] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0154] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for planning the trajectory of a robotic arm, characterized in that, The method includes: The current position and working environment of the robotic arm are obtained; the robotic arm is mounted on the end of the moving part of the motion platform. Based on the current position and working environment of the robotic arm, two planned motion trajectories of the robotic arm are obtained, namely a fast and smooth trajectory segment and a undisturbed trajectory segment. Trajectory planning is performed on the fast and smooth trajectory segment and the undisturbed trajectory segment respectively; The trajectory planning for the fast and smooth trajectory segment includes: The particle swarm optimization algorithm is used to determine the number and position of interpolation points in the fast and smooth trajectory segment. The interpolation points include the start and end points of the robotic arm. Inverse kinematics is performed on the interpolation points to obtain a preset number of joint angle vectors. Joint space trajectory planning is performed between the joint angle vectors. The step of using particle swarm optimization (PSO) to determine the number and location of interpolation points in the fast smooth trajectory segment includes: obtaining the optimal solution using PSO and creating a dataset; training two neural networks based on the dataset; and using the two neural networks to determine the number and location of the interpolation points. The step of determining the number and location of the interpolation points using the two neural networks includes: The two neural networks include a first neural network and a second neural network. The poses of the start and end points and the error constraints are input into the first neural network, which outputs the number of interpolation points. The poses of the start and end points are input into the second neural network, which outputs the poses of the interpolation points. The first projection point on the straight path segment of the first interpolation point is taken as the start point. The second neural network is run again to calculate the position of the second interpolation point. The second projection point on the straight path segment of the second interpolation point is taken as the start point. The second neural network is run again to calculate the position of the next interpolation point. This process continues until the number of interpolation points in the second neural network is equal to the number of interpolation points output by the first neural network.

2. The method according to claim 1, characterized in that, During the movement of the robotic arm in the fast and smooth trajectory segment, it stays away from sparse obstacles, and the movement is smooth and continuous, with stable start and stop, and the trajectory error is within a preset range.

3. The method according to claim 1, characterized in that, The trajectory planning for the undisturbed trajectory segment is performed using trajectory planning in Cartesian space.

4. A robotic arm trajectory planning device, characterized in that, The device includes: An acquisition unit is used to acquire the current position and working environment of the robotic arm; the robotic arm is mounted on the end of the moving part of the motion platform; The trajectory planning unit is used to obtain two planned motion trajectories of the robotic arm based on the current position and working environment of the robotic arm, namely a quick and smooth trajectory segment and a undisturbed trajectory segment; and to perform trajectory planning for the quick and smooth trajectory segment and the undisturbed trajectory segment respectively. The trajectory planning for the fast and smooth trajectory segment includes: The particle swarm optimization algorithm is used to determine the number and position of interpolation points in the fast and smooth trajectory segment. The interpolation points include the start and end points of the robotic arm. Inverse kinematics is performed on the interpolation points to obtain a preset number of joint angle vectors. Joint space trajectory planning is performed between the joint angle vectors. The step of using particle swarm optimization (PSO) to determine the number and location of interpolation points in the fast smooth trajectory segment includes: obtaining the optimal solution using PSO and creating a dataset; training two neural networks based on the dataset; and using the two neural networks to determine the number and location of the interpolation points. The step of determining the number and location of the interpolation points using the two neural networks includes: The two neural networks include a first neural network and a second neural network. The poses of the start and end points and the error constraints are input into the first neural network, which outputs the number of interpolation points. The poses of the start and end points are input into the second neural network, which outputs the poses of the interpolation points. The first projection point on the straight path segment of the first interpolation point is taken as the start point. The second neural network is run again to calculate the position of the second interpolation point. The second projection point on the straight path segment of the second interpolation point is taken as the start point. The second neural network is run again to calculate the position of the next interpolation point. This process continues until the number of interpolation points of the second neural network is equal to the number of interpolation points output by the first neural network.

5. A robotic arm trajectory planning system, characterized in that, The system for implementing the method according to any one of claims 1 to 3, the system comprising: The host computer is used to receive the position and orientation of the object provided by the vision system, convert the reference coordinate system from the vision system's world coordinate system to the robot's base coordinate system, plan the motion trajectory according to the robot's base coordinate system and the working environment, send the trajectory planning results to the motion platform and the robotic arm, control the linkage between the motion platform and the robotic arm, and control the grasping system based on the communication interface to perform grasping or sucking and releasing actions. A motion platform includes a moving component and a supporting component. A robotic arm is mounted at the end of the moving component, which is used to perform horizontal movement and rotational movement with its axis perpendicular to the ground. A robotic arm is used to adjust the actuator of the gripping system to approach the object in any posture. The robotic arm is connected to the end of the moving part of the motion platform. The actuator of the gripping system is installed at the end of the robotic arm, and the camera of the vision system is installed at the end of the robotic arm. The vision system includes an RGBD camera at the end of the robotic arm and an RGB camera array placed around the environment, the RGB camera array being used to monitor the item in real time if it meets preset conditions; The gripping system includes a PLC, a relay, a vacuum pump, and a suction cup. The vacuum pump is connected to the suction cup, and the PLC is used to control the start and stop of the vacuum pump to complete the suction and release of the item.

6. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method described in any one of claims 1 to 3.

7. A storage medium, characterized in that, The storage medium stores a computer program that can be called by a computer, which, when the computer is running, can execute the method described in any one of claims 1 to 3.

Images