A method and system for robotic arm motion planning based on recurrent neural networks

Abstract

A method and system for robot motion planning based on recurrent neural networks (RNNs) are disclosed. The method includes: acquiring a basic environmental point cloud during robot operation; adjusting the basic environmental point cloud in a time series based on the current joint angle and the target joint angle, obtaining a series of environmental point clouds with time indices; establishing a recurrent neural network model to output a series of joint angles with time indices; generating a series of joint positions with time indices for each joint in the robot's base coordinate system from the series of joint angles with time indices; calculating the planning error; adjusting the weights of the recurrent neural network model using a backpropagation algorithm, iteratively reducing the planning error until the iteration ends when the planning error is below a set threshold; and the joint angles with time indices output by the recurrent neural network model are the joint trajectories of the robot. This invention improves the robot's motion planning capability.

Description

Technical Field

[0001] This invention relates to the field of intelligent robots and provides a method and system for planning the motion of a robotic arm based on a recurrent neural network. Based on the current joint position and target joint position of the robotic arm, and combined with the point cloud of the surrounding environment, a smooth, obstacle-avoidable trajectory oriented towards the operation is generated. Background Technology

[0002] Robotic arm motion planning refers to the process by which a robotic arm generates a feasible trajectory from a starting position to an ending position. This has always been a widely concerned issue in the operation of intelligent robots.

[0003] Currently, mainstream robotic arm motion planning methods are mainly divided into sampling-based motion planning and learning-based motion planning methods. Sampling-based motion planning methods, such as RRT (Rapidly-exploring random trees: A new tool for path planning, 1998), sample coordinate points in the robotic arm's workspace, determine whether these points are feasible, and then connect the feasible coordinate points to form the robotic arm's motion trajectory. In recent years, methods such as RRT-Connect and RRT* have improved sampling speed and smoothed the sampled motion trajectories. However, because these sampling-based motion planning methods lack a clear optimization objective, they often sample redundant and unstable motion trajectories.

[0004] Learning-based motion planning methods, such as Motion Planning Networks (Motion planning networks: Bridging the gap between learning-based and classical motionplanners, 2020), build neural network models to encode obstacle information, the current state, and the target state, and then predict the next state. These learning-based methods require collecting a large amount of additional environmental or trajectory information, setting optimization objectives, and training the model. Furthermore, in operational tasks, the objective is both an obstacle to be avoided and an object that can be approached and touched, making it difficult for learning-based motion planning methods to generate suitable optimization objectives.

[0005] Currently, there is still a lack of a method for generating smooth, obstacle-avoiding, and operation-oriented trajectories without requiring additional training data for motion planning of robotic arms. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention proposes a robotic arm motion planning method and system based on recurrent neural networks, which helps the robotic arm generate a smooth, obstacle-avoiding, and reliable motion trajectory during operation tasks.

[0007] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0008] The first aspect of this invention relates to a robotic arm motion planning method based on a recurrent neural network, comprising:

[0009] S1. Acquire basic environmental point cloud during robot operation, and calculate the current end position and target end position based on the current joint angle and the target joint angle;

[0010] S2. Based on the current end position and the target end position, adjust the basic environmental point cloud in time series to obtain a series of environmental point clouds with time sequence numbers;

[0011] S3. Establish a recurrent neural network model. The input of the recurrent neural network model is the current joint angle and the target joint angle of the robotic arm. The output of the recurrent neural network model is a series of joint angles with time sequence numbers.

[0012] S4. Using the positive kinematics of the robotic arm, generate a series of joint positions with time sequence numbers in the robotic arm base coordinate system from the series of joint angles with time sequence numbers.

[0013] S5. Calculate the planning error based on the series of joint positions with time sequence numbers, the joint angles with time sequence numbers, and the environmental point cloud with time sequence numbers;

[0014] S6. Based on the planning error, the weights of the recurrent neural network model are adjusted using the backpropagation algorithm, and the planning error is continuously reduced iteratively. The iteration ends when the planning error is lower than a set threshold.

[0015] S7. After the iteration is completed, the joint angles with time sequence numbers output by the recurrent neural network model are the joint trajectories of the robotic arm.

[0016] Furthermore, during robot operation, a basic environmental point cloud is acquired, and the current end-effector position and target end-effector position are calculated based on the current joint angle and the target joint angle, including:

[0017] Point clouds are obtained by using color and depth images from depth cameras on the robot, and then the point clouds are transformed into the robot arm's base coordinate system to obtain the basic environment point cloud.

[0018] Generally, the point cloud obtained by the depth camera on the robot is a point cloud in the camera coordinate system. Based on the camera mounting position and the robotic arm mounting position on the robot, a transformation matrix can be obtained from the camera coordinate system to the robotic arm base coordinate system. Using this transformation matrix, the point cloud in the camera coordinate system can be converted into the aforementioned basic environment point cloud e. base ;

[0019] The current joint angle and the target joint angle represent the start and end states of the motion trajectory, respectively, and are consistent with the number of joints in the robotic arm.

[0020] Generally, for a robotic arm with k joints, the current joint angle can be expressed as: The target joint angle is

[0021] The current end effector position can be calculated based on the current joint angle using the positive kinematics of the robotic arm.

[0022] Generally, for a robotic arm with k joints, the position of each joint can be calculated using the forward kinematics of the robotic arm, based on the current joint angle, as follows: The position of the last joint. That is, the current end position;

[0023] The position of the target end can be calculated based on the target joint angle using the positive kinematics of the robotic arm.

[0024] Generally, for a robotic arm with k joints, the position of each joint can be calculated from the target joint angles using the forward kinematics of the robotic arm, as follows: The position of the last joint. That is, the target end position.

[0025] Furthermore, based on the current end position and the target end position, the base environmental point cloud is adjusted in time series to obtain a series of environmental point clouds with time sequence numbers, including:

[0026] The series of environmental point clouds with time sequence numbers has T n Each time sequence number corresponds to an independent environmental point cloud. T =[e 0 ,e 1 ,…,e i ,…,e j ,…,e n-1 ,e n ];

[0027] In the initial stage T0—T iIn the environmental point cloud within the time sequence, eliminate the point cloud near the current end position in the basic environmental point cloud.

[0028] T in the intermediate stage i —T j The environmental point cloud within the time sequence number directly uses the aforementioned basic environmental point cloud, e i ~e j =e base ;

[0029] T at the end of the phase j —T n In the environmental point cloud within the time sequence, eliminate the point cloud near the target end position in the basic environmental point cloud.

[0030] By stitching together the environmental point clouds of the start stage, the middle stage, and the end stage, a series of environmental point clouds with time sequence numbers are obtained.

[0031] Furthermore, a recurrent neural network model is established. The input of the recurrent neural network model is the current joint angle and the target joint angle of the robotic arm. The output of the recurrent neural network model is a series of joint angles with time indices, including:

[0032] If the robotic arm has k joints, then the current joint angle and the target joint angle form a 1×k vector, i.e. and

[0033] If the series of joint angles with time sequence numbers has T n Each time sequence number corresponds to an independent joint angle, and the series of joint angles with time sequences constitutes T. n A vector of length ×k can be represented as:

[0034]

[0035] The superscript indicates the time sequence number, and the subscript indicates the joint sequence number.

[0036] Furthermore, utilizing the positive kinematics of the robotic arm, a series of joint positions with time sequences are generated in the robotic arm's base coordinate system from the series of joint angles with time sequences, including:

[0037] Based on different robotic arm structures, the forward kinematic equation FK can be constructed;

[0038] The T-shape is formed by the series of joint angles W with time serial numbers. nThe ×k vector is input into the forward kinematics equations to calculate a series of joint positions with time indices in the base coordinate system of the robotic arm. These joint positions P, with time indices, constitute T. n A vector of length ×k×3, P=FK(W), can be represented as:

[0039]

[0040] Furthermore, based on the series of time-sequenced joint positions, the time-sequenced joint angles, and the time-sequenced environmental point cloud, the planning error is calculated, including:

[0041] Linear interpolation is performed on a series of joint positions P with time indices for adjacent joints to obtain a series of continuous joint positions P with time indices. Linear ;

[0042] Generally, if m linear interpolations are performed for each adjacent joint, then P Linear Composition of T n A vector of length ×((k-1)m+1)×3 has:

[0043]

[0044] Calculate the distance between the series of consecutive joint positions with time sequence numbers and the environmental point cloud with time sequence numbers, as the obstacle avoidance error;

[0045] Specifically, the environmental point cloud e at time i is... i The continuous joint positions at time i. For all points in the network, find the distance between consecutive joint positions at all time points and the closest point in the environmental point cloud at the corresponding time point. This distance is the obstacle avoidance error.

[0046]

[0047] The difference between adjacent joint angles with time numbers is calculated as a smoothing error;

[0048] Specifically, W i Let i represent the joint angle at time i. Then the smoothing error is:

[0049]

[0050] Calculate the difference between the first and last set of joint angles with time sequence numbers and the current and target joint angles of the robotic arm, as the angle error;

[0051] Specifically, W 0 and Wn These are the angles of the first group of joints and the angles of the last group of joints, respectively. and The difference between the current joint angle and the target joint angle is calculated using the mean square error (MSE) function. The angle error is then:

[0052]

[0053] The obstacle avoidance error, smoothing error, and angle error are added together to obtain the planning error, which is L. 规划 =L 避障 +L 平滑 +L 角度 .

[0054] Furthermore, based on the planning error, the weights of the recurrent neural network model are adjusted using the backpropagation algorithm to iteratively reduce the planning error. The iteration ends when the planning error is lower than a set threshold, including:

[0055] Each time the planning error is calculated, the weights of the recurrent neural network model are adjusted using the backpropagation algorithm and the gradient descent algorithm. After adjusting the weights, the recurrent neural network model is used again to infer and calculate a new planning error. This process is iterated.

[0056] Specifically, recurrent neural networks are represented by RNN, planning error is represented by Loss, and backpropagation and gradient descent are represented by BP. Then:

[0057]

[0058] L 规划 =Loss(FK(W)), (8)

[0059] RNN new =BP(L 规划 ), (9)

[0060] Iterate through the above three formulas to update the weights of the RNN;

[0061] Set obstacle avoidance threshold Q 避障 Smoothing threshold Q 平滑 Angle threshold Q 角度 When the obstacle avoidance error L in the planning error 避障 Smoothing error L 平滑 Angular error L 角度 The iteration ends when the values ​​fall below the obstacle avoidance threshold, smoothing threshold, and angle threshold, respectively.

[0062] After the iteration is completed, the joint angles with time sequence numbers output by the recurrent neural network model are the joint trajectories of the robotic arm.

[0063] A second aspect of the present invention relates to a robotic arm motion planning system based on a recurrent neural network, comprising:

[0064] The end-effector position calculation module is used to acquire the basic environmental point cloud during robot operation and calculate the current end-effector position and target end-effector position based on the current joint angle and the target joint angle.

[0065] The environmental point cloud acquisition module adjusts the basic environmental point cloud in time series based on the current end position and the target end position to obtain a series of environmental point clouds with time sequence numbers;

[0066] A recurrent neural network module is used to build a recurrent neural network model. The input of the recurrent neural network model is the current joint angle and the target joint angle of the robotic arm. The output of the recurrent neural network model is a series of joint angles with time sequence numbers.

[0067] The joint position generation module utilizes the positive kinematics of the robotic arm to generate a series of joint positions with time sequence numbers in the robotic arm base coordinate system from the series of joint angles with time sequence numbers.

[0068] The planning error calculation module calculates the planning error based on the series of joint positions with time sequence numbers, the joint angles with time sequence numbers, and the environmental point cloud with time sequence numbers.

[0069] The planning error iterative reduction module adjusts the weights of the recurrent neural network model based on the planning error using the backpropagation algorithm, and iteratively reduces the planning error. The iteration ends when the planning error is lower than a set threshold.

[0070] After the iteration of the joint trajectory generation module of the robotic arm is completed, the joint angles with time sequence numbers output by the recurrent neural network model are the joint trajectories of the robotic arm.

[0071] A third aspect of the present invention relates to a robotic arm motion planning device based on a recurrent neural network, characterized in that it comprises:

[0072] At least one memory and at least one processor;

[0073] The memory is used to store one or more programs;

[0074] When the one or more programs are executed by the at least one processor, the at least one processor implements the robotic arm motion planning method based on recurrent neural networks of the present invention.

[0075] A fourth aspect of the present invention relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the recurrent neural network-based robotic arm motion planning method of the present invention.

[0076] This invention encodes the current joint angles and the target joint angles, and infers and calculates a series of joint angles with time sequences to form the motion trajectory of the robotic arm. Utilizing the forward kinematics of the robotic arm, the series of joint angles with time sequences are converted into a series of joint positions with time sequences in the robotic arm's base coordinate system. Furthermore, the obstacle avoidance error, smoothing error, and angle error of the generated trajectory are calculated. Through backpropagation and gradient descent methods, a feasible robotic arm motion trajectory is iteratively optimized.

[0077] Meanwhile, this invention takes into account the problem in operation tasks where the manipulated object is an obstacle that needs to be avoided to prevent collision during trajectory movement, and needs to be approached and touched at the end of the trajectory movement for operation. Based on environmental information, a series of environmental point clouds with time sequences are generated. A series of joint angles with time sequences generated by a recurrent neural network are used to calculate collisions in the environmental point cloud space with the corresponding time sequences, which can simultaneously meet the requirements of obstacle avoidance in the middle of the trajectory and touch at the end of the trajectory.

[0078] The advantages of this invention are that it enables the robot's robotic arm to generate a smooth, obstacle-avoiding, and reasonable motion trajectory during operation tasks, thus demonstrating good application prospects. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of the robotic arm structure provided in an embodiment of the present invention, as well as the current joint angle and the target joint angle;

[0080] Figure 2 This is a schematic diagram of the basic environmental point cloud provided in an embodiment of the present invention;

[0081] Figure 3 This is a flowchart illustrating the environmental point cloud at the end stage provided in an embodiment of the present invention.

[0082] Figure 4 This is a flowchart illustrating the robotic arm motion planning method based on a recurrent neural network provided in an embodiment of the present invention.

[0083] Figure 5 This is a schematic diagram of the recurrent neural network structure and its input and output provided in an embodiment of the present invention;

[0084] Figure 6 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0085] This invention proposes a robotic arm motion planning method based on recurrent neural networks to improve the operational motion performance of the robotic arm. To make the objectives, technical methods, and solutions of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention.

[0086] Conversely, this invention encompasses any substitutions, modifications, equivalents, and methods made to the spirit and scope of the invention as defined by the claims. Furthermore, to provide the public with a better understanding of the invention, specific details are described in the following detailed description of the invention.

[0087] Example 1:

[0088] Figure 1 This is a schematic diagram of the robotic arm structure provided in an embodiment of the present invention, as well as the current joint angle and the target joint angle; Figure 2 This is a flowchart illustrating the basic environmental point cloud and the environmental point cloud at the end stage provided in the embodiments of the present invention; Figure 3 This is a flowchart illustrating the robotic arm motion planning method based on a recurrent neural network provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of the recurrent neural network structure and its input and output provided in an embodiment of the present invention;

[0089] Step S101: Acquire the basic environmental point cloud during robot operation, and calculate the current end position and target end position based on the current joint angle and the target joint angle.

[0090] Specifically, point clouds are obtained by using color images and depth images from depth cameras on the robot, and then the point clouds are transformed into the robot arm's base coordinate system to obtain the basic environment point cloud;

[0091] Generally, the point cloud obtained by the depth camera on the robot is a point cloud in the camera coordinate system. Based on the camera mounting position and the robotic arm mounting position on the robot, a transformation matrix can be obtained from the camera coordinate system to the robotic arm base coordinate system. Using this transformation matrix, the point cloud in the camera coordinate system can be converted into the aforementioned basic environment point cloud e. base ;

[0092] Specifically, such as in Figure 2 As shown on the left, the environmental point cloud contains a table and a bottle on the table. It should be understandable that the environmental point cloud observed by different robots in different scenarios is different.

[0093] The current joint angle and the target joint angle represent the start and end states of the motion trajectory, respectively, and are consistent with the number of joints in the robotic arm.

[0094] Specifically, such as Figure 1 As shown, the robotic arm has 7 degrees of freedom. The left side shows the current joint angle, which is the starting state of the robotic arm's motion trajectory. The right side shows the target joint angle, which is the ending state of the robotic arm's motion trajectory. The current joint angle can be represented as... The target joint angle is

[0095] The current end effector position can be calculated based on the current joint angle using the positive kinematics of the robotic arm.

[0096] Specifically, for Figure 1 The robotic arm on the left has 7 degrees of freedom. Using the forward kinematics of the robotic arm, the position of each joint can be calculated based on the current joint angle, as shown below: The position of the last joint. That is, the current end position;

[0097] The position of the target end can be calculated based on the target joint angle using the positive kinematics of the robotic arm.

[0098] Specifically, for Figure 1 The robotic arm on the right, with 7 degrees of freedom, can use its forward kinematics to determine the position of each joint of the target object based on the joint angles, as shown below: The position of the last joint. That is, the target end position;

[0099] Step S102: Based on the current end position and the target end position, adjust the basic environmental point cloud in time series to obtain a series of environmental point clouds with time sequence numbers, including:

[0100] Specifically, the series of environmental point clouds with time sequences has 100 time sequences, and each time sequence corresponds to an independent environmental point cloud. T =[e 0 ,e 1 ,…,e 10 ,…,e 90 ,…,e 99 ,e 100 ];

[0101] In the initial stage T0—T i In the environmental point cloud within the time sequence, eliminate the point cloud near the current end position in the basic environmental point cloud.

[0102] Specifically, in the intermediate stage T i —T j The environmental point cloud within the time sequence number directly uses the aforementioned basic environmental point cloud, e10 ~e 90 =e base ,like Figure 2 As shown on the left, e 10 ~e 90 That is, the basic environment point cloud;

[0103] Specifically, in the final stage T j —T n In the environmental point cloud within the time sequence, eliminate the point cloud near the target end position from the basic environmental point cloud. like Figure 2 As shown on the right, the point cloud near the target's end position has been eliminated;

[0104] By stitching together the environmental point clouds of the start stage, the middle stage, and the end stage, a series of environmental point clouds with time sequence numbers are obtained.

[0105] Step S103: Establish a recurrent neural network model. The input of the recurrent neural network model is the current joint angle and the target joint angle of the robotic arm. The output of the recurrent neural network model is a series of joint angles with time sequence numbers.

[0106] Specifically, such as Figure 1 As shown, the robotic arm has 7 joints. Therefore, the current joint angle and the target joint angle form a 1×7 vector, i.e. and

[0107] like Figure 4 As shown, the output of the recurrent neural network model is a series of joint angles with time indices. This series of joint angles has 100 time indices, each corresponding to an independent joint angle. This series of joint angles with time indices forms a 100×7 vector, which can be represented as:

[0108]

[0109] The superscript indicates the time sequence number, and the subscript indicates the joint sequence number;

[0110] Step S104: Using the positive kinematics of the robotic arm, generate a series of joint positions with time sequence numbers in the robotic arm base coordinate system from the series of joint angles with time sequence numbers.

[0111] Depending on the different robotic arm structures, this embodiment uses Figure 1 The mechanical arm structure shown is used to construct the forward kinematic equation FK;

[0112] The 100×7 vector formed by the series of joint angles W with time indices is input into the forward kinematics equations to calculate a series of joint positions with time indices in the base coordinate system of the robotic arm. The series of joint positions P with time indices form a 100×7×3 vector, P=FK(W), which can be expressed as:

[0113]

[0114] Step S105: Calculate the planning error based on the series of joint positions with time serial numbers, the joint angles with time serial numbers, and the environmental point cloud with time serial numbers.

[0115] Specifically, linear interpolation is performed on a series of joint positions P with time indices for adjacent joints to obtain a series of continuous joint positions P with time indices. Linear ;

[0116] like Figure 1 The robotic arm shown has 7 joints. Each adjacent joint undergoes 10 linear interpolations. Then P... Linear The vector that forms a 100×61×3 vector is:

[0117]

[0118] Calculate the distance between a series of consecutive joint positions with time numbers and the environmental point cloud with time numbers, as the obstacle avoidance error;

[0119] Specifically, the environmental point cloud e at time i is... i The continuous joint positions at time i. The distance between all points in the point cloud. For example, at time 0, the distance between points in the environmental point cloud is e. 0 Contains 10,000 points. If it contains 61 points, then calculate e. 0 10,000 points and The distance between each pair of 61 points in the middle; at time 95, due to the elimination of point cloud near the target's end position, the e in the environmental point cloud 95 Contains 9000 points, If the number of points still contains 61, then calculate e. 95 9000 points and The distance between any two of the 61 points.

[0120] Find the point in the environmental point cloud that is closest to the consecutive joint positions for all time numbers; this is the obstacle avoidance error.

[0121]

[0122] The difference between adjacent joint angles with time numbers is calculated as a smoothing error;

[0123] Specifically, W i W represents the joint angle at time i. i+1 Let represent the joint angle at time i+1. The change in the shutdown angle between two time points is calculated by subtraction. The summation of the smoothed error over all time points yields:

[0124]

[0125] Calculate the first and last set of joint angles with time sequence numbers, and the difference between them and the current joint angle and the target joint angle of the robotic arm, as the angle error;

[0126] Specifically, W 0 and W 100 These are the angles of the first group of joints and the angles of the last group of joints, respectively. and The difference between the current joint angle and the target joint angle is calculated using the mean square error (MSE) function. The angle error is then:

[0127]

[0128] The obstacle avoidance error, smoothing error, and angle error are added together to obtain the planning error, which is L. 规划 =L 避障 +L 平滑 +L 角度 ;

[0129] Step S106: Based on the planning error, the weights of the recurrent neural network model are adjusted using the backpropagation algorithm, and the planning error is continuously reduced iteratively. The iteration ends when the planning error is lower than a set threshold.

[0130] Specifically, each calculated planning error is used to adjust the weights of the recurrent neural network model through backpropagation and gradient descent algorithms. After adjusting the weights, the recurrent neural network model is used again to infer and calculate a new planning error, that is, the process of S102-S106 is repeated.

[0131] Generally, recurrent neural networks are represented by RNN, planning error is represented by Loss, and backpropagation and gradient descent are represented by BP. Therefore:

[0132]

[0133] L 规划 =Loss(FK(W)), (17)

[0134] RNN new =BP(L 规划 ), (18)

[0135] Iterate through the above three formulas to update the weights of the RNN;

[0136] Set obstacle avoidance threshold Q 避障 Smoothing threshold Q 平滑 Angle threshold Q 角度 When the obstacle avoidance error L in the planning error 避障 Smoothing error L 平滑 Angular error L 角度 The iteration ends when the values ​​fall below the obstacle avoidance threshold, smoothing threshold, and angle threshold, respectively.

[0137] Generally, the obstacle avoidance threshold is set to 0 to ensure that the robotic arm can completely avoid obstacles, while the smoothing threshold is affected by the performance of the robotic arm's joint motors.

[0138] After the iteration is completed, the joint angles with time sequence numbers output by the recurrent neural network model are the joint trajectories of the robotic arm. Since the environmental point cloud is different at each time sequence number, the trajectory generated by this method can avoid the operation target in the middle of the movement, and can touch the operation target in the final stage of the movement trajectory to help the robotic arm perform the operation.

[0139] Example 2

[0140] This embodiment relates to a computer-readable storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the robotic arm motion planning method based on a recurrent neural network as described in any one of claims 1-7.

[0141] Example 3

[0142] This embodiment relates to a robotic arm motion planning system based on recurrent neural networks, used to implement the robotic arm motion planning method based on recurrent neural networks in Embodiment 1, including:

[0143] The end-effector position calculation module is used to acquire the basic environmental point cloud during robot operation and calculate the current end-effector position and target end-effector position based on the current joint angle and the target joint angle.

[0144] The environmental point cloud acquisition module adjusts the basic environmental point cloud in time series based on the current end position and the target end position to obtain a series of environmental point clouds with time sequence numbers;

[0145] A recurrent neural network module is used to build a recurrent neural network model. The input of the recurrent neural network model is the current joint angle and the target joint angle of the robotic arm. The output of the recurrent neural network model is a series of joint angles with time sequence numbers.

[0146] The joint position generation module utilizes the positive kinematics of the robotic arm to generate a series of joint positions with time sequence numbers in the robotic arm base coordinate system from the series of joint angles with time sequence numbers.

[0147] The planning error calculation module calculates the planning error based on the series of joint positions with time sequence numbers, the joint angles with time sequence numbers, and the environmental point cloud with time sequence numbers.

[0148] The planning error iterative reduction module adjusts the weights of the recurrent neural network model based on the planning error using the backpropagation algorithm, and iteratively reduces the planning error. The iteration ends when the planning error is lower than a set threshold.

[0149] After the iteration of the joint trajectory generation module of the robotic arm is completed, the joint angles with time sequence numbers output by the recurrent neural network model are the joint trajectories of the robotic arm.

[0150] Example 4

[0151] This embodiment provides a robotic arm motion planning device based on a recurrent neural network, comprising:

[0152] One or more processors;

[0153] Memory, used to store one or more programs;

[0154] When the one or more programs are executed by the one or more processors, the one or more processors implement a robotic arm motion planning method based on a recurrent neural network as described in Embodiment 1.

[0155] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0156] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0158] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0160] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0161] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for motion planning of a robot arm based on a recurrent neural network, characterized in that, Comprise: S1. Obtain the basic environment point cloud during the robot running process, and calculate the current end position and target end position according to the current joint angle and target joint angle; S2. Based on the current end position and target end position, adjust the basic environment point cloud in time sequence to obtain a series of environment point clouds with time sequence number; S3. Establish a recurrent neural network model, the input of the recurrent neural network model is the current joint angle and target joint angle of the robot arm, and the output of the recurrent neural network model is a series of joint angles with time sequence number; S4. Using the forward kinematics of the robot arm, a series of joint positions with time sequence number of each joint in the robot arm base coordinate system are generated from the series of joint angles with time sequence number; S5. Based on the series of joint positions with time sequence number, the joint angles with time sequence number and the environment point clouds with time sequence number, the planning error is calculated; S5. Based on the planning error, the weights of the recurrent neural network model are adjusted by using the back propagation algorithm, and the planning error is continuously iterated and reduced, and the iteration ends when the planning error is lower than the set threshold; S6. After iteration, the output of the recurrent neural network model with time sequence number is the joint trajectory of the robot arm.

2. The recurrent neural network based robot arm motion planning method of claim 1, wherein, Obtain the basic environment point cloud during the robot running process, and calculate the current end position and target end position according to the current joint angle and target joint angle, comprising: Obtain the point cloud through the color image and depth image of the depth camera on the robot, and convert the point cloud to the robot arm base coordinate system to obtain the basic environment point cloud; The current joint angle and the target joint angle are respectively the start state and the end state of the motion trajectory, and are consistent with the number of joints of the robot arm; The current end position is calculated according to the current joint angle through the forward kinematics of the robot arm; The target end position is calculated according to the target joint angle through the forward kinematics of the robot arm.

3. The recurrent neural network based robot arm motion planning method of claim 1, wherein, Based on the current end position and target end position, adjust the basic environment point cloud in time sequence to obtain a series of environment point clouds with time sequence number, comprising: The series of environment point clouds with time sequence numbers has a time sequence number, each time sequence number corresponds to an independent environment point cloud; In the beginning phase eliminating, in the time-ordered sequence of environment point clouds, the point cloud in the base environment point cloud in the vicinity of the current end position; In the intermediate stage The environment point cloud within the time sequence directly uses the base environment point cloud; In the end phase In the environment point cloud within the time sequence number, eliminate the point cloud in the base environment point cloud near the target end position. Splice the environment point clouds of the start stage, the middle stage and the end stage to obtain the series of environment point clouds with time sequence number.

4. The recurrent neural network based robot arm motion planning method of claim 1, wherein, Establish a recurrent neural network model, the input of the recurrent neural network model is the current joint angle and target joint angle of the robot arm, and the output of the recurrent neural network model is a series of joint angles with time sequence number, comprising: If the robot arm has k joints, then the current joint angles and target joint angles form a vector ; If the series of joint angles with time indexes has a time index corresponding to an independent joint angle, the series of joint angles with time indexes forms a vector.

5. The recurrent neural network based robot arm motion planning method of claim 1, wherein, Using the forward kinematics of the robot arm, a series of joint positions with time sequence number of each joint in the robot arm base coordinate system are generated from the series of joint angles with time sequence number, comprising: According to different robot arm structures, the forward kinematics equation can be constructed; The series of joint angles with time indexes constitute a vector The vector input positive kinematics equation, and the series of joint positions with time indexes in the base coordinate system of the robot arm are calculated, and the series of joint positions with time indexes constitute a vector ​ 6. The recurrent neural network based robot arm motion planning method of claim 1, wherein, Based on the series of joint positions with time sequence number, the joint angles with time sequence number and the environment point clouds with time sequence number, the planning error is calculated, comprising: Linear interpolation is performed on a series of joint positions with time sequence number of adjacent joints to obtain a series of continuous joint positions with time sequence number; calculating a difference between adjacent time-stamped joint angles as a smoothing error; calculating a difference between the first and last group of time-stamped joint angles and the current and target joint angles of the robot arm as an angle error; adding the obstacle avoidance error, the smoothing error, and the angle error to obtain the planning error. based on the planning error, adjusting the weights of the recurrent neural network model using a backpropagation algorithm, and iteratively reducing the planning error until the planning error is below a set threshold, including:

7. The recurrent neural network based robot arm motion planning method of claim 6, wherein, each time the planning error is calculated, the weights of the recurrent neural network model are adjusted using a backpropagation algorithm and a gradient descent algorithm, and after the weights are adjusted, the recurrent neural network model is used to perform inference to calculate a new planning error, and the process is iterated; setting an obstacle avoidance threshold, a smoothing threshold, and an angle threshold, and when the obstacle avoidance error, the smoothing error, and the angle error in the planning error are below the obstacle avoidance threshold, the smoothing threshold, and the angle threshold, respectively, the iteration is ended; after the iteration is ended, the time-stamped joint angles output by the recurrent neural network model are the joint trajectories of the robot arm. comprising:

8. A recurrent neural network-based robot arm motion planning apparatus, characterized by, at least one memory and at least one processor; the memory is configured to store one or more programs; when the one or more programs are executed by the at least one processor, the at least one processor implements the robot arm motion planning method based on the recurrent neural network as claimed in any one of claims 1-7. The program is executed by the processor to implement the robot arm motion planning method based on the recurrent neural network as claimed in any one of claims 1-7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, comprising:

10. A system for motion planning of a robotic arm based on a recurrent neural network, characterized in that, an end position calculation module configured to obtain a basic environment point cloud during robot operation, and calculate a current end position and a target end position based on current joint angles and target joint angles; an environment point cloud acquisition module configured to adjust the basic environment point cloud based on the current end position and the target end position to obtain a series of time-stamped environment point clouds; a recurrent neural network module configured to establish a recurrent neural network model, wherein the input of the recurrent neural network model is the current joint angles and the target joint angles of the robot arm, and the output of the recurrent neural network model is a series of time-stamped joint angles; a joint position generation module configured to generate a series of time-stamped joint positions of each joint in a robot arm base coordinate system from the series of time-stamped joint angles using forward kinematics of the robot arm; a planning error calculation module configured to calculate a planning error based on the series of time-stamped joint positions, the time-stamped joint angles, and the time-stamped environment point clouds; a planning error iterative reduction module configured to adjust the weights of the recurrent neural network model using a backpropagation algorithm based on the planning error, and iteratively reduce the planning error until the planning error is below a set threshold. ​ The joint trajectory generation module of the mechanical arm, after the iteration is completed, the joint angle with the time sequence number of the output of the recurrent neural network model is the joint trajectory of the mechanical arm.

Images