A torque-controlled robotic arm autonomous trajectory planning method and system

Abstract

The present invention discloses a torque-controlled autonomous trajectory planning method and system for a manipulator. The method comprises: obtaining input data; wherein the input data includes an image of a target point motion region, the end position of the manipulator in Cartesian space, and the angular position, velocity, acceleration, and torque of the manipulator in joint space; inputting the input data into a path planning network model, solving the input data through the path planning network model, and outputting a joint torque control variable; and controlling the manipulator to continuously, smoothly, and quickly move from a current position to a target position based on the joint torque control variable output by the path planning network model. The present invention can autonomously plan a motion trajectory, shortening the time it takes the end of the manipulator to reach the target point.

Description

Technical Field

[0001] The present invention belongs to the technical field of robots, and in particular relates to a method and system for autonomous trajectory planning of a torque-controlled robotic arm. Background Art

[0002] Currently, the main trajectory planning methods for serial manipulators include kinematics, artificial potential fields, rapidly expanding random trees, and neural network models: 1) The kinematic method relies on the manipulator's DH parameters and performs polynomial interpolation in joint space or Cartesian space to construct a smooth trajectory curve. When the manipulator has multiple redundant degrees of freedom, its parameters are numerous. As the order of the polynomial increases, the smoothness improves, but the calculation becomes more complex. 2) The artificial potential field method performs estimation planning and has good real-time performance, but it cannot guarantee path reliability when the manipulator has a high number of degrees of freedom. 3) The rapidly expanding random tree method is simple and easy to implement in theory, but the parameter setting is complex and the algorithm is inefficient. 4) The neural network model method trains the manipulator through reinforcement learning, which can achieve an end-to-end mapping network from joint space to Cartesian space, reducing the reliance on formula derivation and complex calculations, and providing new ideas for manipulator trajectory planning.

[0003] However, none of these methods consider dynamic performance or the torque-velocity curves of the joint motors. In practice, the maximum acceleration a robotic arm can achieve is related to factors such as dynamic performance and motor output torque. Furthermore, motor characteristics are not determined by maximum torque or acceleration, but rather by the torque-velocity curve, or mechanical characteristics. Traditional planning methods struggle to incorporate mechanical characteristic parameters into kinematic equations, making fully utilizing acceleration performance in trajectory planning a major challenge. Summary of the Invention

[0004] The technology of the present invention solves the problem: Overcoming the shortcomings of the existing technology, providing a torque-controlled robotic arm autonomous trajectory planning method and system, which can autonomously plan the motion trajectory so that the end of the robotic arm takes less time to reach the target point.

[0005] In order to solve the above technical problems, the present invention discloses a method for autonomous trajectory planning of a torque-controlled manipulator, comprising:

[0006] Acquire input data; wherein the input data includes: an image of the target point motion area, the end position of the manipulator in Cartesian space, and the angular position, velocity, acceleration, and torque of the manipulator in joint space;

[0007] Input data into the path planning network model, solve the input data through the path planning network model, and output the joint torque control value;

[0008] According to the joint torque control quantity output by the path planning network model, the robotic arm is controlled to move continuously, smoothly and quickly from the current point posture to the target point posture.

[0009] In the above-mentioned torque-controlled robotic arm autonomous trajectory planning method, the path planning network model is obtained through autonomous learning training.

[0010] In the aforementioned torque-controlled robotic arm autonomous trajectory planning method, the path planning network model uses a deep reinforcement learning algorithm to train the target network during autonomous learning training, approximating the optimal strategy by maximizing the discounted future expected cumulative reward. The following nonlinear reward function is used to evaluate the reward r of the robotic arm's current action:

[0011]

[0012]

[0013] r=r pos +r ori +q3r step

[0014] Among them, r pos represents the position reward, r ori represents the posture reward, r step represents the time penalty; Δx, Δy, Δz represent the difference between the target position and the current end position in the manipulator base coordinate system; q1 represents the coefficient for adjusting the position reward slope; Δα, Δβ, Δγ represent the difference between the target posture and the current end posture in the manipulator base coordinate system, q2 represents the coefficient for adjusting the posture reward slope; q3 represents the weight for adjusting the time penalty.

[0015] In the above-mentioned torque-controlled robotic arm autonomous trajectory planning method, the path planning network model updates the target network parameters in the direction of "maximizing the discounted future expected cumulative reward" during autonomous learning training to approach the optimal path planning strategy with the shortest time; wherein, the discounted future expected cumulative reward refers to the state S under the current step number i of the robotic arm. i -Action A i Value Q(S,A).

[0016] In the above-mentioned torque-controlled autonomous trajectory planning method for a robotic arm, the path planning network model collects robotic arm samples into an experience pool during autonomous learning training and uses prioritized experience replay for offline strategy updates. Simultaneously, multiple samples are learned and parameter updates are performed to improve strategy update efficiency and sample independence, thereby addressing the dynamic state distribution problem of the robotic arm in different rounds.

[0017] In the above-mentioned torque-controlled autonomous trajectory planning method for a robotic arm, the path planning network model, during autonomous learning training, can not only learn the kinematic relationship between the joint angles and the end pose of the robotic arm by approximating the optimal strategy, but also learn the torque-speed relationship of the joint drive motor and the dynamic performance of the robotic arm. With the goal of fewer steps, it outputs the joint torque, thereby generating a speed-optimized trajectory that is constrained by both speed and torque.

[0018] In the torque-controlled autonomous trajectory planning method for a robotic arm, the path planning network model imposes various constraints on the path planning network during autonomous learning and training to ensure the safety of the robotic arm itself and the environment. These constraints include: single-step torque constraints, maximum motor torque constraints, robotic arm workspace constraints, robotic arm collision and singularity constraints, and a maximum number of steps per round constraint.

[0019] In the above-mentioned torque-controlled robotic arm autonomous trajectory planning method, the path planning network model extracts features from the input target point motion area image during autonomous learning training. After passing the extracted features through the convolution layer and the global average pooling layer, they are sent to the activation function to obtain the target posture information.

[0020] In the above-mentioned torque-controlled autonomous trajectory planning method for a robotic arm, an image of the target point motion area is acquired by collecting data through a hand-eye camera; and the angular position, velocity, acceleration, and torque of the robotic arm in the joint space are acquired through the associated joint force sensors.

[0021] Accordingly, the present invention also discloses a torque-controlled robotic arm autonomous trajectory planning system, comprising:

[0022] A data acquisition module is used to acquire input data; wherein the input data includes: an image of the target point motion area, the end position of the manipulator in Cartesian space, and the angular position, velocity, acceleration and torque of the manipulator in joint space;

[0023] The solution module inputs the input data into the path planning network model, solves the input data through the path planning network model, and outputs the joint torque control value;

[0024] The control module is used to control the robot arm to move continuously, smoothly and quickly from the current point posture to the target point posture according to the joint torque control quantity output by the path planning network model.

[0025] The present invention has the following advantages:

[0026] (1) The present invention discloses a torque-controlled autonomous trajectory planning method for a robotic arm. Unlike the traditional kinematic trajectory planning method, the present invention takes into account the dynamic performance of the robotic arm and learns the torque-speed relationship of the joint drive motor. It can not only plan a continuous and smooth motion path for the end of the robotic arm to reach the target position, but also consider the speed and acceleration information of each point on the path, thereby generating a speed-optimized trajectory constrained by torque.

[0027] (2) The present invention discloses a method for autonomous trajectory planning of a torque-controlled robotic arm. For a robotic arm with multiple degrees of freedom and redundancy, the path curve obtained by traditional solutions such as spline interpolation and inverse kinematics is often less smooth and computationally complex. The present invention combines a reinforcement learning algorithm with a high-dimensional continuous state space and continuous motion control with the robotic arm trajectory planning task. The trained end-to-end neural network greatly improves the computational efficiency.

[0028] (3) The present invention discloses a method for autonomous trajectory planning of a torque-controlled robotic arm. Deep reinforcement learning does not rely on the physical model of the robotic arm, thus avoiding the problems caused by model uncertainty. There is no need for supervised learning and augmented teaching. For the objective function with complex dynamics, the designed nonlinear reward function enables the training model to have faster convergence speed and stability, thus avoiding the occurrence of local optimality or oscillation and non-convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a schematic diagram illustrating the implementation principle of a torque-controlled autonomous trajectory planning method for a manipulator according to an embodiment of the present invention;

[0030] Figure 2 Schematic diagram of a full-dimensional simulation test platform for a torque-controlled robotic arm autonomous trajectory planning method in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments disclosed in the present invention will be described in further detail below with reference to the accompanying drawings.

[0032] Conventional trajectory planning methods typically rely on position control, which doesn't fully exploit the acceleration performance of the robotic arm. This leads to a trajectory planning problem that considers dynamics. One of the pressing challenges facing researchers is how to learn the robotic arm's dynamics and the speed-torque constraints of each joint's drive motors, autonomously planning the trajectory, and minimizing the time it takes for the robotic arm to reach its target point while satisfying speed-torque constraints.

[0033] In this embodiment, the torque-controlled robotic arm autonomous trajectory planning method includes:

[0034] Step 1: Get input data.

[0035] In this embodiment, the input data is a high-dimensional continuous state space, mainly including: the target point motion area image, the end position of the manipulator in Cartesian space, and the angular position, velocity, acceleration and torque of the manipulator in joint space. Figure 1 , the target point motion area image can be acquired through the hand-eye camera, and the angular position, velocity, acceleration and torque of the robot arm in the joint space can be obtained through the corresponding joint force sensor.

[0036] Step 2: Input the input data into the path planning network model, solve the input data through the path planning network model, and output the joint torque control value.

[0037] In this embodiment, the path planning network model can be trained through autonomous learning training.

[0038] Preferably, the path planning network model uses a deep reinforcement learning algorithm to train the target network during autonomous learning training, and approaches the optimal strategy by maximizing the discounted expected future cumulative reward. Specifically, the following nonlinear reward function can be used to evaluate the reward r of the current action of the robot arm:

[0039]

[0040]

[0041] r=r pos +r ori +q3r step

[0042] Among them, the reward r consists of three parts: position reward r pos , posture reward r ori and time penalty r step For the position reward r pos , Δx, Δy, Δz represent the difference between the target position and the current end position in the manipulator base coordinate system, q1 represents the coefficient for adjusting the position reward slope. The closer the manipulator end is to the target, the greater the reward value. ori , Δα, Δβ, Δγ represent the difference between the target posture and the current end posture in the manipulator base coordinate system, q2 represents the coefficient for adjusting the slope of the posture reward. The closer the manipulator end posture is to the target posture, the greater the reward value. For the time penalty r step , a negative reward is given to the number of steps of the robot in the current round, and q3 represents the weight of adjusting the time penalty. The more steps, the longer it takes for the robot to reach the target point and the lower the score, so that the robot can learn a planning method that takes less time.

[0043] Preferably, the path planning network model updates the target network parameters in the direction of "maximizing the discounted future expected cumulative reward" during autonomous learning training to approach the optimal path planning strategy with the shortest time. The discounted future expected cumulative reward refers to the state S under the current step number i of the robot arm. i -Action A i Value Q(S,A).

[0044] Preferably, when the path planning network model is conducting autonomous learning training, it collects robot arm samples into the experience pool and uses priority experience replay for offline strategy update; it simultaneously learns and updates parameters of multiple samples to improve the efficiency of strategy update, increase sample independence, and solve the problem of dynamic state distribution of the robot arm in different rounds.

[0045] Preferably, during autonomous learning training, the path planning network model receives input information that includes not only the relationship between the target pose and the end-point pose, but also dynamics-related information such as velocity, acceleration, and torque. By approximating the optimal strategy, it learns not only the kinematic relationship between the joint angles and the end-point pose of the robot arm, but also the torque-velocity relationship of the joint drive motors and the dynamic performance of the robot arm. With the goal of achieving fewer steps, the joint torque is output, thereby generating a velocity-optimized trajectory that is constrained by both velocity and torque.

[0046] Preferably, during autonomous learning and training, the path planning network model applies various constraints to the path planning network to ensure the safety of the robot and the environment. These constraints include: single-step torque constraints, maximum motor torque constraints, robot workspace constraints, robot collision and singularity constraints, and a maximum number of steps per round.

[0047] Preferably, when the path planning network model is undergoing autonomous learning training, the input target point motion area image is subjected to feature extraction, the extracted features are passed through the convolution layer and the global average pooling layer, and then fed into the activation function to obtain the target posture information. Among them, the target point motion area image is used as the input state selection action of the path planning network model on the one hand, and the extracted target posture is used to calculate the reward value r of the current state-action. pos Conduct an assessment.

[0048] Preferably, the data output by the path planning network model is high-dimensional continuous motion control, mainly including joint torque control quantities.

[0049] Step 3: According to the output torque of each joint motor of the robotic arm, the robotic arm is controlled to move from the current point posture to the target point posture along the path planning network.

[0050] In summary, the present invention discloses a torque-controlled autonomous trajectory planning method for a manipulator, which solves the problem that the kinematic trajectory planning method cannot fully utilize the acceleration performance of the manipulator. That is, how to learn the dynamic performance of the manipulator and the speed-torque constraint relationship of each joint drive motor under the premise of satisfying the speed-torque constraint, and autonomously plan the motion trajectory so that the end of the manipulator reaches the target point in a shorter time. The torque-controlled autonomous trajectory planning method for a manipulator trains a path planning network with an optimal strategy through a deep reinforcement learning algorithm, obtains a motion area image containing target point posture information through a hand-eye camera, and uses it as the input state of the path planning network. The path planning network outputs the torque of each joint of the manipulator according to the optimal strategy of autonomous learning, and guides the manipulator to reach the target point posture from the current point posture. The method learns the dynamics of the manipulator and the speed-torque constraint relationship, fully utilizes the acceleration performance, and makes the movement time shorter.

[0051] In this embodiment, if Figure 2 As shown in the figure, a full-dimensional simulation test platform for autonomous trajectory planning tasks for a robotic arm has been built. The CoppeliaSim physics engine is used to model and simulate the dynamics and kinematics of the robotic arm and the environment. This allows for the acquisition of status information from sensors such as the hand-eye camera and the reception of torque control signals to control the robotic arm's motion. PyTorch is used to train, update, and test the trajectory planning network's strategy. The simulation environment communicates with Python via the TCP / IP protocol and the RemoteAPI interface. The simulation environment transmits the current state to the training network, which then uses the actions selected by the strategy as motion commands for the robotic arm, controlling the robotic arm's motion and affecting the environment, thus achieving a closed-loop control system.

[0052] During the simulation, an episode is defined as the moment the robot reaches the target point or the robot reaches the maximum number of steps. At the beginning of each episode, the target point's pose is randomly set within the robot's motion range. The simulation environment transmits the current state S0 (hand-eye camera image, robot end-point pose, joint angular position, angular velocity, angular acceleration, and torque) to the trajectory planning network being trained via TCP / IP communication. The network's strategy selects the action A0 with the highest Q value based on the current state, i.e., the torques of each joint. The joint torques are transmitted to the simulation environment via TCP / IP communication to control the robot's motion. After the robot moves, the state information S1 is updated and fed back to the trajectory planning network. A sample [S0, A0, S1, R1] is generated and stored in the experience pool. The hand-eye camera captures an image of the target point's motion area. The path planning network extracts image features, passes them through convolutional layers and global average pooling layers, and then feeds them into an activation function to obtain the target pose information.

[0053] In this example, the neural network executed a total of 1500 rounds, with a maximum step size of 200 steps per round. The network converged in 11 hours on a single CPU, achieving an accuracy of 96%, a position accuracy of less than 3mm, and an attitude accuracy of less than 2°. This ultimately enabled continuous and smooth path planning for the target point's position and attitude in six dimensions.

[0054] Based on the above embodiments, the present invention also discloses a torque-controlled robotic arm autonomous trajectory planning system, including: a data acquisition module, used to obtain input data; wherein the input data includes: an image of the target point motion area, the end posture of the robotic arm in Cartesian space, and the angular position, velocity, acceleration and torque of the robotic arm in the joint space; a solution module, which inputs the input data into a path planning network model, solves the input data through the path planning network model, and outputs a joint torque control amount; a control module, which is used to control the robotic arm to move continuously, smoothly and quickly from the current point posture to the target point posture according to the joint torque control amount output by the path planning network model.

[0055] As for the system embodiment, since it corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0056] Although the present invention has been disclosed above in terms of preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solutions of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the scope of protection of the technical solutions of the present invention.

[0057] The contents not described in detail in the specification of the present invention belong to the common knowledge of professionals in this field.

Claims

1. A torque-controlled robotic arm autonomous trajectory planning method, characterized in that: include: Acquire input data; wherein the input data includes: an image of the target point motion area, the end position of the manipulator in Cartesian space, and the angular position, velocity, acceleration, and torque of the manipulator in joint space; Input data is input into the path planning network model, and the input data is solved by the path planning network model to output the joint torque control value; wherein the path planning network model is trained by an autonomous learning training method and is used to solve the input data and output the joint torque control value; According to the joint torque control quantity output by the path planning network model, the robot arm is controlled to move continuously, smoothly and quickly from the current point posture to the target point posture; During autonomous learning training, the path planning network model uses a deep reinforcement learning algorithm to train the target network, approaching the optimal strategy by maximizing the discounted expected future cumulative reward. The following nonlinear reward function is used to evaluate the reward r of the robot's current action: r＝r pos +r ori +q3r step Among them, r pos represents the position reward, r ori represents the posture reward, r step represents the time penalty; Δx, Δy, Δz represent the difference between the target position and the current end position in the manipulator base coordinate system; q1 represents the coefficient for adjusting the position reward slope; Δα, Δβ, Δγ represent the difference between the target posture and the current end posture in the manipulator base coordinate system, q2 represents the coefficient for adjusting the posture reward slope; q3 represents the weight for adjusting the time penalty.

2. The method for autonomous trajectory planning of a torque-controlled manipulator according to claim 1, characterized in that: When the path planning network model is undergoing autonomous learning training, it updates the target network parameters in the direction of "maximizing the discounted future expected cumulative reward" to approach the optimal path planning strategy with the shortest time; the discounted future expected cumulative reward refers to the state S under the current step number i of the robot arm. i -Action A i Value Q(S,A).

3. The method for autonomous trajectory planning of a torque-controlled manipulator according to claim 1, characterized in that: During autonomous learning and training, the path planning network model collects robot arm samples into the experience pool and uses prioritized experience replay to update offline strategies; it also learns and updates parameters of multiple samples at the same time.

4. The method for autonomous trajectory planning of a torque-controlled manipulator according to claim 1, characterized in that: During autonomous learning and training, the path planning network model generates a speed-optimized trajectory that is subject to both speed and torque constraints by approximating the optimal strategy.

5. The method for autonomous trajectory planning of a torque-controlled manipulator according to claim 4, characterized in that: During autonomous learning and training, the path planning network model imposes various constraints on the path planning network to ensure the safety of the robot itself and the environment. The constraints imposed include: single-step torque constraint, maximum motor torque constraint, robot workspace constraint, robot collision and singularity constraint, and maximum number of steps in a round.

6. The method for autonomous trajectory planning of a torque-controlled manipulator according to claim 1, characterized in that: During autonomous learning training, the path planning network model extracts features from the input target point motion area image, passes the extracted features through the convolution layer and the global average pooling layer, and then sends them to the activation function to obtain the target posture information.

7. The method for autonomous trajectory planning of a torque-controlled manipulator according to claim 1, characterized in that: The image of the target point motion area is obtained through the hand-eye camera; the angular position, velocity, acceleration and torque of the robotic arm in the joint space are obtained through the corresponding joint force sensors.

8. A robotic arm autonomous trajectory planning system for implementing torque control according to the method of claim 1, characterized in that: include: A data acquisition module is used to acquire input data; wherein the input data includes: an image of the target point motion area, the end position of the manipulator in Cartesian space, and the angular position, velocity, acceleration and torque of the manipulator in joint space; The solution module inputs the input data into the path planning network model, solves the input data through the path planning network model, and outputs the joint torque control value; The control module is used to control the robot arm to move continuously, smoothly and quickly from the current point posture to the target point posture according to the joint torque control quantity output by the path planning network model.

Images