A redundant space robot obstacle avoidance planning method based on relaxed null space

Abstract

The application provides a redundant space robot obstacle avoidance planning method based on a relaxed null space, comprising: constructing an end desired trajectory generator based on artificial demonstration; constructing an end desired trajectory tracking controller; constructing a relaxed null space motion strategy of the redundant space robot according to mapping of end motion of the redundant space robot and joint motion; constructing state variables and action variables of the redundant space robot according to parameters related to the redundant space robot and obstacles; constructing an obstacle avoidance reward function of the redundant space robot according to an end relaxed motion vector; constructing a relaxed null space obstacle avoidance motion training strategy according to the variables and functions constructed above; and obtaining a relaxed null space obstacle avoidance motion planner according to the training obstacle avoidance motion planning strategy network and the end desired trajectory generator. According to the technical scheme provided by the embodiment of the application, the redundant space robot can track the end desired trajectory as much as possible while avoiding collision.

Description

[Technical Field]

[0001] This invention belongs to the field of motion planning for space robotic arms, and relates to a redundant space robotic arm obstacle avoidance planning method based on relaxed null space. [Background Technology]

[0002] A space robotic arm is a specialized robotic arm that performs various on-orbit tasks in space. It avoids exposing astronauts to the vacuum, weightlessness, and intense radiation of the space environment, thus reducing operational risks. Its primary objective during mission execution is to avoid collisions with obstacles in the environment. Secondly, it aims to guide the end effector's movement along a predetermined trajectory. However, sometimes the actual trajectory of the end effector can deviate from the predetermined trajectory. For example, in surface inspection missions, the end effector moves according to the predetermined trajectory to ensure that a designated area on the surface can be monitored. If a floating obstacle approaches the robotic arm, it can follow the predetermined trajectory as closely as possible while avoiding collisions, but brief deviations from the predetermined trajectory are also permitted for obstacle avoidance maneuvers.

[0003] Redundant spatial manipulators with more than 7 degrees of freedom at their joints are capable of performing the aforementioned functions. However, most existing obstacle avoidance motion planning methods for redundant spatial manipulators are designed for static scenes, searching for collision-free trajectories using global information. If dynamic obstacles exist in the scene, a suitable collision-free trajectory needs to be found within each time step, consuming a significant amount of computation time and failing to meet the requirements of real-time control and motion continuity.

[0004] In contrast, local obstacle avoidance methods such as artificial potential field methods and gradient projection methods adjust the predetermined trajectory in real time according to the state of the obstacle, greatly reducing computational costs and making real-time obstacle avoidance possible. However, the former does not consider the characteristics of redundant space manipulators, thus altering the trajectory of the end effector during obstacle avoidance and failing to track the desired trajectory. The latter, requiring the end effector to follow the desired trajectory, presents significant challenges in designing trajectory-changing strategies applicable to various obstacle motion scenarios, and becomes ineffective when obstacles encroach on the area traversed by the desired trajectory. Therefore, a new obstacle avoidance motion planning method for redundant space manipulators is needed to ensure that the end effector tracks the desired trajectory as closely as possible while avoiding collisions. [Summary of the Invention]

[0005] In view of this, the present invention provides an obstacle avoidance planning method for redundant spatial robotic arms based on relaxed null space, which enables redundant spatial robotic arms to track the desired trajectory at the end of the arm as much as possible while efficiently avoiding collisions.

[0006] This invention provides a method for obstacle avoidance planning of a redundant space robotic arm based on a relaxed null space, comprising:

[0007] Step S1: Construct an end-effector expected trajectory generator based on manual demonstration to obtain the generated redundant space robot arm end-effector expected pose and velocity;

[0008] Step S2 constructs an end-expected trajectory tracking controller based on the end-expected trajectory generator to obtain the end-expected trajectory tracking speed when there are no obstacles;

[0009] Step S3 constructs a relaxation null space motion strategy for the redundant space manipulator based on the mapping between the end-effector motion and joint motion, the end-effector desired trajectory generator, and the end-effector desired trajectory tracking controller, and obtains the joint null space motion vector and the end-effector relaxation motion vector.

[0010] Step S4 constructs the state variables and motion variables of the redundant space manipulator based on the actual joint angles, actual joint angular velocities, relative vectors between the link and the obstacle, joint zero-space motion vectors, and end-effector relaxation motion vectors.

[0011] Step S5 constructs the obstacle avoidance reward function for the redundant space robotic arm based on the end-effector relaxation motion vector;

[0012] Step S6 constructs a relaxed zero-space obstacle avoidance motion training strategy based on the redundant space manipulator's state variables, motion variables, and obstacle avoidance reward function, and obtains the trained redundant space manipulator relaxed zero-space obstacle avoidance motion planning strategy network.

[0013] Step S7 obtains the redundant space manipulator relaxed zero-space obstacle avoidance motion planner based on the trained redundant space manipulator relaxation zero-space obstacle avoidance motion planning strategy network and the end-effector expected trajectory generator based on manual demonstration.

[0014] In the above method, step S1 includes:

[0015] Based on the dynamic motion primitive method, the trajectory of the redundant spatial manipulator's end effector is represented as the following second-order nonlinear dynamic system model:

[0016]

[0017] Where τ is the scaling factor that enables the trajectory to be scaled proportionally to different time scales, and α y and β y Given system constants, x e For the redundant space robotic arm end effector trajectory x e One dimension component, x eg f(s,x) represents the component of the target position of the trajectory in this dimension. eg Let be the forcing function introduced to the nonlinear dynamic characteristics of the second-order nonlinear dynamic system, expressed as:

[0018]

[0019] Where, x e0 For x e The starting position, ψ i Let i be the Gaussian kernel function numbered i (i = 1 to N), and N be the number of kernel functions, ω i For ψ i The weighting coefficients are given by s, where s is the state variable of the first-order linear system.

[0020] In the above method, step S2 includes:

[0021] Construct an end-effector desired trajectory tracking controller of the following form to achieve end-effector desired trajectory tracking of a redundant space robot arm when there are no obstacles in the working environment:

[0022]

[0023] in, x represents the tracking speed of the robotic arm's end effector in the absence of obstacles, representing the redundant space. ed and The desired pose and velocity of the redundant space robotic arm end effector generated by the end effector trajectory generator, x e and This represents the actual end-effector pose and velocity.

[0024] In the above method, step S3 includes:

[0025] Based on the terminal relaxation characteristics, the actual terminal velocity is expressed as:

[0026]

[0027] in, The tracking speed of the robotic arm end effector in the absence of obstacles is the redundancy space. This is the end-effector relaxation vector;

[0028] Based on the mapping relationship between the end-effector motion and joint motion of the redundant space manipulator, a relaxation null space motion strategy for the redundant space manipulator is constructed, which is represented by the following mapping relationship:

[0029]

[0030] in, Let J be the joint angular velocity of the redundant space manipulator, and J be the Jacobian matrix of the redundant space manipulator. + Let J be the pseudo-inverse matrix, and let I be the identity matrix. x represents the tracking speed of the robotic arm's end effector in the absence of obstacles, representing the redundant space. es For the end-effector relaxation vector, The null space motion vector of the redundant space robotic arm. For the end-point trajectory tracking component, To relax the zero-space obstacle avoidance motion components.

[0031] In the above method, step S4 includes:

[0032] Step S4.1 Construct the state variables s of the redundant space robot at time t. t for:

[0033]

[0034] Where q(t) and Let p0(t) and p be the joint angle vector and joint angular velocity vector at time t, respectively. i (t)(i=1~n) represent the distance vectors between the base, link i and the obstacle, respectively;

[0035] Step S4.2 Construct the motion variable a of the redundant spatial manipulator at time t. t for:

[0036]

[0037] in, The null space motion vector of the redundant space robotic arm. This is the end-effector relaxation vector.

[0038] In the above method, step S5 includes:

[0039] Construct the obstacle avoidance reward function r for the redundant space robotic arm at time t. t for:

[0040] r t =λ o r o +λ s r s

[0041] Where, r o For the obstacle avoidance feedback term, its expression is:

[0042]

[0043] Where, p i Let d be the distance vector between link i of the redundant space robotic arm and the obstacle. s To ensure a safe distance between the redundant space robotic arm and obstacles, r s The stability term is expressed as follows:

[0044]

[0045] in, For the end-effector relaxation vector The i-th dimension component (i = 1 to s), λ is the maximum allowed end-effector relaxation motion vector in this dimension. o and λ s r o and r s The weighting coefficients thus determine the total reward value r t ∈[0,1].

[0046] In the above method, step S6 includes:

[0047] Step S6.1 Constructs the reinforcement learning SAC algorithm network structure required for the relaxed zero-space obstacle avoidance motion training strategy, which includes the redundant space robotic arm relaxed zero-space obstacle avoidance planning strategy network π θ Two main Q networks and Two-target Q-network and Among them, the policy network π θ The state variables of the redundant space manipulator are used as input, and the motion variables of the redundant space manipulator are used as output; main Q-network and the target Q network Both take the state variables and motion variables of the redundant space robotic arm as inputs and output the Q value corresponding to the state variables and motion variables;

[0048] Step S6.2 Relax the zero-space obstacle avoidance planning strategy network π for the redundant space robotic arm according to the following process. θ Conduct training:

[0049] Step S6.2.1 Specifies the initial configuration q of the redundant space robot arm. fix The safe distance d between the redundant space robotic arm and obstacles s The maximum number of rounds k during training num_ep And the longest time t for each round ep_max ;

[0050] Step S6.2.2: Create a data resource pool and initialize the policy network π. θ Main Q network and and the target Q network and

[0051] Step S6.2.3 Set the current round number k = 1;

[0052] Step S6.2.4 Generate a random joint angle Δq, let q init =q fix+Δq, and calculate the initial end-effector pose based on the positive kinematics of the redundant space manipulator.

[0053] Step S6.2.5 Select obstacle location p o Make it satisfy the condition 0 < ||p i ||<d s (i = 1 to n, where n is the number of links in the redundant space manipulator);

[0054] Step S6.2.6 Set the current time t = 0;

[0055] Step S6.2.7 Let the cumulative reward R t =0, and obtain the state variable s of the redundant space robotic arm based on environmental feedback. t ;

[0056] Step S6.2.8 Change the state variable s of the redundant space robot at the current moment. t Input Policy Network π θ Obtain the motion variable a of the redundant space robotic arm. t ;

[0057] Step S6.2.9 According to a t Drive the redundant space robotic arm to obtain the state variable s at time t+1. t+1 And calculate the obstacle avoidance reward function r at time t. t Add it to the cumulative reward R t middle;

[0058] Step S6.2.10: Based on the reinforcement learning SAC algorithm, sample training data from the data resource pool and update the main Q network. and Target Q network and and policy network π θ Network parameters;

[0059] Step S6.2.11 If ||p i ||>d s Or ||p i If || = 0, exit the current round; otherwise, skip this step.

[0060] Step S6.2.12 Let t = t + 1, s t =s t+1 ;

[0061] Step S6.2.13 If t≤t max If the condition is not met, return to step S6.2.8; otherwise, skip this step.

[0062] Step S6.2.14 Let k = k + 1;

[0063] Step S6.2.15 If k < k num_ep If the condition is met, return to step S6.2.4; otherwise, skip this step.

[0064] Step S6.2.16: End training and obtain the trained redundant space robotic arm relaxation zero-space obstacle avoidance motion planning strategy network.

[0065] In the above method, step S7 includes:

[0066] Step S7.1 Deploy the redundancy space robotic arm relaxation zero-space obstacle avoidance motion planning strategy network obtained after training.

[0067] Step S7.2 Set the end-point desired trajectory tracking controller parameter K p K d Set the maximum number of time steps t for a single round. ep_max ;

[0068] Step S7.3 Obtain the desired trajectory using the terminal desired trajectory generator.

[0069] Step S7.4 Set the current time t = 0;

[0070] Step S7.5 Obtain the state variable s of the redundant space robotic arm based on environmental feedback. t ;

[0071] Step S7.6 Obtain the end-effector tracking speed of the redundant space robotic arm based on the desired end-effector trajectory tracking controller.

[0072] Step S7.7: Change the state variable s of the redundant space robot arm at the current moment. t Input Policy Network Obtain the motion variable a of the redundant space robotic arm t ;

[0073] Step S7.8 If Then use a t Drive the redundant space robot arm; otherwise, follow the tracking speed of the redundant space robot arm's end effector when there are no obstacles. Drive redundant space robotic arms;

[0074] Step S7.9 Obtain the state variable s of the redundant space robot at time t+1. t+1 ;

[0075] Step S7.10 Let t = t + 1, s t =s t+1 ;

[0076] Step S7.11 If t ≤ t max Return to step S7.6; otherwise, skip this step.

[0077] Step S7.12 concludes the above steps and completes the motion planning for the relaxation zero-space obstacle avoidance of the redundant space robotic arm. [Attached Image Description]

[0078] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort or labor.

[0079] Figure 1 This is a flowchart illustrating a redundant spatial robotic arm obstacle avoidance planning method based on relaxed null space provided in an embodiment of the present invention.

[0080] Figure 2 This is a schematic diagram of the planar four-degree-of-freedom redundant space robotic arm model used in the simulation experiment of this invention embodiment;

[0081] Figure 3 This is a graph showing the cumulative reward average value recorded during testing of the policy network during the training process in an embodiment of the present invention.

[0082] Figure 4 This is a diagram of the obstacle avoidance trajectory of the end effector of the redundant space robot arm, recorded during testing of the trained policy network in an embodiment of the present invention. [Specific Implementation Examples]

[0083] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0084] It should be understood that the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0085] This invention provides an obstacle avoidance planning method for a redundant space robotic arm based on a relaxed null space. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating a redundant space robotic arm obstacle avoidance planning method based on relaxed null space provided in this invention. Figure 1 As shown, the method includes the following steps:

[0086] Step 101: Construct an end-effector expected trajectory generator based on manual demonstration to obtain the expected pose and velocity of the generated redundant space robotic arm end-effector;

[0087] Specifically, based on the dynamic motion primitive method, the trajectory of the redundant spatial manipulator's end effector is represented as the following second-order nonlinear dynamic system model:

[0088]

[0089] Where τ is the scaling factor that enables the trajectory to be scaled proportionally to different time scales, and α y and β y Given system constants, x e For the redundant space robotic arm end effector trajectory x e One of the components, For x e The first derivative, For x e The second derivative of x eg f(s,x) represents the component of the target position of the trajectory in this dimension. eg Let be the forcing function introduced to the nonlinear dynamic characteristics of the second-order nonlinear dynamic system, where s is the time-dependent state variable of the first-order linear system, i.e.

[0090]

[0091] For equation (1), This constitutes the linear term of the system, therefore in f(s,x) eg Under the condition that ) = 0, the driving variable x e Move in a straight line to x eg The forced function f(s,x) eg This then acts as an interference term on the aforementioned trajectory, by adjusting x at the acceleration level. e To interfere with the operating state of x, thereby making x e The actual trajectory of motion exhibits the desired trend; the forcing function is defined as:

[0092]

[0093] Where, x e0 For x e The starting position; ψ i Let i be the Gaussian kernel function numbered i (i = 1 to N), and N be the number of kernel functions, ω i For ψ i The weighting coefficients. Gaussian kernel function ψ i Defined as:

[0094] ψ i =exp(-h i(sc i ) 2 (4)

[0095] Among them, c i and h i These are the center and variance of the kernel function, respectively.

[0096] Assuming the manual demonstration trajectory of the redundant space robotic arm's end effector is It has T sampling points, i.e., t = 1 to T. One dimension component is denoted as This component can also be written in the form of equation (1), that is...

[0097]

[0098] Therefore, according to f target (t) to fit f(s,x) eg This allows the planned trajectory to have a similar trend to the manually demonstrated trajectory, which involves solving the following optimization problem:

[0099]

[0100] Using the locally weighted regression (LWR) method, the solution to the above equation can be obtained as follows:

[0101]

[0102] in,

[0103]

[0104] Therefore, for the recorded manual demonstration trajectory (k is the spatial dimension; k = 2 when the redundant spatial robotic arm moves on a plane and k = 3 when it moves in three-dimensional space). According to equations (5) to (7), the corresponding kernel function weight coefficients and forcing functions are solved respectively, thus generating a robot that has the same trend as the manual demonstration but can adapt to different task objectives x. eg New trajectory

[0105] Step 102: Based on the desired trajectory generator, construct the desired trajectory tracking controller to obtain the desired trajectory tracking speed when there are no obstacles.

[0106] Specifically, an end-effector desired trajectory tracking controller of the following form is constructed to achieve end-effector desired trajectory tracking of a redundant space robot arm when there are no obstacles in the working environment:

[0107]

[0108] in, The tracking speed of the robotic arm's end effector in the absence of obstacles; x ed and The desired pose and velocity of the redundant spatial manipulator end effector generated by the end effector desired trajectory generator; x e and This represents the actual end-effector pose and velocity.

[0109] Step 103: Based on the mapping of end-effector motion and joint motion of the redundant space manipulator, the end-effector desired trajectory generator and the end-effector desired trajectory tracking controller, construct the relaxation null space motion strategy of the redundant space manipulator to obtain the joint null space motion vector and the end-effector relaxation motion vector.

[0110] Specifically, based on the terminal relaxation characteristics, the actual terminal velocity is expressed as:

[0111]

[0112] in, The tracking speed of the robotic arm end effector in the absence of obstacles; To account for the allowable deviation based on the terminal relaxation characteristics;

[0113] Based on the mapping relationship between the end-effector motion and joint motion of the redundant space manipulator, a relaxation null space motion strategy for the redundant space manipulator is constructed, which is represented by the following mapping relationship:

[0114]

[0115] in, Let J be the joint angular velocity of the redundant space manipulator, and J be the Jacobian matrix of the redundant space manipulator. + Let J be the pseudo-inverse matrix, and let I be the identity matrix. The null space motion vector of the redundant space robotic arm. For the end-point trajectory tracking component, To relax the zero-space obstacle avoidance motion components.

[0116] Step 104: Based on the actual joint angles, actual joint angular velocities, relative vectors between the links and obstacles, joint zero-space motion vectors, and end-effector relaxation motion vectors of the redundant space manipulator, construct the state variables and motion variables of the redundant space manipulator.

[0117] Specifically, firstly, the state variables s of the redundant space robot arm at time t are constructed. t for:

[0118]

[0119] Where q(t) and Let p0(t) and p be the joint angle vector and joint angular velocity vector at time t, respectively.i (t)(i=1~n) represent the distance vectors between the base, link i and the obstacle, respectively;

[0120] Secondly, construct the motion variable a of the redundant spatial manipulator at time t. t for:

[0121]

[0122] in, The null space motion vector of the redundant space robotic arm. This is the end-effector relaxation vector.

[0123] Step 105: Construct the obstacle avoidance reward function for the redundant space robotic arm based on the end-effector relaxation motion vector.

[0124] Specifically, construct the obstacle avoidance reward function r for the redundant space robotic arm at time t. t for:

[0125] r t =λ o r o +λ s r s

[0126] Where, r o For the obstacle avoidance feedback term, its expression is:

[0127]

[0128] Where, p i Let d be the distance vector between link i of the redundant space robotic arm and the obstacle. s The safe distance between the manually set redundant space robotic arm and obstacles, r s The stability term is expressed as follows:

[0129]

[0130] in, End-effector relaxation vector The i-th dimension component (i = 1 to s), λ is the maximum allowed end-effector relaxation motion vector in this dimension. o and λ s r o and r s The weighting coefficients thus determine the total reward value r t ∈[0,1].

[0131] Step 106: Based on the state variables, action variables, and obstacle avoidance reward function of the redundant space manipulator, construct a relaxed zero-space obstacle avoidance motion training strategy to obtain the trained redundant space manipulator relaxed zero-space obstacle avoidance motion planning strategy network.

[0132] Specifically, firstly, the network structure required for training the zero-space obstacle avoidance motion of the redundant space manipulator is constructed based on the SAC algorithm. This structure includes the redundant space manipulator relaxation zero-space obstacle avoidance planning strategy network π. θ Two main Q networks and Two-target Q-network and Among them, the policy network π θ The state variables of the redundant space manipulator are used as input, and the motion variables of the redundant space manipulator are used as output; main Q-network and the target Q network Both take the state variables and motion variables of the redundant space robotic arm as inputs and output the Q value corresponding to the state variables and motion variables;

[0133] Secondly, the zero-space obstacle avoidance planning strategy network π for the redundant space robotic arm is planned according to the following process. θ Conduct training:

[0134] Step 1: Specify the initial configuration q of the redundant space robotic arm fix The safe distance d between the redundant space robotic arm and obstacles s The maximum number of rounds k during training num_ep And the longest time t for each round ep_max ;

[0135] Step 2: Create a data resource pool and initialize the policy network π. θ Main Q network and and the target Q network and

[0136] Step 3: Set the current round number k = 1;

[0137] Step 4: Generate a random joint angle Δq, let q init =q fix +Δq, and calculate the initial end-effector pose based on the positive kinematics of the redundant space manipulator.

[0138] Step 5: Select obstacle location p o Make it satisfy the condition 0 < ||p i ||<d s (i = 1 to n, where n is the number of links in the redundant space manipulator);

[0139] Step 6: Set the current time t = 0;

[0140] Step 7: Let the cumulative reward R t =0, and obtain the state variable s of the redundant space robotic arm based on environmental feedback. t ;

[0141] Step 8: Set the state variable s of the redundant space robot arm at the current moment. t Input Policy Network π θ Obtain the motion variable a of the redundant space robotic arm. t ;

[0142] Step 9: Based on a t Drive the redundant space robotic arm to obtain the state variable s at time t+1. t+1 And calculate the obstacle avoidance reward function r at time t. t Add it to the cumulative reward R t middle;

[0143] Step 10: Based on the reinforcement learning SAC algorithm, sample training data from the data resource pool and update the main Q network. and Target Q network and and policy network π θ Network parameters;

[0144] Step 11: If ||p i ||>d s Or ||p i If || = 0, exit the current round; otherwise, skip this step.

[0145] Step 12: Let t = t + 1, s t =s t+1 ;

[0146] Step 13: If t≤t max Return to step 8; otherwise, skip this step.

[0147] Step 14: Let k = k + 1;

[0148] Step 15: If k < k num_ep Return to step 4; otherwise, skip this step.

[0149] Step 16: End training and obtain the redundant space of the robotic arm to relax the zero-space obstacle avoidance motion planning strategy network.

[0150] Step 107: Based on the trained redundant space manipulator relaxation zero-space obstacle avoidance motion planning strategy network and the end-effector expected trajectory generator based on manual demonstration, obtain the redundant space manipulator relaxation zero-space obstacle avoidance planner.

[0151] Specifically, the following process is used to obtain the relaxation zero-space obstacle avoidance planner for the redundant space manipulator, thereby enabling the relaxed zero-space obstacle avoidance motion of the redundant space manipulator:

[0152] Step 1: Deploy the training-obtained redundant space robotic arm relaxation zero-space obstacle avoidance motion planning strategy network

[0153] Step 2: Set the end-point desired trajectory tracking controller parameter K p K d Set the maximum number of time steps t for a single round. ep_max ;

[0154] Step 3: Obtain the desired trajectory using the terminal desired trajectory generator.

[0155] Step 4: Set the current time t = 0;

[0156] Step 5: Obtain the state variable s of the redundant space robotic arm based on environmental feedback. t ;

[0157] Step 6: Obtain the end-effector tracking speed of the redundant space robotic arm based on the desired end-effector trajectory tracking controller.

[0158] Step 7: Set the state variable s of the redundant space robot arm at the current moment. t Input Policy Network Obtain the motion variable a of the redundant space robotic arm t ;

[0159] Step 8: If Then a t Drive the redundant space robot arm; otherwise, follow the tracking speed of the redundant space robot arm's end effector when there are no obstacles. Drive redundant space robotic arms;

[0160] Step 9: Obtain the state variable s of the redundant space robot arm at time t+1. t+1 ;

[0161] Step 10: Let t = t + 1, s t =s t+1 ;

[0162] Step 11: If t≤t max Return to step 6; otherwise, skip this step.

[0163] Step 15: End planning and complete the zero-space obstacle avoidance motion planning for the redundant space robotic arm.

[0164] Based on the method provided in the embodiments of the present invention, a simulation experiment was conducted on a planar spatial robotic arm with a floating base and a four-degree-of-freedom robotic arm. Please refer to... Figure 2 This is a planar four-degree-of-freedom redundant spatial manipulator model. It only constrains the position of the end effector in the x and y directions in inertial frame I, but does not constrain the attitude of the end effector in the inertial frame. Therefore, the end effector has 3 degrees of freedom and 2 constraints, that is, it has 1 redundant degree of freedom. The DH parameters of this redundant spatial manipulator are shown in Table 1, and the dynamic parameters are shown in Table 2.

[0165] Table 1 Redundant Space Robotic Arm DH Parameter Table

[0166]

[0167] Table 2. Dynamic parameters of redundant space robotic arm

[0168]

[0169] Table 3 Hyperparameters used in the training process

[0170]

[0171] A simulation environment was built in the PyGame module based on the OpenAI Gym framework. The hyperparameters used for training are shown in Table 3, and the technical solution of this embodiment was used for training. The training process lasted for 600 epochs, with each epoch containing 20 rounds, and each round lasting a maximum of 600 time intervals. Once the distance between the obstacle and the redundant space robot arm is greater than the safe distance, or the redundant space robot arm collides with the obstacle, the round will terminate and proceed to the next round. Twenty rounds were set between each epoch for testing to verify the learning effect during the training process. The average cumulative reward obtained in the test after each epoch is shown in Table 3. Figure 3 As shown in the figure, after 500 epochs of training, the cumulative reward gradually converges, the average cumulative reward changes gradually flattening, and the difference between the maximum and minimum values ​​decreases, indicating that the redundant space robotic arm relaxes the zero-space obstacle avoidance planning strategy network. I have gradually completed my studies.

[0172] Extract the policy network after training. Verify it in a test environment, such as Figure 4As shown. In this environment, the end effector of the redundant space robotic arm needs to track a specified dark solid line desired trajectory (its endpoint is the center of the small circle pointed to by the dark solid line arrow), while avoiding light-colored obstacles (i.e., the large circle in the figure) moving in the direction of the light dashed line arrow. From Figure 4 It can be seen that when the distance between the obstacle and the redundant space robotic arm is less than the safe distance, the policy network... The end effector was driven to deviate from the desired trajectory, maintaining a relatively safe distance from the obstacle without contact or collision. Once the obstacle was away, the end effector quickly returned to its original desired trajectory. This demonstrates that the redundant spatial manipulator successfully avoided the obstacle, and the end effector's position deviated within an acceptable range during this process. Therefore, the effectiveness of the obstacle avoidance planning method for the redundant spatial manipulator based on relaxed null space is proven.

[0173] The technical solutions of the embodiments of the present invention have the following beneficial effects:

[0174] This paper introduces end-effector relaxation into a redundant space manipulator and uses reinforcement learning theory to construct the state variables, action variables, and reward function required for the relaxed null space obstacle avoidance motion planning strategy network. A relaxed null space obstacle avoidance motion training strategy is designed, thus forming a redundant space manipulator obstacle avoidance planning method based on relaxed null space. This method enables the redundant space manipulator to effectively avoid obstacles under the condition that the end-effector has an allowable deviation range. It also has good adaptability to unstructured dynamic environments and sufficient computational efficiency.

[0175] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0176] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A method for obstacle avoidance planning of a redundant space robotic arm based on a relaxed null space, characterized in that, The method includes: Step S1: Construct an end-effector expected trajectory generator based on manual demonstration to obtain the generated redundant space robot arm end-effector expected pose and velocity; Step S2 constructs an end-expected trajectory tracking controller based on the end-expected trajectory generator to obtain the end-expected trajectory tracking speed when there are no obstacles; Step S3 constructs a relaxation null space motion strategy for the redundant space manipulator based on the mapping between the end-effector motion and joint motion, the end-effector desired trajectory generator, and the end-effector desired trajectory tracking controller, and obtains the joint null space motion vector and the end-effector relaxation motion vector. Step S4 constructs the state variables and motion variables of the redundant space manipulator based on the actual joint angles, actual joint angular velocities, relative vectors between the link and the obstacle, joint zero-space motion vectors, and end-effector relaxation motion vectors. Step S5 constructs the obstacle avoidance reward function for the redundant space robotic arm based on the end-effector relaxation motion vector; Step S6 constructs a relaxed zero-space obstacle avoidance motion training strategy based on the redundant space manipulator's state variables, motion variables, and obstacle avoidance reward function, and obtains the trained redundant space manipulator relaxed zero-space obstacle avoidance motion planning strategy network. Step S7: Based on the trained redundant space manipulator relaxation zero-space obstacle avoidance motion planning strategy network and the end-effector expected trajectory generator based on manual demonstration, obtain the redundant space manipulator relaxation zero-space obstacle avoidance motion planner. Step S1 includes: Based on the dynamic motion primitive method, the trajectory of the redundant spatial manipulator's end effector is represented as the following second-order nonlinear dynamic system model: in, To achieve a scaling factor that scales the trajectory proportionally to different time scales, and Given system constants, For the end effector trajectory of a redundant space robotic arm One of the components, for The first derivative, for The second derivative, The component of the trajectory's target position in this dimension. The forcing function introduced to the nonlinear dynamic characteristics of the second-order nonlinear dynamic system is expressed as: in, for The starting position, For the number Gaussian kernel function, The range is , The number of kernel functions. for The weighting coefficients, For the state variables of a first-order linear system; Step S2 includes: Construct an end-effector desired trajectory tracking controller of the following form to achieve end-effector desired trajectory tracking of a redundant space robot arm when there are no obstacles in the working environment: in, The tracking speed of the robotic arm end effector in the absence of obstacles is the redundancy space. and The redundant spatial desired pose and velocity of the robotic arm end effector generated by the desired end effector trajectory generator. and This represents the actual end-effector pose and velocity.

2. The method according to claim 1, characterized in that, Step S3 includes: Based on the terminal relaxation characteristics, the actual terminal velocity is expressed as: in, The tracking speed of the robotic arm end effector in the absence of obstacles is the redundancy space. This is the end-effector relaxation vector; Based on the mapping relationship between the end-effector motion and joint motion of the redundant space manipulator, a relaxation null space motion strategy for the redundant space manipulator is constructed, which is represented by the following mapping relationship: in, For the angular velocity of the redundant space robotic arm joints, For the Jacobian matrix of the redundant space manipulator, for The pseudo-inverse matrix, It is the identity matrix. The tracking speed of the robotic arm end effector in the absence of obstacles is the redundancy space. For the end-effector relaxation vector, The null space motion vector of the redundant space robotic arm. For the end-point trajectory tracking component, To relax the zero-space obstacle avoidance motion components.

3. The method according to claim 1, characterized in that, Step S4 includes: Step S4.1 Construction The state variables of the time-redundant space robot arm for: in, and They represent The joint angle vector and joint angular velocity vector at time t. , Representing the base and connecting rod respectively The distance vector between the relative position and the obstacle. Subscript The range is ; Step S4.2 Construction Motion variables of a time-redundant spatial robotic arm for: in, The null space motion vector of the redundant space robotic arm. This is the end-effector relaxation vector.

4. The method according to claim 1, characterized in that, Step S5 includes: structure Obstacle avoidance reward function for a time-redundant spatial robotic arm for: in, For the obstacle avoidance feedback term, its expression is: in, For redundant space robotic arm links The distance vector between the relative position and the obstacle. A manually set safety distance between the redundant space robotic arm and obstacles. The stability term is expressed as follows: in, For the end-effector relaxation vector The dimensional components, where , This represents the maximum allowed end-effector relaxation motion vector in this dimension. and They are respectively and The weighting coefficients thus affect the total reward value. .

5. The method according to claim 1, characterized in that, Step S6 includes: Step S6.1 Constructs the reinforcement learning SAC algorithm network structure required for the relaxed zero-space obstacle avoidance motion training strategy, which includes a redundant space robotic arm relaxed zero-space obstacle avoidance planning strategy network. Two main Q networks and Two target Q-networks and Among them, policy network The state variables of the redundant space manipulator are used as input, and the motion variables of the redundant space manipulator are used as output; main Q-network , and the target Q network , Both take the state variables and motion variables of the redundant space robotic arm as inputs and output the Q value corresponding to the state variables and motion variables; Step S6.2 Relax the zero-space obstacle avoidance planning strategy network for the redundant space robotic arm according to the following process. Conduct training: Step S6.2.1 specifies the initial configuration of the redundant space robot arm. Safe distance between redundant space robotic arm and obstacles Maximum number of rounds during training and the longest time of each round ; Step S6.2.2 Create a data resource pool and initialize the policy network. Main Q network and and the target Q network and ; Step S6.2.3 Set the current round number ; Step S6.2.4 Generate random joint angles ,make The initial end-effector pose is calculated based on the positive kinematics of the redundant space manipulator. ; Step S6.2.5 Select obstacle location To make it meet the conditions ,in Subscript The range is , The number of links in a redundant space robotic arm; Step S6.2.6 Set the current time... ; Step S6.2.7: Accumulate rewards And obtain the state variables of the redundant space robotic arm based on environmental feedback. ; Step S6.2.8: Change the state variables of the redundant space robot at the current moment. Input Policy Network Obtain the motion variables of the redundant space robotic arm. ; Step S6.2.9 according to Drive redundant space robotic arms to obtain State variables at time 1 and calculate Real-time obstacle avoidance reward function Add it to the cumulative rewards middle; Step S6.2.10: Based on the reinforcement learning SAC algorithm, sample training data from the data resource pool and update the main Q network. and Target Q network and and policy networks Network parameters; Step S6.2.11 If or If yes, exit the current round; otherwise, skip this step. Step S6.2.12 , ; Step S6.2.13 If If the condition is not met, return to step S6.2.8; otherwise, skip this step. Step S6.2.14 ; Step S6.2.15 If If the condition is met, return to step S6.2.4; otherwise, skip this step. Step S6.2.16: End training and obtain the trained redundant space robotic arm relaxation zero-space obstacle avoidance motion planning strategy network. .

6. The method according to claim 1, characterized in that, Step S7 includes: Step S7.1 Deploy the redundancy space robotic arm relaxation zero-space obstacle avoidance motion planning strategy network obtained after training. ; Step S7.2 Set the parameters of the end-point desired trajectory tracking controller , Set the maximum number of time steps in a single round. ; Step S7.3 Obtain the desired trajectory using the terminal desired trajectory generator. ; Step S7.4 Set the current time... ; Step S7.5 Obtain the state variables of the redundant space robotic arm based on environmental feedback. ; Step S7.6 Obtain the end-effector tracking speed of the redundant space robotic arm based on the desired end-effector trajectory tracking controller. ; Step S7.7: Change the state variables of the redundant space robot at the current moment. Input Policy Network Obtain the motion variables of the redundant space robotic arm. ; Step S7.8 If Then use Drive the redundant space robot arm; otherwise, follow the tracking speed of the redundant space robot arm's end effector when there are no obstacles. Drive redundant space robotic arms; Step S7.9 Obtain The state variables of the time-redundant space robot arm ; Step S7.10 , ; Step S7.11 If Return to step S7.6; otherwise, skip this step. Step S7.12 concludes the above steps and completes the motion planning for the relaxation zero-space obstacle avoidance of the redundant space robotic arm.

Images