A sac-based dual-arm robot tight coordination carrying motion planning method

Abstract

The application provides a SAC-based dual-arm robot tight coordination carrying motion planning method, comprising the following steps: constructing a kinematics model of a dual-arm robot tight coordination system; dividing active joints and passive joints of the dual-arm robot; constructing a distance vector for describing a distance between an operation object position and a target position; constructing a singularity index for describing whether a mechanical arm configuration singularity occurs in the tight coordination system; constructing state variables and action variables of the dual-arm robot; constructing a motion planning reward function of the dual-arm robot; constructing a deep network structure of SAC; constructing a tight coordination carrying motion training strategy according to the state variables, the action variables, the motion planning reward function of the dual-arm robot and the deep network structure, and obtaining a trained dual-arm robot tight coordination carrying motion planning strategy network. According to the technical scheme provided in the embodiment of the application, the dual-arm robot tight coordination carrying can be realized under the condition that the initial position and the target position of the operation object are uncertain.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of dual-arm robot motion planning, and relates to a dual-arm robot tight coordination carrying motion planning method based on SAC. BACKGROUND

[0002] At present, the application research and development of robots in different fields are rapidly developing, such as industrial manufacturing robots, space robots, surgical robots and the like. However, with the progress of science and technology, the manufacturing demand begins to change, and the defects such as low load capacity, single operation ability and insufficient workspace of traditional single-arm robots cannot meet the new complex manufacturing demand, so the dual-arm collaborative robot with more flexible system structure and control mode enters the field of view of researchers. The dual-arm collaborative robot is often used to perform collaborative assembly, welding, carrying of large or heavy payloads and the like. Among them, the stable clamping and carrying of objects by the end effector of the dual-arm robot is still considered as one of the main challenges in the field of dual-arm robot research.

[0003] When the dual-arm robot cooperatively carries an object, the dual arms and the object are in a tight coordination relationship, forming closed kinematic and dynamic constraints. During the carrying process, the tight contact between the end effector of the dual-arm robot and the object needs to be maintained to the target position, so that there is a complex coupling relationship between the joints of the dual-arm robot tight coordination system and the operated object, which leads to the difficulty in carrying out the carrying task.

[0004] The current dual-arm robot tight coordination carrying motion planning methods mainly include operation space planning and joint space planning. The operation space planning first defines a certain operation object motion trajectory in the operation space, and then calculates the end effector motion trajectory according to the pose constraint between the end effector of the dual-arm robot and the operation object. The operation space planning is simple to calculate, but does not consider the constraints of the dual-arm robot tight coordination system, which is easy to cause the singularity of the tight coordination system. The joint space planning plans the dual-arm motion in the joint space of the dual-arm robot, and finds the joint trajectory that satisfies the constraints of the tight coordination system, which makes up for the shortcomings of the operation space planning. However, the existing motion planning methods need to be planned for fixed tasks and environments, and when facing uncertain operation object target positions, the planning needs to be re-performed, which is time-consuming and laborious. Therefore, a new dual-arm robot tight coordination carrying motion planning method is needed to perform real-time motion planning under the condition that the target position of the operation object is uncertain, so as to avoid the complex and time-consuming multiple motion planning process. SUMMARY

[0005] Therefore, the application provides a dual-arm robot tight coordination carrying motion planning method based on SAC, so as to perform real-time tight coordination carrying motion planning of the dual-arm robot under the condition that the target position of the operation object is uncertain.

[0006] The application provides a SAC-based dual-arm robot tight coordination carrying motion planning method, comprising:

[0007] Step S1 constructs a kinematics model of a dual-arm robot tight coordination system according to constraints formed between two mechanical arms and an operation object in a dual-arm robot tight coordination carrying process;

[0008] Step S2 divides active joints and passive joints of the dual-arm robot;

[0009] Step S3 constructs a distance vector for describing a distance between a position of the operation object and a target position;

[0010] Step S4 constructs a singularity index for describing whether a mechanical arm configuration singularity occurs in the tight coordination system according to a distance between an end effector of the dual-arm robot and the operation object in the carrying process;

[0011] Step S5 constructs state variables and action variables of the dual-arm robot according to a joint angle vector of the dual-arm robot, a joint angular velocity vector, the distance vector and the singularity index;

[0012] Step S6 constructs a motion planning reward function of the dual-arm robot according to the distance vector and the singularity index;

[0013] Step S7 constructs a deep network structure of SAC;

[0014] Step S8 constructs a tight coordination carrying motion training strategy according to the state variables, the action variables, the motion planning reward function of the dual-arm robot and the deep network structure, and obtains a trained dual-arm robot tight coordination carrying motion planning strategy network.

[0015] In the method, the step S1 comprises:

[0016] The kinematics model of the dual-arm robot tight coordination system is constructed as:

[0017]

[0018] wherein, is an operation object velocity, J lo is a Jacobian matrix of a motion chain composed of the left arm and the end effector, is a Jacobian matrix of a motion chain composed of the right arm and the end effector, θ、 are joint angles of the left arm and the right arm, respectively, are joint angular velocities of the left arm and the right arm, respectively.

[0019] In the method, the step S2 comprises:

[0020] The number of joints of the left arm and the right arm of the dual-arm robot is n, and i=1~n represents joint 1~joint n of the left arm, and j=1~n represents joint 1~joint n of the right arm. The 2n joints of the left arm and the right arm are divided into active joints and passive joints according to the following rules:

[0021]

[0022] Wherein, q a is the joint angle of the active joint, q p is the joint angle of the active passive joint, θ1~θ i represents that joints 1~i of the left arm are divided into active joints, φ1~φ j represents that joints 1~j of the right arm are divided into active joints, θ i+1 ~θ n represents that joints i+1~n of the left arm are divided into passive joints, φ j+1 ~φ n represents that joints j+1~n of the right arm are divided into passive joints, [] T represents the transpose of a matrix.

[0023] In the above method, the step S3 comprises:

[0024] Let the position of the operating object be p e , the target position be p g , and construct a distance vector d eg for describing the distance between the operating object position and the target position:

[0025] d eg =p e -p g .

[0026] In the above method, the step S4 comprises:

[0027] According to the maximum distance between the two end effectors and the operating object, a singularity index d Singular for describing whether the tight coordination system is singular is constructed:

[0028]

[0029] Wherein, p le , p re are the positions of the end effectors of the left arm and the right arm, TH s is the threshold value of the distance between the two end effectors of the two arms of the dual-arm robot and the operating object, and when the maximum distance between the two end effectors and the operating object is greater than the threshold value, the dual-arm robot is in a singular state.

[0030] In the above method, the step S5 comprises:

[0031] Step S5.1 constructs the state variable of the dual-arm robot at time t as:

[0032]

[0033] where q a and q p are the active joint angle vector and passive joint angle vector of the dual-arm robot respectively, are the active joint angular velocity vector and passive joint angular velocity vector of the dual-arm robot respectively, d eg is the distance vector between the operating object position and the target position, and d Singular is a singularity index used to describe whether the tight coordination system is singular.

[0034] Step S5.2 constructs the action variable of the dual-arm robot at time t as:

[0035]

[0036] where, is the active joint angular velocity vector of the dual-arm robot.

[0037] In the above method, the step S6 includes:

[0038] Step S6 constructs the motion planning reward function of the dual-arm robot at time t as:

[0039] r t = r eg + r Singular

[0040] where r eg is a distance feedback term between the operating object position and the target position, and is defined as:

[0041] r eg = -||d eg ||

[0042] where d eg is the distance vector between the operating object position and the target position; and r Singular is a singularity index feedback term of the tight coordination system, and is defined as:

[0043]

[0044] where d Singular is a singularity index used to describe whether the tight coordination system is singular.

[0045] In the above method, the step S7 includes:

[0046] Step S7 builds a deep network structure of the SAC, which includes a policy network π θtwo main Q networks and two target Q networks and wherein the policy network π θ takes the state variable s t of the dual-arm robot as input and outputs the action variable a t of the dual-arm robot; the main Q network and the target Q network both take the state variable s t and the action variable a t of the dual-arm robot as input and output the Q value corresponding to s t and a t .

[0047] In the above method, the step S8 comprises:

[0048] Step S8.1 assigns the joints of the dual-arm robot as the main joint q a and the passive joint q p , specifies the initial position p e0-initial of the operated object, the target position p g-initial of the operated object, the position threshold TH d of the operated object near the target position, the maximum time step n max of each round in the training process, and the maximum total time step t max ;

[0049] Step S8.2 creates a data resource pool and initializes the policy network π θ , the main Q network and , and the target Q network and

[0050] Step S8.3 sets the current total time step t = 1;

[0051] Step S8.4 samples random noise from a uniform distribution and adds it to the initial position and the target position of the operated object to generate the random initial position p e0 and the random target position p g of the operated object:

[0052] p e0 = p e0-initial + [U(-0.1,0.1), U(-0.1,0.1), U(-0.1,0.1)]

[0053] p g = p g-initial + [U(-0.2,0.2), U(-0.2,0.2), U(-0.2,0.2)]

[0054] Step S8.5 Set the time step n = 1 in the current round;

[0055] Step S8.6: Set the current state variable s of the dual-arm robot. t Input Policy Network π θ Obtain the motion variable 'a' of the dual-arm robot. t ;

[0056] Step S8.7 Based on the tightly coordinated kinematic model of the dual-arm robot, by a t Calculate the angular velocity vector of the passive joints of the dual-arm robot at the current moment.

[0057] Step S8.8 according to a t and Drive the dual-arm robot and obtain the state variable s at time t+1. t+1 and the reward function value r of exercise planning t+1 Save it to the data resource pool;

[0058] Step S8.9: According to the SAC algorithm, sample training data from the data resource pool and update the main Q network. and Target Q network and and policy network π θ Network parameters;

[0059] Step S8.10 Let n ← n+1;

[0060] Step S8.11 Let t ← t+1;

[0061] Step S8.12 If the distance vector d between the position of the manipulated object and the position of the target object... eg >TH d And n≤n max Return to step S8.6; otherwise, skip this step.

[0062] Step S8.13 Calculate the cumulative reward

[0063] Step S8.14 If t < t max If the condition is not met, return to step S8.4; otherwise, skip this step.

[0064] Step S8.15 concludes the training, obtaining the trained dual-arm robot tightly coordinated handling motion planning strategy network π. θ . [Attached Image Description]

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described below only show some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative and laborious work.

[0066] Figure 1 is a flowchart of the SAC-based dual-arm robot tight coordination carrying motion planning method provided by the embodiments of the present application;

[0067] Figure 2 is a schematic diagram of the dual-arm robot tight coordination carrying scene used in the simulation experiment in the embodiments of the present application;

[0068] Figure 3 is a cumulative reward curve graph recorded by testing the policy network in the training process in the embodiments of the present application;

[0069] Figure 4 is a motion trajectory graph of the operation object in the dual-arm robot tight coordination carrying process recorded by testing the policy network after the training in the embodiments of the present application.

Specific embodiments

[0070] In order to better understand the technical solutions of the present application, the embodiments of the present application will be described in detail below with reference to the drawings.

[0071] It should be clear that the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0072] The embodiments of the present application give a SAC-based dual-arm robot tight coordination carrying motion planning method, please refer to Figure 1 , which is a flowchart of the SAC-based dual-arm robot tight coordination carrying motion planning method provided by the embodiments of the present application, as shown in Figure 1 , the method comprises the following steps:

[0073] Step 101, according to the constraints formed between the two mechanical arms and the operation object in the dual-arm robot tight coordination carrying process, a kinematics model of the dual-arm robot tight coordination system is constructed.

[0074] Specifically, the kinematics model of the dual-arm robot tight coordination system is constructed as follows:

[0075]

[0076] wherein, is the velocity of the operation object, Jlo (θ) is a Jacobian matrix of a kinematic chain composed of the left arm and the end effector, is a Jacobian matrix of a kinematic chain composed of the right arm and the end effector, θ, are joint angles of the left arm and the right arm, respectively, are joint angular velocities of the left arm and the right arm, respectively.

[0077] Step 102, dividing the active joints and passive joints of the dual-arm robot.

[0078] Specifically, the number of joints of the left arm and the right arm of the dual-arm robot is n, and i = 1 ~ n represents the left arm joint 1 ~ joint n, and j = 1 ~ n represents the right arm joint 1 ~ joint n. The 2n joints of the left arm and the right arm are divided into active joints and passive joints according to the following rules:

[0079]

[0080] wherein q a is an active joint angle, q p is an active passive joint angle, θ1~θ i represents that the joints 1~i of the left arm are divided into active joints, φ1~φ j represents that the joints 1~j of the right arm are divided into active joints, θ i+1 ~ θ n represents that the joints i+1~n of the left arm are divided into passive joints, φ j+1 ~ φ n represents that the joints j+1~n of the right arm are divided into passive joints, [] T represents the transpose of a matrix.

[0081] Step 103, constructing a distance vector for describing the distance between the operating object position and the target position.

[0082] Specifically, the operating object position is p e , the target position is p g , and a distance vector d eg for describing the distance between the operating object position and the target position is constructed:

[0083] d eg = p e - p g .

[0084] Step 104, constructing a singularity index for describing whether the mechanical arm configuration singularity of the tight coordination system occurs according to the distance between the end effector of the dual-arm robot and the operating object in the carrying process.

[0085] Specifically, a singularity index d is constructed to describe whether the tight coordination system is singular according to the maximum distance between the two end effectors and the object Singular :

[0086]

[0087] where p le , p re are the positions of the left arm and the right arm end effectors respectively, TH s is a threshold value of the distance between the two end effectors of the dual-arm robot and the object, and when the maximum distance between the two end effectors and the object is greater than the threshold value, the dual-arm robot is in a singular state.

[0088] Step 105, constructing the state variable and the action variable of the dual-arm robot according to the joint angle vector, the joint angular velocity vector, the distance vector and the singularity index of the dual-arm robot.

[0089] Specifically, first, the state variable of the dual-arm robot at time t is constructed as follows:

[0090]

[0091] where q a , q p are the active joint angle vector and the passive joint angle vector of the dual-arm robot respectively, are the active joint angular velocity vector and the passive joint angular velocity vector of the dual-arm robot respectively, d eg is the distance vector between the object position and the target position, and d Singular is the singularity index describing whether the tight coordination system is singular.

[0092] Secondly, the action variable of the dual-arm robot at time t is constructed as follows:

[0093]

[0094] where, is the active joint angular velocity vector of the dual-arm robot.

[0095] Step 106, constructing the motion planning reward function of the dual-arm robot according to the distance vector and the singularity index.

[0096] Specifically, the motion planning reward function of the dual-arm robot at time t is constructed as follows:

[0097] r t = r eg + r Singular

[0098] where r egis a distance feedback term between the operation object position and the target position, which is defined as:

[0099] r eg = -||d eg ||

[0100] where d eg is a distance vector between the operation object position and the target position; r Singular is a singularity index feedback term of the tight coordination system, which is defined as follows:

[0101]

[0102] where d Singular is a singularity index used to describe whether the tight coordination system is singular.

[0103] Step 107, constructing a deep network structure of SAC.

[0104] Specifically, the deep network structure of SAC is built, which includes a policy network π θ , two main Q networks and and two target Q networks and wherein the policy network π θ takes the state variable s t of the dual-arm robot as input and takes the action variable a t of the dual-arm robot as output; the main Q network and the target Q network both take the state variable s t and the action variable a t of the dual-arm robot as input, and take the Q value corresponding to s t and a t as output.

[0105] Step 108, constructing a tight coordination carrying motion training strategy according to the state variable, the action variable, the motion planning reward function of the dual-arm robot and the deep network structure, and obtaining a trained dual-arm robot tight coordination carrying motion planning strategy network.

[0106] Specifically, the dual-arm robot tight coordination carrying motion planning network is trained according to the following process:

[0107] Step 1: assign the joints of the dual-arm robot as active joints q a and passive joints q p , specify the initial position p e0-initial of the operation object, the target position p g-initial of the operation object, and the position threshold TH d, the maximum time step n of each round in the training process max and the maximum total time step t max ;

[0108] Step 2: Create a data resource pool and initialize the policy network π θ , the main Q network and and the target Q network and

[0109] Step 3: Let the current total time step t = 1;

[0110] Step 4: Sample random noise from a uniform distribution and add it to the initial position and target position of the manipulator to generate a random initial position p e0 and a random target position p g of the manipulator:

[0111] p e0 = p e0-initial +[U(-0.1,0.1),U(-0.1,0.1),U(-0.1,0.1)]

[0112] p g = p g-initial +[U(-0.2,0.2),U(-0.2,0.2),U(-0.2,0.2)];

[0113] Step 5: Let the current time step n within the round be 1;

[0114] Step 6: Input the state variable s t of the dual-arm robot at the current time to the policy network π θ to obtain the action variable a t of the dual-arm robot;

[0115] Step 7: According to the kinematics model of the dual-arm robot tight coordination system described in step 101 Rewrite the right half of the equation as:

[0116]

[0117] where J lo (θ) is the Jacobian matrix of the kinematic chain composed of the left arm and the end effector, is the Jacobian matrix of the kinematic chain composed of the right arm and the end effector, θ, are the joint angles of the left arm and the right arm, respectively, are the joint angular velocities of the left arm and the right arm, respectively.

[0118] According to the active joint q a and the passive joint qp The above formula can be rewritten as

[0119]

[0120] wherein, is the Jacobian matrix corresponding to the active joint, is the Jacobian matrix corresponding to the passive joint.H p If the passive joint velocity is reversible, it can be solved as:

[0121]

[0122] According to the above formula, from a t The joint angular velocity vector of the passive joint of the dual-arm robot at the current time is calculated

[0123] Step 8: According to a t and Drive the dual-arm robot to obtain the state variable s t+1 and the motion planning reward function value r t+1 at time t+1, and save it to the data resource pool;

[0124] Step 9: According to the SAC algorithm, sample training data from the data resource pool, and update the network parameters of the main Q network and the target Q network and and the policy network π θ ;

[0125] Step 10: Let n←n+1;

[0126] Step 11: Let t←t+1;

[0127] Step 12: If the distance vector d eg between the operating object position and the target position is greater than TH d , and n≤n max , return to step S8.6, otherwise skip this step;

[0128] Step 13: Calculate the cumulative reward

[0129] Step 14: If t max , return to step S8.4, otherwise skip this step;

[0130] Step 15: End the training, and obtain the trained dual-arm robot tight coordination carrying motion planning strategy network π θ .

[0131] The above method provided by the embodiment of the application is used for simulation experiment on a dual-arm robot with 7 degrees of freedom for each of the left arm and the right arm. Please refer to Figure 2 , wherein the black hollow circle represents a left arm joint, the black solid line connecting the black hollow circle represents a left arm link, the light gray hollow circle represents a right arm joint, the light gray solid line connecting the light gray hollow circle represents a right arm link, the end effector of the dual-arm robot jointly holds a gradually gray spherical operation object, the white solid point is the mass center of the operation object, and the black solid point is a target point to be reached by the mass center of the operation object. The D-H parameters of the left arm of the dual-arm robot are shown in Table 1, and the dynamic parameters are shown in Table 2. The right arm of the dual-arm robot is identical to the left arm.

[0132] Table 1 D-H parameters of the left arm of the dual-arm robot

[0133]

[0134] Table 2 Dynamic parameters of the left arm of the dual-arm robot

[0135]

[0136] Table 3 Parameter values in the training process

[0137]

[0138] A simulation environment is built in a 64-bit Ubuntu 20.04 operating system, and the last three joints of the left arm and the right arm of the dual-arm robot are selected as active joints, that is, The other parameters used for training are shown in Table 3, and the technical solution of the embodiment of the application is used for training. During the training process, the strategy network π θ is tested every 8 rounds, and the cumulative reward obtained is recorded, please refer to Figure 3 . It can be known from Figure 3 that the cumulative reward obtained gradually increases with the increase of the number of training rounds, indicating that the strategy network π θ gradually learns the operation of completing the tight coordination of carrying by the dual-arm robot; after the time step exceeds 75k, the cumulative reward gradually reduces in the promotion range, and the fluctuation gradually decreases, indicating that the training process of the strategy network π θ is gradually completed.

[0139] The strategy network π θ after training is taken out and placed in a simulation environment with randomly generated initial positions and target positions of the operation object for testing, and the test result is shown in Figure 4 . In Figure 4In the figure, the "X" symbol represents the operation mass center position of each time step, and the dark gray line segment connecting the "X" symbol is the operation mass center motion trajectory. It can be seen that the operation mass center moves a large step in the direction of the target point in the first 4 time steps, and then gradually approaches the target point uniformly and slowly until the distance to the target point is less than 0.1 m. Therefore, it can be seen that the dual-arm robot completes the tight coordination carrying, and thus the correctness and effectiveness of the dual-arm robot tight coordination carrying motion planning method based on SAC are proved.

[0140] The technical scheme of the embodiment of the application has the following beneficial effects:

[0141] According to the tight coordination constraint of the dual-arm robot, the dual-arm robot tight coordination system is modeled, the active joint and the passive joint of the dual-arm robot are divided, the SAC theory is introduced, the state variable, the action variable and the reward function of the dual-arm robot are constructed, the tight coordination carrying motion training strategy is constructed, and thus the dual-arm robot tight coordination carrying motion planning method based on SAC is formed. The real-time motion planning of the dual-arm robot tight coordination carrying can be realized under the condition that the initial position and the target position of the operation mass are both uncertain, so that the dual-arm robot can meet higher real-time application requirements in the tight coordination carrying task.

[0142] The above only describes the preferred embodiment of the application, and is not used to limit the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

[0143] The contents not described in detail in the specification of the application are the known technology of the person skilled in the art.

Claims

1. A SAC-based dual-arm robot tight coordination carrying motion planning method, characterized in that, The method includes: Step S1: Based on the constraints formed between the two robotic arms and the manipulated object during the tightly coordinated handling process of the dual-arm robot, construct the kinematic model of the tightly coordinated system of the dual-arm robot. Step S2: Divide the active joints and passive joints of the dual-arm robot; Step S3: Construct a distance vector to describe the position of the manipulated object and the target position; Step S4: Based on the distance between the end effector of the dual-arm robot and the manipulated object during the handling process, construct a singularity index to describe whether a singular configuration of the robotic arm occurs in the tightly coordinated system. Step S5: Based on the joint angle vector, joint angular velocity vector, distance vector, and singularity index of the dual-arm robot, construct the state variables and motion variables of the dual-arm robot; Step S6: Construct the motion planning reward function for the dual-arm robot based on the distance vector and the singularity index; Step S7: Construct the deep network structure of SAC; Step S8: Based on the state variables, action variables, motion planning reward function, and deep network structure of the dual-arm robot, construct a tightly coordinated handling motion training strategy to obtain the trained tightly coordinated handling motion planning strategy network for the dual-arm robot.

2. The method of claim 1, wherein, Step S1 includes: The kinematic model of the tightly coordinated system of the dual-arm robot is constructed as follows: wherein, is the velocity of the object, is the Jacobian matrix of the kinematic chain formed by the left arm and the end effector, is the Jacobian matrix of the kinematic chain formed by the right arm and the end effector, , are the joint angles of the left arm, right arm, respectively, , are the joint angular velocities of the left arm, right arm, respectively.

3. The method of claim 1, wherein, Step S2 includes: The number of joints in the left and right arms of the dual-arm robot is 1. ,by These represent joints 1 through 2 of the left arm. ,by These represent joints 1 through 2 of the right arm. left arm and right arm together Joints are classified into active and passive joints according to the following rules: wherein, joint angle of an active joint, joint angle of a passive joint, denotes the joint 1 of the left arm is divided into active joints, denotes the joint 1 of the right arm is divided into active joints, denotes the joint 1 of the left arm ~ is divided into passive joints, denotes the joint 1 of the right arm ~ is divided into passive joints, denotes the transpose of a matrix.

4. The method of claim 1, wherein, Step S3 includes: recording the object position as , the target position as , and constructing a distance vector between the object position and the target position 。 5. The method of claim 1, wherein, Step S4 includes: A singularity index for describing whether a tight coordination system is singular is constructed in accordance with a maximum distance between two end effectors and an operating object : wherein, , are the end effector positions of the left and right arms, respectively, is a threshold value for the distance between the two end effectors of the two robot arms and the object to be manipulated, the dual-arm robot being in a singular state when the maximum distance between the two end effectors and the object to be manipulated is greater than the threshold value.

6. The method of claim 1, wherein, Step S5 includes: Step S5.1 constructing The state variables of the time-moment dual-arm robot are: wherein, , are active joint angle vector and passive joint angle vector of the dual-arm robot respectively, , are active joint angular velocity vector and passive joint angular velocity vector of the dual-arm robot respectively, is a distance vector between the operating object position and the target position, is a singularity index describing whether the tightly coordinated system is singular or not. Step S5.2 constructing The action variable of the dual-arm robot at the moment is: wherein, is the active joint angular velocity vector of the dual-arm robot.

7. The method of claim 1, wherein, Step S6 includes: structure The motion planning reward function for the dual-arm robot is: in, The distance feedback term between the position of the manipulated object and the position of the target object is defined as follows: in, This is the distance vector between the position of the manipulated object and the position of the target object; The singularity index feedback term for a tightly coordinated system is defined as follows: in, This is a singularity index used to describe whether a tightly coordinated system is singular.

8. The method according to claim 1, characterized in that, Step S7 includes: Construct a deep network structure for SAC, which includes a policy network. Two main network and Two goals network and Among them, policy network State variables of a dual-arm robot As input, the motion variables of the dual-arm robot As output; main network , and goals network , Both are state variables of a dual-arm robot and action variables As input, with and Corresponding The value is output.

9. The method according to claim 1, characterized in that, Step S8 includes: Step S8.1 Assign joints of the dual-arm robot as active joints and passive joints Specify the initial position of the operand Target position of the manipulated object The position threshold of the manipulated object near the target position The maximum time step per round during training and maximum total time steps ; Step S8.2 Create a data resource pool and initialize the policy network. ,host network and and goals network and ; Step S8.3 Set the current total time step ; Step S8.4 Sample random noise from the uniform distribution and add it to the initial position and target position of the object to generate a random initial position of the object. and random target location : ； Step S8.5 Set the time step for the current round. ; Step S8.6: Change the state variables of the dual-arm robot at the current moment. Input Policy Network Obtain the motion variables of the dual-arm robot ; Step S8.7 Based on the tightly coordinated kinematic model of the dual-arm robot, by Calculate the joint angular velocity vector of the passive joints of the dual-arm robot at the current moment. ; Step S8.8 According to and Drive the dual-arm robot to obtain State variables at time 1 and the reward function value of exercise planning Save it to the data resource pool; Step S8.9: Based on the SAC algorithm, sample training data from the data resource pool and update the master. network and ,Target network and and policy networks Network parameters; Step S8.10: Order ; Step S8.11: Let ; Step S8.12 If the distance vector between the position of the manipulated object and the position of the target object... ,and Return to step S8.6; otherwise, skip this step. Step S8.13 Calculate cumulative rewards ; Step S8.14 If If the condition is not met, return to step S8.4; otherwise, skip this step. Step S8.15: End training and obtain the trained dual-arm robot tightly coordinated handling motion planning strategy network. .

Images