Wheel type mobile robot arm coordination control method, system and device

Abstract

The present application relates to unmanned system multi-agent control technical field, especially related to a kind of wheeled mobile manipulator coordination control method, system and device.It includes the following steps: establishing wheeled mobile manipulator model, solving target accessibility area;Calculate the maneuverability index of wheeled mobile manipulator;Calculate the maximum linear velocity and angular velocity of wheeled mobile manipulator, obtain the velocity ellipsoid model of wheeled mobile manipulator;Design the joint state space of multi-agent, design the joint action space of multi-agent;Control algorithm flow is constructed;Set the task scene of wheeled mobile manipulator coordination control algorithm, set training hyperparameter, design task evaluation index;Training setting is carried out, the success rate of task is recorded at the end of training cycle, the effectiveness of flow.The present application solves the problem of solving kinematic equation or objective function optimization under the condition of complex model structure and high dimension.The present application achieves the technical effect of reducing complexity calculation.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-agent control of unmanned systems, and in particular to a coordinated control method, system and device for a wheeled mobile robotic arm. Background Art

[0002] A wheeled mobile manipulator is a new type of intelligent mobile robot system with active operational capabilities, consisting of one or more manipulators added to a wheeled unmanned platform. Compared to traditional wheeled unmanned platforms, the addition of a manipulator can enhance the terrain-traveling capabilities of wheeled unmanned platforms, help improve their stability, and greatly enhance their autonomy. This makes wheeled unmanned platforms no longer just transportation tools, but an important part of current robotic applications. Currently, the main application scenarios for wheeled mobile manipulators include: home services, factory manufacturing, medical assistance, off-road road passability testing, hazardous object recovery, mine clearance and bomb disposal, search and rescue, disaster relief, spray disinfection, casualty evacuation, combat support, etc.

[0003] While the combination of wheeled ground unmanned platforms and robotic arms presents significant advantages, it also presents new challenges. Among these challenges, a major issue for wheeled mobile robotic arms is achieving coordinated vehicle-arm control to complete operational tasks. Over the past few decades, researchers have conducted extensive research on coordinated vehicle-arm control methods for wheeled mobile robotic arms. However, most of the proposed algorithms are traditional control methods that rely on solving kinematic equations or optimizing objective functions in complex and high-dimensional robot models, resulting in high computational costs. Recent breakthroughs in reinforcement and imitation learning have opened up new possibilities for addressing these issues. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a coordinated control method, system and device for a wheeled mobile manipulator to achieve coordinated control of a wheeled mobile manipulator.

[0005] The present invention provides a coordinated control method for a wheeled mobile robotic arm, comprising the following steps:

[0006] S1: establishing a wheeled mobile manipulator model, and solving a target reachability area according to the wheeled mobile manipulator model;

[0007] S2: Calculating the Jacobian matrix and the maneuverability index of the wheeled mobile manipulator according to the wheeled mobile manipulator model;

[0008] S3: calculating the maximum linear velocity and the maximum angular velocity of the wheeled mobile manipulator according to the Jacobian matrix of the wheeled mobile manipulator, and obtaining a velocity ellipsoid model of the wheeled mobile manipulator;

[0009] S4: constructing a joint state space, a joint action space and a reward function according to the target reachability area, the maneuverability index and the velocity ellipsoid model;

[0010] S5: Constructing a coordinated control algorithm flow of a wheeled mobile manipulator based on the DE-MADDPG algorithm according to the joint state space, the joint action space and the reward function;

[0011] S6: Based on the IsaacSim platform, set the task scenario of the coordinated control algorithm process of the wheeled mobile manipulator, set the training hyperparameters, and design the task evaluation indicators;

[0012] S7: Perform training settings based on the task scenario of the wheeled mobile manipulator coordination control algorithm process, the training hyperparameters and the task evaluation indicators, record the success rate of the task at the end of each training cycle, and judge the effectiveness of the wheeled mobile manipulator coordination control algorithm process.

[0013] According to a coordinated control method for a wheeled mobile robotic arm provided by the present invention, step S1 comprises:

[0014] S11: With the center of mass of the unmanned vehicle platform as the origin, the horizontal direction directly behind the unmanned vehicle platform as the positive direction of the x-axis, and the vertical direction upwards to the ground as the positive direction of the z-axis, the positive direction of the y-axis can form a right-handed axial scanning coordinate system together with the positive directions of the x-axis and the z-axis. The axial scanning coordinate system is a wheeled mobile robotic arm model;

[0015] S12: Find the wheeled mobile manipulator operation manual to obtain the working radius of the wheeled mobile manipulator ;

[0016] S13: According to the z-axis coordinate of the target point , the z-axis coordinate of the first joint center of the wheeled mobile robot Calculate the angle between the line perpendicular to the ground starting from the target point and the line extending from the target to the center of the first joint of the manipulator :

[0017] ;

[0018] S14: Get the distance from the center of mass of the unmanned platform to the first joint of the robotic arm on the x-axis. , calculate the reachability area diameter Size:

[0019]

[0020] The size of the reachability area is the reachability area diameter A sphere with a diameter of .

[0021] According to a coordinated control method for a wheeled mobile robotic arm provided by the present invention, step S2 includes:

[0022] S21: Calculate the Jacobian matrix of the wheeled mobile manipulator according to the wheeled mobile manipulator model :

[0023]

[0024] in, is the end effector position Joint angle The partial derivative, is the degree of freedom of the operating space, is the degree of freedom of the joint space;

[0025] S22: According to the Jacobian matrix of the wheeled mobile manipulator , calculate the operability index

[0026]

[0027] in, represents determinant evaluation, for The transposed matrix of .

[0028] According to a coordinated control method for a wheeled mobile robotic arm provided by the present invention, step S3 includes:

[0029] S31: According to the Jacobian matrix of the wheeled mobile manipulator , The upper part of represents the linear velocity of the end effector of the wheeled mobile robot arm. The lower half of represents the angular velocity of the end effector of the wheeled mobile robot. Translated as:

[0030]

[0031] in, It is the linear velocity part, is the angular velocity part;

[0032] S32: Determine the linear velocity of the end of the wheeled mobile robot :

[0033]

[0034] in, yes The transposed matrix of

[0035] S33: Determine the angular velocity of the end of a wheeled mobile manipulator ,

[0036]

[0037] in, yes The transposed matrix of and Draw the velocity ellipsoid model.

[0038] According to a coordinated control method for a wheeled mobile robotic arm provided by the present invention, step S4 includes:

[0039] S41: Divide the structure of the wheeled mobile robotic arm into two agents: an unmanned vehicle agent and a robotic arm agent.

[0040] According to the reachability region and the velocity ellipsoid model, the joint state space of the wheeled mobile manipulator is constructed as ,in, is the displacement of the autonomous vehicle body relative to the starting position and its yaw in the global frame, are the linear and angular velocities of the joints of the robotic agent, is the position coordinate of the end effector of the robotic arm, is the target position coordinate; , is the set of real numbers;

[0041] S42: Constructing a joint action space of the unmanned vehicle agent and the robotic arm agent based on the reachability region and the velocity ellipsoid model ,in is the action space of the unmanned vehicle agent, including the linear velocity and angular velocity of the unmanned vehicle agent. The action space of the robot agent includes the linear velocity and angular velocity of the robot agent. The data of the joint action space is normalized to ;

[0042] S43: The reward function includes a global reward function, a collision penalty function, a reachability area local reward function, and a maneuverability local reward function. Constructing the reward function includes the following steps:

[0043] S431: Constructing a global reward function :

[0044]

[0045]

[0046] in, is the first control coefficient of the reward range, is the second control coefficient of the reward amplitude, is the distance between the end effector of the wheeled mobile robot and the target point, represents the distance between two points, Indicates that the function takes the minimum value;

[0047] S432: Constructing a collision penalty function :

[0048] ;

[0049] S433: Constructing a local reward function for the reachability region of a wheeled mobile manipulator :

[0050]

[0051] in, is the distance between the unmanned vehicle platform and the target point;

[0052] S434: Constructing a local reward function for the maneuverability of a wheeled mobile manipulator :

[0053]

[0054] in, is the operating threshold.

[0055] According to a coordinated control method for a wheeled mobile robotic arm provided by the present invention, step S5 includes:

[0056] S51: Initialize the global evaluation network , initialize the global target evaluation network , initialize the action network of the unmanned vehicle agent , the action network of the robotic arm agent , Evaluation network of unmanned vehicle agent and the evaluation network of the robotic agent , generate the target position within the set range and set the initial state of the wheeled mobile manipulator;

[0057] S52: Get the initial state of the simulation environment: , , ,in, Indicates that at time step The state of the joint state space is Indicates that at time step The action space of the autonomous vehicle agent is Indicates that at time step The action space of the robot agent is represents the policy function of the unmanned vehicle agent, Indicates that at time step The observation input data of the unmanned vehicle agent is Indicates that at time step When the strategy function of the robot agent is Indicates that at time step The observation input data of the robot agent;

[0058] S53: Perform joint actions ,in, , at time step Get the collision penalty function when , at time step The local reward function of the reachable area when and at time step The local reward function for maneuverability ;

[0059] S54: Build storage sample , and store the stored samples into the experience pool, where Indicates that at time step The state of the joint state space when

[0060] S55: training a global evaluation network, randomly sampling from the experience pool Samples make up a batch of samples :

[0061] in, , is the sample ordinal number, , the symbol in the formula Indicates the assignment operation, that is, assigning the result of the expression on the right side of the equal sign to the variable on the left side. Represented in the joint state space Next take The joint action of samples The collision reward function obtained after represents the action taken by the autonomous vehicle agent at the next time step, represents the action taken by the robotic agent in the next time step, Indicates that the autonomous vehicle agent is in the The observation data for the next time step in samples, Indicates that the robot agent is in the The observation data for the next time step in samples, is the policy function of the unmanned vehicle agent under the new policy at the next time step, is the policy function of the robot agent under the new policy at the next time step; For the The joint state space at the next time step when the number of samples is 0;

[0062] S56: Calculating target value and minimizing loss :

[0063]

[0064]

[0065] in, It is The target value of samples, Indicates in In the samples, the agent The global reward obtained at the time step, the agents include the unmanned vehicle agent and the robotic arm agent, , is the discount factor, In the new strategy Next, use the action network parameters The action value function, Indicates in In the samples, the agent The observation data at the next time step, In the new strategy Next, the agent Observation data in the new state Actions taken in It is an intelligent agent The global evaluation network uses the global evaluation network parameters To estimate the value of a given state and action;

[0066] use and Update the global evaluation network and global target evaluation network :

[0067]

[0068]

[0069] in, Indicates a replacement operation. is the evaluation network learning rate, is the global target network parameter, is the inertia update rate; is the loss function About evaluating network parameters gradient;

[0070] S57: Initialize the agent local evaluation network and agent local goal evaluation network , randomly sampled from the experience pool samples form a small batch of samples ,in, is the second sample ordinal number, ;in, , Represented in the joint state space Next take The joint action of samples The collision reward function obtained after Indicates that the autonomous vehicle agent is in the The observation data for the next time step in samples, Indicates that the robot agent is in the The observation data for the next time step in samples; For the The joint state space at the next time step when the number of samples is 0;

[0071] S58: Calculate local target value and local minimization loss :

[0072]

[0073]

[0074] It is The local target value of samples, Indicates in In the samples, the agent The global reward obtained at the time step, It is an intelligent agent The global evaluation network uses the global evaluation network parameters To estimate the value of a given state and action;

[0075] use and Update the agent's local evaluation network and local target evaluation network :

[0076]

[0077]

[0078] in, is the local target network parameter; is the loss function On evaluating local evaluation network parameters gradient;

[0079] S59: Computational Agents i Strategy parameters :

[0080]

[0081] in, is the update rule for the policy parameters, It's a strategy About strategy parameters The gradient of Among the samples, the Observation data of an agent Agents under Joint action , It is a global evaluation network On the joint actions of intelligent agents The gradient, It is The local evaluation function of an agent On the joint actions of intelligent agents The gradient, It is Among the samples, the Observation data of each agent;

[0082] S510: Update and :

[0083]

[0084]

[0085] in, is the agent strategy parameter for the next time step.

[0086] According to a coordinated control method for a wheeled mobile robotic arm provided by the present invention, step S6 includes:

[0087] S61: Set up task scenarios, including simple environments with no obstacles and complex environments with multiple obstacles;

[0088] S62: Setting training hyperparameters, wherein the training hyperparameters include: discount factor, experience pool size, inertia update rate, simulation time step, maximum time step of a single round, evaluation network learning rate, and action network learning rate;

[0089] S63: Design task evaluation indicators include average operability, average operability, and success rate.

[0090] According to a coordinated control method for a wheeled mobile robotic arm provided by the present invention, step S7 includes:

[0091] S71: Training settings are as follows: the maximum length of the round is set according to the task scenario, training hyperparameters, and task evaluation indicators. The initial pose of the wheeled mobile manipulator remains consistent. The design and call of the wheeled mobile manipulator coordination control algorithm are implemented based on the RLGames library. Two independent multi-layer perceptrons are used for DE-MADDPG. The input of the wheeled mobile manipulator coordination control algorithm is the same. In the simulation training environment, the running frequency of the wheeled mobile manipulator simulation is set, and the interaction frequency of the wheeled mobile manipulator coordination control algorithm is set.

[0092] S72: Train and evaluate a coordinated control algorithm for a wheeled mobile manipulator in two different mission scenarios, construct a theoretically infinite test set, in which the target position is randomly generated within a preset range, and record the success rate of the task at the end of each training cycle, wherein the success rate is determined by calculating the percentage of successful cases in a validation set that contains a set number of independent trials in the validation set, and the target point position of each trial is randomly generated within the specified range. The success rate is recorded after each training cycle, and each environment configuration is independently trained a threshold number of times. Each training includes a set number of training cycles and generates a test result containing a random target position.

[0093] S73: Based on the success rate of the test results and the observation results of the operability of the robot arm, the effectiveness of the coordinated control algorithm process of the wheeled mobile robot arm is judged. If the effectiveness is lower than the set threshold, return to step S5 for retraining.

[0094] The present invention also provides a coordinated control system for a wheeled mobile robotic arm, comprising:

[0095] Data acquisition module: establishing a wheeled mobile manipulator model, solving a target reachability area based on the wheeled mobile manipulator model, calculating the Jacobian matrix of the wheeled mobile manipulator and the maneuverability index of the wheeled mobile manipulator based on the wheeled mobile manipulator model, calculating the maximum linear velocity of the wheeled mobile manipulator and the maximum linear velocity of the wheeled mobile manipulator based on the Jacobian matrix of the wheeled mobile manipulator, and obtaining a velocity ellipsoid model of the wheeled mobile manipulator;

[0096] Model building module: Constructs a joint state space, joint action space, and reward function based on the target reachability area, maneuverability index, and velocity ellipsoid model; and constructs a coordinated control algorithm flow for a wheeled mobile manipulator based on the DE-MADDPG algorithm based on the joint state space, joint action space, and reward function.

[0097] Model training module: Build a physical simulation training environment for a wheeled mobile robotic arm based on IsaacSim, set training parameters, design task evaluation indicators, and set up training based on the task scenarios, training hyperparameters, and task evaluation indicators of the coordinated control algorithm process of the wheeled mobile robotic arm. At the end of each training cycle, record the success rate of the task to determine the effectiveness of the coordinated control algorithm process of the wheeled mobile robotic arm.

[0098] The present invention also provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a coordinated control method of a wheeled mobile robotic arm as described above are implemented.

[0099] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0101] Figure 1 Flowchart of a coordinated control method for a wheeled mobile robotic arm according to an embodiment of the present invention.

[0102] Figure 2 Schematic diagram of kinematics of an embodiment of the present invention.

[0103] Figure 3 Schematic diagram of the reachability area of ​​a robot arm in a coordinated control method for a wheeled mobile robot arm according to an embodiment of the present invention.

[0104] Figure 4 This is a visual diagram of the operability of the coordinated control method of the wheeled mobile robotic arm according to an embodiment of the present invention.

[0105] Figure 5 This is a model training diagram of the coordinated control method of the wheeled mobile robotic arm according to an embodiment of the present invention.

[0106] Figure 6Schematic diagram of the training results of the coordinated control method for a wheeled mobile robotic arm according to an embodiment of the present invention.

[0107] Figure 7 Comparison of training results of the coordinated control method for wheeled mobile manipulators according to an embodiment of the present invention Figure 1 .

[0108] Figure 8 Comparison of training results of the coordinated control method of the wheeled mobile manipulator according to the embodiment of the present invention Figure 2 .

[0109] Figure 9 4 is a block diagram of a coordinated control system for a wheeled mobile robotic arm according to an embodiment of the present invention.

[0110] Figure 10 This is a schematic structural diagram of the coordinated control device for a wheeled mobile robotic arm provided by the present invention.

[0111] Reference numerals:

[0112] 101. Data acquisition module; 102. Model building module; 103. Model training module; 810. Processor; 820. Communication interface; 830. Memory; 840. Communication bus. DETAILED DESCRIPTION

[0113] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0114] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0115] Example

[0116] The following combination Figures 1 to 10 Describing the Invention

[0117] like Figure 1 As shown, an embodiment of the present invention provides a coordinated control method for a wheeled mobile robotic arm, comprising the following steps:

[0118] S1: establishing a wheeled mobile manipulator model, and solving a target reachability area according to the wheeled mobile manipulator model;

[0119] S2: Calculating the Jacobian matrix and the maneuverability index of the wheeled mobile manipulator according to the wheeled mobile manipulator model;

[0120] S3: calculating the maximum linear velocity and the maximum angular velocity of the wheeled mobile manipulator according to the Jacobian matrix of the wheeled mobile manipulator, and obtaining a velocity ellipsoid model of the wheeled mobile manipulator;

[0121] S4: Construct the joint state space, joint action space and reward function based on the target reachability area, maneuverability index and velocity ellipsoid model;

[0122] S5: Based on the joint state space, joint action space, and reward function, a coordinated control algorithm flow for a wheeled mobile manipulator based on the DE-MADDPG (Decomposed Multi-Agent Deep Deterministic Policy Gradient) algorithm is constructed;

[0123] S6: Based on the IsaacSim platform, set the task scenario of the coordinated control algorithm of the wheeled mobile manipulator, set the training hyperparameters, and design the task evaluation indicators.

[0124] S7: Perform training based on the task scenarios, training hyperparameters, and task evaluation metrics of the coordinated control algorithm process for the wheeled mobile manipulator. Record the task success rate at the end of each training cycle to determine the effectiveness of the coordinated control algorithm process for the wheeled mobile manipulator.

[0125] Specifically, step S1 includes the following steps: Figure 2 As shown, with the center of mass of the unmanned vehicle platform as the origin, the horizontal direction directly behind the unmanned vehicle platform as the positive direction of the x-axis, and the upward direction perpendicular to the ground as the positive direction of the z-axis, the positive direction of the y-axis can form a right-handed axial scanning coordinate system with the positive directions of the x-axis and the z-axis. This coordinate system is a wheeled mobile robotic arm model.

[0126] Find the Frank Panda operating manual for the wheeled mobile manipulator and get the working radius of the wheeled mobile manipulator. .

[0127] According to the z-axis coordinate of the target point , the z-axis coordinate of the first joint center of the wheeled mobile robot Calculate the angle between the line perpendicular to the ground starting from the target point and the line extending from the target to the center of the first joint of the manipulator :

[0128] ;

[0129] Get the distance from the center of mass of the unmanned platform to the first joint of the robotic arm on the x-axis: , calculate the reachability area diameter Size:

[0130] .

[0131] According to the above steps, we can determine the size of the reachable area. The size of the reachable area is as follows: Figure 3 As shown, the diameter of the accessibility area The size of the wheeled mobile manipulator model and the z-axis coordinate of the target point Determined.

[0132] Specifically, step S2 includes the following steps: according to the wheeled mobile manipulator model, calculating the Jacobian matrix of the wheeled mobile manipulator :

[0133]

[0134] in, is the end effector position Joint angle The partial derivative, is the degree of freedom of the operating space, is the degree of freedom of the joint space. Since the spatial degree of freedom is 6, the degree of freedom of the robot arm is 6. Therefore, ; .

[0135] According to the Jacobian matrix of the wheeled mobile manipulator , calculate the operability index

[0136]

[0137] in, Represents determinant evaluation.

[0138] Specifically, step S3 includes the following steps: according to the Jacobian matrix of the wheeled mobile manipulator , The first three rows represent the linear velocity of the end effector of the wheeled mobile robot. The last three lines represent the angular velocity of the end effector of the wheeled mobile manipulator. The Jacobian matrix of the wheeled mobile manipulator is Translated as:

[0139]

[0140] in, It is the linear velocity part, is the angular velocity part;

[0141] Determine the linear velocity of the end of the wheeled mobile robot and the angular velocity of the end of the wheeled mobile manipulator :

[0142]

[0143]

[0144] according to and Draw an operational visualization diagram, such as Figure 4 As shown in the figure. The shape and direction of the linear velocity ellipsoid are given by The size of the ellipsoid reflects the speed capability of the manipulator in different directions; the shape and direction of the ellipsoid are determined by The size of the ellipsoid reflects the angular velocity capability of the manipulator in different directions. The larger the volume of the ellipsoid, the better the maneuverability of the manipulator.

[0145] Specifically, step S4 of the present invention further includes:

[0146] According to the structure of the wheeled mobile robotic arm, it is divided into two agents: the unmanned vehicle agent and the robotic arm agent:

[0147] According to the reachability region and the velocity ellipsoid model, the joint state space of the wheeled mobile manipulator is constructed as ,in, is the displacement of the autonomous vehicle body relative to the starting position and its yaw in the global frame, are the linear and angular velocities of the joints of the robotic agent, is the position coordinate of the end effector of the robotic arm, is the target position coordinate; , is the set of real numbers.

[0148] Constructing the joint action space of the two intelligent agents, the unmanned vehicle platform and the robotic arm ,in is the motion space of the unmanned vehicle platform, including the linear velocity of the unmanned vehicle platform and angular velocity , is the motion space of the robot arm, including the linear velocity of the end of the robot arm and the angular velocity of the end of the robotic arm All actions are continuous. To stabilize training and avoid overfitting, the value of the action space is normalized to ;

[0149] Constructing a global reward function :

[0150]

[0151]

[0152] in, is the first control coefficient of the reward amplitude and is the second control coefficient of the reward amplitude, is the distance between the end effector of the wheeled mobile robot and the target point, represents the distance between two points, Indicates that the function takes the minimum value;

[0153] Constructing a collision penalty function :

[0154] ;

[0155] Constructing a local reward function for the reachability region of a wheeled mobile manipulator :

[0156]

[0157] in, is the distance between the unmanned vehicle platform and the target point, and the action Consists of two continuous values: Linear speed and angular velocity ,The unmanned platform can locate itself in the global map to determine its relative position to the target location;

[0158] Constructing a local reward function for the maneuverability of a wheeled mobile manipulator :

[0159]

[0160] in, is the operating threshold.

[0161] Specifically, initialize the global evaluation network , initialize the global target evaluation network , initialize the action network of the unmanned vehicle agent , the action network of the robotic arm agent , Evaluation network of unmanned vehicle agent and the evaluation network of the robotic agent , generate the target position within the set range and set the initial state of the wheeled mobile robot arm.

[0162] Get the initial state of the simulation environment: , , ,in, Indicates that at time step The state of the joint state space is Indicates that at time step The action space of the autonomous vehicle agent is Indicates that at time step The action space of the robot agent is represents the policy function of the unmanned vehicle agent, Indicates that at time step The observation input data of the unmanned vehicle agent is Indicates that at time step When the strategy function of the robot agent is Indicates that at time step The observation input data of the robotic arm agent.

[0163] Perform joint actions ,in, , at time step Get the collision penalty function when , at time step The local reward function of the reachable area when and at time step The local reward function for maneuverability .

[0164] Build storage sample , and store the stored samples into the experience pool, where Indicates that at time step The state of the joint state space.

[0165] Train a global evaluation network, randomly sampling from the experience pool Samples make up a batch of samples :

[0166] in, , is the sample ordinal number, , the symbol in the formula Indicates the assignment operation, that is, assigning the result of the expression on the right side of the equal sign to the variable on the left side. Represented in the joint state space Next take The joint action of samples The collision reward function obtained after represents the action taken by the autonomous vehicle agent at the next time step, represents the action taken by the robotic agent in the next time step, Indicates that the autonomous vehicle agent is in the The observation data for the next time step in samples, Indicates that the robot agent is in the The observation data for the next time step in samples, is the policy function of the unmanned vehicle agent under the new policy at the next time step, is the policy function of the robot agent under the new policy at the next time step; For the The joint state space at the next time step when there are samples.

[0167] Calculating target value and minimizing loss :

[0168]

[0169]

[0170] in, It is The target value of samples, Indicates in In the samples, the agent The global reward obtained at the time step, the agents include the unmanned vehicle agent and the robotic arm agent, , is the discount factor, In the new strategy Next, use the action network parameters The action value function, Indicates in In the samples, the agent The observation data at the next time step, In the new strategy Next, the agent Observation data in the new state Actions taken in It is an intelligent agent The global evaluation network uses the global evaluation network parameters To estimate the value of a given state and action;

[0171] use and Update the global evaluation network and global target evaluation network :

[0172]

[0173]

[0174] in, Indicates a replacement operation. is the evaluation network learning rate, is the global target network parameter, is the inertia update rate; is the loss function About evaluating network parameters gradient.

[0175] Initialize the agent's local evaluation network and agent local goal evaluation network , randomly sampled from the experience pool samples form a small batch of samples ,in, is the second sample ordinal number, ;in, , Represented in the joint state space Next take The joint action of samples The collision reward function obtained after Indicates that the autonomous vehicle agent is in the The observation data for the next time step in samples, Indicates that the robot agent is in the The observation data for the next time step in samples; For the The joint state space at the next time step when there are samples.

[0176] Calculate local objective value and local minimization loss :

[0177]

[0178]

[0179] It is The local target value of samples, Indicates in In the samples, the agent The global reward obtained at the time step, It is an intelligent agent The global evaluation network uses the global evaluation network parameters To estimate the value of a given state and action;

[0180] use and Update the agent's local evaluation network and local target evaluation network :

[0181]

[0182]

[0183] in, is the local target network parameter; is the loss function On evaluating local evaluation network parameters gradient.

[0184] Computational Agents i Strategy parameters :

[0185]

[0186] in, is the update rule for the policy parameters, It's a strategy About strategy parameters The gradient of Among the samples, the Observation data of an agent Agents under Joint action , It is a global evaluation network On the joint actions of intelligent agents The gradient, It is The local evaluation function of an agent On the joint actions of intelligent agents The gradient, It is Among the samples, the Observation data of each agent;

[0187] renew and :

[0188]

[0189]

[0190] in, is the agent strategy parameter for the next moment.

[0191] The specific step S6 includes: setting the task scene including Task 1: simple environment without obstacles; Task 2: complex environment with multiple obstacles. In addition to the self-collision problem, the complex environment with multiple obstacles also needs to consider the collision problem between obstacles and the platform; setting the training hyperparameters as: discount factor , experience pool size M=30000, inertia update rate , simulation time step , the maximum time step of a single round is 500, and the learning rate of the evaluation network is , action network learning rate ; Design task evaluation indicators include average operability, average operability and success rate.

[0192] Table 1 Experimental parameters

[0193]

[0194] According to the data in Table 1, the embodiment of the present invention uses IsaacSim to build a simulation training environment. Figure 5 As shown. Among them, Figure 5 (a) is a simple barrier-free environment for task 1. Figure 5 Middle (b) is the complex environment with many obstacles in task 2.

[0195] Specifically, step S7 training is set as follows: the maximum length of the round is 500 steps, the initial position of the wheeled mobile manipulator is kept consistent, the design and call of the wheeled mobile manipulator coordination control algorithm is implemented based on the RLGames library, two independent multi-layer perceptrons are used for DE-MADDPG, the input of the wheeled mobile manipulator coordination control algorithm is the same, the running frequency of the wheeled mobile manipulator simulation in the simulation training environment is set to 120Hz, and the interaction frequency of the wheeled mobile manipulator coordination control algorithm is set to 60Hz; the wheeled mobile manipulator coordination control algorithm is trained and evaluated in two different task scenarios to construct a theoretically infinite test set , where the target position is randomly generated within a preset range, and the success rate of the task is recorded at the end of each training cycle, where the success rate is determined by calculating the percentage of successful cases in a validation set containing 50 independent trials. The target point position of each trial is randomly generated within the specified range, and the success rate is recorded after every 20 training cycles. Each environment configuration is trained independently 3 times, each training includes 500 training cycles, and a test set containing 100 random target positions is generated; based on the success rate of the test results and the operability observation results of the robot arm, the coordinated control algorithm process of the wheeled mobile robot arm based on the DE-MADDPG algorithm is evaluated.

[0196] The simulation was run at 120 Hz, the interaction frequency of the reinforcement learning algorithm was set at 60 Hz, and the maximum episode length was set to 500 steps. The initial pose of the wheeled mobile manipulator remained consistent throughout all experiments.

[0197] According to the observation of training data, multiple training processes in two tasks show consistent learning curves for the three strategies. Figure 6 As shown. Among them, Figure 6 (a) is a simple barrier-free environment for task 1. Figure 6 (b) illustrates Task 2, a complex environment with multiple obstacles. Overall, the DE-MADDPG-based learning curve proposed in this embodiment of the present invention achieves the highest learning speed and convergence success rate, indicating better accessibility. In particular, the DE-MADDPG-based coordinated control strategy proposed in this embodiment of the present invention exhibits significantly higher convergence speed and success rate in complex task scenarios.

[0198] For both tasks, we used the average maneuverability of the entire task trajectory as the maneuverability metric. Tables 2 and 3 present the experimental results in a simulation environment for 50 test runs of the test set. During the evaluation process, the proposed DE-MADDPG-based control strategy achieved the best performance in both task settings, and achieved significant improvements in average maneuverability and success rate compared to the other two control strategies.

[0199] Table 2 Experimental results of reachability and operability in the simulation environment of Task 1

[0200]

[0201] Table 3 Experimental results of reachability and operability in the simulation environment of Task 2

[0202]

[0203] According to the experimental results of 50 tests in the simulation environment, the Figure 7 and Figure 8 .in, Figure 7 (a) to (d) are task 1 barrier-free simple environment. Figure 8 (a) to (d) illustrate Task 2, a complex environment with multiple obstacles. A side-by-side comparison shows that in Task 1, a simple, obstacle-free environment, all three control strategies were able to achieve the task objective. DE-MADDPG and DDPG (Deep Deterministic Policy Gradient) achieved significantly better control performance than MADDPG (Multi-Agent Deep Deterministic Policy Gradient). The operability curves show that DE-MADDPG significantly improves operability compared to the other two control strategies. In Task 2, a complex environment with multiple obstacles, none of the MADDPG-based control strategies were able to achieve the target. Compared to DDPG, DE-MADDPG demonstrates significant improvements in both control efficiency and operability.

[0204] A longitudinal comparison of the two experimental results shows that the MADDPG-based control strategy failed to complete the arrival task in both complex environments. While the DDPG-based control strategy completed the required tasks in both experimental scenarios, it suffered from slow control efficiency and poor operability. In comparison, the DE-MADDPG-based control strategy achieved higher control efficiency in both environments, significantly improved operability, and demonstrated better adaptability to different task scenarios.

[0205] like Figure 9 As shown, Figure 9 An example of a coordinated control system for a wheeled mobile robotic arm is provided, comprising:

[0206] Data acquisition module 101: establishing a wheeled mobile manipulator model, solving a target reachability region based on the wheeled mobile manipulator model, calculating the Jacobian matrix of the wheeled mobile manipulator and the maneuverability index of the wheeled mobile manipulator based on the wheeled mobile manipulator model, calculating the maximum linear velocity of the wheeled mobile manipulator and the maximum linear velocity of the wheeled mobile manipulator based on the Jacobian matrix of the wheeled mobile manipulator, and obtaining a velocity ellipsoid model of the wheeled mobile manipulator;

[0207] Model building module 102: constructing a joint state space, a joint action space, and a reward function based on the target reachability area, the maneuverability index, and the velocity ellipsoid model, and constructing a coordinated control algorithm flow of the wheeled mobile manipulator based on the DE-MADDPG algorithm based on the joint state space, the joint action space, and the reward function;

[0208] Model training module 103: Build a physical simulation training environment for a wheeled mobile manipulator based on IsaacSim, set training parameters, design task evaluation indicators, perform training settings based on the task scenarios of the wheeled mobile manipulator coordination control algorithm process, the training hyperparameters and the task evaluation indicators, record the success rate of the task at the end of each training cycle, and judge the effectiveness of the wheeled mobile manipulator coordination control algorithm process.

[0209] Figure 10 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 10 As shown, the electronic equipment may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute a coordinated control method for a wheeled mobile robotic arm.

[0210] Furthermore, the logic instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A coordinated control method for a wheeled mobile robotic arm, characterized in that: The following steps are involved: S1: establishing a wheeled mobile manipulator model, and solving a target reachability area according to the wheeled mobile manipulator model; S2: Calculating the Jacobian matrix and the maneuverability index of the wheeled mobile manipulator according to the wheeled mobile manipulator model; S3: calculating the maximum linear velocity and the maximum angular velocity of the wheeled mobile manipulator according to the Jacobian matrix of the wheeled mobile manipulator, and obtaining a velocity ellipsoid model of the wheeled mobile manipulator; S4: constructing a joint state space, a joint action space and a reward function according to the target reachability area, the maneuverability index and the velocity ellipsoid model; S5: Constructing a coordinated control algorithm flow of a wheeled mobile manipulator based on the DE-MADDPG algorithm according to the joint state space, the joint action space and the reward function; S6: Based on the IsaacSim platform, set the task scenario of the coordinated control algorithm process of the wheeled mobile manipulator, set the training hyperparameters, and design the task evaluation indicators; S7: Perform training settings based on the task scenario of the wheeled mobile manipulator coordination control algorithm process, the training hyperparameters and the task evaluation indicators, record the success rate of the task at the end of each training cycle, and judge the effectiveness of the wheeled mobile manipulator coordination control algorithm process.

2. A coordinated control method for a wheeled mobile robotic arm according to claim 1, characterized in that: Step S1 includes: S11: With the center of mass of the unmanned vehicle platform as the origin, the horizontal direction directly behind the unmanned vehicle platform as the positive direction of the x-axis, and the vertical direction upwards to the ground as the positive direction of the z-axis, the positive direction of the y-axis can form a right-handed axial scanning coordinate system together with the positive directions of the x-axis and the z-axis. The axial scanning coordinate system is a wheeled mobile robotic arm model; S12: Find the wheeled mobile manipulator operation manual to obtain the working radius of the wheeled mobile manipulator ; S13: According to the z-axis coordinate of the target point , the z-axis coordinate of the first joint center of the wheeled mobile robot Calculate the angle between the line perpendicular to the ground starting from the target point and the line extending from the target to the center of the first joint of the manipulator : ； S14: Get the distance from the center of mass of the unmanned platform to the first joint of the robotic arm on the x-axis. , calculate the reachability area diameter Size: The size of the reachability area is the reachability area diameter A sphere with a diameter of .

3. A coordinated control method for a wheeled mobile robotic arm according to claim 1, characterized in that: Step S2 includes: S21: Calculate the Jacobian matrix of the wheeled mobile manipulator according to the wheeled mobile manipulator model : in, is the end effector position Joint angle The partial derivative, is the degree of freedom of the operating space, is the degree of freedom of the joint space; S22: According to the Jacobian matrix of the wheeled mobile manipulator , calculate the operability index in, represents determinant evaluation, for The transposed matrix of .

4. A coordinated control method for a wheeled mobile robotic arm according to claim 3, characterized in that: Step S3 includes: S31: According to the Jacobian matrix of the wheeled mobile manipulator , The upper part of represents the linear velocity of the end effector of the wheeled mobile robot arm. The lower half of represents the angular velocity of the end effector of the wheeled mobile robot. Translated as: in, It is the linear velocity part, is the angular velocity part; S32: Determine the linear velocity of the end of the wheeled mobile robot : in, yes The transposed matrix of S33: Determine the angular velocity of the end of a wheeled mobile manipulator , in, yes The transposed matrix of and Draw the velocity ellipsoid model.

5. A coordinated control method for a wheeled mobile robotic arm according to claim 3, characterized in that: Step S4 includes: S41: Divide the structure of the wheeled mobile robotic arm into two agents: an unmanned vehicle agent and a robotic arm agent. According to the reachability region and the velocity ellipsoid model, the joint state space of the wheeled mobile manipulator is constructed as ,in, is the displacement of the autonomous vehicle body relative to the starting position and its yaw in the global frame, are the linear and angular velocities of the joints of the robotic agent, is the position coordinate of the end effector of the robotic arm, is the target position coordinate; , is the set of real numbers; S42: Constructing a joint action space of the unmanned vehicle agent and the robotic arm agent based on the reachability region and the velocity ellipsoid model ,in is the action space of the unmanned vehicle agent, including the linear velocity and angular velocity of the unmanned vehicle agent. The action space of the robot agent includes the linear velocity and angular velocity of the robot agent. The data of the joint action space is normalized to ; S43: The reward function includes a global reward function, a collision penalty function, a reachability area local reward function, and a maneuverability local reward function. Constructing the reward function includes the following steps: S431: Constructing a global reward function : in, is the first control coefficient of the reward range, is the second control coefficient of the reward amplitude, is the distance between the end effector of the wheeled mobile robot and the target point, represents the distance between two points, Indicates that the function takes the minimum value; S432: Constructing a collision penalty function : ； S433: Constructing a local reward function for the reachability region of a wheeled mobile manipulator : in, is the distance between the unmanned vehicle platform and the target point; S434: Constructing a local reward function for the maneuverability of a wheeled mobile manipulator : in, is the operating threshold.

6. A coordinated control method for a wheeled mobile robotic arm according to claim 5, characterized in that: Step S5 includes: S51: Initialize the global evaluation network , initialize the global target evaluation network , initialize the action network of the unmanned vehicle agent , the action network of the robotic arm agent , Evaluation network of unmanned vehicle agent and the evaluation network of the robotic agent , generate the target position within the set range and set the initial state of the wheeled mobile manipulator; S52: Get the initial state of the simulation environment: , , ,in, Indicates that at time step The state of the joint state space is Indicates that at time step The action space of the autonomous vehicle agent is Indicates that at time step The action space of the robot agent is represents the policy function of the unmanned vehicle agent, Indicates that at time step The observation input data of the unmanned vehicle agent is Indicates that at time step When the strategy function of the robot agent is Indicates that at time step The observation input data of the robot arm agent; S53: Perform joint actions ,in, , at time step Get the collision penalty function when , at time step The local reward function of the reachable area when and at time step The local reward function for maneuverability ; S54: Build storage sample , and store the stored samples into the experience pool, where Indicates that at time step The state of the joint state space when S55: training a global evaluation network, randomly sampling from the experience pool Samples make up a batch of samples : in, , is the sample ordinal number, , the symbol in the formula Indicates the assignment operation, that is, assigning the result of the expression on the right side of the equal sign to the variable on the left side. Represented in the joint state space Next take The joint action of samples The collision reward function obtained after represents the action taken by the autonomous vehicle agent at the next time step, represents the action taken by the robotic agent in the next time step, Indicates that the autonomous vehicle agent is in the The observation data for the next time step in samples, Indicates that the robot agent is in the The observation data for the next time step in samples, is the policy function of the unmanned vehicle agent under the new policy at the next time step, is the policy function of the robot agent under the new policy at the next time step; For the The joint state space at the next time step when the number of samples is 0; S56: Calculating target value and minimizing loss : in, It is The target value of samples, Indicates in In the samples, the agent The global reward obtained at the time step, the agents include the unmanned vehicle agent and the robotic arm agent, , is the discount factor, In the new strategy Next, use the action network parameters The action value function, Indicates in In the samples, the agent The observation data at the next time step, In the new strategy Next, the agent Observation data in the new state Actions taken in It is an intelligent agent The global evaluation network uses the global evaluation network parameters To estimate the value of a given state and action; use and Update the global evaluation network and global target evaluation network : in, Indicates a replacement operation. is the evaluation network learning rate, is the global target network parameter, is the inertia update rate; is the loss function About evaluating network parameters gradient; S57: Initialize the agent local evaluation network and agent local goal evaluation network , randomly sampled from the experience pool samples form a small batch of samples ,in, is the second sample ordinal number, ;in, , Represented in the joint state space Next take The joint action of samples The collision reward function obtained after Indicates that the autonomous vehicle agent is in the The observation data for the next time step in samples, Indicates that the robot agent is in the The observation data for the next time step in samples; For the The joint state space at the next time step when the number of samples is 0; S58: Calculate local target value and local minimization loss : It is The local target value of samples, Indicates in In the samples, the agent The global reward obtained at the time step, It is an intelligent agent The global evaluation network uses the global evaluation network parameters To estimate the value of a given state and action; use and Update the agent's local evaluation network and local target evaluation network : in, is the local target network parameter; is the loss function On evaluating local evaluation network parameters gradient; S59: Computational Agents i Strategy parameters : in, is the update rule for the policy parameters, It's a strategy About strategy parameters The gradient of Among the samples, the Observation data of an agent Agents under Joint action , It is a global evaluation network On the joint actions of intelligent agents The gradient, It is The local evaluation function of an agent On the joint actions of intelligent agents The gradient, It is Among the samples, the Observation data of each agent; S510: Update and : in, is the agent strategy parameter for the next time step.

7. A coordinated control method for a wheeled mobile robotic arm according to claim 1, characterized in that: Step S6 includes: S61: Set up task scenarios, including simple environments with no obstacles and complex environments with multiple obstacles; S62: Setting training hyperparameters, wherein the training hyperparameters include: discount factor, experience pool size, inertia update rate, simulation time step, maximum time step of a single round, evaluation network learning rate, and action network learning rate; S63: Design task evaluation indicators include average operability, average operability, and success rate.

8. The coordinated control method of a wheeled mobile manipulator according to claim 1, characterized in that: Step S7 includes: S71: Training settings are as follows: the maximum length of the round is set according to the task scenario, training hyperparameters, and task evaluation indicators. The initial pose of the wheeled mobile manipulator remains consistent. The design and call of the wheeled mobile manipulator coordination control algorithm are implemented based on the RLGames library. Two independent multi-layer perceptrons are used for DE-MADDPG. The input of the wheeled mobile manipulator coordination control algorithm is the same. In the simulation training environment, the running frequency of the wheeled mobile manipulator simulation is set, and the interaction frequency of the wheeled mobile manipulator coordination control algorithm is set. S72: Train and evaluate a coordinated control algorithm for a wheeled mobile manipulator in two different mission scenarios, construct a theoretically infinite test set, in which the target position is randomly generated within a preset range, and record the success rate of the task at the end of each training cycle, wherein the success rate is determined by calculating the percentage of successful cases in a validation set that contains a set number of independent trials in the validation set, and the target point position of each trial is randomly generated within the specified range. The success rate is recorded after each training cycle, and each environment configuration is independently trained a threshold number of times. Each training includes a set number of training cycles and generates a test result containing a random target position. S73: Based on the success rate of the test results and the observation results of the operability of the robot arm, the effectiveness of the coordinated control algorithm process of the wheeled mobile robot arm is judged. If the effectiveness is lower than the set threshold, return to step S5 for retraining.

9. A coordinated control method system for a wheeled mobile manipulator, for executing a coordinated control method for a wheeled mobile manipulator according to any one of claims 1 to 8, characterized in that: include: Data acquisition module: establishing a wheeled mobile manipulator model, solving a target reachability area based on the wheeled mobile manipulator model, calculating the Jacobian matrix of the wheeled mobile manipulator and the maneuverability index of the wheeled mobile manipulator based on the wheeled mobile manipulator model, calculating the maximum linear velocity of the wheeled mobile manipulator and the maximum linear velocity of the wheeled mobile manipulator based on the Jacobian matrix of the wheeled mobile manipulator, and obtaining a velocity ellipsoid model of the wheeled mobile manipulator; Model building module: constructing a joint state space, a joint action space and a reward function according to the target reachability area, the maneuverability index and the velocity ellipsoid model, and constructing a coordinated control algorithm flow of a wheeled mobile manipulator based on the DE-MADDPG algorithm according to the joint state space, the joint action space and the reward function; Model training module: Build a physical simulation training environment for a wheeled mobile robotic arm based on IsaacSim, set training parameters, design task evaluation indicators, perform training settings based on the task scenarios of the wheeled mobile robotic arm coordinated control algorithm process, the training hyperparameters, and the task evaluation indicators, record the task success rate at the end of each training cycle, and judge the effectiveness of the wheeled mobile robotic arm coordinated control algorithm process.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the coordinated control method of a wheeled mobile robotic arm as described in any one of claims 1 to 8 are implemented.

Images