Control methods, equipment, and media for quadruped robot integrated robotic arm systems

Abstract

This invention discloses a control method, device, and medium for a quadruped robot integrated robotic arm system, belonging to the field of quadruped robot integrated robotic arm system control. The method includes: Step 1, using a trajectory optimization algorithm based on a single rigid body model, obtaining the quadruped robot's motion prior through a manually defined movement trajectory, and inputting the adversarial motion prior into a reinforcement learning network for training; Step 2, generating feasible trajectories for the robotic arm using a trajectory planner based on model predicted path integrals, and executing trajectory commands through a vector field-based trajectory follower; Step 3, combining the system's ontology perception, historical observations, and target point information and inputting them into a target-conditional reinforcement learning network to obtain the movement trajectory and control the system's movement. This method has stronger motion capabilities and safety, enabling the system to complete movement tasks in various unstructured environments, and has low computational complexity.

Description

Technical Field

[0001] This invention relates to the field of quadruped robot and robotic arm movement control technology, and in particular to a control method, device and medium for a quadruped robot integrated robotic arm system. Background Technology

[0002] Integrating quadruped robots and robotic arms into a unified system to perform mobility tasks can expand the robotic arm's operational space. Traditional control methods rely on predefined assumptions and model simplifications to achieve computationally complex optimized trajectories. However, the heavy computational load, environmental uncertainties, and sensor drift limit the performance of such traditional methods in unstructured environments.

[0003] One effective approach to addressing these issues is reinforcement learning, which teaches quadruped robots to move and manipulate objects through continuous interaction with their environment. Deep reinforcement learning, by defining a reward function and employing multi-stage training, enables quadruped robots with integrated robotic arm systems to track both the base velocity and the end effector position. However, the complex reward function involves a massive parameter tuning process, which significantly wastes time and resources.

[0004] In view of this, the present invention is hereby proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a control method, device and medium for a quadruped robot integrated robotic arm system, which can improve the mobility and operation capabilities of the quadruped robot integrated robotic arm system based on deep reinforcement learning, thereby solving the above-mentioned technical problems existing in the prior art.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A control method for a quadruped robot integrated robotic arm system, used to control the quadruped robot integrated robotic arm system to complete movement operation tasks in an unstructured environment, including:

[0008] Step 1: Using a trajectory optimization algorithm based on a single rigid body model, the prior motion of the quadruped robot integrated robotic arm system is obtained through a manually defined movement trajectory. The prior motion of the movement operation is then input into a reinforcement learning network through an adversarial motion prior to obtain a reinforcement learning strategy that can complete the movement operation task.

[0009] Step 2: Use a trajectory planner based on model prediction path integral to generate a feasible trajectory for the quadruped robot integrated manipulator system, and execute the trajectory command through a trajectory follower based on vector field.

[0010] Step 3: Combine the body perception, historical observation and target point information of the quadruped robot integrated robotic arm system and input them into the reinforcement learning network trained in Step 1 to obtain the movement operation trajectory. Control the movement operation of the quadruped robot integrated robotic arm system through the obtained movement operation trajectory.

[0011] A processing apparatus, comprising:

[0012] At least one memory for storing one or more programs;

[0013] At least one processor is capable of executing one or more programs stored in the memory, such that when the processor executes one or more programs, the processor can implement the method of the present invention.

[0014] A readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the methods described in this invention.

[0015] Compared with the prior art, the automatic motion control method, equipment and medium for rope-traction flexible robot joints provided by the present invention have the following advantages:

[0016] By combining traditional model-based quadruped robot control methods with reinforcement learning, and using model prediction integral control to control the robotic arm, the mobility of the quadruped robot integrated robotic arm system is improved. Motion priors for mobility operations are designed through trajectory optimization, and deep reinforcement learning training is assisted by adversarial motion priors to further enhance the mobility of the quadruped robot integrated robotic arm system. Asynchronous communication control using a trajectory planner based on model prediction path integral and a trajectory follower based on vector fields avoids collisions with the robotic arm. Finally, the information from the quadruped robot integrated robotic arm system's ontology perception, historical observations, and target points is combined and input into a target-conditional reinforcement learning network to acquire mobility operation skills. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the control method for a quadruped robot integrated robotic arm system provided in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the architecture of the control method for the quadruped robot integrated robotic arm system provided by the present invention. Detailed Implementation

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the specific content of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, which do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0021] First, the following explanations are provided for the terms that may be used in this article:

[0022] The term "and / or" means that either or both can be achieved simultaneously. For example, X and / or Y means that it includes both "X" or "Y" as well as the three cases of "X and Y".

[0023] The terms "comprising," "including," "containing," "having," or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.) should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0024] The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim.

[0025] Unless otherwise explicitly specified or limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this document according to the specific circumstances.

[0026] When concentration, temperature, pressure, size, or other parameters are expressed as numerical ranges, such ranges should be understood to specifically disclose all ranges formed by any pairing of upper limits, lower limits, or preferred values ​​within that range, regardless of whether the range is explicitly stated; for example, if the numerical range "2 to 8" is stated, then that range should be interpreted to include ranges such as "2 to 7", "2 to 6", "5 to 7", "3 to 4 and 6 to 7", "3 to 5 and 7", "2 and 5 to 7", etc. Unless otherwise stated, the numerical ranges described herein include both their endpoints and all integers and fractions within that range.

[0027] The terms “center,” “longitudinal,” “lateral,” “length,” “width,” “thickness,” “upper,” “lower,” “front,” “back,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” “outer,” “clockwise,” and “counterclockwise” indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience and simplification of description and do not imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this document.

[0028] The solution provided by this invention will be described in detail below. Contents not described in detail in the embodiments of this invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, they shall be performed according to conventional conditions in the art or conditions recommended by the manufacturer. Reagents or instruments used in the embodiments of this invention whose manufacturers are not specified are all conventional products that can be purchased commercially.

[0029] like Figure 1 As shown, this invention provides a control method for a quadruped robot integrated robotic arm system. This method is a mobile operation control method for a quadruped robot integrated robotic arm system based on adversarial motion prior deep reinforcement learning. It is used to control the quadruped robot integrated robotic arm system to complete mobile operation tasks in unstructured environments. Compared with traditional model-based control methods or whole-body controllers for quadruped robot integrated robotic arms designed entirely using reinforcement learning, this method has stronger motion capabilities and safety, enabling the quadruped robot integrated robotic arm system to complete mobile operation tasks in various unstructured environments, and has low computational complexity. The method includes:

[0030] Step 1: Using a trajectory optimization algorithm based on a single rigid body model, the prior motion of the quadruped robot integrated robotic arm system is obtained through a manually defined movement trajectory. The prior motion of the movement operation is then input into a reinforcement learning network through an adversarial motion prior to obtain a reinforcement learning strategy that can complete the movement operation task.

[0031] Step 2: Use a trajectory planner based on model prediction path integral to generate a feasible trajectory for the quadruped robot integrated manipulator system, and execute the trajectory command through a trajectory follower based on vector field.

[0032] Step 3: Combine the body perception, historical observation and target point information of the quadruped robot integrated robotic arm system and input them into the reinforcement learning network trained in Step 1 to obtain the movement operation trajectory. Control the movement operation of the quadruped robot integrated robotic arm system through the obtained movement operation trajectory.

[0033] Preferably, in step 1 of the above method, a trajectory optimization algorithm based on a single rigid body model is used to obtain the prior motion of the quadruped robot integrated robotic arm system through a manually defined movement trajectory, including:

[0034] In a simulation environment, a single-body kinematic model of the quadruped robot is established, and a manually defined expert motion trajectory is input into the single-body kinematic model. The prior motion of the movement operation was obtained through trajectory optimization calculation. .

[0035] Preferably, in the above method, the manually defined expert motion trajectory include:

[0036] Linear position of the quadruped robot Orientation of the quadruped robot The position of the four legs of the quadruped robot ;

[0037] The motion prior of the movement operation obtained through trajectory optimization calculation include:

[0038] The joint angles of the quadruped robot ,in, This represents the number of joints in a quadruped robot.

[0039] Preferably, in the above method, the trajectory optimization finds the objective function through the following nonlinear programming. Minimize motion priors ,Right now: And simultaneously satisfy the inequality constraints Sum of equality constraints Objective function for:

[0040] ;

[0041] in, The cost of the quadratic linear regulator that tracks the motion trajectory of the expert; Represents the end cost of generating the trajectory; Indicates the time; T represents the prior duration of the generated movement operation;

[0042] The feasibility of the prior motion of the movement operation is guaranteed by the following complementary contact constraints:

[0043] ;

[0044] ;

[0045] in, This represents the z-axis component of the contact force between the i-th foot of the quadruped robot and the ground. This represents the z-axis component of the position of the i-th leg of the quadruped robot.

[0046] Preferably, in step 1 of the above method, the adversarial motion prior is obtained in the following manner:

[0047] First train a parameter The neural network that determines the discriminator The state transition in the motion prior of the movement operation. and state transitions generated by reinforcement learning training Input to discriminator In this way, we can distinguish whether the state generated during training matches the prior motion of the movement operation;

[0048] Training the discriminator objective function in the process Defined as:

[0049] ;

[0050] The meanings of each parameter are as follows: For The objective function is the parameter. To train the discriminator, its essence is to use A multilayer perceptron with parameters; Let t be the prior state of the movement. This represents the prior state of the movement at time t+1; Let be the state of the quadruped robot at time t. This represents the state of the quadruped robot at time t+1. For parameters Calculate the gradient. The first two terms in the objective function encourage the discriminator to distinguish whether the obtained state is obtained from the motion prior of the movement operation or generated through reinforcement learning, and the last term of the objective function adds a gradient penalty.

[0051] In step 1 of the above method, a robotic arm trajectory generator based on model path integral and a trajectory tracker based on vector field are designed. The goal is to calculate the robotic arm speed command given the target position of the end effector and the joint state of the robotic arm. This ensures safety during the movement operation, as the robotic arm needs to meet certain constraints during operation, such as avoiding collisions.

[0052] Preferably, in step 2 of the above method, a feasible trajectory for the quadruped robot integrated robotic arm system is generated using a trajectory planner based on model-predicted path integrals, including:

[0053] The trajectory planner based on model-predicted path integrals is defined as a discrete-time, continuous-state system, i.e.:

[0054] ;

[0055] in, This indicates the joint state of the robotic arm at time t+1; This represents the joint state of the robotic arm at time t; To satisfy the normal distribution The nominal speed, This indicates the state of the joint at time t. The nominal displacement at the start; The covariance of the nominal displacement;

[0056] In each iteration round, a set of robotic arm joint states are given. Where H represents the control range from which M nominal sequences are sampled. j=1,...,M; then, the trajectory planner based on the model predicts the path integral and calculates the cost function. To evaluate and update the feasible trajectory of the quadruped robot integrated robotic arm system;

[0057] The cost function Includes: Collision costs ), action cost and target cost );in,

[0058] The collision cost is defined as follows: ;

[0059] The cost of the action is defined as: ;

[0060] The target cost is defined as follows: ;

[0061] in, This represents the value of the directed distance field calculated using externally input point cloud information and the state of the robotic arm itself. ) represents the joint state of the robotic arm at the end of the generated trajectory at time t; This represents the forward dynamics calculation of the robotic arm; This indicates the joint states of the robotic arm at the end of the generated trajectory; This indicates the target position of the robotic arm's end effector.

[0062] Preferably, in step 2 of the above method, the trajectory command is executed by the vector field-based trajectory follower in the following manner:

[0063] A feasible trajectory for the generated robotic arm is tracked using a vector field-based trajectory tracker to output joint velocity commands for the robotic arm. The calculation formula is:

[0064] ;

[0065] Where k is a constant greater than 0; This indicates the gradient; V(q) represents the vector field. Indicates the joint status of the robotic arm; This represents the farthest point in the generated feasible trajectory of the robotic arm; It is 0.0001.

[0066] Preferably, in step 3 of the above method, the information of the quadruped robot integrated robotic arm system's body perception, historical observation, and target point is combined and input into the reinforcement learning network trained in step 1 in the following manner to obtain the movement trajectory. The movement operation of the quadruped robot integrated robotic arm system is controlled by the obtained movement trajectory, including:

[0067] During reinforcement learning training, the proprioception of the quadruped robot integrated with the robotic arm system is incorporated. Robotic historical observation and privilege information Input into reinforcement learning network In the middle, it is:

[0068] ;

[0069] in, The reinforcement learning network outputs a twelve-dimensional vector representing the offset of the joint target position, i.e.:

[0070] ;

[0071] in, Indicates the target position of the joint after offset; This indicates the default joint target position;

[0072] The joint target position output by the reinforcement learning network is tracked by the proportional-differential controller at the bottom layer of the quadruped robot integrated robotic arm system, which outputs control torque. Direct control of the joint motor is as follows:

[0073] ;

[0074] in, Indicates the scaling factor; Indicates the current joint position of the quadruped robot; ; Indicates the velocity of the joint target after offset; This indicates the current joint velocity of the quadruped robot;

[0075] Multi-objective reward computational loss of reinforcement learning networks for:

[0076] ;

[0077] in, For use in directly calculating the distance between the robot's end effector position and the target position. The task reward for positional loss; Style rewards are used to mimic motion priors in mobile operations;

[0078] After each training round's reward, the expected reward of the policy is maximized using the nearest neighbor policy optimization algorithm. for:

[0079] ;

[0080] The meanings of each parameter are as follows: For is the expected return of the parameter; E[·] is the expected return; Let be the discount factor at time t; Let T be the reward at time t, and T be the trajectory duration.

[0081] This invention also provides a processing apparatus, comprising:

[0082] At least one memory for storing one or more programs;

[0083] At least one processor is capable of executing one or more programs stored in the memory, such that when the processor executes one or more programs, the processor can implement the methods described above.

[0084] The present invention further provides a readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0085] In summary, the method provided by the embodiments of the present invention has at least the following advantages and positive effects: It combines traditional model-based quadruped robot control methods with reinforcement learning, and controls the robotic arm through model prediction integral control, thereby improving the mobility of the quadruped robot integrated robotic arm system; it enhances the mobility of the quadruped robot integrated robotic arm system by designing mobility operation priors through trajectory optimization and training deep reinforcement learning with adversarial motion priors; it avoids robotic arm collisions through asynchronous communication control of a robotic arm trajectory planner based on model prediction path integral and a trajectory follower based on vector fields; and it combines the body perception, historical observation, and target point information of the quadruped robot integrated robotic arm system and inputs them into a target-conditional reinforcement learning network to acquire mobility operation skills. Compared with traditional model-based control methods or whole-body controllers for quadruped robot integrated robotic arm systems designed entirely using reinforcement learning, this method has stronger mobility and safety, enabling the quadruped robot integrated robotic arm system to complete mobility operation tasks in various unstructured environments, with low computational complexity.

[0086] To more clearly demonstrate the technical solution and its effects provided by the present invention, the following detailed description of the solution provided by the embodiments of the present invention is provided with reference to specific examples.

[0087] Example 1

[0088] like Figure 1 , Figure 2 As shown, this embodiment provides a control method for a quadruped robot integrated robotic arm system based on adversarial motion prior deep reinforcement learning. This method combines a model-based quadruped robot control method with reinforcement learning. By optimizing the trajectory to design the motion prior for movement operations and using adversarial motion prior to assist deep reinforcement learning training, the mobile operation capability of the quadruped robot integrated robotic arm system is improved. The robotic arm trajectory planner based on model prediction path integral and the trajectory follower based on vector field are used. Through their asynchronous communication control, the quadruped robot integrated robotic arm system can avoid robotic arm collisions during the movement operation.

[0089] The control method for the quadruped robot integrated robotic arm system includes the following steps:

[0090] Step 1: Using a trajectory optimization algorithm based on a single rigid body model, the prior motion of the quadruped robot's movement operation is obtained by manually defining the movement operation skill trajectory, and then input into the reinforcement learning network through adversarial movement prior.

[0091] Step 2: Generate a feasible trajectory for the robotic arm using a trajectory planner based on model-predicted path integrals, and execute trajectory commands using a trajectory follower based on vector fields.

[0092] Step 3: Combine the ontological perception, historical observation and target point information of the quadruped robot integrated robotic arm system and input them into the target conditional reinforcement learning network to acquire mobile operation skills.

[0093] In step 1 of the above method, the processing is as follows:

[0094] In the simulation environment, a single rigid body kinematic model of the quadruped robot is first established, and then a manually defined expert motion trajectory is input. The motion priors for the movement operation are obtained through trajectory optimization calculation. Among them, the expert trajectory Including the linear position of quadruped robots The robot's orientation And the position of the robot's four legs Motion priors for movement operations Including the angles of each joint of the quadruped robot ,in, Let be the number of joints in the quadruped robot. The trajectory optimization described above can be expressed as a nonlinear programming problem:

[0095] ;

[0096] st ;

[0097] ;

[0098] The goal of this invention is to find a way to make the objective function Minimize motion priors At the same time, it satisfies the inequality constraints. Sum of equality constraints The cost function is defined as:

[0099] ;

[0100] in, The cost of the quadratic linear regulator that tracks the expert's trajectory; Let represent the end-point cost; T represents the trajectory duration. To ensure the feasibility of the generated trajectory, the following contact complementary constraints are designed:

[0101] ;

[0102] ;

[0103] in, This represents the z-axis component of the force exerted by the robot's i-th foot on the ground. Let z represent the z-axis direction component of the position of the i-th foot of the robot; in order to combine the acquired motion prior of the movement operation with reinforcement learning, an adversarial motion prior is introduced, and a discriminator is trained. It is a parameter The neural network determines the movement operation by transferring states from the prior motion. and state transitions generated by reinforcement learning training The input is fed into the discriminator to distinguish whether the state style generated during training matches the motion prior of the movement operation. This is done during discriminator training. In the process, the objective function is defined as:

[0104] ;

[0105] The first two terms in the objective function encourage the discriminator to distinguish whether the obtained state is acquired from a priori motion of the movement operation or generated through reinforcement learning. The last term in the objective function adds a gradient penalty to prevent overfitting.

[0106] In step 1 of the above method, the processing is as follows:

[0107] To ensure safety during mobile operations, the robotic arm needs to meet certain constraints, such as avoiding collisions. This invention designs a robotic arm trajectory generator based on model path integral and a trajectory tracker based on vector field. The goal is to determine the target position of the given end effector. Joint states of the robotic arm The robot arm speed command is calculated. Trajectory planners based on model-predicted path integrals are defined as discrete-time, continuous-state systems. .in, Satisfies a normal distribution Represents the state of the joint at time t. The nominal displacement at the start. Let its covariance be given. In each iteration round, given a set of robot arm joint states... Sample M nominal sequences j=1,...,M. Where H represents the control range. Then, the trajectory planner based on the model-predicted path integral calculates the cost function. To evaluate and update the robotic arm trajectory. Cost function. It includes three parts: collision cost ), action cost and target cost ), defined as follows:

[0108] The collision cost is: ;

[0109] The cost of the action is: ;

[0110] The target cost is: ;

[0111] in, This represents the CSDF value calculated using externally input point cloud information and the robotic arm's own state. () represents the joint state of the robotic arm at the end of the generated trajectory at time t. This indicates the forward dynamics calculation of the robotic arm.

[0112] To track the generated trajectory, this invention designs a vector field-based trajectory tracker to output joint velocity commands for the robotic arm. The joint speed command is calculated as follows:

[0113] ;

[0114] in, Let V(q) represent the farthest point in the generated trajectory, V(q) represent the vector field, and k be a constant greater than 0. It is a very small positive number.

[0115] The specific processing method for step 3 of the above method is as follows:

[0116] To train the mobility of the quadruped robot integrated robotic arm system, this invention sets up two common real-world environments: a platform and a pipe.

[0117] During reinforcement learning training, the robot's body perception is integrated into the quadruped robot's robotic arm system. Robotic historical observation and privilege information Input into the reinforcement learning network:

[0118] ;

[0119] Reinforcement learning networks output a twelve-dimensional vector. , representing the offset of the joint target position, that is:

[0120] ;

[0121] in, Indicates the target position of the joint after offset; This indicates the default joint target position;

[0122] The target joint position output by the network is ultimately tracked by the underlying proportional-derivative controller, and the output control torque directly controls the joint motor.

[0123] ;

[0124] in, Indicates the scaling factor; Indicates the current joint position of the quadruped robot; ; Indicates the velocity of the joint target after offset; This indicates the current joint velocity of the quadruped robot;

[0125] The loss of a reinforcement learning network is calculated using a location-based multi-objective reward mechanism, mainly consisting of: task reward. It is used to directly calculate the distance between the robot's end effector position and the target position. Position loss; style reward This is used to mimic prior knowledge of movement operations, accelerating the learning process; therefore, the reward definition of reinforcement learning networks... for:

[0126] ;

[0127] After calculating the policy's reward in each training round, the expected return of the policy is maximized using the nearest neighbor policy optimization algorithm:

[0128] ;

[0129] In summary, the control method of this invention is used to train a quadruped robot integrated robotic arm system to complete movement tasks in an unstructured environment.

[0130] The advantages and positive effects of this invention are as follows: It combines traditional model-based quadruped robot control methods with reinforcement learning, and improves the mobility of the quadruped robot integrated manipulator system by controlling the robotic arm through model prediction integral control; it enhances the mobility of the quadruped robot integrated manipulator system by designing mobility operation priors through trajectory optimization and training deep reinforcement learning with adversarial motion priors; it avoids robotic arm collisions through asynchronous communication control of the robotic arm trajectory planner based on model prediction path integral and the trajectory follower based on vector field; finally, it combines the body perception, historical observation, and target point information of the quadruped robot integrated manipulator system and inputs them into the target conditional reinforcement learning network to acquire mobility operation skills. Compared with traditional model-based control methods or whole-body controllers of quadruped robot integrated manipulator systems designed entirely using reinforcement learning, this method has stronger mobility and safety, enabling the quadruped robot integrated manipulator system to complete mobility operation tasks in various unstructured environments, with low computational complexity.

[0131] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0132] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A control method for a quadruped robot integrated robotic arm system, characterized in that, This is used to control a quadruped robot integrated robotic arm system to perform movement tasks in unstructured environments, including: Step 1: Using a trajectory optimization algorithm based on a single rigid body model, the prior motion of the quadruped robot integrated robotic arm system is obtained through a manually defined movement trajectory. The prior motion of the movement operation is then input into a reinforcement learning network through an adversarial motion prior to obtain a reinforcement learning strategy that can complete the movement operation task. Step 2: Use a trajectory planner based on model prediction path integral to generate a feasible trajectory for the quadruped robot integrated manipulator system, and execute the trajectory command through a trajectory follower based on vector field. Step 3: Combine the body perception, historical observation and target point information of the quadruped robot integrated robotic arm system and input them into the reinforcement learning network trained in Step 1 to obtain the movement operation trajectory. Control the movement operation of the quadruped robot integrated robotic arm system through the obtained movement operation trajectory.

2. The control method for the quadruped robot integrated robotic arm system according to claim 1, characterized in that, In step 1, a trajectory optimization algorithm based on a single rigid body model is used to obtain the prior motion of the quadruped robot integrated robotic arm system through a manually defined movement trajectory, including: In a simulation environment, a single-body kinematic model of the quadruped robot is established, and a manually defined expert motion trajectory is input into the single-body kinematic model. The prior motion of the movement operation was obtained through trajectory optimization calculation. .

3. The control method for the quadruped robot integrated robotic arm system according to claim 2, characterized in that, The human-defined expert motion trajectory include: Linear position of the quadruped robot Orientation of the quadruped robot The position of the four legs of the quadruped robot ; The motion prior of the movement operation obtained through trajectory optimization calculation include: The joint angles of the quadruped robot ,in, This represents the number of joints in a quadruped robot.

4. The control method for the quadruped robot integrated robotic arm system according to claim 3, characterized in that, The trajectory optimization is achieved by finding the objective function through the following nonlinear programming. Minimize motion priors ,Right now: And simultaneously satisfy the inequality constraints Sum of equality constraints ; objective function for: ； in, The cost of the quadratic linear regulator that tracks the motion trajectory of the expert; Represents the end cost of generating the trajectory; Indicates the time; T represents the prior duration of the generated movement operation; The feasibility of the prior motion of the movement operation is guaranteed by the following complementary contact constraints: ； ； in, This represents the z-axis component of the contact force between the i-th foot of the quadruped robot and the ground. This represents the z-axis component of the position of the i-th leg of the quadruped robot.

5. The control method for the quadruped robot integrated robotic arm system according to any one of claims 1-4, characterized in that, In step 1, the adversarial motion priors used are obtained in the following ways: First train a parameter The neural network that determines the discriminator The state transition in the motion prior of the movement operation. and state transitions generated by reinforcement learning training Input to discriminator In this way, we can distinguish whether the state generated during training matches the prior motion of the movement operation; Training the discriminator objective function in the process Defined as: ； The meanings of each parameter are as follows: For The objective function is the parameter. To train the discriminator, its essence is to use A multilayer perceptron with parameters; Let t be the prior state of the movement. This represents the prior state of the movement at time t+1; Let be the state of the quadruped robot at time t. This represents the state of the quadruped robot at time t+1. For parameters Find the gradient.

6. The control method for the quadruped robot integrated robotic arm system according to any one of claims 1-4, characterized in that, In step 2, a feasible trajectory for the quadruped robot integrated robotic arm system is generated using a trajectory planner based on model-predicted path integrals, in the following manner: The trajectory planner based on model-predicted path integrals is defined as a discrete-time, continuous-state system, i.e.: ； in, This indicates the joint state of the robotic arm at time t+1; This represents the joint state of the robotic arm at time t; To satisfy the normal distribution The nominal speed, This indicates the state of the joint at time t. The nominal displacement at the start; The covariance of the nominal displacement; In each iteration round, a set of robotic arm joint states are given. Where H represents the control range from which M nominal sequences are sampled. j=1,...,M; then, the trajectory planner based on the model predicts the path integral and calculates the cost function. To evaluate and update the feasible trajectory of the quadruped robot integrated robotic arm system; The cost function Includes: Collision costs ), action cost and target cost );in, The collision cost is defined as follows: ; The cost of the action is defined as: ; The target cost is defined as follows: ; in, This represents the value of the directed distance field calculated using externally input point cloud information and the state of the robotic arm itself. ) represents the joint state of the robotic arm at the end of the generated trajectory at time t; This represents the forward dynamics calculation of the robotic arm; This indicates the joint states of the robotic arm at the end of the generated trajectory; This indicates the target position of the robotic arm's end effector.

7. The control method for the quadruped robot integrated robotic arm system according to claim 6, characterized in that, In step 2, the trajectory command is executed via a vector field-based trajectory follower in the following manner: A feasible trajectory for the generated robotic arm is tracked using a vector field-based trajectory tracker to output joint velocity commands for the robotic arm. The calculation formula is: ； Where k is a constant greater than 0; This indicates the gradient; V(q) represents the vector field. Indicates the joint status of the robotic arm; This represents the farthest point in the generated feasible trajectory of the robotic arm; It is 0.0001.

8. The control method for the quadruped robot integrated robotic arm system according to any one of claims 1-4, characterized in that, In step 3, the information from the quadruped robot integrated robotic arm system's body perception, historical observation, and target point is combined and input into the reinforcement learning network trained in step 1 in the following manner to obtain the movement trajectory. The movement operation of the quadruped robot integrated robotic arm system is controlled by the obtained movement trajectory, including: During reinforcement learning training, the proprioception of the quadruped robot integrated with the robotic arm system is incorporated. Robotic historical observation and privilege information Input into reinforcement learning network In the middle, it is: ； in, The reinforcement learning network outputs a twelve-dimensional vector representing the offset of the joint target position, i.e.: ； in, Indicates the target position of the joint after offset; This indicates the default joint target position; The joint target position output by the reinforcement learning network is tracked by the proportional-differential controller at the bottom layer of the quadruped robot integrated robotic arm system, which outputs control torque. Direct control of the joint motor is as follows: ； in, Indicates the scaling factor; Indicates the current joint position of the quadruped robot; ; Indicates the velocity of the joint target after offset; This indicates the current joint velocity of the quadruped robot; Multi-objective reward computational loss of reinforcement learning networks for: ； in, For use in directly calculating the distance between the robot's end effector position and the target position. The task reward for positional loss; Style rewards are used to mimic motion priors in mobile operations; After each training round's reward, the expected reward of the policy is maximized using the nearest neighbor policy optimization algorithm. for: ； The meanings of each parameter are as follows: For is the expected return of the parameter; E[·] is the expected return; Let be the discount factor at time t; Let T be the reward at time t, and T be the trajectory duration.

9. A processing device, characterized in that, include: At least one memory for storing one or more programs; At least one processor is capable of executing one or more programs stored in the memory, such that when the one or more programs are executed by the processor, the processor can perform the method according to any one of claims 1-8.

10. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it can implement the method described in any one of claims 1-8.

Images