Variable load balance control method and balance control system of double-wheel-foot type robot and experimental platform

Through reinforcement learning methods, variable load balancing controllers are generated and real environments are simulated through domain randomization technology, which solves the problem that dual-wheel foot robots are difficult to achieve stable balancing control in complex environments, and realizes efficient and flexible balancing control and system. stability.

CN119396197APending Publication Date: 2025-02-07SOUTH CHINA UNIV OF TECH

Patent Information

Application Number
CN202411509081.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-28
Publication Date
2025-02-07

AI Technical Summary

Technical Problem

It is difficult for the dual-wheel foot robot to achieve stable variable load balancing control in complex and changing environments. Traditional control methods have problems such as difficult to obtain models, many parameters, and complex programming and debugging.

Method used

Using reinforcement learning methods to build a simulated physical environment, conduct large-scale parallel training through deep reinforcement learning algorithms, generate variable load balancing controllers, and simulate disturbances and noise in the real environment through domain randomization technology, and finally migrate and deploy the controller to the real In the robot.

Benefits of technology

The control algorithm generation process is simplified, the system reliability and maintainability are improved, the anti-interference ability and balance recovery ability are enhanced, and the two-wheeled foot robots operate more stably and reliably in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119396197A_ABST
    Figure CN119396197A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of robot control, and discloses a variable load balance control method, a balance control system and an experiment platform of a double-wheel-foot robot, and the method comprises the steps: building a simulation physical environment of the double-wheel-foot robot through a reinforcement learning experiment simulation platform; in a reinforcement learning experiment simulation platform, a reinforcement learning algorithm double-wheel-foot type robot variable load balance controller is used; a variable load balance controller is migrated and deployed to a real balance control system of the double-wheel-foot robot, and stable balance control in a real environment is realized in a double-wheel-foot robot experiment platform. By means of the method, the generation process of the control algorithm of the double-wheel-legged robot is simplified, the problem of real machine migration deployment is solved, and the reliability and maintainability of the system are improved. And meanwhile, the anti-interference capability and the stability of the double-wheel-foot type robot during variable load balance control are improved, and self-adaptive control of balance is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of robot control, and in particular to a variable load balancing control method, a balancing control system and an experimental platform for a two-wheeled legged robot. Background Art

[0002] A two-wheeled legged robot is a robot structure that controls the motion of the robot body through driving wheels and leg structures. It has fast and flexible movement capabilities and can be applied to a variety of application scenarios. Since the two-wheeled legged robot is an unstable under-actuated system, there is a problem of balance control. At the same time, for complex and changeable working conditions and tasks that require load, a balance control algorithm with strong robustness and adaptability is the basis for the two-wheeled legged robot to complete various tasks.

[0003] In the related art, the balance control is performed by a whole-body dynamics controller. However, when the two-wheeled legged robot carries a load and the load mass and center of mass are unknown, the balance control effect is often very poor. At the same time, traditional control methods based on robot kinematics and dynamics often have problems such as difficulty in obtaining an accurate mathematical model of the robot system, many control parameters, difficulty in parameter setting, and complex and time-consuming programming and debugging processes. Compared with traditional motion control methods, the motion control of two-wheeled legged robots using reinforcement learning has the advantages of no model, strong environmental applicability, easy parameter setting, and autonomous strategy generation. However, control methods based on reinforcement learning often remain in the simulation stage and are difficult to actually migrate and deploy to real robots. This is because the real physical system has various disturbance noises, which are quite different from the simulated physical system, and this difference will be continuously amplified in actual motion.

[0004] How to achieve stable variable load balance control for a two-wheeled legged robot in a complex and changing environment is an urgent problem to be solved. A new solution is urgently needed to solve the above technical problems. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies of the above-mentioned prior art and to provide a variable load balancing control method, a balancing control system and an experimental platform for a two-wheeled legged robot.

[0006] The object of the present invention is achieved by the following technical solution: The variable load balancing control method of the two-wheeled legged robot comprises the following steps:

[0007] S1. Use the reinforcement learning experimental simulation platform to build a simulated physical environment for a two-wheeled legged robot;

[0008] S2. In the reinforcement learning experiment simulation platform of step S1, a reinforcement learning algorithm is used to perform large-scale parallel training of a variable load balancing controller of a two-wheeled legged robot model to realize autonomous generation of a control strategy;

[0009] S3. Migrate and deploy the variable load balancing controller of step S2 to the balance control system of a real two-wheeled legged robot, and realize stable balance control in a real environment in a two-wheeled legged robot experimental platform.

[0010] A better option is that step S1 includes the following steps:

[0011] S11, exporting a two-wheeled legged robot model from a 3D modeling software, and then importing the two-wheeled legged robot model into a reinforcement learning experiment simulation platform;

[0012] S12, building a simulation platform environment in the reinforcement learning experiment simulation platform of step S11, and setting the physical properties of the simulation platform environment;

[0013] S13. Using domain randomization technology, the simulation platform environment of step S12 is used to simulate the real situation.

[0014] A better option is that step S13 includes the following steps:

[0015] S131, using domain randomization technology to randomize the environmental physical parameters of the physical attributes of step S12;

[0016] S132, using domain randomization technology to add various noises and disturbances to the simulation platform environment;

[0017] S133. Using domain randomization technology, the body mass and center of mass position of the two-wheeled legged robot model are randomly distributed within a given range to simulate the variable load situation of the two-wheeled legged robot in a real scene.

[0018] A better option is that step S2 includes the following steps:

[0019] S21. Acquire various posture state information required by the two-wheeled legged robot model in the simulation platform;

[0020] S22. Design reinforcement learning framework;

[0021] S23. Based on the reinforcement learning framework, balance control training is performed on the two-wheeled legged robot model with the help of deep reinforcement learning to obtain simulation training results;

[0022] S24. According to the simulation training result of step S23, continue to use the proximal policy optimization algorithm to continuously optimize the trained policy network, and select the policy network with better effect as the variable load balancing controller of the two-wheeled legged robot model.

[0023] A better option is that the reinforcement learning framework of step S22 includes the state space, action space and reward function of the reinforcement learning algorithm.

[0024] A better option is that step S3 includes the following steps:

[0025] S31, converting the variable load balancing controller of step S2 into a variable load balancing controller program that can be read and written by a computing device;

[0026] S32. Directly migrate and deploy the variable load balancing controller program to the balance control system of a real two-wheeled legged robot.

[0027] The balancing control system using the variable load balancing control method comprises a data acquisition module, a primary control module, a secondary control module and an execution module connected in sequence;

[0028] The data acquisition module is used to acquire real-time motion state data of the two-wheeled legged robot;

[0029] The primary control module is used to read the real-time motion state data, input the required data into the balance controller, and output the desired joint positions and the desired driving wheel speeds required to control the two-wheeled legged robot to perform the control task;

[0030] The secondary control module is used to receive the desired joint positions and the desired driving wheel speeds from the primary control module, and obtain the joint and driving wheel torques required to control the two-wheeled legged robot to perform the control task after calculation and processing;

[0031] The execution module executes the joint and driving wheel torques output by the secondary control module to drive the two-wheeled legged robot to restore or maintain balance.

[0032] The balance control experimental platform of the two-wheeled and legged robot includes a mechanical body structure, a measuring unit, a computing device, a robot real-time control module and a power supply module. The measuring unit and the robot real-time control module are both installed on the mechanical body structure, the measuring unit is connected to the computing device, the power supply module is respectively connected to the measuring unit, the computing device and the robot real-time control module, and the computing device is used to install the balance control system.

[0033] A more preferred option is that the mechanical body mechanism includes a driving wheel, a leg structure and a fuselage, the upper end of the leg structure is rotatably connected to the fuselage, and the lower end of the leg structure is rotatably connected to the driving wheel.

[0034] A better choice is that the robot real-time control module includes a wheel hub motor, a knee joint motor and a hip joint motor, the wheel hub motor is installed on the driving wheel, the knee joint motor is installed on the leg structure, the hip joint motor is installed on the fuselage, and the wheel hub motor, the knee joint motor and the hip joint motor are all connected to the power supply module.

[0035] The present invention has the following advantages and beneficial effects compared with the prior art:

[0036] The present invention can simplify the control algorithm generation process of the two-wheeled legged robot through the variable load balance control method, balance control system and experimental platform of the two-wheeled legged robot, make it more efficient and flexible, and solve the problem of real machine migration deployment by introducing a variety of randomized disturbance noises similar to reality, thereby improving the reliability and maintainability of the system. At the same time, it can also improve the anti-interference ability of the two-wheeled legged robot during balance control, provide its balance recovery ability under variable load conditions and stability in the balance state, so that the two-wheeled legged robot can operate more stably and reliably in a complex external environment, and realize balanced adaptive control. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic structural diagram of the two-wheeled legged robot of the present invention;

[0038] Figure 2 It is a flow chart of a variable load balancing control method of a two-wheeled legged robot of the present invention;

[0039] Figure 3 It is a flow chart of the dual-wheeled legged robot reinforcement learning training of the variable load balancing control method of the dual-wheeled legged robot of the present invention;

[0040] Figure 4 It is a schematic flow chart of the migration of a variable load balancing controller of the variable load balancing control method of the two-wheeled legged robot of the present invention;

[0041] Figure 5 It is a schematic flow chart of the balance control system of the two-wheeled legged robot of the present invention;

[0042] Figure 6 Schematic diagram of the experimental platform of the two-wheeled legged robot of the present invention;

[0043] The markings of the components in the accompanying drawings are: 1-fuselage; 2-leg structure; 3-driving wheel. DETAILED DESCRIPTION

[0044] The purpose of the present invention is further described in detail below with reference to the accompanying drawings and specific examples. The examples cannot be described one by one here, but the implementation methods of the present invention are not therefore limited to the following examples.

[0045] A two-wheeled legged robot is a robot structure that uses a leg structure with joints and a drive wheel to control the motion of the robot body. Since the contact points between the two-wheeled legged robot and the ground only include the contact points between the drive wheel and the ground, it is an unstable under-actuated system and has balance control problems.

[0046] The variable load balancing controller is a method for balancing the two-wheeled legged robot pre-set in this embodiment, which is generated by deep reinforcement learning training on the NVIDIA Isaac Gym platform. It is migrated and deployed to the real two-wheeled legged robot through code programming and other methods to achieve the balance motion control of the two-wheeled legged robot.

[0047] Deep reinforcement learning (DRL) is a fusion of deep learning and reinforcement learning. It combines the advantages of deep learning and reinforcement learning, can solve many complex decision-making and control problems, and has strong learning and adaptive capabilities. It can directly extract useful features from input data and achieve end-to-end learning from raw input to final action. This means that the system can automatically optimize strategies from environmental feedback without determining intermediate steps or features in advance, and thus exhibits excellent performance in complex and changing environments and tasks. In addition, the structure of deep neural networks enables DRL to process different types of data and has strong generalization capabilities. This means that the experience learned from the simulation environment can be effectively transferred to the actual environment, and reasonable decisions can still be made in new situations outside the simulation environment.

[0048] When deep reinforcement learning is combined with a two-wheeled legged robot, the advantages of both can be fully utilized. With the help of deep reinforcement learning, the two-wheeled legged robot can better understand and adapt to various complex environments and tasks. Through large-scale parallel training on the GPU, continuous trial and error and learning, it can quickly self-generate control strategies, simplify the robot control algorithm design and generation process, reduce the workload of debugging and optimization, shorten the development cycle, and improve efficiency. At the same time, the introduction of a variety of randomized disturbance noises similar to reality solves the problem of real machine migration deployment, and helps to improve the reliability and maintainability of the system, so that the real prototype has strong robustness and good adaptability when facing complex environments, and achieves balanced adaptive control.

[0049] like Figure 1As shown, the mechanical body mechanism includes a driving wheel, a leg structure and a fuselage, and the robot real-time control module includes a wheel hub motor, a knee joint motor and a hip joint motor. The upper end of the leg structure is rotatably connected to the fuselage, and the lower end of the leg structure is rotatably connected to the driving wheel. The wheel hub motor is mounted on the driving wheel, the knee joint motor is mounted on the leg structure, and the hip joint motor is mounted on the fuselage. The wheel hub motor, the knee joint motor and the hip joint motor are all connected to the power supply module.

[0050] like Figure 2 As shown, the variable load balancing control method of the two-wheeled legged robot is used to quickly self-generate a control strategy to realize the balancing control of the two-wheeled legged robot, which includes the following steps:

[0051] S1. Use the reinforcement learning experiment simulation platform to build a simulated physical environment for the two-wheeled legged robot. The reinforcement learning experiment simulation platform is the open source NVIDIA Isaac Gym platform (the first framework launched by NVIDIA that focuses on high-performance physical simulation and reinforcement learning development, designed for large-scale parallel computing and deep reinforcement learning). The simulated physical environment includes the simulated physical model of the two-wheeled legged robot and the physical properties of the surrounding environment. Domain randomization technology is used to introduce a large number of random changes to simulate actual conditions, reduce the gap from simulation to reality (Sim-to-Real), and improve the migration success rate of the deployment of real two-wheeled legged robots; at the same time, for variable load conditions, domain randomization technology is used during the training process to add different loads to the two-wheeled legged robot model to improve the robustness and adaptability of the two-wheeled legged robot under variable load conditions. The simulation model and environment can be displayed in a visual way in the platform. Step S1 includes the following steps:

[0052] S11. Export a two-wheeled legged robot model from 3D modeling software (i.e., Solidworks software), and then import the two-wheeled legged robot model into a reinforcement learning experimental simulation platform (i.e., NVIDIA Isaac Gym platform); establish a two-wheeled legged robot model based on the relative relationship between the joints and connecting rods of a real two-wheeled legged robot, mass inertia properties, physical geometric characteristics, and collision model.

[0053] S12, building a simulation platform environment in the reinforcement learning experiment simulation platform of step S11, and setting the physical properties of the simulation platform environment. The construction of the simulation platform environment takes into account the actual control conditions, such as the control cycle, the external force on the robot, the real ground friction and the real terrain conditions.

[0054] S13, using domain randomization technology, using the simulation platform environment of step S12 to simulate the real situation. Using domain randomization technology, reduce the gap between simulation and reality (Sim-to-Real) and improve generalization ability. Randomize the physical parameters of the environment, such as the physical model parameters, initial conditions, friction coefficient, gravity, etc. of the two-wheeled legged robot simulation, and add noise and disturbances such as sensor noise, actuator noise, random external force disturbances. Step S13 includes the following steps:

[0055] S131, using domain randomization technology to randomize the environmental physical parameters of the physical attributes of step S12;

[0056] S132, using domain randomization technology to add various noises and disturbances to the simulation platform environment;

[0057] S133. Using domain randomization technology, the body mass and center of mass position of the two-wheeled legged robot model are randomly distributed within a given range to simulate the variable load situation of the two-wheeled legged robot in a real scene.

[0058] S2, such as Figure 3 As shown, in a reinforcement learning experimental simulation platform (i.e., the NVIDIA Isaac Gym platform), a reinforcement learning algorithm is used to perform large-scale parallel training of a variable load balancing controller of a two-wheeled legged robot model to achieve autonomous generation of control strategies. In the NVIDIA Isaac Gym platform, various posture state information of the two-wheeled legged robot model required for reinforcement learning training is obtained from the simulation environment, and the parameters affecting the balance control performance of the two-wheeled legged robot model are determined and optimized. The proximal policy optimization (PPO) algorithm is used in the interaction between the two-wheeled legged robot model and the simulation environment, and large-scale parallel training of the variable load balancing controller of the two-wheeled legged robot model is performed on a GPU (graphics processing unit) to achieve autonomous generation of control strategies. Step S2 includes the following steps:

[0059] S21. Obtain various posture state information required by the two-wheeled and legged robot model in the simulation platform, such as height, position, speed, force, torque, etc.

[0060] S22. Design a reinforcement learning framework, which includes two different neural networks, namely the actor network and the critic network.

[0061] Actor network (actor): takes the current state as input and outputs the probability distribution of actions or continuous action values. It can fully explore the current environment and slowly update the strategy. Its structure is [64×32×32].

[0062] Critic network: evaluates the action value of the current state to evaluate the quality of the strategy, and helps reduce the deviation of the RL algorithm in estimating the policy gradient. Its structure is [256×128×64].

[0063] The reinforcement learning framework includes the state space, action space and reward function of the reinforcement learning algorithm:

[0064] State space: The state space includes the observations of the two-wheeled legged robot (actor network) and the privileged observations (critic network). The observations of the two-wheeled legged robot include the body attitude angular velocity, body attitude angle, command, joint angle, joint angular velocity, and the output value of the neural network at the previous moment. The privileged observations include the observations of the two-wheeled legged robot, body linear velocity, joint relative rotation angle, joint angular acceleration, joint torque, mass attribute parameters, friction coefficient, and information related to interaction with the terrain.

[0065] Action space: includes the expected torque of the four joints of the leg structure and the expected torque of the two driving wheels; the expected torque of the four joints of the leg structure is calculated by the output expected joint position through a PD controller (proportional plus derivative controller), and the expected torque of the two driving wheels is calculated by the output expected speed through a P controller (that is, a proportional controller).

[0066] Reward function: includes command tracking, stability and collision; command tracking includes linear velocity reward, yaw angular velocity reward, pitch angle reward, roll angle reward, altitude reward, vertical velocity penalty; stability includes action smoothness reward, posture penalty, left and right fork leg penalty, joint constraint penalty, torque penalty, joint angular velocity penalty, joint angular acceleration penalty; collision includes fuselage collision and leg structure collision.

[0067] S23. Based on the reinforcement learning framework, the balance control training of the two-wheeled legged robot model is carried out with the help of deep reinforcement learning. The proximal policy optimization (PPO) algorithm is used to train the actor network and the critic network. By continuously updating the parameter values ​​of the policy network (that is, the actor network) in the PPO algorithm, the parameter values ​​of the action space are obtained from the policy network, and the parameter values ​​of the action space are directly applied to the simulation environment. The two-wheeled legged robot executes the target task given by the instruction. At this time, the parameter values ​​of the state space are updated and the domain randomization technology is continued to be used to add noise and interference. The reward value is calculated and the next iteration is performed until the performance index converges or the maximum number of iterations is reached to realize the autonomous generation of the control strategy.

[0068] S24. According to the simulation training results, you can choose to continue to use the PPO deep reinforcement learning algorithm to continuously optimize the trained policy network, and select the policy network with better results as the variable load balancing controller of the two-wheeled legged robot.

[0069] S3, such as Figure 4 As shown, the variable load balancing controller of step S2 is migrated and deployed to the balance control system of a real two-wheeled legged robot, and stable balance control in a real environment is realized in the two-wheeled legged robot experimental platform. Step S3 includes the following steps:

[0070] S31, directly converting the control parameters used in the training (i.e., the parameters of the P controller and the PD controller) and the policy network obtained through the training into a variable load balancing controller program that can be read and written by a computing device through a relevant tool chain;

[0071] S32, directly migrate and deploy the variable load balance controller program to the balance control system of the real two-wheeled legged robot. Obtain real-time motion state data through the data acquisition module, input the required data into the variable load balance controller, and output the expected joint angle and driving wheel speed of the two-wheeled legged robot. And as in the training process, the expected joint angle is calculated by the secondary control module (the joint is the PD controller and the driving wheel is the P controller), and the corresponding torque output is obtained, and the torque output is applied to the execution module of the two-wheeled legged robot, realizing the variable load balance control of the two-wheeled legged robot experimental platform in a real environment.

[0072] like Figure 5 As shown, the balance control system of the two-wheeled legged robot includes a data acquisition module, a primary control module, a secondary control module and an execution module which are connected in sequence.

[0073] The data acquisition module refers to sensors such as the inertial measurement unit (IUM) and joint encoders, and is converted into the real-time motion state data we need through the corresponding program, which is used to obtain the real-time motion state data of the two-wheeled legged robot. The real-time motion state data is used to represent the current motion characteristics of the two-wheeled legged robot;

[0074] The primary control module refers to the variable load balancing controller program converted from the strategy network, which is used to read the real-time motion state data, input the required data into the balancing controller, and output the desired joint positions and driving wheel speeds required to control the two-wheeled legged robot to perform the control task;

[0075] The secondary control module refers to the PD controller and P controller corresponding to the action space in the simulation training, which are used to receive the expected joint positions and driving wheel speeds from the primary control module, and obtain the joint and driving wheel torques required to control the two-wheeled legged robot to perform the control task after calculation and processing;

[0076] The execution module refers to the joint motors and wheel hub motors and their driver programs, which execute the joint torque output by the secondary control module to drive the two-wheeled legged robot to restore or maintain balance.

[0077] like Figure 6 As shown, the balance control experimental platform of the two-wheeled legged robot includes a mechanical body structure, a measuring unit, a computing device, a robot real-time control module and a power supply module. The measuring unit and the robot real-time control module are both installed on the mechanical body structure, the measuring unit is connected to the computing device, and the power supply module is respectively connected to the measuring unit, the computing device and the robot real-time control module.

[0078] Mechanical body mechanism: includes driving wheels, leg structures and fuselage, wherein the leg structure includes thighs and calves, the thighs are connected to the fuselage through hip joints and can rotate around the hip joints, the calves are connected to the thighs through knee joints and can rotate around the knee joints, and the driving wheels are connected to the calves through ankle joints and can rotate around the ankle joints.

[0079] Measuring unit: including sensors such as inertial measurement unit (IMU) and joint encoders, which can measure the current position and other states of the mechanical body mechanism and are used to obtain the mechanical body mechanism information.

[0080] Computing device: a real-time kernel system is configured to receive feedback signals from the measuring unit and output control instructions through each feedback signal, and the computing device includes two parts: a memory and a processor.

[0081] A memory storing at least one instruction, at least one program, a code set or an instruction set, that is, storing a balance control system, wherein the at least one instruction, the at least one program, the code set or the instruction set can be loaded and executed by the processor to implement the control method of the two-wheeled legged robot as described above;

[0082] The processor can read relevant instructions or programs from the memory and execute the instructions or programs, so that the computing device can execute the control method of the two-wheeled legged robot described in any of the above embodiments.

[0083] Robot real-time control module: includes wheel hub motors, knee joint motors and hip joint motors, etc., receives control instructions from the computing device, and is used to perform real-time control on the mechanical body mechanism.

[0084] Power supply module: provides energy for the measurement unit, computing device and robot real-time control module.

[0085] The above specific implementation modes are preferred embodiments of the present invention and cannot be used to limit the present invention. Any other changes or other equivalent replacement methods that do not deviate from the technical solution of the present invention are included in the protection scope of the present invention.

Claims

1. A variable load balancing control method for a two-wheeled legged robot, characterized in that: The following steps are involved: S1. Use the reinforcement learning experimental simulation platform to build a simulated physical environment for a two-wheeled legged robot; S2. In the reinforcement learning experiment simulation platform of step S1, a reinforcement learning algorithm is used to perform large-scale parallel training of a variable load balancing controller of a two-wheeled legged robot model to realize autonomous generation of a control strategy; S3. Migrate and deploy the variable load balancing controller of step S2 to the balance control system of a real two-wheeled legged robot, and realize stable balance control in a real environment in a two-wheeled legged robot experimental platform.

2. The variable load balancing control method of the two-wheeled legged robot according to claim 1, characterized in that: Step S1 includes the following steps: S11, exporting a two-wheeled legged robot model from a 3D modeling software, and then importing the two-wheeled legged robot model into a reinforcement learning experiment simulation platform; S12, building a simulation platform environment in the reinforcement learning experiment simulation platform of step S11, and setting the physical properties of the simulation platform environment; S13. Using domain randomization technology, the simulation platform environment of step S12 is used to simulate the real situation.

3. The variable load balancing control method of the two-wheeled legged robot according to claim 2, characterized in that: Step S13 includes the following steps: S131, using domain randomization technology to randomize the environmental physical parameters of the physical attributes of step S12; S132, using domain randomization technology to add various noises and disturbances to the simulation platform environment; S133. Using domain randomization technology, the body mass and center of mass position of the two-wheeled legged robot model are randomly distributed within a given range to simulate the variable load situation of the two-wheeled legged robot in a real scene.

4. The variable load balancing control method of the two-wheeled legged robot according to claim 1, characterized in that: Step S2 includes the following steps: S21. Acquire various posture state information required by the two-wheeled legged robot model in the simulation platform; S22. Design reinforcement learning framework; S23. Based on the reinforcement learning framework, balance control training is performed on the two-wheeled legged robot model with the help of deep reinforcement learning to obtain simulation training results; S24. According to the simulation training result of step S23, continue to use the proximal policy optimization algorithm to continuously optimize the trained policy network, and select the policy network with better effect as the variable load balancing controller of the two-wheeled legged robot model.

5. The variable load balancing control method of the two-wheeled legged robot according to claim 4, characterized in that: The reinforcement learning framework of step S22 includes the state space, action space and reward function of the reinforcement learning algorithm.

6. The variable load balancing control method of the two-wheeled legged robot according to claim 1, characterized in that: Step S3 includes the following steps: S31, converting the variable load balancing controller of step S2 into a variable load balancing controller program that can be read and written by a computing device; S32. Directly migrate and deploy the variable load balancing controller program to the balance control system of a real two-wheeled legged robot.

7. A balancing control system using the variable load balancing control method according to any one of claims 1 to 6, characterized in that: It includes a data acquisition module, a primary control module, a secondary control module and an execution module which are connected in sequence; The data acquisition module is used to acquire real-time motion state data of the two-wheeled legged robot; The primary control module is used to read the real-time motion state data, input the required data into the balance controller, and output the desired joint positions and the desired speeds of the driving wheels required to control the two-wheeled legged robot to perform the control task; The secondary control module is used to receive the desired joint positions and the desired driving wheel speeds from the primary control module, and obtain the joint and driving wheel torques required to control the two-wheeled legged robot to perform the control task after calculation and processing; The execution module executes the joint and driving wheel torques output by the secondary control module to drive the two-wheeled legged robot to restore or maintain balance.

8. The balance control experimental platform of the two-wheeled and footed robot is characterized by: It includes a mechanical body structure, a measuring unit, a computing device, a robot real-time control module and a power supply module. The measuring unit and the robot real-time control module are both installed on the mechanical body structure. The measuring unit is connected to the computing device. The power supply module is respectively connected to the measuring unit, the computing device and the robot real-time control module. The computing device is used to install the balance control system described in claim 7.

9. The balance control experimental platform of the two-wheeled legged robot according to claim 8, characterized in that: The mechanical body mechanism comprises a driving wheel, a leg structure and a fuselage, the upper end of the leg structure is rotatably connected to the fuselage, and the lower end of the leg structure is rotatably connected to the driving wheel.

10. The balance control experimental platform of the two-wheeled legged robot according to claim 9, characterized in that: The robot real-time control module includes a wheel hub motor, a knee joint motor and a hip joint motor. The wheel hub motor is installed on the driving wheel, the knee joint motor is installed on the leg structure, and the hip joint motor is installed on the fuselage. The wheel hub motor, the knee joint motor and the hip joint motor are all connected to the power supply module.

Citation Information

Patent Citations

  • Quadruped robot motion control self-generation and real object migration method

    CN112631131A

  • Foot type robot gait self-learning method

    CN115016325A

  • Control method of lower limb exoskeleton rehabilitation robot with balance assisting function

    CN115157247A

Cited By

  • A load adaptive control method for a dual-wheel dual-arm carrying robot

    CN122807924A