Robot gait control method and device suitable for walking on slippery road surface and biped robot

Abstract

The invention discloses a robot gait control method and device suitable for walking on a slippery road surface and a biped robot. A reinforcement learning and MPC combined control mode is adopted, the robot is controlled to carry out normal gait walking, and data of joint motors and foot sensors are collected in real time; the state of the robot is judged, specifically, when the robot possibly overturns, the robot enters a slow gait mode, and the judgment standard that overturning possibly occurs is that the states of n steps in the future are deduced in the simulation deduction environment, and whether the n step presents the overturning trend or not is judged; and when the robot overturns, the robot enters a high-frequency gait mode, and a related gait regulation and control mode is designed. The problem that the biped robot easily slips on a wet and slippery road surface is solved; and when slipping occurs, the problem is difficult to remedy. And the stable walking capability of the robot on a complex road surface is improved.

Description

Technical Field

[0001] The present invention relates to the field of robots, and particularly to a gait control method, device and biped robot for walking on slippery roads. Background Art

[0002] A biped robot is an anthropomorphic machine designed to mimic human movement and behavior. However, they are prone to slipping on wet and slippery ground, which may cause very costly losses. Despite significant progress in robot movement, walking on slippery floors remains a difficult and unsolved problem with great potential for further improvement. Just as humans can use sensory feedback information to predict when they are about to slip and take adaptive measures to avoid slipping, we believe that biped robots should rely on feedback mechanisms to achieve the same goal and improve their ability to walk steadily on slippery roads.

[0003] To achieve motion efficiency trajectory tracking, the design of a biped robot requires additional complexity. For this purpose, two main parallel controllers need to be implemented: one for high-level trajectory tracking and the other related to maintaining a specified motion gait, that is, the specified phase relationship between the legs. Commonly used high-level control methods in biped robots rely on model predictive control (MPC). MPC is a well-known control strategy that has been widely used in academic research and industrial fields in recent decades. In particular, linear model predictive control (LMPC) assumes that the system dynamics can be effectively approximated as linear, simplifies the robot model configuration, reduces the leg inertia, and solves the high-level steering control problem. In past research, researchers assumed that adjusting the robot's controller according to various known floor friction coefficients could enable the robot to move efficiently on these surfaces. Based on the development of foot sensors and machine learning algorithms, the friction coefficients of various floor types can be detected in real time, thereby adaptively adjusting the control model input to enhance the movement ability of humanoid robots on smooth surfaces.

[0004] However, in real situations, slipping always occurs suddenly, and the robot may not have enough time to react to the slip. In addition, due to the complex explicit dynamic model of biped robots, the controller cannot use it to calculate accurate and appropriate responses. Traditional LMPC also has significant limitations. Due to complex inertial interactions, it is particularly difficult to formalize the accurate non-linear analysis model of real robots when moving on slippery surfaces; on the other hand, directly deploying such control strategies in biped robots is very dangerous, especially when requiring movement on slippery terrains. Therefore, using an accurate dynamic simulation environment is crucial for correctly evaluating the robot's performance and application limits. Summary of the Invention

[0005] To address these challenges, this paper proposes a method, device, and robot for gait adjustment on slippery surfaces based on adaptive reinforcement learning. This method utilizes real-time sensor data to provide road surface information, employs simulation and real-time planning to predict and mitigate potential risks, and adaptively adjusts gait to quickly maintain balance when a slip is imminent. The research explores a method for the robot to maintain stable walking on complex surfaces.

[0006] The robot walking gait control method based on adaptive reinforcement learning involved in the present invention includes the following process: A control method combining reinforcement learning and MPC is used to control the robot to walk normally and collect data from joint motors and foot sensors in real time. Perform force analysis on the real-time collected data and monitor the robot's walking status: When the robot is likely to overturn, it enters the slow gait mode. The judgment criteria for the possibility of overturning are to deduce the future in the simulation environment. n The status of the step, judge the n Whether the step shows a tendency to overturn; When the robot overturns, it enters the high-frequency gait mode; The normal gait, high-frequency gait and slow gait are all controlled by a combination of reinforcement learning and model predictive control (MPC), with gait frequency adjustment performed only at the output layer. After the high-frequency gait pattern or the slow gait pattern is completed, it is determined whether to switch to normal gait walking. The determination includes setting a fixed time or setting conditions for determination.

[0007] Furthermore, the optimization strategy of the reinforcement learning is: in, is the robot's trajectory, It is the execution strategy, express exist The likelihood estimate under r t Represents the total reward, rforward represents the forward reward, rrotation represents the yaw angle reward, rsmooth represents the torque transmission reward, rground represents the contact force feedback reward, rslip represents the lateral sliding speed of the foot, and rregular represents the regular regularization penalty.

[0008] Furthermore, each reward item is defined as follows: in, v x is the robot's forward speed, v y is the speed of the robot in the direction orthogonal to the forward direction, ωyaw represents the lateral angular velocity, F is the ground reaction force collected by the aforementioned sensors, the foot velocity is represented by, and the binary indication vector representing the foot contact state is used to ensure the non-negativity of the reward term.

[0009] Preferably, in real-time monitoring, zero moment is used to judge the position of the robot's center of mass, thereby judging whether the robot is on the verge of tipping over.

[0010] Preferably, in the prediction of the robot's next n steps, the robot is simplified to an inverted pendulum system with a concentrated mass at the center of mass, the contact point between the supporting foot and the ground is regarded as the fulcrum of the pendulum, a dynamic model is established, and the linear inverted pendulum model LIPM is used. Assuming that the height of the center of mass is fixed, the following dynamic equation is satisfied: where, x is the horizontal position of the center of mass, z 0 is the height of the center of mass, g represents the acceleration due to gravity; With the comprehensive index E t indicating the future period t i.e., in the future n steps, the cumulative error of the robot's click angle deviation, foot force change, and center-of-mass dynamic behavior: where represents the deviation of the joint angle, represents the fluctuation of the foot force, represents the fluctuation of the energy consumption, is the weight parameter set according to the importance of the slip risk. In the simulation environment, Et is calculated in real time. If Et is greater than the slip threshold Ethreshold, it is determined that the robot will slip.

[0011] The linear inverted pendulum model LIPM can greatly simplify the robot model while losing fewer features, achieve a high fidelity rate while reducing the computational cost, and is very suitable for scenarios with high real-time requirements.

[0012] Preferably, the adjustment process of the high-frequency gait is as follows: Design an additional reward for the high-frequency gait pattern to achieve the goal:

[0013] where, constrains the gait frequency, as the reference limit frequency control; is the gait symmetry reward; training the network to output a smaller T stance , T swing , that is, the swing frequency of the two feet is increased.

[0014] An additional definition is made for the cost function of the reinforcement learning algorithm to minimize the step distance as much as possible during the optimization process:

[0015] Among them, represents the relative distance between the centroid and the supporting foot.

[0016] Preferably, the adjustment process of the slow gait includes: Based on the original reward function of reinforcement learning, further design is made:

[0017] Among them, is the weighted reward in the original reinforcement learning; represents the current stride; is the set small stride target, output by the reinforcement learning strategy; represents the linear velocity of the swinging foot, punishing too fast swinging; , , are the corresponding weight coefficients; by online updating the policy parameters, the amplitude and frequency of adjusting the gait are made more accurate, and at the same time, the gait parameters are dynamically optimized in combination with the development trend of the slipping risk, so that it quickly converges to a more stable gait pattern when predicting the occurrence of slipping.

[0018] Based on the same inventive concept, the present invention also designs a bipedal humanoid robot for a gait control method suitable for walking on a slippery road surface, including: including a robot body and a motion control module; The robot body shown selects a high burst and high torque servo motor, and the foot is equipped with a force sensor to real-time feedback the force information; The motion control module adopts a combined control method of reinforcement learning and MPC to control the robot to walk in a normal gait and real-time collect data of each joint motor and foot sensor; Perform a force analysis on the real-time collected data and monitor the walking state of the robot: When the robot may tip over, it enters the slow gait mode. The criterion for judging possible tipping over is to deduce the state of the next n steps in the simulation environment and judge whether the n step shows a tipping trend; When the robot tips over, it enters the high-frequency gait mode; The normal gait, high-frequency gait and slow gait all adopt a combined control method of reinforcement learning and model predictive control MPC, and only adjust the gait frequency at the output layer; After determining the completion of the high-frequency gait pattern or the slow gait pattern, the robot switches to normal gait walking. The determination includes setting a fixed time or setting conditions for judgment.

[0019] Based on the same inventive concept, the present invention also designs a computer-readable storage medium storing computer instructions for causing a computer to execute a robot gait control method adapted to walking on a slippery road surface.

[0020] Based on the same inventive concept, the present invention also designs an electronic device including a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute a robot gait control method adapted to walking on a slippery road surface.

[0021] The advantages of the present invention are as follows: 1. The present invention innovatively constructs a simulation-deduction - real-time planning mode. Pre-play in the simulation environment can give early warnings of incorrect steps and correct the gait in advance. Real-time planning is the actual implementation of the simulation-deduction and fine-tunes in combination with the actual environment and the body state to achieve optimal control, enhancing the stability of the robot on a slippery road surface.

[0022] 2. The present invention innovatively proposes a high-frequency gait strategy. According to the critical balance condition of the robot, the gait is adjusted in a timely manner to avoid slipping. This strategy can reduce the risk of falling and the loss of damage to the body.

[0023] 3. The present invention innovatively proposes an adaptive reinforcement learning gait adjustment method, which adaptively adjusts the gait according to different environmental conditions, adopts a normal gait during normal walking, a slow gait when it is predicted that slipping will occur, and a high-frequency gait when slipping has occurred, and smoothly realizes the transition between gaits. This method enhances the robustness of the robot's walking and effectively prevents unpredictable losses caused by accidents such as locking. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic diagram of the overall process of the present invention.

[0025] Figure 2 is an architecture diagram of the combined control algorithm of reinforcement learning and MPC of the present invention.

[0026] Figure 3 is a front view of the force on the foot of a simplified biped robot.

[0027] Figure 4 is a flowchart of the simulation-deduction - real-time planning mode of the present invention.

[0028] Figure 5It is a schematic diagram of the specific implementation details of the simulation deduction - real - time planning of the present invention.

[0029] Figure 6 It is a flowchart of the adaptive reinforcement learning training of the present invention. Detailed implementation manners

[0030] To facilitate the understanding and implementation of the present invention by those of ordinary skill in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0031] Embodiment 1 The present invention designs three gaits: normal gait, high - frequency gait, and slow gait. Under normal circumstances, the robot will walk with a normal gait; when the robot has an unavoidable slip, it will switch to a high - frequency gait; when the robot predicts that a slip may occur, it will switch to a slow gait. Since there are significant dynamic differences between the three gaits, all three gaits adopt a control method based on reinforcement learning, share the network structure and training parameters, and only make targeted adjustments in the output layer. This can largely avoid the sudden change of torque parameters during gait switching and achieve an adaptive and smooth transition between different gaits.

[0032] This embodiment discloses a gait control method for a robot adapted to walk on a slippery road surface, which includes the following steps: Step S1, the robot adopts a control method combining reinforcement learning (RL) and model predictive control (MPC) to walk normally on a flat or slippery road surface with a certain set gait, and realizes the unity of flexibility and stability through hierarchical control. During this process, RL is responsible for generating high - level gait strategies and dynamically adjusting global parameters such as step length and step frequency according to the environment and robot state; MPC, as a low - level controller, optimizes the robot state trajectory in a short time and generates joint torque or position control instructions to ensure dynamic balance and walking stability. During the walking process of the robot, each joint motor collects data such as angle and torque in real time through a high - precision encoder to feedback the current motion state. The specific implementation process is described as follows: S1 - 1, the target speed or position target of the robot's torso is converted into a state trajectory and transmitted to RL. RL evaluates the optimal state and input parameters of the system according to the optimization algorithm. Then, MPC uses these optimized states and input parameters to calculate the required joint torque. This torque is sent to the motor controller of the robot as a feed - forward term, and at the same time, it is combined with low - gain joint - space position and speed PD control instructions to reduce the impact force when the foot touches the ground and improve the tracking accuracy of the motion. For this reason, a reinforcement learning optimization strategy is designed: Among them, is the motion trajectory of the robot, is the execution strategy, denotes the likelihood estimate under ; r t denotes the total reward, r forward represents the forward reward, r rotation represents the yaw angle reward, r smooth represents the torque transmission reward, r ground represents the contact force feedback reward, r slip represents the lateral foot sliding speed, r regular represents the conventional regularization penalty, denotes the discount factor of future rewards for immediate rewards, t represents the time step.

[0033] A preferred way is to define the above sub - reward terms as follows: Among them, v x is the speed of the robot in the forward direction, v y is the speed of the robot in the direction orthogonal to the forward direction, ω yaw represents the lateral angular velocity, F is the ground feedback force collected by the aforementioned sensors, represents the foot speed, and is a binary indicator vector representing the foot contact state. To ensure the non - negativity of the reward terms. To ensure the consistency of the scales of each reward term, the scaling factors are designed as 20, 23, 0.005, 0.03, 0.9 respectively; S1 - 2, During the normal walking process of the robot, the foot sensors continuously monitor the contact information between it and the ground, including data such as vertical support force, horizontal shear force, and contact time. The inertial measurement unit IMU real - time senses the overall posture and acceleration information of the robot, providing a basis for dynamic adjustment of the control algorithm. The joint torques and rotation angles of each motor are obtained in real - time, where i takes values of 0, 1, 2, 3, 4 to identify different motor numbers; a takes values of 0, 1 to represent the left and right legs. The foot sensors feedback the reaction force of the sole of the foot received from the ground F .

[0034] Specifically, the robot designed in this embodiment has 10 freely movable joints. To distinguish each motor, a identifies the left and right legs, and within the same leg,i Denote hip roll, hip yaw, hip pitch, knee pitch and ankle pitch.

[0035] Step S2: When the robot suddenly slips, since the explicit dynamic model of the biped robot is very complex, the controller may not have enough time to react to the slip. Therefore, a high-priority adaptation method with fast response ability is designed, and the specific implementation steps are as follows: S2-1: The movement of the robot is mainly affected by three forces: gravity, driving force generated by the motor, and foot support force. Among them, the driving force of the motor has an interaction relationship to counteract the support force of the ground on the sole of the foot, while the gravity is constant. Therefore, the model can be simplified to only consider the effect of the foot support force. When slipping occurs, the reading of the foot pressure sensor drops suddenly, and the lateral value of the body IMU linear accelerometer changes abruptly.

[0036] Furthermore, the ZMP describes a specific point when the robot contacts the ground. At this point, the resultant force and moment of the force exerted by the robot on the ground have no tipping moment, that is, all dynamic effects can be described by pure force without the tendency to tip over. According to the zero moment point theory, when the ZMP is inside the support polygon, the robot is considered to be in a dynamically stable state. The polygon refers to the area enclosed by the contact points of the support feet. The ability of the support polygon to enclose the ZMP means that the resultant force and moment acting on the robot can be offset by the foot support, thus preventing tipping. On the contrary, when the ZMP exceeds the support polygon, the robot will tip over due to loss of balance. Therefore, the balance of the robot can be judged by the position information of this point: Among them, x zmp , y zmp are the coordinates of the ZMP in the x- , y- directions, x , y are the two-dimensional positions of the robot, a x , a y are the accelerations of the robot in the x- , y- directions; S2-2: The system will continuously monitor whether the critical conditions of the previous step occur. If they occur, the robot enters the high-frequency gait mode. In this mode, the robot adopts a high-frequency gait. The motion cycle of the legged robot is divided into the stance phase and the swing phase. The reinforcement learning algorithm is used to adaptively control the time of the stance phase and the swing phase, generating rapid replacement of the support feet, which is beneficial to the adjustment of the center of mass position and the restoration of the balance condition.

[0037] To quickly restore balance, the robot immediately switches from the normal gait to the high-frequency gait mode. In the high-frequency gait, the walking cycle of the robot is significantly shortened, and the steps are smaller. This adjustment helps to quickly adjust the position of the center of mass and bring it back into the support polygon. The design goal of the high-frequency gait is to restore the position of the center of mass and dynamic balance through high-speed response.

[0038] Furthermore, according to the Zero Moment Point (ZMP) theory, shortening the motion cycle can accelerate the change of the dynamic trajectory of the center of mass, enabling the ZMP to return to the support polygon faster. The position of the center of mass is judged as follows: where, P CoM is the position of the center of mass, T stance 、 T swing are the stance phase and swing phase times respectively, v 、 v 2 represent the center of mass velocity and the horizontal component of the center of mass velocity respectively, z 0 represents the height of the center of mass, ω yaw represents the tilt angle of the center of mass in the horizontal direction. In the normal gait mode, the motion cycle is longer, that is, T stance 、 T swing are larger, and the response speed is limited; while in the high-frequency gait mode, the shortened walking cycle makes the update of the support foot position more frequent, and can adjust the position of the support foot faster to follow the change of the center of mass.

[0039] Furthermore, the role of reinforcement learning in gait transition is reflected in its adaptability to complex and changing environments. In order to obtain appropriate time parameters, additional rewards are designed for the high-frequency gait mode to achieve the goal:

[0040] where, constrains the gait frequency, and acts as a reference limit frequency control to avoid the actual frequency being too fast; is the gait symmetry reward, emphasizing that the symmetry of the left and right legs helps natural walking. Thus, the trained network outputs smaller T stance 、 T swing , that is, the swing frequency of both feet is increased.

[0041] Furthermore, an additional definition is made for the cost function of the Proximal Policy Optimization (PPO) reinforcement learning algorithm to minimize the step distance during the optimization process:

[0042] where, Represents the relative distance between the centroid and the support feet.

[0043] Step S3, as the core of feedforward prediction in the system, is responsible for evolving and deducing several steps of actions in advance in the virtual environment in an accelerated simulation manner while the robot actually executes actions. This module adopts a simplified dynamics model and control strategy. By setting the simulation step size smaller than the actual control period, during the process of the robot completing the current gait cycle, the simulation can predict the future n state sequence of steps. During the deduction process, the system continuously records the reference joint positions, speeds, desired torques, and sole contact information of each step, especially paying attention to whether there are signs of slipping. Among these deduction results, the action instruction at the earliest moment is processed by the PD controller and then used as the initial control reference input for the next step to the real-time planning module, thereby providing stable feedforward guidance for the real robot. The simulation deduction also provides a warning basis for abnormal situations, and the system can send a strategy adjustment signal in advance to switch to a slow gait. This design greatly enhances the predictability and stability of the system while ensuring real-time performance, effectively improving the gait safety of the robot on slippery roads.

[0044] To detect possible slips in advance, an asynchronous parallel feedback architecture of simulation deduction - real-time planning is designed. After the robot starts walking, the model deduction and real-time planning are carried out simultaneously. In the simulation environment, the strategy will be calculated at a speed faster than the actual n steps, and the calculation result of the first step is processed by the PD controller and then sent to the real robot. Furthermore, the strategy combines sensor data and simulation input for real-time planning and outputs the final torque. Then, the real robot executes the action of this step and feeds back the state to form a closed loop. General methods based on reinforcement learning can be used to fine-tune different gaits, but for better integration of multiple gaits and smooth transitions. The preferred control process adopted in this embodiment is specifically as follows: S3-1, based on the current body state of the robot and the ground feedback data, a three-dimensional simulation model that matches the actual situation is generated in real time. This simulation model can accurately map information such as the current posture of the robot, the position of the landing point, and the terrain features. In the simulation environment, a large number of key parameters involved in the motion control process are obtained through simulation calculations, such as the centroid position, desired landing point, etc.; since the basic rigid body parameter information of the robot is known, for a biped robot with 10 degrees of freedom, the geometric relative position relationship in space is obtained using kinematic parameters, and the center of gravity position of the robot is accurately calculated; S3-2. Since the robot motion system has extremely high requirements for real-time performance, the computational cost of accurately modeling the fuselage is often unaffordable and unnecessary. Therefore, the robot is simplified to an inverted pendulum system with a concentrated mass at the center of mass, and the contact point between the supporting foot and the ground is regarded as the fulcrum of the pendulum, thus ignoring the complexity of the multi-link structure and focusing on the dynamic behavior of the center of mass. To establish the dynamic model, a linear inverted pendulum model LIPM is adopted, assuming that the height of the center of mass is fixed and satisfying the following dynamic equation: Where, x is the horizontal position of the center of mass, z 0 is the height of the center of mass, g represents the acceleration due to gravity; S3-3. To determine whether the biped robot will slip due to cumulative errors, a comprehensive error analysis framework is established. Combining information such as the motor angle offset, foot force change, and center-of-mass dynamic behavior generated during the robot's operation, it is evaluated in real time. Using the comprehensive index E t to represent the cumulative error generated by the above changes in a future period of time t , that is, the cumulative error in the future n : Where, represents the offset of the joint angle, represents the fluctuation of the foot force, represents the fluctuation of the energy consumption, is the weight parameter set according to the importance of the slip risk. In the simulation environment, calculate E t in real time. If it is greater than the slip threshold E threshold , it is determined that the robot will slip; S3-4. In the asynchronous parallel feedback architecture of simulation deduction - real-time planning, the real-time planning is implemented based on a PD controller in the form of position control. In each control cycle, according to the sensor feedback of the robot and the reference motion state provided by the deduction module, the control torque of each driving joint is calculated. Specifically, the deduction module predicts the expected motion trajectory of the robot in the future n steps in an accelerated simulation manner, and the expected joint position and velocity of each step are cached in the planner. The real-time planning extracts the reference target joint position and velocity of the current step, and takes the difference with the current actual state , , and uses the standard PD control law:

[0045] to calculate the control torque , and apply it to each driving joint of the robot. To adapt to the contact uncertainty brought by different friction conditions, the PD gains K p ,K d It can be adaptively fine-tuned through an online policy adjustment module. When foot-end sliding is detected (such as a rapid decrease in plantar contact force or abnormal IMU lateral acceleration), it automatically reduces K p to relieve excessive tracking rigidity and increase stability. In addition, real-time planning integrates the error between sensor input and simulated predicted state to form a state residual signal, dynamically corrects the reference trajectory, and prevents control drift caused by the accumulation of simulation errors, thus realizing a high-response control process combining "feed-forward prediction + real-time feedback".

[0046] S3-5. To ensure that the robot can quickly perceive environmental changes and respond, and reduce the influence of external interference and its own errors in a dynamic environment, data is adopted at a frequency of 1000 Hz to ensure the continuity and fineness of key parameters in time, and avoid control deviations caused by information lag.

[0047] Step S4. If a risk signal of slipping is detected in step S3, the current simulation deduction and real-time planning operations are immediately aborted. The swinging foot will give priority to executing the risk response strategy, and the robot will gradually shorten the stride length and change to a slow gait with small steps and slow swings. The specific control process is as follows: The reinforcement learning model uses the model adaptive control strategy to adaptively adjust the gait amplitude and frequency. Based on the original reward function, further design is made for the more stable and conservative walking behavior of this gait:

[0048] wherein, is the weighted reward described in step S1; represents the current stride length, which can be converted into the relative distance between the center of mass and the supporting foot described in step S2-2; is the set small-stride target output by the reinforcement learning strategy; represents the linear velocity of the swinging foot and punishes too fast a swing; 、 、 are the corresponding weight coefficients. By online updating the policy parameters, the adjustment of the gait amplitude and frequency is made more accurate, and at the same time, the gait parameters are dynamically optimized in combination with the development trend of the slipping risk, so that it can quickly converge to a more stable gait pattern when predicting the occurrence of slipping.

[0049] Step S5 realizes smooth and adaptive adjustment from a special gait to a normal gait. This process can be inherited from Step S1 or resumed after Step S2 or Step S4 is completed. The execution cycle of the special gait is 5 seconds. When a cycle ends, the system determines whether it is stable according to the above conditions. If so, it resumes the normal gait; otherwise, it enters the next special gait cycle. During the adaptive adjustment process, a reinforcement learning algorithm is used to train various gaits and their transition states in a simulation environment. The system learns the dynamic responses and optimal switching strategies between different gaits to ensure the continuity and stability of gait switching in any state. Through a large amount of training data covering various environmental conditions, terrain complexities, and dynamic disturbances, the robot can quickly generate gait adjustment strategies and ensure a smooth and efficient switch from a special gait (high-frequency gait or slow gait) to a normal gait.

[0050] This method designs an asynchronous parallel architecture of simulation and deduction - real-time planning to improve the robot's ability to avoid risks on slippery roads; it creates an adaptive gait adjustment control with shared parameters to improve the robot's robustness when walking on slippery roads.

[0051] Embodiment 2 Based on the same inventive concept, the present invention also designs a bipedal humanoid robot based on a robot gait control method adapted to walking on slippery roads, including: It includes a robot body and a motion control module; The shown robot body selects high - burst and high - torque servo motors to provide precise control and fast response. The feet are equipped with force sensors to real - time feedback the force information; it conforms to human aesthetics, each leg has 5 degrees of freedom, and uses a modular design for easy assembly and maintenance. It is equipped with a high - precision IMU to accurately sense the robot's posture and speed, and integrates a microcomputer to meet real - time computing tasks.

[0052] The motion control module uses a combined control method of reinforcement learning and MPC to control the robot to walk in a normal gait and real - time collect data of each joint motor and foot sensor; Conduct a force analysis on the real - time collected data and monitor the robot's walking state: When the robot may tip over, it enters the slow gait mode. The criterion for judging possible tipping over is to deduce the state of the next n steps in the simulation and deduction environment and judge whether the n step shows a tipping - over trend; When the robot tips over, it enters the high - frequency gait mode; The normal gait, high - frequency gait, and slow gait all adopt a combined control method of reinforcement learning and model predictive control MPC, and only adjust the gait frequency at the output layer; After determining to enter normal gait walking after completing the high-frequency gait pattern or the slow gait pattern, the determination includes making a determination by setting a fixed time or setting conditions.

[0053] The biped robot described in this embodiment is a robot that implements the robot walking gait control method based on adaptive reinforcement learning in Embodiment 1. For the specific process content, refer to the corresponding part of the above method embodiment, which will not be elaborated here.

[0054] Embodiment 3 Based on the same inventive concept, the present invention also provides an electronic device, including one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in Embodiment 1.

[0055] Since the device introduced in Embodiment 3 of the present invention is the electronic device used to implement the robot gait control method for walking on a slippery road surface in Embodiment 1 of the present invention, based on the method introduced in Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and deformation of this electronic device, so it will not be elaborated here. Any electronic device used in the method of Embodiment 1 of the present invention belongs to the scope of protection of the present invention.

[0056] Embodiment 4 Based on the same inventive concept, the present invention also provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method described in Embodiment 1.

[0057] Since the device introduced in Embodiment 4 of the present invention is the computer-readable medium used to implement the robot gait control method for walking on a slippery road surface in Embodiment 1 of the present invention, based on the method introduced in Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and deformation of this electronic device, so it will not be elaborated here. Any electronic device used in the method of Embodiment 1 of the present invention belongs to the scope of protection of the present invention.

[0058] The specific embodiments described herein are only illustrative of the spirit of the present invention. Those skilled in the art of the present invention can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A gait control method for a robot adapted to walk on a slippery road surface, characterized in that: Adopt a combined control method of reinforcement learning and MPC to control the robot to walk in a normal gait, and collect data of each joint motor and foot sensor in real time; Conduct a force analysis on the real-time collected data and monitor the walking state of the robot: When the robot is likely to overturn, it enters the slow gait mode. The judgment criteria for the possibility of overturning are to deduce the future in the simulation environment. n The status of the step, judge the n Whether the step shows a tendency to overturn; When the robot overturns, it enters the high-frequency gait mode; The normal gait, high-frequency gait, and slow gait all adopt a combined control method of reinforcement learning and model predictive control MPC, and only adjust the gait frequency at the output layer; After judging to complete the high-frequency gait mode or the slow gait mode, switch to walking in a normal gait. The judgment includes judging by setting a fixed time or setting conditions.

2. The gait control method for a robot adapted to walk on a slippery road surface according to claim 1, characterized in that: The optimization strategy of the reinforcement learning is: Among them, is the motion trajectory of the robot, is the execution strategy, denotes the likelihood estimate under, r t denotes the total reward, r forward represents the forward reward, r rotation represents the yaw angle reward, r smooth represents the torque transmission reward, r ground represents the contact force feedback reward, r slip represents the lateral sliding speed of the foot, r regular represents the conventional regularization penalty, denotes the discount factor of future rewards for immediate rewards, t represents the time step.

3. The robot gait control method for adapting to walking on a slippery road according to claim 2, characterized in that: The definitions of each reward item are as follows: Among them, v x is the speed in the forward direction of the robot, v y is the speed in the direction orthogonal to the forward direction of the robot, ω yaw represents the lateral angular velocity, F is the ground feedback force collected by the aforementioned sensor. The foot speed is represented by, and the binary indicator vector representing the foot contact state is used to ensure the non-negativity of the reward term.

4. The gait control method for a robot adapted to walk on a slippery road surface according to claim 1, characterized in that: During real-time monitoring, the zero moment is used to judge the position of the robot's center of mass, so as to judge whether the robot is overturned.

5. The robot gait control method for adapting to walking on a slippery road according to claim 1, characterized in that: In the prediction of the robot's future n steps, the robot is simplified into an inverted pendulum system with a concentrated mass located at the center of mass. The contact point between the supporting foot and the ground is regarded as the fulcrum of the pendulum, and a dynamic model is established. The linear inverted pendulum model LIPM is adopted, assuming that the height of the center of mass is fixed, and the following dynamic equation is satisfied: Among them, x is the horizontal position of the centroid, z 0 is the centroid height, g represents the gravitational acceleration; Using comprehensive indicators E t to represent the cumulative errors of the robot's click angle deviation, foot force change, and center-of-mass dynamic behavior within a certain period in the future, that is, in the future t steps n : where, $\Delta\theta$ represents the offset of the joint angle, $\Delta F$ represents the fluctuation of the foot force, and $\Delta E$ represents the fluctuation of the energy consumption. is a weight parameter set according to the importance of the slip risk and is calculated in real time in the simulation environment. E t , and judge E t is greater than the slip threshold E threshold , then it is determined that the robot will slip.

6. The gait control method for a robot adapted to walk on a slippery road surface according to claim 5, characterized in that: The adjustment process of the high-frequency gait is: Design additional rewards for the high-frequency gait mode to achieve the goal: Among them, constrain the gait frequency, and control it as the reference limit frequency; is the gait symmetry reward; the trained network outputs a smaller T stance 、 T swing , that is, it speeds up the swing frequency of both feet; Make an additional definition of the cost function of the reinforcement learning algorithm to make it minimize the step distance as much as possible during the optimization process: Among them, represents the relative distance between the centroid and the support feet.

7. The gait control method for a robot adapted to walk on a slippery road surface according to claim 1, characterized in that: The adjustment process of the slow gait includes: Based on the original reward function of reinforcement learning, make a further design: Among them, is the weighted reward in the original reinforcement learning; represents the current stride; is the set small stride target, output by the reinforcement learning strategy; represents the linear velocity of the swinging foot, punishing too fast swinging; and and are the corresponding weight coefficients; by online updating the policy parameters, the amplitude and frequency of the adjusted gait are made more accurate, and at the same time, the gait parameters are dynamically optimized in combination with the development trend of the slip risk, so that it quickly converges to a more stable gait pattern when predicting the occurrence of slip.

8. A biped robot based on the gait control method for a robot adapted to walk on a slippery road surface according to any one of claims 1-7, characterized in that: It includes a robot body and a motion control module; The robot body selects high-explosion and high-torque servo motors, and the feet are equipped with force sensors to feedback the force information in real time; The motion control module adopts a combined control method of reinforcement learning and MPC to control the robot to walk in a normal gait, and collect data of each joint motor and foot sensor in real time; Conduct a force analysis on the real-time collected data and monitor the walking state of the robot: When the robot may tip over, it enters the slow gait mode. The criterion for judging the possible tipping over is to deduce the state of the next n steps in the simulation and deduction environment, and judge whether the n step shows a tipping trend; When the robot overturns, it enters the high-frequency gait mode; The normal gait, high-frequency gait, and slow gait all adopt a combined control method of reinforcement learning and model predictive control MPC, and only adjust the gait frequency at the output layer; After judging to complete the high-frequency gait mode or the slow gait mode, switch to walking in a normal gait. The judgment includes judging by setting a fixed time or setting conditions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the robot gait control method for adapting to walking on a slippery road according to any one of claims 1-7.

10. An electronic device, characterized in that, Comprising: A memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the robot gait control method for adapting to walking on a slippery road according to any one of claims 1-7.

Images