Gait control method, device and biped robot for robot walking on slippery road surface

Abstract

The application discloses a robot gait control method and device suitable for walking on slippery road surfaces and a biped robot. A reinforcement learning and MPC combined control mode is adopted to control the robot to walk in a normal gait, and joint motor and foot sensor data are collected in real time; the robot state is judged: when the robot may overturn, a slow gait mode is entered, and the possible overturning judgment standard is to deduce the state of the next n steps in the simulation deduction environment, and whether the n-th step presents an overturning trend is judged; when the robot overturns, a high-frequency gait mode is entered, and a related gait control mode is designed. The problems that the biped robot is prone to slipping on a wet and slippery road surface and is difficult to remedy when slipping are solved. The ability of the robot to walk stably on a complex road surface is improved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more specifically to a gait control method, device, and bipedal robot adapted to walking on slippery surfaces. Background Technology

[0002] Bipedal robots are anthropomorphic machines designed to mimic human movement and behavior. However, they are prone to slipping on wet surfaces, potentially causing very costly damage. Despite significant advancements in robotic locomotion, walking on slippery floors remains a challenging and unresolved problem with considerable potential for further improvement. Just as humans can use sensory feedback to anticipate and adaptively avoid slips, we believe bipedal robots should rely on feedback mechanisms to achieve the same goal, improving their ability to walk steadily on slippery surfaces.

[0003] To achieve efficient trajectory tracking, the design of bipedal robots requires additional complexity. This necessitates the implementation of two main parallel controllers: one for advanced trajectory tracking and the other responsible for maintaining a specified gait, i.e., a defined phase relationship between the legs. Commonly used advanced control methods in bipedal robots rely on Model Predictive Control (MPC). MPC is a well-known control strategy that has been widely applied in both academic research and industry for decades. In particular, Linear Model Predictive Control (LMPC) assumes that the system dynamics can be effectively approximated as linear, simplifying robot model configuration, mitigating leg inertia, and solving advanced steering control problems.

[0004] In previous research, researchers hypothesized that adjusting a robot's controller based on known coefficients of friction for various floor surfaces would enable the robot to move efficiently on these surfaces. Building upon the development of foot sensors and machine learning algorithms, it is possible to detect the coefficients of friction for various floor types in real time, thereby adaptively adjusting the control model input to enhance the humanoid robot's mobility on smooth surfaces.

[0005] In real-world scenarios, slips always occur suddenly, and the robot may not have enough time to react. Furthermore, the explicit dynamics models of bipedal robots are complex, making it difficult for controllers to calculate precise and appropriate responses. Traditional LMPC also has significant limitations; due to complex inertial interactions, accurate nonlinear analysis models of real-world robots are particularly difficult to formalize when moving on slippery surfaces. On the other hand, directly deploying such control strategies in bipedal robots is extremely dangerous, especially when movement on slippery terrain is required. Therefore, using accurate dynamic simulation environments is crucial for properly evaluating robot performance and application limits. Summary of the Invention

[0006] To address the above problems, this invention proposes a method, device, and robot for gait adjustment on slippery surfaces based on adaptive reinforcement learning. Utilizing real-time feedback of road surface information from sensor data, the invention employs simulation and real-time planning to predict and mitigate potential risks. When a slip is imminent, the robot adaptively adjusts its gait to quickly maintain balance, thus researching and exploring a method for stable robot walking on complex surfaces.

[0007] The robot walking gait control method based on adaptive reinforcement learning involved in this invention includes the following process:

[0008] The robot is controlled to walk with a normal gait by using a combination of reinforcement learning and MPC, and data from the motors of each joint and the foot sensors are collected in real time.

[0009] Force analysis is performed on the real-time collected data to monitor the robot's walking status:

[0010] When the robot is likely to tip over, it enters a slow gait mode. The criterion for determining whether a tipping over is likely is based on future projections in a simulation environment. n The state of the step, determine the first step n Does the trend show signs of collapse?

[0011] When the robot tip over, it enters a high-frequency gait mode;

[0012] The normal gait, high-frequency gait, and slow gait all adopt a control method based on reinforcement learning and model predictive control (MPC), with gait frequency adjustment only performed at the output layer.

[0013] The system determines whether a high-frequency gait mode or a slow gait mode has been completed and then transitions to normal gait walking. This determination can be made by setting a fixed time or by setting specific conditions.

[0014] Furthermore, the optimization strategy for the reinforcement learning is as follows:

[0015] in, It is the robot's movement trajectory. It is an execution strategy. express exist The following likelihood estimate, r t The total reward is represented by rforward, rrotation, rsmooth, rground, rslip, and rregular.

[0016] Furthermore, the definitions of each reward item are as follows:

[0017] in, v x It is the robot's forward speed. v y It is the robot's velocity in a direction orthogonal to its forward direction. ω yaw Represents lateral angular velocity. F The ground feedback force is collected by the aforementioned sensor, and the foot velocity is represented by , which is a binary indicator vector representing the foot contact state to ensure the non-negativity of the reward term.

[0018] Preferably, in real-time monitoring, zero torque is used to determine the position of the robot's center of gravity, thereby determining whether the robot is overturning.

[0019] Preferably, in predicting the robot's next n steps, the robot is simplified as an inverted pendulum system with concentrated mass at its center of mass. The contact point between the supporting foot and the ground is considered the fulcrum of the pendulum. A dynamic model is established using the linear inverted pendulum model LIPM, assuming a fixed height of the center of mass, satisfying the following dynamic equations:

[0020] in, x The center of mass is at a horizontal position. z 0 is the height of the center of mass. g Represents gravitational acceleration;

[0021] Based on comprehensive indicators E t Indicates the future period of time t Inside, that is, the future n Step, cumulative error of robot click angle offset, foot force change, and center of mass dynamic behavior:

[0022] Among them, represents the deviation of the joint angle, represents the fluctuation of the force on the foot, and represents the fluctuation of energy consumption. The weighted parameters are set according to the importance of the slip risk. In the simulation environment, Et is calculated in real time. If Et is greater than the slip threshold Ethreshold, it is determined that the robot will slip.

[0023] The Linear Inverted Pendulum Model (LIPM) can significantly simplify robot models while losing fewer features, achieving high fidelity and reducing computational overhead, making it ideal for scenarios with high real-time requirements.

[0024] Preferably, the high-frequency gait adjustment process is as follows:

[0025] Design additional rewards for high-frequency gait patterns to achieve the goal:

[0026]

[0027] in, Constrain gait frequency, As a reference limit frequency control; It is a gait symmetry reward; the training network outputs relatively small T stance , T swing This increases the frequency of foot swings.

[0028] We define an additional cost function for the reinforcement learning algorithm to minimize the step distance during optimization:

[0029]

[0030] in, This indicates the relative distance between the center of mass and the supporting foot.

[0031] Preferably, the process of adjusting to a slow gait includes:

[0032] Based on the original reward function of reinforcement learning, further design is made:

[0033]

[0034] in, It is the weighted reward in the original reinforcement learning; Indicates the current stride length; The set small-step target is output by the reinforcement learning strategy; This indicates the speed of the swing line, and penalizes excessively fast swings; , , These are the corresponding weighting coefficients; by updating the strategy parameters online, the amplitude and frequency of gait adjustments are made more accurate. At the same time, the gait parameters are dynamically optimized in combination with the development trend of slip risk, so that they can quickly converge to a more stable gait pattern when a slip occurs.

[0035] Based on the same inventive concept, this invention also designs a bipedal humanoid robot with a gait control method adapted to walking on slippery surfaces, comprising:

[0036] Includes the robot body and motion control module;

[0037] The robot shown uses a high-burst, high-torque servo motor, and its feet are equipped with force sensors that provide real-time feedback of force information.

[0038] The motion control module uses a combination of reinforcement learning and MPC control to control the robot to walk in a normal gait and collect data from the joint motors and foot sensors in real time.

[0039] Force analysis is performed on the real-time collected data to monitor the robot's walking status:

[0040] When the robot is likely to tip over, it enters a slow gait mode. The criterion for determining whether a tipping over is likely is based on future projections in a simulation environment. n The state of the step, determine the first step n Does the trend show signs of collapse?

[0041] When the robot tip over, it enters a high-frequency gait mode;

[0042] The normal gait, high-frequency gait, and slow gait all adopt a control method based on reinforcement learning and model predictive control (MPC), with gait frequency adjustment only performed at the output layer.

[0043] The system determines whether a high-frequency gait mode or a slow gait mode has been completed and then transitions to normal gait walking. This determination can be made by setting a fixed time or by setting specific conditions.

[0044] Based on the same inventive concept, the present invention also designs a computer-readable storage medium storing computer instructions for causing the computer to execute a robot gait control method adapted to walking on slippery surfaces.

[0045] Based on the same inventive concept, the present invention also designs an electronic device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform a robot gait control method adapted to walking on slippery surfaces.

[0046] The advantages of this invention are:

[0047] 1. This invention innovatively constructs a simulation-real-time planning model. Pre-simulation in the simulation environment can provide early warnings of erroneous steps and correct gait in advance; real-time planning is the actual implementation of the simulation, and fine-tunes are made based on the actual environment and robot status to achieve optimal control. This enhances the robot's stability on slippery surfaces.

[0048] 2. This invention innovatively proposes a high-frequency gait strategy that adjusts the gait in a timely manner based on the robot's critical balance conditions to avoid slipping. This strategy can reduce the risk of falls and minimize losses due to damage to the robot body.

[0049] 3. This invention innovatively proposes an adaptive reinforcement learning gait adjustment method. This method adaptively adjusts the gait according to different environmental conditions: a normal gait is used during normal walking, a slow gait is used when a slip is predicted, and a high-frequency gait is used when a slip has already occurred, smoothly transitioning between gaits. This method enhances the robustness of robot walking and effectively prevents unpredictable losses caused by unexpected situations such as wheel lock-up. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0051] Figure 2 This is a diagram of the reinforcement learning and MPC combined control algorithm architecture of the present invention.

[0052] Figure 3 This is a simplified front view of the forces acting on the feet of a bipedal robot.

[0053] Figure 4 This is a flowchart of the simulation-real-time planning mode of the present invention.

[0054] Figure 5 This is a schematic diagram illustrating the specific implementation details of the simulation-real-time planning method of this invention.

[0055] Figure 6 This is a flowchart of the adaptive reinforcement learning training process of the present invention. Detailed Implementation

[0056] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0057] Example 1

[0058] This invention designs three gait modes: normal gait, high-frequency gait, and slow gait. Under normal circumstances, the robot will walk with a normal gait; when the robot experiences an unavoidable slip, it will switch to a high-frequency gait; when the robot anticipates a possible slip, it will switch to a slow gait. Since there are significant dynamic differences between the three gait modes, all three modes employ a reinforcement learning-based control method and share the same network structure and training parameters, with targeted adjustments only made at the output layer. This largely avoids abrupt changes in torque parameters during gait switching, achieving adaptive and smooth transitions between different gait modes.

[0059] This embodiment discloses a robot gait control method adapted to walking on slippery surfaces, which includes the following steps:

[0060] Step S1: The robot employs a control method combining reinforcement learning (RL) and model predictive control (MPC) to walk normally on flat or slippery surfaces with a predetermined gait. This hierarchical control achieves a balance between flexibility and stability. In this process, RL is responsible for generating high-level gait strategies, dynamically adjusting global parameters such as stride length and stride frequency based on the environment and robot state. MPC, as a low-level controller, optimizes the robot's trajectory in a short time, generating joint torque or position control commands to ensure dynamic balance and walking stability. During robot movement, each joint motor collects angle, torque, and other data in real time via a high-precision encoder, providing feedback on the current motion state. The specific implementation process is described below:

[0061] In step S1-1, the target velocity or position of the robot's torso is converted into a state trajectory and passed to the Reinforcement Learning (RL). The RL evaluates the optimal state and input parameters of the system using an optimization algorithm. Then, the Multiprocessor Control (MPC) uses these optimized state and input parameters to calculate the required joint torque. This torque is sent as a feedforward term to the robot's motor controller, and combined with low-gain joint spatial position and velocity (PD) control commands, it is used to reduce the impact force when the feet contact the ground and improve motion tracking accuracy. To this end, a reinforcement learning optimization strategy is designed:

[0062] in, It is the robot's movement trajectory. It is an execution strategy. express exist The following likelihood estimate, r t Indicates the total reward. r forward Represents forward rewards. r rotation Represents yaw angle bonus, r smooth Represents torque transmission award, r ground Represents feedback rewards based on contact force. r slip This represents the lateral sliding speed of the foot. r regular Represents the regularization penalty. This represents the discount factor between future rewards and immediate rewards. t Represents a time step.

[0063] A preferred approach is to define the aforementioned sub-reward items as follows:

[0064] in, v x It is the robot's forward speed.v y It is the robot's velocity in a direction orthogonal to its forward direction. ω yaw Represents lateral angular velocity. F The ground feedback force collected by the aforementioned sensors represents the foot velocity, and the binary indicator vector representing the foot contact state is used to ensure the non-negativity of the reward items. To ensure consistent scaling across all reward items, scaling factors are designed to be 20, 23, 0.005, 0.03, and 0.9, respectively.

[0065] S1-2, During normal robot walking, foot sensors continuously monitor the contact information between the robot and the ground, including data such as vertical support force, horizontal shear force, and contact time. The inertial measurement unit (IMU) senses the robot's overall posture and acceleration information in real time, providing a basis for dynamic adjustments to the control algorithm. The joint torque and rotation angle of each motor are acquired in real time. i The values ​​are 0, 1, 2, 3, and 4, which identify different motor serial numbers; a The value is 0 or 1, representing the left and right legs. The foot sensor provides feedback on the reaction force of the foot against the ground. F .

[0066] Specifically, the robot designed in this embodiment has 10 freely movable joints. To distinguish the various motors, [further details are needed]. a Identify the left and right legs, and within the same leg... i This indicates hip roll, hip yaw, hip pitch, knee pitch, and ankle pitch.

[0067] Step S2: When the robot suddenly slips, due to the complexity of the explicit dynamics model of the bipedal robot, the controller may not have enough time to react to the slip. Therefore, a high-priority adaptive method with rapid response capability is designed, and the specific implementation steps are as follows:

[0068] In S2-1, the robot's motion is mainly due to the combined effects of gravity, the driving force generated by the motors, and the supporting force of the feet. The motor driving force counteracts the supporting force of the ground on the feet, creating an interaction relationship. Since gravity is constant, the model can be simplified to consider only the effect of the foot supporting force. When a slip occurs, the foot pressure sensor reading drops sharply, and the lateral value of the linear accelerometer on the robot's IMU changes abruptly.

[0069] Furthermore, the Zero Moment Point (ZMP) describes a specific point in the robot's contact with the ground where the resultant force and torque exerted by the robot on the ground produces no overturning moment; that is, all dynamic effects can be described by pure forces without any tendency to tip over. According to the zero moment point theory, when the ZMP is located inside the supporting polygon, the robot is considered to be in a dynamically stable state. The polygon refers to the area enclosed by the contact points of the supporting feet. The fact that the supporting polygon can enclose the ZMP means that the resultant force and torque acting on the robot can be counteracted by the foot support, thus preventing tipping. Conversely, when the ZMP extends beyond the supporting polygon, the robot will tip over due to loss of balance. Therefore, the location information of this point is used to help determine the robot's balance.

[0070] in, x zmp , y zmp It's ZMP in x- , y- Coordinates in direction x , y It is the robot's two-dimensional position. a x , a y It's a robot. x- , y- Acceleration in the direction of;

[0071] In step S2-2, the system monitors in real time whether the critical condition of the previous step has occurred. If it has, the robot enters a high-frequency gait mode, in which it adopts a high-frequency gait. The movement cycle of the legged robot is divided into a standing phase and a swing phase. The reinforcement learning algorithm is used to adaptively control the time of the standing and swing phases, resulting in rapid alternation of the supporting feet, which is beneficial for adjusting the center of mass and restoring balance.

[0072] To quickly regain balance, the robot immediately switches from a normal gait to a high-frequency gait mode. In high-frequency gait, the robot's walking cycle is significantly shortened, and its strides are smaller. This adjustment helps to quickly reposition the center of mass, bringing it back within the supporting polygon. The design goal of high-frequency gait is to achieve the recovery of the center of mass position and dynamic balance through a high-speed response.

[0073] Furthermore, according to the Zero Moment Point (ZMP) theory, shortening the motion period can accelerate the change in the dynamic trajectory of the centroid, allowing the ZMP to return to the supporting polygon more quickly. The centroid position is determined as follows:

[0074] in, P CoM It is the location of the center of mass. T stance , T swingThese are the standing phase and the swinging phase times, respectively. v , v 2 represents the center-of-mass velocity and the horizontal component of the center-of-mass velocity, respectively. z 0 represents the height of the center of mass. ω yaw This represents the angle of inclination of the center of mass along the horizontal direction. In normal gait patterns, the movement cycle is relatively long, i.e. T stance , T swing The larger size results in a limited response speed; however, in the high-frequency gait mode, the shortened walking cycle allows for more frequent updates to the position of the supporting foot, enabling faster adjustments to the supporting foot's position to follow changes in the center of gravity.

[0075] Furthermore, the role of reinforcement learning in gait transition is reflected in its ability to adapt to complex and changing environments. To obtain suitable time parameters, additional rewards are designed for high-frequency gait patterns to achieve the goal:

[0076]

[0077] in, Constrain gait frequency, and As a reference limit for frequency control, to avoid the actual frequency from being too fast; It is a gait symmetry reward, emphasizing that symmetry between the left and right legs helps with natural walking. Therefore, the training network outputs relatively small. T stance , T swing This increases the frequency of foot swings.

[0078] Furthermore, the cost function of the reinforcement learning algorithm PPO is additionally defined to minimize the step distance during the optimization process:

[0079]

[0080] in, This indicates the relative distance between the center of mass and the supporting foot.

[0081] Step S3, simulation and deduction, serves as the core of the feedforward prediction system. It is responsible for performing several steps of motion evolution prediction in a virtual environment while the robot is actually executing its actions, using accelerated simulation. This module employs a simplified dynamics model and control strategy. By setting the simulation step size to be smaller than the actual control cycle, the simulation can predict future gait cycles as the robot completes its current gait. nThe system continuously records the state sequence of each step. During the simulation, the system continuously records the reference joint position, velocity, expected torque, and foot contact information for each step, paying particular attention to any signs of slippage. The action command from the previous moment is processed by the PD controller and used as the initial control reference input for the real-time planning module, thus providing stable feedforward guidance for the real robot. The simulation also provides early warning for abnormal situations, allowing the system to issue a strategy adjustment signal in advance to switch to a slower gait. This design significantly enhances the predictability and stability of the system while ensuring real-time performance, effectively improving the robot's gait safety on slippery surfaces.

[0082] To detect potential slips in advance, an asynchronous parallel feedback architecture of simulation-real-time planning is designed. Once the robot begins walking, model inference and real-time planning occur simultaneously. In the simulation environment, the strategy is implemented faster than in actual operation. n The step speed is calculated, and the result of the first step is processed by the PD controller and sent to the real robot. Then, the strategy combines sensor data and analog input for real-time planning, outputting the final torque. Afterwards, the real robot executes the step and provides feedback on its state, forming a closed loop. While general reinforcement learning-based methods can be used to fine-tune different gaits, this embodiment employs a preferred control process for better integration of multiple gaits and smooth transitions:

[0083] S3-1 generates a 3D simulation model that matches the actual situation in real time, based on the robot's current body state and ground feedback data. This simulation model can accurately map the robot's current posture, landing point position, and terrain features. In the simulation environment, a large number of key parameters involved in motion control are obtained through simulation calculations, such as the center of gravity position and the desired landing point. Since the robot's basic rigid body parameters are known, for a bipedal robot with 10 degrees of freedom, the geometric relative positional relationship in space is obtained using kinematic parameters, and the robot's center of gravity position is accurately calculated.

[0084] S3-2. Due to the extremely high real-time requirements of robot motion systems, the computational burden of accurately modeling the robot body is often unacceptable and unnecessary. Therefore, the robot is simplified as an inverted pendulum system with concentrated mass at its center of mass. The contact points between the supporting legs and the ground are considered the fulcrum of the pendulum, thus ignoring the complexity of the multi-link structure and focusing on the dynamic behavior of the center of mass. To establish the dynamic model, a linear inverted pendulum model LIPM is adopted, assuming a fixed height of the center of mass, satisfying the following dynamic equations:

[0085] in, x The center of mass is at a horizontal position. z 0 is the height of the center of mass. g Represents gravitational acceleration;

[0086] S3-3, to determine whether a bipedal robot will slip due to cumulative errors, a comprehensive error analysis framework is established. This framework combines information such as motor angle offsets, changes in foot force, and dynamic behavior of the center of gravity generated during robot operation to perform real-time evaluation. A comprehensive index is used to evaluate this performance. E t Indicates the future period of time t The cumulative error resulting from the aforementioned changes within the future, i.e. n Cumulative error:

[0087] Among them, represents the deviation of the joint angle, represents the fluctuation of the force on the foot, and represents the fluctuation of energy consumption. Weighting parameters are set according to the importance of slip risk. Calculated in real time within the simulation environment. E t If it is greater than the slip threshold E threshold If so, it is determined that the robot will slip;

[0088] In S3-4, within the asynchronous parallel feedback architecture of simulation-real-time planning, real-time planning is implemented using a PD controller based on position control. In each control cycle, the control torque of each driven joint is calculated based on sensor feedback from the robot and the reference motion state provided by the simulation module. Specifically, the simulation module predicts the robot's future... n The expected motion trajectory for each step is cached in the planner, where the expected joint position and velocity for each step are stored. Real-time planning extracts the reference target joint position for the current step. and speed and the current actual state , Perform differential analysis using the standard PD control law:

[0089]

[0090] Calculate the control torque This is then applied to the robot's various drive joints. To adapt to the contact uncertainties caused by different friction conditions, the PD gain... K p , K d The system can be adaptively fine-tuned via an online strategy adjustment module, automatically reducing speed when foot slippage is detected (e.g., rapid decrease in plantar contact force or abnormal IMU lateral acceleration). K pThis alleviates excessive tracking rigidity and increases stability. In addition, real-time planning integrates the error between sensor input and simulated predicted state to form a state residual signal, which dynamically corrects the reference trajectory and prevents control drift caused by the accumulation of simulation errors. This achieves a high-response control process that combines "feedforward prediction + real-time feedback".

[0091] S3-5. In order to ensure that the robot can quickly perceive environmental changes and respond to them, and reduce the impact of external interference and its own errors in the dynamic environment, data is used at a frequency of 1000Hz to ensure that key parameters are continuous and precise in time, and to avoid control deviations caused by information lag.

[0092] Step S4: If a slip risk signal is detected in step S3, the current simulation and real-time planning operation is immediately stopped. The swinging foot will prioritize the risk response strategy, and the robot will gradually shorten its stride, switching to a slow gait with small strides and slow swinging motion. The specific control process is as follows:

[0093] The reinforcement learning model utilizes an adaptive control strategy to adaptively adjust gait amplitude and frequency. Based on the original reward function, further designs are made to address the more stable and conservative walking behavior of this gait.

[0094]

[0095] in, It is the weighted reward described in step S1; The current stride length can be converted into the relative distance between the center of mass and the supporting foot as described in step S2-2. These are small-step objectives set and output by the reinforcement learning strategy; This indicates the speed of the swing line, and penalizes excessively fast swings; , , These are the corresponding weighting coefficients. By updating the strategy parameters online, the amplitude and frequency of gait adjustments are made more accurate. At the same time, the gait parameters are dynamically optimized in conjunction with the development trend of slip risk, so that they can quickly converge to a more stable gait pattern when a slip occurs.

[0096] Step S5 achieves smooth, adaptive adjustment from special gait to normal gait. This process can be inherited from step S1 or resumed after step S2 or S4. The execution cycle of the special gait is 5 seconds. When a cycle ends, the system determines whether it is stable based on the above conditions. If so, it resumes normal gait; otherwise, it enters the next special gait cycle. During the adaptive adjustment process, reinforcement learning algorithms are used to train various gaits and their transition states in a simulation environment. The system learns the dynamic response and optimal switching strategy between different gaits, ensuring the continuity and stability of gait switching under any condition. Through a large amount of training data covering various environmental conditions, terrain complexity, and dynamic disturbances, the robot can quickly generate gait adjustment strategies and ensure a smooth and efficient transition from special gait (high-frequency gait or slow gait) to normal gait.

[0097] This method designs an asynchronous parallel architecture of simulation and real-time planning to improve the robot's ability to avoid risks on slippery surfaces; and creates adaptive gait adjustment control with shared parameters to improve the robot's robustness when walking on slippery surfaces.

[0098] Example 2

[0099] Based on the same inventive concept, this invention also designs a bipedal humanoid robot based on a robot gait control method adapted to walking on slippery surfaces, comprising:

[0100] Includes the robot body and motion control module;

[0101] The robot shown uses high-power, high-torque servo motors for precise control and rapid response. Force sensors in its feet provide real-time force feedback. Adopting an ergonomic design, each leg has five degrees of freedom and features a modular design for easy assembly and maintenance. A high-precision IMU accurately senses the robot's posture and speed, while a microcomputer handles real-time computation.

[0102] The motion control module uses a combination of reinforcement learning and MPC control to control the robot to walk in a normal gait and collect data from the joint motors and foot sensors in real time.

[0103] Force analysis is performed on the real-time collected data to monitor the robot's walking status:

[0104] When the robot is likely to tip over, it enters a slow gait mode. The criterion for determining whether a tipping over is likely is based on future projections in a simulation environment. n The state of the step, determine the first step n Does the trend show signs of collapse?

[0105] When the robot tip over, it enters a high-frequency gait mode;

[0106] The normal gait, high-frequency gait, and slow gait all adopt a control method based on reinforcement learning and model predictive control (MPC), with gait frequency adjustment only performed at the output layer.

[0107] The system determines whether a high-frequency gait mode or a slow gait mode has been completed and then transitions to normal gait walking. This determination can be made by setting a fixed time or by setting specific conditions.

[0108] The bipedal robot described in this embodiment is the robot that implements the robot walking gait control method based on adaptive reinforcement learning in Embodiment 1. For details of the process, please refer to the corresponding part of the above method embodiment, which will not be repeated here.

[0109] Example 3

[0110] Based on the same inventive concept, the present invention also provides an electronic device, including one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in Embodiment 1.

[0111] Since the device described in Embodiment 3 of this invention is an electronic device used in implementing the robot gait control method for walking on slippery surfaces as described in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this electronic device based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All electronic devices used in any method of this invention fall within the scope of protection of this invention.

[0112] Example 4

[0113] Based on the same inventive concept, the present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in Embodiment 1.

[0114] Since the device described in Embodiment 4 of this invention is a computer-readable medium used to implement the robot gait control method for walking on slippery surfaces in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this electronic device based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All electronic devices used in any method of this invention fall within the scope of protection of this invention.

[0115] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A gait control method for a robot adapted to walking on slippery surfaces, characterized in that: The robot is controlled to walk with a normal gait by using a combination of reinforcement learning and MPC, and data from the motors of each joint and the foot sensors are collected in real time. Force analysis is performed on the real-time collected data to monitor the robot's walking status: When the robot is likely to tip over, it enters a slow gait mode. The criterion for determining whether a tipping over is likely is based on future projections in a simulation environment. n The state of the step, determine the first step n Does the trend show signs of collapse? The adjustment process for the slow gait pattern includes: Based on the original reward function of reinforcement learning, further design is made: in, It is the weighted reward in the original reinforcement learning; Indicates the current stride length; The set small-step target is output by the reinforcement learning strategy; This indicates the speed of the swing line, and penalizes excessively fast swings; , , These are the corresponding weighting coefficients; by updating the strategy parameters online, the amplitude and frequency of gait adjustment are made more accurate, and the gait parameters are dynamically optimized in combination with the development trend of slip risk, so that it can quickly converge to a more stable gait pattern when a slip occurs. When the robot tip over, it enters a high-frequency gait mode; The normal gait, high-frequency gait, and slow gait all adopt a control method based on reinforcement learning and model predictive control (MPC), with gait frequency adjustment only performed at the output layer. The system determines whether a high-frequency gait mode or a slow gait mode has been completed and then transitions to normal gait walking. This determination can be made by setting a fixed time or by setting specific conditions.

2. The robot gait control method for adapting to slippery surfaces according to claim 1, characterized in that: The optimization strategy for reinforcement learning is as follows: in, It is the robot's movement trajectory. It is an execution strategy. express exist The following likelihood estimate, r t Indicates the total reward. r forward Represents forward rewards. r rotation Represents yaw angle bonus, r smooth Represents torque transmission award, r ground Represents feedback rewards based on contact force. r slip This represents the lateral sliding speed of the foot. r regular Represents the regularization penalty. This represents the discount factor between future rewards and immediate rewards. t Represents a time step.

3. The robot gait control method for adapting to slippery surfaces according to claim 2, characterized in that: The definitions of each reward item are as follows: in, v x It is the robot's forward speed. v y It is the robot's velocity in a direction orthogonal to its forward direction. ω yaw Represents lateral angular velocity. F The ground feedback force collected by the aforementioned sensor, and the foot speed are represented by... express, A binary indicator vector representing the foot contact state, in Ensure that the reward items are non-negative.

4. The robot gait control method for adapting to slippery surfaces according to claim 1, characterized in that: In real-time monitoring, zero torque is used to determine the position of the robot's center of gravity, thereby determining whether the robot is about to tip over.

5. The robot gait control method for adapting to slippery surfaces according to claim 1, characterized in that: In predicting the robot's next n steps, the robot is simplified as an inverted pendulum system with concentrated mass at its center of mass. The contact point between the supporting foot and the ground is considered the fulcrum of the pendulum. A dynamic model is established using the linear inverted pendulum model LIPM, assuming a fixed height of the center of mass, satisfying the following dynamic equations: in, The center of mass is at a horizontal position. It is the height of the center of mass. Represents gravitational acceleration; Based on comprehensive indicators E t Indicates a period of time in the future t Inside, that is, the future n Step, cumulative error of robot click angle offset, foot force change, and center of mass dynamic behavior: in, Indicates the offset of the joint angle. This indicates the fluctuation of force on the feet. This indicates fluctuations in energy consumption. In a simulation environment, weighting parameters set according to the importance of slip risk are calculated in real time. E t ,judge E t Greater than the slip threshold E threshold If so, it is determined that the robot will slip.

6. The robot gait control method for adapting to slippery surfaces according to claim 5, characterized in that: The adjustment process for high-frequency gait is as follows: Design additional rewards for high-frequency gait patterns to achieve the goal: in, Constrain gait frequency, As a reference limit frequency control; It is a gait symmetry reward; the training network outputs relatively small T stance , T swing This increases the frequency of leg swings; We define an additional cost function for the reinforcement learning algorithm to minimize the step distance during optimization: in, This indicates the relative distance between the center of mass and the supporting foot.

7. A bipedal robot based on the robot gait control method for walking on slippery surfaces according to any one of claims 1-6, characterized in that: Includes the robot body and motion control module; The robot shown uses a high-burst, high-torque servo motor, and its feet are equipped with force sensors that provide real-time feedback of force information. The motion control module uses a combination of reinforcement learning and MPC control to control the robot to walk in a normal gait and collect data from the joint motors and foot sensors in real time. Force analysis is performed on the real-time collected data to monitor the robot's walking status: When the robot is likely to tip over, it enters a slow gait mode. The criterion for determining whether a tipping over is likely is based on future projections in a simulation environment. n The state of the step, determine the first step n Does the trend show signs of collapse? When the robot tip over, it enters a high-frequency gait mode; The normal gait, high-frequency gait, and slow gait all adopt a control method based on reinforcement learning and model predictive control (MPC), with gait frequency adjustment only performed at the output layer. The system determines whether a high-frequency gait mode or a slow gait mode has been completed and then transitions to normal gait walking. This determination can be made by setting a fixed time or by setting specific conditions.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the robot gait control method for walking on slippery surfaces as described in any one of claims 1-6.

9. An electronic device, characterized in that, include: The system includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the robot gait control method for walking on slippery surfaces as described in any one of claims 1-6.

Images