Adaptive control method for quadruped robot based on reinforcement learning and nmpc-wbc

Abstract

The application discloses a kind of four-wheel foot type robot adaptive control method based on reinforcement learning and NMPC-WBC, through gradient descent method and joint soft constraint combination, realize the autonomous start under the initial posture of robot and leg posture alignment, without foot end force sensor and external perception equipment, only through ontology multi-source state data, combined with reinforcement learning network realizes implicit contact sensing and terrain parameter estimation, reduce hardware cost, NMPC optimization module with dynamic soft constraint and time-event hybrid triggering mechanism cooperate, real-time adjustment gait phase and foot end trajectory, adapt to complex terrain change, improve motion stability, wheel-step switching WBC control model realizes the adaptive switching of pure wheel mode and leg-lifting wheel-step mode, WBC hierarchical force distribution strategy based on task priority, priority guarantee body posture and center of mass stability, improve control precision and system robustness, can be applied to various four-wheel foot type robot inspection, detection scene.

Description

Technical Field

[0001] This invention relates to the field of robot motion control technology, and in particular to an adaptive control method for a four-wheeled legged robot based on reinforcement learning and NMPC-WBC. Background Technology

[0002] Quadrupedal robots combine the high mobility of wheeled robots with the adaptability to complex terrain of legged robots, and have broad application prospects in fields such as inspection, field exploration, and emergency rescue. However, existing quadrupedal robot control methods still have many technical bottlenecks that limit their practical application effects: First, their autonomous start-up capability is insufficient, making it impossible to achieve stable start-up in any initial posture, and joint jamming or posture imbalance is prone to occur; Second, contact perception relies on external devices such as foot force sensors, which not only increases hardware costs but also reduces reliability in harsh environments; Third, traditional control architectures have not achieved deep collaboration between reinforcement learning, NMPC, and WBC, resulting in insufficient accuracy in trajectory optimization and force distribution in complex terrain; Fourth, existing WBC control models mostly follow the design concept of pure quadrupedal robots, only considering legged motion and not taking into account pure wheeled motion and wheel-walking mode switching, making it impossible to achieve adaptive mode adjustment according to terrain changes and body posture requirements; Fifth, the body posture stability control is insufficient, and it is difficult to maintain the preset posture (such as the fixed shooting posture of the gimbal) when the terrain is rugged or there is external interference, affecting the accuracy of operation.

[0003] To address the aforementioned issues, some existing patents have proposed NMPC-WBC collaborative control methods for quadruped robots, but these do not involve wheel-stepping mode switching and cannot adapt to the structural characteristics of quadruped robots. Some patents have implemented wheel-stepping switching, but they do not combine it with body posture changes for adaptive judgment and do not deeply integrate it with reinforcement learning and contact perception, resulting in insufficient switching accuracy and stability. At the same time, most control methods rely on external sensors to achieve contact perception and terrain recognition, which results in high hardware costs and weak anti-interference capabilities. Summary of the Invention

[0004] Purpose of the invention: To address the above-mentioned problems, the purpose of this invention is to provide an adaptive control method for a four-wheeled robot that does not require external sensors, can achieve adaptive switching of wheel steps, and ensures stable body posture.

[0005] Technical solution: An adaptive control method for a four-wheeled legged robot based on reinforcement learning and NMPC-WBC, comprising the following steps:

[0006] S1: Adaptive Start-up Trajectory Planning and Leg Posture Alignment: Initialize system parameters, collect robot initial joint angles and body posture, define target start-up posture, construct joint angle error function, use gradient descent method to iteratively update joint angles, introduce joint soft constraints and emergency adjustment mechanism to achieve adaptive start-up and leg posture alignment under any initial posture.

[0007] S2: Proprioceptive State Acquisition and Preprocessing: Multi-source state data is acquired through robot body sensors to construct a time-series state sequence. z-score normalization preprocessing is used to eliminate dimensional differences and output a standardized time-series state sequence.

[0008] S3: Reinforcement Learning Implicit Contact Sensing and Terrain Parameter Estimation: Input the standardized temporal state sequence into the reinforcement learning network, and output the foot contact confidence, contact temporal correction and terrain stiffness estimate to achieve implicit contact sensing and terrain parameter recognition without external sensors.

[0009] S4: NMPC Optimization and Gait Scheduling: Based on contact perception results and terrain parameters, an NMPC optimization problem with dynamic soft constraints is constructed and solved using a sequential quadratic programming algorithm. Combined with a time-event hybrid triggering mechanism, adaptive gait phase scheduling is achieved.

[0010] S5: Wheel-step switching WBC control and torque optimization: Construct a wheel-step switching WBC control model to achieve adaptive switching between pure wheel mode and leg-lifting wheel-step mode. Perform hierarchical force distribution based on task priority management strategy, introduce compliant control mechanism to compensate for disturbances, and output optimal joint torque command.

[0011] The wheel-step switching WBC control model includes mode switching determination logic, wheel-step hybrid dynamics model, and layered force distribution strategy;

[0012] The expression for the wheel-step switching WBC control model is: ,in, This is the coefficient for switching between wheel-step modes. Indicates pure wheel mode. This indicates the leg-lifting wheel step mode. For the wheel-step switching coupling inertia matrix, To enable the switching of the coupling between the Coriolis force and the centrifugal force matrix, The gravity vector Let the joint position vector be... , These are the joint velocity and joint acceleration vectors, respectively. Select a matrix for joint torques. This is the joint torque vector. The Jacobian matrix is ​​adapted for wheel-walking modes. Pure wheel mode corresponds to the wheel center Jacobian, and leg-lifting wheel-walking mode corresponds to the foot contact Jacobian. The contact force vector at the foot;

[0013] S6: Closed-loop feedback execution: The optimal joint torque command is sent to the actuator through the communication bus, and the body status data is collected in real time and fed back to the preprocessing module, forming a complete closed loop of acquisition-sensing-optimization-control-feedback.

[0014] Furthermore, in step S1:

[0015] The joint angle error function is: ,in, This is the current joint angle vector. Initiate the attitude joint angle vector for the target;

[0016] The formula for iterative updating using the gradient descent method is as follows: ,in, The iteration step size, The gradient of the error function;

[0017] The joint soft constraint is ,in, , These represent the minimum and maximum constraint values ​​for the joint angle, respectively.

[0018] Furthermore, in step S2, the z-score normalization preprocessing formula is: ,in, The normalized state data, This is the original state data. The statistical mean of the state data in the training set. The statistical variance of the training set state data; the length of the time-series state sequence is set to... , recorded as It is used to preserve the historical temporal characteristics of multi-source ontology state data.

[0019] Furthermore, in step S3:

[0020] The reinforcement learning network adopts an LSTM-GRU hybrid network structure, containing two LSTM hidden layers and one GRU hidden layer. The input is the length. The standardized time-series state sequence is output as the foot contact confidence level. Contact timing correction and terrain stiffness estimate ;

[0021] The network loss function is ,in, For foot contact confidence loss, For contact timing correction amount loss, Loss due to terrain stiffness estimation. , , This is the loss weighting coefficient.

[0022] Furthermore, in step S4:

[0023] The dynamic soft constraint is linked to the foot contact confidence level, and the dynamic soft constraint function is: ,in, , These are the minimum and maximum penalty coefficients for soft constraints, respectively. For foot contact confidence;

[0024] The weighted transformation formula for friction cone constraints is as follows: ,in, The contact force vector at the foot end. The ground normal vector, The ground friction coefficient; through soft constraint terms A friction cone constraint is introduced to achieve dynamic adjustment of the constraint;

[0025] The objective function of the NMPC optimization problem is: ,in, for The system state vector at any given time. for System status reference value at any time for Control the input vector at all times. , , These are the system state tracking weight matrix, the control input penalty weight matrix, and the terminal state tracking weight matrix, respectively. For NMPC prediction in the time domain;

[0026] A sequential quadratic programming algorithm is used to solve the NMPC optimization problem, and the solution cycle is consistent with the robot control cycle.

[0027] The time-event hybrid triggering mechanism is as follows: To preset the confidence threshold for foot contact,

[0028] when Furthermore, when the foot touches the ground prematurely, it triggers an early gait phase switch, shortens the duration of the current gait phase, and avoids a rigid collision between the foot and the ground.

[0029] when Furthermore, if the foot fails to touch the ground on time, a gait phase delay switch is triggered, extending the duration of the current gait phase, while also adjusting the contact timing based on the contact timing. Adjust foot height Ensure that the feet touch the ground stably;

[0030] The time-triggered mechanism serves as the basic scheduling logic, pre-setting the baseline duration of each gait phase. The event-triggered mechanism serves as the dynamic correction logic, adjusting the gait phase in real time based on foot contact confidence and foot ground contact status. The two work together to achieve adaptive scheduling of gait phases.

[0031] Furthermore, the mode switching determination logic is based on the fuselage attitude change amount, and the determination formula is as follows: ,in, The current fuselage attitude angle vector, roll angle Pitch angle Yaw angle , Let be the desired fuselage attitude angle vector; when hour, As the attitude change threshold, if the terrain change is too large or there is significant external interference, switch to the leg-lifting wheel-walking mode to compensate for the fuselage attitude deviation through leg posture adjustment. At that time, maintain pure wheel mode.

[0032] Furthermore, the expression for the wheel-walking hybrid power dynamics model is as follows: ,in, Here is the system inertia matrix. The matrix of Coriolis force and centrifugal force. The gravity vector Let the joint position vector be... , These are the joint velocity and joint acceleration vectors, respectively. Select a matrix for joint torques. This is the joint torque vector. For foot contact Jacobian matrix, This is the contact force vector at the foot.

[0033] Furthermore, the hierarchical force allocation strategy divides the control tasks into three levels according to priority, from high to low:

[0034] Highest priority: Centroid position and pose tracking task, objective function is ,in, Let the linear velocity of the center of mass be denoted as . This is the reference value for the linear velocity of the center of mass. For fuselage attitude angle, For the desired attitude angle of the fuselage, , This is the weight matrix for the corresponding task;

[0035] Second highest priority: Foot position tracking task, objective function is... ,in, The actual position of the foot. For the reference position of the foot, This is the task weight matrix. To reinforce the terrain stiffness estimate output by the learning process, It is the identity matrix;

[0036] Lowest priority: Joint torque minimization task, objective function is ,in, This is the torque penalty weight matrix.

[0037] Furthermore, in step S5, the compliance control mechanism is used to compensate for external disturbances caused by contact estimation errors, and its expression is... ,in, The nominal joint torque is obtained by solving the quadratic programming algorithm. This is the disturbance compensation gain matrix. This is the disturbance estimate corresponding to the motor current deviation. , This represents the actual output torque of the motor. This is the command torque for the motor.

[0038] Beneficial effects: The adaptive control method for four-wheeled legged robots of the present invention enables autonomous start-up and leg posture alignment of the robot under any initial posture, without the need for foot force sensors and external sensing devices. It adjusts the gait phase and foot trajectory in real time, adapts to complex terrain changes, improves motion stability, and realizes adaptive switching between pure wheel mode and leg-lifting wheel step mode, thereby improving control accuracy and system robustness. It can be applied to various four-wheeled legged robot inspection and detection scenarios. Detailed Implementation

[0039] The present invention will be further illustrated below with reference to specific embodiments.

[0040] An adaptive control method for a four-wheeled legged robot based on reinforcement learning and NMPC-WBC includes the following steps S1 to S6.

[0041] S1: Adaptive Start-up Trajectory Planning and Leg Posture Alignment: Initialize system parameters, collect robot initial joint angles and body posture, define target start posture, construct joint angle error function, use gradient descent method to iteratively update joint angles, introduce joint soft constraints and emergency adjustment mechanism to achieve adaptive start-up and leg posture alignment under any initial posture.

[0042] The joint angle error function is ,in, This is the current joint angle vector. Initiate the attitude joint angle vector for the target.

[0043] The formula for iterative updates using gradient descent is: ,in, The iteration step size, Let be the gradient of the error function.

[0044] joint soft constraint is ,in, , These represent the minimum and maximum constraint values ​​for the joint angle, respectively.

[0045] S2: Proprioceptive State Acquisition and Preprocessing: Multi-source state data is acquired through the robot's body sensors to construct a time-series state sequence. z-score normalization preprocessing is used to eliminate dimensional differences and output a standardized time-series state sequence.

[0046] The z-score normalization preprocessing formula is: ,in, The normalized state data, This is the original state data. The statistical mean of the state data in the training set. The statistical variance of the training set state data; the length of the time-series state sequence is set to... , recorded as It is used to preserve the historical temporal characteristics of multi-source ontology state data.

[0047] S3: Reinforcement Learning Implicit Contact Sensing and Terrain Parameter Estimation: The standardized temporal state sequence is input into the reinforcement learning network, which outputs foot contact confidence, contact temporal correction, and terrain stiffness estimate, thereby achieving implicit contact sensing and terrain parameter recognition without external sensors.

[0048] The reinforcement learning network employs a hybrid LSTM-GRU network structure, consisting of two LSTM hidden layers (128 dimensions) and one GRU hidden layer (64 dimensions). The input is the length. The standardized time-series state sequence is output as the foot contact confidence level. Contact timing correction and terrain stiffness estimate .

[0049] The network loss function is ,in, For foot contact confidence loss, For contact timing correction amount loss, Loss due to terrain stiffness estimation. , , This is the loss weighting coefficient.

[0050] S4: NMPC Optimization and Gait Scheduling: Based on contact perception results and terrain parameters, an NMPC optimization problem with dynamic soft constraints is constructed and solved using a sequential quadratic programming algorithm. Combined with a time-event hybrid triggering mechanism, adaptive gait phase scheduling is achieved.

[0051] The dynamic soft constraint is linked to the foot contact confidence level, and the dynamic soft constraint function is: ,in, , These are the minimum and maximum penalty coefficients for soft constraints, respectively. The confidence level is the foot contact confidence level.

[0052] The weighted transformation formula for friction cone constraints is as follows: ,in, The contact force vector at the foot end. The ground normal vector, is the coefficient of friction of the ground.

[0053] Through soft constraint terms Introducing friction cone constraints enables dynamic adjustment of the constraints: when the foot contacts the confidence level... Higher ( )hour, The dynamic soft constraint function degenerates into a friction cone constraint, imposing strict rigid constraints on the contact force to ensure the stability of the support phase; when the foot contact confidence level Lower ( )hour, The constraint on the right side is reduced as a whole, and the friction cone constraint is dynamically relaxed, allowing the contact force to exceed the range of the friction cone to a certain extent, thus avoiding the optimization problem becoming unsolvable due to contact estimation errors.

[0054] The objective function of the NMPC optimization problem is: ,in, for The system state vector at any given time. for System status reference value at any time for Control the input vector at all times. , , These are the system state tracking weight matrix, the control input penalty weight matrix, and the terminal state tracking weight matrix, respectively. For NMPC prediction in the time domain.

[0055] The Sequential Quadratic Programming (SQP) algorithm is used to solve the NMPC optimization problem, and the solution cycle is kept consistent with the robot control cycle to ensure real-time control.

[0056] The time-event hybrid triggering mechanism is as follows: To preset the confidence threshold for foot contact,

[0057] when Furthermore, when the foot touches the ground prematurely, it triggers an early gait phase switch, shortens the duration of the current gait phase, and avoids a rigid collision between the foot and the ground.

[0058] when Furthermore, if the foot fails to touch the ground on time, a gait phase delay switch is triggered, extending the duration of the current gait phase, while also adjusting the contact timing based on the contact timing. Adjust foot height Ensure that the feet touch the ground stably;

[0059] The time-triggered mechanism serves as the basic scheduling logic, pre-setting the baseline duration of each gait phase. The event-triggered mechanism serves as the dynamic correction logic, adjusting the gait phase in real time based on foot contact confidence and foot ground contact status. The two work together to achieve adaptive scheduling of gait phases, improving the robot's motion stability in complex terrain.

[0060] S5: Wheel-step switching WBC control and torque optimization: Construct a wheel-step switching WBC control model to achieve adaptive switching between pure wheel mode and leg-lifting wheel-step mode. Perform hierarchical force distribution based on task priority management strategy, introduce an adaptive control mechanism to compensate for disturbances, and output the optimal joint torque command.

[0061] The wheel-step switching WBC control model includes mode switching determination logic, wheel-step hybrid dynamics model, and hierarchical force distribution strategy.

[0062] The expression for the wheel-step switching WBC control model is: ,in, This is the coefficient for switching between wheel-step modes. Indicates pure wheel mode. This indicates the leg-lifting wheel step mode. For the wheel-step switching coupling inertia matrix, To enable the switching of the coupling between the Coriolis force and the centrifugal force matrix, The gravity vector Let the joint position vector be... , These are the joint velocity and joint acceleration vectors, respectively. Select a matrix for joint torques. This is the joint torque vector. The Jacobian matrix is ​​adapted for wheel-walking modes. Pure wheel mode corresponds to the wheel center Jacobian, and leg-lifting wheel-walking mode corresponds to the foot contact Jacobian. This is the contact force vector at the foot.

[0063] The mode switching determination logic is based on the fuselage attitude change, and the determination formula is as follows: ,in, The current fuselage attitude angle vector, roll angle Pitch angle Yaw angle , The desired camera body attitude angle vector can correspond to preset attitudes such as fixed shooting posture of the gimbal; when hour, As the attitude change threshold, if the terrain change is too large or there is significant external interference, switch to the leg-lifting wheel-walking mode to compensate for the fuselage attitude deviation through leg posture adjustment. At the same time, maintain pure wheel mode to ensure exercise efficiency.

[0064] The expression for the wheel-walking hybrid power model is: ,in, Here is the system inertia matrix. The matrix of Coriolis force and centrifugal force. The gravity vector Let the joint position vector be... , These are the joint velocity and joint acceleration vectors, respectively. Select a matrix for joint torques. This is the joint torque vector. For foot contact Jacobian matrix, This represents the contact force vector at the foot. The wheel-walking hybrid dynamics model is calculated using the Robot Dynamics Library (RBDL) to ensure both model accuracy and computational efficiency.

[0065] The hierarchical force allocation strategy divides control tasks into three levels according to priority, from high to low:

[0066] Highest priority: Centroid position and pose tracking task, objective function is ,in, Let the linear velocity of the center of mass be denoted as . This is the reference value for the linear velocity of the center of mass. For fuselage attitude angle, This represents the desired attitude angle of the fuselage, which corresponds to the fixed attitude of the gimbal. , This is the weight matrix for the corresponding task;

[0067] Second highest priority: Foot position tracking task, objective function is... ,in, The actual position of the foot. For the reference position of the foot, This is the task weight matrix. To reinforce the terrain stiffness estimate output by the learning process, It is an identity matrix; the foot is also the center of the wheel.

[0068] Lowest priority: Joint torque minimization task, objective function is ,in, This is the torque penalty weight matrix.

[0069] The null-space projection method is used to achieve hierarchical optimization of tasks with different priorities, ensuring that high-priority tasks are satisfied first.

[0070] The compliance control mechanism is used to compensate for external disturbances caused by contact estimation errors, and its expression is as follows: ,in, The nominal joint torque is obtained by solving the quadratic programming algorithm. This is the disturbance compensation gain matrix. This is the disturbance estimate corresponding to the motor current deviation. , This represents the actual output torque of the motor. This is the command torque for the motor.

[0071] S6: Closed-loop feedback execution: The optimal joint torque command is sent to the actuator through the communication bus, and the body status data is collected in real time and fed back to the preprocessing module to form a complete closed loop of "acquisition-sensing-optimization-control-feedback" to ensure the real-time performance and stability of control.

[0072] The adaptive control method for quadruped robots based on reinforcement learning and NMPC-WBC of the present invention has the following advantages:

[0073] 1. Strong autonomous start capability: By combining gradient descent method with soft joint constraints, the robot can achieve autonomous start and leg posture alignment under any initial posture without human intervention. It also has an emergency adjustment mechanism to avoid joint jamming or posture imbalance during the start-up process, thus improving the reliability of the start-up.

[0074] 2. Low perception cost and strong robustness: No foot force sensor or external sensing device is required. Implicit contact perception and terrain parameter estimation are achieved only through multi-source state data of the body combined with reinforcement learning network, which reduces hardware cost and improves anti-interference ability in harsh environments.

[0075] 3. Trajectory optimization and adaptive gait scheduling: The NMPC optimization module with dynamic soft constraints works in conjunction with the time-event hybrid triggering mechanism to adjust the gait phase and foot trajectory in real time based on contact perception results and terrain parameters, adapting to complex terrain changes and improving movement stability.

[0076] 4. Flexible and adaptable wheel-step switching: The wheel-step switching WBC control model is proposed, which breaks through the limitation of the traditional quadruped WBC that only considers leg movement. It realizes the adaptive switching between pure wheel mode and leg-lifting wheel-step mode. The switching logic is based on the body attitude change, which can effectively compensate for the attitude deviation caused by terrain changes and external force interference, ensuring that the body (and gimbal) maintains the preset attitude and improves the operation accuracy.

[0077] 5. High force distribution accuracy and strong anti-disturbance capability: The WBC hierarchical force distribution strategy based on task priority prioritizes the stability of the fuselage attitude and center of gravity. Combined with the compliant control mechanism, it compensates for contact estimation errors and external disturbances, reduces joint torque loss, and improves control accuracy and system robustness.

[0078] 6. High engineering practicality: The control process is simple and the quantitative parameters are clear. It adopts the mature RBDL dynamics library and SQP and OSQP solution algorithms. The control cycle is matched with the response speed of the actuator. It can be directly applied to various four-wheeled legged robot inspection and detection scenarios.

[0079] Example 1

[0080] Using an 80kg explosion-proof wheeled inspection robot with 16 degrees of freedom (12 drives + 4 wheels) as the verification platform, the adaptive control method for a four-wheeled robot based on reinforcement learning and NMPC-WBC was verified in a scenario with a 15° gentle slope and continuous potholes (maximum pothole depth 15cm). The test cycle was 1kHz and the test duration was 30 seconds.

[0081] I. Implementation Process and Core Data

[0082] 1. S1: System startup and attitude alignment (startup performance test)

[0083] Initial state: The robot is in a random overturned state, with a torso roll angle error of +2.5° and a pitch angle error of −1.8°, and a high risk of joint torque saturation.

[0084] Processing procedure: The system constructs a joint angle error function. Soft constraints are introduced into the joints using the gradient descent method. Alignment is completed after 30 iterations.

[0085] Test results:

[0086] Alignment time: only 0.45s (far lower than the industry standard startup time of more than 1.5s).

[0087] Attitude accuracy: The final torso roll / pitch angle error converges to within ±0.2°.

[0088] Torque performance: Joint torque fluctuations are controlled within ±15 N⋅m, with no severe impact, and a smooth standing up is successfully achieved.

[0089] 2. S2 & S3: Sensing and Terrain Estimation (Disturbance Resistance Test)

[0090] Processing procedure: The system collects IMU, motor current, and force data to construct... The standardized time-series state sequence is output as foot contact confidence by an LSTM-GRU network. and terrain parameters.

[0091] Test results:

[0092] Contact recognition accuracy: confidence level of foot contact The real-time update rate reached 100%, successfully distinguishing "gravel roads" ( ) and "grassland" ).

[0093] Time delay performance: The terrain stiffness estimation delay is less than 5ms, which effectively compensates for the impact of ground disturbance on NMPC prediction.

[0094] 3. S4: NMPC Optimization and Gait Scheduling (Constrained Adaptive Testing)

[0095] Processing procedure: Based on foot contact confidence Dynamically adjust soft constraint function Gait fine-tuning is achieved by combining a time-event hybrid triggering mechanism.

[0096] Test results:

[0097] Constraint effectiveness: In high-confidence contact ( During the contact transition phase, the friction cone constraint is strictly effective; Constraints are automatically softened to avoid unsolvable optimization problems.

[0098] Solution efficiency: The SQP algorithm is used to solve the NMPC problem (N=15), and the time taken for a single solution is consistently less than 4ms, which meets the real-time requirements.

[0099] 4. S5: Wheel-foot hybrid WBC control (obstacle crossing test in complex terrain)

[0100] Processing procedure: For 15° slopes and continuous potholes, the system dynamically switches the WBC weights (wheel system torque weight 0.6 / attitude weight 0.4) and performs layered torque distribution.

[0101] Test results:

[0102] Climbing performance: It can maintain a stable forward speed of 3.2m / s on a 15° slope, which is a significant improvement compared to the traditional wheeled control method (1.8m / s).

[0103] Pothole adaptation: When encountering continuous potholes with a depth of 15cm, the system automatically switches to the "leg lift wheel step" mode, and the leg lift height is adaptively adjusted.

[0104] Torque tracking: The error between the WBC output torque and the actual motor torque tracking is ≤2.5N⋅m, achieving high-precision torque tracking.

[0105] 5. S6: Closed-loop execution and system robustness (final performance verification)

[0106] Processing procedure: Control commands are sent to the motor driver to form a closed-loop feedback.

[0107] The final test data is summarized in the table below:

[0108]

[0109] II. Summary of Technical Effects

[0110] The above-mentioned actual test data shows that the present invention, through the complete technical path of steps S1 to S6, solves the technical pain points of traditional four-wheeled legged robots, such as lag during mode switching, poor terrain adaptability, and low control precision. The test results demonstrate that the method of the present invention, under complex explosion-proof terrain, combines high mobility (speed 3.2 m / s) with high stability (attitude fluctuation <1°), significantly outperforming existing technologies.

Claims

1. An adaptive control method for a four-wheeled legged robot based on reinforcement learning and NMPC-WBC, characterized in that... Includes the following steps: S1: Adaptive Start-up Trajectory Planning and Leg Posture Alignment: Initialize system parameters, collect robot initial joint angles and body posture, define target start-up posture, construct joint angle error function, use gradient descent method to iteratively update joint angles, introduce joint soft constraints and emergency adjustment mechanism to achieve adaptive start-up and leg posture alignment under any initial posture; S2: Proprioceptive State Acquisition and Preprocessing: Multi-source state data is acquired through robot body sensors to construct a time-series state sequence. z-score normalization preprocessing is used to eliminate dimensional differences and output a standardized time-series state sequence. S3: Reinforcement Learning Implicit Contact Sensing and Terrain Parameter Estimation: Input the standardized temporal state sequence into the reinforcement learning network, and output the foot contact confidence, contact temporal correction and terrain stiffness estimate to achieve implicit contact sensing and terrain parameter recognition without external sensors. S4: NMPC Optimization and Gait Scheduling: Based on contact perception results and terrain parameters, an NMPC optimization problem with dynamic soft constraints is constructed and solved using a sequential quadratic programming algorithm. Combined with a time-event hybrid triggering mechanism, adaptive gait phase scheduling is achieved. S5: Wheel-step switching WBC control and torque optimization: Construct a wheel-step switching WBC control model to achieve adaptive switching between pure wheel mode and leg-lifting wheel-step mode. Perform hierarchical force distribution based on task priority management strategy, introduce an adaptive control mechanism to compensate for disturbances, and output the optimal joint torque command. The wheel-step switching WBC control model includes mode switching determination logic, wheel-step hybrid dynamics model, and layered force distribution strategy; The expression for the wheel-step switching WBC control model is: ,in, This is the coefficient for switching between wheel-step modes. Indicates pure wheel mode. This indicates the leg-lifting wheel step mode. For the wheel-step switching coupling inertia matrix, To enable the switching of the coupling between the Coriolis force and the centrifugal force matrix, The gravity vector Let the joint position vector be... , These are the joint velocity and joint acceleration vectors, respectively. Select a matrix for joint torques. This is the joint torque vector. The Jacobian matrix is ​​adapted for wheel-walking modes. Pure wheel mode corresponds to the wheel center Jacobian, and leg-lifting wheel-walking mode corresponds to the foot contact Jacobian. The contact force vector at the foot; S6: Closed-loop feedback execution: The optimal joint torque command is sent to the actuator through the communication bus, and the body status data is collected in real time and fed back to the preprocessing module, forming a complete closed loop of acquisition-sensing-optimization-control-feedback.

2. The adaptive control method for a quadrupedal robot based on reinforcement learning and NMPC-WBC as described in claim 1, characterized in that: In step S1: The joint angle error function is: ,in, This is the current joint angle vector. Initiate the attitude joint angle vector for the target; The formula for iterative updating using the gradient descent method is as follows: ,in, The iteration step size, The gradient of the error function; The joint soft constraint is ,in, , These represent the minimum and maximum constraint values ​​for the joint angle, respectively.

3. The adaptive control method for a quadrupedal robot based on reinforcement learning and NMPC-WBC as described in claim 1, characterized in that: In step S2, the z-score normalization preprocessing formula is: ,in, The normalized state data, This is the original state data. The statistical mean of the state data in the training set. The statistical variance of the training set state data; the length of the time-series state sequence is set to... , recorded as It is used to preserve the historical temporal characteristics of multi-source ontology state data.

4. The adaptive control method for a quadrupedal robot based on reinforcement learning and NMPC-WBC as described in claim 1, characterized in that: In step S3: The reinforcement learning network adopts an LSTM-GRU hybrid network structure, containing two LSTM hidden layers and one GRU hidden layer. The input is the length. The standardized time-series state sequence is output as the foot contact confidence level. Contact timing correction and terrain stiffness estimate ; The network loss function is ,in, For foot contact confidence loss, For contact timing correction amount loss, Loss due to terrain stiffness estimation. , , This is the loss weighting coefficient.

5. The adaptive control method for a four-wheeled robot based on reinforcement learning and NMPC-WBC according to claim 1, characterized in that: In step S4: The dynamic soft constraint is linked to the foot contact confidence level, and the dynamic soft constraint function is: ,in, , These are the minimum and maximum penalty coefficients for soft constraints, respectively. For foot contact confidence; The weighted transformation formula for friction cone constraints is: ,in, The contact force vector at the foot end. The ground normal vector, The ground friction coefficient; through soft constraint terms A friction cone constraint is introduced to achieve dynamic adjustment of the constraint; The objective function of the NMPC optimization problem is: ,in, for The system state vector at any given time. for System status reference value at any time for Control the input vector at all times. , , These are the system state tracking weight matrix, the control input penalty weight matrix, and the terminal state tracking weight matrix, respectively. For NMPC prediction in the time domain; A sequential quadratic programming algorithm is used to solve the NMPC optimization problem, and the solution cycle is consistent with the robot control cycle. The time-event hybrid triggering mechanism is as follows: To preset the confidence threshold for foot contact, when Furthermore, when the foot touches the ground prematurely, it triggers an early gait phase switch, shortens the duration of the current gait phase, and avoids a rigid collision between the foot and the ground. when Furthermore, if the foot fails to touch the ground on time, a gait phase delay switch is triggered, extending the duration of the current gait phase, while also adjusting the contact timing based on the contact timing. Adjust foot height Ensure that the feet touch the ground stably; The time-triggered mechanism serves as the basic scheduling logic, pre-setting the baseline duration of each gait phase. The event-triggered mechanism serves as the dynamic correction logic, adjusting the gait phase in real time based on foot contact confidence and foot ground contact status. The two work together to achieve adaptive scheduling of gait phases.

6. The adaptive control method for a quadrupedal robot based on reinforcement learning and NMPC-WBC as described in claim 1, characterized in that: The mode switching determination logic is based on the fuselage attitude change amount, and the determination formula is as follows: ,in, The current fuselage attitude angle vector, roll angle Pitch angle Yaw angle , Let be the desired fuselage attitude angle vector; when hour, As the attitude change threshold, if the terrain change is too large or there is significant external interference, switch to the leg-lifting wheel-walking mode to compensate for the fuselage attitude deviation through leg posture adjustment. At that time, maintain pure wheel mode.

7. The adaptive control method for a quadrupedal robot based on reinforcement learning and NMPC-WBC as described in claim 1, characterized in that: The expression for the wheel-walking hybrid power model is: ,in, Here is the system inertia matrix. The matrix of Coriolis force and centrifugal force. The gravity vector Let the joint position vector be... , These are the joint velocity and joint acceleration vectors, respectively. Select a matrix for joint torques. This is the joint torque vector. For foot contact Jacobian matrix, This is the contact force vector at the foot.

8. The adaptive control method for a quadrupedal robot based on reinforcement learning and NMPC-WBC according to claim 1, characterized in that: The hierarchical force allocation strategy divides control tasks into three levels according to priority, from high to low: Highest priority: Centroid position and pose tracking task, objective function is ,in, Let the linear velocity of the center of mass be denoted as . This is the reference value for the linear velocity of the center of mass. For fuselage attitude angle, The desired attitude angle of the fuselage. , This is the weight matrix for the corresponding task; Second highest priority: Foot position tracking task, objective function is... ,in, The actual position of the foot. For the reference position of the foot, This is the task weight matrix. To reinforce the terrain stiffness estimate output by the learning process, It is the identity matrix; Lowest priority: Joint torque minimization task, objective function is ,in, This is the torque penalty weight matrix.

9. The adaptive control method for a quadrupedal robot based on reinforcement learning and NMPC-WBC as described in claim 1, characterized in that: In step S5, the compliance control mechanism is used to compensate for external disturbances caused by contact estimation errors, and its expression is as follows: ,in, The nominal joint torque is obtained by solving the quadratic programming algorithm. This is the disturbance compensation gain matrix. This is the disturbance estimate corresponding to the motor current deviation. , This represents the actual output torque of the motor. This is the command torque for the motor.

Images