Quadruped robot with bionic multi-degree-of-freedom spine and control method

Abstract

This invention proposes a quadruped robot with a biomimetic multi-degree-of-freedom spine and its control method. The robot includes a forequarter unit, a posterior trunk unit, four leg components, and a spine module. The spine module has three cascaded rotational degrees of freedom, namely yaw, pitch, and roll, from front to back. The peak torque of the spine module's joints is greater than that of the leg modules. A reinforcement learning framework based on a proximal policy optimization algorithm is constructed for training the robot's policy network. The training employs a dual-course learning mechanism that includes adaptive velocity range and acceleration constraints. The reward functions used in training include a gait reward function based on the phase difference of ipsilateral limb ground contact time, a spine undulation reward function, and a spine turning reward function. This invention utilizes a three-degree-of-freedom active spine and reward functions to enable the quadruped robot to perform more biologically-like rotational gait, highly maneuverable on-the-spot turning, and agile movements such as mid-air righting.

Description

Technical Field

[0001] This invention belongs to the field of robotics technology, specifically relating to a quadruped robot with a biomimetic multi-degree-of-freedom spine and its control method. Background Technology

[0002] Although modern quadruped robots have achieved unprecedented agility by combining high-torque actuators with reinforcement learning, mainstream high-performance platforms still generally adopt a rigid body design in pursuit of control simplicity and structural robustness. This means that even at peak speeds, they can only rely on a conservative trotting gait and cannot reproduce the explosive "galloping gait" unique to living organisms, fundamentally limiting further breakthroughs in performance.

[0003] While some studies have attempted to incorporate spinal joints, significant limitations remain: Limited degrees of freedom: Most designs only achieve single-degree-of-freedom (e.g., pitch or roll only) or two-degree-of-freedom, failing to replicate the complete rotational characteristics of the biological spine across three spatial axes (yaw, pitch, and roll). Rigid movement patterns: Although some studies have incorporated the spine, they often employ symmetrical movement patterns for control, inconsistent with the asymmetrical flexion and extension movements observed in organisms. Limited functionality: Currently, no research has been able to simultaneously utilize the spine within a unified framework to achieve complex agile behaviors such as high-speed running, flexible steering, and feline-like mid-air righting. Summary of the Invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a solution to the problems of rigid body trunk limitation performance, insufficient spinal degrees of freedom, and lack of efficient collaborative control strategies in the prior art.

[0005] To achieve the above objectives and other related objectives, the present invention proposes a quadruped robot with a biomimetic multi-degree-of-freedom spine, comprising a forequarter unit, a hindquarter unit, four leg assemblies, and a spine module connecting the forequarter unit and the hindquarter unit. Each of the leg components described has three degrees of freedom; The spinal module is configured with three series-connected rotational degrees of freedom, including a yaw spinal unit, a pitch spinal unit, and a roll spinal unit connected in series from front to back; the front end of the yaw spinal unit is connected to the anterior trunk unit, and the rear end of the roll spinal unit is connected to the posterior trunk unit. The spinal module's joint actuator is configured to have an output torque greater than the peak torque of the leg component's joint actuator, so as to drive the robot to perform high-speed running and stationary turning maneuvers through the active flexion, extension and torsion of the spinal module.

[0006] In one embodiment of the present invention, the series connection method of the spinal module is specifically as follows: The front end of the yaw spine unit is connected to the rear end face of the forequarter unit via a flange, and its axis of rotation is arranged in the vertical direction. The pitch spine unit is connected in series at the rear end of the yaw spine unit, and its rotation axis is arranged horizontally. The lateral spinal unit is connected in series at the rear end of the lateral spinal unit, and its rotation axis is arranged longitudinally along the front and rear. The rear end of the lateral spinal unit is connected to the front end of the posterior trunk unit.

[0007] In one embodiment of the present invention, the yaw angle, pitch angle and roll angle corresponding to the yaw spine unit, pitch spine unit and roll spine unit are all set with preset activity range thresholds.

[0008] This invention also proposes a control method for a quadruped robot, applied to a quadruped robot with a biomimetic multi-degree-of-freedom spine as described in any of the above embodiments, comprising: A reinforcement learning framework is constructed, which includes a policy network and an evaluation network, and is trained using a proximal policy optimization algorithm; The trained policy network is deployed in the controller of the quadruped robot; During the control execution phase, the strategy network outputs the target joint angles of the four leg components and the spine module at a preset frequency. The target joint angles are converted into joint torques by a proportional-derivative position controller to drive the robot's movement. The reinforcement learning framework employs a dual-curricular learning strategy during training, which includes: Adaptive velocity range course: The linear velocity command range and angular velocity command range of the robot are set as dynamically adjustable variables, and the linear velocity command range and angular velocity command range are independently and gradually expanded according to the convergence of the policy network during the training process. Command generator with acceleration constraints: When generating motion commands, the acceleration of the commands is limited to smooth the speed transition, and when switching from linear motion to steering motion, the effective time of the angular velocity command is set with a random delay.

[0009] In one embodiment of the present invention, the training reward function of the reinforcement learning framework includes gait reward, spinal undulation reward and spinal turning reward; The gait reward is calculated based on the temporal phase difference of ipsilateral limb ground contact events and is used to guide the robot to learn a rotational sprint gait with two airborne phases. The spinal wave reward includes an amplitude incentive, which determines the flexion or extension target of the spine based on the direction of leg velocity and sets asymmetric flexion angle threshold and extension angle threshold. The spinal steering reward is calculated based on a dead zone threshold mechanism. When the target angular velocity is within the preset dead zone range, the reward value is zero. When the target angular velocity exceeds the preset dead zone range, the consistency score between the spinal yaw angle and the target steering direction is calculated.

[0010] In one embodiment of the present invention, the method further includes: The system comprises a comprehensive reward function system consisting of speed tracking rewards, agile motion rewards for achieving high-speed driving, flexible turning, and precise path tracking, and self-righting rewards for achieving mid-air self-righting, combined with auxiliary regularization penalties. The policy network is iteratively trained using the reinforcement learning framework and the complete reward function system to drive the quadruped robot to learn and achieve the agile movement behavior.

[0011] In one embodiment of the present invention, the calculation process of the gait reward includes: For ipsilateral limb pairs, calculate the actual time phase difference of their ground contact events. Phase difference with target The shortest phase distance Δδ between them; Calculate the reward value of a unilateral limb pair using the Gaussian kernel function. : ,in, This refers to the phase tolerance coefficient. Total gait reward Forelimbs responding to rewards Rewards for hind limbs The product of: .

[0012] In one embodiment of the present invention, the spinal wave reward is used to guide the flexion-extension movement of the spinal joints to produce large amplitude and phase-coordinated flexion-extension movements; the spinal wave reward The calculation process includes: Based on effective leg speed With spinal angular velocity Product calculation phase coordination reward The formula is as follows: in, This is the sensitivity coefficient. It is determined by the average of the hind limb thigh velocity and the forelimb thigh velocity after sign reversal; Based on spinal state s and spinal joint angle Calculate amplitude reward and to Threshold truncation is performed to obtain ; like Exceeding the threshold Then calculate the over-limit penalty term. The formula is as follows: in, Weighting for exceeding the limit penalty; The final spinal undulation reward is calculated by combining the above factors: in, Weights are added to the amplitude.

[0013] In one embodiment of the present invention, the amplitude excitation calculation step specifically includes: Based on the effective leg speed The symbol determines the desired spinal state. And calculate the basic amplitude bonus. ; right Threshold truncation is performed to obtain : , wherein, according to the spinal condition Dynamically adjust the expected threshold When the spine is in a flexed position, the flexion threshold is used. When the spine is in an extended state, the extension threshold is used. ,and > .

[0014] In one embodiment of the present invention, the spinal steering reward is used to incentivize the yaw spinal joint to yaw in a direction consistent with the commanded angular velocity; The spinal turning reward A piecewise function is used, and a dead zone threshold is introduced. To distinguish between linear motion and turning motion; When the absolute value of the command angular velocity Less than the dead zone threshold At that time, the spinal rotation reward is zero; When the absolute value of the command angular velocity Greater than or equal to the dead zone threshold At that time, the spinal turning reward is calculated according to the following formula: ,in, This represents the actual deflection angle of the yaw spine joint. ω is the command angular velocity, and k is the scaling factor for controlling the reward saturation velocity.

[0015] In one embodiment of the present invention, the specific implementation steps of the adaptive speed range course include: Set up linear velocity courses and angular velocity courses, where the upper limit of the target range for the linear velocity course is a first preset value, and the upper limit of the target range for the angular velocity course is a second preset value; In the early stages of training, the linear velocity command range is limited to a first preset range, and the angular velocity command range is limited to a second preset range. The monitoring strategy network's average reward value over the most recent N training rounds is used. When the average round tracking reward of the linear velocity course exceeds 80% of a preset threshold, the upper and lower limits of the linear velocity command range are increased by a preset step size Δ=0.1. When the average round tracking reward of the angular velocity course exceeds 60% of a preset threshold, the upper and lower limits of the angular velocity command range are increased by a preset step size Δ=0.1, until the target range is reached respectively.

[0016] In one embodiment of the present invention, the instruction generator with acceleration constraints smooths the instruction using the following formula when executing step S2: , in, The smoothed instruction velocity at time t+1 Let be the command speed at time t. For the target speed, The preset maximum acceleration limit value, To control the time step This invention proposes a quadruped robot equipped with a biomimetic serial three-DOF active spine, which can effectively reproduce the three-axis rotational motion of a quadruped spine. By employing a dedicated reinforcement learning framework with a custom reward function, the robot achieves a variety of agile movement behaviors, including high-speed running, maneuvering, precise path tracking, and emerging feline-like aerial righting abilities.

[0017] Through comprehensive cross-validation on both IsaacSim and MuJoCo simulators, this study established a highly robust control strategy with minimal transfer differences between simulators. In the simulation environment, the robot, employing a rotational-rotational gait, achieved a maximum running speed of 6.9 m / s and a stationary turning angular velocity of 7.2 rad / s, significantly surpassing the performance of the rigid body baseline model and the current state-of-the-art spinal quadruped robot in simulators. The analysis results fully demonstrate that the introduced three-degree-of-freedom spine significantly enhances the robot's speed, maneuverability, accuracy, and stability.

[0018] Compared with the prior art, the present invention has the following beneficial effects: Complete three-degree-of-freedom spine design: This invention designs a biomimetic three-degree-of-freedom active spine composed of three tandem spinal joints: yaw, pitch, and roll. It can achieve three-axis rotation in space, fully reproduce the flexible movement of the spine of a biological quadruped in three dimensions, and overcome the limitations of existing spinal quadruped robots that only have single or two degrees of freedom and cannot achieve complete spatial rotation.

[0019] Optimized spinal configuration arrangement: The present invention adopts a spinal configuration arrangement of "prioritizing yaw guidance in the front section, concentrating pitch power in the middle section, and taking into account roll compliance in the rear section", which enables the robot to exhibit motion dynamics characteristics that are closer to those of biological quadrupeds.

[0020] Gait reward function: The gait reward function designed in this invention guides the strategy to naturally explore a complete rotational sprint gait through minimum foot placement pattern constraints, including two airborne phases and a distinctive circular foot placement sequence, thus avoiding gait rigidity caused by excessive constraints.

[0021] Spinal undulation reward function: The spinal undulation reward function designed in this invention actively drives the spine to produce phase-coordinated flexion and extension movements, and introduces an asymmetric threshold to faithfully simulate the movement characteristics of the biological spine, effectively avoiding the problem in passive methods where the strategy converges to invalid spinal movements due to reward cheating.

[0022] Spinal steering reward function: The spinal steering reward function designed in this invention utilizes the yaw spinal joint to achieve a steering mechanism similar to front wheel steering, and achieves effective decoupling of forward motion and yaw maneuvering through dead zone threshold.

[0023] Dual-course learning strategy: This invention adopts a dual-course learning strategy, which significantly improves training stability and enables the robot to simultaneously achieve a variety of highly agile movement behaviors such as high-speed running, flexible turning, precise path tracking, and feline-like aerial self-righting within a unified reinforcement learning framework. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the structure of a quadruped robot.

[0026] Figure 2 This is a schematic diagram of the spinal module, where (a) and (b) represent different states, respectively.

[0027] Figure 3 This is an exploded view of the spinal module.

[0028] Figure 4 Training diagram for reinforcement learning framework.

[0029] Figure 5The robots exhibited agile movement capabilities in various scenarios, including (a) high-speed running, (b) flexible turning in place and turning while moving, and (c) emerging feline-like aerial self-righting.

[0030] Figure 6 This describes the linear velocity tracking performance during high-speed driving.

[0031] Figure 7 This refers to the changes in the angles of the three spinal joints during high-speed movement.

[0032] Figure 8 This refers to the angular velocity tracking performance during the turning process.

[0033] Figure 9 This refers to the changes in the angles of the three spinal joints during the steering process.

[0034] Figure 10 The performance evaluation for the figure-eight path tracking task includes (a) the tracking trajectories of the spinal robot and the rigid body robot, (b) the changes in yaw angle and yaw rate during path tracking, and (c) the changes in the angles of the three spinal joints during path tracking.

[0035] Figure 11 This refers to the changes in roll angle and roll velocity during the in-flight self-turning process.

[0036] Figure 12 This represents the changes in the angles of the three spinal joints during the mid-air self-righting process. Detailed Implementation

[0037] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0038] It should be noted that the illustrations provided in this embodiment are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components relevant to the present invention, and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0039] Please see Figures 1 to 3As shown, this embodiment provides a quadruped robot with a biomimetic multi-degree-of-freedom spine. The quadruped robot includes a forequarter unit 100, a posterior trunk unit 200, four leg assemblies 300, and a spine module 400 connecting the forequarter unit 100 and the posterior trunk unit 200. The robot's torso consists of the forequarter unit and the posterior trunk unit, which are connected by the spine module. Each leg consists of a hip unit, a thigh unit, and a lower leg unit. The robot has a total mass of 20 kg, a total body length of 625 mm, a width of 380 mm, a leg length of 580 mm, and a standing height of approximately 440 mm.

[0040] Please see Figures 1 to 3 As shown, each leg assembly 300 has three degrees of freedom, specifically, the hip unit, thigh unit, and lower leg unit each provide one rotational degree of freedom. The spine module 400 is configured with three series-connected rotational degrees of freedom, including a yaw spine unit 410, a pitch spine unit 420, and a roll spine unit 430 connected in series from front to back; the front end of the yaw spine unit 410 is connected to the front trunk unit 100, and the rear end of the roll spine unit 430 is connected to the rear trunk unit 200. Thus, the quadruped robot has a total of fifteen degrees of freedom, including twelve degrees of freedom from the four leg assemblies and three degrees of freedom from the spine module.

[0041] Please see Figures 1 to 3 As shown, the joint actuators of the spine module 400 are configured to have an output torque greater than the peak torque of the joint actuators of the leg assembly 300, so as to drive the robot to perform high-speed running and stationary turning maneuvers through the active flexion, extension and torsion of the spine module 400. Specifically, the peak torque of the leg joint motors is 33.5 N·m. To drive the robot's movement more effectively, the spine joint motors are designed with a higher peak torque, reaching 50 N·m. This high torque configuration ensures that the spine joints can actively generate sufficient force and motion, thereby effectively driving the robot to achieve high-speed running and agile turning.

[0042] This embodiment introduces a biomimetic three-degree-of-freedom active spine composed of three tandem spinal joints: yaw, pitch, and roll. This enables three-axis rotation in space, fully replicating the flexible movement of the spine of a biological quadruped in three dimensions. It overcomes the limitations of existing spinal quadruped robots that only have a single or two-degree-of-freedom spine and cannot achieve complete spatial rotation, providing a physical basis for subsequent high-agility movements such as high-speed running and flexible turning.

[0043] Please see Figures 1 to 3 As shown, this embodiment further defines the series connection method of the spinal module 400. Specifically, the series connection method of the spinal module 400 is as follows: Please see Figures 1 to 3 As shown, the front end of the yaw spine unit 410 is connected to the rear end face of the forequarter unit 100 via a flange, and its rotation axis is arranged in the vertical direction. The yaw spine unit is located at the very front of the spine module, enabling the robot's front half to perform yaw movements independently, simulating the line-of-sight-guided turning behavior of quadrupeds. By changing the deflection of the forelimbs relative to the hindlimbs in the workspace, the robot naturally guides its body into a turn.

[0044] Please see Figures 1 to 3 As shown, the pitching spine unit 420 is connected in series at the rear end of the yaw spine unit 410, and its axis of rotation is arranged horizontally. The pitching spine unit is located in the middle of the spine module, at the mechanical center between the anterior and posterior hip joints, to maximize the stride gain from spinal flexion and extension. If it is arranged off-center, an unbalanced lever arm will be generated, thereby reducing the effectiveness of spinal energy storage.

[0045] Please see Figures 1 to 3 As shown, the roll spine unit 430 is connected in series at the rear end of the pitch spine unit 420, with its rotation axis arranged longitudinally along the front-rear direction. The rear end of the roll spine unit 430 is connected to the front end face of the hindquarters unit 200. Located at the rear end of the spine module, close to the pelvis and hind limbs, the roll spine unit causes the hindquarters to twist relative to the forequarters, simulating the independent adjustment ability of the pelvis when quadrupeds move on complex terrain, and the behavior of the hind limbs to counteract centrifugal force through lateral roll during high-speed turns.

[0046] This embodiment employs a spinal configuration arrangement that prioritizes yaw guidance in the initial stage, concentrates pitch power in the middle stage, and balances roll with compliant movement in the final stage, enabling the robot to exhibit motion dynamics more closely resembling those of a biological quadruped. Experiments have demonstrated that this design allows the pitch joint to exhibit significant asymmetric changes during high-speed movement (the flexion amplitude is approximately twice the extension amplitude), while the roll spinal joints maintain minimal change to ensure lateral stability, fully conforming to biological principles.

[0047] Please see Figures 1 to 3As shown, in this embodiment, the key joint parameters of the quadruped robot are specifically defined. Specifically, the peak torque of the joint motors of the leg assembly 300 is 33.5 N·m, and the peak torque of the joint motors of the spine module (400) is 50 N·m. The range of motion of the yaw spine unit 410, pitch spine unit 420, and roll spine unit (430) are: yaw angle ±1 rad, pitch angle ±1.5 rad, and roll angle ±2 rad, respectively. The above parameters have been carefully optimized. The large range of motion of the spine joints (especially the ±2 rad of the roll joint) enables the robot to perform large-angle aerial attitude adjustments, providing physical possibilities for behaviors such as aerial self-righting. At the same time, the high peak torque of 50 N·m ensures that the spine joints can overcome inertial loads and actively generate the required flexion, extension, and torsional torques during high-speed movement, thereby achieving a 15%-25% speed increase and a 16.1%-83.3% increase in steering angular velocity.

[0048] Please see Figure 4 As shown, in terms of control methods, the present invention also provides a reinforcement learning control method applied to the above-mentioned quadruped robot, comprising the following steps: Step S1: Construct the simulation environment. Establish a full dynamic model of the quadruped robot in the physics simulation engine, including a forequarter unit 100, a hindquarter unit 200, four leg components 300, and a spine module 400 with three serial active degrees of freedom. Set the output torque range of each joint of the spine module 400 to ±50 N·m, and the range of motion of each joint as described in the above embodiment.

[0049] Step S2: Design the reward function. Design a composite reward function for training high-speed driving, stationary turning, and mid-air flipping tasks. This reward function includes: Speed ​​bonus: Encourages the robot's center of mass to reach the target high speed (e.g., 4.0 m / s or higher); Gait reward: Calculated based on the temporal phase difference of ipsilateral limb ground contact events, used to guide the robot to learn a rotational sprint gait with two airborne phases; Spinal undulation reward: includes an amplitude incentive item, which determines the flexion or extension target of the spine based on the direction of leg velocity, and sets asymmetric flexion angle thresholds and extension angle thresholds. Spinal turning reward: Calculated based on the dead zone threshold mechanism. When the target angular velocity is within the preset dead zone range, the reward value is zero. When the target angular velocity exceeds the preset dead zone range, the consistency score between the spinal yaw angle and the target turning direction is calculated. Regularization penalty: Penalizes excessive work done by the spinal joints to encourage energy-efficient movement patterns.

[0050] Step S3: Train the policy network using a deep reinforcement learning algorithm. The current state of the robot is used as input, and the network outputs target position or torque commands for each joint of the spine module 400 and leg components 300. Millions of iterations are performed in a simulation environment until the policy network converges.

[0051] Step S4: Deployment and Physical Control. The trained policy network is converted into real-time control code and deployed to the onboard computer of the quadruped robot. During physical operation, the onboard computer collects IMU and joint encoder data in real time, calculates state vectors, and obtains drive commands for the spine module 400 and leg components 300 through policy network inference, thereby realizing complex agile behaviors such as high-speed running, turning in place, and self-righting in mid-air.

[0052] Please see Figure 4 As shown, this embodiment provides a control method for a quadruped robot, applied to a quadruped robot with a biomimetic multi-degree-of-freedom spine as described in the above embodiments. Figure 4 As shown, this method employs a reinforcement learning framework based on a proximal policy optimization algorithm to control the robot, specifically including: A reinforcement learning framework is constructed, which includes a policy network and an evaluation network, and is trained using a proximal policy optimization algorithm; The trained policy network is deployed in the controller of the quadruped robot; During the control execution phase, the strategy network outputs the target joint angles of the four leg components and the spine module at a preset frequency. The target joint angles are converted into joint torques by a proportional-derivative position controller to drive the robot's movement. During the control execution phase, only the Actor (policy network) is used. The policy network outputs scaled target joint angles for the four leg components and the 15 joints (12 leg joints and 3 spinal joints) of the spine module at a frequency of 50 Hz. These target joint angles are then input into a simple proportional-derivative position controller, which calculates the corresponding joint torques to drive the robot's movement.

[0053] The reinforcement learning framework employs a dual-curricular learning strategy during training, which significantly improves training stability and final performance. The dual-curricular learning strategy includes: Adaptive Velocity Range Curriculum: The robot's linear velocity and angular velocity command ranges are set as dynamically adjustable variables, and are independently and progressively expanded during training based on the convergence of the policy network. Specifically, when the average tracking reward per round exceeds a preset threshold (80% for linear velocity and 60% for angular velocity), the range increases in increments of Δ=0.1 in each training iteration until the full range is reached. The learning processes for linear velocity and angular velocity are independent of each other, enabling the agent to master forward motion and yaw control separately at its own pace.

[0054] An acceleration-constrained command generator: When generating motion commands, the acceleration of the commands is limited to smooth the velocity transition. Instead of directly issuing step-like target velocities, the command velocity smoothly transitions to the target velocity in each simulation step. Furthermore, when switching from linear motion to steering motion, a random delay is set for the activation time of angular velocity commands. For steering tasks, the angular velocity command is further delayed by a random amount of time after the linear velocity command is issued, allowing the robot to reach a stable forward speed before starting the steering maneuver.

[0055] The dual-course learning strategy adopted in this embodiment avoids the non-convergence problem caused by the difficulty of tasks in the early training by using an adaptive speed range, and avoids physically unfollowable speed abrupt changes by using acceleration constraints and turning delays, thereby significantly improving training stability. This enables the robot to simultaneously achieve a variety of highly agile movement behaviors such as high-speed running, flexible turning, precise path tracking, and feline-like aerial self-righting within a unified reinforcement learning framework.

[0056] In this embodiment, the training reward function of the reinforcement learning framework is specifically defined. In addition to velocity tracking rewards and auxiliary regularization penalties, the training reward function of the reinforcement learning framework also specifically includes gait rewards, spinal undulation rewards, and spinal turning rewards.

[0057] Velocity tracking reward is the core task reward of the entire reward system, used to evaluate the degree of matching between the robot's actual motion speed (including forward linear velocity, lateral linear velocity, and yaw angular velocity) and the command target speed. Auxiliary regularization penalty is a set of restrictive negative rewards, including joint torque penalty, motion smoothness penalty, joint velocity penalty, base posture penalty, foot sliding penalty, collision penalty, etc., used to constrain the behavior of the policy network within a physically reasonable, safe and controllable range.

[0058] The core innovation of this invention lies in the following three behavior shaping reward functions specifically designed for spinal quadruped robots: Gait reward: The gait reward is calculated based on the temporal phase difference of ipsilateral limb ground-touching events and is used to guide the robot to learn a rotational sprint gait with two airborne phases. The core idea of ​​this reward is to specify only the minimum landing pattern that can distinguish rotational sprint from other gaits, without rigidly specifying the complete gait cycle, duty cycle, or stride frequency. By constraining only the phase relationship between ipsilateral limb pairs, this reward provides sufficient guidance to prevent the strategy from converging to a jumping gait (simultaneous landing of ipsilateral limbs) while maintaining the flexibility of gait generation.

[0059] Spinal undulation reward: The spinal undulation reward is used to actively drive the flexion and extension joints to produce large-amplitude and phase-coordinated wave-like flexion and extension movements. This reward includes an amplitude incentive, which determines the flexion or extension target of the spine based on the direction of leg velocity and sets asymmetric flexion and extension angle thresholds (flexion thresholds). >Stretch threshold (This is to faithfully simulate the dynamic characteristics of a biological spine where there are significant differences in angular limitations between flexion and extension.)

[0060] Spinal yaw reward: The spinal yaw reward is calculated based on a dead zone threshold mechanism and is used to incentivize the yaw spinal joint to yaw in a direction consistent with the commanded angular velocity. When the target angular velocity is within the preset dead zone range (linear motion), the reward value is zero to prevent unnecessary spinal yaw; when the target angular velocity exceeds the preset dead zone range, a consistency score between the spinal yaw angle and the target turning direction is calculated.

[0061] In this embodiment, the three reward functions act as behavioral guidance signals in the learning framework, shaping the movement behavior of the policy network from three dimensions: gait coordination, spinal undulation, and spinal turning. The three functions are parallel, complementary, and irreplaceable, working together to ensure that the policy network learns complete, coordinated, and fully utilized spinal mechanism movement behavior.

[0062] The method also includes: a comprehensive speed tracking reward, an agile movement reward for achieving high-speed running, flexible turning and precise path tracking (i.e., gait reward, spinal undulation reward and spinal turning reward in the above embodiments), and a self-righting reward for achieving self-righting in the air (this self-righting ability is an emergent behavior during training, which is jointly guided by the degree of freedom of the spinal module and the reward function), and combined with auxiliary regularization penalty, to form a complete reward function system.

[0063] Using the reinforcement learning framework, the policy network is iteratively trained based on the complete reward function system to drive the quadruped robot to learn and achieve the agile movement behavior. Experiments show that the trained robot can achieve: 1) High-speed running: Achieving a maximum running speed of 6.9 m / s using a rotational running gait; 2) Flexible turning: Achieving a turning angular velocity of 7.2 rad / s in place and 7.8 rad / s while moving; 3) Precise path tracking: Tracking the reference path more accurately than the rigid body torso reference model in the figure-eight path tracking task; 4) Mid-air self-righting: When falling from extreme initial postures such as complete inversion, it can adjust its posture in mid-air using a three-degree-of-freedom spine to achieve stable landing on all four legs.

[0064] like Figure 5 As shown, the characteristic of the rotational sprint gait is that the limbs touch down in a circular sequence and have two airborne phases. Our gait reward design captures only the temporal asymmetry between ipsilateral limb pairs (the leading leg touches down slightly earlier than the following leg) as a key feature to distinguish sprint gaits from jumping gaits.

[0065] The calculation process for the gait reward includes: For ipsilateral limb pairs (forelimb pairs and hindlimb pairs), calculate the actual time phase difference of their ground contact events. Phase difference with target The shortest phase distance Δδ between the measured phase difference and the target phase difference is calculated, taking into account the periodicity of the gait cycle.

[0066] Calculate the reward value of a unilateral limb pair using the Gaussian kernel function. : ,in, This is the phase tolerance coefficient, used to control the tolerance around the target phase difference.

[0067] Total gait reward Forelimbs responding to rewards Rewards for hind limbs The product of: .

[0068] It is worth noting that this embodiment imposes explicit constraints on the phase relationship between the forelimb and hindlimb pairs, and does not limit the duration of the flight period. By setting different target phase differences for the forelimb and hindlimb pairs, the reward provides sufficient guidance to prevent the strategy from converging to a jumping gait. =0), while maintaining flexibility in gait generation. This deliberate non-sufficiency constraint design, combined with auxiliary rewards such as airtime and velocity tracking, allows the strategy to naturally explore a complete rotational-rotational sprint gait, including its two distinct airtime phases and the signature circular foot-landing sequence. Experiments show that, using this gait reward, the robot equipped with an active spine achieved a stable speed of 6.9 m / s in IsaacSim, a 15% improvement compared to the rigid body baseline model's 6.0 m / s; and reached 6.0 m / s in MuJoCo, a 25% improvement compared to the baseline model's 4.8 m / s.

[0069] This embodiment defines the specific calculation process for the spinal undulation reward. This reward is used to guide the flexion-extension joints to produce large-amplitude and phase-coordinated flexion-extension movements. The spinal undulation reward... The calculation process includes the following three parts: Calculate phase coordination reward Based on effective leg speed With spinal angular velocity The product calculation. Effective leg speed. The phase coordination reward is determined by the average of the hind limb thigh velocity and the forelimb thigh velocity after sign reversal. A positive value indicates a phase requiring spinal extension, while a negative value indicates a phase requiring spinal flexion. The phase coordination reward formula is as follows: ,in, This is the sensitivity coefficient.

[0070] Calculate the amplitude excitation term and handle the over-limit penalty: based on the spinal state s (by... The sign is determined, s∈{-1,+1} represents flexion or extension respectively, and the angle of the spinal joint is determined. Calculate the basic amplitude reward Then on Threshold truncation is performed to obtain the truncated amplitude reward. : .

[0071] Among them, the expected threshold is dynamically adjusted according to the spinal state s. When the spine is in a flexed position, the flexion threshold is used. When the spine is in an extended state, the extension threshold is used. ,and > To simulate the asymmetric characteristics of the biological spine, where the flexion amplitude is greater than the extension amplitude.

[0072] like Exceeding the threshold Then calculate the over-limit penalty term. : in, This is the weight for exceeding the limit penalty.

[0073] Finally, the final spinal undulation reward is calculated by combining the above items: ,in, Weights are added to the amplitude.

[0074] This design ensures that the spine is rewarded when it moves with sufficient amplitude in the correct phase, and penalized when it exceeds safe mechanical limits. Experiments have shown that with this spinal undulation reward, the flexion-extension joint exhibits significant asymmetric changes during high-speed movement, with an angle range of [-0.5, 0.25] rad in IsaacSim and [-0.6, 0.3] rad in MuJoCo. The flexion amplitude is approximately twice the extension amplitude, perfectly replicating the kinematic characteristics of a biological spine, in stark contrast to existing methods that rigidly prescribe symmetrical spinal movements.

[0075] In this embodiment, the amplitude excitation calculation steps are further defined.

[0076] The amplitude excitation calculation steps specifically include: Based on the effective leg speed The symbol determines the desired spinal state. And calculate the basic amplitude bonus. .

[0077] right Threshold truncation is performed to obtain the truncated amplitude reward. : , The desired threshold is dynamically adjusted based on the spinal state s. The flexion threshold is used when the spine is in a flexed position. When the spine is in an extended state, the extension threshold is used. And satisfy > .

[0078] This embodiment faithfully simulates the natural kinematic characteristics of a tetrapod during high-speed running, where the flexion amplitude of the spine is significantly greater than the extension amplitude, by introducing asymmetric flexion and extension thresholds. This effectively avoids the problem in existing methods that violate natural kinematic laws due to rigidly defining symmetrical spinal movements.

[0079] In this embodiment, the specific calculation process for the spinal steering reward is defined. This reward is used to incentivize the yaw spinal joint to yaw in a direction consistent with the commanded angular velocity.

[0080] Traditional rigid-body quadruped robots can only achieve steering through the differential speed of their left and right legs, making it difficult to decouple forward movement from yaw maneuvering. The introduction of the yaw spinal joint solves this limitation, functioning similarly to a front-wheel-guided vehicle steering mechanism.

[0081] The spinal turning reward A piecewise function is used, and a dead zone threshold is introduced. To distinguish between linear motion and turning motion: When the absolute value of the command angular velocity Less than the dead zone threshold At that time, the spinal turning reward is zero: =0. This effectively prevents any unwanted spinal deflection during linear motion.

[0082] When the absolute value of the command angular velocity Greater than or equal to the dead zone threshold At that time, the spinal turning reward is calculated according to the following formula: ,in, This represents the actual deflection angle of the yaw spine joint. ω is the command angular velocity, and k is the scaling factor for controlling the reward saturation velocity.

[0083] This formula uses the denominator It naturally incorporates the symbols (positive and negative) of the instructions, ensuring that when the spine deflects in the correct direction ( and A positive reward is given when the sign is the same, while a negative penalty is given when the sign is wrong, thus providing a clear and directional learning signal.

[0084] Experiments have shown that, with the addition of this spinal steering reward, the robot equipped with an active spine achieved a yaw rate of 7.2 rad / s in the stationary turning task, a 16.1% improvement over the rigid baseline model; and 7.8 rad / s in the moving turning task, a 22.9% improvement over the rigid baseline model. These yaw rates significantly exceed the previously reported highest level of 3 rad / s for quadruped robots with yaw-spinning joints.

[0085] In this embodiment, the specific implementation steps of the adaptive speed range course are defined.

[0086] The specific implementation steps of the adaptive speed range course include: Set linear velocity courses and angular velocity courses, where the upper limit of the target range of the linear velocity course is a first preset value (e.g., the maximum safe speed set according to the physical limits of the robot), and the upper limit of the target range of the angular velocity course is a second preset value (e.g., the maximum safe angular velocity set according to the joint range of motion and torque capability).

[0087] In the early stages of training, the linear velocity command range is limited to a first preset range (a smaller velocity range), and the angular velocity command range is limited to a second preset range (a smaller angular velocity range) to avoid non-convergence problems caused by excessive task difficulty in the early training.

[0088] The monitoring strategy network's average reward value over the most recent N training rounds is used. When the average round tracking reward of the linear velocity course exceeds 80% of a preset threshold, the upper and lower limits of the linear velocity command range are increased by a preset step size Δ=0.1; when the average round tracking reward of the angular velocity course exceeds 60% of a preset threshold, the upper and lower limits of the angular velocity command range are increased by a preset step size Δ=0.1, until the target ranges are reached respectively. The course progress of linear velocity and angular velocity is independent of each other, enabling the agent to master forward motion and yaw control separately according to their respective progress.

[0089] In this embodiment, the smoothing process of the instruction generator with acceleration constraints is specifically defined.

[0090] The instruction generator with acceleration constraints uses the following formula to smooth the instructions: ,in, The smoothed instruction velocity at time t+1 Let be the command speed at time t. For the target speed, The maximum acceleration limit is a preset value (sampled from a predefined range). To control the time step.

[0091] This formula ensures that the change in command speed at each time step does not exceed [a certain value]. This avoids physically unpredictable speed abrupt changes. In this way, the commanded speed can smoothly transition to the target speed, ensuring that the generated motion commands remain within the robot's physical capabilities. For turning tasks, the angular velocity command is further delayed by a random amount of time after the linear velocity command is issued, allowing the robot to reach a stable forward speed before starting the turn, further improving the stability and controllability of the turning motion.

[0092] Understandably, within the reinforcement learning framework, the three core reward functions mentioned above serve as behavioral shaping signals, guiding the policy network in a targeted manner from three orthogonal dimensions: limb coordination, trunk dynamics, and steering maneuvering. Gait reward function: As a guiding signal for limb coordination, its core function is to constrain the timing of limb ground contact and prevent the strategy from falling into unstable gait states such as bounds. By applying only minimum footing pattern constraints, this function guides the policy network to converge to the Rotary Gallop gait unique to biological quadrupeds, ensuring that the robot achieves maximum speed and energy efficiency.

[0093] Spinal undulation reward function: As the driving signal for the trunk's participation in forward movement, this function is a key innovation that distinguishes it from rigid trunk robots. Instead of forcing the spine to execute a preset regular waveform, it drives the policy network to autonomously explore and generate rhythmic, large-amplitude flexion and extension movements that conform to biomechanics through an implicit incentive mechanism, thereby enhancing propulsion by utilizing spinal dynamics.

[0094] Spinal steering reward function: As a guiding signal for trunk participation in steering, this function incentivizes the strategy to actively coordinate the yaw spinal joints under turning commands, simulating the biological line-of-sight guidance mechanism to achieve coordinated steering of the trunk and limbs.

[0095] The three reward functions mentioned above are computed in parallel at each time step of training, each independently evaluating motion performance in a specific dimension. Finally, they are weighted and integrated in step six to form a unified total reward signal, which is fed back into the reinforcement learning framework to guide the PPO algorithm in updating the policy network parameters. These three functions are parallel and complementary, working together to ensure that the policy learns to fully utilize the complete agility movement skills of the multi-degree-of-freedom spine.

[0096] Speed ​​tracking reward is the core reward of the entire reward system, used to evaluate the degree of match between the robot's actual movement speed (including forward linear velocity, lateral linear velocity, and yaw rate) and the commanded target speed. The closer the actual speed is to the commanded speed, the larger the reward value. This reward defines the robot's fundamental motion task: moving and turning at the commanded speed.

[0097] Auxiliary regularization penalties are a set of restrictive negative rewards, including joint torque penalties, motion smoothness penalties, joint velocity penalties, base posture penalties, foot sliding penalties, collision penalties, etc., used to constrain the behavior of the policy network within a physically reasonable, safe and controllable range, thereby improving the overall quality of motion.

[0098] It's also understandable that the evaluation network is a standard internal component of the PPO algorithm. Its role is to estimate the state value function during training, providing an advantage estimation benchmark for the parameter updates of the policy network. In all subsequent training steps, the evaluation network implicitly participates: the reward signals from the gait reward function, spinal undulation reward function, and spinal turning reward function are all transformed into advantage functions by the evaluation network and used to guide the updates of the policy network.

[0099] Since the structure and training method of the evaluation network follow the standard process of the PPO algorithm, and only the policy network is deployed after training (the evaluation network is discarded during the deployment phase), it is not described separately in the aforementioned steps.

[0100] To individually verify the contribution of the multi-jointed spine, this invention compares two configurations in a simulation environment: (1) a complete robot model equipped with a 3-DOF active spine; and (2) a rigid body baseline model with locked spinal joints. Both configurations share the same hyperparameters, observation / action space, and reward function (excluding spine-specific terms). Furthermore, both were trained with similar number of iterations to ensure fairness and rigor in the comparison.

[0101] 1. High-speed acceleration powered by the spine Rotary-rotational sprinting is the fastest gait employed by agile quadrupeds such as cheetahs, relying on coordinated spinal flexion and extension to enhance limb propulsion. For example... Figure 5 As shown in (a), the framework proposed in this invention successfully realizes this complex asymmetrical gait and effectively utilizes spinal dynamics to improve athletic performance.

[0102] To evaluate the speed limit, we issued high-speed tracking commands in both simulators, such as... Figure 6 As shown, in IsaacSim, the active spinal robot achieved a stable speed of 6.9 m / s, a 15% improvement compared to the baseline rigid body model's speed of 6.0 m / s. In MuJoCo, the performance improvement is even more significant: the active spinal robot reached 6.0 m / s, a 25% improvement compared to the baseline model's 4.8 m / s. It is worth noting that both results significantly surpass the current highest level achieved in simulations of spinal quadruped robots, namely 4.9 m / s.

[0103] Figure 7 Kinematic analysis of the spinal joints revealed significant asymmetric changes in the pitch joints, ranging from [-0.5, 0.25] rad in IsaacSim and [-0.6, 0.3] rad in MuJoCo. This behavior aligns with biological principles, where spinal flexion during high-speed movement is approximately twice the amplitude of extension. Simultaneously, the roll joints exhibit minimal change, ensuring lateral stability. This bio-inspired asymmetry contrasts sharply with existing model predictive control (MPC) approaches, which rigidly prescribe symmetrical spinal movements (e.g., [-0.42, 0.42] rad), a simplification that contradicts the natural kinematic characteristics of rapid movement.

[0104] 2. Spinal-assisted agile steering 2.1 Steering Capability Assessment To evaluate the improvement in steering performance brought about by the multi-jointed spine, we conducted angular velocity tracking experiments in two simulators. We measured the maximum yaw angular velocity in both the active spine configuration and the rigid torso configuration under two conditions: stationary steering and moving steering (tracking a constant forward velocity of 1 m / s). Figure 5 As shown in (b).

[0105] like Figure 8 As shown, the active spine configuration consistently and significantly outperforms the rigid torso baseline model. During stationary turning, the active spine robot achieved a stable maximum yaw rate of 7.2 rad / s in IsaacSim, a 16.1% improvement over the rigid baseline model; and 5.3 rad / s in MuJoCo, a 60.1% improvement over its rigid control group. In the moving turning task, the performance gap widened further, with the active spine robot achieving 7.8 rad / s in IsaacSim, a 22.9% improvement over the rigid baseline model; and 6.6 rad / s in MuJoCo, a remarkable 83.3% improvement over the rigid torso model. Notably, these yaw rates significantly surpass the previously reported highest level of 3 rad / s for quadruped robots with yaw spine articulation.

[0106] Figure 9 Kinematic analysis of the spinal joints reveals the intrinsic mechanisms driving this agility. During high-speed turns, the spinal yaw joint operates near its mechanical limits to maximize rotational capability. Simultaneously, the spinal roll joint actively adjusts to perform a roll-over maneuver, i.e., tilting towards the inside of the curve. This biomimetic compensation effectively counteracts centrifugal force, significantly enhancing the robot's dynamic stability during rapid changes of direction.

[0107] 2.2 Path tracking test To further explore how a 3-DOF spine enhances mobility, we conducted a figure-eight path tracking task on MuJoCo. The trajectory consisted of two circular arcs (R=1m) connected by intersecting straight line segments (L=4m). During the task, the robot maintained a constant commanded forward velocity v_x, while a proportional controller adjusted the steering by converting heading error into target yaw rate for the strategy to track.

[0108] like Figure 10 As shown in the spatial trajectory of (a), the active spine robot tracks the reference path more accurately than the rigid torso baseline model, achieving smooth turning by utilizing the degrees of freedom in the middle of its torso. A representative example is taken with v_x = 2.0 m / s. Figure 10(b) Quantitative analysis shows that the active spine configuration produces a higher transient yaw rate when entering a curve, thus enabling faster heading adjustments than the rigid body trunk baseline model. Figure 10 (c) The spinal kinematics curve further reveals this coordination: when the robot transitions to a turning state, the spinal yaw angle increases to reduce the turning radius, while the roll angle is adjusted simultaneously to provide roll support.

[0109] 3. Spinal-driven aerial self-righting During training, we observed an emergent capability in the active spinal robot: it was able to achieve a stable quadrupedal landing from a random initial posture during a fall, whereas the rigid body control group frequently experienced torso collisions with the ground in the same task. To systematically evaluate this, we designed a drop experiment from a height of 2m with three extreme initial postures: (pitch angle θ = 90°), (roll angle φ = 135°, pitch angle θ = 45°), and a completely inverted posture (roll angle φ = 180°).

[0110] like Figure 5 As shown in (c), the active spinal robot uses its 3-DOF spinal joints to quickly adjust its body posture during descent. Taking a fully inverted position (φ=180°) as a representative example, the robot coordinates its spinal rotation to readjust the chassis orientation in mid-air, ensuring that the feet are in the optimal ground contact position. This process greatly mimics the mid-air righting reflex of felines. Figure 11 Quantitative analysis confirms that, due to the lack of internal momentum manipulation capabilities, rigid-body trunk robots cannot significantly adjust their roll angle φ, while active spine robots... Figure 12 The coordinated joint angular trajectory shown generates a large roll velocity ω_x to correct its attitude. In contrast, rigid body torso models typically slam into the ground back-on and can only rely entirely on the limbs after landing for attitude recovery.

[0111] In summary, the biomimetic serial 3-DOF active spinal quadruped robot and its control method proposed in this invention, through the organic combination of structural innovation and control method innovation, achieved a maximum running speed of 6.9 m / s and a turning angular velocity of 7.2 rad / s in a simulation environment, which significantly surpasses the performance of the rigid body trunk benchmark model and the current state-of-the-art spinal quadruped robot. It also exhibits complex and agile behaviors such as self-righting in mid-air, fully demonstrating the technical advancement and beneficial effects of this invention.

[0112] In summary, this invention proposes a quadruped robot equipped with a biomimetic serial three-DOF active spine, which can effectively reproduce the three-axis rotational motion of a quadrupedal spine. By employing a dedicated reinforcement learning framework with a custom reward function, the robot achieves a variety of agile movement behaviors, including high-speed running, maneuvering, precise path tracking, and emerging feline-like aerial righting abilities.

[0113] Through comprehensive cross-validation on both IsaacSim and MuJoCo simulators, this study established a highly robust control strategy with minimal transfer differences between simulators. In the simulation environment, the robot, employing a rotational-rotational gait, achieved a maximum running speed of 6.9 m / s and a stationary turning angular velocity of 7.2 rad / s, significantly surpassing the performance of the rigid body baseline model and the current state-of-the-art spinal quadruped robot in simulators. The analysis results fully demonstrate that the introduced three-degree-of-freedom spine significantly enhances the robot's speed, maneuverability, accuracy, and stability.

[0114] Compared with the prior art, the present invention has the following beneficial effects: Complete three-degree-of-freedom spine design: This invention designs a biomimetic three-degree-of-freedom active spine composed of three tandem spinal joints: yaw, pitch, and roll. It can achieve three-axis rotation in space, fully reproduce the flexible movement of the spine of a biological quadruped in three dimensions, and overcome the limitations of existing spinal quadruped robots that only have single or two degrees of freedom and cannot achieve complete spatial rotation.

[0115] Optimized spinal configuration arrangement: The present invention adopts a spinal configuration arrangement of "prioritizing yaw guidance in the front section, concentrating pitch power in the middle section, and taking into account roll compliance in the rear section", which enables the robot to exhibit motion dynamics characteristics that are closer to those of biological quadrupeds.

[0116] Gait reward function: The gait reward function designed in this invention guides the strategy to naturally explore a complete rotational sprint gait through minimum foot placement pattern constraints, including two airborne phases and a distinctive circular foot placement sequence, thus avoiding gait rigidity caused by excessive constraints.

[0117] Spinal undulation reward function: The spinal undulation reward function designed in this invention actively drives the spine to produce phase-coordinated flexion and extension movements, and introduces an asymmetric threshold to faithfully simulate the movement characteristics of the biological spine, effectively avoiding the problem in passive methods where the strategy converges to invalid spinal movements due to reward cheating.

[0118] Spinal steering reward function: The spinal steering reward function designed in this invention utilizes the yaw spinal joint to achieve a steering mechanism similar to front wheel steering, and achieves effective decoupling of forward motion and yaw maneuvering through dead zone threshold.

[0119] Dual-course learning strategy: This invention adopts a dual-course learning strategy, which significantly improves training stability and enables the robot to simultaneously achieve a variety of highly agile movement behaviors such as high-speed running, flexible turning, precise path tracking, and feline-like aerial self-righting within a unified reinforcement learning framework.

[0120] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A quadruped robot with a biomimetic multi-degree-of-freedom spine, characterized in that, It includes an anterior trunk unit (100), a posterior trunk unit (200), four leg assemblies (300), and a spinal module (400) connecting the anterior trunk unit (100) and the posterior trunk unit (200). Each of the leg components (300) has three degrees of freedom; The spinal module (400) is configured with three series-connected rotational degrees of freedom, including a yaw spinal unit (410), a pitch spinal unit (420) and a roll spinal unit (430) connected in series from front to back; the front end of the yaw spinal unit (410) is connected to the anterior trunk unit (100), and the rear end of the roll spinal unit (430) is connected to the posterior trunk unit (200). The joint actuator of the spinal module (400) is configured to have an output torque greater than the peak torque of the joint actuator of the leg assembly (300) to drive the robot to perform high-speed running and turning maneuvers through the active flexion, extension and torsion of the spinal module (400).

2. The quadruped robot according to claim 1, characterized in that, The specific series connection method of the spinal module (400) is as follows: The front end of the yaw spine unit (410) is connected to the rear end face of the forequarter unit (100) via a flange, and its rotation axis is arranged in the vertical direction. The pitching spine unit (420) is connected in series at the rear end of the yaw spine unit (410), and its rotation axis is arranged horizontally. The rolling spine unit (430) is connected in series to the rear end of the pitching spine unit (420), with its rotation axis arranged longitudinally along the front and rear, and the rear end of the rolling spine unit (430) is connected to the front end of the posterior trunk unit (200).

3. The quadruped robot according to claim 1, characterized in that, The yaw angle, pitch angle, and roll angle of the yaw spine unit (410), pitch spine unit (420), and roll spine unit (430) are all set with preset activity range thresholds.

4. A control method for a quadruped robot, applied to a quadruped robot with a biomimetic multi-degree-of-freedom spine as described in any one of claims 1 to 3, characterized in that, include: A reinforcement learning framework is constructed, which includes a policy network and an evaluation network, and is trained using a proximal policy optimization algorithm; The trained policy network is deployed in the controller of the quadruped robot; During the control execution phase, the strategy network outputs the target joint angles of the four leg components and the spine module at a preset frequency. The target joint angles are converted into joint torques by a proportional-derivative position controller to drive the robot's movement. The reinforcement learning framework employs a dual-curricular learning strategy during training, which includes: Adaptive velocity range course: The linear velocity command range and angular velocity command range of the robot are set as dynamically adjustable variables, and the linear velocity command range and angular velocity command range are independently and gradually expanded according to the convergence of the policy network during the training process. Command generator with acceleration constraints: When generating motion commands, the acceleration of the commands is limited to smooth the speed transition, and when switching from linear motion to steering motion, the effective time of the angular velocity command is set with a random delay.

5. The control method for a quadruped robot according to claim 4, characterized in that, The training reward function of the reinforcement learning framework includes gait reward, spinal undulation reward, and spinal turning reward; The gait reward is calculated based on the temporal phase difference of ipsilateral limb ground contact events and is used to guide the robot to learn a rotational sprint gait with two airborne phases. The spinal wave reward includes an amplitude incentive, which determines the flexion or extension target of the spine based on the direction of leg velocity and sets asymmetric flexion angle threshold and extension angle threshold. The spinal steering reward is calculated based on a dead zone threshold mechanism. When the target angular velocity is within the preset dead zone range, the reward value is zero. When the target angular velocity exceeds the preset dead zone range, the consistency score between the spinal yaw angle and the target steering direction is calculated.

6. The control method for a quadruped robot according to claim 5, characterized in that, The method further includes: The system integrates speed tracking rewards, agile motion rewards for achieving high-speed driving, flexible turning, and precise path tracking, and combines them with auxiliary regularization penalties to form a complete reward function system. The policy network is iteratively trained using the reinforcement learning framework and the complete reward function system to drive the quadruped robot to learn and achieve the agile movement behavior.

7. The control method according to claim 5, characterized in that, The calculation process for the gait reward includes: For ipsilateral limb pairs, calculate the actual time phase difference of their ground contact events. Phase difference with target The shortest phase distance Δδ between them; Calculate the reward value of a unilateral limb pair using the Gaussian kernel function. : ,in, This refers to the phase tolerance coefficient. Total gait reward Forelimbs responding to rewards Rewards for hind limbs The product of: .

8. The control method according to claim 5, characterized in that, The spinal wave reward is used to guide the flexion and extension movements of the spinal joints to produce large amplitude and phase coordination; the spinal wave reward The calculation process includes: Based on effective leg speed With spinal angular velocity Product calculation phase coordination reward The formula is as follows: in, This is the sensitivity coefficient. It is determined by the average of the hind limb thigh velocity and the forelimb thigh velocity after sign reversal; Based on spinal state s and spinal joint angle Calculate amplitude reward and to Threshold truncation is performed to obtain ; like Exceeding the threshold Then calculate the over-limit penalty term. The formula is as follows: in, Weighting for exceeding the limit penalty; The final spinal undulation reward is calculated by combining the above factors: in, Weights are added to the amplitude.

9. The control method according to claim 8, characterized in that, The amplitude excitation calculation steps specifically include: Based on the effective leg speed The symbol determines the desired spinal state. And calculate the basic amplitude bonus. ; right Threshold truncation is performed to obtain : , wherein, according to the spinal condition Dynamically adjust the expected threshold When the spine is in a flexed position, the flexion threshold is used. When the spine is in an extended state, the extension threshold is used. ,and > .

10. The control method according to claim 5, characterized in that, The spinal steering reward is used to incentivize the yaw spinal joint to yaw in a direction consistent with the commanded angular velocity; The spinal turning reward A piecewise function is used, and a dead zone threshold is introduced. To distinguish between linear motion and turning motion; When the absolute value of the command angular velocity Less than the dead zone threshold At that time, the spinal rotation reward is zero; When the absolute value of the command angular velocity Greater than or equal to the dead zone threshold At that time, the spinal turning reward is calculated according to the following formula: ,in, This represents the actual deflection angle of the yaw spine joint. ω is the command angular velocity, and k is the scaling factor for controlling the reward saturation velocity.

11. The control method according to claim 4, characterized in that, The specific implementation steps of the adaptive speed range course include: Set up linear velocity courses and angular velocity courses, where the upper limit of the target range for the linear velocity course is a first preset value, and the upper limit of the target range for the angular velocity course is a second preset value; In the early stages of training, the linear velocity command range is limited to a first preset range, and the angular velocity command range is limited to a second preset range. The monitoring strategy network's average reward value over the most recent N training rounds is used. When the average round tracking reward of the linear velocity course exceeds 80% of a preset threshold, the upper and lower limits of the linear velocity command range are increased by a preset step size Δ=0.

1. When the average round tracking reward of the angular velocity course exceeds 60% of a preset threshold, the upper and lower limits of the angular velocity command range are increased by a preset step size Δ=0.1, until the target range is reached respectively.

12. The control method according to claim 11, characterized in that, When executing step S2, the instruction generator with acceleration constraints uses the following formula to smooth the instruction: ， in, The smoothed instruction velocity at time t+1 Let be the command speed at time t. For the target speed, The preset maximum acceleration limit value, To control the time step.

Images