Method and device for fall recovery of rope-driven differential omni-directional wheeled humanoid robot

Abstract

This disclosure provides a fall recovery method and apparatus for a tethered differential omnidirectional wheeled humanoid robot, relating to the field of control technology. The method includes: inputting the current angle, angular velocity, and current of the motors into a forward mapping network to obtain joint state estimates; inputting the joint state estimates, the current posture and angular velocity of the base, and the robot's current contact state into a reinforcement learning policy network for policy reasoning to obtain joint space action commands; inputting the joint space action commands into a reverse mapping network to obtain motor space commands; and controlling the motors to execute fall recovery actions based on the motor space commands. This disclosure absorbs the non-ideal characteristics of the tethered differential control interface layer through the forward and reverse mapping networks, enabling the policy to learn and think in joint space. After deployment, the learned mappings correctly drive the motors, improving the accuracy and smoothness of the robot's recovery actions.

Description

Technical Field

[0001] This disclosure relates to the field of robot control technology, and more specifically, to a method and apparatus for self-recovery after a fall in a rope-driven differential omnidirectional wheeled humanoid robot. Background Technology

[0002] In simulations, a rope-driven differential omnidirectional wheeled humanoid robot can only be modeled and its joints controlled, while in a live robot, only the motor encoders can be observed and the motors controlled. The differential mapping between the two is theoretically linear, but in practice, it is highly nonlinear due to rope elasticity, friction, and slack.

[0003] This results in technical issues such as insufficient precision and smoothness in the movements of the rope-driven differential omnidirectional wheeled humanoid robot during the fall recovery process, leading to shaking and deviation. Summary of the Invention

[0004] This disclosure provides a method and apparatus for self-recovery after a fall in a rope-driven differential omnidirectional wheeled humanoid robot, which is used to solve the above-mentioned technical problems.

[0005] In a first aspect, this disclosure provides a fall recovery method for a rope-driven differential omnidirectional wheeled humanoid robot, comprising: Obtain the current angle, angular velocity, and current of the motor; The current angle, angular velocity, and current of the motor are input into the forward mapping network to obtain the joint state estimate output by the forward mapping network; The joint state estimate, the current posture and angular velocity of the base, and the current contact state of the robot are input into the reinforcement learning policy network for policy reasoning to obtain the joint space action command output by the reinforcement learning policy network. The joint space motion command is input into the inverse mapping network to obtain the motor space command output by the inverse mapping network; Based on the aforementioned motor space command, the motor is controlled to perform a fall recovery action.

[0006] In some embodiments, the method further includes: The reinforcement learning policy network is pre-trained using simulation. The steps for performing simulation pre-training on the reinforcement learning policy network are as follows: Construct a contact state space and a contact state transition diagram, and build a robot simulation model based on a rigid body dynamics simulator in joint space; Reinforcement learning strategy training is performed based on the contact state transition graph. An intermediate reward is given for each completed contact state transition step until the strategy success rate is higher than the first threshold and the current gravity value reaches the standard gravity value. At this point, reinforcement learning strategy training stops, and pre-trained strategy parameters are obtained.

[0007] In some embodiments, domain randomization perturbations of the rope-driven differential transmission characteristics are injected during the training of reinforcement learning policies.

[0008] In some embodiments, the domain randomization perturbation of the rope-driven differential transmission characteristics includes at least one of the following: Apply asymmetric torque constraints to the joint; A low-pass filter with a random time constant is inserted between the joint command and the execution torque; Noise proportional to the coupled second joint angular velocity is injected into the observed value of the first joint angular velocity; Inject nonlinear noise related to joint angle and angular velocity into the joint angle observations; Apply a random delay to joint space motion commands.

[0009] In some embodiments, during the reinforcement learning policy training process, whenever the policy success rate under the current gravity is higher than the first threshold, the gravity value is increased; whenever the policy success rate under the current gravity is lower than the second threshold, the gravity value is decreased.

[0010] In some embodiments, the intermediate reward includes the total prior reward for stage-adaptive morphological decoupling motion; The phased adaptive morphological decoupling motion prior total reward includes: style reward and coordination reward; The style reward is determined based on an adversarial discriminator trained on human standing motion capture data; The coordinated reward is determined based on an adversarial discriminator trained under self-supervised conditions using simulated successful trajectories.

[0011] In some embodiments, the weight of the style reward differs depending on the contact state; The smaller the graph distance between the contact state and the target state, the greater the weight of the style reward; The greater the graph distance between the contact state and the target state, the smaller the weight of the style reward.

[0012] In some embodiments, the intermediate reward includes positive rewards and negative rewards; A positive reward is given when the graph distance between the contact state and the target state decreases. An increase in the graph distance between the contact state and the target state results in a negative reward.

[0013] In some embodiments, the method further includes: The forward mapping network and the reverse mapping network are trained; The steps for training the forward mapping network and the reverse mapping network are as follows: Deploy the pre-trained strategy parameters on a real device and collect training data; The training data is used to train the forward mapping network and the reverse mapping network.

[0014] In some embodiments, the method further includes: The system identifies the physical parameters of the simulation based on the state error between the simulated trajectory and the actual machine trajectory, and updates the parameters of the robot simulation model.

[0015] In some embodiments, the method further includes: The learned forward mapping network and the backward mapping network are integrated into the simulation training loop, and interface mapping error is injected. Load the pre-trained strategy parameters; Continue training with the reinforcement learning strategy to obtain the fine-tuned policy parameters.

[0016] In some embodiments, the method further includes: The residual dynamics network is trained using the training data; The state transition is corrected through the residual dynamics network.

[0017] In some embodiments, the contact state space includes at least one of the following states: forward-lying state, backward-lying state, side-flipping state, overturning state, half-squatting state, and standing state.

[0018] Secondly, this disclosure provides a fall recovery device for a rope-driven differential omnidirectional wheeled humanoid robot, comprising: The acquisition module is used to acquire the current angle, angular velocity, and current of the motor. The forward mapping module is used to input the current angle, angular velocity and current of the motor into the forward mapping network to obtain the joint state estimate output by the forward mapping network; The reasoning module is used to input the joint state estimate, the current posture and angular velocity of the base, and the current contact state of the robot into the reinforcement learning policy network for policy reasoning, and obtain the joint space action command output by the reinforcement learning policy network. The reverse mapping module is used to input the joint space motion command into the reverse mapping network and obtain the motor space command output by the reverse mapping network; The control module is used to control the motor to perform fall recovery actions based on the motor space commands.

[0019] In some embodiments, the apparatus further includes a simulation pre-training module; The simulation pre-training module is used to perform simulation pre-training on the reinforcement learning policy network. The steps for performing simulation pre-training on the reinforcement learning policy network are as follows: Construct a contact state space and a contact state transition diagram, and build a robot simulation model based on a rigid body dynamics simulator in joint space; Reinforcement learning strategy training is performed based on the contact state transition graph. An intermediate reward is given for each completed contact state transition step until the strategy success rate is higher than the first threshold and the current gravity value reaches the standard gravity value. At this point, reinforcement learning strategy training stops, and pre-trained strategy parameters are obtained.

[0020] In some embodiments, domain randomization perturbations of the rope-driven differential transmission characteristics are injected during the training of reinforcement learning policies.

[0021] In some embodiments, the domain randomization perturbation of the rope-driven differential transmission characteristics includes at least one of the following: Apply asymmetric torque constraints to the joint; A low-pass filter with a random time constant is inserted between the joint command and the execution torque; Noise proportional to the coupled second joint angular velocity is injected into the observed value of the first joint angular velocity; Inject nonlinear noise related to joint angle and angular velocity into the joint angle observations; Apply a random delay to joint space motion commands.

[0022] In some embodiments, during the reinforcement learning policy training process, whenever the policy success rate under the current gravity is higher than the first threshold, the gravity value is increased; whenever the policy success rate under the current gravity is lower than the second threshold, the gravity value is decreased.

[0023] In some embodiments, the intermediate reward includes the total prior reward for stage-adaptive morphological decoupling motion; The phased adaptive morphological decoupling motion prior total reward includes: style reward and coordination reward; The style reward is determined based on an adversarial discriminator trained on human standing motion capture data; The coordinated reward is determined based on an adversarial discriminator trained under self-supervised conditions using simulated successful trajectories.

[0024] In some embodiments, the weight of the style reward differs depending on the contact state; The smaller the graph distance between the contact state and the target state, the greater the weight of the style reward; The greater the graph distance between the contact state and the target state, the smaller the weight of the style reward.

[0025] In some embodiments, the intermediate reward includes positive rewards and negative rewards; A positive reward is given when the graph distance between the contact state and the target state decreases. An increase in the graph distance between the contact state and the target state results in a negative reward.

[0026] In some embodiments, the apparatus further includes an interface mapping network training module; The interface mapping network training module is used to train the forward mapping network and the reverse mapping network; The steps for training the forward mapping network and the reverse mapping network are as follows: Deploy the pre-trained strategy parameters on a real device and collect training data; The training data is used to train the forward mapping network and the reverse mapping network.

[0027] In some embodiments, the apparatus further includes a model parameter update module; The model parameter update module is used to identify the system physical parameters of the simulation based on the state error between the simulation trajectory and the real machine trajectory, and update the parameters of the robot simulation model.

[0028] In some embodiments, the apparatus further includes a policy network fine-tuning module; The policy network fine-tuning module is used to integrate the learned forward mapping network and the reverse mapping network into the simulation training loop, and inject interface mapping error; Load the pre-trained strategy parameters; Continue training with the reinforcement learning strategy to obtain the fine-tuned policy parameters.

[0029] In some embodiments, the policy network fine-tuning module is further configured to: The residual dynamics network is trained using the training data; The state transition is corrected through the residual dynamics network.

[0030] In some embodiments, the contact state space includes at least one of the following states: forward-lying state, backward-lying state, side-flipping state, overturning state, half-squatting state, and standing state.

[0031] Thirdly, this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0032] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0033] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0034] The method and apparatus for fall recovery of a rope-driven differential omnidirectional wheeled humanoid robot disclosed herein absorbs the non-ideal characteristics of the control interface layer of the rope-driven differential system through forward and reverse mapping networks, enabling the strategy to learn and think in the joint space. After deployment, the learned mapping is used to correctly drive the motor, thereby improving the accuracy and smoothness of the robot's recovery action. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0036] Figure 1 This is a flowchart illustrating the fall recovery method for a rope-driven differential omnidirectional wheeled humanoid robot provided in an embodiment of this disclosure.

[0037] Figure 2 This is a schematic diagram of the contact state provided in an embodiment of this disclosure.

[0038] Figure 3 A schematic diagram of a phase-adaptive morphological decoupling motion prior reward function provided in an embodiment of this disclosure.

[0039] Figure 4 This is a schematic diagram of the overall logic flow provided for embodiments of this disclosure.

[0040] Figure 5 This is a schematic diagram of the simulation pre-training logic flow provided in the embodiments of this disclosure.

[0041] Figure 6 This is a schematic diagram of the logic flow for real device data collection and interface mapping learning provided in the embodiments of this disclosure.

[0042] Figure 7 A schematic diagram of the logic flow for fine-tuning reinforcement learning strategies provided in the embodiments of this disclosure.

[0043] Figure 8 This is a schematic diagram of the real device deployment and online execution logic flow provided in the embodiments of this disclosure.

[0044] Figure 9 This is a schematic diagram of the structure of the self-recovery device for a rope-driven differential omnidirectional wheeled humanoid robot that has fallen, as provided in an embodiment of this disclosure.

[0045] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0046] The embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions of the embodiments of this disclosure.

[0047] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this disclosure mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element are connected through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term, for example, “A and / or B” or “A, B” indicates implementation as “A,” or implementation as “B,” or implementation as “A and B.”

[0048] The prefixes "first," "second," etc., used in the embodiments of this disclosure are merely for distinguishing different descriptive objects and do not impose restrictions on the position, order, priority, quantity, or content of the descriptive objects. The description of the descriptive objects is found in the claims or the context of the embodiments, and the use of prefixes should not constitute unnecessary restrictions. For example, if the descriptive object is a port of an element and / or component, abbreviated as "end," then the ordinal number before "end" in "first end" and "second end" does not restrict the position or order of the "ends." "First" and "second" do not restrict whether the "ends" they modify are adjacent, nor do they restrict the order of "first end" and "second end." Similarly, if the descriptive object is a "level," then the ordinal number before "level" in "first level" and "second level" does not restrict the priority between "levels." Furthermore, the number of descriptive objects is not limited by ordinal numbers and can be one or more. For example, in "first device," the number of "devices" can be one or more. Furthermore, the objects modified by different prefixes can be the same or different. For example, if the object being described is "device", then "first device" and "second device" can be the same device or different devices, and their types can be the same or different. Similarly, if the object being described is "information", then "first information" and "second information" can be the same information or different information, and their content can be the same or different.

[0049] In this embodiment of the disclosure, "determine B based on A" means that factor A must be considered when determining B. It is not limited to "B can be determined based solely on A," but should also include: "determine B based on A and C," "determine B based on A, C, and E," "determine C based on A, and further determine B based on C," etc. Additionally, it can include using A as a condition for determining B, for example, "when A meets the first condition, determine B using the first method"; or "when A meets the second condition, determine B," etc.; or "when A meets the third condition, determine B based on the first parameter," etc. Of course, it can also be a condition where A is a factor in determining B, for example, "when A meets the first condition, determine C using the first method, and further determine B based on C," etc.

[0050] Figure 1 This is a flowchart illustrating the fall recovery method for a rope-driven differential omnidirectional wheeled humanoid robot provided in an embodiment of this disclosure. Figure 1 As shown, during the actual deployment and online execution phases, the fall self-recovery method for the rope-driven differential omnidirectional wheeled humanoid robot may include the following steps: Step 101: Obtain the current angle, angular velocity, and current of the motor; Step 102: Input the current angle, angular velocity and current of the motor into the forward mapping network to obtain the joint state estimate output by the forward mapping network; Step 103: Input the estimated joint state, the current posture and angular velocity of the base, and the current contact state of the robot into the reinforcement learning policy network for policy reasoning to obtain the joint space action command output by the reinforcement learning policy network. Step 104: Input the joint space motion command into the inverse mapping network to obtain the motor space command output by the inverse mapping network; Step 105: Control the motor to perform a fall recovery action based on the motor space command.

[0051] Specifically, the Sim-to-Real method (e.g., ASAP) employs a strategy combining system identification and residual dynamics learning to correct the dynamic discrepancies between simulation and reality. The specific process is as follows: (1) Execute the excitation trajectory on the real machine and collect state transition data; (2) Optimize the physical parameters of the simulator (e.g., mass, inertia, friction coefficient) through system identification; (3) Train the residual dynamics network, inputting the current state and action, and outputting the state transition residuals between the simulation prediction and the actual observation; (4) Fine-tune the strategy in the corrected simulation environment. This method assumes that the main differences between simulation and reality stem from inaccurate physical parameters and unmodeled dynamic effects, and that parameter optimization and residual compensation can effectively reduce these discrepancies.

[0052] The rope-driven differential omnidirectional wheeled humanoid robot system has two types of coupled simulation-reality differences: (1) dynamic gap: the difference between simulation physics and real physics; (2) control interface gap: the simulation controls the joints, while the real robot controls the motors. In the simulation, only the joints can be modeled and controlled, while the real robot can only observe the motor encoder and control the motor. The differential mapping between the two is theoretically linear, but in reality it is highly nonlinear due to rope elasticity / friction / slack, and the differential mapping between the two is not ideal. The Sim-to-Real method corrects the difference between simulation physics and real physics through system identification and residual dynamics network. It assumes that the simulation and reality use the same control interface (both control the joints), only corrects the simulation dynamics, and does not deal with this control interface level deviation (Gap). When this method is applied to the rope-driven differential omnidirectional wheeled humanoid robot, the robot's movements are not precise and smooth enough during the fall recovery process, and there are technical problems of shaking and deviation.

[0053] In the Sim-to-Real method, the residual dynamics network takes the current state and action as input and outputs the state transition residual, but does not explicitly model the configuration dependency of the tethered system. It corrects the difference between simulated and real state transitions. For example, residual dynamics methods (such as ASAP) train neural networks... The prediction targets dynamic residuals that the simulation physics engine fails to accurately model, and the correction target is the state transition function. = ( , )+ ( , ),in, Indicates the state at the next moment. ( , () indicates the state of the simulation engine's calculations. ( , ) represents a neural network The output state is used to compensate for distortion during state transitions in the simulation. Indicates the current state. This represents the action at the current moment. The dynamic gap of the tethered differential is small in normal postures, but increases significantly in extreme postures (such as fall recovery), and varies with joint angles and load direction. Residual networks with fixed structures struggle to capture this strong configuration dependence.

[0054] The Sim-to-Real method estimates joint states using only the motor encoder angle and angular velocity, without utilizing the motor current signal. Motor current indirectly reflects rope tension and load conditions; under heavy loads and extreme configurations, pure kinematic calculations (ideal difference formulas) have significant errors, while the current signal can provide additional mechanical information to correct the estimate.

[0055] To address the aforementioned technical problems, embodiments of this disclosure insert a pair of learnable bidirectional mapping networks (forward mapping network and backward mapping network) between the strategy and the real robot, learning from real machine data to replace ideal difference kinematics.

[0056] In some embodiments, forward mapping networks Motor-side observables (e.g., motor encoder angles, angular velocities, motor current) can be mapped to joint state estimates. Compared to ideal differential positive kinematics, this network additionally utilizes motor current information—current indirectly reflecting rope tension and load conditions—providing more accurate joint state estimates than pure kinematic calculations, especially under heavy loads and extreme configurations.

[0057] In some embodiments, the tethered differential omnidirectional wheeled humanoid robot employs differential joints, with two motors coupling to control two joints. The input to the forward mapping network may include the encoder angle of motor A. Motor B encoder angle Angular velocity of motor A Angular velocity of motor B Motor A current Motor B current A 6-dimensional vector. The output is the angle of joint 1. 2. Joint angle Angular velocity of joint 1 Angular velocity of joint 2 It is a 4-dimensional vector.

[0058] In some embodiments, forward mapping networks The network structure can be a 3-layer fully connected neural network with hidden layer dimensions of [128, 128] and activation function ELU. In this embodiment, by introducing a current signal, when the joint is under a large load, the current of the driving motor increases significantly, while the current of the driven motor may decrease or even reverse (due to rope slack), resulting in a current difference. - This directly reflects the load asymmetry of the differential mechanism, and the forward mapping network Use this information to revise the ideal kinematics formula =( + The deviation caused by the elasticity of the rope in () / (2r) is where r represents the conversion coefficient from motor motion to joint motion.

[0059] Additional input motor current to the forward mapping network , By using current signals to correct joint state estimation, accuracy is significantly improved in high-load scenarios such as fall recovery.

[0060] In some embodiments, reverse mapping networks Joint space commands can be mapped to motor space commands. Input can include the target joint position (joint position can be replaced with joint angles) and an estimate of the current joint state (from...). This makes the mapping configuration-dependent—adaptively adjusting the mapping relationship under different configurations.

[0061] In some embodiments, the input to the inverse mapping network may include the target joint 1 angle. Target joint 2 angles Current joint 1 angle estimation (from) Current joint 2 angle estimation It is a 4-dimensional vector. The output is the target angle of motor A. Target angle of motor B A 2-dimensional vector. Inverse mapping network. The network structure can be a 3-layer fully connected network with hidden layer dimensions [128, 128]. Configuration dependency is implemented by inverse mapping the network. It not only learns the mapping from target joint angles to motor angles, but also the modulation effect of the current configuration (current joint angles) on that mapping. For example, when the joint approaches its motion limit, the cable tension distribution becomes highly nonlinear, and the ideal inverse kinematics formula... =r( + This will produce a large error; inverse mapping network By learning the current configuration , With target configuration , The relationship between them adaptively compensates for this error.

[0062] The input to the inverse mapping network includes the current joint state estimate, making the mapping configuration-dependent and adaptively adjusting the mapping relationship under different configurations.

[0063] This disclosure proposes a learnable bidirectional interface mapping (LBM) to train a forward mapping network (motor encoder + current → joint state estimation) and a reverse mapping network (joint command → motor command), and to use the motor current signal to correct the nonlinear characteristics of the rope drive differential.

[0064] This disclosure addresses the differences between the policy observation / execution interface and the actual sensors / actuators, additionally applying conventional domain randomization to fill the remaining dynamic gap. The interface mapping corrects the observation function. = ( ) and execution function = ( , The two act at different levels, among which, This represents the sensor's predicted value at the current moment. This includes motor encoders, etc. For cable-driven differential systems, interface mapping is particularly critical because even if the simulated physics is perfectly accurate, the action output by the strategy in joint space will differ. It cannot be directly converted into motor commands—it must go through a nonlinear, stateful, and hysteretic differential mapping.

[0065] This embodiment learns the mapping through a data-driven approach, avoiding the complexity of manually modeling rope dynamics. It simultaneously addresses the dynamic gap and the control interface gap, absorbing all non-ideal characteristics of the control interface layers of the rope-driven differential model through a bidirectional mapping network. This allows the strategy to learn and think in joint space, and during deployment, it correctly drives the motors using the learned mapping, improving the accuracy and smoothness of the robot's motion recovery.

[0066] In some embodiments, the method further includes: The reinforcement learning policy network is pre-trained using simulation. The steps for performing simulation pre-training on the reinforcement learning policy network are as follows: Construct a contact state space and a contact state transition diagram, and build a robot simulation model based on a rigid body dynamics simulator in joint space; Reinforcement learning strategy training is performed based on the contact state transition graph. An intermediate reward is given for each completed contact state transition step until the strategy success rate is higher than the first threshold and the current gravity value reaches the standard gravity value. At this point, reinforcement learning strategy training stops, and pre-trained strategy parameters are obtained.

[0067] Specifically, to address the fall recovery problem of bipedal humanoid robots, reinforcement learning can be used to train the robot to stand up from a fall, employing a terrain-based learning strategy: initially using flat terrain, gradually increasing the difficulty (e.g., slopes, steps, uneven surfaces) to guide the strategy in learning robust standing movements. The core assumption of this method is that the robot has four limbs (two legs and two arms), and after a fall, these limbs can still contact the ground and exert force, allowing it to directly attempt to stand up. Sparse rewards are defined during training (positive rewards are given for successful standing), and effective standing sequences are explored through extensive trial and error. After a fall, a bipedal humanoid robot can still exert force with its limbs, and its contact state space includes different combinations of hands and feet (e.g., hands on the ground, one hand and one foot on the ground, etc.). However, wheeled humanoid robots, after tipping over / overturning, have their wheels suspended in the air, completely losing their driving ability, and must go through a multi-stage process of righting, wheels touching the ground, and standing up. In the forward-leaning and overturned states, the wheels are completely unusable; in the backward-leaning state, all four wheels can be used for auxiliary force exertion; and in the side-flipping state, some wheels can be used for fine-tuning their position. This contact state space is fundamentally different from that of bipedal robots, making bipedal methods unsuitable for direct application. Terrain learning (flat surfaces, slopes, steps) used for fall recovery in bipedal humanoid robots is ineffective for righting wheeled chassis, as rewards are extremely sparse (only a full standing up provides a reward), and terrain learning cannot provide effective guidance. Bipedal humanoid robot fall recovery assumes joints are driven by ideal actuators, allowing simulation-based joint control to be directly mapped to the actual robot. However, in a rope-driven differential omnidirectional wheeled humanoid robot system, simulation controls joints while the actual robot controls motors. The differential mapping between the two is highly nonlinear due to rope elasticity / friction / slack, making the ideal differential kinematics formula... =( + ) / (2r), =( The error of ) / (2r) is significant under extreme configurations and heavy loads. The bipedal approach does not address the simulation-reality difference at this control interface level.

[0068] For fall recovery in wheeled quadruped robots, a hierarchical control strategy based on contact states can be adopted. This method decomposes the recovery process into multiple sub-tasks: identifying the current contact state (which legs are on the ground), planning the recovery path (the sequence of contact states from the current state to the standing state), and executing local control (joint control in each contact state). The contact state is defined based on the contact combination of the four legs with the ground, and the recovery process is realized through state machine switching. This method assumes that the robot has four legs, each of which can independently contact the ground and provide support. The contact states of wheeled quadruped robots are based on different combinations of the four legs (such as three-legged support, diagonal two-legged support, etc.), and each leg can independently contact the ground and provide support. However, the contact states of tethered differential omnidirectional wheeled humanoid robots are postures such as forward / backward / side-over / tumbling, involving complex combinations of hands, torso, and wheels, and the wheels cannot provide any support when suspended in the air. The contact state modes of the two are fundamentally different, and the state definition and transition logic of the quadruped method cannot be directly transferred. The recovery movements of wheeled quadruped robots do not need to be anthropomorphic; they only need to functionally stand up. Wheeled quadruped robots do not involve the learning and optimization of movement styles.

[0069] In the field of tethered differential omnidirectional wheeled humanoid robots, righting and standing up of the wheeled chassis presents a novel and extremely challenging problem. After the chassis tilts or overturns, the wheels are suspended in the air and must undergo a multi-stage process, including righting, wheel landing, and standing up. Traditional curriculum learning (progressing according to posture difficulty) cannot effectively guide the exploration of this process.

[0070] This disclosure, taking into account the special form of wheeled humanoid robots, redefines the contact state space and transition diagram, clearly distinguishing different stages where wheels are available / unavailable.

[0071] This disclosure provides a Contact State Graph-Guided Gravity Progressive Course (CSG-GC) learning method. A contact state space and state transition graph specific to wheeled humanoid robots are defined. Contact states are defined by combinations of contact between the wheels, hands, torso, etc., and the ground. All possible contact states are organized into a directed graph, where paths represent feasible recovery routes from a fall to standing. A reward shaping based on the contact state graph is designed. An intermediate reward is given after each contact state transition (towards the target direction). This solves the core problem of sparse rewards in fall recovery—rewards are not only given after fully standing, but each step (e.g., more wheel contact, changing from full-body contact to hand contact) provides immediate guidance.

[0072] In some embodiments, Figure 2 This is a schematic diagram of the contact state diagram provided in the embodiments of this disclosure, such as... Figure 2As shown, the contact state diagram includes at least one of the following key state nodes: forward-leaning state (all four wheels off the ground, torso / hands on the ground), backward-leaning state (all four wheels on the ground, torso leaning backward), side-rolling state (partially on the ground, fuselage lying on its side), overturned state (upside down, all four wheels off the ground), semi-squatting state (hands off the ground, legs bent, all four wheels on the ground), and target / standing state (all four wheels on the ground, torso upright). State transition conditions are determined based on contact sensor signals and inertial measurement unit (IMU) attitude angles. Figure 2 In this context, r1, r2, and r3 represent the reward values ​​obtained from the contact state transition. The values ​​of r1, r2, and r3 can be the same or different.

[0073] This disclosure proposes a Contact State Graph (CSG) to explicitly define the contact state space unique to wheeled humanoid robots, including the contact combinations of wheels, hands, torso, and the ground, and organizes them into a directed graph to guide recovery path planning.

[0074] In some embodiments, the intermediate reward includes positive rewards and negative rewards; A positive reward is given when the graph distance between the contact state and the target state decreases. An increase in the graph distance between the contact state and the target state results in a negative reward.

[0075] Specifically, the reward shaping function provides decreasing intermediate rewards based on the graph distance (shortest path length) from the current state to the target state, and a positive reward for each successful transition to a state with a smaller graph distance. For example, the positive reward is denoted as... , = • (Reduction in distance), of which, This is the shaping weight coefficient.

[0076] In some embodiments, during the reinforcement learning policy training process, whenever the policy success rate under the current gravity is higher than the first threshold, the gravity value is increased; whenever the policy success rate under the current gravity is lower than the second threshold, the gravity value is decreased.

[0077] Specifically, gravity is used as a parameter in the curriculum. Under low gravity, flipping the chassis is extremely easy (it flips with a gentle push), and the strategy can quickly learn the basic righting sequence. As gravity gradually returns to normal, the strategy is forced to optimize force efficiency and coordination. For example, the initial gravity is set to... =0.3· (30% of Earth's gravity), whenever the success rate of the strategy under the current gravity reaches a threshold. When the gravity reaches 80%, the increase in gravity is Δg = 0.1. until reached In low-gravity environments, the torque required for a robot to right itself is significantly reduced, and the strategy can quickly explore the basic sequence of movements: arm pushing the ground → torso twisting → chassis flipping. As gravity increases, the strategy must learn more precise timing of force application, greater joint torque output, and better body coordination patterns to avoid failure to right itself due to insufficient strength or a second fall due to excessive force.

[0078] This disclosure proposes a gravity-based progressive learning curriculum, which makes it extremely easy to right the chassis under low gravity, allowing the strategy to quickly learn the basic righting sequence. As gravity returns to normal, the strategy is forced to optimize its force application efficiency. Simultaneously, it incorporates contact state graph shaping rewards, providing intermediate rewards for each completed contact state transition (e.g., more wheels touching the ground), transforming sparse rewards into dense guidance.

[0079] In this embodiment, the contact state diagram tells the strategy which way to go, and the gravity course allows the strategy to learn from simpler ones. Compared to traditional posture difficulty courses (gradually transitioning from simple postures such as small-angle side rolls to difficult postures such as complete rollovers), the gravity course in this embodiment reduces the difficulty simultaneously in all postures, enabling the strategy to explore recovery paths for multiple fall postures in parallel, significantly improving training efficiency.

[0080] In some embodiments, the intermediate reward includes the total prior reward for stage-adaptive morphological decoupling motion; The phased adaptive morphological decoupling motion prior total reward includes: style reward and coordination reward; The style reward is determined based on an adversarial discriminator trained on human standing motion capture data; The coordinated reward is determined based on an adversarial discriminator trained under self-supervised conditions using simulated successful trajectories.

[0081] Specifically, the Standard Adversarial Motion Prior (AMP) method achieves anthropomorphic motion by mimicking human reference movements. This method trains an adversarial discriminator to distinguish between policy-generated trajectories and human motion capture data. The discriminator's output serves as an additional reward signal to guide policy learning. The training objective is to jointly optimize task rewards and style rewards. = + · ,in The weights are fixed. This method assumes that the reference motion is completely consistent with the robot's shape, and the discriminator acts on the state of all joints of the robot.

[0082] The standard AMP method uses a single discriminator to evaluate whether the robot's overall movement conforms to reference data. The discriminator input includes all joint states with fixed weights, requiring the robot's overall shape to perfectly match the reference data. However, the tethered differential omnidirectional wheeled humanoid robot has a hybrid form (humanoid upper body, wheeled lower body). The human standing reference data includes unmappable lower limb movements (kneeling, pushing off the ground, etc.). The standard AMP method cannot handle this form mismatch, and the anthropomorphic priority differs at different stages of standing (function priority when righting, posture priority after standing), making direct mapping to the wheeled chassis impossible. Using the entire human body as a reference would create physically infeasible constraints.

[0083] The standard AMP method uses fixed style reward weights. The humanoid posture remains constant throughout the mission. However, in the fall recovery task of the rope-driven differential omnidirectional wheeled humanoid robot, the intensity of the humanoid requirement varies at different stages: the righting stage requires functional priority (allowing non-human but effective movements), while the final standing stage requires an elegant posture. Fixed weights cannot balance this contradiction—too high a weight will hinder righting exploration, while too low a weight will lead to an unnatural final posture.

[0084] Figure 3 A schematic diagram of the phase-adaptive morphological decoupling motion prior reward function provided in an embodiment of this disclosure. (See diagram below.) Figure 3 As shown, this disclosure provides a phase-adaptive morphology-decoupled motion prior (PA-MDMP) method.

[0085] In some embodiments, the motion prior is decomposed into two independent parts: 1. Upper body style prior: The adversarial discriminator only acts on the joint status of the upper body (arms / waist / torso) and the base posture, and is trained using data captured from human standing up movements.

[0086] In some embodiments, the upper body style discriminator The input consists of upper body joint angle sequences (e.g., 3 degrees of freedom for the left shoulder, 3 degrees of freedom for the right shoulder, 1 degree of freedom for the left elbow, 1 degree of freedom for the right elbow, and 2 degrees of freedom for the waist, totaling 10 dimensions) and base postures (roll, pitch, and yaw, totaling 3 dimensions). The time window length is 1 second (50 frames, control frequency 50Hz), and the input dimension is 13×50. The discriminator structure is a 1D convolutional neural network and fully connected layers. The output is a scalar score indicating whether the action segment comes from human reference data. The training data comes from motion capture data of humans standing up from the ground, extracting only the upper body joints and trunk postures, ignoring the lower limb joints. The adversarial training objective is: the upper body movements generated by the strategy should be indistinguishable from human movements by the discriminator.

[0087] II. Full-body coordination prior: This system does not rely on human data but is self-supervised trained from the trajectories of successful standing up in simulations. The discriminator evaluates the coordination between changes in upper body movement and changes in the base state; that is, whether the base response is consistent with successful recovery when the upper body performs a certain movement.

[0088] In some embodiments, the whole-body coordination discriminator The input consists of a 10-dimensional sequence of upper body joint angular velocities and a 6-dimensional sequence of base linear / angular velocities, with a time window of 1 second and an input dimension of 16×50. The discriminator evaluates causal coordination: when the upper body performs a certain action (e.g., the right arm pushes forward), the base should produce a corresponding motion response (e.g., rotating to the left or lifting upward). Training data comes from the trajectory of successful standing up in simulation (episodes where the task reward is greater than the threshold). The discriminator learns the temporal correlation pattern between upper body actions and base responses in successful trajectories. Unlike the upper body style discriminator, the coordination discriminator does not focus on whether the action is anthropomorphic, but rather on whether the action is physically reasonable and effective. For example, if the policy causes the robot to push the ground with both arms simultaneously to the same side, the base will rotate unbalancedly, and the coordination discriminator will give a low score, guiding the policy to learn symmetrical or alternating pushing patterns.

[0089] In this embodiment, the upper body style a priori ensures the anthropomorphism of the movements, while the whole body coordination a priori ensures the rationality of the movements (appropriate coordination between the upper and lower body). The two complement each other.

[0090] In some embodiments, the weight of the style reward differs depending on the contact state; The smaller the graph distance between the contact state and the target state, the greater the weight of the style reward; The greater the graph distance between the contact state and the target state, the smaller the weight of the style reward.

[0091] In some embodiments, the reward weight for style priors is not fixed, but dynamically adjusted based on the current contact state.

[0092] Specifically, in the righting phase (contact state away from the target): style weight is low, task reward dominates, allowing strategies to use non-human but effective righting movements. In the standing phase (contact state approaching the target): style weight gradually increases, guiding movements to be elegant and human-like. In the final standing phase: style weight is highest, ensuring the final posture is natural.

[0093] In some embodiments, the weight adjustment function is: ( )= +( - )·(1- ( ) / ),in ( () represents the graph distance from the current contact state to the target state. The maximum graph distance from the initial fall state to the target state. =0.1、 =0.8 is the weight range. This function ensures that: in the early stages of righting (when the graph distance is large), the style weight is only 0.1, and the strategy can freely explore various righting strategies, including non-human but effective movements (such as a single-arm push or a large torso twist); as the contact state gradually approaches the target (when the graph distance decreases), the style weight increases linearly, and the strategy is guided to adopt movements that are closer to human style; finally, when standing (when the graph distance is 0), the style weight reaches 0.8, ensuring the anthropomorphism of the final posture.

[0094] The total prior reward for phase-adaptive morphological decoupling motion can be used express, The expression is as follows: +

[0095] in, Indicates style weight, Indicates style rewards, This indicates a coordinated reward.

[0096] This disclosure proposes a phase-adaptive morphology-decoupled motion prior (PA-MDMP) in its embodiments, prioritizing functionality in the righting phase and gradually increasing anthropomorphic weight in the standing phase, thus balancing functionality and aesthetics.

[0097] This disclosure proposes a morphologically decoupled motion prior, decomposing the discriminator into an upper-body style prior (acting only on upper-body joints and trained using human data) and a full-body coordination prior (self-supervised training from successful simulated trajectories, evaluating upper and lower-body coordination). The upper-body style prior ensures anthropomorphic movements, while the full-body coordination prior ensures reasonable movements.

[0098] This embodiment of the disclosure breaks through the limitations of the standard AMP method. The upper body style discriminator only focuses on mappable upper body joints, while the lower body (wheeled chassis) is completely unconstrained by human reference data. The full-body coordination discriminator constrains the coordination of the upper and lower body from a physical rationality perspective, without requiring human data. In addition, the style prior reward weight is dynamically adjusted according to the current contact state: low weight during the righting phase (contact state away from the target), gradually increasing weight during the standing phase (contact state close to the target), and finally reaching the highest weight when standing. This satisfies the different anthropomorphic needs at different stages and solves the trade-off between functionality and anthropomorphism.

[0099] Existing methods do not model the differentiated availability of wheeled chassis under different fall postures. The four omnidirectional wheels are completely unusable when falling forward / overturning, fully usable for assisting force when falling backward, and partially usable for fine-tuning position when rolling sideways. Without distinguishing these situations, the strategy may incorrectly attempt to use the wheels when they are unavailable, or fail to fully utilize the wheels when they are available. This disclosure, in its embodiments, explicitly identifies the wheel availability state based on a contact state diagram. The strategy adaptively adjusts the level of wheeled chassis engagement according to the current contact state, fully utilizing the assistive capabilities of the wheels when they are available.

[0100] In some embodiments, domain randomization perturbations of the rope-driven differential transmission characteristics are injected during the training of reinforcement learning policies.

[0101] In some embodiments, the domain randomization perturbation of the rope-driven differential transmission characteristics includes at least one of the following: Apply asymmetric torque constraints to the joint; A low-pass filter with a random time constant is inserted between the joint command and the execution torque; Noise proportional to the coupled second joint angular velocity is injected into the observed value of the first joint angular velocity; Inject nonlinear noise related to joint angle and angular velocity into the joint angle observations; Apply a random delay to joint space motion commands.

[0102] In this embodiment, an equivalent domain randomization perturbation (joint coupling noise, asymmetric torque limitation, execution delay, etc.) is designed for the characteristics of rope-driven differential transmission to improve the robustness of the strategy during the training phase.

[0103] The methods in the above embodiments are further illustrated below with some specific examples: This disclosure addresses the practical deployment scenarios of rope-driven differential omnidirectional wheeled humanoid robots, resolving the unavoidable fall problem that occurs during task execution. Robots may fall into various postures, such as forward-leaning, backward-leaning, sideways, or overturned (upside down), due to uneven ground, collisions, or excessive loads. Traditional solutions require manual intervention to lift the robot after each fall, severely limiting its autonomous working capabilities and deployment value.

[0104] This disclosure enables the robot to autonomously recover its standing position from any fall posture without human intervention. After deployment, even if the robot falls during operation, it can automatically complete the entire process of righting its chassis, standing up, and resuming its working state, ensuring task continuity.

[0105] Specifically, the system features include the following: 1. Self-recovery capability in any posture: The robot can recognize the current fall posture (forward, backward, leftward, rightward, overturned, etc.) and automatically plan and execute a sequence of recovery actions. No matter what posture the robot falls in, the system can find a feasible recovery path.

[0106] 2. Human-like movement style: The robot's upper body movements during the standing process (arm support, torso exertion rhythm, and body coordination) closely resemble the natural human standing motion, avoiding stiff, mechanical movements. This not only enhances the robot's approachability but also makes the movements more in line with human intuitive understanding of a reasonable way to stand, making it easier for users to understand and accept.

[0107] 3. Phased Movement Strategy: The system adaptively adjusts its movement style according to different stages of the recovery process. During the chassis righting stage, the system prioritizes functionality, allowing efficient but not entirely human-like righting movements (such as large arm swings and torso twists). As the system approaches standing, it gradually increases the elegance of the movements, ultimately maintaining a natural and stable standing posture. Users experience a smooth process of first quickly righting themselves and then gracefully standing up.

[0108] 4. Differentiated Utilization of Omnidirectional Wheels: The system can intelligently determine the availability of omnidirectional wheels based on the fall posture. In a backward fall, all four wheels are in contact with the ground, and the system will use the wheels to assist in accelerating the recovery; in a side roll, some wheels are in contact with the ground, and the system will use the available wheels to fine-tune their positions; in a forward fall or rollover, the wheels are suspended in the air, and the system relies entirely on the upper body to right the user. Users do not need to worry about the underlying control logic; the system automatically selects the optimal strategy.

[0109] 5. Rope-driven Differential Transmission Adaptation: For the rope-driven differential joint transmission method used in the robot (two motors drive two joints via ropes and a differential mechanism), the system has a built-in control interface mapping mechanism. This mechanism automatically compensates for control deviations caused by non-ideal characteristics such as rope elasticity, friction, and slack, ensuring that the joint commands output by the strategy can be accurately translated into motor movements. Users experience precise and smooth motion execution, rather than jitter or deviations.

[0110] Application scenarios for rope-driven differential omnidirectional wheeled humanoid robots may include: In home service scenarios: When a robot falls over due to carpet edges or obstacles while moving items or cleaning the floor, it can stand up on its own and continue working without the user having to interrupt their task to help it up.

[0111] Commercial deployment scenarios: When robots work in public places such as restaurants, hotels, and office buildings, they can quickly and autonomously recover from occasional collisions with people or slippery floors, avoiding prolonged occupation of passageways or impact on service quality.

[0112] Industrial inspection scenario: When robots are inspecting complex environments such as factories and warehouses, they can get up autonomously and continue the inspection task if they fall due to uneven ground or collision with equipment, reducing the cost of human intervention.

[0113] Improved user experience is reflected in: Task continuity: When the robot falls, it no longer means the task is interrupted. Users do not need to intervene frequently, and the robot can independently complete long-term, multi-tasking work.

[0114] Deployment flexibility: Users can deploy robots in more complex and challenging environments (such as uneven ground or narrow spaces) without worrying about them not recovering after a fall.

[0115] Psychological acceptance: The anthropomorphic standing action makes the robot's behavior more in line with human expectations, reducing users' concerns about the robot going out of control or acting abnormally, and increasing trust.

[0116] Reduced maintenance costs: Reduces maintenance work such as manually lifting, inspecting, and restarting due to falls, thus lowering operating costs.

[0117] As part of a robot motion control system, the interface relationships between this disclosed embodiment and other system modules are as follows: 1. Sensing system interface: Input: Base attitude (pitch angle, roll angle, yaw angle) and angular velocity provided by the IMU (Inertial Measurement Unit).

[0118] Input: Motor angle and angular velocity provided by the motor encoder.

[0119] Input: Motor current signal provided by motor current sensor.

[0120] Input: Contact status of the wheel, hand, torso, and ground provided by contact sensors (optional).

[0121] 2. Execute system interface: Output: Motor space commands (target angle or target torque for each motor).

[0122] Output: Omnidirectional wheel chassis speed command ( , , ),in, Indicates chassis extension Velocity in the axial direction, Indicates chassis extension Velocity in the axial direction, Indicates chassis wrap Angular velocity of the axis.

[0123] 3. Task planning system interface: Input: Fall detection signal (triggered by the upper-level task planning module or the fall detection logic built into this module).

[0124] Output: Recovery complete signal (notifies the upper system that the robot has resumed standing and can continue to perform its tasks).

[0125] 4. Security monitoring system interface: Output: Safety-related state variables such as joint torque, velocity, and acceleration.

[0126] Input: Emergency stop signal (If a dangerous situation is detected during the recovery process, the upper-level system can stop the recovery action).

[0127] From the user's perspective, the complete process of a robot's self-recovery after a fall is as follows: 1. Fall Detection: The robot detects abnormal base posture (such as pitch or roll angle exceeding the threshold) via its IMU and determines it to be in a fall state. The system automatically pauses the current task and enters self-recovery mode.

[0128] 2. Posture Recognition: The system identifies the current fall posture type (forward, backward, leftward, rightward, overturn, etc.) based on the IMU posture and contact status (whether the wheels, palms, and torso are on the ground).

[0129] 3. Recovery Action Execution: Based on the identified posture type, the system automatically executes the corresponding recovery action sequence. For example: Forward prone position: Arms on the ground → torso lifted → chassis righted → wheels on the ground → stand up.

[0130] Backward lean posture: Use four-wheel assist to generate force → lean the torso forward → support yourself with your arms on the ground → stand up.

[0131] Side rollover posture: Use some wheels to make minor adjustments to position → brace yourself with your arms → right the chassis → stand up.

[0132] Overturning posture: Arms swinging wildly → torso twisting → chassis flipping → wheels touching the ground → standing up.

[0133] 4. Stable standing: After the robot stands up, the system performs a brief balance adjustment to ensure the stability of the base posture (pitch angle and roll angle are close to zero).

[0134] 5. Resume Task: The system sends a recovery completion signal to the upper-level task planning module, and the robot resumes normal operation, continuing to execute the task before the interruption or waiting for new task instructions.

[0135] The entire process requires no user intervention; the robot autonomously completes the entire process from falling to recovery. Users only need to perform initial configuration (such as setting the work area and task type) when the robot is first deployed. In subsequent operations, the robot can autonomously handle fall recovery issues.

[0136] Comparison with existing products: Existing products: After a fall, the user needs to be helped up manually, or can only recover from a specific posture (such as leaning back), and cannot handle complex postures such as side rolls or overturns.

[0137] This disclosed embodiment supports self-recovery in any posture, covering all common falling postures such as forward lying, backward lying, side rolling, and flipping over.

[0138] Existing products: The standing motion is mechanical and lacks an anthropomorphic style, resulting in low user acceptance.

[0139] In this embodiment, the upper body movements closely resemble the natural standing posture of a human, resulting in smooth and coordinated movements that enhance user trust and acceptance.

[0140] Existing products: They are not specifically optimized for the characteristics of rope-driven differential transmission, resulting in limited control accuracy and deviations or jitters in motion execution.

[0141] This embodiment of the disclosure features a built-in rope-driven differential control interface mapping mechanism that automatically compensates for non-ideal transmission characteristics, ensuring precise and smooth operation.

[0142] Through the aforementioned functional features and user experience improvements, the embodiments of this disclosure significantly enhance the autonomous working capability and practical deployment value of the rope-driven differential omnidirectional wheeled humanoid robot, enabling the robot to independently complete long-term, complex tasks in a wider range of scenarios.

[0143] Figure 4 A schematic diagram of the overall logic flow provided for embodiments of this disclosure, such as Figure 4 As shown, the overall logical flow provided in this embodiment of the disclosure may include the following four stages: Phase 1: Simulation Pre-training Phase Figure 5 This is a schematic diagram of the simulation pre-training logic flow provided in the embodiments of this disclosure, such as... Figure 5 As shown, the simulation pre-training provided in this embodiment may include the following steps: 1.1 Simulation Environment Construction A robot model is constructed using a joint-space-based rigid body dynamics simulator. The robot model includes: Upper body: Head 1 degree of freedom (pitch), neck 2 degrees of freedom (yaw and pitch), waist 3 degrees of freedom (roll, pitch, yaw), both arms 7 degrees of freedom each (shoulder 3 degrees of freedom, elbow 1 degree of freedom, wrist 3 degrees of freedom).

[0144] Lower body: The wheeled chassis acts as a single rigid body, and the four omnidirectional wheels achieve omnidirectional movement through kinematic constraints.

[0145] Contact model: The palm, torso, head, and wheels are all equipped with contact collision bodies, and the coefficient of friction is set to 0.8~1.2 (ground).

[0146] The simulation time step was set to 0.002 seconds (500Hz physical simulation), and the policy control frequency was 50Hz (policy inference was performed once every 10 simulation steps).

[0147] 1.2 Domain Randomization of Rope-Driven Differential Transmission Characteristics To address the five non-ideal characteristics of rope-driven differential transmission, an equivalent domain randomization perturbation is designed: (1) Unidirectional force characteristics of the rope: In the simulation, an asymmetric torque limit is applied to the joint. When the joint movement direction is opposite to the direction of the rope tension, the upper limit of the torque is randomly reduced to 60%~90% of the nominal value.

[0148] (2) Transmission compliance and elastic delay: A first-order low-pass filter is inserted between the joint command output by the strategy and the joint torque executed by the simulation. The time constant is randomly set to 0.01~0.05 seconds to simulate the response delay caused by the elasticity of the rope.

[0149] (3) Differential coupling noise: For differentially coupled joint pairs ( , ), in the joint Injecting and Coupled noise proportional to angular velocity: Observation = Real + · ,in In each training cycle, samples are randomly taken in radians (-0.05, 0.05) per second.

[0150] (4) Configuration-dependent motor-joint bias: Injecting nonlinear noise related to joint angles and angular velocities into joint angle observations: Noise = sin( )+ · ·| | in, Random sampling is performed in radians within the range of [-0.02, 0.02]. Random sampling was performed in the range [-0.01, 0.01]. Indicates the first The angle of each joint Indicates the first angular velocity of each joint .

[0151] (5) Execution delay and hysteresis: The action command output by the strategy takes effect with a probability of 0.3 and a random delay of 1 to 3 control cycles (20 to 60 milliseconds) to simulate the hysteresis caused by rope friction and slack.

[0152] The aforementioned domain randomization parameters are resampled at the beginning of each training episode to ensure that the policy learns robustness to the non-ideal characteristics of the rope drive.

[0153] 1.3 Gravity Progression Curriculum Design Gravity Course Parameters Current Gravity From initial gravity =2.0 m / s² gradually increased to standard gravity =9.81 m / s². The course progression strategy employs adaptive adjustment based on success rate: Initial stage: =2.0 m / s².

[0154] After collecting every 1000 episodes, calculate the standing success rate of the most recent 1000 episodes. .

[0155] like >0.85, then ← +0.5 m / s² (increasing gravity).

[0156] like <0.60, then ←max( 0.3 m / s², (Reverse gravity).

[0157] when achieve and When >0.80, fixed = Continue training until convergence, and obtain the pre-trained policy. .

[0158] The course is designed to enable the strategy to quickly learn the basic righting sequence under low gravity (about 5,000 episodes), and then optimize the force efficiency and coordination under gradually increasing gravity (about 20,000 episodes to reach standard gravity).

[0159] 1.4 Reinforcement Learning Training Configuration The policy network is trained using the Proximal Policy Optimization (PPO) algorithm. ( | The network structure is as follows: Input: Observation vector (Approximately 150 dimensions, including joint angles, angular velocities, base orientation, angular velocities, contact states, etc.)

[0160] Hidden layers: 3 fully connected layers, each with 512 neurons, and the activation function is ELU.

[0161] Output: Mean value of motion and standard deviation (Dimensions equal to the number of controllable joints, approximately 25 degrees of freedom).

[0162] PPO hyperparameter settings: Learning rate: 3×10 -4 (Using linear decay to 1×10) -5 ).

[0163] Discount factor =0.99.

[0164] GAE parameters =0.95.

[0165] PPO cutting parameters =0.2.

[0166] Entropy regularization coefficient: 0.01 (initial) → 0.001 (decay).

[0167] Mini-batch size: 4096 time steps.

[0168] Training rounds: Five rounds of PPO updates are performed after each data collection.

[0169] Parallel training configuration: Experience is collected simultaneously using 4096 parallel simulation environments, each independently sampling the initial fall posture and domain randomization parameters. The total training steps are approximately 2 × 10⁻⁶. 8 Steps (approximately 50 hours).

[0170] Phase 2: Real Device Data Collection and Interface Mapping Learning Figure 6 This is a schematic diagram of the real device data collection and interface mapping learning logic flow provided in the embodiments of this disclosure, such as... Figure 6 As shown, real device data collection and interface mapping learning can include the following steps: 2.1 Real Device Data Collection Protocol Deploying pre-trained strategies on real devices (Or use teleoperation) Collect data (which can be used to train the forward and inverse mapping networks), recording at a frequency of 100Hz, and recording the following: Motor side: Dual motor encoder angle ( , ), angular velocity ( , ), current ( , ).

[0171] Sensors: IMU attitude (quaternion), angular velocity, linear acceleration.

[0172] Command: Joint space target position output by the strategy ( , ).

[0173] Data collection strategy: Randomly initialize the robot with different falling postures (forward, backward, leftward, rightward, and overturned).

[0174] Collect 50-100 tracks for each posture (recovery tracks from falling posture to standing posture), with each track lasting 5-15 seconds.

[0175] Total data volume: approximately 10,000 trajectories, approximately 150 hours of time-series data.

[0176] 2.2 Forward Mapping Network train The forward mapping network maps motor-side observables to joint state estimates: ( , , , )= ( , , , , , , , , , ) in, This represents the estimated joint angle of joint 1. This represents the estimated joint angle of joint 2. This represents the estimated joint angular velocity of joint 1. This represents the estimated joint angular velocity of joint 2. This indicates the motor angle of motor A. This indicates the motor angle of motor B. This indicates the speed of motor A. This indicates the speed of motor B. This represents the motor current of motor A. This represents the motor current of motor B. This indicates the historical joint angle of joint 1. This indicates the historical joint angle of joint 2. This represents the historical joint angular velocity of joint 1. This represents the historical joint angular velocity of joint 2.

[0177] Network structure: Input layer: 10-dimensional (6-dimensional motor signal, 2-dimensional historical joint position, 2-dimensional historical joint velocity (joint velocity can be used to replace joint angular velocity)).

[0178] Hidden layers: 3 fully connected layers, each with 256 neurons, and the activation function is ReLU.

[0179] Output layer: 4D joint state estimation (position and velocity estimation of 2 joints).

[0180] Training objective: Minimize joint state estimation error.

[0181] Since the joint angle cannot be directly measured on a real machine, the following indirect monitoring signals are used: By using the base pose changes obtained from IMU integration and combining them with forward kinematics, the joint state can be inferred.

[0182] During periods of rest or slow motion, assuming the ideal difference kinematics holds, we use... =( + ) / (2 () as a pseudo tag.

[0183] During the dynamic motion phase, temporal consistency loss is used: it requires that the joint state estimates at adjacent time steps satisfy the dynamic constraints.

[0184] Loss function: = ·MSE( , )+ ·MSE( , )+ ·

[0185] in, =1.0, =0.5, =0.3. Indicates use =( + ) / (2 ( ) as the joint angle of the pseudo-label, Indicates use =( + ) / (2 ) as the joint angular velocity of the pseudo-label, This represents the loss of timing consistency.

[0186] Temporal consistency loss Defined as: =|| ( + ·Δt)||² in, This represents the estimated joint angle at time t. This represents the estimated joint angle at time t-1. This represents the estimated joint angular velocity at time t-1, where Δt is the time difference between time t and time t-1.

[0187] Training configuration: Adam optimizer, learning rate 1×10⁻⁶ - ³, batch size 256, training for 50 epochs.

[0188] 2.3 Inverse Mapping Network train The reverse mapping network maps joint space commands to motor space commands: ( , )= ( , , , , , ) in, This represents the target motor angle for motor A. This indicates the target motor angle of motor B. This indicates the target joint angle of joint 1. This indicates the target joint angle of joint 2. This represents the current estimated joint angle of joint 1. This represents the current estimated joint angle of joint 2. This represents the current estimated joint angular velocity of joint 1. This represents the current estimated joint angular velocity of joint 2.

[0189] Network structure: Input layer: 6-dimensional (2-dimensional target joint position and 4-dimensional current joint state estimation).

[0190] Hidden layers: 3 fully connected layers, each with 256 neurons, and the activation function is ReLU.

[0191] Output layer: 2D (target angle of dual motors).

[0192] Training objective: To minimize the error between the actual joint position and the target joint position.

[0193] Using behavioral cloning, we learn from the collected data: Input: Joint commands output by the strategy and current joint status estimation (Depend on (Estimation). Joint space motion commands output by the strategy. Joint commands are obtained after transformation. ,For example, , where m and n are coefficients.

[0194] Tag: Actual instructions sent to the motor .

[0195] Loss function: =MSE( , )+ ·|| - ||².

[0196] in, This represents the predicted value from the reverse network. The calculated value is the inverse kinematics of the ideal difference. =0.1 is used for regularization to prevent the mapping from deviating too far from the ideal value.

[0197] Training configuration: Adam optimizer, learning rate 1×10⁻⁶ - ³, batch size 256, training for 50 epochs.

[0198] 2.4 Simulation Parameter System Identification Simultaneously perform system identification of simulation physical parameters and optimize the following parameters: Linkage mass and inertia (±20% adjustment range).

[0199] Joint friction coefficient (0.01~0.5 N·m).

[0200] Contact stiffness and damping (ground-robot contact).

[0201] Motor torque constant (±15% adjustment range).

[0202] The Bayesian optimization method is used, with the objective function being the state error between the simulated trajectory and the actual machine trajectory: =Σ|| - ||²+|| - ||² in, Simulation values ​​(simulation states) representing joint and / or motor angles. Represents the actual value of the joint and / or motor angles (actual machine state). Simulation values ​​(simulation states) representing the angular velocities of the joints and / or motors. Represents the actual value of the joint and / or motor angular velocity (actual machine state).

[0203] Optimize and iterate 50-100 times, evaluating 10 sets of parameters in each round, and selecting the optimal parameters. Update the robot (simulation) model with the smallest set of parameters.

[0204] Phase 3: Fine-tuning of reinforcement learning strategies Figure 7 This is a schematic diagram of the logic flow for fine-tuning reinforcement learning strategies provided in the embodiments of this disclosure, such as... Figure 7 As shown, fine-tuning a reinforcement learning strategy may include the following steps: 3.1 Simulation of Interface Mapping Error The learned interface mapping network is integrated into the simulation training loop to simulate the characteristics of the real machine interface. (1) Observation noise injection: the ideal joint state output by the simulation environment Policy observations are obtained after forward mapping of residual distribution noise. : = +

[0205] in, Sampled from the prediction error distribution of the forward mapping network on the validation set (mean 0, standard deviation approximately 0.01–0.03 radians).

[0206] (2) Motion error injection: Joint commands output by the strategy The simulation execution instructions are obtained after inverse mapping of the residual distribution noise. : = +

[0207] in, Samples were taken from the prediction error distribution of the inverse mapping network on the validation set (mean 0, standard deviation approximately 0.02–0.05 radians).

[0208] 3.2 Residual Dynamics Network (Optional) If a significant dynamic gap still exists after system identification, a residual dynamics network can be trained. Correct state transition: = + ( , ) in, This represents the correction value for the state at the next time step (t+1). The simulated value representing the state at the next time step (t+1 time) ( , ) represents the residual dynamics network The output value, This represents the estimated joint state at the current time (time t). This represents the joint space motion command output by the strategy at the current time (time t).

[0209] Residual network structure: Input: Forward mapping network Output of current joint state estimate (Joint position, velocity, base posture, velocity) and joint space motion commands output by the strategy. .

[0210] Hidden layers: 2 fully connected layers, each with 128 neurons, and the activation function is Tanh.

[0211] Output: State increment Δ (and (Same dimension).

[0212] Training data: State transition pairs collected from real machines ( , , The loss function is: =MSE( + ( , ), ) in, This represents the true value of the state at the next time step (t+1).

[0213] Training configuration: Adam optimizer, learning rate 5×10 -4 Batch size 512, training for 30 epochs.

[0214] 3.3 Fine-tuning the training configuration Continue training the strategy in the modified simulation environment. → : Initialization: Load the pre-trained strategy The weight.

[0215] Learning rate: Reduced to 1×10 -4 (Avoid deviating too much from the pre-training strategy).

[0216] Training steps: Approximately 5 x 10 7 Steps (approximately 10 hours).

[0217] Other PPO hyperparameters remain unchanged.

[0218] Fine-tuning objective: To adapt the strategy to the characteristics of the real machine interface (interface mapping error) and residual dynamics, while maintaining the stand-up capability learned in the pre-training phase.

[0219] Continue training the reinforcement learning strategy to obtain the fine-tuned policy parameters. .

[0220] Phase 4: Real Device Deployment and Online Execution Figure 8 This is a schematic diagram of the real device deployment and online execution logic flow provided in the embodiments of this disclosure, such as... Figure 8 As shown, real-device deployment and online execution can include the following steps: 4.1 The data flow during real device deployment is as follows: (1) Motor encoder reading: Read the angle of the dual motors at a frequency of 100Hz. , ), angular velocity ( , ) and current ( , ).

[0221] (2) Forward mapping: Inputting motor signals The network yields joint state estimates. , , , ).

[0222] (3) Strategy reasoning: Combine joint state estimation, IMU data, contact state, etc. into an observation vector. Input policy network Receive joint space motion commands .

[0223] (4) Reverse mapping: mapping joint commands and current joint state estimation input The network receives motor space commands ( , ).

[0224] (5) Motor control: Sending motor commands to the underlying motor controller (PD controller, = 50 N·m / rad, = 2 N·m·s / rad).

[0225] The end-to-end latency of the entire process is controlled within 20 milliseconds (approximately 5 milliseconds for policy inference, approximately 2 milliseconds each for mapping network inference, and approximately 10 milliseconds for communication and control).

[0226] 4.2 Security Monitoring and Protection Implement the following security mechanisms during deployment: Joint angle limit: If the joint angle exceeds the safe range (±150° for each joint), the motor will be stopped immediately and an emergency stop will be triggered.

[0227] Torque Limitation: The motor output torque is limited to within 80% of the rated value (approximately 40 N·m) to prevent overload.

[0228] Attitude monitoring: If the base pitch angle or roll angle exceeds 60° and lasts for more than 2 seconds, it is determined that the standing up has failed and the attempt is stopped.

[0229] Collision detection: If the motor current suddenly changes by more than 10A / 10ms, it is determined to be an accidental collision and the operation is stopped immediately.

[0230] 4.3 Iterative Optimization Process After deployment, the following processes can be iteratively executed to further reduce the Sim-to-Real Gap: (1) Collect new failure cases and boundary case data on the actual machine.

[0231] (2) Retrain or fine-tune the interface mapping network using new data. and .

[0232] (3) Update simulation parameters (system identification) and residual dynamics network.

[0233] (4) Continue to fine-tune the strategy in the correction simulation.

[0234] (5) Redeploy and evaluate performance.

[0235] Typically, after 2-3 rounds of iteration, the success rate of deployment can be increased from 60%-70% in the initial deployment to over 85%-90%.

[0236] This disclosure achieves anthropomorphic self-recovery capability of a tethered differential omnidirectional wheeled humanoid robot under any fall posture by using gravity-progressive learning guided by contact state diagrams, learnable bidirectional interface mapping for tethered differential systems, and phase-adaptive morphological decoupling motion priors. Compared with existing technologies, it has the following advantages: 1. Solving the exploratory challenges of righting wheeled chassis. Contact state graph shaping rewards transform the sparse reward problem into a dense reward problem, providing immediate guidance for each step (e.g., more wheels on the ground), significantly accelerating the exploratory learning of wheeled chassis righting. Experimental data show that, compared to the sparse reward baseline without shaping, contact state graph guidance improves training convergence speed by approximately 3-5 times, enabling the policy to learn the basic righting sequence within the first 5 million steps, while the sparse reward baseline still fails to converge stably within 20 million steps.

[0237] The progressive gravity training program enables the strategy to quickly learn basic righting sequences under low gravity (0.3g-0.5g), avoiding blind exploration under normal gravity. In the low gravity phase, the strategy achieves a righting success rate of over 90% with only about 1 million steps, and then gradually optimizes force efficiency as gravity increases. Compared to training directly under normal gravity, this program increases the final success rate by approximately 40%, and the movements are smoother and more efficient.

[0238] The directed graph structure of the contact state diagram provides differentiated recovery path guidance for different fall postures: in a forward-lying position, it prioritizes hand-to-toe support → torso lift → wheels touch the ground; in a backward-lying position, it allows for direct standing up with four-wheel assist; and in a side-fall position, it guides fine-tuning of position using some of the wheels that are in contact with the ground. This differentiated guidance reduces the average recovery time for all postures by approximately 30%.

[0239] 2. Gap in the control interface of the modified rope-driven differential transmission The learnable bidirectional interface mapping absorbs all the non-ideal characteristics of the control interface layers in rope-driven differential systems, enabling the strategy to learn and think in joint space and correctly drive the motor during deployment using the learned mapping. This fundamentally solves the simulation-to-real control interface gap problem, a capability that existing methods that only correct the dynamic gap lack.

[0240] Using motor current signals to correct joint state estimation, compared to the ideal difference kinematics formula =( + ) / (2r), =( The forward mapping network () / (2r) significantly improves accuracy under heavy loads and extreme configurations (such as unconventional postures during fall recovery). Experimental data show that in fall recovery scenarios, the forward mapping network... The joint angle estimation error is reduced by approximately 45%-60% compared to ideal kinematics, and the joint angular velocity estimation error is reduced by approximately 35%-50%. The introduction of motor current signals enables the network to sense rope tension and load status, providing reliable estimates even when the rope is slack or overloaded.

[0241] Inverse mapping network Its configuration-dependent design enables it to adaptively adjust motor commands under different joint angles and load conditions. Compared to a fixed ideal inverse mapping, this network reduces joint command tracking errors by approximately 40%, with a particularly significant improvement in tracking performance when the joint is near its limit or undergoing rapid changes in direction.

[0242] Joint training via bidirectional mapping ensures closed-loop consistency between forward estimation and inverse control, avoiding estimation-control mismatch issues that may arise from individual training. This allows the joint-space control laws learned in simulation to be seamlessly transferred to the actual machine's motor space for execution.

[0243] 3. Achieve anthropomorphic restoration of hybrid robots The decoupled motion prior addresses the issue of hybrid-morphology robots (humanoid upper body + wheeled lower body) being unable to directly utilize human reference data. The upper body style prior ensures human-like movements, while the full-body coordination prior ensures reasonable movements; the two complement each other. Experiments show that, compared to the standard AMP method (which requires complete morphological consistency), the decoupled prior disclosed in this paper improves the anthropomorphism score (assigned by human evaluators) of the final standing posture by approximately 35%-45%, while maintaining the same success rate for righting the robot.

[0244] The phase-adaptive weighting mechanism achieves a balance between functionality and anthropomorphism: In the righting-out phase (contact state away from the target), the style weight is set to 0.1-0.3, allowing for non-human but effective movements, prioritizing task success rate; in the standing-up phase (contact state approaching the target), the style weight increases to 0.5-0.7, guiding graceful movements; finally, in the standing-up phase, the style weight reaches 0.8-1.0, ensuring a natural final posture. This dynamic weighting strategy increases the righting-out success rate by approximately 25% compared to a fixed high weight, while simultaneously improving the final posture anthropomorphism score by approximately 40% compared to a fixed low weight.

[0245] A full-body coordination prior discriminator is self-supervised and trained from a simulated successful trajectory to assess the coordination between upper body movement changes and base state changes. This discriminator teaches the strategy how the base should respond when the upper body exerts force, and how arm swings should coordinate with trunk rotation, thus avoiding failures caused by mismatched upper and lower body movements. Experimental data show that introducing coordination priors improves the smoothness score of the recovery process by approximately 30% and energy efficiency by approximately 15%.

[0246] 4. Fully utilize the differentiated capabilities of omnidirectional wheels The omnidirectional wheel differentiation strategy adaptively adjusts the chassis engagement based on the fall posture, fully utilizing the auxiliary capabilities of the wheeled chassis when available. In a backward fall posture with all four wheels in contact with the ground, the strategy learns to use the forward and backward movement of the wheels to assist in force generation, reducing recovery time by approximately 40% compared to relying solely on upper body force. In a side roll posture with some wheels in contact with the ground, the strategy learns to use available wheels to fine-tune the base position, increasing the recovery success rate by approximately 20%. In forward and overturned postures with the wheels suspended in the air, the strategy focuses on the upper body righting motion, avoiding interference from ineffective wheel commands.

[0247] This differentiated strategy allows the robot to flexibly adjust its recovery method based on the actual contact status, reducing the average recovery time by about 25% and energy consumption by about 18% compared to a fixed strategy (either relying entirely on the upper body or always trying to use wheels).

[0248] 5. Improve the robustness of real-device deployment. Domain randomization design for rope-driven differential transmission characteristics addresses the unidirectional force, elastic delay, differential coupling, relaxation, and friction of the rope by injecting equivalent perturbations during simulation training, including joint coupling noise, asymmetric torque constraints, and execution delays. This enhances the robustness of the strategy to the non-ideal characteristics of the rope-driven system during real-world deployment.

[0249] Experimental data shows that, compared to general domain randomization, targeted domain randomization increases the first-time success rate of real-device deployment from approximately 60% to approximately 85%, and reduces the average recovery time from approximately 12 seconds to approximately 8 seconds. The strategy significantly improves tolerance to abnormal conditions such as rope slack and sudden load changes, maintaining a success rate of over 90% even with slight rope slack (tension reduction within 30%).

[0250] The synergistic effect of bidirectional interface mapping and domain randomization further enhances the simulation-to-real migration effect: interface mapping corrects systematic deviations at the control level, while domain randomization covers random disturbances at the dynamics level. Their complementary nature brings the real machine performance close to the optimal simulation level. After multiple iterations of optimization, the real machine recovery success rate reaches over 95%, approaching the 98% success rate of the simulation environment.

[0251] 6. Supports unified processing of various falling postures. The method disclosed herein can handle various falling postures, including forward, backward, sideways (left / right), and rollover, without requiring separate strategy training for each posture. The unified representation of the contact state graph allows the recovery process of different postures to be modeled as different paths in the same state space, and the strategy automatically selects the appropriate recovery action sequence by observing the current contact state.

[0252] Experiments show that the single strategy achieves an average success rate of over 92% across all pose types, with a performance decrease of no more than 5% compared to dedicated strategies optimized for a single pose, while significantly reducing development and maintenance costs. The strategy exhibits good generalization ability to small changes in the initial pose (such as a roll angle deviation of ±15°), and recovery can be initiated without precise pose classification.

[0253] 7. Reduce reliance on precise modeling The learnable interface mapping learns the motor-joint mapping relationship from real machine data, eliminating the need for precise modeling of difficult-to-measure physical parameters such as the rope elasticity coefficient, friction coefficient, and differential mechanism clearance. This significantly reduces the difficulty and cost of system identification, making the method easier to extend to other rope-driven differential robot platforms.

[0254] Experiments show that even if the simulated rope parameters deviate from the actual values ​​by ±30%, interface mapping learning can still enable the actual machine performance to reach more than 90% of the level under accurate modeling. Compared with the traditional precision modeling + parameter identification process, this data-driven correction method shortens the development cycle by about 50% and has stronger adaptability to parameter drift caused by mechanical wear, aging, etc.

[0255] In some embodiments, the contact state graph shaping reward can be replaced by a continuous reward function based on the number of contact points.

[0256] Implementation method: Instead of explicitly constructing a contact state graph, a continuous reward function is designed. = ·(Number of wheels grounded)+ • (Number of times palms touch the ground) - • (Torso ground contact area), where the weight , , The system dynamically adjusts based on the current base posture. For example, it increases the elevation when the base tilt angle exceeds a threshold. (Encourage hands to support the weight), raise the weight as the inclination angle approaches upright. (Encourage the use of wheels for load-bearing).

[0257] Advantages: Simple to implement, no predefined state transition graph required. The reward signal is continuously differentiable, which is beneficial for gradient optimization. Better generalization across different robot configurations.

[0258] In some embodiments, milestone rewards based on success trajectory clustering can be used to replace contact state graph shaping rewards.

[0259] Implementation: First, a small number of successfully recovered trajectories are collected through random exploration or teleoperation. Keyframes (base posture + contact state) in the trajectories are clustered to obtain several milestone states. During training, whenever the robot's state enters the neighborhood of a milestone, a one-time reward is given, and the reward value increases as the milestone approaches the target state.

[0260] Advantages: Data-driven, automatically discovers key intermediate states, reducing manual design. Milestone rewards are sparse but informative, resulting in good training stability. Can be combined with offline reinforcement learning methods using demonstration data.

[0261] In some embodiments, a learnable bidirectional interface mapping can be replaced by a physical model-based adaptive compensator.

[0262] Implementation: Retain the ideal difference kinematics as the basic mapping, and additionally train a lightweight compensation network. and Forward mapping is = ( )+ ( , , ), inversely mapped to = ( )+ ( , The compensation network only learns the residuals, retaining the physical priors.

[0263] Advantages: High data efficiency, small number of compensation network parameters, and low training data requirements. Strong physical interpretability, facilitating debugging and safety verification. Approaches the ideal model under normal operating conditions, activating compensation only under extreme configurations.

[0264] In some embodiments, an end-to-end recurrent neural network mapping can be used instead of a learnable bidirectional interface mapping.

[0265] Implementation: The timing mapping relationship between the motor and joints is directly modeled using an LSTM or GRU network. The forward mapping input consists of the time series of the motor encoder and current. (tk),..., (t), (tk),..., (t)}, output joint state estimate (t). The reverse mapping is similar, taking the joint command sequence and historical joint states as input, and outputting the motor command sequence.

[0266] Advantages: Automatically captures the dynamic characteristics (elastic delay, hysteresis) of the rope-driven system. No explicit modeling of the physical mechanism is required, making it highly adaptable. It can handle long-term dependencies (such as the recovery process after rope slack).

[0267] In some embodiments, phase-adaptive motion priors can be replaced by linear weighted scheduling based on task progress.

[0268] Implementation method: Define task progress indicators ∈[0,1], for example =(Current base height - Initial height) / (Target height - Initial height), or normalized progress based on time steps. Style prior weights are set to... (t)= +( )· (t), which means it grows linearly with the progress of the task.

[0269] Advantages: Extremely simple to implement, no contact state detection required. Smooth weight changes and stable training. Good generalization to different fall postures (not dependent on specific contact state definitions).

[0270] In some embodiments, adaptive motion priors based on discriminator confidence can be used to replace phase adaptive motion priors.

[0271] Implementation: Instead of pre-setting weights, the style discriminator dynamically adjusts its weights based on the output confidence of the style discriminator. When the discriminator gives a low score to the anthropomorphism of the current action, the style prior weights are reduced (allowing the strategy to deviate from human style to complete the task); when the score is high, the weights are increased (reinforcing the anthropomorphic behavior). The specific implementation is as follows: (t)= ·sigmoid( ·( (s, a) )),in For the discriminator output, The threshold value is used.

[0272] Advantages: Fully adaptive, requiring no manual scheduling strategy design. Automatically balances task completion and style constraints, avoiding premature entry into local optima. Automatically adjusts weights for fall postures of varying difficulty.

[0273] In some embodiments, gravity-based progressive courses can be replaced by adaptive difficulty adjustments based on success rates.

[0274] Implementation: Instead of using gravity as a course parameter, the difficulty distribution of the initial fall postures is dynamically adjusted based on the success rate of the current strategy. A posture difficulty pool is maintained, containing fall postures with different tilt and rotation angles. During training, if the recent success rate is higher than a threshold (e.g., 80%), the sampling probability of difficult postures is increased; if the success rate is lower than the threshold (e.g., 30%), the sampling probability of easy postures is increased.

[0275] Advantages: The curriculum is directly tailored to strategic abilities, avoiding the mismatch between gravity parameters and actual difficulty. It is applicable to any type of fall recovery task, not limited to righting wheeled chassis. It can be used in combination with other curriculum parameters (such as ground friction coefficient and initial angular velocity).

[0276] In some embodiments, gravity-based progressive courses can be replaced by courses based on auxiliary reward decay.

[0277] Implementation: In the initial stages of training, abundant auxiliary rewards are provided (such as rewards for each joint approaching the target posture and rewards for a decrease in base angular velocity). As training progresses, the weight of these auxiliary rewards is gradually reduced, eventually retaining only the task completion reward. The reduction strategy can be exponential decay. (t)= ·exp(- t) or segmented decay based on the number of training steps.

[0278] Advantages: It does not alter the physical environment, avoiding the engineering complexity of adjusting simulator gravity parameters. Auxiliary rewards can be flexibly designed for specific sub-tasks (e.g., stabilizing the base before moving the arm). It is orthogonal to other course learning methods and can be used in combination.

[0279] Figure 9 This is a schematic diagram of the fall recovery device for a rope-driven differential omnidirectional wheeled humanoid robot provided in an embodiment of this disclosure. Figure 9 As shown in the embodiments of this disclosure, the fall recovery device for a rope-driven differential omnidirectional wheeled humanoid robot includes: The acquisition module 901 is used to acquire the current angle, angular velocity and current of the motor; The forward mapping module 902 is used to input the current angle, angular velocity and current of the motor into the forward mapping network to obtain the joint state estimate output by the forward mapping network; The reasoning module 903 is used to input the joint state estimate, the current posture and angular velocity of the base, and the current contact state of the robot into the reinforcement learning policy network for policy reasoning, and obtain the joint space action command output by the reinforcement learning policy network. The reverse mapping module 904 is used to input the joint space motion command into the reverse mapping network to obtain the motor space command output by the reverse mapping network; The control module 905 is used to control the motor to perform a fall recovery action based on the motor space command.

[0280] In some embodiments, the apparatus further includes a simulation pre-training module; The simulation pre-training module is used to perform simulation pre-training on the reinforcement learning policy network. The steps for performing simulation pre-training on the reinforcement learning policy network are as follows: Construct a contact state space and a contact state transition diagram, and build a robot simulation model based on a rigid body dynamics simulator in joint space; Reinforcement learning strategy training is performed based on the contact state transition graph. An intermediate reward is given for each completed contact state transition step until the strategy success rate is higher than the first threshold and the current gravity value reaches the standard gravity value. At this point, reinforcement learning strategy training stops, and pre-trained strategy parameters are obtained.

[0281] In some embodiments, domain randomization perturbations of the rope-driven differential transmission characteristics are injected during the training of reinforcement learning policies.

[0282] In some embodiments, the domain randomization perturbation of the rope-driven differential transmission characteristics includes at least one of the following: Apply asymmetric torque constraints to the joint; A low-pass filter with a random time constant is inserted between the joint command and the execution torque; Noise proportional to the coupled second joint angular velocity is injected into the observed value of the first joint angular velocity; Inject nonlinear noise related to joint angle and angular velocity into the joint angle observations; Apply a random delay to joint space motion commands.

[0283] In some embodiments, during the reinforcement learning policy training process, whenever the policy success rate under the current gravity is higher than the first threshold, the gravity value is increased; whenever the policy success rate under the current gravity is lower than the second threshold, the gravity value is decreased.

[0284] In some embodiments, the intermediate reward includes the total prior reward for stage-adaptive morphological decoupling motion; The phased adaptive morphological decoupling motion prior total reward includes: style reward and coordination reward; The style reward is determined based on an adversarial discriminator trained on human standing motion capture data; The coordinated reward is determined based on an adversarial discriminator trained under self-supervised conditions using simulated successful trajectories.

[0285] In some embodiments, the weight of the style reward differs depending on the contact state; The smaller the graph distance between the contact state and the target state, the greater the weight of the style reward; The greater the graph distance between the contact state and the target state, the smaller the weight of the style reward.

[0286] In some embodiments, the intermediate reward includes positive rewards and negative rewards; A positive reward is given when the graph distance between the contact state and the target state decreases. An increase in the graph distance between the contact state and the target state results in a negative reward.

[0287] In some embodiments, the apparatus further includes an interface mapping network training module; The interface mapping network training module is used to train the forward mapping network and the reverse mapping network; The steps for training the forward mapping network and the reverse mapping network are as follows: Deploy the pre-trained strategy parameters on a real device and collect training data; The training data is used to train the forward mapping network and the reverse mapping network.

[0288] In some embodiments, the apparatus further includes a model parameter update module; The model parameter update module is used to identify the system physical parameters of the simulation based on the state error between the simulation trajectory and the real machine trajectory, and update the parameters of the robot simulation model.

[0289] In some embodiments, the apparatus further includes a policy network fine-tuning module; The policy network fine-tuning module is used to integrate the learned forward mapping network and the reverse mapping network into the simulation training loop, and inject interface mapping error; Load the pre-trained strategy parameters; Continue training with the reinforcement learning strategy to obtain the fine-tuned policy parameters.

[0290] In some embodiments, the policy network fine-tuning module is further configured to: The residual dynamics network is trained using the training data; The state transition is corrected through the residual dynamics network.

[0291] In some embodiments, the contact state space includes at least one of the following states: forward-lying state, backward-lying state, side-flipping state, overturning state, half-squatting state, and standing state.

[0292] The apparatus of this disclosure embodiment can execute the method provided in this disclosure embodiment, and its implementation principle is similar, and it has corresponding technical effects. The actions performed by each module in the apparatus of each embodiment of this disclosure correspond to the steps in the method of each embodiment of this disclosure. For a detailed functional description of each module of the apparatus, please refer to the description in the corresponding method shown above, and it will not be repeated here.

[0293] This disclosure provides an electronic device (computer apparatus / device / system) including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method provided in any optional embodiment of this disclosure.

[0294] In one alternative embodiment, an electronic device is provided, such as Figure 10 As shown, Figure 10 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this disclosure.

[0295] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0296] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0297] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0298] The memory 4003 is used to store computer programs that execute embodiments of the present disclosure, and is controlled by the processor 4001 to execute them. The processor 4001 is used to execute the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0299] This disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0300] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0301] It should be understood that although arrows indicate various operation steps in the flowcharts of the embodiments of this disclosure, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of this disclosure, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of this disclosure do not limit this.

[0302] The above description is only an optional implementation method for some implementation scenarios of this disclosure. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this disclosure without departing from the technical concept of this disclosure also fall within the protection scope of the embodiments of this disclosure.

Claims

1. A method for fall recovery of a rope-driven differential omnidirectional wheeled humanoid robot, characterized in that, include: Obtain the current angle, angular velocity, and current of the motor; The current angle, angular velocity, and current of the motor are input into the forward mapping network to obtain the joint state estimate output by the forward mapping network; The joint state estimate, the current posture and angular velocity of the base, and the current contact state of the robot are input into the reinforcement learning policy network for policy reasoning to obtain the joint space action command output by the reinforcement learning policy network. The joint space motion command is input into the inverse mapping network to obtain the motor space command output by the inverse mapping network; Based on the aforementioned motor space command, the motor is controlled to perform a fall recovery action.

2. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 1, characterized in that, The method further includes: The reinforcement learning policy network is pre-trained using simulation. The steps for performing simulation pre-training on the reinforcement learning policy network are as follows: Construct a contact state space and a contact state transition diagram, and build a robot simulation model based on a rigid body dynamics simulator in joint space; Reinforcement learning strategy training is performed based on the contact state transition graph. An intermediate reward is given for each completed contact state transition step until the strategy success rate is higher than the first threshold and the current gravity value reaches the standard gravity value. At this point, reinforcement learning strategy training stops, and pre-trained strategy parameters are obtained.

3. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 2, characterized in that, Inject domain randomization perturbations of the rope-driven differential transmission characteristics during the training process of reinforcement learning strategies.

4. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 3, characterized in that, The domain randomization perturbation of the rope-driven differential transmission characteristics includes at least one of the following: Apply asymmetric torque constraints to the joint; A low-pass filter with a random time constant is inserted between the joint command and the execution torque; Noise proportional to the coupled second joint angular velocity is injected into the observed value of the first joint angular velocity; Inject nonlinear noise related to joint angle and angular velocity into the joint angle observations; Apply a random delay to joint space motion commands.

5. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 2, characterized in that, During the reinforcement learning policy training process, whenever the policy success rate under the current gravity is higher than the first threshold, the gravity value is increased; whenever the policy success rate under the current gravity is lower than the second threshold, the gravity value is decreased.

6. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 2, characterized in that, The intermediate reward includes the phase-adaptive morphological decoupling motion prior total reward; The phased adaptive morphological decoupling motion prior total reward includes: style reward and coordination reward; The style reward is determined based on an adversarial discriminator trained on human standing motion capture data; The coordinated reward is determined based on an adversarial discriminator trained under self-supervised conditions using simulated successful trajectories.

7. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 6, characterized in that, The weight of the style reward varies depending on the contact state; The smaller the graph distance between the contact state and the target state, the greater the weight of the style reward; The greater the graph distance between the contact state and the target state, the smaller the weight of the style reward.

8. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 2, characterized in that, The intermediate rewards include positive rewards and negative rewards; A positive reward is given when the graph distance between the contact state and the target state decreases. An increase in the graph distance between the contact state and the target state results in a negative reward.

9. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to any one of claims 2 to 8, characterized in that, The method further includes: The forward mapping network and the reverse mapping network are trained; The steps for training the forward mapping network and the reverse mapping network are as follows: Deploy the pre-trained strategy parameters on a real device and collect training data; The training data is used to train the forward mapping network and the reverse mapping network.

10. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 9, characterized in that, The method further includes: The system identifies the physical parameters of the simulation based on the state error between the simulated trajectory and the actual machine trajectory, and updates the parameters of the robot simulation model.

11. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 10, characterized in that, The method further includes: The learned forward mapping network and the backward mapping network are integrated into the simulation training loop, and interface mapping error is injected. Load the pre-trained strategy parameters; Continue training with the reinforcement learning strategy to obtain the fine-tuned policy parameters.

12. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 11, characterized in that, The method further includes: The residual dynamics network is trained using the training data; The state transition is corrected through the residual dynamics network.

13. The fall recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 2, 3, 4, 5, 6, 7, 8, 10, 11, or 12, is characterized in that... The contact state space includes at least one of the following states: forward-lying state, backward-lying state, side-flipping state, overturning state, half-squatting state, and standing state.

14. The fall self-recovery method for a rope-driven differential omnidirectional wheeled humanoid robot according to claim 9, characterized in that, The contact state space includes at least one of the following states: forward-lying state, backward-lying state, side-flipping state, overturning state, half-squatting state, and standing state.

15. A fall recovery device for a rope-driven differential omnidirectional wheeled humanoid robot, characterized in that, include: The acquisition module is used to acquire the current angle, angular velocity, and current of the motor. The forward mapping module is used to input the current angle, angular velocity and current of the motor into the forward mapping network to obtain the joint state estimate output by the forward mapping network; The reasoning module is used to input the joint state estimate, the current posture and angular velocity of the base, and the current contact state of the robot into the reinforcement learning policy network for policy reasoning, and obtain the joint space action command output by the reinforcement learning policy network. The reverse mapping module is used to input the joint space motion command into the reverse mapping network and obtain the motor space command output by the reverse mapping network; The control module is used to control the motor to perform fall recovery actions based on the motor space commands.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method described in any one of claims 1 to 14.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 14.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 14.

Images