Whole body contact manipulation control method and device for humanoid robot

Abstract

The present application relates to the technical field of humanoid robot control, and particularly relates to a humanoid robot whole-body contact manipulation control method and device, comprising: obtaining base action output by a frozen base strategy, residual original action output by a trainable residual strategy, and real-time tracking error of each joint; obtaining high-pass residual of the residual original action through first-order high-pass filtering; calculating a gating coefficient to obtain a gating residual through a gating function according to the tracking error; dynamically calculating an upper bound of the residual amplitude according to a sliding average of a training return, symmetrically truncating the gating residual to obtain a final residual, and superimposing the final residual to the base action to obtain a final control action. The present application is suitable for a humanoid robot whole-body contact manipulation task, can decouple the residual and the base strategy in the frequency domain, realize the residual intervention on demand, dynamically adjust the residual amplitude according to the training performance, and solve the problems of frequency domain coupling, residual always activated and fixed amplitude in the existing residual mode.

Description

Technical Field

[0001] This invention relates to the field of humanoid robot control technology, and in particular to a method and apparatus for full-body contact manipulation and control of a humanoid robot. Background Technology

[0002] Humanoid robot control involves whole-body motion control, aiming to coordinate the robot's gait stability, limb movements, and external interactions within a unified framework, enabling the robot to perform a series of actions such as walking, manipulation, and contact. With advancements in the performance of high-degree-of-freedom electrically driven joints and torque-controllable actuators, as well as the widespread application of deep reinforcement learning in robot control, learning-based motion tracking and residual correction are gradually becoming one of the main technical approaches to achieving whole-body control of humanoid robots.

[0003] In contact-intensive manipulation tasks such as carrying boxes, moving chairs, and picking up objects while kneeling or squatting, robots not only need to maintain their dynamic balance and stable gait, but also need to establish and maintain precise contact relationships with external objects during movement. These tasks present a dual challenge to control strategies: on the one hand, robots need to maintain the stability of low-frequency periodic movements at the gait level to avoid falls or gait disturbances caused by disturbances; on the other hand, robots need to have the ability to quickly and precisely correct high-frequency information such as contact forces and pose deviations at the manipulation level. Therefore, the key technical challenge in the full-body contact manipulation control of humanoid robots is how to balance low-frequency gait stability and high-frequency manipulation precision within the same control framework.

[0004] In the full-body control of humanoid robots based on residual reinforcement learning, a motion tracking policy is typically trained first as a base policy and then frozen. A trainable residual policy is then superimposed on the base policy to obtain contact-level manipulation capabilities. In existing technologies, the raw output of the residual policy is usually truncated by a fixed amplitude and directly superimposed on the base motion. The residual is injected across the entire frequency band, and its low-frequency components couple with the low-frequency periodic gait already learned by the base, which can easily cause joint oscillations and training failures. The residual remains active throughout the entire control round, and even if the base policy can track the reference motion well, the residual continues to inject disturbances, resulting in torque energy consumption and non-smooth motion. The residual truncation amplitude depends on a fixed value set manually and cannot be adaptively adjusted with the training process. If the value is set too high, it can easily destroy the base policy in the early stages of training; if the value is set too low, it will limit the residual's correction ability in complex contact tasks in the later stages of training. Although existing technologies have introduced mechanisms such as frequency separation, performance signal gating, or training step-driven amplitude modulation, the cutoff parameter is a fixed value and cannot be adjusted adaptively. Amplitude modulation driven by training steps cannot perceive training quality. None of the above mechanisms have systematically solved the problems of frequency domain coupling, residual constant activation, and fixed amplitude.

[0005] Therefore, there is an urgent need for a humanoid robot full-body contact manipulation control method that can systematically solve the problems of frequency domain coupling, constant residual activation, and fixed amplitude. This method can decouple the residual from the base strategy in the frequency domain, enable the residual to be intervened on demand, and allow the residual amplitude to be dynamically adjusted according to the training performance. Summary of the Invention

[0006] The purpose of this invention is to provide a method and device for full-body contact manipulation and control of a humanoid robot, so as to solve the problems in the prior art where residuals and base strategies are coupled in the frequency domain, residuals cannot be intervened as needed, and residual amplitudes cannot be adaptively adjusted.

[0007] The technical solution of this invention is: a method for full-body contact manipulation and control of a humanoid robot, comprising: The system acquires the base motion output by the frozen base strategy, the residual raw motion output by the trainable residual strategy, and the real-time tracking error for each joint at the current moment. The residual raw motion is filtered using a learnable first-order high-pass filter operator for each joint to obtain the high-pass residual. Based on the real-time tracking error for each joint, a gating coefficient is calculated using a differentiable gating function for each joint. The gating coefficient is then multiplied element-wise by the high-pass residual to obtain the gated residual. The moving average of the training reward is acquired, and the upper bound of the residual amplitude is dynamically calculated based on this average. The gated residual is symmetrically truncated using this upper bound to obtain the final residual. The final residual is then superimposed on the base motion to obtain the final control motion, and the final control motion is sent to the actuators of each joint of the humanoid robot. Furthermore, the cutoff parameter of the joint-by-joint learnable first-order high-pass filter operator is obtained by mapping the learnable parameters to the sigmoid function, and the learnable parameters are updated end-to-end through reinforcement learning.

[0008] Furthermore, the first-order high-pass filter operator maintains the low-pass state at each joint, and updates the low-pass state by weighted recursion of the low-pass state and the original residual action based on the cutoff parameter. The high-pass residual is the original residual action minus the updated low-pass state.

[0009] Furthermore, the joint-by-joint differentiable gating function is a sigmoid function, and the gating coefficient is calculated based on the maximum value of the gating coefficient, the sigmoid function, the gating sharpness, the gating threshold, and the real-time tracking error of the joint.

[0010] Furthermore, the gating sharpness and the gating threshold are joint-by-joint learnable positive parameters.

[0011] Furthermore, the upper bound of the residual amplitude is calculated based on the minimum value of the upper bound of the residual amplitude, the maximum value of the upper bound of the residual amplitude, the normalized ratio between the moving average of the training reward and the reference reward of the base policy running alone; wherein, the normalized ratio is obtained by cropping the ratio of the moving average of the training reward to the reference reward, and the value range is [0,1].

[0012] Furthermore, the moving average of the training reward is calculated based on the exponential moving average decay coefficient and the average round reward of the current iteration.

[0013] Furthermore, the maximum upper bound of the residual amplitude is constrained by a predetermined safety boundary.

[0014] Furthermore, the joint-by-joint low-pass state maintained by the first-order high-pass filter operator is cleared to zero when the control round is reset.

[0015] This invention also provides a humanoid robot full-body contact manipulation and control device, comprising: a motion acquisition module for acquiring the base motion output by the frozen base strategy at the current moment, the residual raw motion output by the trainable residual strategy, and the real-time tracking error for each joint; a frequency-aware residual module for filtering the residual raw motion through a learnable first-order high-pass filter operator for each joint to obtain a high-pass residual; a confidence-gated fusion module for calculating a gating coefficient based on the real-time tracking error for each joint using a differentiable gating function for each joint, and multiplying the gating coefficient element-wise with the high-pass residual to obtain a gated residual; an adaptive amplitude course module for acquiring the moving average of the training reward, dynamically calculating the upper bound of the residual amplitude based on the average, and symmetrically truncating the gated residual with the upper bound of the residual amplitude to obtain a final residual; and a motion synthesis and output module for superimposing the final residual onto the base motion to obtain a final control motion, and sending the final control motion to the actuators of each joint of the humanoid robot.

[0016] Compared with the prior art, the advantages of the present invention are: (1) By using a learnable high-pass filter for each joint, the low-frequency periodic gait already learned by the base is no longer affected by the coupling interference of the residual low-frequency components, reducing the continuous bias of the residual low-frequency components on the periodic movements of the base. Combined with gating fusion and amplitude limiting, the risk of joint oscillation and training instability is reduced. The cutoff parameters are updated end-to-end through reinforcement learning, so that each joint can automatically learn different cutoff parameters according to its own dynamic characteristics and task requirements, without the need for manual setting and more in line with the needs of each joint.

[0017] (2) The residual is intervened on demand by using confidence gating fusion. The residual is only intervened when the tracking error of each joint is large, and the intervention is reduced when the tracking error is small. This avoids the situation where the residual is still injected when the base can track the reference motion well, thus reducing torque energy consumption and motion jitter.

[0018] (3) By using an amplitude course coupled with training rewards, the upper bound of the residual amplitude is dynamically adjusted with the moving average of the training rewards, thereby releasing the correction capability required to complete complex contact tasks. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 A flowchart of a full-body contact manipulation and control method for a humanoid robot provided by the present invention; Figure 2 A structural block diagram of a humanoid robot full-body contact manipulation and control device provided by the present invention; Figure 3 This is a schematic diagram of the initial standing stage of the chair-moving task provided by the present invention; Figure 4 A schematic diagram illustrating the bending-to-approach-to-chair stage of the chair-moving task provided by the present invention; Figure 5 A schematic diagram illustrating the chair-lifting and standing phase of the chair-moving task provided by the present invention; Figure 6 A schematic diagram illustrating the bending-over and placing stage of a chair-moving task provided by the present invention; Figure 7 This is a schematic diagram illustrating the standing phase after the chair-moving task is completed, as provided by the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to specific embodiments: It should be noted that the term "acquisition" generally refers to the process of obtaining data or information, which can be achieved through direct acquisition by hardware or by reading stored data. The term "based on" generally refers to a description of relationships that are processed, calculated, deduced, or judged using certain data, conditions, rules, or models as input or basis, and does not exclude the simultaneous processing in conjunction with other data, conditions, rules, or models. The term "comprising" is intended to cover non-exclusive inclusion, meaning that a process, method, system, or apparatus that includes a series of elements is not limited to the explicitly listed elements, but may also include other elements not explicitly listed or inherent. The term "at least" is used to indicate a minimum quantity or level of limitation, and does not exclude situations where there is more than that minimum quantity or level.

[0021] The "base strategy" in this invention refers to a motion tracking strategy that is frozen after pre-training convergence and no longer participates in subsequent training, used to provide gait maintenance and basic motion imitation capabilities for humanoid robots. The "residual strategy" in this invention refers to a trainable strategy superimposed on the frozen base strategy; its output, after processing, is added to the base action, used to provide contact-level manipulation correction capabilities for humanoid robots. The "residual raw action" in this invention refers to the forward output of the residual strategy network, without any post-processing. The "high-pass residual" in this invention refers to the result of high-frequency components retained after high-pass filtering of the residual raw action. The "gated residual" in this invention refers to the result of element-wise multiplication of the high-pass residual by gating coefficients. The "end-to-end update" in this invention refers to incorporating the learnable parameters and the network parameters of the residual strategy network into the same optimization framework and performing joint updates based on the training objective of the residual strategy. The "reference motion frame" in this invention refers to the target value of the desired joint motion trajectory at the current moment, pre-recorded or generated. In this invention, "round" refers to a complete interactive process in reinforcement learning, from the robot's initial state to the triggering of a termination condition. "Privileged information" refers to environmental state information that is available during the training phase but not during the deployment phase.

[0022] In one embodiment of the present invention, the pedestal policy used is derived from teacher-student distillation training. In a simulation environment, a teacher policy that can access privileged information such as future reference frames and truth physics states is first trained using reinforcement learning. Then, it is distilled into a student policy that is observable only by the ontology through behavioral cloning. After the student policy training converges, it is frozen and used as the pedestal policy in one embodiment of the present invention.

[0023] This invention proposes a full-body contact manipulation control method for humanoid robots. It employs a combination of three processing operators: frequency-aware residual, confidence-gated fusion, and adaptive amplitude curriculum. The original output of the residual strategy is sequentially subjected to high-pass filtering, gating fusion, and adaptive amplitude limiting, and then superimposed onto the base motion. This decouples the residual from the base in the frequency domain, adaptively adjusts the gating residual according to the tracking error, and dynamically adjusts the residual amplitude with training rewards.

[0024] Figure 1 A flowchart illustrating a full-body contact manipulation and control method for a humanoid robot provided by this invention. Figure 1As shown, the process of determining the control action in this exemplary embodiment includes at least the following steps: S1, acquiring robot body observations, reference motion frames, and object state observations, inputting a frozen base policy and a trainable residual policy respectively, to obtain the base action, the residual original action, and the joint-by-joint tracking error; S2, applying a frequency-aware residual operator to the residual original action, and outputting a high-pass residual; S3, applying a confidence-gated fusion operator to the high-pass residual based on the joint-by-joint real-time tracking error, and outputting a gated residual; S4, acquiring the moving average of the training reward, applying an adaptive amplitude course operator to the gated residual, and outputting the final residual; S5, synthesizing the base action and the final residual; S6, sending the synthesized control action to the actuators of each joint of the humanoid robot. The above exemplary embodiment will be described in detail below.

[0025] In step S1, each control step acquires robot body observations, a reference motion frame, and object state observations. The robot body observations and the reference motion frame are then input into the base freeze strategy to obtain the base motion. The robot body observations, reference motion frames, and object state observations are input into the trainable residual policy to obtain the residual original motion. Simultaneously, the reference joint position is read from the environment. and actual joint position The joint-by-joint real-time tracking error was calculated. .like Figure 1 As shown, the output of the frozen base strategy The output is sent directly to S5 action synthesis, bypassing the processing from S2 to S4; it can train residual policies. Then it is sent to S2 for residual processing.

[0026] In step S2, the original action of the residual is performed. Apply a frequency-aware residual operator to output a high-pass residual. First, perform the original action on the residual. Perform a first-order low-pass recursion, extract the low-frequency components, and maintain them as low-pass states. Then, the original residual action is... With the low-pass state Subtraction yields the high-pass residual that retains only the high-frequency components. Cutoff parameters It is applied to each joint component in an element-wise multiplication manner. These are learnable parameters, updated end-to-end through reinforcement learning. The low-pass state is cleared to zero when the control round is reset to ensure that the filter starts from zero at the beginning of each round.

[0027] In one implementation, the low-pass state The update method is as follows: ,in For element-wise multiplication, This is the low-pass state of the previous control step. Initialize to So that the initial cutoff parameter is ,satisfy ,in It is the inverse function of the sigmoid function.

[0028] In step S3, a confidence-gated fusion operator is applied to the high-pass residual to output the gated residual. This is used for real-time joint-by-joint error tracking. For input, via Gating function calculation of joint-by-joint gating coefficients The gating coefficient With the aforementioned high-pass residual Element-wise multiplication yields the gated residual. The gating coefficient This indicates the level of confidence in the residual corrections for each joint at the current moment.

[0029] In one implementation, the joint-by-joint gating coefficient for: ,in yes function, It is the maximum value of the gating coefficient. It's about gating sharpness. It refers to the gating threshold and gating sharpness. Used to control the gating coefficient at the gating threshold. The steepness of the nearby transition, and All are joint-by-joint learnable positive parameters, in this embodiment Initialization is approximately 10. The initial value is approximately 0.1.

[0030] In step S4, the gated residual is... Apply the adaptive amplitude course operator and output the final residual. First, based on the exponential moving average decay coefficient... and the average round return of the current iteration The moving average of the training reward is updated using an exponential moving average method. Then, based on the aforementioned moving average value... Reference return when operating independently of the base strategy The ratio is used to calculate the upper bound of the residual magnitude in the current iteration. Finally, the upper bound of the residual amplitude is used. For the gated residual Perform symmetric truncation to obtain the final residual. ,in, It is a symmetric truncation function, when Greater than Time output ,when Less than Time output ,when Time output itself.

[0031] In one implementation, the moving average for: The upper bound of the residual amplitude for: Among them, the normalized ratio , The function is used to... Clip to the interval [0,1]. It is the minimum value of the upper bound of the residual amplitude. It is the maximum value of the upper bound of the residual amplitude, which satisfies the following in this embodiment. , , Reference Returns It can be measured offline, when When the value is zero or cannot be measured, the normalized ratio can be calculated using a self-normalized preheating mechanism. ,in, This indicates that the self-normalized preheating mechanism uses the moving average itself as the normalization benchmark, and the preheating refers to the preheating process described in the reference return. This mechanism can be used during the previous transition phase.

[0032] In one implementation, the predetermined safety boundary constraints are determined based on Input-to-State Stability (ISS) theory. Specifically, Constrained by the ISS security community, and meets ,in, It is the stability margin of the base closed-loop system, representing the maximum allowable range of the system's deviation from equilibrium after being disturbed; This is the Lipschitz constant of the base closed-loop system to motion disturbances, reflecting the sensitivity of the system output to changes in motion disturbances. The safety boundary constraint can be determined through theoretical derivation, system identification, or experimental estimation, and the specific determination method is not limited to the aforementioned ISS method.

[0033] Due to the moving average of training rewards It changes with training performance; when training performance does not improve... Residual amplitude remains at a low level, upper bound Correspondingly maintained at Nearby; as training performance continues to improve Gradually increase, upper limit of residual amplitude from Towards Smooth expansion. Thus, the upper bound of the residual amplitude is coupled with the moving average of the training reward, enabling the upper bound of the residual amplitude to be dynamically adjusted according to the training reward.

[0034] In step S5, the base is activated. With the final residual Perform motion synthesis. Final control motion. Simultaneously record the cutoff parameter The mean, the gate coefficient The mean and the upper bound of the residual amplitude Intermediate values ​​are used for training monitoring and analysis.

[0035] In step S6, the final control action is performed. The data is sent to the actuators of each joint of the humanoid robot, and the PD (proportional-derivative) controller then... Drive joint movement to complete the action of the current control step.

[0036] It should be noted that the three processing operators are independent of each other and can be enabled or disabled individually. When all three operators are disabled, this invention degenerates into the existing residual scheme with fixed amplitude truncation.

[0037] The following is a feasible example of the present invention. For ease of understanding, a 23-DOF humanoid robot performing a kneeling picking task is used as an example. The control frequency is set to 50 Hz, and the motion dimensions are... The number of parallel environments for large-scale parallel simulation training This method is also applicable to contact-level tasks such as squatting to pick up objects, walking to move boxes, and moving chairs, as well as humanoid characters with other degrees of freedom. Based on steps S1 to S6 above, a feasible pseudocode implementation is shown in Table 1 below: Table 1 Pseudocode for Residual Synthesis

[0038] Wherein, `use_far` indicates whether the frequency-aware residual operator is enabled, `use_cgf` indicates whether the confidence-gated fusion operator is enabled, and `use_aac` indicates whether the adaptive amplitude curriculum operator is enabled; `a_base` is the base action output by the frozen base policy, with dimensions of... a_raw represents the residual raw action output by the trainable residual policy, with a dimension of . ;e represents the joint-by-joint real-time tracking error, with dimensions of lpf_prev and lpf_new are the joint-wise low-pass states maintained by the frequency-aware residual operator, with dimensions of [missing information]. beta is the joint-by-joint learnable cutoff parameter, with dimensions of . ; eps is the upper limit of the residual amplitude; a is the final residual.

[0039] In the pseudocode above, line 1 defines the residual synthesis function `compose`, with input parameters including the residual raw action `a_raw`, the base action `a_base`, the joint-by-joint real-time tracking error `e`, the low-pass state `lpf_prev` of the previous control step, the frequency-aware residual operator enable switch `use_far`, the confidence-gated fusion operator enable switch `use_cgf`, and the adaptive amplitude course operator `use_aac`. Line 2 initializes the final residual `a` to the residual raw action. Line 3 initializes the low-pass state `lpf_new` of the current control step to the low-pass state of the previous control step. Lines 4 to 7 correspond to the frequency-aware residual processing in step S2. When `use_far` is true, the cutoff parameter `beta` is obtained based on the learnable parameter `phi` through the sigmoid function mapping, the low-pass state `lpf_new` is updated, and the updated value is subtracted from the residual raw action. The low-pass state is then used to obtain the high-pass residual; lines 8 to 10 correspond to the confidence-gated fusion processing in step S3. When use_cgf is true, the gating coefficient alpha is calculated using the sigmoid function based on the maximum gating coefficient alpha_max, the logarithmically parameterized gating sharpness k, and the gating threshold theta. This gating coefficient is then multiplied element-wise with the high-pass residual to obtain the gating residual; lines 11 to 12 correspond to the adaptive amplitude course processing in step S4. When use_aac is true, the amplitude course scheduler obtains the upper bound of the current residual amplitude eps. Otherwise, a fixed amplitude eps_fixed is used as eps, and the gating residual is symmetrically truncated using eps to obtain the final residual; line 13 corresponds to the action synthesis in step S5. The final residual is added to the base action to obtain the final control action, and the updated low-pass state is returned for use in the next control step.

[0040] Based on the above method embodiments, the present invention also provides a humanoid robot full-body contact manipulation and control device. Figure 2 This invention provides a structural block diagram of a humanoid robot's full-body contact manipulation and control device. (See diagram below.) Figure 2 As shown, the device includes at least the following modules: motion acquisition module, frequency sensing residual module, confidence gating fusion module, adaptive amplitude course module, and motion synthesis and output module. The motion acquisition module transmits the base motion to the motion synthesis and output module, and finally synthesizes the final result and sends it to the actuators of each joint of the humanoid robot.

[0041] The motion acquisition module is used to acquire the base motion output by the frozen base policy, the residual raw motion output by the trainable residual policy, and the joint-by-joint real-time tracking error.

[0042] The frequency-aware residual module is used to perform joint-by-joint learnable first-order high-pass filtering on the original residual motion to obtain the high-pass residual.

[0043] The confidence-gated fusion module is used to calculate the joint-by-joint gating coefficients based on the real-time tracking error, and then multiply the gating coefficients element-by-element with the high-pass residuals to obtain the gating residuals.

[0044] The adaptive amplitude course module is used to dynamically calculate the upper bound of the residual amplitude constrained by a predetermined safety boundary based on the moving average of the training reward, and to symmetrically truncate the gated residual to obtain the final residual.

[0045] The motion synthesis and output module is used to superimpose the final residual onto the base motion to obtain the final control motion and send it to the actuators of each joint of the humanoid robot.

[0046] Figure 3 This is a schematic diagram of the initial standing stage of the chair-moving task provided by the present invention. Figure 4 This is a schematic diagram illustrating the bending-over approach stage of the chair-moving task provided by the present invention. Figure 5 This is a schematic diagram illustrating the chair-lifting and standing phase of a chair-moving task provided by the present invention. Figure 6 This is a schematic diagram illustrating the bending over and placing stage of a chair-moving task provided by the present invention. Figure 7 This is a schematic diagram illustrating the standing phase after the chair-moving task is completed, as provided by the present invention. Figures 3 to 7 The execution process of the present invention in a real contact-level task is shown in a time series manner.

[0047] exist Figure 3 In the initial standing phase shown, the robot is in an upright posture. The base strategy tracks the reference standing posture well, the tracking error of each joint approaches zero, the gating coefficient decreases, the residual contribution decreases, and the system operates primarily with a pure base strategy. The frequency-aware residual operator maintains a low-pass state in preparation for subsequent activation, and the upper bound of the amplitude is at a stable value after training convergence.

[0048] exist Figure 4 As shown in the bending-to-chair approach phase, the robot performs a bending motion, causing the torso and leg joints to deviate from the reference standing posture, increasing the tracking error. A high-pass filter removes low-frequency components from the residuals, ensuring that the low-frequency motion during the bending process is primarily driven by the base strategy. Only high-frequency fine-position adjustment components are injected into the residuals, without interfering with the low-frequency stability of the bending process. The gating coefficient gradually activates the residual correction of the corresponding joints as the error increases, enabling on-demand intervention during the contact preparation phase. The upper bound of the residual amplitude, within a safe range, provides correction capability for precise contact.

[0049] exist Figure 5 As shown in the stage of picking up the chair and returning to a standing position, the robot establishes contact with the chair, picks it up, and then returns to a standing posture. The additional gravity disturbance of carrying the chair causes the tracking error of some joints to remain high. Steady-state gravity compensation is mainly handled by the base strategy and PD controller, while the residual only provides dynamic correction for contact establishment and disengagement transients. The high-pass filter continuously filters the low-frequency components of the residual to prevent the holding disturbance from affecting the base gait through the low-frequency components of the residual. The gating coefficient maintains the activation of the corresponding joint residual correction to ensure the holding balance, and the upper bound of the residual amplitude provides the correction magnitude to resist the holding disturbance.

[0050] exist Figure 6 As shown in the bending-over-to-place-chair phase, the robot bends over again to place the chair back on the ground. High-pass filtering reduces the persistent bias of low-frequency residual components on the base gait, ensuring low-frequency stability during placement. The gating coefficient adaptively adjusts the residual strength based on the real-time tracking error of each joint, achieving precise placement.

[0051] exist Figure 7 After the chair is placed, the robot returns to its standing position. Once the chair is in place, the robot disengages and resumes its initial upright posture. The tracking errors of each joint approach zero, the gating coefficient automatically decreases, the residual correction intervention level decreases, and the system returns to a pure base-strategy-dominated state. Figure 3 The initial state is symmetrically closed.

[0052] The above execution process demonstrates the coordinated effect of the three operators: the frequency-aware residual operator ensures gait stability by filtering out low-frequency components of the residual; the confidence-gated fusion operator automatically activates during the contact phase and suppresses residuals during the disengagement phase by utilizing changes in real-time tracking errors; and the adaptive amplitude curriculum operator dynamically adjusts the correction magnitude of the residuals based on training performance. The cascaded interaction of these three operators reduces the risk of gait disorder or control instability caused by residual aggregation throughout the entire task cycle.

[0053] The above embodiments are merely illustrative of the technical concept and features of the present invention, intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and should not be construed as limiting the scope of protection of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the above embodiments should be considered exemplary rather than restrictive, and all variations within the meaning and scope of equivalent elements of the technical features of the present invention are included within the present invention.

Claims

1. A method for full-body contact manipulation and control of a humanoid robot, characterized in that, include: It obtains the base motion output by the frozen base policy at the current moment, the residual raw motion output by the trainable residual policy, and the real-time tracking error for each joint. The original residual action is filtered by a joint-by-joint learnable first-order high-pass filter operator to obtain the high-pass residual; Based on the real-time tracking error of each joint, the gating coefficient is calculated using a differentiable gating function of each joint, and the gating coefficient is multiplied element-wise with the high-pass residual to obtain the gating residual; Obtain the moving average of the training reward, dynamically calculate the upper bound of the residual amplitude based on the average, and symmetrically truncate the gated residual with the upper bound of the residual amplitude to obtain the final residual; The final residual is superimposed on the base action to obtain the final control action, and the final control action is sent to the actuators of each joint of the humanoid robot.

2. The method for full-body contact manipulation and control of a humanoid robot according to claim 1, characterized in that, The cutoff parameter of the joint-by-joint learnable first-order high-pass filter operator is obtained by mapping the learnable parameters to the sigmoid function, and the learnable parameters are updated end-to-end through reinforcement learning.

3. The method for full-body contact manipulation and control of a humanoid robot according to claim 2, characterized in that, The first-order high-pass filter operator maintains the low-pass state for each joint, and updates the low-pass state by weighted recursion of the low-pass state and the original residual action based on the cutoff parameter. The high-pass residual is the original residual action minus the updated low-pass state.

4. The method for full-body contact manipulation and control of a humanoid robot according to claim 1, characterized in that, The joint-by-joint differentiable gating function is a sigmoid function, and the gating coefficient is calculated based on the maximum value of the gating coefficient, the sigmoid function, the gating sharpness, the gating threshold, and the real-time tracking error of the joint.

5. The method for full-body contact manipulation and control of a humanoid robot according to claim 4, characterized in that, The gating sharpness and the gating threshold are joint-by-joint learnable positive parameters.

6. The method for full-body contact manipulation and control of a humanoid robot according to claim 1, characterized in that, The upper bound of the residual amplitude is calculated based on the minimum value of the upper bound of the residual amplitude, the maximum value of the upper bound of the residual amplitude, the normalized ratio between the moving average of the training reward and the reference reward of the base policy running alone; The normalized ratio is obtained by cropping the ratio of the moving average of the training reward to the reference reward, and its value ranges from [0,1].

7. The method for full-body contact manipulation and control of a humanoid robot according to claim 6, characterized in that, The moving average of the training reward is calculated based on the exponential moving average decay coefficient and the average reward of the current iteration.

8. The method for full-body contact manipulation and control of a humanoid robot according to claim 6, characterized in that, The maximum upper bound of the residual amplitude is constrained by a predetermined safety boundary.

9. A method for full-body contact manipulation and control of a humanoid robot according to claim 3, characterized in that, The joint-by-joint low-pass state maintained by the first-order high-pass filter operator is cleared to zero when the control round is reset.

10. A humanoid robot full-body contact manipulation and control device, characterized in that, include: The motion acquisition module is used to acquire the base motion output by the frozen base policy at the current moment, the residual raw motion output by the trainable residual policy, and the real-time tracking error for each joint. The frequency-aware residual module is used to filter the original residual action through a joint-by-joint learnable first-order high-pass filter operator to obtain the high-pass residual. The confidence-gated fusion module is used to calculate the gating coefficients based on the joint-by-joint real-time tracking error using a joint-by-joint differentiable gating function, and then multiply the gating coefficients element-by-element with the high-pass residual to obtain the gating residual. The adaptive amplitude course module is used to obtain the moving average of the training reward, dynamically calculate the upper bound of the residual amplitude based on the average, and symmetrically truncate the gated residual with the upper bound of the residual amplitude to obtain the final residual. The motion synthesis and output module is used to superimpose the final residual onto the base motion to obtain the final control motion, and to send the final control motion to the actuators of each joint of the humanoid robot.

Images