Robot control method and device, humanoid robot and storage medium

Abstract

The application discloses a robot control method and device, a humanoid robot and a storage medium, and relates to the technical field of robot control. The method comprises the following steps: constructing a digital twin robot matched with the hardware parameters and physical characteristics of a target humanoid robot in a simulation environment, and constructing an adaptive joint loss function for jointly training a high-level planner and a bottom controller. The high-level planner is provided with a differentiable safe projection layer, and the output high-level instruction is boundary-constrained according to bottom physical execution limit information. The joint loss function comprises an instruction executability loss item quantifying the degree to which the instruction exceeds the physical limit and a planning execution distribution alignment loss item measuring the deviation between the feasible range of the planning instruction and the executable ability range of the bottom layer. The parameters of the high-level planner and the bottom controller are synchronously updated through the loss function, a hierarchical control strategy is obtained, and the hierarchical control strategy is deployed to the real machine, so that the real machine always keeps coordination under the physical constraint, and the task success rate and the execution stability are significantly improved.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to robot control methods, robot control devices, humanoid robots, and storage media. Background Technology

[0002] In the hierarchical control of humanoid robots, the high-level planner and the low-level controller are usually optimized independently. When generating target instructions such as base movement speed and end-effector pose, the high-level planner often lacks physical execution limit information such as the upper limit of joint torque, foot friction cone constraint, and end-effector reachable space boundary that the low-level controller can actually provide.

[0003] Therefore, when deployed in a real physical environment, the motion and operation commands required by the high-level planner often exceed the range of commands that the low-level controller can actually implement while maintaining the overall balance and stable contact. This leads to phenomena such as torque saturation, foot slippage, or excessive end-effector pose deviation in the low-level controller during the tracking process. This directly disrupts the coordination between the humanoid robot's base movement, hand operation, and overall balance, significantly reducing the mission success rate. Summary of the Invention

[0004] The main objective of this application is to provide a robot control method, a robot control device, a humanoid robot, and a storage medium, aiming to solve the technical problem in existing solutions where the high-level planner lacks information about the actual physical execution limits that the low-level controller can provide, resulting in poor coordination of the humanoid robot and a low task success rate.

[0005] To achieve the above objectives, this application proposes a robot control method, the method comprising: In a simulation environment, a digital twin robot is constructed that matches the hardware parameters and physical characteristics of the target humanoid robot; An adaptive joint loss function is constructed for jointly training the high-level planner and the low-level controller of the digital twin robot. The high-level planner includes a differentiable safe projection layer for boundary constraints on the high-level planning instructions output by the high-level planner based on the physical execution limit information of the low-level controller. The adaptive joint loss function includes an instruction executability loss term for quantifying the degree to which the high-level planning instructions exceed the physical execution limit information, and a planning execution distribution alignment loss term for quantifying the deviation between the feasible range of the high-level planning instructions and the capability range of the low-level executable instructions. The parameters of the high-level planner and the low-level controller are updated synchronously using the adaptive joint loss function, and a hierarchical control strategy is obtained based on the synchronously updated parameters. The hierarchical control strategy is deployed to the target humanoid robot to perform collaborative control of the target humanoid robot in a real physical environment.

[0006] In one embodiment, the network structure of the high-level planner of the digital twin robot is: a state-scene condition encoder, a diffusion time-step modulation module, a motion branch denoiser, an operation branch denoiser, a coupling gated residual module, and a differentiable safe projection layer. The state-scene condition encoder consists of 6 stacked condition coding layers. The diffusion time step modulation module is located after the state-scene condition encoder; The motion branch denoising unit is composed of four sequentially stacked motion branch denoising layers, and the operation branch denoising unit is composed of four sequentially stacked operation branch denoising layers. The coupling gated residual module is disposed between the second and third motion branch denoising layers of the motion branch denoising device and between the second and third operation branch denoising layers of the operation branch denoising device. The differentially secure projection layer is disposed after the motion branch denoiser and the operational branch denoiser.

[0007] In one embodiment, in the high-level planner, The state-scene condition encoder is used to receive conditional input data and output conditional coding features. The conditional input data includes the state data of the target humanoid robot, external environment data, diffusion time step, foot contact state, and task stage code. The diffusion time step modulation module is used to take the diffusion time step as input, output scaling coefficient and bias coefficient, and input the scaling coefficient and bias coefficient to each motion branch denoising layer and each operation branch denoising layer respectively, so as to modulate the denoising features of each denoising layer. The conditional coding features are respectively input into the motion branch denoiser and the operation branch denoiser, and the denoising results of each are concatenated to obtain the predicted high-level planning instruction. Specifically, after every two denoising layers of both the motion branch denoiser and the operation branch denoiser, the hidden states of the motion branch and the operation branch, along with the conditional coding features, are input into the coupled gated residual module to obtain updated hidden states of the motion branch and the operation branch. The updated hidden states of the motion branch and the operation branch are then input into the next denoising layer of the motion branch denoiser and the next denoising layer of the operation branch denoiser, respectively, to obtain their respective denoising results. The hidden states of the motion branch are obtained by diffusion denoising performed by the motion branch denoiser, and the hidden states of the operation branch are obtained by diffusion denoising performed by the operation branch denoiser. After the predicted high-rise planning instructions are input into the differentiable safe projection layer for boundary constraints, the actual high-rise planning instructions are output.

[0008] In one embodiment, the step of constructing an adaptive joint loss function for jointly training the high-level planner and the low-level controller of the digital twin robot includes: The diffusion denoising loss term is determined based on the consistency between the noise predicted by the diffusion network in the high-level planner and the actual noise. Based on the degree of violation of the underlying physical constraints fed back when the underlying controller executes the high-level planning instructions output by the high-level planner, the instruction executability loss item is determined, wherein the underlying physical constraints include torque saturation, foot friction cone overrun, and end reachable space overrun. The bottom tracking loss term is determined based on the deviation between the trajectory corresponding to the high-level planning instruction output by the high-level planner and the actual execution trajectory of the bottom-level controller. Based on the update step size limit of the underlying controller during policy updates, determine the pruning target loss term; The overall stability loss term is determined based on the minimum distance from the projection point of the center of mass of the digital twin robot on the horizontal plane along the direction of gravity to each side of the supporting polygon, the torso angular velocity and foot sliding velocity of the digital twin robot. Based on the deviation between the feasible range of the high-level planning instructions output by the high-level planner and the capability range of the low-level executable instructions of the low-level controller, the planning execution distribution alignment loss term is determined. Based on the joint torques, torque change rates collected during the task execution of the digital twin robot, and the high-level planning instructions output by the high-level planner, the energy consumption smoothing loss term is determined. An adaptive joint loss function is constructed based on the diffusion denoising loss term, the instruction executability loss term, the underlying tracking loss term, the pruning target loss term, the overall system stability loss term, the planning execution distribution alignment loss term, and the energy consumption smoothing loss term.

[0009] In one embodiment, the step of synchronously updating the parameters of the high-level planner and the low-level controller using the adaptive joint loss function includes: During the execution of the high-level planning instructions output by the high-level planner by the low-level controller, the physical feedback quantity of the digital twin robot is obtained; Based on the physical feedback quantity, the weights of each loss term in the adaptive joint loss function are dynamically adjusted to obtain the adjusted weights; The adaptive joint loss function is updated with the adjusted weights, the loss value of the adaptive joint loss function is calculated, and the gradient of the loss value is backpropagated to the high-level planner and the low-level controller, and the parameters of the high-level planner and the low-level controller are updated synchronously.

[0010] In one embodiment, the physical feedback quantities include foot slip rate, joint torque saturation ratio, and end-effector tracking error; the adaptive joint loss function includes instruction executability loss term, low-level tracking loss term, overall system stability loss term, planning execution distribution alignment loss term, and energy consumption smoothing loss term; the step of dynamically adjusting the weights of each loss term in the adaptive joint loss function based on the physical feedback quantities includes: When the foot slip rate exceeds a preset slip rate threshold, the weight of the overall stability loss term is increased, and the weight of the underlying tracking loss term is decreased. When the proportion of joint torque saturation exceeds a preset saturation threshold, the weight of the instruction executability loss item is increased; After a preset number of consecutive task executions, if the end tracking error decreases multiple times consecutively and the task success rate does not increase compared to the previous time, the weight of the planning execution distribution alignment loss term is increased. When the task success rate reaches a preset success rate threshold and the joint energy consumption of the digital twin robot is higher than the baseline energy consumption, the weight of the energy consumption smoothing loss term is increased.

[0011] In one embodiment, after the step of deploying the hierarchical control strategy to the target humanoid robot, the following is included: Collect operational status data and task execution results of the target humanoid robot when it performs tasks in a real physical environment; Based on the running status data and the task execution results, the weights of each loss term in the adaptive joint loss function are optimized and adjusted; In the simulation environment, based on the optimized and adjusted adaptive joint loss function, the steps of synchronously updating the parameters of the high-level planner and the low-level controller through the adaptive joint loss function, and obtaining the hierarchical control strategy based on the synchronously updated parameters are executed to generate an iterative hierarchical control strategy, and the iterative hierarchical control strategy is redeployed to the target humanoid robot.

[0012] Furthermore, to achieve the above objectives, this application also proposes a robot control device, which includes: The building blocks are used to construct digital twin robots that match the hardware parameters and physical characteristics of the target humanoid robot in a simulation environment. The loss module is used to construct an adaptive joint loss function for jointly training the high-level planner and the low-level controller of the digital twin robot. The high-level planner includes a differentiable safe projection layer for boundary constraints on the high-level planning instructions output by the high-level planner based on the physical execution limit information of the low-level controller. The adaptive joint loss function includes an instruction executability loss term for quantifying the degree to which the high-level planning instructions exceed the physical execution limit information, and a planning execution distribution alignment loss term for quantifying the deviation between the feasible range of the high-level planning instructions and the capability range of the low-level executable instructions. The update module is used to synchronously update the parameters of the high-level planner and the low-level controller through the adaptive joint loss function, and obtain the hierarchical control strategy based on the synchronously updated parameters; The control module is used to deploy the hierarchical control strategy to the target humanoid robot and perform collaborative control of the target humanoid robot in a real physical environment.

[0013] In addition, to achieve the above objectives, this application also proposes a humanoid robot, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the robot control method described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the robot control method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the robot control method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: This application constructs a digital twin robot in a simulation environment that matches the hardware parameters and physical characteristics of a target humanoid robot (i.e., a real robot). This digital twin robot provides physical execution limit information, such as upper limits of joint torques, foot friction cone constraints, and end-effector reachable space boundaries, consistent with the real robot. An adaptive joint loss function is constructed for jointly training the high-level planner and the low-level controller of this digital twin robot. The high-level planner includes a differentiable, safe projection layer that constrains the high-level planning instructions output by the high-level planner based on the physical execution limit information of the low-level controller. This layer directly limits the corrections to base velocity, hand pose, and center of mass offset from the instruction generation stage to the executable envelope formed by the upper limits of joint torques, foot friction cones, center of mass support polygons, and end-effector reachable space, preventing high-level planning instructions from exceeding the physical implementation capabilities of the low-level controller. Simultaneously, the adaptive joint loss function includes an instruction executability loss term to quantify the degree to which high-level planning instructions exceed physical execution limit information, and a planning execution distribution alignment loss term to quantify the deviation between the feasible range of high-level planning instructions and the capability range of low-level executable instructions.

[0017] Because the adaptive joint loss function can directly incorporate real-time physical constraints from the feedback of the digital twin robot during computation, the high-level planning instructions generated by the high-level planner are gradually constrained to the actual executable range achievable by the low-level controller during the synchronous update of high-level planner and low-level controller parameters. This hierarchical control strategy, based on the synchronously updated parameters, fundamentally eliminates the instruction over-limit problem caused by the decoupling of high-level planning and low-level control in traditional solutions. Through joint optimization of planning instructions and execution capabilities, the resulting hierarchical control strategy, when deployed to the target humanoid robot, ensures that base movement, hand manipulation, and overall robot balance remain coordinated under physical constraints in a real physical environment. Therefore, it significantly improves the success rate and execution stability of humanoid robots performing long-term complex tasks in real physical environments. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating an embodiment of the robot control method of this application. Figure 1 ; Figure 2 A flowchart illustrating an embodiment of the robot control method of this application. Figure 2 ; Figure 3 A flowchart illustrating an embodiment of the robot control method of this application. Figure 3 ; Figure 4 This is a schematic diagram of the module structure of the robot control device of this application; Figure 5 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the robot control method of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] In the field of humanoid robot control, the entire control architecture is divided into two layers: a high-level planner and a low-level controller. The high-level planner operates at a lower frequency and is responsible for generating expected high-level planning instructions based on environmental observations and task phase information. These instructions include the robot's base velocity, end-effector poses, and center-of-gravity offset corrections. Its output is not joint torque, but rather reflects the robot's intention to "do" motion. The low-level controller operates at a higher frequency and is responsible for converting the high-level planning instructions output by the high-level planner into torque control quantities for each joint in real time. While tracking the high-level planning instructions, it maintains overall robot balance and stable foot contact, thus achieving precise execution of "how to do it." Functionally, the two form a clear hierarchical structure of decision-making and execution, rather than being integrated into a single policy network that directly outputs torque. During joint training, although the high-level planner and the low-level controller synchronously update their parameters through an adaptive joint loss function, they each have independent network structures and parameter configurations, and communicate using different operating frequencies and interface protocols, reflecting the "layered" characteristic.

[0025] Therefore, the hierarchical control strategy obtained in this application is essentially composed of the updated networks of the high-level planner and the low-level controller (i.e., the updated parameters combined with a fixed network structure). During joint training, the gradient generated by the adaptive joint loss function synchronously updates the network weights of the high-level planner and the low-level controller. After the update is completed, the updated parameters and their respective pre-set network structures jointly determine the mapping relationship between the two networks from input to output. This mapping relationship is the final hierarchical control strategy. The high-level planner network is responsible for mapping environmental observations into high-level planning instructions, and the low-level controller network is responsible for mapping high-level planning instructions into joint torque control quantities. The two do not require additional conversion steps. They only need to load the updated parameters into the corresponding network structures to be directly used for hierarchical collaborative control of digital twin robots or real humanoid robots.

[0026] It should be noted that the execution entity in this embodiment can be the onboard controller of the target humanoid robot itself, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, server, etc., or an embedded processing device or cloud computing platform capable of realizing the above functions. The following description uses a control device connected to the target humanoid robot (hereinafter referred to as the control device) as an example to illustrate this embodiment and the following embodiments.

[0027] Based on this, the embodiments of this application provide a robot control method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the robot control method of this application.

[0028] In one feasible embodiment, the application scenarios of the technical solution of this application can be: material handling in industrial manufacturing scenarios, autonomous door opening in service robot scenarios, and stair climbing in complex terrain, etc., which are long-term and complex operation tasks. The application objects are humanoid robots with mobile base, dual arms and full-body balance control requirements, such as the 29-DOF BoosterT1 humanoid robot.

[0029] In this embodiment, the robot control method includes steps S1 to S4: Step S1: In a simulation environment, construct a digital twin robot that matches the hardware parameters and physical characteristics of the target humanoid robot. It should be noted that in order to transfer the control strategy trained subsequently to the real robot with low bias, it is first necessary to build a virtual simulation environment that can accurately reflect the physical behavior of the real robot.

[0030] In this step, the control device uses Modelica (an object-oriented multi-domain physical modeling language) on the MWORKS platform to construct a 1:1 full-parameter digital twin model of the target humanoid robot.

[0031] Optionally, the simulation environment refers to a simulation platform that uses the Modelica standard multibody library, robot library, contact mechanics library and drive library to fully cover the rigid body linkage structure, joint servo drive characteristics, foot nonlinear contact model and overall dynamic balance characteristics of the robot with 20+ degrees of freedom. The average deviation rate between the simulation results and the actual machine's operating characteristics is less than 5%. This simulation environment is a high-fidelity simulation environment.

[0032] Optionally, constructing a digital twin robot that matches the hardware parameters and physical characteristics of the target humanoid robot refers to: building a 1:1 digital twin model using Modelica language based on the actual hardware parameters of the target humanoid robot (including link dimensions, mass, moment of inertia, joint angle limits, rotational speed limits, peak torque, static / dynamic friction coefficients, torque-rotation response bandwidth, etc.), and completing fidelity verification. Alternatively, a digital twin robot can be constructed solely based on the calibrated hardware parameters.

[0033] Optionally, fidelity verification involves importing the standard motion trajectory of the real robot into the digital twin robot (also known as a simulation model) for simulation. The simulation output is compared with the key data such as joint angles, torques, and foot displacements of the real robot. When the average deviation rate is less than 5%, the construction of the digital twin robot is confirmed to be complete.

[0034] Optionally, the control device also leverages the bidirectional fusion capabilities of MWORKS' Syslab (scientific computing environment) and Sysplorer (modeling environment) to build a real-time data interaction interface between Python and Modelica, supporting reinforcement learning algorithms for real-time command input, status data feedback, and batch parallel simulation scheduling of the digital twin robot.

[0035] Step S2: Construct an adaptive joint loss function for jointly training the high-level planner and the low-level controller of the digital twin robot. The high-level planner includes a differentiable safe projection layer for boundary constraints on the high-level planning instructions output by the high-level planner based on the physical execution limit information of the low-level controller. The adaptive joint loss function includes an instruction executability loss term for quantifying the degree to which the high-level planning instructions exceed the physical execution limit information, and a planning execution distribution alignment loss term for quantifying the deviation between the feasible range of the high-level planning instructions and the capability range of the low-level executable instructions. It should be noted that, in order to achieve a precise match between planning and execution and to prevent high-level instructions from exceeding the execution capabilities of the low-level controller, this step constructs a loss function that can simultaneously constrain planning quality and execution feasibility, and sets up a safe projection mechanism for the high-level planner.

[0036] Optionally, joint training refers to updating the parameters of both the high-level planner and the low-level controller simultaneously.

[0037] Optionally, the high-level planner is a motion-manipulation coupled gated diffusion network MMCG-DiT (Motion-Manipulation Coupled Gated Diffusion Transformer), which generates high-level control commands containing base linear velocity, angular velocity, hand 3D position, attitude angle, and center of mass offset correction based on the robot's body state, environmental observations, foot contact state, and task stage encoding.

[0038] Optionally, the underlying controller is a reinforcement learning controller based on the PPO (Proximal Policy Optimization) algorithm, used to convert high-level planning instructions into torque control quantities for each joint of the robot.

[0039] Optionally, the adaptive joint loss function refers to a function that includes multiple sub-loss terms, and the weights of the sub-loss terms can be dynamically adjusted according to simulation feedback.

[0040] Optionally, the high-level planner includes a differentiable safe projection layer, which is located at the output of MMCG-DiT. This layer is a differentiable mathematical mapping module used to limit the predicted base velocity, hand pose, and center of mass offset within the executable envelope of the lower-level controller based on the physical execution limit information of the lower-level controller.

[0041] Optionally, the physical execution limit information includes the upper limit of joint torque, the boundary of the foot friction cone, the range of the center of mass support polygon, and the reachable space at the end point. This information can be obtained through batch simulation using digital twins.

[0042] Optionally, the high-level planning instruction refers to the desired control quantity output by the high-level planner, specifically including the robot base linear velocity, angular velocity, three-dimensional position of both hands, attitude angle, and center of mass offset correction.

[0043] Optionally, boundary constraints refer to cropping or scaling the predicted values ​​output by the higher-level planner through a differentiable safe projection layer, so that they fall within a predefined safe and executable range, and the projection operation is differentiable, thereby allowing gradient backpropagation to the higher-level planner.

[0044] Optionally, the instruction executability loss term is a sub-term of the adaptive joint loss function, used to quantify the degree to which high-level planning instructions exceed the physical execution limits. Specifically, it consists of joint torque saturation, foot friction cone overrun, and end-effector reachability overrun.

[0045] Optionally, the planning execution distribution alignment loss is a sub-term of the adaptive joint loss function, which uses the maximum mean discrepancy (MMD) metric to quantify the deviation between the feasible range of high-level planning instructions and the capability range of low-level executable instructions.

[0046] Optionally, the feasible range of high-level planning instructions refers to the distribution of instructions constrained by the differentiable safety projection layer; the capability range of low-level executable instructions refers to the distribution of instructions that the low-level controller can actually stably track, which is obtained by collecting instruction-tracking error data during the operation of the low-level controller.

[0047] Step S3: Through the adaptive joint loss function, the parameters of the high-level planner and the low-level controller are updated synchronously, and a hierarchical control strategy is obtained based on the synchronously updated parameters. It should be noted that, in order to enable the output distribution of the planner and the execution capability distribution of the controller to converge in a coordinated manner, this step will backpropagate the gradient of the adaptive joint loss function to both modules simultaneously, thereby achieving end-to-end joint optimization.

[0048] Optionally, synchronous update means that in the same training iteration, the parameters of the high-level planner and the parameters of the low-level controller are updated simultaneously according to the gradient of the same adaptive joint loss function.

[0049] Optionally, the parameters include: the weights and biases of the state-scene condition encoder in the high-level planner MMCG-DiT, the weights and biases of the motion branch denoiser, the weights and biases of the operation branch denoiser, the gating factor weights Wg, Wo, and Wm in the coupled gating residual module, the scaling and bias coefficients of the FiLM (Feature-wise Linear Modulation) in the diffusion time step modulation module, and the value network weights and policy network weights in the low-level PPO controller.

[0050] Optionally, the synchronous update is implemented as follows: during the joint fine-tuning process, the control device smoothly upscales the 10Hz output command of the high-level planner to the 1000Hz command of the low-level controller through the differentiable frequency adaptation module, and makes the gradient backpropagate through the differentiable module; at the same time, the gradients generated by the low-level tracking loss term and the pruning target loss term in the adaptive joint loss are backpropagated through the value network of the low-level controller to the safety projection layer and denoising network of the high-level planner.

[0051] Optionally, the hierarchical control strategy based on the synchronously updated parameters refers to combining the updated high-level planner and the low-level controller into a complete control link according to the original interface and frequency: the high-level planner outputs the base velocity, hand pose and center of mass offset correction at a frequency of 10Hz, and the low-level controller receives these instructions and outputs joint torque at a frequency of 1000Hz. The two are connected through a differential frequency adapter module.

[0052] Step S4: Deploy the hierarchical control strategy to the target humanoid robot and perform collaborative control of the target humanoid robot in a real physical environment.

[0053] It should be noted that, in order to verify the real effectiveness of the simulation optimization strategy and achieve continuous iteration, this step transfers the trained hierarchical control strategy to the real robot (i.e., the target humanoid robot) and optimizes it through a data feedback closed loop.

[0054] Optionally, deploying the hierarchical control strategy to the target humanoid robot means: using the MWORKS code generation tool, converting the optimized hierarchical control strategy (including the network structure and parameters of the MMCG-DiT high-level planner and the network structure and parameters of the low-level PPO controller) into C / C++ code executable by the robot controller, and loading it into the real-time control system of the target humanoid robot.

[0055] Optionally, the real physical environment refers to actual working scenarios with non-ideal factors such as ground friction, joint servo delay, sensor noise, and external interference, such as factory floors, indoor corridors, and doorways with thresholds.

[0056] Optionally, the collaborative control of the target humanoid robot refers to the following: during the robot's execution of long-term tasks (such as walking-grasping-carrying-placing, autonomous door opening, and stair climbing), after deploying a hierarchical control strategy, the high-level planner generates the base velocity, hand pose, and center of mass offset correction, which are then smoothed by the differential frequency adaptation module and transmitted to the low-level controller. The low-level controller outputs joint torques to drive the robot's movement, so that the mobile base, arms, and whole-body balance coordinate in physical time.

[0057] Optionally, the control device also collects full state data and task execution results of the real machine (i.e., the real robot) and imports them into the digital twin robot for comparative analysis, and further optimizes parameters and weights. It can also supplement training data of extreme scenarios and out-of-distribution scenarios in the simulation environment, and repeatedly execute the joint fine-tuning process to achieve a closed-loop solution of "simulation optimization - real machine deployment - data feedback - model iteration".

[0058] This embodiment provides a robot control method. By constructing a digital twin robot in a simulation environment that matches the hardware parameters and physical characteristics of a target humanoid robot (i.e., a real robot), the digital twin robot can provide physical execution limit information consistent with the real robot, such as upper limits of joint torques, foot friction cone constraints, and end-effector reachable space boundaries. An adaptive joint loss function is constructed for jointly training the digital twin robot's high-level planner and low-level controller. The high-level planner includes a differentiable, safe projection layer to constrain the high-level planning instructions output by the high-level planner based on the physical execution limit information of the low-level controller. This layer directly limits the correction amounts of base velocity, hand pose, and center of mass offset from the instruction generation stage to the executable envelope formed by the upper limits of joint torques, foot friction cones, center of mass support polygons, and end-effector reachable space, preventing high-level planning instructions from exceeding the physical implementation capabilities of the low-level controller. Simultaneously, the adaptive joint loss function includes an instruction executability loss term to quantify the degree to which high-level planning instructions exceed physical execution limit information, and a planning execution distribution alignment loss term to quantify the deviation between the feasible range of high-level planning instructions and the capability range of low-level executable instructions.

[0059] Because the adaptive joint loss function can directly incorporate real-time physical constraints from the feedback of the digital twin robot during computation, the high-level planning instructions generated by the high-level planner are gradually constrained to the actual executable range achievable by the low-level controller during the synchronous update of high-level planner and low-level controller parameters. This hierarchical control strategy, based on the synchronously updated parameters, fundamentally eliminates the instruction over-limit problem caused by the decoupling of high-level planning and low-level control in traditional solutions. Through joint optimization of planning instructions and execution capabilities, the resulting hierarchical control strategy, when deployed to the target humanoid robot, ensures that base movement, hand manipulation, and overall robot balance remain coordinated under physical constraints in a real physical environment. Therefore, it significantly improves the success rate and execution stability of humanoid robots performing long-term complex tasks in real physical environments.

[0060] In one feasible implementation, the network structure of the high-level planner of the digital twin robot is: a state-scene condition encoder, a diffusion time-step modulation module, a motion branch denoiser, an operation branch denoiser, a coupling gated residual module, and a differentiable safe projection layer. The state-scene condition encoder consists of 6 stacked condition coding layers. The diffusion time step modulation module is located after the state-scene condition encoder; The motion branch denoising unit is composed of four sequentially stacked motion branch denoising layers, and the operation branch denoising unit is composed of four sequentially stacked operation branch denoising layers. The coupling gated residual module is disposed between the second and third motion branch denoising layers of the motion branch denoising device and between the second and third operation branch denoising layers of the operation branch denoising device. The differentially secure projection layer is disposed after the motion branch denoiser and the operational branch denoiser.

[0061] It should be noted that, in order to enable the high-level planner to explicitly couple the moving base, two-hand operation and overall balance constraints during the diffusion denoising process, and to avoid the instruction distribution offset caused by the general Transformer structure, a dedicated motion-operation coupled gated diffusion network was designed in this step.

[0062] Optionally, the state-scene conditional encoder consists of six stacked conditional encoding layers. Each conditional encoding layer includes a multi-head self-attention mechanism, followed by RMSnorm (Root Mean Square Normalization) normalization, SwiGLU (Swish Gated Linear Unit) activation of the feedforward network, and pre-normalized residual connections. The multi-head self-attention mechanism has 8 heads, a hidden dimension of 384, a feedforward expansion factor of 2.5, and the residual connections use a pre-normalized form of x+DropPath(F(RMSNorm(x))). This state-scene conditional encoder maps conditional input data to conditional encoded features.

[0063] Optionally, the number of diffusion time step modulation modules is 1, and it is set after the state-scene condition encoder.

[0064] Optionally, the motion branch denoiser consists of four sequentially stacked motion branch denoising layers, and the operation branch denoiser consists of four sequentially stacked operation branch denoising layers. The motion branch denoiser is used to output the base linear velocity, angular velocity, and center of mass offset, while the operation branch denoiser is used to output the three-dimensional position of both hands, attitude angle, and grasping stage variables. The two branches do not share the parameters of the last two denoising layers to avoid excessive interference of the base velocity gradient on the end-effector pose accuracy.

[0065] Optionally, the number of coupled gated residual modules is one, and it is set between the second and third motion branch denoising layers of the motion branch denoiser and between the second and third operation branch denoising layers of the operation branch denoiser. Specifically, the gate factor g of the coupled gated residual module is calculated by the formula g=sigmoid(Wg[hm, ho, cfoot, zcom]), where hm is the hidden state of the motion branch, ho is the hidden state of the operation branch, cfoot is the foot contact encoding, zcom is the centroid stability margin encoding, and Wg is a learnable weight matrix that belongs to the training parameters inside the coupled gated residual module.

[0066] Optionally, the motion branch is updated to hm' = hm + g·Woho, where Woho is a learnable weight matrix used to project the hidden state ho of the operation branch into the feature space of the motion branch.

[0067] Optionally, the operation branch is updated to ho'=ho+(1-g)·Wmhm, where Wmhm is a learnable weight matrix used to project the hidden state hm of the motion branch into the feature space of the operation branch.

[0068] Optionally, through the dynamic gating mechanism of the coupled gating residual module, the information flow and fusion strength between the base movement and the hand operation can be adjusted in real time according to the foot contact state and the center of mass stability state, so as to realize adaptive information exchange. This allows the target humanoid robot to prioritize meeting the operation requirements under stable conditions and prioritize ensuring balance under unstable conditions, ultimately improving the collaborative control performance of the humanoid robot in long-term tasks.

[0069] Optionally, the number of differentiable safe projection layers is one, and it is set after the motion branch denoiser and the operation branch denoiser. The differentiable safe projection layer is a differentiable mathematical mapping module, which is used to restrict the base velocity predicted by the motion branch denoiser, the centroid offset, and the hand pose predicted by the operation branch denoiser within the executable envelope of the underlying controller based on the physical execution limit information of the underlying controller (including the upper limit of joint torque, the boundary of the foot friction cone, the range of the centroid support polygon, and the reachable space of the end effector), and ensure that the projection operation is differentiable, so that the gradient can be backpropagated to the preceding network layer. The preceding network layer is the motion branch denoiser (specifically the 4th, 3rd, 2nd, and 1st motion branch denoiser layers) and the operation branch denoiser (specifically the 4th, 3rd, 2nd, and 1st operation branch denoiser layers) before the differentiable safe projection layer, as well as the coupled gating residual module located before these denoising layers, and also includes the state-scene condition encoder (its 6-layer condition coding layer).

[0070] Understandably, the aforementioned 6-layer state-scene condition encoder, 4-layer motion branch denoising layer, 4-layer operation branch denoising layer, the position settings of the coupled gated residual module, and the addition of the differentiable safe projection layer constitute a dedicated network structure for motion-operation cooperative control of humanoid robots. This network structure was verified through ablation experiments: when the coupled gated residual module was removed, the average task success rate decreased from 96.3% to 88.9%, and when the differentiable safe projection layer was removed, the proportion of torque saturation frames increased from 3.1% to 9.7%.

[0071] In one feasible implementation, in the high-level planner... The state-scene condition encoder is used to receive conditional input data and output conditional coding features. The conditional input data includes the state data of the target humanoid robot, external environment data, diffusion time step, foot contact state, and task stage code. The diffusion time step modulation module is used to take the diffusion time step as input, output scaling coefficient and bias coefficient, and input the scaling coefficient and bias coefficient to each motion branch denoising layer and each operation branch denoising layer respectively, so as to modulate the denoising features of each denoising layer. The conditional coding features are respectively input into the motion branch denoiser and the operation branch denoiser, and the denoising results of each are concatenated to obtain the predicted high-level planning instruction. Specifically, after every two denoising layers of both the motion branch denoiser and the operation branch denoiser, the hidden states of the motion branch and the operation branch, along with the conditional coding features, are input into the coupled gated residual module to obtain updated hidden states of the motion branch and the operation branch. The updated hidden states of the motion branch and the operation branch are then input into the next denoising layer of the motion branch denoiser and the next denoising layer of the operation branch denoiser, respectively, to obtain their respective denoising results. The hidden states of the motion branch are obtained by diffusion denoising performed by the motion branch denoiser, and the hidden states of the operation branch are obtained by diffusion denoising performed by the operation branch denoiser. After the predicted high-rise planning instructions are input into the differentiable safe projection layer for boundary constraints, the actual high-rise planning instructions are output.

[0072] It should be noted that the input data is processed sequentially by each module in the high-rise planner, and the actual high-rise planning instructions after boundary constraints are output. It should be noted that, in order for the high-level planner to generate instructions that conform to the execution capabilities of the underlying controller based on the robot's state (which can be the state data of the digital twin robot or the state data of the target humanoid robot) and environmental observations (i.e., external environment data), this step involves the data flow and module collaboration relationship from the input of conditional input data to the final output of high-level planning instructions.

[0073] Specifically, the state-scene condition encoder first receives conditional input data and outputs conditional encoding features. Optionally, the conditional input data includes the target humanoid robot's state data (such as joint angles, joint velocities, foot contact force torque, IMU (Inertial Measurement Unit) data, denoted as stab), external environment data (such as visual point clouds, target pose, and obstacles, denoted as obs), diffusion time step (representing the current iteration number in the diffusion denoising process, denoted as t), foot contact state (also called foot contact encoding, which obtains the encoding of whether the left and right feet are in contact with the ground based on the contact state / slip velocity, denoted as cfoot), and task stage encoding (such as the encoding of the walking stage, grasping stage, and placement stage). It may also include the centroid stability margin (such as the support polygon distance, denoted as zcom).

[0074] Optionally, the state-scene conditional encoder consists of six stacked conditional coding layers. Each layer includes a multi-head self-attention network, RMSnorm normalization, a SwiGLU activation feedforward network, and a pre-normalized residual connection, which are used to conditionally embed and concatenate these multi-source input data and then fuse them into a unified conditional coding feature.

[0075] Optionally, the diffusion time-step modulation module receives the diffusion time step as input, outputs a scaling factor gamma_t and a bias factor beta_t, and inputs the scaling factor and the bias factor to each residual block inside each motion branch denoising layer and each residual block inside each operation branch denoising layer, respectively. It then performs gamma_t·h+beta_t modulation on the input feature h within the residual block using FiLM (Feature-wise Linear Modulation). This modulation does not change the network structure of the denoiser but dynamically adjusts the feature representation according to the denoising progress. Specifically, the diffusion time-step modulation module first performs sinusoidal time-step embedding encoding on the diffusion time step, mapping it to a high-dimensional vector; then, through a small network composed of linear layers and activation functions, it converts this high-dimensional vector into scaling factors (vectors with the same dimension as the denoised features, used to scale the denoised features) and bias factors (vectors with the same dimension as the denoised features, used to translate the denoised features).

[0076] Understandably, by introducing scaling and bias coefficients related to the diffusion time step into each denoising layer, the feature representation can be adaptively adjusted according to the current denoising progress, thereby improving denoising stability and generation quality under conditions of limited demonstration data.

[0077] Optionally, the conditional coding features are input into the motion branch denoiser and the operation branch denoiser (also called a motion-operation dual-branch diffusion denoiser), respectively, and the denoising results of each are concatenated to obtain the predicted high-level planning instruction. It should be noted that each residual block within each denoising layer of the motion branch denoiser and the operation branch denoiser has received scaling and bias coefficients from the diffusion time-step modulation module, meaning that denoising operations are performed under modulated conditions.

[0078] Optionally, the motion branch denoising unit consists of four sequentially stacked motion branch denoising layers, used to output the predicted base linear velocity, angular velocity, and centroid offset; the operation branch denoising unit consists of four sequentially stacked operation branch denoising layers, used to output the predicted hand 3D position, attitude angle, and grasping stage variables; the above prediction results together constitute (i.e., splice together) the predicted high-level planning instructions.

[0079] Optionally, after every two denoising layers of the motion branch denoiser and the operation branch denoiser (i.e., after the output of the second denoising layer and before the input of the third denoising layer), the motion branch hidden state, the operation branch hidden state, and the conditional coding features are input into the coupled gated residual module to obtain the updated motion branch hidden state and the updated operation branch hidden state.

[0080] Optionally, the motion branch hidden state, denoted as hm, is the intermediate feature vector obtained by the motion branch denoiser performing diffusion denoising during forward propagation, and the operation branch hidden state, denoted as ho, is the intermediate feature vector obtained by the operation branch denoiser performing diffusion denoising. Diffusion denoising refers to the process of starting from random noise and gradually removing noise through iteration to recover meaningful instruction data.

[0081] Optionally, the number of coupled gated residual modules is 1. Internally, the gate factor g is first calculated; then the hidden states of motion branches and operation branches are updated to obtain the updated hidden states of motion branches, denoted as hm' and the updated hidden states of operation branches, denoted as ho'; finally, the updated hidden states of motion branches are input into the next denoising layer of the motion branch denoiser (i.e., the 3rd motion branch denoising layer), and the updated hidden states of operation branches are input into the next denoising layer of the operation branch denoiser (i.e., the 3rd operation branch denoising layer).

[0082] Optionally, after all four layers of processing by the motion branch denoiser and the operation branch denoiser, the predicted high-level planning instruction obtained by splicing the updated motion branch hidden state and the updated operation branch hidden state is input into the differentiable safe projection layer, and the actual high-level planning instruction (or high-level control instruction) is output after boundary constraints.

[0083] Optionally, a differentiable safe projection layer of one size is positioned after the motion branch denoiser and the operation branch denoiser. This differentiable safe projection layer clips or scales the predicted base velocity, hand pose, and centroid offset based on the physical execution limit information of the underlying controller (including the upper limit of joint torque, the boundary of the foot friction cone, the range of the centroid support polygon, and the reachable space of the end effector), constraining them within the executable envelope and maintaining the differentiability of the projection operation. Finally, it outputs the actual high-level planning instructions for subsequent transmission to the underlying controller for execution. This layer is located at the output end and is used for output and safety constraints.

[0084] Understandably, through the aforementioned sequential conditional encoding, bi-branch denoising, coupled gated residual update, and safe projection, the high-level planner can adaptively coordinate movement, manipulation, and balance constraints based on foot contact and centroid state during the diffusion denoising process, generating actual high-level planning instructions that match the execution capabilities of the lower-level controller.

[0085] In one feasible implementation, the step of constructing a digital twin robot that matches the hardware parameters and physical characteristics of the target humanoid robot includes: Step A1: The preset demonstration trajectory is used as the high-level planning instruction of the digital twin robot to drive the low-level controller with frozen parameters to execute low-level actions, thereby obtaining the demonstration dataset. It should be noted that, in order to collect expert data for training the high-level planner, this step generates high-quality demonstration trajectories manually or heuristically, and performs closed-loop acquisition in a simulation environment.

[0086] Optionally, the preset demonstration trajectory refers to the robot's expected behavior sequence generated by manual teleoperation or a heuristic behavior planner, such as the expected base velocity sequence and hand pose sequence in a "walk-grasp-carry-place" task.

[0087] Optionally, using a preset demonstration trajectory as a high-level planning instruction for the digital twin robot means that the control device directly sends the desired state (such as desired base velocity and desired hand pose) in the demonstration trajectory as a high-level planning instruction to the low-level controller.

[0088] Optionally, the underlying controller with frozen parameters refers to the underlying RL (Reinforcement Learning) motion-operation controller whose parameters remain unchanged and are not updated during the acquisition process. Its parameters are fixed in the pre-training stage to ensure that the acquired data only reflects the mapping relationship between the demonstration trajectory and the underlying execution capability, without introducing interference caused by changes in the controller's own policy.

[0089] Optionally, driving the frozen parameters of the bottom-level controller to perform bottom-level actions means that the control device inputs the high-level planning instructions into the bottom-level controller, and the bottom-level controller outputs the torque control quantities of each joint to drive the digital twin robot to perform actions.

[0090] Optionally, the demonstration dataset refers to the offline data collection acquired through the above simulation process. Each data point includes at least: robot body state (joint angles, joint velocities, etc.), environmental observation (target object pose, etc.), motion commands (high-level planning commands and their corresponding low-level execution results), task rewards, foot contact state, centroid projection margin, and low-level controller saturation.

[0091] Optionally, the foot contact state and the saturation of the underlying controller are used as executability constraints in the subsequent joint loss function.

[0092] Step A2: Based on the state-scene conditional encoder, motion branch denoiser, operation branch denoiser, coupled gated residual module and differentiable safe projection layer, the initial diffusion network is constructed. It should be noted that in order to obtain a high-level planning model that can couple motion and operation during the diffusion denoising process, this step constructs an untrained diffusion network.

[0093] Optionally, the initial diffusion network refers to a high-level planner network whose parameters are randomly initialized or initialized in a preset manner and have not been trained in any way. Its network structure includes: a state-scene condition encoder with 6 layers stacked in sequence, a motion branch denoiser with 4 layers stacked in sequence, an operation branch denoiser with 4 layers stacked in sequence, a coupled gated residual module set between the second and third layers of the motion branch denoiser and between the second and third layers of the operation branch denoiser, and a differentiable safe projection layer set after the motion branch denoiser and the operation branch denoiser.

[0094] Optionally, constructing the initial diffusion network means that the control device creates the above network structure, assigns corresponding learnable parameters (including multi-head self-attention weights in the state-scene condition encoder, diffusion denoising network weights in the motion branch denoiser and operation branch denoiser, Wg, Woho, and Wmhm weight matrices in the coupled gated residual module, and boundary scaling parameters that may exist in the differentiable safe projection layer), and assigns initial values ​​to these parameters according to a preset method (e.g., normal distribution random initialization).

[0095] Step A3: Pre-train the initial diffusion network based on the demonstration dataset to obtain a preset diffusion network, and use the preset diffusion network as the high-level planner of the digital twin robot.

[0096] It should be noted that, in order for the initial diffusion network to learn to predict reasonable high-level planning instructions from robot state and environmental observations, this step uses the demonstration dataset collected in step A1 to conduct supervised learning training on the initial diffusion network constructed in step A2.

[0097] Optionally, pre-training the initial diffusion network based on the demonstration dataset means that the control device samples data in batches from the demonstration dataset, takes the robot body state, environmental observation, diffusion time step, foot contact state and task stage encoding in the data as input, takes the high-level planning instructions in the data (i.e. the expected base velocity, hand pose and centroid offset correction in the demonstration trajectory) as the prediction target, uses the diffusion denoising loss term (constraining the consistency between the predicted noise and the real noise of the diffusion network) as the optimization target, and updates all learnable parameters of the initial diffusion network through gradient descent.

[0098] Optionally, the pre-defined diffusion network refers to the network whose parameters have converged after pre-training. At this point, the network has the ability to generate reasonable planning instructions from the input conditions, but has not yet undergone joint fine-tuning with the underlying controller.

[0099] Optionally, using a preset diffusion network as the high-level planner of the digital twin robot means that the control device fixes the parameters of the trained diffusion network (or uses them as initial values ​​for subsequent joint fine-tuning) and deploys it in a hierarchical control framework to receive robot state and environmental observations and output actual high-level planning instructions to the low-level controller.

[0100] Understandably, through the aforementioned pre-training, the high-level planner can mimic the behavior distribution demonstrated by experts, possessing a good initial strategy for subsequent joint training.

[0101] Based on the above embodiments of this application, in another embodiment of this application, the same or similar content as the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, step S2, which constructs an adaptive joint loss function for jointly training the high-level planner and the low-level controller of the digital twin robot, includes: Step S21: Determine the diffusion denoising loss term based on the consistency between the noise predicted by the diffusion network in the high-level planner and the actual noise. It should be noted that, in order for the high-level planner to accurately recover reasonable planning instructions from random noise, this step calculates the loss value of the diffusion denoising loss term L_diff (also called the diffusion denoising loss value) to constrain the prediction accuracy of the diffusion network. Optionally, the diffusion denoising loss value refers to the mean square error between the noise predicted by the diffusion network (i.e., MMCG-DiT) in the high-level planner during training and the actual noise pre-added to the real instruction data. Specifically, during the training process, the control device first adds standard Gaussian noise stepwise to the real high-level planning instructions (such as base velocity and hand pose) in the demonstration dataset according to the diffusion process to obtain the noisy instructions; then, the noisy instructions, along with the conditional inputs (robot state, environmental observation, diffusion time step, etc.), are input into the diffusion network, and the diffusion network outputs a predicted noise; finally, the squared difference between the predicted noise and the actual added noise is calculated, and the average value is taken as the diffusion denoising loss value. The smaller the diffusion denoising loss value, the stronger the denoising ability of the diffusion network, and the closer the generated high-level planning instructions are to the real demonstration data.

[0102] Step S22: Determine the instruction executability loss item based on the degree of violation of the underlying physical constraints fed back when the underlying controller executes the high-level planning instruction output by the high-level planner. The underlying physical constraints include torque saturation, foot friction cone overrun, and end reachable space overrun. It should be noted that, to ensure that high-level planning instructions do not exceed the physical execution capabilities of the low-level controller, this step quantifies the degree to which instructions violate physical constraints through the loss value of the instruction executability loss term L_exec (also called instruction executability loss value). Optionally, the instruction executability loss value is obtained by weighted summation of three sub-items: joint torque saturation, foot friction cone out-of-bounds amount, and end-effector reachability out-of-bounds amount. Optionally, joint torque saturation refers to the portion of the ratio of the actual output torque of each joint of the robot to the rated torque of the joint exceeding the safety threshold when the low-level controller executes high-level instructions; the control device obtains the torque value of each joint through MWORKS Modelica digital twin simulation, calculates the magnitude of exceeding the upper limit of the rated torque, and sums them. Optionally, foot friction cone out-of-bounds amount refers to the degree to which the ratio of the horizontal component to the vertical component of the robot's foot contact force exceeds the friction cone boundary (determined by the static friction coefficient); the control device obtains the three-dimensional contact force of the foot from the simulation, calculates the ratio of the horizontal resultant force to the vertical force, and records the excess if it exceeds the friction coefficient. Optionally, the end-effector reachability exceeds the robot's kinematic reachability by referring to the degree to which the three-dimensional position and attitude angles of the hands in the high-level planning command exceed the robot's kinematic reachability. The control device compares the end-effector pose in the command with the pre-calculated reachability boundary to calculate the three-dimensional Euclidean distance reachability. After normalizing each of the three factors, a weighted sum is obtained to obtain the command executability loss value.

[0103] Step S23: Determine the bottom tracking loss term based on the deviation between the trajectory corresponding to the high-level planning instruction output by the high-level planner and the actual execution trajectory of the bottom-level controller; It should be noted that, in order to measure the ability of the lower-level controller to follow the planning instructions of the higher-level controller, this step calculates the loss value of the lower-level tracking loss term L_track (also called the lower-level tracking loss value). Optionally, the lower-level tracking loss value refers to the deviation between the expected trajectory output by the higher-level planner (such as the expected base linear velocity, the expected hand pose sequence) and the actual trajectory measured after the lower-level controller actually executes the plan. Specifically, at each time step of the joint fine-tuning, the control device collects the expected base velocity, expected hand pose, and expected centroid offset correction amount output by the higher-level planner, and simultaneously collects the actual base velocity, actual hand pose, and actual centroid position fed back by the digital twin robot after the lower-level controller executes the plan; calculates the mean square error between the expected value and the actual value, and sums and averages them over the time steps. The smaller this loss value, the more accurately the lower-level controller can follow the higher-level instructions, and the smaller the deviation between planning and execution.

[0104] Step S24: Determine the target loss term to be pruned based on the update step size limit of the underlying controller during policy update; It should be noted that, in order to ensure the stability of the training of the underlying RL controller and avoid policy collapse caused by excessively large single update steps, this step introduces the pruning target loss term of the PPO algorithm.

[0105] Optionally, the loss value of the pruning target loss term L_ppo (also known as the pruning target loss value) refers to the pruning alternative target loss value used by the underlying controller when adopting the PPO algorithm.

[0106] Specifically, the control device calculates the probability ratio between the old and new strategies; then it trims this probability ratio to the interval [1-ε, 1+ε] (e.g., ε=0.2). Based on the original probability ratio without trimming and the trimmed probability ratio, the control device takes the minimum of these two values, calculates the average of the minimum values ​​(i.e., the expected value), and then takes the opposite value to obtain the target trimming loss value. This target trimming loss value limits the magnitude of strategy updates, avoiding oscillations in control performance due to excessively large single-step updates.

[0107] Step S25: Determine the overall stability loss term based on the minimum distance from the projection point of the center of mass of the digital twin robot along the direction of gravity on the horizontal plane to each side of the supporting polygon, the torso angular velocity of the digital twin robot, and the foot sliding velocity. It should be noted that, in order to maintain the balance and stability of the humanoid robot during the motion-operation process, this step constructs a whole-machine stability loss term L_stab, which penalizes behaviors that may lead to falling or instability.

[0108] Optionally, the loss value of the overall stability loss term (i.e., the overall stability loss value) is composed of a weighted sum of three sub-terms: the distance from the centroid projection to the boundary of the supporting polygon, the torso attitude angular velocity, and the foot slip velocity.

[0109] Optionally, the minimum distance from the projection point of the center of mass on the horizontal plane along the direction of gravity to each side of the supporting polygon is calculated by the control device from the Modelica simulation. The control device obtains the horizontal projection coordinates of the center of mass and the linear equations of each side of the supporting polygon (a convex polygon formed by the foot contact points), calculates the vertical distance from the projection point to each side, and takes the minimum value. The smaller this distance is, the closer the robot is to the instability boundary and the greater the loss value.

[0110] Optionally, the torso attitude angular velocity: the control device obtains the roll, pitch and yaw angular velocities of the torso from the simulation, calculates the sum of the squares or the sum of the absolute values ​​of the three components. The larger the angular velocity, the more severe the fuselage shaking and the greater the loss value.

[0111] Optionally, foot slip velocity: The control device obtains the relative slip velocity between the sole of the foot and the ground from the simulation, calculates its modulus, and the larger the slip velocity, the worse the contact stability and the greater the loss value.

[0112] Step S26: Determine the planning execution distribution alignment loss term based on the deviation between the feasible range of the high-level planning instructions output by the high-level planner and the capability range of the low-level executable instructions of the low-level controller. It should be noted that, in order to eliminate the distribution offset between the planning instruction distribution and the underlying execution capability distribution, this step introduces a planning execution distribution alignment loss term.

[0113] Optionally, the loss value of the planning execution distribution alignment loss term L_align (i.e. the planning execution distribution alignment loss value) is measured using the maximum mean difference (MMD) metric, which is used to quantify the deviation between the feasible range of high-level planning instructions and the capability range of low-level executable instructions.

[0114] Optionally, the feasible range of high-level planning instructions refers to the distribution of output instructions from the high-level planner after being constrained by the differentiable safe projection layer. The capability range of low-level executable instructions refers to the distribution of instructions that the low-level controller can actually stably track. This distribution is estimated by collecting "input instruction - actual tracking error" pairs during the low-level controller's operation and filtering out instruction samples with tracking errors less than a threshold. The control device samples a batch of samples from both distributions and calculates the MMD distance. The larger the MMD value, the greater the difference between the two distributions, and the larger the loss value. This execution distribution alignment loss value prompts the high-level planner to adjust its output distribution through gradient backpropagation, gradually aligning it with the executable capability range of the low-level controller.

[0115] Step S27: Determine the energy consumption smoothing loss term based on the joint torque, torque change rate collected during the task execution of the digital twin robot and the high-level planning instructions output by the high-level planner. It should be noted that, in order to encourage the robot to move in an energy-efficient and smooth manner, this step calculates the loss value of the energy consumption smoothing loss term L_cost (i.e., the energy consumption smoothing loss value).

[0116] Optionally, the energy consumption smoothing loss value is composed of a weighted sum of three sub-items: the sum of squares of joint torques, the sum of squares of torque change rates, and the second-order difference sum of squares of high-level planning instructions.

[0117] Optionally, the sum of squares of joint torques: the control device collects the torque values ​​of all joints from the simulation, calculates their sum of squares, and represents the instantaneous energy consumption; the sum of squares of the rate of change of torque: the sum of squares of the differences in joint torques between adjacent time steps is calculated, and then summed, representing the severity of torque jitter; the sum of squares of the second-order differences of the high-level planning instructions: the second-order differences (i.e., approximate accelerations) of the instruction sequence output by the high-level planner (such as the base velocity sequence, hand pose sequence) are calculated, and then squared and summed, representing the smoothness of the instructions. The smaller these three terms are, the lower the energy consumption, the smoother the motion, and the more continuous the instructions.

[0118] Step S28: Based on the diffusion denoising loss term, the instruction executability loss term, the underlying tracking loss term, the pruning target loss term, the overall system stability loss term, the planning execution distribution alignment loss term, and the energy consumption smoothing loss term, an adaptive joint loss function is constructed.

[0119] It should be noted that, in order to simultaneously optimize diffusion denoising accuracy, instruction executability, low-level tracking capability, policy update stability, overall system balance, distribution alignment, and energy consumption smoothing in a single objective function, this step weights and sums the above seven losses to construct an adaptive joint loss function.

[0120] Optionally, the adaptive joint loss function L_joint has the following specific form: L_joint = lambda_d·L_diff + lambda_e·L_exec + lambda_t·L_track + lambda_r·L_ppo + lambda_s·L_stab + lambda_a·L_align + lambda_c·L_cost.

[0121] Where lambda_d, lambda_e, lambda_t, lambda_r, lambda_s, lambda_a, and lambda_c are the weight coefficients of each loss term. For example, lambda_d = 0.30, lambda_e = 0.20, lambda_t = 0.15, lambda_r = 0.15, lambda_s = 0.10, lambda_a = 0.05, and lambda_c = 0.05. The sum of the weight coefficients of each sub-item is λ_total, denoted as λ_total. , where f is the index of the loss term traversal, f ∈ {d, e, t, r,s, a, c}.

[0122] All losses have been normalized and are dimensionless scalars with values ​​ranging from [0,1].

[0123] Among them, the diffusion denoising loss term is: ];in, This indicates the calculation of mathematical expectation. Gaussian noise was added to the real object. The noise predicted by the diffusion network in the high-level planner is represented by k ~ U(0, K), indicating that the diffusion time step k is uniformly and randomly sampled in the range of 0 to K. The noise vector is represented by each dimension of which independently follows a Gaussian distribution with a mean of 0 and a variance of 1. z_k is the intermediate noise-added hidden state (i.e., the intermediate variable after noise addition) formed during the k-th step of the diffusion noise addition process. c is the condition input (including robot state, environmental observation, diffusion time step, etc.), k is the diffusion time step, and K is the total number of diffusion steps. The smaller the value, the more consistent the noise predicted by the diffusion network is with the actual added Gaussian noise, meaning the diffusion network's noise estimation is more accurate. In other words, the higher the quality of the high-level planner's generated high-level planning instructions, the closer they are to the distribution of the demonstration trajectories in the training data, as the planning instructions gradually recovered from the random noise.

[0124] Instruction executability loss items: ;in, For joint torque saturation, The amount of friction cone exceeding the limit at the foot end. This refers to the amount of space exceeding the limit at the end of the reachable space. , , These are the weighting coefficients for the joint torque saturation, the foot friction cone overrun, and the end-effector reach space overrun, respectively. , , All have been normalized. The smaller the value, the lower the degree to which the planning instructions output by the higher-level planner violate the underlying physical constraints. This means that there is less joint torque saturation, less foot contact force exceeding the friction cone boundary, and less end effector position exceeding the reachable space. In other words, the higher-level planner can actively avoid physical limits that the lower-level controller cannot achieve when generating instructions, making the planning instructions more physically executable.

[0125] Underlying tracking loss term: Where N is the number of sampling time steps, i is the i-th sampling time step out of N sampling time steps, p represents position, and v represents velocity. The weighting coefficients represent the position tracking error. The weighting coefficients represent the speed tracking error. To determine the desired position corresponding to the desired position instruction output by the high-level planner at the i-th sampling time step, The actual arrival position fed back by the digital twin robot or real machine after the actual execution by the underlying controller at the i-th sampling time step; For the desired velocity corresponding to the desired velocity command output by the high-level planner at the i-th sampling time step, This refers to the actual speed fed back by the digital twin robot or the real machine after the underlying controller actually executes the code at the i-th sampling time step. The smaller the value, the smaller the deviation between the expected trajectory output by the high-level planner and the actual trajectory executed by the low-level controller, meaning the lower-level controller has higher tracking accuracy of the high-level planning instructions. In other words, the instructions planned by the high-level planner are more closely matched to the dynamic characteristics of the low-level controller, and the planner will not require the low-level controller to perform actions beyond its physical capabilities.

[0126] Pruning target loss item: ;in, This indicates the calculation of mathematical expectation. This indicates that state s and action a follow the old policy before the update. The distribution is sampled; This represents the robot's current observation state. t represents the action command the robot will take based on its current observation state; t is the time step. For the new strategy, The importance sampling ratio is the ratio of the probabilities of the new policy and the old policy under the same state-action pair. For the estimation of the advantage function, (·) is the clipping function, which restricts the input value to a certain range. Within the range; The pruning hyperparameter (usually 0.1~0.2) is used to limit the step size of policy updates and ensure training stability; (·) represents the smaller value. Its physical meaning is to impose constraints on the update step size during policy update. The lower the value of this loss term, the smoother the policy update, and the more effectively the update step size is limited within a safe range, thereby ensuring the stability of the underlying controller training.

[0127] Overall stability loss items: ;in, Let be the distance projected from the centroid onto the boundary of the supporting polygon. The angular velocity of the torso is the attitude angle. The sliding speed of the foot; , and These are the weighting coefficients corresponding to the distance from the centroid projection to the boundary of the supporting polygon, the torso attitude angular velocity, and the foot sliding velocity, respectively. The smaller the value, the more stable the robot's overall balance is during task execution. This means the center of mass projection is farther from the boundary of the supporting polygon, the torso angular velocity is smaller, and the foot sliding speed is lower. In other words, the robot can maintain good dynamic balance during base movement and hand manipulation, and is less prone to falling or losing control.

[0128] Planning execution distribution alignment loss term: Where p is the sample index of the planned distribution, and its value ranges from 1, 2, 3... ; The total number of samples collected from the planning distribution is the distribution of output instructions of the high-level planner after being constrained by a differentiable safe projection layer. The feature map induced by the kernel function (commonly Gaussian kernel) is used to map the original sample to the regenerated kernel Hilbert space; Let p be the p-th sample sampled from the planning distribution, representing a set of planning instructions (such as base velocity, hand pose, centroid offset, etc.) output by the high-level planner under the current conditions. q is the sample index of the execution distribution, ranging from 1, 2, 3... ; The total number of samples collected from the execution distribution, which is the range of instructions that the underlying controller can actually implement while maintaining overall machine balance and stable contact. Let q be the sampled from the execution distribution, representing the instruction sample that the underlying controller can stably track in actual operation; H is the regenerating kernel Hilbert space. The smaller the value, the closer the distribution of instructions output by the high-level planner is to the distribution of instructions that the low-level controller can actually execute. In other words, the planning instructions are less likely to exceed the physical execution capability of the low-level controller, and the higher the degree of "distribution alignment" between planning and execution.

[0129] Energy consumption smoothing loss term: ;in, This represents the weighting coefficient for the squared term of the joint torque; This is the weighting coefficient for the squared term of the rate of change of torque; These are the weighting coefficients for the second-order difference squared terms of the planning instructions; This represents the total number of joints in the robot. This is a joint index, with values ​​ranging from 1, 2, 3... , indicating the first One joint; For the first The actual output torque of each joint; For the first The torque change rate of each joint; L is the total number of time steps contained in the high-level planning instruction; n is the time index of the high-level planning instruction, with a value range of 2, 3, ..., L-1, representing the nth time step; Let be the high-level planning instruction vector at time n; This is the high-level planning instruction vector at time n+1; This is the high-level planning instruction vector at time n-1. This represents the second-order difference (i.e., discrete acceleration) of the high-level planning instruction at time n, used to quantify the smoothness of the instruction change. The smaller the value, the smaller the joint torque, the lower the rate of torque change, and the smoother the high-level planning instruction sequence during robot task execution. This means the robot consumes less energy, experiences less motion shock, and has more continuous instruction changes. In other words, the robot can perform tasks in a more energy-efficient and gentler manner, thereby extending hardware lifespan and reducing operating costs.

[0130] Optionally, through the above adaptive weighting, the joint loss function can balance various optimization objectives, enabling the high-level planner and the low-level controller to converge collaboratively during the training process.

[0131] In one feasible implementation, step S3, which involves synchronously updating the parameters of the high-level planner and the low-level controller using the adaptive joint loss function, includes: Step S31: During the execution of the high-level planning instructions output by the high-level planner by the low-level controller, the physical feedback quantity of the digital twin robot is obtained; It should be noted that, in order to obtain real-time physical state information for dynamically adjusting the weights of the joint loss terms, this step continuously collects the robot's physical feedback data during the simulation execution.

[0132] Optionally, during the execution of high-level planning instructions output by the high-level planner by the low-level controller, the control device inputs the high-level planning instructions (the actual high-level planning instructions constrained by the differentiable safety projection layer) into the low-level controller, and the low-level controller outputs joint torques to drive the digital twin robot to move, and data is collected within each simulation step (e.g., every 1ms) during the interaction process.

[0133] Optionally, the physical feedback quantities obtained for the digital twin robot refer to the following data read and recorded in real time by the control device from the MWORDS Modelica digital twin simulation environment: foot slip ratio (the ratio of the foot's sliding speed relative to the ground to its forward speed), torque saturation frame percentage (the ratio of the number of joints whose torque exceeds the rated torque limit in the current simulation step to the total number of joints), end-effector tracking error (the Euclidean distance between the desired hand pose in the high-level command and the actual hand pose obtained during execution), task success rate (the proportion of successful completions in the most recent rounds of tasks), and energy consumption baseline (the average of the sum of squares of all joint torques per unit time). These physical feedback quantities are used to determine whether the weights of various loss terms need to be adjusted.

[0134] Step S32: Based on the physical feedback quantity, dynamically adjust the weights of each loss term in the adaptive joint loss function to obtain the adjusted weights; It should be noted that, in order to enable the joint loss function to adapt to different training stages and robot states, this step dynamically adjusts the proportion of each sub-loss in the total loss based on the physical feedback collected in the previous step.

[0135] Optionally, the weights of each loss term in the adaptive joint loss function are dynamically adjusted based on the physical feedback quantity. This means that after the control device completes N rounds of sampling (e.g., N=10 rounds), the physical feedback quantity obtained in the current round is compared with a preset threshold. The weights can be updated or adjusted using EMA (Exponential Moving Average, a weighted average method that gives higher weight to recent data). Alternatively, the corresponding weight coefficients can be increased or decreased according to the following adjustment rules.

[0136] Optionally, the single-round adjustment amount of any weight adjusted due to a triggering condition. Calculated according to the adjustment rules. The triggering condition is that the physical feedback quantity obtained from the current round of statistics reaches or exceeds the corresponding preset threshold. The adjustment rules are: ;in, This refers to the physical quantities monitored in the current round (foot slip rate, torque saturation frame percentage, or energy consumption). For the preset threshold of the corresponding physical quantity, These are the response coefficients. The range of values ​​for each response coefficient is... , , , .

[0137] Specifically: when the foot slip rate exceeds its corresponding preset threshold, the weight of the overall stability loss term lambda_s is increased according to the adjustment rules, and the weight of the underlying tracking loss term lambda_t is decreased by the same amount; when the proportion of torque saturation frames exceeds its corresponding preset threshold, the weight of the instruction executability loss term lambda_e is increased according to the adjustment rules; when the task success rate reaches the preset value and the current energy consumption is higher than the baseline, the weight of the energy consumption smoothing loss term lambda_c is increased according to the adjustment rules.

[0138] Optionally, for the planning execution distribution alignment loss weight, the trigger condition is that the end-point tracking error continuously decreases but the task success rate does not increase. In this case, a fixed adjustment step size is used. Increase the weight of lambda_a in the planning execution distribution alignment loss term.

[0139] Optionally, if multiple conditions are triggered simultaneously within a single round, the adjustment amounts generated by each condition on the same weight are summed algebraically, and the summed weight satisfies | |≤0.2· This is to avoid training oscillations caused by drastic changes in weights.

[0140] Optionally, after each round of adjustment, the weights of each sub-item are normalized and constrained according to the following formula: Furthermore, after normalization, the weights of each sub-item satisfy 0 < 0. <1 and the sum of the weights of all loss items after adjustment is still 1. , ∈ {d', e', t', r',s', a', c'}.

[0141] The final output is a new set of weight coefficients: lambda_d', lambda_e', lambda_t', lambda_r', lambda_s', lambda_a', lambda_c'.

[0142] Step S33: Update the adaptive joint loss function with the adjusted weights, calculate the loss value of the adaptive joint loss function, backpropagate the gradient of the loss value to the high-level planner and the low-level controller, and synchronously update the parameters of the high-level planner and the low-level controller.

[0143] It should be noted that, in order to achieve coordinated convergence of the planner and the controller, this step uses a dynamically adjusted joint loss function to calculate the gradient and updates both modules simultaneously through backpropagation.

[0144] Optionally, updating the adaptive joint loss function with the adjusted weights means that the control device substitutes the adjusted weights obtained in step S32 into the joint loss formula: L_joint = lambda_d'·L_diff + lambda_e'·L_exec + lambda_t'·L_track + lambda_r'·L_ppo + lambda_s'·L_stab + lambda_a'·L_align + lambda_c'·L_cost. Then, the loss value of this loss function at the current position is calculated.

[0145] Optionally, backpropagating the gradient of the loss value to the higher-level planner and the lower-level controller means that the control device uses an automatic differentiation mechanism to calculate the partial derivative of each learnable parameter from L_joint and then propagates the gradient to each network layer of the network structure in the higher-level planner and the lower-level controller.

[0146] Specifically, the gradient will pass sequentially through the differentiable safe projection layer (differentiable) of the high-level planner, the motion branch denoiser and the operation branch denoiser, the coupled gated residual module, the diffusion time step modulation module, and the state-scene condition encoder, and will also be passed to the policy network and value network of the lower-level controller.

[0147] Optionally, synchronously updating the parameters of the high-level planner and the low-level controller means that the control device updates all weights in the high-level planner (including attention weights in the state-scene condition encoder, feedforward network weights, weights in the diffusion time-step modulation module, motion / operation branch denoising weights, and weights in the coupled gating residual module, etc.) and the policy network weights and value network weights in the low-level controller simultaneously based on the gradient obtained from backpropagation (e.g., using the Adam optimizer with learning rates set to 1e-5 and 3e-5, respectively). After the update, the parameters of both modules have changed in the same iteration step, thereby maintaining the coordinated convergence of the planning instruction distribution and the execution capability distribution.

[0148] In one feasible implementation, the physical feedback quantities include foot slip rate, joint torque saturation ratio, and end-effector tracking error; the adaptive joint loss function includes instruction executability loss term, low-level tracking loss term, overall system stability loss term, planning execution distribution alignment loss term, and energy consumption smoothing loss term; step S32, the step of dynamically adjusting the weights of each loss term in the adaptive joint loss function based on the physical feedback quantities, includes: Step A31: When the foot slip rate exceeds a preset slip rate threshold, increase the weight of the overall stability loss term and decrease the weight of the underlying tracking loss term. It should be noted that, in order to prioritize the stability of the robot's foot contact and avoid slippage that could lead to mission failure, this step adjusts the weight of the corresponding loss items based on the foot slippage situation.

[0149] Optionally, the foot slip ratio refers to the ratio of the sliding speed of the foot relative to the ground to the forward speed of the robot when it is in contact with the ground. It is obtained by the control device from the MWORKS Modelica digital twin simulation. For example, if the foot slip speed is 0.05 m / s and the forward speed is 0.5 m / s, then the slip ratio is 0.1.

[0150] Optionally, the preset slip rate threshold is a pre-set safe upper limit for slip rate, such as 5%. When the actual slip rate exceeds this threshold, it indicates that the foot has slipped significantly.

[0151] Optionally, the overall stability loss term L_stab is a sub-term in the adaptive joint loss function, used to penalize behaviors such as excessively small distance between the centroid projection and the boundary of the supporting polygon, excessively large torso angular velocity, and excessively large foot slip velocity.

[0152] Optionally, the underlying tracking loss term L_track is another sub-term in the adaptive joint loss function, used to measure the deviation between the robot's actual trajectory and the trajectory planned by the higher-level system. When the control device detects that the foot slip rate exceeds a preset slip rate threshold, it adjusts the weight of the overall stability loss term lambda_s by 20% and decreases the weight of the underlying tracking loss term lambda_t by 20% according to a preset adjustment range, so that the joint optimization process focuses more on penalizing foot slip, temporarily allowing a certain amount of trajectory tracking deviation in exchange for better ground contact stability.

[0153] Step A32: When the joint torque saturation ratio exceeds a preset saturation threshold, increase the weight of the instruction executability loss item; It should be noted that, in order to prevent high-level planning instructions from exceeding the actual output torque range of the low-level controller, this step adjusts the weight of the instruction executability loss term based on the joint torque saturation level.

[0154] Optionally, the joint torque saturation ratio refers to the ratio of the number of joints whose actual output torque exceeds the rated torque limit to the total number of joints in the current simulation step or a sampling period. It is obtained by the control device collecting the torque value of each joint from the Modelica simulation and comparing it with the rated torque limit of that joint.

[0155] Optionally, the preset saturation threshold is a pre-set upper limit of saturation ratio, such as 8%. When the actual ratio exceeds this threshold, it indicates that many joints are approaching or have reached the torque output limit.

[0156] Optionally, the instruction executability loss term L_exec is a sub-term of the adaptive joint loss function, weighted by joint torque saturation, foot friction cone out-of-bounds amount, and end-effector reachability out-of-bounds amount. It is used to quantify the degree to which high-level planning instructions violate low-level physical execution constraints. When the control device detects that the proportion of joint torque saturation exceeds a preset saturation threshold, it adjusts the joint optimization process by a preset amount (e.g., increasing the weight of the instruction executability loss term lambda_e by 30%) to penalize instructions that exceed the torque limit, thereby prompting the high-level planner to constrain the output instructions within the torque envelope executable by the low-level controller.

[0157] Step A33: After executing the task a preset number of times, when the end tracking error decreases continuously for multiple times and the task success rate does not increase compared to the previous time, increase the weight of the planning execution distribution alignment loss term. It should be noted that, in order to address the issue of improved end-point tracking accuracy but stagnant overall task success rate (indicating a distributional offset between planning instructions and execution capabilities), this step adjusts the weight of the distribution alignment loss term based on the changing trends of tracking error and task success rate.

[0158] Optionally, continuously executing a preset number of tasks means that the control device performs an evaluation after completing a certain number of task rounds (e.g., every 10 iterations, each round containing 200 sampling trajectories) during the joint fine-tuning process.

[0159] Optionally, the end-point tracking error refers to the weighted sum of the Euclidean distance and angle deviations between the expected hand poses (3D position and attitude angles) output by the high-level planner and the actual hand poses fed back by the digital twin robot after the low-level controller actually executes the simulation. The average error over the entire task trajectory is calculated by the control device at the end of the simulation.

[0160] Optionally, "continuous decline" refers to the fact that the end tracking error shows a downward trend in two or more consecutive evaluation periods (e.g., a decline of more than 5% in each round).

[0161] Optionally, the task success rate is the proportion of times a task is successfully completed in the most recent rounds of tasks, which is calculated by the control equipment based on task completion indicators (such as successfully placing an object from the starting point to the target point in material handling).

[0162] Optionally, "not higher than the previous one" means that the task success rate calculated in the current evaluation period is less than or equal to the success rate in the previous evaluation period.

[0163] Optionally, the planning execution distribution alignment loss term L_align is a sub-term of the adaptive joint loss function, using the maximum mean difference (MMD) to measure the deviation between the feasible range of the high-level planning instructions and the capability range of the low-level executable instructions. When the control device detects that the above conditions are met, it adjusts the planning execution distribution alignment loss term by a preset amount (e.g., increasing the weight lambda_a of the planning execution distribution alignment loss term by 50%), making the optimization process more focused on forcing the distribution of planning instructions to align with the capability range of the low-level controller, thereby breaking the deadlock where tracking accuracy improves but the success rate does not.

[0164] Step A34: When the task success rate reaches a preset success rate threshold and the joint energy consumption of the digital twin robot is higher than the baseline energy consumption, increase the weight of the energy consumption smoothing loss term.

[0165] It should be noted that after the task success rate has reached the target, this step further optimizes the robot's energy efficiency by increasing the weight of the energy consumption smoothing loss term to guide the strategy towards convergence in the direction of low energy consumption and smooth motion.

[0166] Optionally, the preset success rate threshold is a pre-set satisfactory success rate, such as 90%, which is monitored by the control device during the training process to see if the current task success rate reaches the preset success rate threshold.

[0167] Optionally, joint energy consumption refers to the time integral or average of the sum of squares of all joint torques during the execution of a task by a digital twin robot. This is obtained by the control device acquiring the torque values ​​of each joint from the Modelica simulation, calculating the squares, summing them, and then dividing by the time. Optionally, baseline energy consumption refers to the average joint energy consumption value measured beforehand by running the same task using a standard strategy (e.g., a pre-trained strategy that has not undergone energy optimization), serving as a comparison benchmark.

[0168] Optionally, the energy consumption smoothing loss term L_cost is a sub-term of the adaptive joint loss function, weighted by the sum of squares of joint torques, the sum of squares of torque change rates, and the sum of squares of the second-order differences of high-level planning instructions. When the control device detects that the task success rate has reached a preset success rate threshold (e.g., ≥90%) and the current joint energy consumption is higher than the baseline energy consumption (e.g., more than 1.2 times the baseline), it adjusts the weight of the energy consumption smoothing loss term lambda_c by a preset amount (e.g., increasing it by 40%) to allow the joint optimization to penalize high-energy-consuming and non-smooth instructions while maintaining a high success rate, thus driving the strategy to evolve towards energy saving and stability.

[0169] Based on the above embodiments of this application, in another embodiment of this application, the same or similar content as the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, after step S4, the step of deploying the hierarchical control strategy to the target humanoid robot, the following is included: Step S41: Collect the operating status data and task execution results of the target humanoid robot when it performs tasks in a real physical environment; It should be noted that, in order to obtain execution feedback in a real physical environment for subsequent iterative optimization, this step involves continuously collecting runtime data after deployment on a real device.

[0170] Optionally, collecting the operational status data of the target humanoid robot when performing tasks in a real physical environment means that the control device records joint angles, joint speeds, joint torques, foot contact forces, torso postures, end effector poses, and environmental observation data (such as the position of the target object and the distribution of obstacles) in real time through sensors on the robot body (such as joint encoders, IMU inertial measurement units, foot force sensors, cameras, etc.).

[0171] Optionally, the task execution results include whether the task was successful (e.g., whether the door is fully opened in the door opening task, or whether the material is accurately placed in the target area in the material handling task), execution time, number of foot slippage occurrences, proportion of joint torque saturation frames, end-point tracking error trajectory, etc., which are used to adjust the loss weights and simulation environment parameters in the next step.

[0172] Step S42: Based on the running status data and the task execution results, optimize and adjust the weights of each loss term in the adaptive joint loss function; It should be noted that, in order to eliminate the dynamic deviation between the simulation environment and the real machine, this step uses feedback data from the real machine to recalibrate the weight coefficients in the joint loss function and supplements extreme scenario data.

[0173] Optionally, based on the running status data and the task execution results, the weights of each loss term in the adaptive joint loss function are optimized and adjusted. This means that the control device compares the physical feedback quantities (such as foot slip rate, torque saturation frame ratio, end tracking error, task success rate, and joint energy consumption) obtained from the actual machine operation with the corresponding indicators in the simulation environment and calculates the deviation coefficient. Then, the weights of each loss term can be optimized and adjusted according to the dynamic adjustment rules in steps A31 to A34.

[0174] Meanwhile, the control equipment also supplements the simulation environment with training data for corresponding extreme scenarios and out-of-distribution scenarios based on out-of-distribution conditions that appear in the real machine's operating data (such as occasional foot slippage, joint response delay, and small ground bumps in the real machine). For example, it randomly disturbs the ground friction coefficient within the range of ±0.2, and randomly adds slight collision interference or ground unevenness parameters to the simulation environment, so that the simulation environment can cover various physical deviations observed in the real machine's operation.

[0175] Optionally, the optimized weights refer to a new set of weight coefficients obtained after dynamic adjustment driven by the aforementioned real machine data.

[0176] Step S43: In the simulation environment, based on the optimized and adjusted adaptive joint loss function, the step of synchronously updating the parameters of the high-level planner and the low-level controller through the adaptive joint loss function, and obtaining the hierarchical control strategy based on the synchronously updated parameters is executed to generate the iterative hierarchical control strategy, and the iterative hierarchical control strategy is redeployed to the target humanoid robot.

[0177] It should be noted that, in order to form a closed loop of "simulation optimization - real machine deployment - data feedback - model iteration", this step uses the loss function calibrated by the real machine feedback to perform joint fine-tuning again, and then redeploys the optimized strategy to the real machine. In this process, joint fine-tuning can also be performed based on training data supplemented with extreme scenarios, or it can be performed jointly based on the loss function calibrated by the real machine feedback and the training data of extreme scenarios.

[0178] Optionally, the control device can construct a joint loss function using the optimized and adjusted new weights, and can also mix the supplementary extreme scenario data into the original demonstration dataset and sampling trajectory pool.

[0179] Optionally, the step of synchronously updating the parameters of the high-level planner and the low-level controller through the adaptive joint loss function and obtaining the hierarchical control strategy based on the synchronously updated parameters refers to repeatedly executing the joint fine-tuning process in step S3, that is, in the updated simulation environment, calculating the gradient through the adaptive joint loss function, backpropagating and updating all learnable parameters of the high-level planner and the low-level controller simultaneously until the task success rate in the simulation environment reaches a preset success threshold (e.g., above 95%).

[0180] Optionally, generating an iterative hierarchical control strategy refers to combining the high-level planner and the low-level controller, which have completed a new round of joint fine-tuning, into a new hierarchical control strategy.

[0181] Optionally, redeploying the iterative hierarchical control strategy to the target humanoid robot means replacing the original hierarchical control strategy with a new one, enabling the target humanoid robot to have better performance and stronger generalization and anti-interference capabilities when performing tasks in a real physical environment. Through the above-mentioned simulation supplementation and joint fine-tuning iteration driven by real machine data, the hierarchical control strategy can be continuously evolved in complex real environments.

[0182] In one feasible embodiment, a real-machine experiment was conducted. The hierarchical control strategy obtained through the robot control method of this application was deployed to the Booster T1 robot real-machine controller. The real-machine test results showed that the success rate reached 62% in 100 material handling tasks, and the average task completion time was shortened by 18%. When the position of the material box was moved by human intervention within ±10cm, the robot could autonomously adjust its actions and re-complete the grasping and handling, with significantly better anti-interference capabilities than the baseline model. At the same time, in real-machine tests for autonomous door opening and stair climbing tasks (indoor normal temperature, no extreme environment, target position deviation ±10cm), the success rates reached 76% and 72% respectively in 100 tests, both meeting the preset requirements. The task completion rate was high and the control performance was good.

[0183] For example, to help understand the data flow of the high-level planner in combination with the above embodiments, please refer to... Figure 2 , specifically: The high-level planner first acquires the conditional inputs (i.e., conditional input data), which specifically include the robot's body state (state), such as joint angles / velocities / torques / IMU; environmental observations (obs), such as visual point clouds / target poses / obstacles; diffusion time steps (t); foot contact encoding (cfoot) (used to encode the foot contact state and sliding velocity); and centroid stability margin (zcom) (i.e., centroid stability margin obtained based on the distance of the supporting polygons).

[0184] The conditional input enters the state-scene conditional encoder. The state, obs, cfoot, and zcom, after being concatenated through conditional embedding, are then used for feature extraction by a stacked 6-layer conditional coding layer. The internal structure of each conditional coding layer is as follows: multi-head attention (Head=8, hidden dimension 384, feedforward expansion ratio 2.5), followed by activation and feature processing via RMSnorm + SwiGLU, using pre-normalized residual connections (in the form: x + DropPath(F(RMSNorm(x)))). Furthermore, the diffusion time step t synchronously enters the diffusion time step modulation module. After sinusoidal time step embedding, FiLM modulation (t -> gamma_t, beta_t) generates scaling coefficients gamma_t and bias coefficients beta_t, which are used for feature modulation in subsequent denoising layers.

[0185] The output of the state-scene condition encoder is used as conditional features (also called conditional encoded features) and input to the motion-operation dual-branch diffusion denoising unit (composed of a motion branch denoising unit and an operation branch denoising unit). Each denoising unit contains four layers: the motion branch denoising unit outputs a latent state hm during diffusion, corresponding to the base linear velocity / angular velocity and centroid offset; the operation branch denoising unit outputs a latent state ho during diffusion, corresponding to the three-dimensional position / attitude angle of both hands and the variables of the grasping stage.

[0186] After every two denoising layers of the two denoisers, a cross-branch coupled gated residual module is inserted. This coupled gated residual module receives the hidden state hm of the motion branch, the hidden state ho of the operation branch, the foot contact code cfoot, and the centroid stability margin zcom. First, the gating factor is calculated: g = sigmoid(Wg(hm, ho, cfoot, zcom)); then, the hidden states of the two denoisers are updated respectively: the motion branch is updated to hm' = hm + g. Woho, the operation branch is updated to ho' = ho + (1 - g) Whmhm.

[0187] Finally, in the output and safety constraint stage, the updated motion hidden state hm' and the updated operation hidden state ho' are concatenated and input into the differentiable safe projection layer. During projection, this differentiable safe projection layer utilizes information from the executable envelope source for constraints. This executable envelope source information is obtained through MWORDS Modelica digital twin batch simulation and includes the physical execution limits of the underlying RL controller, specifically: the upper limit of joint torque, the boundary of the foot friction cone, the range of the center of mass support polygon, and the reachable space of the end effector. The differentiable safe projection layer uses these physical constraints to limit the predicted base velocity, hand pose, and center of mass offset, ultimately outputting the actual high-level control commands (i.e., base velocity, hand pose, and center of mass offset, also called high-level planning commands). These high-level control commands are ensured to be within the underlying physical executable envelope.

[0188] For example, to help understand the implementation flow of the robot control method obtained by combining this embodiment with the above embodiments, please refer to... Figure 3 Specifically: The robot control method provided in this application is divided into four closed-loop iterations, as follows: Phase 1: Full-parameter digital twin modeling based on MWORKS Modelica First, data is collected through hardware parameter calibration of the actual robot (i.e., the target humanoid robot). A 1:1 full-parameter digital twin model is then created using Modelica, constructing a simulation model (i.e., a digital twin robot) with over 20 degrees of freedom and multi-body contact / collision capabilities. Fidelity verification is then performed: a two-way simulation-algorithm interface (Python-Modelica real-time interaction) is established to compare the deviation between the simulation model output and the actual robot's running data. When the average deviation rate is below 5%, it is considered to meet the high-fidelity matching requirements; otherwise, accuracy iterations are performed until the simulation environment requirements are met. Finally, the twin parameters (i.e., the parameters of the simulation model) are updated using the actual robot data. Phase Two: Construction of the REFINE-DP Hierarchical Control Framework This phase comprises three steps, outputting a hierarchical control strategy: First (Data Acquisition): High-quality demonstration datasets are obtained through manual teleoperation and a heuristic planner. Second (MMCG-DiT High-Level Diffusion Planner Pre-training): The initial high-level planner (MMCG-DiT) is pre-trained using the demonstration dataset, enabling it to generate reasonable planning instructions from states and scenarios. Third (REFINE-DP Joint Fine-tuning): In a simulation environment, the pre-trained high-level planner (i.e., the pre-defined diffusion network) is combined with the underlying controller structure, and the PPO algorithm is used to synchronously fine-tune the pre-trained high-level planner and the underlying controller, ultimately outputting a hierarchical control strategy as the initial state for phase three interactive sampling.

[0189] Phase 3: Joint Fine-tuning Closed-Loop Optimization Based on Modelica Digital Twin In the MWORKS simulation environment, a randomized augmentation of the scenario is constructed, followed by joint interactive sampling. The DP planner (i.e., the high-level planner) outputs high-level planning instructions at a frequency of 10Hz, while the RL controller (i.e., the low-level controller) executes low-level actions at a frequency of 1000Hz. During execution, multi-dimensional joint reward calculations are performed (including task completion, tracking error, foot smoothness, stability, and safety, etc.), and the high-level planner and low-level controller are updated synchronously via PPO. During this process, the L_joint adaptive joint loss function adaptively adjusts the weights of its various loss terms based on physical feedback, using EMA to update or adjust the weights, with the adjustment range controlled within ±20% per round. Through iterative optimization, one criterion is "Has the task success rate met?". If yes, it indicates that the high-level planner and low-level controller have completed optimization, and the optimized hierarchical control strategy is output; if no, interactive sampling continues until the condition is met. The optimized hierarchical control strategy can serve as the initial state for the next joint fine-tuning in stage two.

[0190] Phase Four: Real Device Deployment and Virtual-Real Closed-Loop Iteration The optimized hierarchical control strategy from Phase 3 is converted into C / C++ executable code using the MWORKS code generation tool and deployed to a real machine for practical operation. During the real machine operation, all data (including operational status data) is collected. A digital twin comparative analysis is performed between the real machine data and the simulation data (i.e., comparing the data obtained from the target humanoid robot's operation with the simulation data obtained from the digital twin robot), optimizing model parameters (i.e., optimizing the digital twin robot) and loss weights (i.e., adjusting the weights of each loss term in the adaptive joint loss function). Simultaneously, based on new situations encountered during real machine operation (such as slippage, impact, etc.), datasets of extreme or out-of-distribution scenarios are supplemented. These datasets are then used to repeatedly fine-tune the high-level planner and low-level controller in the simulation environment. The fine-tuned hierarchical control strategy is used to provide feedback optimization for Phase 3 (i.e., serving as the starting point for a new round of synchronous updates in Phase 3), forming a closed-loop process of "simulation optimization - real machine deployment - data feedback - model iteration," continuously improving the control performance of the real machine.

[0191] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the robot control method of this application. Any simple variations based on this technical concept, such as the interaction and combination of various embodiments, are all within the protection scope of this application.

[0192] This application also provides a robot control device, please refer to... Figure 4 The robot control device includes: Module 10 is used to build a digital twin robot that matches the hardware parameters and physical characteristics of the target humanoid robot in a simulation environment. Loss module 20 is used to construct an adaptive joint loss function for jointly training the high-level planner and the low-level controller of the digital twin robot. The high-level planner includes a differentiable safe projection layer for boundary constraints on the high-level planning instructions output by the high-level planner based on the physical execution limit information of the low-level controller. The adaptive joint loss function includes an instruction executability loss term for quantifying the degree to which the high-level planning instructions exceed the physical execution limit information, and a planning execution distribution alignment loss term for quantifying the deviation between the feasible range of the high-level planning instructions and the capability range of the low-level executable instructions. The update module 30 is used to synchronously update the parameters of the high-level planner and the low-level controller through the adaptive joint loss function, and obtain a hierarchical control strategy based on the synchronously updated parameters; The control module 40 is used to deploy the hierarchical control strategy to the target humanoid robot and perform collaborative control of the target humanoid robot in a real physical environment.

[0193] Optionally, the robot control device includes: a planning network module for constructing a high-level planner; specifically, the network structure of the high-level planner of the digital twin robot is: a state-scene condition encoder, a diffusion time-step modulation module, a motion branch denoiser, an operation branch denoiser, a coupling gated residual module, and a differentiable safe projection layer. The state-scene condition encoder consists of 6 stacked condition coding layers. The diffusion time step modulation module is located after the state-scene condition encoder; The motion branch denoising unit is composed of four sequentially stacked motion branch denoising layers, and the operation branch denoising unit is composed of four sequentially stacked operation branch denoising layers. The coupling gated residual module is disposed between the second and third motion branch denoising layers of the motion branch denoising device and between the second and third operation branch denoising layers of the operation branch denoising device. The differentially secure projection layer is disposed after the motion branch denoiser and the operational branch denoiser.

[0194] Specifically, in the high-level planner, The state-scene condition encoder is used to receive conditional input data and output conditional coding features. The conditional input data includes the state data of the target humanoid robot, external environment data, diffusion time step, foot contact state, and task stage code. The diffusion time step modulation module is used to take the diffusion time step as input, output scaling coefficient and bias coefficient, and input the scaling coefficient and bias coefficient to each motion branch denoising layer and each operation branch denoising layer respectively, so as to modulate the denoising features of each denoising layer. The conditional coding features are respectively input into the motion branch denoiser and the operation branch denoiser, and the denoising results of each are concatenated to obtain the predicted high-level planning instruction. Specifically, after every two denoising layers of both the motion branch denoiser and the operation branch denoiser, the hidden states of the motion branch and the operation branch, along with the conditional coding features, are input into the coupled gated residual module to obtain updated hidden states of the motion branch and the operation branch. The updated hidden states of the motion branch and the operation branch are then input into the next denoising layer of the motion branch denoiser and the next denoising layer of the operation branch denoiser, respectively, to obtain their respective denoising results. The hidden states of the motion branch are obtained by diffusion denoising performed by the motion branch denoiser, and the hidden states of the operation branch are obtained by diffusion denoising performed by the operation branch denoiser. After the predicted high-rise planning instructions are input into the differentiable safe projection layer for boundary constraints, the actual high-rise planning instructions are output.

[0195] Optionally, the loss module 20 is used to determine the diffusion denoising loss term based on the consistency between the noise predicted by the diffusion network in the high-level planner and the actual noise. Based on the degree of violation of the underlying physical constraints fed back when the underlying controller executes the high-level planning instructions output by the high-level planner, the instruction executability loss item is determined, wherein the underlying physical constraints include torque saturation, foot friction cone overrun, and end reachable space overrun. The bottom tracking loss term is determined based on the deviation between the trajectory corresponding to the high-level planning instruction output by the high-level planner and the actual execution trajectory of the bottom-level controller. Based on the update step size limit of the underlying controller during policy updates, determine the pruning target loss term; The overall stability loss term is determined based on the minimum distance from the projection point of the center of mass of the digital twin robot on the horizontal plane along the direction of gravity to each side of the supporting polygon, the torso angular velocity and foot sliding velocity of the digital twin robot. Based on the deviation between the feasible range of the high-level planning instructions output by the high-level planner and the capability range of the low-level executable instructions of the low-level controller, the planning execution distribution alignment loss term is determined. Based on the joint torques, torque change rates collected during the task execution of the digital twin robot, and the high-level planning instructions output by the high-level planner, the energy consumption smoothing loss term is determined. An adaptive joint loss function is constructed based on the diffusion denoising loss term, the instruction executability loss term, the underlying tracking loss term, the pruning target loss term, the overall system stability loss term, the planning execution distribution alignment loss term, and the energy consumption smoothing loss term.

[0196] Optionally, the update module 30 is used to obtain the physical feedback quantity of the digital twin robot during the process of the lower-level controller executing the high-level planning instructions output by the high-level planner; Based on the physical feedback quantity, the weights of each loss term in the adaptive joint loss function are dynamically adjusted to obtain the adjusted weights; The adaptive joint loss function is updated with the adjusted weights, the loss value of the adaptive joint loss function is calculated, and the gradient of the loss value is backpropagated to the high-level planner and the low-level controller, and the parameters of the high-level planner and the low-level controller are updated synchronously.

[0197] Optionally, the update module 30 is used to increase the weight of the overall stability loss term and decrease the weight of the underlying tracking loss term when the foot slip rate exceeds a preset slip rate threshold. When the proportion of joint torque saturation exceeds a preset saturation threshold, the weight of the instruction executability loss item is increased; After a preset number of consecutive task executions, if the end tracking error decreases multiple times consecutively and the task success rate does not increase compared to the previous time, the weight of the planning execution distribution alignment loss term is increased. When the task success rate reaches a preset success rate threshold and the joint energy consumption of the digital twin robot is higher than the baseline energy consumption, the weight of the energy consumption smoothing loss term is increased.

[0198] Optionally, the robot control device includes: a feedback adjustment module, used to collect the operating status data and task execution results of the target humanoid robot when performing tasks in a real physical environment; Based on the running status data and the task execution results, the weights of each loss term in the adaptive joint loss function are optimized and adjusted; In the simulation environment, based on the optimized and adjusted adaptive joint loss function, the steps of synchronously updating the parameters of the high-level planner and the low-level controller through the adaptive joint loss function, and obtaining the hierarchical control strategy based on the synchronously updated parameters are executed to generate an iterative hierarchical control strategy, and the iterative hierarchical control strategy is redeployed to the target humanoid robot.

[0199] The robot control device provided in this application, employing the robot control method described in the above embodiments, can solve the technical problem in existing solutions where the high-level planner lacks information about the actual physical execution limits that the low-level controller can provide, resulting in poor coordination of humanoid robots and low task success rates. Compared with the prior art, the beneficial effects of the robot control device provided in this application are the same as those of the robot control method provided in the above embodiments, and other technical features in the robot control device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0200] This application provides a humanoid robot, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the robot control method in the first embodiment described above.

[0201] The following is for reference. Figure 5 It shows a structural schematic diagram suitable for implementing the humanoid robot of the present application embodiments. Figure 5 The humanoid robot shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0202] like Figure 5As shown, the humanoid robot may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the humanoid robot. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the humanoid robot to communicate wirelessly or wiredly with other devices to exchange data. Although the diagram shows a humanoid robot with various systems, it should be understood that it is not required to implement or possess all of the systems shown. Alternatively, more or fewer systems may be implemented.

[0203] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0204] The humanoid robot provided in this application, employing the robot control method described in the above embodiments, solves the technical problem in existing solutions where the high-level planner lacks information about the actual physical execution limits provided by the low-level controller, resulting in poor coordination of the humanoid robot and a low task success rate. Compared with the prior art, the beneficial effects of the humanoid robot provided in this application are the same as those of the robot control method provided in the above embodiments, and other technical features of this humanoid robot are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0205] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0206] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0207] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the robot control method in the above embodiments.

[0208] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0209] The aforementioned computer-readable storage medium may be included in the humanoid robot; or it may exist independently and not assembled into the humanoid robot.

[0210] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0211] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0212] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0213] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described robot control method. This addresses the technical problem in existing solutions where the high-level planner lacks information about the actual physical execution limits provided by the low-level controller, resulting in poor coordination of the humanoid robot and a low task success rate. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the robot control method provided in the above embodiments, and will not be elaborated upon here.

[0214] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the robot control method described above.

[0215] The computer program product provided in this application solves the technical problem in existing solutions where the high-level planner lacks information about the actual physical execution limits that the low-level controller can provide, resulting in poor coordination of humanoid robots and low task success rates. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the robot control method provided in the above embodiments, and will not be repeated here.

[0216] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A robot control method, characterized in that, The robot control method includes: In a simulation environment, a digital twin robot is constructed that matches the hardware parameters and physical characteristics of the target humanoid robot; An adaptive joint loss function is constructed for jointly training the high-level planner and the low-level controller of the digital twin robot. The high-level planner includes a differentiable safe projection layer for boundary constraints on the high-level planning instructions output by the high-level planner based on the physical execution limit information of the low-level controller. The adaptive joint loss function includes an instruction executability loss term for quantifying the degree to which the high-level planning instructions exceed the physical execution limit information, and a planning execution distribution alignment loss term for quantifying the deviation between the feasible range of the high-level planning instructions and the capability range of the low-level executable instructions. The parameters of the high-level planner and the low-level controller are updated synchronously using the adaptive joint loss function, and a hierarchical control strategy is obtained based on the synchronously updated parameters. The hierarchical control strategy is deployed to the target humanoid robot to perform collaborative control of the target humanoid robot in a real physical environment.

2. The method as described in claim 1, characterized in that, The network structure of the high-level planner of the digital twin robot is as follows: a state-scene condition encoder, a diffusion time-step modulation module, a motion branch denoiser, an operation branch denoiser, a coupling gated residual module, and a differentiable safe projection layer. The state-scene condition encoder consists of 6 stacked condition coding layers. The diffusion time step modulation module is located after the state-scene condition encoder; The motion branch denoising unit is composed of four sequentially stacked motion branch denoising layers, and the operation branch denoising unit is composed of four sequentially stacked operation branch denoising layers. The coupling gated residual module is disposed between the second and third motion branch denoising layers of the motion branch denoising device and between the second and third operation branch denoising layers of the operation branch denoising device. The differentially secure projection layer is disposed after the motion branch denoiser and the operational branch denoiser.

3. The method as described in claim 2, characterized in that, In the high-level planner, The state-scene condition encoder is used to receive conditional input data and output conditional coding features. The conditional input data includes the state data of the target humanoid robot, external environment data, diffusion time step, foot contact state, and task stage code. The diffusion time step modulation module is used to take the diffusion time step as input, output scaling coefficient and bias coefficient, and input the scaling coefficient and bias coefficient to each motion branch denoising layer and each operation branch denoising layer respectively, so as to modulate the denoising features of each denoising layer. The conditional coding features are respectively input into the motion branch denoiser and the operation branch denoiser, and the denoising results of each are concatenated to obtain the predicted high-level planning instruction. Specifically, after every two denoising layers of both the motion branch denoiser and the operation branch denoiser, the hidden states of the motion branch and the operation branch, along with the conditional coding features, are input into the coupled gated residual module to obtain updated hidden states of the motion branch and the operation branch. The updated hidden states of the motion branch and the operation branch are then input into the next denoising layer of the motion branch denoiser and the next denoising layer of the operation branch denoiser, respectively, to obtain their respective denoising results. The hidden states of the motion branch are obtained by diffusion denoising performed by the motion branch denoiser, and the hidden states of the operation branch are obtained by diffusion denoising performed by the operation branch denoiser. After the predicted high-rise planning instructions are input into the differentiable safe projection layer for boundary constraints, the actual high-rise planning instructions are output.

4. The method as described in claim 1, characterized in that, The step of constructing an adaptive joint loss function for jointly training the high-level planner and the low-level controller of the digital twin robot includes: The diffusion denoising loss term is determined based on the consistency between the noise predicted by the diffusion network in the high-level planner and the actual noise. Based on the degree of violation of the underlying physical constraints fed back when the underlying controller executes the high-level planning instructions output by the high-level planner, the instruction executability loss item is determined, wherein the underlying physical constraints include torque saturation, foot friction cone overrun, and end reachable space overrun. The bottom tracking loss term is determined based on the deviation between the trajectory corresponding to the high-level planning instruction output by the high-level planner and the actual execution trajectory of the bottom-level controller. Based on the update step size limit of the underlying controller during policy updates, determine the pruning target loss term; The overall stability loss term is determined based on the minimum distance from the projection point of the center of mass of the digital twin robot on the horizontal plane along the direction of gravity to each side of the supporting polygon, the torso angular velocity and foot sliding velocity of the digital twin robot. Based on the deviation between the feasible range of the high-level planning instructions output by the high-level planner and the capability range of the low-level executable instructions of the low-level controller, the planning execution distribution alignment loss term is determined. Based on the joint torques, torque change rates collected during the task execution of the digital twin robot, and the high-level planning instructions output by the high-level planner, the energy consumption smoothing loss term is determined. An adaptive joint loss function is constructed based on the diffusion denoising loss term, the instruction executability loss term, the underlying tracking loss term, the pruning target loss term, the overall system stability loss term, the planning execution distribution alignment loss term, and the energy consumption smoothing loss term.

5. The method as described in claim 1, characterized in that, The step of synchronously updating the parameters of the high-level planner and the low-level controller using the adaptive joint loss function includes: During the execution of the high-level planning instructions output by the high-level planner by the low-level controller, the physical feedback quantity of the digital twin robot is obtained; Based on the physical feedback quantity, the weights of each loss term in the adaptive joint loss function are dynamically adjusted to obtain the adjusted weights; The adaptive joint loss function is updated with the adjusted weights, the loss value of the adaptive joint loss function is calculated, and the gradient of the loss value is backpropagated to the high-level planner and the low-level controller, and the parameters of the high-level planner and the low-level controller are updated synchronously.

6. The method as described in claim 5, characterized in that, The physical feedback quantities include foot slip rate, joint torque saturation ratio, and end-effector tracking error. The adaptive joint loss function includes instruction executability loss, low-level tracking loss, overall system stability loss, planning execution distribution alignment loss, and energy consumption smoothing loss. The step of dynamically adjusting the weights of each loss term in the adaptive joint loss function based on the physical feedback quantities includes: When the foot slip rate exceeds a preset slip rate threshold, the weight of the overall stability loss term is increased, and the weight of the underlying tracking loss term is decreased. When the proportion of joint torque saturation exceeds a preset saturation threshold, the weight of the instruction executability loss item is increased; After a preset number of consecutive task executions, if the end tracking error decreases multiple times consecutively and the task success rate does not increase compared to the previous time, the weight of the planning execution distribution alignment loss term is increased. When the task success rate reaches a preset success rate threshold and the joint energy consumption of the digital twin robot is higher than the baseline energy consumption, the weight of the energy consumption smoothing loss term is increased.

7. The method as described in claim 1, characterized in that, Following the step of deploying the hierarchical control strategy to the target humanoid robot, the following is included: Collect operational status data and task execution results of the target humanoid robot when it performs tasks in a real physical environment; Based on the running status data and the task execution results, the weights of each loss term in the adaptive joint loss function are optimized and adjusted; In the simulation environment, based on the optimized and adjusted adaptive joint loss function, the steps of synchronously updating the parameters of the high-level planner and the low-level controller through the adaptive joint loss function, and obtaining the hierarchical control strategy based on the synchronously updated parameters are executed to generate an iterative hierarchical control strategy, and the iterative hierarchical control strategy is redeployed to the target humanoid robot.

8. A robot control device, characterized in that, The robot control device includes: The building blocks are used to construct digital twin robots that match the hardware parameters and physical characteristics of the target humanoid robot in a simulation environment. The loss module is used to construct an adaptive joint loss function for jointly training the high-level planner and the low-level controller of the digital twin robot. The high-level planner includes a differentiable safe projection layer for boundary constraints on the high-level planning instructions output by the high-level planner based on the physical execution limit information of the low-level controller. The adaptive joint loss function includes an instruction executability loss term for quantifying the degree to which the high-level planning instructions exceed the physical execution limit information, and a planning execution distribution alignment loss term for quantifying the deviation between the feasible range of the high-level planning instructions and the capability range of the low-level executable instructions. The update module is used to synchronously update the parameters of the high-level planner and the low-level controller through the adaptive joint loss function, and obtain the hierarchical control strategy based on the synchronously updated parameters; The control module is used to deploy the hierarchical control strategy to the target humanoid robot and perform collaborative control of the target humanoid robot in a real physical environment.

9. A humanoid robot, characterized in that, The humanoid robot includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the robot control method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the robot control method as described in any one of claims 1 to 7.

Images