Movement control method and device of humanoid robot, terminal equipment and storage medium

Abstract

The invention discloses a movement control method and device of a humanoid robot, terminal equipment and a storage medium, and belongs to the technical field of robot control, and the method comprises the steps: training a movement control model of the humanoid robot through a teacher-student distillation training mode, and carrying out the distillation training in a simulation environment, and constructing a simulation model of the humanoid robot and a plurality of simulation terrain point clouds, and training the teacher network. Further compressing input data input into the student network, and performing distillation training on the student network by adopting the compressed training data set by taking the difference minimization between a second joint driving instruction output by the student network and the first joint driving instruction as a target; the trained student network not only maintains the strong ability of the teacher model to cope with complex terrains, but also meets the real-time requirement of the humanoid robot for the control strategy, and overcomes the problem of poor movement stability of the humanoid robot in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular to a mobile control method and device for a humanoid robot, a terminal device and a storage medium. BACKGROUND

[0002] In the current field of humanoid robot perception and control, complex terrains such as steps, slopes, gravel and deep snow, and other unstructured environments, pose a serious challenge to the dynamic stability and autonomous navigation capability of robots. To improve the terrain adaptability of robots, the mainstream method generally adopts the strategy of large-scale training in a simulation environment. By constructing a large number of complex terrain scenes in a virtual world to train the control strategy network, it learns how to maintain balance and efficiently move under various extreme conditions.

[0003] However, in order to cope with increasingly complex terrain environments, the structural design of the control strategy network has to be deep and complex. Although these large models exhibit excellent performance in simulation, the cost is a very high computational complexity and resource demand. When these models that perform well in ideal simulation environments are deployed to physical robots, their performance often deteriorates significantly due to the limited computing power of embedded hardware, real-time requirements, and real sensor noise. The gap between the theoretical performance in the simulation environment and the actual performance in the real world directly leads to the fact that the adaptability of current humanoid robots in real complex terrains is still far from the level shown in the simulation environment, which seriously restricts their widespread application in real-world scenarios. Therefore, the existing technology still has the problem of poor movement stability of the humanoid robot. SUMMARY

[0004] The present application provides a mobile control method and device for a humanoid robot, a terminal device and a storage medium, which can solve the problem of poor movement stability of the humanoid robot in the prior art.

[0005] An embodiment of the present application provides a mobile control method for a humanoid robot, comprising: obtaining body pose data of the humanoid robot and terrain point cloud data; compressing the terrain point cloud data to generate a target terrain feature vector, and inputting the body pose data and the target terrain feature vector into a preset mobile control model to make the mobile control model output joint driving instructions of the humanoid robot; controlling the humanoid robot to move according to the joint driving instructions; wherein the construction of the mobile control model comprises: constructing a teacher network and a student network, wherein the student network is a lightweight network model; In a preset simulator, a simulation model of the humanoid robot, a plurality of simulation postures of the simulation model, and a plurality of simulation terrain point clouds are constructed; The teacher network is trained by taking the movement stability and the movement efficiency maximization of the humanoid robot as a target and by using the simulation postures and the simulation terrain point clouds. The simulation postures and the simulation terrain point clouds are combined into a plurality of initial data groups in a random manner, and the initial data groups are input into the trained teacher network, so that the teacher network outputs a plurality of first joint driving instructions. The simulation postures in each initial data group are pruned, and the simulation terrain point clouds are compressed, to construct a training data group; wherein the data dimension of the pruned simulation postures is the same as the data dimension of the body posture data. The student network is distilled trained by taking the minimization of the difference between the second joint driving instruction output by the student network and the first joint driving instruction as a target, and using the training data group, and the trained student network is used as the movement control model.

[0006] Further, the teacher network is trained by taking the movement stability and the movement efficiency maximization of the humanoid robot as a target and by using the simulation postures and the simulation terrain point clouds, comprising: An reward function is constructed by taking the movement stability and the movement efficiency maximization of the humanoid robot as a target. An agent is constructed, and the agent is used to repeatedly perform optimization operations on the teacher network until the training of the teacher network is completed. The optimization operation comprises: An optimization teacher network is obtained; wherein initially, the optimization teacher network is an untrained teacher network. A plurality of state sequence samples are constructed according to the simulation postures and the simulation terrain point clouds, so that the agent generates an action sequence composed of a plurality of joint driving instructions according to each state sequence sample and the optimization teacher network, and updates the simulation model according to the action sequence. According to the reward function and the response state sequence generated after the simulation model is updated, a corresponding reward function value is generated, and a plurality of experience sequences are constructed according to the state sequence sample, the action sequence, the reward function value, and the response state sequence. The optimization teacher network is optimized according to the experience sequence to generate an evaluation teacher network. When a plurality of reward function values tend to converge, it is determined that the training of the teacher network is completed; otherwise, the evaluation teacher network is used as the optimization strategy network of the next round of optimization operation.

[0007] Further, the state sequence samples are constructed according to the simulation posture and the simulation terrain point cloud, including: An iteration number of the optimization operation is obtained. When the iteration number is greater than a first preset threshold, random disturbance is injected into the simulation terrain point cloud to generate a first simulation terrain point cloud, and the state sequence samples are constructed according to the simulation posture and the first simulation terrain point cloud. When the iteration number is greater than a second preset threshold, random disturbance is injected into the first simulation terrain point cloud to generate a second simulation terrain point cloud, and the state sequence samples are constructed according to the simulation posture and the second simulation terrain point cloud; wherein the first preset threshold is less than the second preset threshold.

[0008] Further, the compression of the simulation terrain point cloud includes: The simulation terrain point cloud is input into a preset spatial compression model, so that the spatial compression model identifies the spatial context features between the point clouds in the simulation terrain point cloud, and outputs a terrain feature vector representing the terrain features of the simulation terrain point cloud according to the spatial context features.

[0009] Further, the student network is distilled and trained by using the training data set, with the difference between the second joint driving instruction output by the student network and the first joint driving instruction being minimized, including: A loss function for calculating the mean square error of each joint angle increment of the second joint driving instruction and the first joint driving instruction is constructed The student network is iteratively trained by using the training data set, and in each round of iterative training, the loss function value is calculated according to the second joint driving instruction output by the student network for each training data set and the corresponding first joint driving instruction by using the loss function, the network parameters of the student network are adjusted according to the loss function value when the loss function value does not converge, and the student network training is determined to be completed when the loss function value converges.

[0010] An embodiment of the present application also provides a mobile control device of a humanoid robot, including: A data acquisition module is configured to acquire body posture data and terrain point cloud data of the humanoid robot. An instruction generation module is configured to compress the terrain point cloud data to generate a target terrain feature vector, and input the body posture data and the target terrain feature vector into a preset mobile control model, so that the mobile control model outputs a joint driving instruction of the humanoid robot. An instruction driving module is configured to control the humanoid robot to move according to the joint driving instruction. The construction of the movement control model comprises: A teacher network and a student network are constructed, wherein the student network is a lightweight network model. In a preset simulator, a simulation model of the humanoid robot, a plurality of simulation poses of the simulation model, and a plurality of simulation terrain point clouds are constructed. The teacher network is trained by the simulation poses and the simulation terrain point clouds, with the movement stability of the humanoid robot and the maximization of movement efficiency as the target. The simulation poses and the simulation terrain point clouds are randomly combined into a plurality of initial data groups, and the initial data groups are input into the trained teacher network, so that the teacher network outputs a plurality of first joint driving instructions. The simulation poses in each initial data group are pruned, and the simulation terrain point clouds are compressed to construct a training data group, wherein the data dimension of the pruned simulation pose is the same as that of the body pose data. The student network is distilled trained by the training data group, with the minimization of the difference between the second joint driving instruction output by the student network and the first joint driving instruction as the target, and the trained student network is used as the movement control model.

[0011] Further, the training of the teacher network by the simulation poses and the simulation terrain point clouds, with the movement stability of the humanoid robot and the maximization of movement efficiency as the target, comprises: A reward function is constructed with the movement stability of the humanoid robot and the maximization of movement efficiency as the target. An agent is constructed, and the agent is repeatedly used to optimize the teacher network until the training of the teacher network is completed. The optimization operation comprises: An optimized teacher network is obtained, wherein initially, the optimized teacher network is an untrained teacher network. A plurality of state sequence samples are constructed according to the simulation poses and the simulation terrain point clouds, so that the agent generates an action sequence composed of a plurality of joint driving instructions according to each state sequence sample and the optimized teacher network, and updates the simulation model according to the action sequence. A corresponding reward function value is generated according to the reward function and the response state sequence generated after the simulation model is updated, and a plurality of experience sequences are constructed according to the state sequence sample, the action sequence, the reward function value, and the response state sequence. Based on the aforementioned experience sequence, the teacher network to be optimized is optimized to generate a teacher network to be evaluated. When several reward function values ​​tend to converge, the teacher network training is considered complete; otherwise, the teacher network to be evaluated is used as the policy network to be optimized in the next round of optimization.

[0012] Furthermore, based on the simulated pose and the simulated terrain point cloud, several state sequence samples are constructed, including: Get the number of iterations for the optimization operation; When the number of iterations exceeds a first preset threshold, random perturbations are injected into several of the simulated terrain point clouds to generate a first simulated terrain point cloud, and several state sequence samples are constructed based on the simulated pose and the first simulated terrain point cloud. When the number of iterations exceeds the second preset threshold, random perturbation is injected into the first simulated terrain point cloud to generate a second simulated terrain point cloud, and several state sequence samples are constructed based on the simulated pose and the second simulated terrain point cloud; wherein, the first preset threshold is less than the second preset threshold.

[0013] This application also provides a terminal device, including: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a humanoid robot movement control method as described in the above embodiments of the invention.

[0014] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a humanoid robot movement control method as described in the above embodiments.

[0015] The following benefits can be obtained by implementing the present invention: This invention provides a method, apparatus, terminal device, and storage medium for the movement control of a humanoid robot. The method trains a humanoid robot's movement control model through a teacher-student distillation training approach. In a simulation environment, a simulation model of the humanoid robot and several simulated terrain point clouds are constructed to train the teacher network, enabling it to fully utilize the advantages of the simulation environment and learn terrain adaptation strategies. The input data to the student network is then compressed to improve its operational efficiency. With the goal of minimizing the difference between the second joint drive commands output by the student network and the first joint drive commands, the compressed training data is used to distill the student network. This ensures that the trained student network maintains the teacher model's strong ability to handle complex terrain while meeting the real-time requirements of the humanoid robot's control strategy, effectively improving the movement stability of the humanoid robot equipped with the movement control model and overcoming the problem of poor movement stability in existing humanoid robots. Attached Figure Description

[0016] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a motion control method for a humanoid robot according to a certain embodiment of this application; Figure 2 This is a schematic diagram of the structure of a humanoid robot motion control device provided in a certain embodiment of this application; Figure 3 This is a schematic diagram of the structure of a terminal device provided in a certain embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0020] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0023] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0024] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0025] See Figure 1 To address the problems in the prior art, an embodiment of the present invention provides a method for controlling the movement of a humanoid robot, comprising: S1. Acquire the body posture data and terrain point cloud data of the humanoid robot; In a preferred embodiment of the present invention, the terrain point cloud data is the point cloud data within a certain range in the forward direction detected by the radar on the humanoid robot. The body posture data includes joint data and core posture data. The joint data includes: 12 joint positions (12-dimensional) and velocity (12-dimensional), for a total of 24 dimensions. The core posture includes: Euler angles (3-dimensional), three-dimensional position (3-dimensional), and 2 foot contact forces (front / rear foot, 2-dimensional), for a total of 8 dimensions.

[0026] S2. Compress the terrain point cloud data to generate a target terrain feature vector, and input the body posture data and the target terrain feature vector into a preset motion control model so that the motion control model outputs the joint drive commands of the humanoid robot. S3. Control the humanoid robot to move according to the joint drive command; The construction of the motion control model includes: S21. Construct a teacher network and a student network, wherein the student network is a lightweight network model; S22. In a preset simulator, construct a simulation model of the humanoid robot, several simulation poses of the simulation model, and several simulation terrain point clouds. In a preferred embodiment of the present invention, the simulated posture is the robot's full-state information (64-dimensional), specifically including: joint data: position (12-dimensional), velocity (12-dimensional), and torque (12-dimensional) of 12 joints, totaling 36 dimensions (each leg has 6 joints, and the humanoid robot has 2 legs; each joint is one-dimensional, so there are 12 dimensions of joint angle, velocity, and torque input); posture and position: Euler angle rotation (3-dimensional: a method for describing the rotational speed of an object in three-dimensional space, namely: yaw, pitch, roll), quaternions (4-dimensional: using four parameters (w, x, y, ...). The system uses 13 dimensions to represent rotation, 3D position (X, Y, Z position of the object in world coordinates), and 3D acceleration (3D gravity acting on the object in the X, Y, Z directions in the world), and interactive feedback: 12 foot contact forces (12D: forces and torques acting on the two feet in the X, Y, Z directions), gait cycle (2D: gait cycle duration (T) and stride length (L)), and leg lift height (1D: the expected height of the foot in the Z direction when the robot takes a step), for a total of 15 dimensions.

[0027] The simulated terrain point cloud is the point cloud data of a 5m×3m area in front of the simulation model. Specifically, it is downsampled to 16×16×6 voxel features (16×16×6=1536 dimensions) through VoxelGrid, then compressed to 16×8×8=1024 dimensions through one convolutional layer (3×3×3 kernel, 16 output channels), and finally reduced to 192 dimensions through a fully connected layer.

[0028] S23. With the goal of maximizing the movement stability and movement efficiency of the humanoid robot, the teacher network is trained using the simulated posture and the simulated terrain point cloud. Preferably, the step of maximizing the movement stability and efficiency of the humanoid robot, and training the teacher network using the simulated posture and the simulated terrain point cloud, includes: A reward function is constructed with the goal of maximizing the mobility stability and efficiency of the humanoid robot; an intelligent agent is constructed, and the intelligent agent is used to repeatedly perform optimization operations on the teacher network until the teacher network training is completed; wherein, the optimization operations include: A teacher network to be optimized is obtained; initially, the teacher network to be optimized is an untrained teacher network; based on the simulated pose and the simulated terrain point cloud, several state sequence samples are constructed so that the agent generates an action sequence consisting of several joint drive commands based on each state sequence sample and the teacher network to be optimized, and updates the simulation model based on the action sequence; based on the reward function and the response state sequence generated after the simulation model is updated, a corresponding reward function value is generated, and several experience sequences are constructed based on the state sequence samples, action sequences, reward function values, and response state sequences; the teacher network to be optimized is optimized based on the experience sequences to generate a teacher network to be evaluated; when several reward function values ​​tend to converge, the teacher network training is determined to be complete; otherwise, the teacher network to be evaluated is used as the policy network to be optimized in the next round of optimization.

[0029] Preferably, the step of constructing several state sequence samples based on the simulated attitude and the simulated terrain point cloud includes: The number of iterations for the optimization operation is obtained; when the number of iterations is greater than a first preset threshold, random perturbations are injected into several simulated terrain point clouds to generate a first simulated terrain point cloud, and several state sequence samples are constructed based on the simulated attitude and the first simulated terrain point cloud; when the number of iterations is greater than a second preset threshold, random perturbations are injected into the first simulated terrain point cloud to generate a second simulated terrain point cloud, and several state sequence samples are constructed based on the simulated attitude and the second simulated terrain point cloud; wherein, the first preset threshold is less than the second preset threshold.

[0030] In a preferred embodiment of the present invention, the teacher network structure includes 4 layers of feature extraction and 2 layers of task branches. Specifically, the feature extraction layers (4 layers in total, all with ReLU activation function) are as follows: Layer 1: Fully connected layer, input 256 dimensions → output 512 dimensions, dropout probability 0.2 (to prevent overfitting); Layer 2: Fully connected layer, 512 dimensions → 1024 dimensions, with layer normalization (LayerNorm); Layer 3: Residual connected layer, 1024 dimensions → 1024 dimensions (to alleviate gradient vanishing and adapt to complex terrain feature learning); Layer 4: Fully connected layer, 1024 dimensions → 512 dimensions, outputting a "global feature vector" (fusing state and terrain information). Secondly, the task branch layer (2 parallel branches, adapted for multi-objective optimization) has the following branches: stability branch: 512-dimensional → 256-dimensional → 64-dimensional, outputting "attitude correction coefficients" (3-dimensional, corresponding to roll / pitch / yaw angle adjustments); gait planning branch: 512-dimensional → 256-dimensional → 64-dimensional, outputting "joint control commands" (12-dimensional, corresponding to 12 joint angle increments) + "gait parameters" (2-dimensional, gait period T and stride length L).

[0031] The input to the teacher network consists of the robot's full state information (64-dimensional) and terrain point cloud features (192-dimensional), for a total of 256 dimensions. The output consists of core motion commands: 12 joint angle increments (±5° range, 0.1° accuracy); and auxiliary control parameters: gait period T (0.5-1.0s), stride length L (0.2-0.4m), and posture correction coefficient (±2° range).

[0032] It should be further explained that a noise-free ideal training platform is constructed in the simulation environment. By inputting the full state information of the humanoid robot under various terrains and the noise-free terrain height map, the teacher policy is obtained by training with reinforcement learning algorithm. The full state information includes the position, joint speed, body posture and position information of each joint of the robot. The terrain height map is the local terrain data in front of or around the robot. During the teacher strategy training process, a course learning mechanism was adopted to gradually increase the difficulty of the terrain, while domain randomization was carried out to inject various disturbance factors into the simulation environment, including terrain deformation, step height disturbance, perceived noise and friction coefficient changes. Specifically, the training of the teacher's strategy adopts a reinforcement learning algorithm with privileged information. The reinforcement learning strategy uses a noiseless height map and the robot's full state as joint inputs. It is optimized by setting a multi-objective reward function that includes trajectory tracking, joint smoothing and posture stabilization, so that the generated action sequence has high stability and high transferability under different terrains.

[0033] Reward function of teacher network Employing a weighted summation approach, this method focuses on optimizing both stability and efficiency. It avoids predefined motion references (such as periodic contact or fixed gait) and constrains robot behavior solely through regularization, as shown in the following formula: ; Among them, linear velocity tracking reward This is used to measure the consistency between the robot's actual linear velocity and the commanded velocity, ensuring efficient movement. ; in, The actual linear velocity of the robot's joints. For command linear velocity, This is the attenuation coefficient.

[0034] Angular velocity tracking reward It is used to ensure steering accuracy and works in conjunction with linear velocity tracking to achieve efficient movement. ; in, This refers to the actual angular velocity at the robot's joints. The command angular velocity, This is the attenuation coefficient.

[0035] Attitude stability reward Used to constrain the fuselage attitude and prevent tipping or excessive tilting: ; in, , These are the fuselage roll angle and pitch angle (yaw angle is unconstrained to accommodate steering requirements). This represents the attitude constraint coefficient.

[0036] Smoothness of reward It is used to suppress abrupt changes in joint motion, thereby improving stability and mechanical life. ; in, Total number of joints ( ), For the first angular velocity of each joint This is the smoothing coefficient.

[0037] Joint force punishment This is used to avoid joint overload and reduce mechanical wear. ; in, For the first The torque of each joint, This is the maximum permissible torque of the joint (determined by the robot's hardware parameters).

[0038] Collision Penalty This is used to prevent the aircraft from making non-foot contact with the ground (such as knee contact). ; Based on the objectives of unleashing robot capabilities and balancing stability and efficiency, the weighting coefficients are set as follows: (linear velocity) (angular velocity) (Attitude stability) (Smoothness) (Joint force penalty) (Collision penalty), satisfied To ensure consistent reward standards.

[0039] This implementation introduces privileged information during the teacher policy training phase to construct a reinforcement learning system that accelerates policy convergence and enhances the quality of the output action sequence. Privileged information refers to complete observation information that is available in a simulation training environment but not in actual deployment, such as full-state information and high-precision noise-free terrain height maps. The input includes the position and velocity data of all robot joints, the position and attitude of the robot body in the world coordinate system, and the terrain elevation image in front of the robot's direction of movement. This height map is a continuously rasterized two-dimensional matrix that reflects the local terrain undulations.

[0040] Reinforcement learning algorithms (such as Proximal Policy Optimization, PPO) use the aforementioned joint input as the input layer of the policy network, optimizing the policy by continuously interacting with the environment, performing actions, and receiving reward feedback. To improve the stability and generalization ability of the action sequence, a multi-objective reward function system is designed, including: a trajectory tracking error term, which measures the deviation between the robot's actual position and the desired trajectory; a joint motion smoothness term, which penalizes drastic changes in angular velocity to suppress jitter; and a posture stability term, which maintains center of gravity balance by constraining the robot's tilt angle. In addition, auxiliary objectives such as energy consumption penalty and contact stability reward can be added to guide the policy towards convergence in a stable, energy-efficient, and highly adaptable direction.

[0041] During training, the terrain samples in the environment cover a variety of structures, including flat land, slopes, steps, and undulating random terrain. The strategy learns to cope with the situation through repeated trials, forming a highly robust and generalizable action generation mechanism. The final generated action sequence has excellent terrain adaptability and platform portability.

[0042] During the iterative training phase, when the number of rounds is between 1 and 5000, training is conducted using unperturbed simulated terrain point clouds, with the goals of gait stability, joint smoothness, and posture balance. When the number of rounds is between 5001 and 15000, training is conducted using 50% intensity perturbation, with terrain adaptation as the perturbation correction. When the number of rounds is 15001+, training is conducted using 100% intensity plus extreme perturbation, with the goals of balancing multiple objectives: efficiency, energy consumption, and stability.

[0043] This implementation introduces privileged information from the simulation environment, enabling the teacher's strategy to fully utilize more precise state observations and terrain structure information during the training phase. This accelerates training convergence and yields a superior control strategy. The multi-objective reward function system provides multi-dimensional constraints and optimizations for robot behavior, effectively improving the stability, naturalness, and safety of the action execution process. The resulting action sequences exhibit stronger generalization ability and robustness when facing complex and unknown terrain, providing high-quality supervision signals for subsequent student strategy imitation learning, significantly enhancing the transfer performance and practical value of the entire perception-control method.

[0044] Domain randomization perturbation further includes perturbation of the parameters of the physical environment in which the robot is located. Specifically, it includes randomly injecting friction coefficient perturbation, load perturbation and terrain material changes during training. The combination of parameter perturbations improves the robustness and generalization ability of the strategy in actual deployment.

[0045] This implementation introduces an enhanced domain randomization mechanism during teacher strategy training, specifically dynamically perturbing parameters closely related to physical interactions in the simulation environment to improve the adaptability and fault tolerance of the strategy in real-world environments. This mechanism automatically samples and perturbs key environmental physical parameters before each training episode begins, including: Friction coefficient perturbation: In the contact model between the robot's foot and the ground, set the upper and lower fluctuation range of the sliding friction coefficient, such as randomly taking values ​​between 0.4 and 1.0, to simulate friction changes between wet and slippery surfaces, rough surfaces, or different material surfaces.

[0046] Load perturbation: Introduce variable additional mass blocks into the robot model to simulate uneven loads on the equipment or environment. Perturbation methods include applying loads of different directions and masses to the upper limbs, back, or torso to dynamically change the center of mass distribution and interfere with the original gait control balance strategy.

[0047] Terrain Material Variation: By switching the material properties of the terrain contact surface, such as different combinations of contact response parameters (e.g., elastic modulus, damping coefficient, surface stiffness, etc.) from hard cement to gravel, mud, ice, etc., the traversability and contact feedback of the terrain can be simulated in a variety of ways.

[0048] The aforementioned physical perturbation parameters are all randomly sampled using a defined statistical distribution (such as a uniform or normal distribution) and continuously applied during training to ensure that the strategy forms a stable response strategy under environments with high variance and high uncertainty. By injecting more challenging perturbation combinations during the simulation phase, the strategy retains strong decision-making adaptability when faced with uncontrollable or difficult-to-model situations in reality.

[0049] This implementation significantly enhances the tolerance of the teacher's strategy to variable factors in the real physical world. By introducing friction coefficient perturbations during the training phase, the strategy can adapt to slippage or skidding caused by sudden changes in ground friction; by introducing load perturbations, the strategy can maintain gait stability even when the center of mass changes dynamically; and by changing terrain material, the robot learns to adjust its gait strategy under different ground tactile sensations. The high-dimensional parameter perturbation space formed by the combination of these three perturbation types effectively improves the robustness and generalization performance of the strategy, avoids deployment failures caused by overly idealized training environments, and provides a strong guarantee for the real-world transfer of the strategy.

[0050] S24. Randomly combine the simulated posture and the simulated terrain point cloud into several initial data groups, and input the initial data groups into the trained teacher network so that the teacher network outputs several first joint driving commands. S25. The simulated poses in each initial data set are reduced, and the simulated terrain point cloud is compressed to construct a training data set; wherein, the data dimension of the reduced simulated poses is the same as the data dimension of the ontological pose data. Preferably, the compression of the simulated terrain point cloud includes: The simulated terrain point cloud is input into a preset spatial compression model so that the spatial compression model can identify the spatial context features between each point cloud in the simulated terrain point cloud, and output a terrain feature vector representing the terrain features of the simulated terrain point cloud based on the spatial context features.

[0051] In a preferred embodiment of the present invention, the spatial compression model is a variational autoencoder (VAE) that spatially encodes the point cloud. The encoder of the VAE maps the original data to a low-dimensional latent space and controls the KL divergence constraint of the latent vector to retain as much key terrain structure information as possible while filtering out invalid or interfering information.

[0052] The spatial compression model first sorts or groups the unordered point cloud data according to spatial relationships and inputs it into a Bidirectional Long Short-Term Memory (BiLSTM) network. Utilizing the temporal modeling capabilities of BiLSTM, it understands the spatial contextual relationships between points (e.g., whether the area around a point is flat or convex), thus better understanding the continuity and structure of the terrain (e.g., the edges of steps, the slope of a slope). Secondly, the contextual features extracted by BiLSTM are fed into a VAE encoder composed of a Multilayer Perceptron (MLP). The encoder no longer outputs a deterministic vector, but rather a probability distribution: a 32-dimensional mean vector (μ) and a 32-dimensional variance vector (σ²). During training, a 32-dimensional latent vector (z) is randomly sampled from this distribution. This process introduces uncertainty, forcing the model to learn not just the input, but the most robust and critical essential features of the data (e.g., terrain contours, slope), thus ignoring randomly occurring noise.

[0053] The overall goal of VAE is to minimize the evidence lower bound (ELBO) loss: ; Among them, the reconstruction items are: Based on encoder output latent distribution Sampling, through the decoder Reconstructing the noiseless privileged state The reconstruction error is calculated using the mean squared error (MSE), i.e.: ; Where d s For the privileged state dimension, Let i be the privileged state of the i-th dimension reconstructed by the decoder.

[0054] KL divergence regularization term: Constrained latent distribution Approximating the standard normal distribution To avoid overfitting, the formula is: ; in, For the latent variable dimension, denoted as the j-th dimension mean and standard deviation of the encoder output, respectively.

[0055] Annealing coefficient The formula is dynamically adjusted according to the number of training steps t, balancing reconstruction accuracy and noise filtering. ; Suggested value: , , (1 / 4 of the total training steps) to avoid premature compression of the latent space.

[0056] Because if the KL divergence weight is too large in the early stages of training, it will force the model to overemphasize the regularity of the distribution and ignore reconstruction, leading to feature learning failure. Understandably, β=0: ELBO loss only retains the reconstruction loss, and the encoder can freely output latent vectors (without distribution constraints), suitable for the early training stages (first 1000 rounds), prioritizing the decoder's learning of basic data features (such as the height distribution pattern of terrain point clouds); 0<β<1: gradually increasing the KL divergence loss weight, guiding the latent vector distribution to approach a standard normal distribution, suitable for the middle training stages (1001-5000 rounds), balancing reconstruction accuracy and latent distribution regularity; β=1: ELBO loss restores the standard form, the KL divergence loss weight is maximized, forcing the latent vector distribution to conform to the prior, suitable for the later training stages (after 5001 rounds), ensuring the continuity and interpretability of latent features (such as linear interpolation of latent features for terrains with different slopes). Therefore, this embodiment uses dynamic annealing, gradually increasing the weight coefficient (β) of the KL divergence.

[0057] Early stage (rounds 1-1000): β=0, focus on reconstruction and capturing basic terrain features. Mid-stage (rounds 1001-5000): β increases linearly from 0 to 1, gradually strengthening the regularization constraint on the spatial distribution of latent potential targets. Dynamic updates using linear functions are employed. : ; Where t is the current training round, t 0= 1000 (starting round of growth), t1=5000 (ending round of growth). .

[0058] Later stages (5000+ rounds): β=1, strictly constrained distribution to ensure continuity and interpretability of the latent space. Fixed. Simultaneously monitor the KL divergence loss value: if the KL divergence loss < 0.1 (the latent distribution has approximated the prior), then maintain... If the KL divergence loss is >0.5 (the latent distribution deviates too much), temporarily... Increased to 1.2 (reverted to 1.0 after 100 rounds), with forced constraint distribution.

[0059] After the above steps, the VAE encoder finally outputs a 32-dimensional latent vector. This vector is a "sterilized," low-dimensional, semantically rich representation of terrain features, containing key information such as "a 15° slope ahead" and "a 10cm high step to the left front," while suppressing sensor noise to the maximum extent.

[0060] S26. With the goal of minimizing the difference between the second joint drive command output by the student network and the first joint drive command, the student network is distilled using the training data set, and the trained student network is used as the motion control model.

[0061] Preferably, the step of using the training data set to perform distillation training on the student network with the objective of minimizing the difference between the second joint drive command output by the student network and the first joint drive command includes: A loss function is constructed to calculate the mean square error of the second joint drive command and the first joint drive command for each joint angle increment. The training data set is used to iteratively train the student network. In each round of iterative training, the loss function is used to calculate the loss function value based on the second joint drive command and the corresponding first joint drive command output by the student network for each training data set. If the loss function value does not converge, the network parameters of the student network are adjusted according to the loss function value. When the loss function value converges, the training of the student network is determined to be complete.

[0062] In a preferred embodiment of the present invention, the student network comprises three fully connected layers: Layer 1: Fully connected layer, 64-dimensional input → 128-dimensional output, ReLU activation function, no dropout (due to limited computing resources on embedded devices, prioritizing inference speed); Layer 2: Fully connected layer, 128-dimensional → 256-dimensional, with layer normalization (to adapt to fluctuations in input data in real-world environments); Layer 3: Fully connected layer, 256-dimensional → 128-dimensional, outputting a "reduced feature vector" (retaining only features relevant to real-time control). The output consists of 14 dimensions of movement commands, including: core control commands: 12 joint angle increments (consistent with the teacher network, ±5° range); simplified gait parameters: 1 gait cycle T (0.5-1.0s, no step length L required, step length is indirectly controlled through the cycle) + 1 "emergency correction signal" (1-dimensional, 0 = normal, 1 = posture adjustment required); The input to the student network is the same as the ontology pose data. To simplify the state information (32-dimensional), it includes joint data and core pose data. The joint data includes: 12 joint positions (12-dimensional) and velocities (12-dimensional), for a total of 24 dimensions. The core pose includes: Euler angles (3-dimensional), three-dimensional position (3-dimensional), 2 plantar contact forces (front / rear plantar, 2-dimensional), for a total of 8 dimensions, and lightweight terrain features compressed by VAE (32-dimensional).

[0063] The total number of input parameters required is only 8.2M (64.5M for teacher networks), with a video memory usage of ≤100MB, making it compatible with embedded devices; and "outlier detection" is added to the input layer (if the joint speed exceeds the threshold, the data from the previous frame is used to replace it) to compensate for the information loss caused by the simplified input.

[0064] Furthermore, during the student strategy training phase, a behavior cloning method is employed, using the actions output by the teacher's strategy as supervisory signals and constructing an imitation loss function. The training process combines sensor data obtained from the robot's actual operation with action records generated by the teacher's strategy. By minimizing the weighted sum of the imitation loss and the VAE reconstruction loss, the student network's action output is optimized, enabling it to approximate the teacher's strategy's performance under ideal conditions even when the input is only perceptual information.

[0065] Furthermore, regarding the loss function, during the training of the student policy, a joint optimization objective composed of multiple loss functions is constructed to simultaneously constrain the action output accuracy, dynamic behavior smoothness, and semantic representation ability of the student policy's perceptual features, thereby improving the quality and robustness of policy learning. This loss function system consists of the following main parts: First, a motion error loss term is defined to measure the difference between the student's strategy output motion and the reference motion provided by the teacher's strategy. This term uses the square of the Euclidean distance as the metric. Specifically, in each training sample, the element-wise difference between the joint control vector output by the student's strategy and the target vector generated by the corresponding teacher's strategy is calculated, the squares are summed, and then the average is taken to form a penalty for the overall motion difference. The purpose of this loss is to force the student's strategy to approximate the teacher's motion performance as closely as possible during training, ensuring effective imitation.

[0066] Secondly, an angular velocity smoothing loss term is set to constrain the continuity of the student policy's behavior in the dynamic dimension. This term uses an error metric called "SmoothL1," which provides stable gradient output when the angular velocity differences are small, and adopts linear processing when the differences are large, avoiding gradient explosion or jitter during training. During training, for each time step, the difference between the joint angular velocities generated by the student policy and the corresponding angular velocities of the teacher policy is calculated. The error is calculated based on this smoothing function and weighted averaged, serving as a constraint on the dynamic smoothness of the policy and effectively suppressing the oscillation problem of the control signal.

[0067] Specifically, the loss function formula is: ; in, (Weighting coefficients for balancing reconstruction and imitation). For the evidence lower bound (ELBO) loss of variational autoencoders (VAEs), Mean squared error (MSE) loss to mimic the teacher's strategy.

[0068] Specifically, the dataset aggregation (DAgger) method is used to minimize the action difference between student policies and teacher policies (OraclePolicy), as shown in the following formula: ; in, For aggregated datasets, containing Training trajectory of each iteration: .

[0069] For movement dimensions (joint angle increments, gait cycles, etc.). Teacher strategies in a noise-free privileged state The next Dimensional action output, Student strategies in latent variables The next Dimensional action output.

[0070] The time step length of a single trajectory. This represents the total number of trajectories in the dataset.

[0071] Combining the above components, the complete expansion of the student network's total loss function is as follows: .

[0072] Understandably, in this implementation, during the student's strategy perception input phase, the robot's onboard 3D sensors collect real-time 3D dense point cloud data within a 2-meter radius centered on the robot. This point cloud not only contains the 3D coordinates of each spatial point but also includes multiple semantically enhanced attributes, such as terrain height, reflectivity (used to distinguish surface materials like metal, rock, and soil), and the confidence level of the system estimate (reflecting the stability or reliability of the data acquisition). This information collectively constitutes a multi-dimensional feature vector representation of the point cloud, providing a foundation for subsequent terrain understanding.

[0073] To effectively encode this high-dimensional, unstructured data, a variational autoencoder (VAE) structure combining a bidirectional long short-term memory network (BiLSTM) and a multilayer perceptron (MLP) is employed. First, the point cloud is sorted or grouped according to its spatial distribution and then input into the BiLSTM module, utilizing its bidirectional temporal modeling capabilities to extract spatial contextual dependencies and terrain continuity features between points. Subsequently, the temporal features output by the BiLSTM are input into the MLP network for nonlinear transformation and abstract extraction, ultimately encoded as a 32-dimensional latent vector representing the terrain semantic information contained in the currently observed point cloud, such as slope, protrusions, obstacle edges, and material continuity.

[0074] The 32-dimensional latent space vector serves as the primary input to the student policy network, and is further combined with the action module to generate actions. Through this low-dimensional, semantically robust terrain encoding method, the policy model can effectively understand and adapt to ambiguity, local missing information, or noise interference in perceptual information, providing more stable and abstract representational support for subsequent action decisions.

[0075] This implementation achieves effective compression and semantic enhancement of high-dimensional point cloud data. By introducing additional attributes such as height, reflectivity, and confidence into each point, the ability to analyze complex terrain surface conditions is improved. The BiLSTM network is used to model the latent spatial structural continuity in the point cloud, enhancing the model's ability to perceive global terrain features. The 32-dimensional latent vector significantly reduces data dimensionality and computational burden compared to the original point cloud, while retaining crucial geometric and semantic information for policy decisions. This encoding method significantly improves the student policy's response speed, understanding depth, and robustness to terrain changes in complex environments, making it particularly suitable for noisy sensor outputs and unstructured scenarios.

[0076] Furthermore, this embodiment also employs an action segmentation mechanism by dividing the robot action sequence into segments of fixed length and weighting them, wherein the prediction length of each segment is 100 steps. A decreasing weighting function is used to assign exponentially decaying weights to historical actions in the time series, so as to achieve long-term dependency modeling and temporal noise reduction and smoothing of the action sequence.

[0077] This implementation addresses the short-term motion instability, oscillation, and response delay issues that are common in humanoid robot motion control. It proposes a motion sequence modeling method based on a block-based and weighted mechanism, integrated into the motion output stage of the student policy network. The core idea is to divide the continuous time-step motion signal sequence into multiple independent time blocks (action segments) of fixed length, and apply temporal weighting to the historical actions within each block to improve the stability and long-term dependency of motion generation.

[0078] In its implementation, the robot policy network generates a predicted sequence of actions for the next 100 steps in each control cycle, forming an action block. These 100 steps are numbered sequentially according to time, from the earliest step to the current step, corresponding to step lengths t1, t2, ..., t100. The system assigns a weight value to the historical actions at each time step, calculated using a decreasing function, such as an exponential decay function. ; Here, λ is the decay rate parameter (adjustable), with actions closer to the current time having a larger weight and earlier historical actions having a lower weight. The weighted action signals are then integrated through a weighted average to obtain a smoothed action output, avoiding the influence of single time step errors or abrupt changes. This mechanism essentially introduces a first-order time filter at the output of the policy network while retaining long-term action trend information, effectively enhancing the adaptability of the action generation model to state changes and its temporal consistency.

[0079] This implementation enhances the student policy's ability to model historical actions in a time series by introducing fixed-length blocks and an exponentially decreasing weighting strategy without introducing additional computational overhead. This method not only mitigates instantaneous control jitter caused by perceptual errors or sensor delays but also helps maintain the natural flow and rhythmic consistency of behavior. Compared to traditional single-step prediction, the action block mechanism provides stronger temporal modeling capabilities, effectively reducing the risk of discontinuity caused by action switching. It is particularly suitable for scenarios requiring coherent responses in complex terrain walking or disturbed movements, significantly improving the controllability, continuity, and stability of the policy's output actions.

[0080] See Figure 2 This invention provides a mobile control device for a humanoid robot, comprising: The data acquisition module is used to acquire the humanoid robot's body posture data and terrain point cloud data; The instruction generation module is used to compress the terrain point cloud data, generate a target terrain feature vector, and input the body posture data and the target terrain feature vector into a preset motion control model so that the motion control model outputs the joint drive instructions of the humanoid robot. The instruction drive module is used to control the movement of the humanoid robot according to the joint drive instructions; The construction of the motion control model includes: Construct a teacher network and a student network, wherein the student network is a lightweight network model; In a preset simulator, a simulation model of the humanoid robot, several simulation poses of the simulation model, and several simulation terrain point clouds are constructed. With the goal of maximizing the mobility stability and mobility efficiency of the humanoid robot, the teacher network is trained using the simulated posture and the simulated terrain point cloud. The simulated pose and the simulated terrain point cloud are randomly combined into several initial data sets, and the initial data sets are input into the trained teacher network so that the teacher network outputs several first joint drive commands. The simulated poses in each initial data set are reduced, and the simulated terrain point cloud is compressed to construct a training data set; wherein, the data dimension of the reduced simulated poses is the same as the data dimension of the ontological pose data. With the goal of minimizing the difference between the second joint drive command output by the student network and the first joint drive command, the student network is distilled and trained using the training data set, and the trained student network is used as the motion control model.

[0081] Furthermore, the step of maximizing the movement stability and efficiency of the humanoid robot, and training the teacher network using the simulated posture and the simulated terrain point cloud, includes: To maximize the mobility stability and efficiency of the humanoid robot, a reward function is constructed. Construct an intelligent agent and use the intelligent agent to repeatedly perform optimization operations on the teacher network until the teacher network training is completed; The optimization operation includes: Obtain the teacher network to be optimized; initially, the teacher network to be optimized is an untrained teacher network. Based on the simulated posture and the simulated terrain point cloud, several state sequence samples are constructed so that the agent can generate an action sequence consisting of several joint driving instructions based on each state sequence sample and the teacher network to be optimized, and update the simulation model based on the action sequence. Based on the reward function and the response state sequence generated after the simulation model is updated, a corresponding reward function value is generated, and several empirical sequences are constructed based on the state sequence samples, action sequences, reward function values, and response state sequences. Based on the aforementioned experience sequence, the teacher network to be optimized is optimized to generate a teacher network to be evaluated. When several reward function values ​​tend to converge, the teacher network training is considered complete; otherwise, the teacher network to be evaluated is used as the policy network to be optimized in the next round of optimization.

[0082] Furthermore, based on the simulated pose and the simulated terrain point cloud, several state sequence samples are constructed, including: Get the number of iterations for the optimization operation; When the number of iterations exceeds a first preset threshold, random perturbations are injected into several of the simulated terrain point clouds to generate a first simulated terrain point cloud, and several state sequence samples are constructed based on the simulated pose and the first simulated terrain point cloud. When the number of iterations exceeds the second preset threshold, random perturbation is injected into the first simulated terrain point cloud to generate a second simulated terrain point cloud, and several state sequence samples are constructed based on the simulated pose and the second simulated terrain point cloud; wherein, the first preset threshold is less than the second preset threshold.

[0083] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can implement the humanoid robot movement control method provided by any of the above-described method embodiments of the present invention.

[0084] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0085] See Figure 3 One embodiment of this application also provides a terminal device, including: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a humanoid robot movement control method as described above.

[0086] The processor controls the overall operation of the terminal device to complete all or part of the steps of the aforementioned humanoid robot mobility control method. The memory stores various types of data to support the operation of the terminal device. This data may include, for example, instructions for any application or method operating on the terminal device, as well as application-related data. The memory can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0087] In an exemplary embodiment, the terminal device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute a humanoid robot motion control method as described in any of the foregoing embodiments and achieve the same technical effects as the foregoing method.

[0088] In another exemplary embodiment, a computer-readable storage medium including a computer program is also provided. When executed by a processor, the computer program implements the steps of a humanoid robot mobility control method as described in any of the foregoing embodiments. For example, the computer-readable storage medium may be the aforementioned memory including the computer program, which may be executed by a processor of a terminal device to complete the humanoid robot mobility control method as described in any of the foregoing embodiments and achieve the same technical effects as the aforementioned method.

[0089] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A mobile control method for a humanoid robot, characterized by, The method comprises the following steps: acquiring body pose data and terrain point cloud data of a humanoid robot; compressing the terrain point cloud data to generate target terrain feature vectors, and inputting the body pose data and the target terrain feature vectors into a preset movement control model to make the movement control model output joint driving instructions of the humanoid robot; controlling movement of the humanoid robot according to the joint driving instructions; wherein the construction of the movement control model comprises: constructing a teacher network and a student network, wherein the student network is a lightweight network model; in a preset simulator, constructing a simulation model of the humanoid robot, a plurality of simulation poses of the simulation model and a plurality of simulation terrain point clouds; training the teacher network by taking maximization of movement stability and movement efficiency of the humanoid robot as a target and by using the simulation poses and the simulation terrain point clouds; randomly combining the simulation poses and the simulation terrain point clouds into a plurality of initial data groups, and inputting the initial data groups into the trained teacher network to make the teacher network output a plurality of first joint driving instructions; constructing training data groups by reducing the simulation poses in each initial data group and compressing the simulation terrain point clouds, wherein the data dimension of the reduced simulation poses is the same as that of the body pose data; adopting the training data groups to perform distillation training on the student network by taking minimization of the difference between second joint driving instructions output by the student network and the first joint driving instructions as a target, and taking the trained student network as the movement control model.

2. The mobile control method of a humanoid robot according to claim 1, wherein The training of the teacher network by taking maximization of movement stability and movement efficiency of the humanoid robot as a target and by using the simulation poses and the simulation terrain point clouds comprises: constructing a reward function by taking maximization of movement stability and movement efficiency of the humanoid robot as a target; constructing an agent, and repeatedly performing optimization operations on the teacher network by using the agent until the training of the teacher network is completed; wherein the optimization operation comprises: acquiring a teacher network to be optimized; wherein initially, the teacher network to be optimized is an untrained teacher network; constructing a plurality of state sequence samples according to the simulation poses and the simulation terrain point clouds, so that the agent generates an action sequence composed of a plurality of joint driving instructions according to each state sequence sample and the teacher network to be optimized, and updates the simulation model according to the action sequence; generating corresponding reward function values according to the reward function and a response state sequence generated after the simulation model is updated, and constructing a plurality of experience sequences according to the state sequence samples, the action sequence, the reward function values and the response state sequence; optimizing the teacher network to be optimized according to the experience sequences to generate a teacher network to be evaluated; when a plurality of reward function values tend to converge, it is determined that the training of the teacher network is completed; otherwise, the teacher network to be evaluated is taken as the teacher network to be optimized in the next round of optimization operation.

3. The mobile control method of a humanoid robot according to claim 2, wherein constructing a plurality of state sequence samples according to the simulation posture and the simulation terrain point cloud, comprising: acquiring an iteration number of an optimization operation; when the iteration number is greater than a first preset threshold, injecting a random disturbance into a plurality of the simulation terrain point cloud to generate a first simulation terrain point cloud, and constructing a plurality of state sequence samples according to the simulation posture and the first simulation terrain point cloud; when the iteration number is greater than a second preset threshold, injecting a random disturbance into the first simulation terrain point cloud to generate a second simulation terrain point cloud, and constructing a plurality of state sequence samples according to the simulation posture and the second simulation terrain point cloud; wherein the first preset threshold is less than the second preset threshold.

4. The mobile control method of a humanoid robot according to claim 3, wherein the compression of the simulation terrain point cloud comprises: inputting the simulation terrain point cloud into a preset spatial compression model, so that the spatial compression model identifies the spatial context features between each point cloud in the simulation terrain point cloud, and outputs a terrain feature vector representing the terrain features of the simulation terrain point cloud according to the spatial context features.

5. The mobile control method of a humanoid robot according to claim 4, wherein The student network is distilled and trained using the training data set, with the difference between the second joint driving instruction output by the student network and the first joint driving instruction being minimized, comprising: constructing a loss function for calculating the mean square error of each joint angle increment of the second joint driving instruction and the first joint driving instruction iteratively training the student network using the training data set, and in each round of iterative training, using the loss function to calculate the loss function value according to the second joint driving instruction output by the student network for each training data set and the corresponding first joint driving instruction, adjusting the network parameters of the student network according to the loss function value when the loss function value does not converge, and determining that the student network training is complete when the loss function value converges.

6. A mobile control device for a humanoid robot, characterized by comprising: a data acquisition module for acquiring body posture data and terrain point cloud data of a humanoid robot; an instruction generation module for compressing the terrain point cloud data to generate a target terrain feature vector, and inputting the body posture data and the target terrain feature vector into a preset movement control model to make the movement control model output joint driving instructions of the humanoid robot; an instruction driving module for controlling the movement of the humanoid robot according to the joint driving instructions; wherein the construction of the movement control model comprises: constructing a teacher network and a student network, wherein the student network is a lightweight network model; constructing a simulation model of the humanoid robot, a plurality of simulation postures of the simulation model, and a plurality of simulation terrain point clouds in a preset simulator; maximizing the movement stability and movement efficiency of the humanoid robot, and training the teacher network through the simulation postures and the simulation terrain point clouds; randomly combining the simulation postures and the simulation terrain point clouds into a plurality of initial data sets, and inputting the initial data sets into the trained teacher network to make the teacher network output a plurality of first joint driving instructions; pruning the simulation poses in each initial data set, and compressing the simulation terrain point clouds, to construct a training data set; wherein a data dimension of the pruned simulation poses is same as a data dimension of the body pose data; minimizing a difference between a second joint driving instruction output by the student network and the first joint driving instruction as a target, and performing distillation training on the student network by using the training data set, and taking the student network trained as the mobile control model.

7. The mobile control device of a humanoid robot according to claim 6, wherein training the teacher network by using the simulation poses and the simulation terrain point clouds as a target to maximize a moving stability and a moving efficiency of the humanoid robot, including: constructing a reward function as a target to maximize the moving stability and the moving efficiency of the humanoid robot; constructing an agent, and repeatedly performing an optimization operation on the teacher network by using the agent until the teacher network is trained; wherein the optimization operation includes: obtaining a to-be-optimized teacher network; wherein initially, the to-be-optimized teacher network is an untrained teacher network; constructing a plurality of state sequence samples according to the simulation poses and the simulation terrain point clouds, so that the agent generates a motion sequence composed of a plurality of joint driving instructions according to each of the state sequence samples and the to-be-optimized teacher network, and updates the simulation model according to the motion sequence; generating a corresponding reward function value according to the reward function and a response state sequence generated after the simulation model is updated, and constructing a plurality of experience sequences according to the state sequence samples, the motion sequence, the reward function value, and the response state sequence; optimizing the to-be-optimized teacher network according to the experience sequences to generate a to-be-evaluated teacher network; when a plurality of reward function values tend to converge, determining that the teacher network is trained; otherwise, taking the to-be-evaluated teacher network as a to-be-optimized strategy network of a next round of optimization operation.

8. The mobile control device of a humanoid robot according to claim 7, wherein constructing a plurality of state sequence samples according to the simulation poses and the simulation terrain point clouds, including: obtaining an iteration number of the optimization operation; when the iteration number is greater than a first preset threshold, injecting a random disturbance into a plurality of the simulation terrain point clouds to generate a first simulation terrain point cloud, and constructing a plurality of state sequence samples according to the simulation poses and the first simulation terrain point cloud; when the iteration number is greater than a second preset threshold, injecting a random disturbance into the first simulation terrain point cloud to generate a second simulation terrain point cloud, and constructing a plurality of state sequence samples according to the simulation poses and the second simulation terrain point cloud; wherein the first preset threshold is less than the second preset threshold.

9. A terminal device, comprising: including: one or more processors; a memory coupled to the processors and storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors realize a mobile control method of a humanoid robot as claimed in any one of claims 1-5.

10. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to realize a mobile control method of a humanoid robot as claimed in any one of claims 1-5.

Images