Motion control model training method, robot motion control method, and electronic device

Abstract

The application provides a motion control model training method, a robot motion control method and an electronic device. The model training method comprises: using a pre-trained terrain reconstruction model to reconstruct terrain according to historical proprioceptive data and a plurality of consecutive local depth images to obtain a current local terrain geometric feature map; obtaining current observation features according to current proprioceptive data, the current local terrain geometric feature map and a preset control instruction of a driving joint in a simulation robot; using a teacher model and a student model respectively to generate a first action instruction and a second action instruction according to the current observation features; training the student model according to the first action instruction and the second action instruction to obtain a target action strategy model for motion control of a humanoid robot. The application realizes stable adaptive motion of the humanoid robot on complex terrain, reduces terrain reconstruction error and perception delay, reduces hardware dependence, and improves training efficiency and environmental generalization ability.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and more specifically, to a motion control model training method, a robot motion control method, and an electronic device. Background Technology

[0002] Humanoid robots possess enormous potential for autonomous mobility applications in human-centered, unstructured environments. Humanoid robots, in particular, are robots designed to mimic human appearance and behavior, especially those with physiques similar to humans.

[0003] In related technologies, motion control of humanoid robots is still mainly limited to two paradigms: end-to-end learning methods based on depth images and methods based on elevation maps. The first paradigm directly maps depth images to control actions through end-to-end learning, while the second paradigm creates elevation maps and completes motion planning and control in a structured geometric space.

[0004] However, the first paradigm is limited by the gap between simulation and real environment, resulting in low training efficiency and poor generalization ability. It performs particularly poorly in sensor noise and occlusion scenarios. The second paradigm relies on multiple sensors and positioning systems, which are prone to latency and drift. Summary of the Invention

[0005] In view of this, embodiments of this application provide a motion control model training method, a robot motion control method, and an electronic device to solve the problems of the first paradigm being limited by the migration gap from simulation to the real environment, having poor generalization ability, and performing particularly poorly in sensor noise and occlusion scenarios, and the second paradigm relying on multiple sensors and positioning systems, which are prone to delay and drift problems.

[0006] In a first aspect, embodiments of this application provide a method for training a motion control model, including: Acquire current proprioceptive data, historical proprioceptive data, and multiple consecutive frames of local depth images of the simulated robot in a preset terrain environment; A pre-trained terrain reconstruction model is used to reconstruct the terrain based on the historical ontological perception data and the continuous multi-frame local depth images, thereby obtaining the current local terrain geometric feature map of the preset terrain environment. Based on the current proprioceptive data, the current local terrain geometric feature map, and the preset control commands of the drive joints in the simulation robot, the current observed features of the drive joints are obtained; Based on the current observation characteristics, the teacher model and the student model are used respectively to generate the first action command and the second action command of the driving joint; The student model is trained according to the first action instruction and the second action instruction to obtain the target action strategy model; The output includes a robot motion control model comprising the pre-trained terrain reconstruction model and the target action strategy model, for motion control of the humanoid robot.

[0007] In an optional implementation, the pre-trained terrain reconstruction model includes: a feature fusion module, a recurrent memory unit, and a terrain reconstruction module; The pre-trained terrain reconstruction model is used to reconstruct the terrain based on the historical ontological perception data and the continuous multi-frame local depth images, obtaining the current local terrain geometric feature map of the preset terrain environment, including: Based on the historical ontological perception data and multiple consecutive frames of local depth images, the feature fusion module is used to perform feature fusion to obtain fused terrain features. The fused terrain features are accumulated over time using the circular memory unit to obtain accumulated terrain features; The terrain reconstruction module generates the current local terrain geometric feature map based on the accumulated terrain features.

[0008] In an optional implementation, the feature fusion module includes: a visual encoder, a state encoder, and a cross-attention network; The step of fusing features based on the historical ontological perception data and multiple consecutive frames of local depth images using the feature fusion module to obtain fused terrain features includes: The visual encoder and the state encoder are used to encode the continuous multi-frame local depth images and the historical ontology perception data, respectively, to obtain the depth embedding vector and the ontology perception embedding vector. The cross-attention network is used to perform cross-attention feature fusion based on the deep embedding vector and the ontology-aware embedding vector to obtain the fused terrain features.

[0009] In an optional implementation, the terrain reconstruction module includes: a decoder and a conditional network; the step of generating the current local terrain geometric feature map using the terrain reconstruction module based on the accumulated terrain features includes: The accumulated terrain features are decoded using the decoder to obtain the current coarse height map; The conditional network is used to refine the current coarse height map using the deep embedding vector as a condition, thereby obtaining the current local terrain geometric feature map.

[0010] In an optional implementation, the method further includes: A preset attitude estimator is used to estimate the current proprioceptive data to obtain the current estimated features; The step of obtaining the current observed features of the drive joint based on the current proprioceptive data, the current local terrain geometric feature map, and the preset control commands of the drive joint in the simulated robot includes: The current observed features are obtained based on the current ontological perception data, the current estimated features, the current local terrain geometric feature map, and the preset control commands.

[0011] In an optional implementation, the step of generating the first and second motion commands for the drive joint using a teacher model and a student model respectively, based on the current observation features, includes: The current proprioceptive data, the current estimated features, and the preset control commands in the current observed features are determined as blind control input features; Using the first blind control backbone network in the teacher model, the first blind control action of the driving joint is generated based on the blind control input features; Using the first perception network in the teacher model, the first modulation action of the drive joint is generated based on the current local terrain geometric feature map, the first blind control action, and the blind control input feature in the current observation features, and the first action command is generated based on the first blind control action and the first modulation action. Using the second blind control backbone network in the student model, the second blind control action of the driving joint is generated based on the blind control input features; Using the second perception network in the student model, the second modulation action of the drive joint is generated based on the current local terrain geometric feature map in the current observation features, the second blind control action, and the blind control input features. The second action command is generated based on the second blind control action and the second modulation action.

[0012] In an optional implementation, the method further includes: Obtain the robot state information after the simulated robot executes the second action command; The robot state information corresponding to the preset human action state data is obtained from the preset demonstration database as reference demonstration data. The preset demonstration database includes: state data of multiple preset human actions. A preset discriminator is used to generate action reward parameters based on the robot's state information and the reference demonstration data; Obtain the task reward parameters when the simulated robot executes the second action instruction; The student model is tuned based on the action reward parameters and the task reward parameters.

[0013] Secondly, embodiments of this application also provide a robot motion control method, including: Acquire target proprioceptive data of a humanoid robot in a target terrain environment, historical proprioceptive data of the target proprioceptive data, and continuous multi-frame target local depth images; Based on the historical prophetic perception data of the target prophetic perception data and the continuous multi-frame target local depth images, terrain reconstruction is performed using a pre-trained terrain reconstruction model in the robot motion control model to obtain the target local terrain geometric feature map of the target terrain environment; wherein, the robot motion control model is a model trained using the method described in any of the first aspects. Based on the target body perception data, the target local terrain geometric feature map, and the target control commands for the drive joints in the humanoid robot, the target observation features are obtained; Based on the target observation characteristics, the target action strategy model in the robot motion control model is used to generate the target action commands that drive the joints of the humanoid robot. The humanoid robot is controlled to perform actions according to the target action command.

[0014] In an optional implementation, the step of generating target motion commands for driving joints in the humanoid robot based on the target observation features and using the target motion strategy model in the robot motion control model includes: A preset attitude estimator is used to estimate the target body perception data to obtain target estimation features, and the target observation features further include the target estimation features; The target ontological sensing data, the target estimation features, and the target control commands in the target observation features are determined as target blind control input features; Using the blind control backbone network in the target action strategy model, the target blind control action of the driving joint is generated based on the target blind control input features; Using the perception network in the target action strategy model, the target local terrain geometric feature map, the target blind control action, and the target blind control input feature in the target observation features are used to generate the target modulation action of the drive joint, and the target action command is generated based on the target blind control action and the target modulation action.

[0015] Thirdly, embodiments of this application also provide an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to perform the method described in any of the first aspects.

[0016] This application provides a motion control model training method, a robot motion control method, and an electronic device. The model training method includes: using a pre-trained terrain reconstruction model, reconstructing the terrain based on historical proprioceptive data and continuous multi-frame local depth images to obtain a current local terrain geometric feature map; obtaining the current observed features of the drive joints based on the current proprioceptive data, the current local terrain geometric feature map, and preset control commands for the drive joints in the simulated robot; generating a first action command and a second action command for the drive joints using both a teacher model and a student model based on the current observed features; training the student model based on the first and second action commands to obtain a target action strategy model; and outputting a robot motion control model including the pre-trained terrain reconstruction model and the target action strategy model for motion control of a humanoid robot. This enables stable adaptive motion of the humanoid robot in complex terrain, reduces terrain reconstruction errors and perception latency, reduces hardware dependence, and improves training efficiency and environmental generalization ability. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 1 ; Figure 2 A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 2 ; Figure 3 This is a schematic diagram of the structure of the initial terrain reconstruction model provided in the embodiments of this application; Figure 4 A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 3 ; Figure 5 A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 4 ; Figure 6A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 5 ; Figure 7 A flowchart illustrating the robot motion control method provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of the motion control model training device provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the robot motion control device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] To address the shortcomings of current humanoid robot motion control, this application trains a robot motion control model, including a pre-trained terrain reconstruction model and a target action strategy model, to control the motion of the humanoid robot. This enables the embodied intelligent humanoid robot to achieve stable and adaptive motion in complex terrains (slopes, stairs, gaps, movable platforms, etc.), reducing terrain reconstruction errors and perception delays, reducing hardware dependence, and improving training efficiency and environmental generalization ability.

[0021] Figure 1 A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 1 In this embodiment, the execution entity can be a model training device, such as a computer device.

[0022] like Figure 1 As shown, the method may include: S101. Acquire the current proprioceptive data, historical proprioceptive data, and multiple consecutive frames of local depth images of the simulated robot in the preset terrain environment.

[0023] The simulated robot is a physically simulated humanoid robot. The preset terrain environment is an unstructured environment, which may include slopes, stairs, gaps, movable platforms, etc.

[0024] Current proprioceptive data refers to the proprioceptive data of the simulated robot at the current moment in the preset terrain environment, while historical proprioceptive data refers to the proprioceptive data of the simulated robot at historical moments in the preset terrain environment. Proprioceptive data refers to the joint data of the driven joints collected by the simulated robot through internal sensors, which may include joint position, joint velocity, joint angular velocity, etc. For example, if the current moment is time t, then the historical moment may include the moments between time t-50 and time t.

[0025] A series of consecutive local depth images are depth images of a local area of ​​the simulated robot observed at the current moment under a preset terrain environment. The local area can be, for example, a depth image of a preset range in front of the simulated robot (e.g., a range of 1.0m × 1.0m in front of the simulated robot, where "in front" refers to the direction of movement of the simulated robot as a reference). There are multiple frames of local depth images, such as 5 frames, which is called temporal depth observation. .

[0026] Among them, the local depth image is used to indicate the depth information of a preset range in front of the simulated robot in a preset terrain environment.

[0027] S102. Using a pre-trained terrain reconstruction model, terrain reconstruction is performed based on historical ontological perception data and multiple consecutive frames of local depth images to obtain the current local terrain geometric feature map of the preset terrain environment.

[0028] Historical proprioceptive data and multiple consecutive frames of local depth images are used as input. A pre-trained terrain reconstruction model is used for processing to generate a current local terrain geometric feature map of the preset terrain environment. The current local terrain geometric feature map is a height map of a preset range in front of the simulated robot at the current moment. The height map is a two-dimensional discrete grid, and each grid cell stores an elevation value. The elevation value refers to the vertical distance of the grid cell relative to the robot's floating base (such as the center reference point of the robot body).

[0029] The current local terrain geometry feature map is used to indicate the height features of a preset range in front of the simulated robot in the preset terrain environment.

[0030] S103. Based on the current proprioceptive data, the current local terrain geometric feature map, and the preset control commands of the drive joints in the simulation robot, obtain the current observed features of the drive joints.

[0031] Among them, the drive joint is the joint that drives the motion of the simulated robot. The preset control command of the drive joint can carry the target linear velocity (planned linear velocity) and target yaw rate (planned yaw rate) of the drive joint. The target linear velocity and target yaw rate are determined from the preset control command. The target linear velocity is used to indicate how fast the robot's center of mass or reference point moves in space along the direction of motion, and the target yaw rate is used to indicate how fast the robot rotates about an axis perpendicular to the plane of motion (xy plane). The preset control command can be expressed as: , These are the target linear velocities in the x and y directions, respectively, within the local coordinate system where the simulated robot is located (with the robot's motion direction as the x-direction and the direction perpendicular to the x-direction as the y-direction) at the current time t. The target yaw rate.

[0032] During the movement of the simulated robot in a preset terrain environment, it can obtain the preset control commands of the driven joints in the simulated robot at the current moment, and obtain the current observation features of the driven joints based on the current proprioceptive data, the current local terrain geometric feature map, and the preset control commands of the driven joints in the simulated robot. The current observation features include: the current proprioceptive data, the current local terrain geometric feature map, and the preset control commands of the driven joints in the simulated robot.

[0033] S104. Based on the current observation characteristics, the teacher model and student model are used respectively to generate the first and second motion commands to drive the joints.

[0034] Based on the current observation characteristics, a teacher model is used to generate the first action command for driving the joint. The first action command is the action command for driving the joint output by the teacher model at the current time. Based on the current observation characteristics, a student model is used to generate the second action command for driving the joint. The second action command is the action command for driving the joint output by the student-teacher model at the current time.

[0035] S105. Train the student model according to the first action instruction and the second action instruction to obtain the target action strategy model.

[0036] S106. Output a robot motion control model including a pre-trained terrain reconstruction model and a target action strategy model, to perform motion control on the humanoid robot.

[0037] Based on the first action instruction and the second action instruction, the action loss is calculated, and the student model is trained based on the action loss. The student model obtained when the preset iteration stopping condition is reached is used as the target action strategy model.

[0038] The robot motion control model includes a pre-trained terrain reconstruction model and a target motion strategy model. This model outputs motion commands to the humanoid robot to control its motion.

[0039] In this embodiment, a robot motion control model, including a pre-trained terrain reconstruction model and a target action strategy model, is trained to control the motion of the humanoid robot. This enables the embodied intelligent humanoid robot to achieve stable adaptive motion in complex terrains (slopes, stairs, gaps, movable platforms, etc.), reducing terrain reconstruction errors and perception delays, reducing hardware dependence, and improving training efficiency and environmental generalization ability.

[0040] Figure 2 A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 2 ,like Figure 2 As shown, in an optional implementation, the pre-trained terrain reconstruction model includes: a feature fusion module, a recurrent memory unit, and a terrain reconstruction module.

[0041] Step S102 above uses a pre-trained terrain reconstruction model to reconstruct the terrain based on historical ontological perception data and multiple consecutive frames of local depth images, obtaining the current local terrain geometric feature map of the preset terrain environment, which may include: S201. Based on historical ontological perception data and multiple consecutive frames of local depth images, feature fusion is performed using a feature fusion module to obtain fused terrain features.

[0042] In an optional implementation, the feature fusion module includes: a visual encoder, a state encoder, and a cross-attention network. Based on historical ontological perception data and multiple consecutive frames of local depth images, the feature fusion module performs feature fusion to obtain fused terrain features, including: A visual encoder and a state encoder are used to encode multiple consecutive frames of local depth images and historical ontology perception data, respectively, to obtain depth embedding vectors and ontology perception embedding vectors. A cross-attention network is used to perform cross-attention feature fusion based on the depth embedding vectors and ontology perception embedding vectors to obtain fused terrain features.

[0043] The visual encoder and state encoder can be Convolutional Neural Network (CNN) encoders. The cross-attention network can be a Transformer network based on the cross-attention mechanism.

[0044] A visual encoder is used to encode multiple consecutive frames of local depth images to obtain a depth embedding vector, which is a compact spatial depth feature. A state encoder is used to encode historical proprioceptive data to obtain a proprioceptive embedding vector, which is used to capture the kinematic and dynamic states of the simulated robot.

[0045] For both deep embedding vectors and ontology-aware embedding vectors, a cross-attention network is used. The ontology-aware embedding vectors are used as the query (Q), and the deep embedding vectors are used as the key (K) and value (V) for feature fusion, resulting in fused terrain features. These fused terrain features are represented as follows:

[0046] in, In order to integrate terrain features, For cross-attention networks, For ontology-aware embedding vectors, This is a deep embedding vector.

[0047] It's important to note that the query is an ontology-aware embedding vector, meaning the fusion starts from the robot's own state. The key and value are deep embedding vectors, representing all the contextual information provided by the environment. The key is used to calculate the relevance to the query, while the value is the specific feature content to be extracted and aggregated. In other words, a cross-attention network is used to calculate the similarity between the query and the key, obtaining a set of attention weights. These weights are then used to weight and sum the values ​​to obtain the fused feature vector. This fused feature vector is the result of filtering and modulating the deep features of the environment through the robot's own state.

[0048] In some embodiments, virtual rays are projected onto a 3D mesh consisting of static terrain and the robot's articulated geometry within a local area of ​​the robot, based on Graphics Processing Unit (GPU) ray casting technology. This generates a 600×480 resolution local depth image. The local depth image undergoes domain randomization and noise reduction to simulate the axial noise, lateral disturbances, and depth holes of a real depth sensor. Further processing techniques, such as cropping and resampling, and out-of-range cropping, are then employed to further refine the local depth image, reducing the gap between the simulation and real-world data. Finally, based on historical ontological perception data and multiple consecutive frames of processed depth images, a feature fusion module is used to perform feature fusion to obtain terrain fusion features.

[0049] It should be noted that combining self-occlusion perception of light projection and noise perception modeling to synthesize realistic depth images can reduce terrain reconstruction errors.

[0050] S202. Use a circular memory unit to perform temporal accumulation of the fused terrain features to obtain the accumulated terrain features.

[0051] The recurrent memory unit is a gated recurrent unit (GRU) that acquires the historical fused terrain features from the previous moment. The recurrent memory unit uses the historical fused terrain features and the current fused terrain features to comprehensively determine the cumulative terrain features. In other words, based on the historical fused terrain features, the fused terrain features are accumulated over time to obtain the cumulative terrain features. Thus, the terrain features are understood based on continuously accumulated experience, so that the robot's terrain cognition is no longer an instantaneous snapshot, but becomes an experience base that evolves and enriches over time.

[0052] S203. The terrain reconstruction module generates a local terrain geometric feature map based on the accumulated terrain features.

[0053] Using accumulated terrain features as input, a terrain reconstruction module is employed to generate the current local terrain geometric features.

[0054] In an optional implementation, the terrain reconstruction module includes: a decoder and a conditional network; step S203 above, which uses the terrain reconstruction module to generate a current local terrain geometric feature map based on accumulated terrain features, may include: A decoder is used to decode the accumulated terrain features to obtain the current coarse height map; a conditional network is used with the deep embedding vector as a condition to refine the current coarse height map to obtain the current local terrain geometric feature map.

[0055] The conditional network can be a U-Net network, which uses a decoder to decode the accumulated terrain features to obtain the current coarse height map. However, the current coarse height map suffers from blurred edges and uneven surfaces, limiting its effectiveness as model input. Therefore, a conditional network is used with deep embedding vectors as conditions to refine the current coarse height map, resulting in the current local terrain geometric feature map, or refined height map. Compared to the current coarse height map, the edges are clearer and the surface is smoother, as shown below:

[0056] in, This is the current local terrain geometry. This is the current coarse height map. This is a deep embedding vector.

[0057] In this embodiment, by aligning the proprioceptive query with the deep contextual features, the model can selectively highlight the key features required for reconstruction in the visual feature stream according to the robot's state, without the need for an external positioning system, and also reduces terrain reconstruction errors.

[0058] Figure 3 This is a schematic diagram of the structure of the initial terrain reconstruction model provided in the embodiments of this application, as shown below. Figure 3 As shown, the initial terrain reconstruction model includes: a visual encoder (CNN), a state encoder (CNN), a cross-attention network (Transformer), a recurrent memory unit (GRU), a decoder, and a conditional network (U-Net).

[0059] Visual encoders and state encoders are used to process the historical ontological perception data of the samples, respectively. and local depth images of consecutive multi-frame samples Encoding is performed to obtain sample depth embedding vectors and sample ontology-aware embedding vectors. A cross-attention network is used to perform cross-attention feature fusion with the sample ontology-aware embedding vector as Q and the sample depth embedding vector as K and V to obtain sample fused terrain features. Then, a recurrent memory unit is used to accumulate the sample fused terrain features in time to obtain sample accumulated terrain features. A decoder is used to decode the sample accumulated terrain features to obtain sample coarse height maps. A conditional network is used with the sample depth embedding vector as a condition to refine the sample coarse height maps to obtain sample fine height maps.

[0060] in, This is a roughness height map of the sample. For a detailed height map of the sample, To create a height map, based on the sample coarse height map. and height map Calculate loss Based on the sample detailed height map and height map Calculate loss The total loss Based on the total loss, the visual encoder, state encoder, cross-attention network, recurrent memory unit, decoder, and conditional network in the initial terrain reconstruction model are tuned to obtain the pre-trained terrain reconstruction model.

[0061] Figure 4 A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 3 ,like Figure 4 As shown, in an optional implementation, the method may further include: S301. Using a preset attitude estimator, the current proprioceptive data is estimated to obtain the current estimated features.

[0062] The preset attitude estimator is used to estimate the attitude of the simulated robot based on the current proprioceptive data to obtain the current estimated features. The current estimated features may include, for example, the estimated projected gravity and estimated linear velocity of the simulated robot. The estimated projected gravity is used to indicate the degree of tilt of the simulated robot's attitude relative to the direction of gravity. The estimated linear velocity is used to indicate the speed and direction of the robot's base (center of mass) translational motion in space. It is usually a 3-dimensional vector, corresponding to the linear velocities in the three coordinate directions.

[0063] It should be noted that the preset attitude estimator is distributed on the simulated robot and is used to collect and estimate the projected gravity and the linear velocity.

[0064] Step S103 above, which obtains the current observed features of the drive joints based on the current proprioceptive data, the current local terrain geometric feature map, and the preset control commands of the drive joints in the simulated robot, may include: S302. Based on the current proprioceptive data, current estimated features, current local terrain geometric feature map, and preset control commands, obtain the current observed features.

[0065] The current observation features include current ontological perception data, current estimated features, current local terrain geometric feature map, and preset control commands.

[0066] Figure 5 A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 4 ,like Figure 5 As shown, in an optional implementation, step S104 above, which generates a first motion command and a second motion command for driving the joint using the teacher model and the student model respectively based on the current observation features, may include: S401. Determine the current proprioceptive data, current estimated features, and preset control commands in the current observation features as blind control input features.

[0067] The blind control input features include: current proprioceptive data, current estimated features, and preset control commands.

[0068] In some embodiments, the blind control input feature may further include: a phase variable, which indicates the periodic phase offset of the two legs of the simulated robot.

[0069] Represented as:

[0070] in, For blind control input features, These represent joint position, joint velocity, joint angular velocity, estimated projected gravity, preset control command, phase variable, and estimated linear velocity, respectively.

[0071] S402. Using the first blind control backbone network in the teacher model, the first blind control action to drive the joint is generated based on the blind control input characteristics.

[0072] The teacher model includes a first blind control backbone network. Using this network, the first blind control action that drives the joint is generated based on the blind control input features, as shown below:

[0073] in, This is the first blind control action. It forms the first blind control backbone network.

[0074] S403. Using the first perception network in the teacher model, based on the current local terrain geometric feature map, the first blind control action, and the blind control input features in the current observation features, a first modulation action to drive the joint is generated, and a first action command is generated based on the first blind control action and the first modulation action.

[0075] The teacher model also includes: a first perceptual network, which generates a first modulated action to drive the joint based on the current local terrain geometry feature map, the first blind control action, and the blind control input features, represented as:

[0076] in, The first sensing network receives the sensing input features, and the first sensing network outputs the first modulation action. , This is a map showing the current local terrain geometry.

[0077] The teacher model may also include a first fusion network. Using this first fusion network, a first action instruction is generated based on the first blind control action and the first modulation action. The action corresponding to the first action instruction can be a weighted sum of the first blind control action and the first modulation action, represented as follows:

[0078] in, The action corresponding to the first action instruction. For weight values, .

[0079] S404. Using the second blind control backbone network in the student model, a second blind control action to drive the joint is generated based on the blind control input features.

[0080] S405. Using the second perception network in the student model, based on the current local terrain geometric feature map, the second blind control action, and the blind control input features in the current observation features, a second modulation action to drive the joint is generated, and a second action command is generated based on the second blind control action and the second modulation action.

[0081] The student model includes a second blind control backbone network and a second perception network. The second blind control backbone network is used to generate a second blind control action to drive the joint based on the blind control input features. The second perception network is used to generate a second modulation action to drive the joint based on the current local terrain geometry feature map, the second blind control action, and the blind control input features.

[0082] For the specific expressions of the second blind control action and the second modulation action, please refer to the expressions of the first blind control action and the first modulation action mentioned above.

[0083] The student model may also include a second fusion network. The second fusion network generates a second action instruction based on the second blind control action and the second modulation action. The action corresponding to the second action instruction can be a weighted sum of the second blind control action and the second modulation action. For a specific expression, please refer to the expression of the action corresponding to the first action instruction mentioned above.

[0084] It should be noted that training the student model includes: tuning the parameters of the second blind control backbone network, the second perception network, and the second fusion network in the student model to obtain the target action policy model, wherein the target action policy model includes: the trained blind control backbone network, perception network, and fusion network.

[0085] In this embodiment, a terrain-aware motion control strategy with a blind backbone network is adopted. A pre-trained terrain reconstruction model is used to guide reinforcement learning training with minimal visual input. Furthermore, a multimodal cross-attention network is used to reconstruct structured terrain representations from noisy depth images.

[0086] Figure 6 A flowchart illustrating the motion control model training method provided in this application embodiment. Figure 5 ,like Figure 6 As shown, in an optional implementation, the method may further include: S501. Obtain the robot status information after the simulated robot executes the second action command.

[0087] The simulation robot is controlled to perform the corresponding action according to the second action command, and the robot state information after the robot executes the second action command is obtained. The robot state information refers to the robot pose information, which is used to describe the robot's position and posture in space.

[0088] S502. Obtain the state data of the preset human actions corresponding to the robot state information from the preset demonstration database as reference demonstration data.

[0089] The preset demonstration database includes: state data of multiple preset human actions. Preset human actions refer to actions performed by humans. The state data of preset human actions includes: human state information after performing preset human actions. Human state information refers to human pose information, which is used to describe the human's position and posture in space.

[0090] According to the second action instruction, a preset human action with a similarity to the action corresponding to the second action instruction exceeding a preset threshold is determined from the preset demonstration database. This preset human action is used as the preset human action corresponding to the robot's state information, and the state information of this preset human action is used as reference demonstration data.

[0091] S503: A preset discriminator is used to generate action reward parameters based on robot state information and reference demonstration data.

[0092] To achieve more natural and human-like interaction between robots and their environment, the Adversarial Motion Priors (AMP) framework is introduced. AMP introduces a pre-defined discriminator and, through adversarial learning, enables the robot to learn and imitate the natural movement styles contained in human state information while completing specific tasks. The goal is to generate movements that the discriminator cannot distinguish, that is, to be as close as possible to the reference demonstration data.

[0093] Based on the robot's state information and reference demonstration data, a preset discriminator is trained, and based on the trained discriminator and the robot's state information, action reward parameters are generated.

[0094] In some embodiments, a discriminant loss function is calculated based on robot state information and reference demonstration data, and a preset discriminator is trained based on the discriminant loss function to obtain a trained discriminator, wherein the discriminant loss function is expressed as:

[0095] in, To determine the loss function, This indicates the robot's state information before the simulated robot executes the second action command. And the robot state information after the simulated robot executes the second action command. The corresponding state transition process, This represents the human state information before a human performs a pre-set human action. Human state information after humans perform preset human actions The corresponding state transition process, This is the gradient penalty coefficient (hyperparameter). Let D represent the gradient of the output of the preset discriminator with respect to the input. It expresses expectation.

[0096] In some embodiments, the action reward parameter is represented as:

[0097] in, Here, D represents the action reward parameter, and D represents the discriminator after training. In this formula... , These represent the robot's state information before the simulated robot executes the second action command, and the robot's state information after the simulated robot executes the second action command, respectively.

[0098] S504. Obtain the task reward parameters when the simulation robot executes the second action instruction.

[0099] Based on the second action command, the simulated robot is controlled to execute the corresponding action, and the task reward parameters when the simulated robot executes the second action command are obtained. These task reward parameters can be determined based on the following reward items: x-direction velocity deviation, y-direction velocity deviation, z-direction velocity deviation, angular velocity, attitude deviation, torque penalty, joint velocity, degree-of-freedom position restriction, torque restriction, tripping penalty, and tripping penalty during the swing phase (foot leaving the ground, swinging forward to prepare for the next landing). The tripping penalty and the swing phase tripping penalty are used to punish unexpected collisions between the foot and obstacles or terrain edges.

[0100] The specific formula for the task reward parameters is not particularly limited and can be selected according to the actual situation. It should be noted that the task reward function can guide the robot to maintain a stable posture and achieve smooth and efficient movement.

[0101] S505. Adjust the parameters of the student model based on the action reward parameters and the task reward parameters.

[0102] Based on the action reward parameters and task reward parameters, the total reward parameters are calculated. During the training of the student model based on the action loss, the parameters of the student model are simultaneously tuned based on the total reward parameters until the target action policy model is obtained.

[0103] In some embodiments, the total reward parameter is represented as:

[0104] in, For the total reward parameter, Action reward parameters The weight, For the set of reward items The Middle The weight of each reward item, For the first Reward parameters for each reward item.

[0105] In this embodiment, during the student model training phase, the model parameters are adjusted by the reward parameters, thereby optimizing the strategy through reinforcement learning until the strategy successfully imitates human behavior, thus guiding the robot to maintain a stable posture and achieve smooth and efficient movement.

[0106] Figure 7 This is a flowchart illustrating the robot motion control method provided in this embodiment. The executing entity in this embodiment can be a humanoid robot.

[0107] like Figure 7 As shown, the method may include: S601. Acquire target proprioception data of the humanoid robot in the target terrain environment, historical proprioception data of the target proprioception data, and continuous multi-frame target local depth images.

[0108] The target terrain environment can be any type of real terrain environment, such as other slopes, stairs, gaps, movable platforms, etc.

[0109] Target proprioceptive data refers to the proprioceptive data of the humanoid robot at the current moment in the target terrain environment, while historical proprioceptive data refers to the proprioceptive data of the humanoid robot at historical moments in the preset terrain environment. Proprioceptive data refers to the joint data of the humanoid robot's driving joints collected by internal sensors, such as joint position, joint velocity, joint angular velocity, etc. For example, if the current moment is time t, then the historical moment can include the moments between time t-50 and time t.

[0110] The humanoid robot is equipped with a single-depth camera, which is mounted on the robot's head or shoulder to cover a preset area in front of it, and is used to acquire local depth images at a preset frequency (such as 30Hz).

[0111] The continuous multi-frame target local depth images are depth images of a local area of ​​the humanoid robot observed at the current moment using a single depth camera in the target terrain environment. The local area can be, for example, a depth image of a preset range in front (such as a range of 1.0m × 1.0m in front of the humanoid robot, where "in front" refers to the direction of movement of the humanoid robot as a reference). There are multiple frames of target local depth images, such as 5 frames.

[0112] S602. Based on the historical proprioceptive data of the target proprioceptive data and the continuous multi-frame target local depth images, the pre-trained terrain reconstruction model in the robot motion control model is used to reconstruct the terrain and obtain the target local terrain geometric feature map of the target terrain environment.

[0113] The robot motion control model is a model trained using the aforementioned motion control model training method.

[0114] The historical proprioceptive data of the target and the target local depth images of multiple consecutive frames are used as input. A pre-trained terrain reconstruction model is used to reconstruct the terrain, and the target local terrain geometric feature map of the target terrain environment is obtained. The target local terrain geometric feature map is the height map of the preset range in front of the humanoid robot at the current moment.

[0115] It should be noted that the specific process of generating the target local terrain geometric feature map is similar to the process of generating the current local terrain geometric feature map, and can be found in the relevant descriptions above.

[0116] S603. Based on the target's propagational perception data, the target's local terrain geometric feature map, and the target control commands for the drive joints in the humanoid robot, acquire the target observation features.

[0117] Among them, the drive joint is the joint that drives the humanoid robot's movement. The target control command for the drive joint can carry the target linear velocity (planned linear velocity) and the target yaw rate (planned yaw rate) of the drive joint. The target linear velocity and the target yaw rate are determined from the target control command. The target linear velocity is used to indicate how fast the robot's center of mass or reference point moves in space along the direction of motion, and the target yaw rate is used to indicate how fast the robot rotates about an axis perpendicular to the plane of motion (xy plane).

[0118] During the movement of the humanoid robot in the target terrain environment, the target control commands of the driving joints of the humanoid robot at the current moment can be obtained. Based on the target body perception data, the target local terrain geometric feature map, and the target control commands of the driving joints of the humanoid robot, the target observation features can be obtained. The target observation features include: target body perception data, target local terrain geometric feature map, and target control commands.

[0119] S604. Based on the target observation characteristics, the target action strategy model in the robot motion control model is used to generate the target action instructions for driving the joints in the humanoid robot.

[0120] S605. Perform motion control on the humanoid robot according to the target motion command.

[0121] Based on the target observation characteristics, a target motion strategy model is adopted to generate target motion commands for the drive joints of the humanoid robot. Based on the target motion commands, the drive joints of the humanoid robot are controlled to move, so as to perform motion control on the humanoid robot.

[0122] In an optional implementation, step S604 above, which generates target motion commands for driving joints in the humanoid robot based on target observation features and using the target motion strategy model in the robot motion control model, may include: A preset attitude estimator is used to estimate the target proprioceptive data to obtain target estimation features. The target observation features also include target estimation features. The target proprioceptive data, target estimation features, and target control commands in the target observation features are determined as target blind control input features. The blind control backbone network in the target action strategy model is used to generate target blind control actions that drive the joints based on the target blind control input features. The perception network in the target action strategy model is used to generate target modulation actions that drive the joints based on the target local terrain geometric feature map, target blind control actions, and target blind control input features in the target observation features. Target action commands are then generated based on the target blind control actions and target modulation actions.

[0123] The preset attitude estimator is used to estimate the attitude of the humanoid robot based on the target proprioceptive data to obtain target estimation features. The target estimation features may include, for example, the estimated projected gravity and estimated linear velocity of the humanoid robot. The estimated projected gravity is used to indicate the degree of tilt of the humanoid robot's attitude relative to the direction of gravity. The estimated linear velocity is used to indicate the speed and direction of the humanoid robot's base (center of mass) translational motion in space. It is usually a 3-dimensional vector, corresponding to the linear velocity in the three coordinate directions.

[0124] The target observation features also include: target estimation features, which determine the target ontological perception data, target estimation features and target control commands in the target observation features as target blind control input features. The target action strategy model includes: a blind control backbone network and a perception network. Using the blind control backbone network, target blind control actions that drive joints are generated based on the target blind control input features. Using the perception network, target modulation actions that drive joints are generated based on the target local terrain geometric feature map, target blind control actions and target blind control input features in the target observation features.

[0125] Then, based on the target blind control action and the target modulation action, a target action command is generated. The action corresponding to the target action command can be a weighted sum of the actions corresponding to the target blind control action and the actions corresponding to the target modulation action.

[0126] It should be noted that the specific process for generating target action instructions is the same as described above. Figure 5The generation process of the second action instruction in the embodiment is similar, and please refer to the relevant description above for details.

[0127] Among them, the preset attitude estimator is distributed on the humanoid robot and is used to collect and estimate the projected gravity and the estimated linear velocity.

[0128] In this embodiment, a low-latency, highly robust perception and motion control scheme is implemented for humanoid robots using only a single depth camera, solving problems such as self-occlusion, noise interference, and difficulty in transferring simulation to reality. At the same time, it enables stable adaptive motion of embodied intelligent humanoid robots in complex terrains (slopes, stairs, gaps, movable platforms, etc.), reducing terrain reconstruction errors and perception latency, reducing hardware dependence, and improving training efficiency and environmental generalization ability.

[0129] In summary, this approach, by integrating a terrain-aware motion control strategy with a blind backbone network, a terrain reconstruction model, and a real depth image synthesis pipeline, effectively suppresses perceptual noise, occlusion, and domain disparity while maintaining training efficiency and exhibiting superior generalization capabilities. The humanoid robot can achieve flexible adaptive motion on stairs, ramps, gaps, and movable platforms, and end-to-end fine-tuning significantly reduces tripping frequency and perceptual latency. Therefore, the fusion of structured terrain reasoning and reinforcement learning provides a robust implementation path for reliable humanoid robot motion control in unstructured environments.

[0130] Figure 8 This is a schematic diagram of the structure of the motion control model training device provided in the embodiments of this application. The device can be integrated into a model training equipment.

[0131] like Figure 8 As shown, the device may include: The acquisition module 701 is used to acquire the current proprioceptive data, historical proprioceptive data and continuous multi-frame local depth images of the simulated robot in a preset terrain environment. Processing module 702 is used to perform terrain reconstruction based on historical ontological perception data and continuous multi-frame local depth images using a pre-trained terrain reconstruction model to obtain the current local terrain geometric feature map of the preset terrain environment. The acquisition module 701 is also used to acquire the current observed features of the driving joint based on the current body perception data, the current local terrain geometric feature map, and the preset control commands of the driving joint in the simulation robot. The processing module 702 is also used to generate a first action command and a second action command to drive the joint based on the current observation features, using the teacher model and the student model respectively; The processing module 702 is also used to train the student model according to the first action instruction and the second action instruction to obtain the target action strategy model; Output module 703 is used to output a robot motion control model including a pre-trained terrain reconstruction model and a target action strategy model, so as to perform motion control on the humanoid robot.

[0132] In one optional implementation, the pre-trained terrain reconstruction model includes: a feature fusion module, a recurrent memory unit, and a terrain reconstruction module; Processing module 702 is specifically used for: Based on historical ontological perception data and multiple consecutive frames of local depth images, a feature fusion module is used to perform feature fusion to obtain fused terrain features. A circular memory unit is used to accumulate the fused terrain features over time to obtain the accumulated terrain features; The terrain reconstruction module generates a local terrain geometry feature map based on accumulated terrain features.

[0133] In an optional implementation, the feature fusion module includes: a visual encoder, a state encoder, and a cross-attention network; the processing module 702 is specifically used for: Visual encoders and state encoders are used to encode multiple consecutive frames of local depth images and historical proprioceptive data, respectively, to obtain depth embedding vectors and proprioceptive embedding vectors. A cross-attention network is used to fuse cross-attention features based on deep embedding vectors and ontology-aware embedding vectors to obtain fused terrain features.

[0134] In an optional implementation, the terrain reconstruction module includes: a decoder, a conditional network; and a processing module 702, specifically used for: The accumulated terrain features are decoded using a decoder to obtain the current coarse height map; A conditional network is used with deep embedding vectors as conditions to refine the current coarse height map and obtain the current local terrain geometric feature map.

[0135] In an optional implementation, the processing module 702 is further configured to: A preset attitude estimator is used to estimate the current proprioceptive data, and the current estimated features are obtained. Module 701 is used specifically for: Based on the current proprioceptive data, current estimated features, current local terrain geometric feature map, and preset control commands, obtain the current observed features.

[0136] In an optional implementation, the processing module 702 is specifically used for: The current proprioceptive data, current estimated features, and preset control commands in the current observed features are identified as blind control input features; The first blind control backbone network in the teacher model is used to generate the first blind control action to drive the joint based on the blind control input features; Using the first perception network in the teacher model, the first modulation action to drive the joint is generated based on the current local terrain geometric feature map, the first blind control action, and the blind control input features in the current observation features. Based on the first blind control action and the first modulation action, the first action command is generated. The second blind control backbone network in the student model is used to generate the second blind control action to drive the joint based on the blind control input features; The second perception network in the student model is used to generate the second modulation action to drive the joint based on the current local terrain geometric feature map, the second blind control action, and the blind control input features in the current observation features. The second action command is generated based on the second blind control action and the second modulation action.

[0137] In an optional implementation, the acquisition module 701 is further configured to: Obtain the robot's state information after the simulated robot executes the second action command; The preset human motion state data corresponding to the robot state information is obtained from the preset demonstration database as reference demonstration data. The preset demonstration database includes: state data of multiple preset human motions. The processing module 702 is also used to generate action reward parameters based on robot state information and reference demonstration data using a preset discriminator; The acquisition module 701 is also used to acquire the task reward parameters when the simulated robot executes the second action instruction; The processing module 702 is also used to adjust the parameters of the student model based on the action reward parameters and the task reward parameters.

[0138] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0139] Figure 9 This is a schematic diagram of the structure of a robot motion control device provided in an embodiment of this application. This device can be integrated into a humanoid robot.

[0140] like Figure 9 As shown, the device may include: The acquisition module 801 is used to acquire target proprioceptive data of the humanoid robot in the target terrain environment, historical proprioceptive data of the target proprioceptive data, and continuous multi-frame target local depth images. The processing module 802 is used to perform terrain reconstruction based on the historical prophetic perception data of the target prophetic perception data and the target local depth images of multiple consecutive frames, using the pre-trained terrain reconstruction model in the robot motion control model, to obtain the target local terrain geometric feature map of the target terrain environment; wherein, the robot motion control model is a model trained using the above method. The acquisition module 801 is also used to acquire target observation features based on target body perception data, target local terrain geometric feature map, and target control commands for driving joints in the humanoid robot; The processing module 802 is also used to generate target action instructions for driving joints in a humanoid robot by adopting the target action strategy model in the robot motion control model based on the target observation characteristics. The control module 803 is used to control the humanoid robot's movements according to the target action instructions.

[0141] In an optional implementation, the processing module 802 is specifically used for: A preset attitude estimator is used to estimate the target body perception data to obtain target estimation features. The target observation features also include: target estimation features; The target ontological sensing data, target estimation features, and target control commands in the target observation features are identified as the target blind control input features; The blind control backbone network in the target action strategy model is used to generate target blind control actions that drive the joints based on the target blind control input features. The perceptual network in the target action strategy model is used to generate target modulation actions that drive the joints based on the target local terrain geometric feature map, target blind control actions and target blind control input features in the target observation features. Target action commands are then generated based on the target blind control actions and target modulation actions.

[0142] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0143] Figure 10 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 10 As shown, the device may include a processor 901 and a memory 902. The memory 902 stores machine-readable instructions that can be executed by the processor 901. When the electronic device is running, the processor 901 executes the machine-readable instructions to perform the above-mentioned motion control model training or robot motion control method.

[0144] The electronic device can be either the aforementioned model training device or a humanoid robot.

[0145] This application also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the above-described method.

[0146] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0147] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0148] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0150] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0151] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0152] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for training a motion control model, characterized in that, include: Acquire current proprioceptive data, historical proprioceptive data, and multiple consecutive frames of local depth images of the simulated robot in a preset terrain environment; A pre-trained terrain reconstruction model is used to reconstruct the terrain based on the historical ontological perception data and the continuous multi-frame local depth images, thereby obtaining the current local terrain geometric feature map of the preset terrain environment. Based on the current proprioceptive data, the current local terrain geometric feature map, and the preset control commands of the drive joints in the simulation robot, the current observed features of the drive joints are obtained; Based on the current observation characteristics, the teacher model and the student model are used respectively to generate the first action command and the second action command of the driving joint; The student model is trained according to the first action instruction and the second action instruction to obtain the target action strategy model; The output includes a robot motion control model comprising the pre-trained terrain reconstruction model and the target action strategy model, for motion control of the humanoid robot.

2. The method according to claim 1, characterized in that, The pre-trained terrain reconstruction model includes: a feature fusion module, a recurrent memory unit, and a terrain reconstruction module; The pre-trained terrain reconstruction model is used to reconstruct the terrain based on the historical ontological perception data and the continuous multi-frame local depth images, obtaining the current local terrain geometric feature map of the preset terrain environment, including: Based on the historical ontological perception data and multiple consecutive frames of local depth images, the feature fusion module is used to perform feature fusion to obtain fused terrain features. The fused terrain features are accumulated over time using the circular memory unit to obtain accumulated terrain features; The terrain reconstruction module generates the current local terrain geometric feature map based on the accumulated terrain features.

3. The method according to claim 2, characterized in that, The feature fusion module includes: a visual encoder, a state encoder, and a cross-attention network; The step of fusing features based on the historical ontological perception data and multiple consecutive frames of local depth images using the feature fusion module to obtain fused terrain features includes: The visual encoder and the state encoder are used to encode the continuous multi-frame local depth images and the historical ontology perception data, respectively, to obtain the depth embedding vector and the ontology perception embedding vector. The cross-attention network is used to perform cross-attention feature fusion based on the deep embedding vector and the ontology-aware embedding vector to obtain the fused terrain features.

4. The method according to claim 3, characterized in that, The terrain reconstruction module includes: a decoder and a conditional network; the step of generating the current local terrain geometric feature map based on the accumulated terrain features using the terrain reconstruction module includes: The accumulated terrain features are decoded using the decoder to obtain the current coarse height map; The conditional network is used to refine the current coarse height map using the deep embedding vector as a condition, thereby obtaining the current local terrain geometric feature map.

5. The method according to claim 1, characterized in that, The method further includes: A preset attitude estimator is used to estimate the current proprioceptive data to obtain the current estimated features; The step of obtaining the current observed features of the drive joint based on the current proprioceptive data, the current local terrain geometric feature map, and the preset control commands of the drive joint in the simulated robot includes: The current observed features are obtained based on the current ontological perception data, the current estimated features, the current local terrain geometric feature map, and the preset control commands.

6. The method according to claim 5, characterized in that, The step of generating the first and second motion commands for the drive joint based on the current observed features, using both the teacher and student models respectively, includes: The current proprioceptive data, the current estimated features, and the preset control commands in the current observed features are determined as blind control input features; Using the first blind control backbone network in the teacher model, the first blind control action of the driving joint is generated based on the blind control input features; Using the first perception network in the teacher model, the first modulation action of the drive joint is generated based on the current local terrain geometric feature map, the first blind control action, and the blind control input feature in the current observation features, and the first action command is generated based on the first blind control action and the first modulation action. Using the second blind control backbone network in the student model, the second blind control action of the driving joint is generated based on the blind control input features; Using the second perception network in the student model, the second modulation action of the drive joint is generated based on the current local terrain geometric feature map in the current observation features, the second blind control action, and the blind control input features. The second action command is generated based on the second blind control action and the second modulation action.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the robot state information after the simulated robot executes the second action command; The robot state information corresponding to the preset human action state data is obtained from the preset demonstration database as reference demonstration data. The preset demonstration database includes: state data of multiple preset human actions. A preset discriminator is used to generate action reward parameters based on the robot's state information and the reference demonstration data; Obtain the task reward parameters when the simulated robot executes the second action instruction; The student model is tuned based on the action reward parameters and the task reward parameters.

8. A robot motion control method, characterized in that, include: Acquire target proprioceptive data of a humanoid robot in a target terrain environment, historical proprioceptive data of the target proprioceptive data, and continuous multi-frame target local depth images; Based on the historical proprioceptive data of the target proprioceptive data and the continuous multi-frame target local depth images, a pre-trained terrain reconstruction model in the robot motion control model is used to reconstruct the terrain to obtain the target local terrain geometric feature map of the target terrain environment; wherein, the robot motion control model is a model trained by the method described in any one of claims 1-7. Based on the target body perception data, the target local terrain geometric feature map, and the target control commands for the drive joints in the humanoid robot, the target observation features are obtained; Based on the target observation characteristics, the target action strategy model in the robot motion control model is used to generate the target action commands that drive the joints of the humanoid robot. The humanoid robot is controlled to perform actions according to the target action command.

9. The method according to claim 8, characterized in that, The step of generating target motion commands for driving joints in the humanoid robot based on the target observation features and using the target motion strategy model in the robot motion control model includes: A preset attitude estimator is used to estimate the target body perception data to obtain target estimation features, and the target observation features further include the target estimation features; The target ontological sensing data, the target estimation features, and the target control commands in the target observation features are determined as target blind control input features; Using the blind control backbone network in the target action strategy model, the target blind control action of the driving joint is generated based on the target blind control input features; Using the perception network in the target action strategy model, the target local terrain geometric feature map, the target blind control action, and the target blind control input feature in the target observation features are used to generate the target modulation action of the drive joint, and the target action command is generated based on the target blind control action and the target modulation action.

10. An electronic device, characterized in that, The method includes: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to perform the method according to any one of claims 1 to 9.

Images