Robot motion control methods and devices

By determining the trustworthy state of depth image blocks and masking untrustworthy blocks in a quadruped robot, and combining a teacher-student architecture encoder and policy network, the problem of control failure caused by visual degradation is solved, and highly robust and agile motion control in complex environments is achieved.

CN123103692APending Publication Date: 2026-11-17INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611586010.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-14
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

When quadruped robots perform agile movements in complex terrain, visual degradation caused by factors such as visual sensor noise, occlusion, and reflection leads to terrain misjudgment and control failure. Existing methods are difficult to effectively integrate vision and proprioception for control.

Method used

By acquiring depth images and ontology-aware sequences, the trustworthiness of image patches is determined, untrusted image patches are masked, and an encoder and policy network trained using a teacher-student architecture are used to fuse visual and ontology-aware information to generate accurate control commands.

Benefits of technology

In cases of local or global visual degradation, it improves the robustness and agility of robot motion, reduces the impact of erroneous data on control commands, and enhances the accuracy of control commands.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123103692A_ABST
    Figure CN123103692A_ABST
Patent Text Reader

Abstract

This application provides a robot motion control method and apparatus, relating to the field of robotics. The method includes: acquiring a depth image of the environment in which the robot to be controlled is currently located and a proprioception sequence within a preset time period; determining the trustworthiness state of each image block in the depth image, and masking image blocks with a trustworthiness state of untrustworthiness in the depth image to obtain a masking image and a trustworthiness mask for the image blocks; inputting the masking image, the proprioception sequence, and the trustworthiness mask for the image blocks into a first encoder to obtain terrain latent variables; inputting the proprioception sequence into a second encoder to obtain context latent variables; and inputting the terrain latent variables, context latent variables, and the proprioception data at the current moment into a policy network to obtain control commands for the robot to be controlled; wherein the first encoder, the second encoder, and the policy network are trained based on a teacher-student architecture. This application can fuse vision and proprioception to control the robot according to the actual situation of the depth image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of robotics, and more specifically, to a robot motion control method and apparatus. Background Technology

[0002] When quadruped robots perform agile movements such as parkour in complex terrain, they usually rely on depth vision to perceive the terrain in advance in order to plan their actions. However, in actual deployment environments, depth images often suffer from local or global visual degradation due to sensor noise, occlusion, reflection, or motion blur. Directly using unreliable depth images can lead to terrain misjudgment and control failure. Related methods either completely trust vision or directly switch to pure proprioception when vision degrades, which is difficult to adjust according to the actual situation.

[0003] Therefore, how to integrate vision and proprioception to control robots according to actual conditions has become a technical problem that the industry urgently needs to solve. Summary of the Invention

[0004] This application provides a robot motion control method and apparatus to solve the technical problem in related technologies of how to control a robot by integrating vision and proprioception according to actual conditions.

[0005] In a first aspect, this application provides a robot motion control method, including: Acquire depth images of the environment in which the robot to be controlled is currently located, as well as the body perception sequence within a preset time period; Determine the trust status of each image block in the depth image, and mask the image blocks with the trust status as untrusted in the depth image to obtain a masking image and a trust mask for the image blocks in the depth image; The masked image, the ontology-aware sequence, and the image patch trusted mask are input into the first encoder to obtain the terrain latent variables output by the first encoder; The ontology-aware sequence is input into the second encoder to obtain the context latent variables output by the second encoder; The terrain latent variables, the context latent variables, and the current body perception data are input into the policy network to obtain the control instructions output by the policy network. The control instructions are used to control the robot to be controlled to move. The first encoder, the second encoder, and the policy network are trained based on a teacher-student architecture.

[0006] In some embodiments, before acquiring the depth image of the environment in which the robot to be controlled is currently located, the method further includes: A reference image set is constructed based on multiple frames of depth images acquired by the robot under control when its vision is unobstructed. Each reference image in the reference image set is divided into multiple reference image blocks, and reference features of each reference image block are extracted; The reference features of the reference image blocks at the same image location in the reference image set are aggregated to obtain a block memory library; Determining the confidence state of each image patch in the depth image includes: The trusted state of each image block is determined based on the block memory.

[0007] In some embodiments, determining the trusted state of each image block based on the block memory includes: Extract image block features for each of the image blocks; Determine the feature distance between each image block feature and the reference feature corresponding to the image block feature in the block memory; The image block whose feature distance is less than a preset distance threshold is marked as trustworthy; Image blocks whose feature distance is greater than or equal to the preset distance threshold are marked as untrustworthy.

[0008] In some embodiments, determining the confidence state of each image patch in the depth image includes: The depth image is reconstructed to obtain a reconstructed image; Obtain the reconstruction error between the reconstructed image and the depth image; If the reconstruction error is greater than a preset whole frame error threshold, the trust status of the depth image is marked as untrusted, so that the trust mask of the image block indicates that the trust status of all image blocks is untrusted. When the reconstruction error is less than or equal to the preset whole frame error threshold, the confidence state of each image block is determined based on the block memory, which stores the reference features of each image block.

[0009] In some embodiments, the step of inputting the masking image, the ontology-aware sequence, and the image patch trusted mask into a first encoder to obtain the terrain latent variables output by the first encoder includes: The masking image, the ontology-aware sequence, and the image patch trusted mask are input into the first encoder, and the first encoder is controlled to perform the following steps: The masked image and the image block trusted mask are encoded into block visual features, and the ontology-aware sequence is encoded into motion features; The query vector is determined based on the motion features and the image patch confidence mask; The segmented visual features are concatenated with the motion features to obtain multimodal features; The query vector is controlled to perform multi-head attention weight allocation on the multimodal features based on the image patch trust mask; The multimodal features are fused based on the assigned attention weights to obtain the terrain latent variables.

[0010] In some embodiments, controlling the query vector to perform multi-head attention weight allocation on the multimodal features based on the image patch trust mask includes: When the image patch trust mask indicates the existence of an image patch with a trust status, the query vector is controlled to allocate attention weights to the block visual features and the motion features in the multimodal features; When the image patch trust mask indicates that the trust status of all image patches is untrusted, the query vector is controlled to only allocate attention weights to the motion features in the multimodal features.

[0011] In some embodiments, the training of the first encoder, the second encoder, and the policy network includes the following steps: Construct an initial unified network, which includes a teacher path, a student path, and an initial policy network shared by the teacher path and the student path, wherein the student path includes an initial first encoder and an initial second encoder; The privileged information in the sample simulation environment is input into the teacher pathway to obtain the teacher terrain latent variable and teacher context latent variable output by the teacher pathway. The privileged information includes the local terrain elevation map and physical dynamic parameters in the sample simulation environment. The sample masking image, the sample image patch trusted mask, and the sample ontology perception sequence are input into the student pathway to obtain the student terrain latent variable and the student context latent variable output by the student pathway. The data output from the student channel or the data output from the teacher channel is input into the initial policy network to obtain the control instructions output by the initial policy network. The network parameters of the initial unified network are optimized based on the data output from the student channel, the data output from the teacher channel, the control instructions, and the preset optimization objectives. After the network parameters are optimized, the teacher path is discarded, and the initial first encoder, the initial second encoder, and the initial policy network retained in the initial unified network are respectively used as the first encoder, the second encoder, and the policy network.

[0012] In some embodiments, the optimization objective includes a distillation objective and a reinforcement learning objective. Optimizing the network parameters of the initial unified network based on the data output from the student pathway, the data output from the teacher pathway, the control instructions, and the preset optimization objective includes: The network parameters of the initial unified network are optimized with the goal of approximating the teacher's terrain latent variable with the student's terrain latent variable and approximating the teacher's context latent variable with the student's context latent variable. Based on the interaction between the control command and the sample simulation environment, a control trajectory is generated, and the single-step reward corresponding to the control trajectory is obtained, so as to maximize the cumulative reward and optimize the network parameters for the reinforcement learning objective.

[0013] In some embodiments, it also includes: Gradually increase the proportion of data output from the student channel to be input into the initial policy network, and gradually decrease the proportion of data output from the teacher channel to be input into the initial policy network, until all inputs to the initial policy network are switched to data output from the student channel.

[0014] Secondly, this application provides a robot motion control device, comprising: The acquisition module is used to acquire depth images of the environment in which the robot to be controlled is currently located, as well as the body perception sequence within a preset time period. The determination module is used to determine the trust status of each image block in the depth image, and to mask the image blocks with the trust status as untrusted in the depth image to obtain a masking image and a trust mask for the image blocks in the depth image; The first encoding module is used to input the masking image, the body-aware sequence and the image patch trusted mask into the first encoder to obtain the terrain latent variable output by the first encoder; The second encoding module is used to input the ontology-aware sequence into the second encoder to obtain the context latent variables output by the second encoder; The control module is used to input the terrain latent variables, the context latent variables, and the current body perception data into the policy network to obtain the control instructions output by the policy network. The control instructions are used to control the robot to be controlled to move. The first encoder, the second encoder, and the policy network are trained based on a teacher-student architecture.

[0015] The robot motion control method and apparatus provided in this application, by determining the credible state of each image block in the depth image and masking untrusted image blocks, can accurately isolate visual degradation areas caused by noise or local occlusion according to the actual situation, avoiding erroneous visual cues from misleading motion decisions; by inputting the masked image, the body perception sequence, and the credible mask of the image block into the first encoder, it can guide the effective fusion of visual information and body perception information based on the credible mask of the image block, thereby generating accurate and interference-resistant terrain latent variables; by combining terrain latent variables and context latent variables to generate control commands, it can reduce the impact of erroneous data on the generated control commands in various scenarios of local visual degradation or global visual failure, improve the accuracy of the generated control commands, and thus effectively improve the robot's motion robustness and agility in various scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the robot motion control method provided in an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of the structure of the initial unified network provided in the embodiments of this application.

[0019] Figure 3 A performance comparison diagram of the robot motion control method provided in this application embodiment and other methods on a progressive obstacle course.

[0020] Figure 4 These are schematic diagrams of various depth images provided in the embodiments of this application.

[0021] Figure 5 The image shows a comparison of the robustness of the robot motion control method provided in this application embodiment with other methods under simulated deep damage.

[0022] Figure 6 An outdoor deployment route map provided for an embodiment of this application.

[0023] Figure 7 This is a schematic diagram of the structure of the robot motion control device provided in the embodiments of this application.

[0024] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0027] Figure 1 This is a flowchart illustrating the robot motion control method provided in the embodiments of this application, as shown below. Figure 1 As shown, the method includes the following steps. These steps are merely one possible implementation of this application.

[0028] Step 110: Obtain a depth image of the environment in which the robot to be controlled is currently located, as well as a body perception sequence within a preset time period.

[0029] Specifically, the robot motion control method provided in this application is applicable to a terminal, which can be various electronic devices that are communicatively connected to the robot to be controlled, including servers, smartphones, tablets, laptops, and desktop computers; or it can be an onboard computing platform installed on the robot body to be controlled.

[0030] The robot motion control method provided in this application is executed by a robot motion control device, which can be a hardware device independently set in the terminal or a software program running in the terminal.

[0031] The robot to be controlled can be a quadruped robot, and visual sensors such as depth cameras and body perception sensors can be installed in the robot.

[0032] A depth image is a two-dimensional pixel array that provides geometric information about the surrounding environment, denoted as . , For time. Depth images, as visual sensing data, provide the robot to be controlled with visual observation information for terrain prediction.

[0033] An ontology perception sequence refers to a set of multiple ontology perception data arranged chronologically from the current moment back to a preset time period, denoted as . Body perception data can include the joint positions, joint velocities, body angular velocities, body orientation, and foot contact status of the robot to be controlled.

[0034] The preset duration can be determined by combining empirical settings with simulation experiment optimization.

[0035] The depth camera acquires real-time depth images of the terrain in front of the robot to be controlled, while the body perception sensor acquires the body perception sequence.

[0036] Step 120: Determine the trust status of each image block in the depth image, and mask the image blocks with a trust status of untrust in the depth image to obtain the image block trust mask of the masked image and the depth image.

[0037] Specifically, a trustworthy state is a binary label indicating whether the depth measurement of each image patch in a depth image is reliable, including trustworthy and untrustworthy.

[0038] A masked image is a depth image obtained by setting the trusted image patches to zero in the original depth image, retaining only the depth information of the trusted image patches.

[0039] An image patch confidence mask is a binary mask matrix that records the confidence state of each image patch in a depth image. Its dimension is the same as the number of image patches in the depth image.

[0040] The acquired full-frame depth image is divided into multiple local image blocks, and the credibility status of each image block is evaluated. For image blocks whose data is distorted due to physical occlusion, sensor noise, or surface reflection, their credibility status is determined to be untrustworthy; for image blocks with normal data, their credibility status is determined to be trustworthy.

[0041] In the depth image, untrusted image patches are masked, and only trustworthy image patches are retained to form a masked image. At the same time, a trustworthy image patch mask that records the location distribution of trustworthy state of each image patch is generated. In this way, while retaining reliable terrain forward-looking cues, local untrustworthy image patches are isolated to deal with situations where local visual degradation or global visual failure is caused by sensor noise, physical occlusion or lighting artifacts.

[0042] Step 130: Input the masked image, the body-aware sequence, and the image patch trusted mask into the first encoder to obtain the terrain latent variables output by the first encoder.

[0043] Specifically, the masked image, the body perception sequence, and the image block trusted mask are input into the first encoder. The first encoder uses the image block trusted mask as a guiding condition and dynamically fuses the block visual features corresponding to the image block trusted mask and the motion features corresponding to the body perception sequence according to the trusted state of each data block indicated by the image block trusted mask.

[0044] For example, in areas where visual information is reliable, i.e., where data blocks in a reliable state are located, the first encoder relies on block visual features to predict the terrain, or relies on block visual features and motion features to predict the terrain. In areas where visual information is unreliable, i.e., where data blocks in an unreliable state are located, or when depth images are missing, the first encoder relies only on motion features to predict the terrain, and finally outputs terrain latent variables that can accurately characterize the geometric state of the terrain around the robot to be controlled.

[0045] Step 140: Input the ontology perception sequence into the second encoder to obtain the context latent variables output by the second encoder.

[0046] Specifically, the body perception sequence within a preset time period is input into the second encoder. The second encoder performs feature extraction and compression on these time series data, infers the implicit dynamic physical properties and body response patterns, and generates contextual latent variables that reflect the current motion state of the robot to be controlled, providing contextual reference for the continuous motion of the robot to be controlled.

[0047] Step 150: Input the terrain latent variables, context latent variables and the current body perception data into the policy network to obtain the control instructions output by the policy network. The control instructions are used to control the robot to be controlled to move. The first encoder, the second encoder, and the policy network are trained based on a teacher-student architecture.

[0048] Specifically, terrain latent variables representing environmental geometry, contextual latent variables representing motion state, and the latest ontology perception data are input into the policy network. The policy network integrates these features and outputs control commands required to keep the robot under control stable and perform specific tasks. The control commands are then sent to the robot execution layer. The control commands include multi-dimensional motion vectors, with each dimension of the motion vector corresponding to the target position and / or velocity command of the robot's leg or wheel hub joint.

[0049] This embodiment also uses a trained unified network, which includes a first encoder, a second encoder, and a policy network.

[0050] The unified network is a neural network trained on an initial unified network based on sample depth images, sample ontology perception sequences, and sample control commands corresponding to the sample depth images and sample ontology perception sequences.

[0051] Figure 2 A schematic diagram of the structure of the initial unified network provided in the embodiments of this application, as shown below. Figure 2 As shown, the initial unified network is built on a teacher-student architecture, including teacher pathways, student pathways, and an initial policy network shared by teacher and student pathways. The student pathway includes an initial first encoder and an initial second encoder.

[0052] By distilling loss, the data output from the student path is made closer to the data output from the teacher path. The network parameters of the initial unified network are updated by combining proximal policy optimization (PPO). After the update is completed, the teacher path is discarded, and the final first encoder, second encoder and policy network are obtained.

[0053] The robot motion control method provided in this application determines the credible state of each image block in the depth image and masks untrusted image blocks. It can accurately isolate visual degradation areas caused by noise or local occlusion according to the actual situation, avoiding erroneous visual cues from misleading motion decisions. By inputting the masked image, the body perception sequence, and the credible mask of the image block into the first encoder, it can guide the effective fusion of visual information and body perception information based on the credible mask of the image block, thereby generating accurate and interference-resistant terrain latent variables. By combining terrain latent variables and context latent variables to generate control commands, it can reduce the impact of erroneous data on the generated control commands in various scenarios of local visual degradation or global visual failure, improve the accuracy of the generated control commands, and thus effectively improve the robot's motion robustness and agility in various scenarios.

[0054] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0055] In some embodiments, prior to step 110, the method further includes: A reference image set is constructed based on multiple frames of depth images acquired by the robot under control when its vision is unobstructed. Each reference image in the reference image set is divided into multiple reference image blocks, and reference features of each reference image block are extracted; The reference features of reference image blocks at the same image location in the reference image set are aggregated to obtain a block memory library; Step 120 includes: The trusted state of each image block is determined based on the block memory.

[0056] The process of determining the trust status of each image block based on a block memory includes: Extract image patch features from each image patch; Determine the feature distance between each image patch feature and the reference feature corresponding to the image patch feature in the block memory; Image blocks with feature distances less than a preset distance threshold are marked as trustworthy. Image blocks whose feature distance is greater than or equal to a preset distance threshold are marked as untrustworthy.

[0057] Step 120 also includes: Image reconstruction is performed on the depth image to obtain the reconstructed image; Obtain the reconstruction error between the reconstructed image and the depth image; If the reconstruction error is greater than the preset whole frame error threshold, the trust status of the depth image is marked as untrusted, so that the trust mask of the image block indicates that the trust status of all image blocks is untrusted. When the reconstruction error is less than or equal to the preset whole frame error threshold, the reliable state of each image block is determined based on the block memory, which stores the reference features of each image block.

[0058] Specifically, when the robot to be controlled experiences local visual degradation during movement, if the entire depth image is input into the policy network, incorrect terrain geometry information will be introduced into the policy network, thereby affecting the accuracy of the output control commands. If a simple whole-frame masking mechanism is used, the effective visual area will be discarded. Therefore, it is crucial to accurately identify credible visual areas in the depth image. To this end, this application embodiment designs a two-level visual credibility state evaluation mechanism of whole frames and image blocks.

[0059] In this embodiment, a block memory library is pre-built and a deep autoencoder (AE) is trained to evaluate image reconstruction errors.

[0060] Specifically, during the robot's unobstructed movement, normal depth trajectories are collected. These trajectories are then used to train a convolutional neural network deep autoencoder containing an encoder and a decoder. Features extracted from the trained autoencoder are then used to construct a block-based memory. Multiple frames of depth images acquired by the robot under control when its vision is unobstructed are used as multi-frame reference images to construct a reference image set. Each reference image in the reference image set is then divided into... Each image patch is used as a reference feature, extracted from the pre-trained autoencoder. From a set of pre-defined normal reference images with different terrain categories and difficulty levels, reference features corresponding to the location of the image are uniformly extracted and aggregated into a subset of reference features specific to that location. By integrating subsets of reference features from all locations, a complete block memory is obtained. , .

[0061] After acquiring the depth image of the environment in which the robot to be controlled is currently located, in order to avoid global erroneous inputs from passing the local check incorrectly due to a small number of image blocks having abnormally low scores, i.e. to prevent the entire depth image from being severely damaged, but individual image blocks not being intercepted by the local filtering mechanism because they happen to be similar to normal features, a whole frame confidence status assessment and an image block confidence status assessment will be performed. The whole frame confidence status assessment and the image block confidence status assessment can be performed simultaneously or sequentially.

[0062] If performed sequentially, first use After Effects (AE) to perform a full-frame confidence state assessment, then use the depth image... The image is input to After Effects (AE), which performs image reconstruction on the depth image and outputs the reconstructed image. The formula is .

[0063] in, This represents the reconstruction operation of a depth autoencoder.

[0064] The reconstruction error between the reconstructed image and the depth image is obtained. The reconstruction error may include the reconstruction mean square error and the gradient mean square error. The preset whole frame error threshold is set to the average error of the frames collected when the vision sensor of the real robot is completely occluded. The point clouds of these frames are all invalid data.

[0065] If either the reconstruction mean square error or the gradient mean square error exceeds the preset whole frame error threshold, the whole frame depth image is masked, and the trust status of the depth image is marked as untrustworthy. When the trust status of the depth image is untrustworthy, the trust mask of the subsequent image patch will indicate that the trust status of all image patches is untrustworthy. The policy network is essentially outputting a control policy based on the context latent variables and the ontology perception data at the current moment. When the reconstruction error is less than or equal to the preset whole frame error threshold, the reliability status of each image block is determined based on the block memory.

[0066] Extracting location from depth image using the encoder in the trained autoencoder Image patch features And determine the image patch features and their location in the block memory. Corresponding reference features The characteristic distance between them, i.e., the nearest neighbor score, characteristic distance The calculation formula is as follows: ; Feature distance less than a preset distance threshold Image patches whose feature distance is greater than or equal to a preset distance threshold are marked as trustworthy, and image patches whose feature distance is greater than or equal to a preset distance threshold are marked as untrustworthy. The determination formula is as follows: ; The preset distance threshold is obtained by calibrating the normal trajectory according to its position. For indicator functions; To indicate the result of the function, This indicates that the image patch at this location is considered trustworthy, and the image patch is retained. The image patch at that location is considered untrustworthy, and the image patch is rejected. This location-based comparison method can preserve reliable regions when there is local visual degradation.

[0067] It should be noted that both the image patch features and the reference features are depth space features extracted from the encoder hidden layer in the autoencoder, and the depth image and the block memory are divided in the same way. Therefore, the image patch features of each image patch can find the corresponding reference features in the block memory.

[0068] The confidence state assessment process described above transforms the depth image into a confidence representation formula, as shown below: ; in, Represents the trusted state assessment process; image patch trusted mask Masking images The system masks image patches that are considered untrusted, retaining only trusted regions, i.e., the regions where image patches are considered trustworthy. This representation can isolate local faults while retaining reliable terrain forward-looking cues.

[0069] If the process is synchronous, and if the credibility state of the depth image is untrusted while there are image patches with a credibility state, the image patch credibility mask still indicates that the credibility state of all image patches is untrusted. The policy network essentially outputs based on the context latent variables and the ontology perception data at the current moment.

[0070] Furthermore, during the training phase of the unified network, instead of relying on the aforementioned online reliability assessment process, the network initially inputs visually normal, locally degraded, and globally degraded depth images through a masked course. This allows the teacher-student-based unified network to learn a unified interface under various visual conditions. Instead of relying on a single ideal camera condition.

[0071] The robot motion control method provided in this application combines a depth autoencoder and a block memory library to perform reliable state assessment, which can accurately isolate local visual faults and retain reliable terrain forward cues, thus avoiding global misjudgment caused by a few abnormal features.

[0072] In some embodiments, step 130 includes: The masked image, the body-aware sequence, and the image patch trusted mask are input into the first encoder, which is then controlled to perform the following steps: The masked image and image patch trusted mask are encoded into block visual features, and the ontology-aware sequence is encoded into motion features; The query vector is determined based on motion features and image patch confidence masks; By concatenating the segmented visual features with motion features, multimodal features are obtained; The query vector is controlled by multi-head attention weight allocation for multimodal features based on image patch trust masks; Multimodal features are fused based on assigned attention weights to obtain terrain latent variables.

[0073] The control query vector is used to perform multi-head attention weight allocation on multimodal features based on image patch trust masks, including: When the image patch trust mask indicates the existence of an image patch with a trust status, the control query vector assigns attention weights to the block visual features and motion features in the multimodal features; When the image patch trust mask indicates that the trust status of all image patches is untrusted, the control query vector only assigns attention weights to motion features in the multimodal features.

[0074] Specifically, block visual features are the set of visual feature vectors of all image blocks.

[0075] Motion features are compact feature vectors that characterize the motion state of a robot to be controlled in the near future.

[0076] In the multi-head attention mechanism, the query vector is a normalized importance coefficient assigned to each term in the multimodal features, which is constrained by the image patch credibility mask.

[0077] The first encoder is used to receive masked images, ontology-aware sequences, and image patches, and to construct an accurate terrain representation, i.e., terrain latent variables.

[0078] The first encoder includes a depth encoder and a history encoder, utilizing the depth encoder. The trusted masking depth frame is the masking image. and image patch trusted mask Mapping encoding into block visual features The formula is shown below: ; At the same time, utilizing the historical encoder Received body perception sequence Summarized as compact motion characteristics The formula is shown below: ; Subsequently, using the first learnable projection Construct query vectors The formula is shown below: ; The query vector effectively encodes the motion context and visual confidence conditions of the robot to be controlled. Then, the segmented visual features and motion features are concatenated into words to form a set of visual and ontology perception words. , This set is the multimodal feature set, where “;” represents word concatenation.

[0079] During the fusion phase, the first encoder uses a multi-head attention (MHA) mechanism to control the query vector to reallocate the weights of block visual features and motion features based on the image patch trust mask. The internal dynamic allocation logic is as follows: if the image patch trust mask indicates that there are image patches with a trust state, these trust image patches will participate in the allocation of attention weights to predict obstacles ahead, while image patches with an untrust state will not dominate subsequent terrain inference because they are masked; if the image patch trust mask indicates that the trust state of all current image patches is untrusted, the attention will automatically mask visual information, i.e., block visual features, and only act on the ontology perception features. This means that pure ontology perception control is an automatic degradation limit case under the fusion mechanism, without the need to switch independent policy branches.

[0080] Finally, based on the assigned attention weights, the segmented visual features and motion features are sequentially passed through a Gated Recurrent Unit (GRU) and a second learnable projection. The process is performed to calculate the fused terrain latent variables. Modal fusion is completed, and the formula is shown below: ; The robot motion control method provided in this application dynamically allocates the weights of multimodal features in a multi-head attention mechanism based on image patch trusted masks. This can retain usable forward-looking prediction cues when visual local damage occurs, and smoothly revert to pure ontology perception when global failure occurs, thereby improving the robustness of robot control.

[0081] In some embodiments, training the first encoder, the second encoder, and the policy network includes the following steps: Construct an initial unified network, which includes teacher pathways, student pathways, and an initial policy network shared by teacher and student pathways. The student pathway includes an initial first encoder and an initial second encoder. The privileged information in the sample simulation environment is input into the teacher pathway to obtain the teacher terrain latent variables and teacher context latent variables output by the teacher pathway. The privileged information includes the local terrain elevation map and physical dynamic parameters in the sample simulation environment. Input the sample masking image, the sample image patch trusted mask, and the sample ontology perception sequence into the student pathway to obtain the student terrain latent variable and the student context latent variable output by the student pathway. Input the data output from the student channel or the teacher channel into the initial policy network to obtain the control instructions output by the initial policy network. The network parameters of the initial unified network are optimized based on the data output from the student pathway, the data output from the teacher pathway, control instructions, and the preset optimization objectives. After the network parameters are optimized, the teacher path is discarded, and the initial first encoder, initial second encoder and initial policy network retained in the initial unified network are used as the first encoder, second encoder and policy network, respectively.

[0082] The optimization objectives include distillation objectives and reinforcement learning objectives. Based on data output from the student pathway, data output from the teacher pathway, control instructions, and the preset optimization objectives, the network parameters of the initial unified network are optimized, including: The network parameters of the initial unified network are optimized by distillation objectives of approximating the teacher's latent variables with the student's latent variables of terrain and approximating the teacher's latent variables with the student's latent variables of context. The control trajectory is generated by interacting with the sample simulation environment based on the control command, and the single-step reward corresponding to the control trajectory is obtained. The network parameters are optimized with the goal of maximizing the cumulative reward.

[0083] This also includes: Gradually increase the proportion of data output from the student pathway into the initial policy network, and gradually decrease the proportion of data output from the teacher pathway into the initial policy network, until all inputs to the initial policy network are switched to data output from the student pathway.

[0084] Specifically, the initial unified network is a unified network that has not yet been trained or whose training has not yet ended.

[0085] The teacher path is an encoder branch used solely for training, comprising a privileged encoder and a scan encoder, used to provide distillation targets for the student path.

[0086] The student pathway is a deployable encoder branch.

[0087] The initial policy network is a policy network that has not yet been trained or whose training has not yet ended.

[0088] Based on a single-stage teacher-student architecture and an asymmetric policy-value network, the initial unified network includes teacher pathways, student pathways, and an initial policy network shared by teacher and student pathways.

[0089] In this embodiment, to perform effective reinforcement learning, motion under visual degradation conditions is modeled as a partially observable Markov decision process. The formula is shown below: ; Among them, terrain distribution is defined as ; It is a visually degenerate distribution; State space represents the set of kinematic and dynamic states of the robot's joints and its physical environment. The action space represents the set of all target instructions that can be executed by each joint of the robot to be controlled. This refers to the deployment observation space, which is the data that the robot to be controlled can actually acquire through its onboard sensors when it is running in the actual physical environment, such as a collection of depth images and body perception sequences. This represents the state transition process, which is the evolutionary pattern of the environment and the physical state of the robot changing and transitioning to the next state after the robot performs a certain action in the current state. This indicates a single-step reward.

[0090] Privileged information includes local terrain elevation maps and physical dynamic parameters in the sample simulation environment. The physical dynamic parameters include the body mass of the robot to be controlled, the friction coefficient of the terrain surface, the motor strength, and contact information.

[0091] like Figure 2 As shown, during the training process, privileged information from the sample simulation environment is input into the teacher pathway, where the privileged encoder in the teacher pathway receives physical dynamic parameters. Encode it as a teacher contextual latent variable Simultaneously, the scanning encoder receives local terrain elevation maps. The terrain ahead is geometrically encoded as a latent variable for the teacher. .

[0092] Simultaneously, the sample depth images are subjected to a credibility state assessment to obtain sample masking images and sample image patch credibility masks. In the student pathway, the initial first encoder fuses the sample masking images, sample image patch credibility masks, and sample ontology perception sequences to generate student terrain latent variables. The initial second encoder receives the sample ontology perception sequence and generates student contextual latent variables. .

[0093] Next, the data output from the student pathway or the data output from the teacher pathway is input into the interface of the initial policy network. The data output from the student pathway consists of student latent variables, which include student terrain latent variables and student context latent variables. The data output from the teacher pathway consists of teacher latent variables, which include teacher terrain latent variables and teacher context latent variables.

[0094] At each control moment Initial policy network They will all receive the body perception data at that moment. Terrain latent variables and adaptive contextual latent variables During training, the interface index is set. The initial policy network dynamically determines whether to receive teacher or student latent variables. When the simulation is in a privileged environment with a global view, the initial policy network selects teacher latent variables for learning; when simulating realistic observation conditions, the initial policy network selects student latent variables for learning. The control commands output by the initial policy network... It can include target commands for 16-dimensional actions, i.e. Each dimension of the motion corresponds to the target command of the robot's leg or hub joints, such as the target expected angle or target output torque for each joint, as shown in the formula below: ; The network parameters of the initial unified network are optimized based on the preset optimization objectives, as well as the distillation objective and the reinforcement learning objective.

[0095] For reinforcement learning objectives, control trajectories are generated based on the interaction between control commands and the sample simulation environment, and single-step rewards are obtained. Single-step rewards include forward velocity and yaw command tracking rewards, as well as a regularization term to ensure stable transition from simulation to reality and applicability to the hardware. The forward velocity and yaw command tracking rewards positively encourage the robot to follow the expected movement speed and direction; the regularization term penalizes excessive joint torque consumption, violent collisions, or unstable postures, thereby preventing damage to the real physical hardware during deployment and compensating for the physical differences between the simulation environment and the real world. A value function is configured solely for training. Receive the full privilege state, which is the full privilege state. It is a comprehensive feature variable formed by splicing and fusing the ontology perception data at a given time, the local terrain elevation map, the physical dynamic parameters, and the ontology perception sequence of the sample. The formula is as follows: ; Privileged information enables the value function to more accurately guide the optimization of network parameters in the initial unified network.

[0096] The sequence of control commands at multiple consecutive time steps arranged in chronological order constitutes the control trajectory. In each control trajectory generated based on the interaction between the control commands and the environment, the network parameters of the initial unified network are optimized using the Proximal Policy Optimization (PPO) algorithm.

[0097] In the reinforcement learning architecture, the initial policy network serves as the policy model for generating actions, and the value function serves as the value model for evaluating actions. Based on a scheduling mechanism that gradually switches the input ratio of student latent variables / teacher latent variables, the initial policy network and the value function are simultaneously optimized when the initial policy network is in different data input interface states. The optimization objective of maximizing the cumulative reward in reinforcement learning is as follows: ; in, It controls the sequence length of the trajectory. It is a discount factor.

[0098] For the distillation target, since the high-scan representation differs significantly in fidelity from the student representation composed of degraded vision and proprioception, the student simultaneously uses the following loss function. Learning Teacher Latent Variables: ; in, It is a stopping gradient operator used to freeze the parameter updates of the teacher path, so that the student terrain latent variable approximates the teacher terrain latent variable and the student context latent variable approximates the teacher context latent variable, while the parameters of the teacher path do not change due to the distillation loss. It should be noted that the autoencoder used to generate the image patch credibility mask is optimized independently from the initial policy network and the value network. The autoencoder is not updated in the reinforcement learning loss. The training samples of the autoencoder are image frames that have not been degraded or damaged and are randomly sampled from normal depth trajectories.

[0099] To ensure stable convergence of the model, the control trajectory is first generated using the data output from the teacher pathway. The student pathway learns the teacher's latent variables through truncated time backpropagation in all environments to quickly explore the terrain and form effective geometric representations. Subsequently, as training progresses, the proportion of student latent variables input to the initial policy network interface is gradually increased, while the proportion of teacher latent variables input is gradually decreased, until all inputs to the initial policy network are switched to student latent variables output from the student pathway. At this point, the scan encoder is frozen as the geometric learning target. This approach allows the initial policy network to gradually adapt to the student latent variables without forcing the teacher and student representations to be exactly the same.

[0100] After the network parameters are optimized, the teacher path is discarded, and the initial first encoder, initial second encoder and initial policy network retained in the initial unified network are used as the first encoder, second encoder and policy network used in actual deployment control, respectively. Unlike conventional motion perception, since the policy may encounter multiple visual degradations in a control trajectory, this process learns a unified network. When the vision is reliable, it is used to predict the terrain; otherwise, effective proprioceptive control is maintained.

[0101] The robot motion control method provided in this application optimizes network parameters by combining a teacher-student architecture with privileged information distillation and a dynamic scheduling mechanism, effectively improving the robot's robustness in the face of visual degradation.

[0102] The robot motion control method provided in this application integrates ontological perception sequence observation and reliable visual observation, including identifying and masking unreliable regions in depth images, and a student-centered progressive training scheme that guides the initial unified network from privileged geometric information in the simulation environment to a practically deployable student access interface during a reinforcement learning process.

[0103] To verify the technical effectiveness of the proposed method, the robot motion control device was deployed on a wheeled quadruped robot and tested in laboratory and outdoor environments. The results show that the proposed method can stably complete climbing, leaping, and traversing various real terrains under different degrees of visual degradation. The maximum climbing height is 0.8 m, which is twice the robot's standing height; the maximum ditch crossing width is 0.9 m, which is twice the robot's body length. The experiments demonstrate that the proposed method can maintain robust movement in complex outdoor environments with visual degradation and has practical deployment potential.

[0104] To verify the effectiveness of the proposed solution, we evaluated it from three aspects: (1) comparing the overall obstacle crossing performance with the open source baseline; (2) evaluating the robustness of the structured depth image under damage by removing the trusted state determination and the attention dynamic allocation mechanism through ablation experiments; and (3) evaluating the physical deployment performance of the robot under indoor and outdoor obstacle and occlusion conditions.

[0105] Experimental setup: The initial unified network was trained using training samples constructed in a parallel environment with 4096 parallel environments in the Isaac Gym simulator. Progressive courses sampled terrains including flat ground, platforms, ravines, and stairs. The initial unified network ran at 50 Hz, while depth image acquisition, image patch confidence state determination, and mask-based feature fusion all ran at 10 Hz. The robot to be controlled was a wheeled quadruped robot with a standing height of 0.40 m and a body length of 0.45 m.

[0106] The first step is to conduct simulation experiments: Comparison Methods and Baseline: Robustness was assessed under standard parkour conditions and deep damage conditions. Mean X-Displacement (MXD) represents the distance moved normalized to the forward route length, and the success rate represents the proportion of experimental tracks completed. Comparison methods included: Prior upper bound (Oracle): Directly provides teacher terrain latent variables and teacher context latent variables from the teacher pathway output as upper bounds for simulation performance.

[0107] Global Degradation Baseline: This is an extreme case setting used for comparative testing. Under this test condition, the trustworthy states of all image patches in the depth image are forcibly marked as untrustworthy. This is used to test and evaluate the backtracking control capability of the unified network trained in this application, which maintains stable robot motion solely by relying on the ontological perception sequence when all effective visual information is completely lost.

[0108] Remove trusted state determination: Cancel trusted state recognition, and degraded depth observations are directly input to the first encoder without trusted masking.

[0109] Remove dynamic attention allocation: Cancel the multi-head attention weight allocation based on image patch trusted masks and replace the mechanism with a simple multilayer perceptron (MLP) fusion module.

[0110] Binary Switch (0 / 1 Switch): Uses a frame-level binary switch to control the input depth image, without performing image-level confidence estimation. It uses the maximum reconstruction difference from the normal training depth image according to relevant technical thresholds, rather than a threshold calibrated on real occluded data.

[0111] DreamWaQ refers to the ontological perception baseline that infers terrain information through passive contact in scenarios with global visual degradation.

[0112] Concurrent Teacher-Student Reinforcement Learning (CTS): A single-stage parallel teacher-student baseline. During training, in each batch of training environments, half of the initial policy network is input using the teacher's latent variables, and the other half is input using the student's latent variables. There is no progressive scheduling or switching of latent variable weights.

[0113] Extreme Parkour: A visual parkour baseline for navigating challenging obstacles.

[0114] Progressive obstacle course evaluation: For three terrain types—ditches, steps, and stairs—ten similar obstacles were arranged in increasing difficulty to form a test route. The maximum ditch width was 0.90 m, the maximum step height was 0.80 m, and the maximum stair step height was 0.20 m. Under a forward command of 1 m / s, 512 trajectory segments were evaluated for each terrain type. All methods used the same protocol. MXD measured normalized forward progress, and success rate measured the proportion of trajectories that completed the entire route.

[0115] Figure 3 The graph shows a performance comparison between the robot motion control method provided in this application and other methods on a progressive obstacle course. Figure 3 The left side shows data under a scenario without global visual degradation, and the right side shows data under a scenario with global visual degradation. Figure 3 As shown, the performance of the proposed method is close to the prior upper bound using privileged information and outperforms the visual parkour baseline in all three routes. Both CTS and extreme parkour are affected by the difference between privileged representations and deployable representations. CTS requires the same policy network to adapt to mismatched teacher and student latent variable distributions simultaneously. Extreme parkour, on the other hand, transfers privileged policies to deployable visual observations through two-stage distillation. The progressive single-stage framework proposed in this application trains student pathways in all environments and then gradually switches the policy network interface from teacher latent variables to deployable student latent variables, thereby reducing this difference. Compared with parallel and two-stage baselines, this single-stage framework improves cross-terrain generalization ability and task completion rate.

[0116] Even after masking all image blocks, the overall performance of the proposed method is still higher than the DreamWaQ baseline, which relies solely on ontology perception, especially on steps and stairs. This indicates that the proposed method retains strong ontology perception backtracking capability. The other two methods without visual prediction were unable to cross the ditch, demonstrating that reliable terrain forward-looking information is crucial for take-off decisions. These results show that the proposed method retains both visual prediction capability and ontology perception backtracking capability.

[0117] Structured visual damage testing: Figure 4 These are schematic diagrams of various depth images provided in the embodiments of this application. Figure 4 It includes four subgraphs: (a), (b), (c), and (d). Figure 4 (a) shows a depth image with artifacts; Figure 4 (b) shows a depth image with bright patches; Figure 4 Image (c) shows a depth image with Gaussian noise; Figure 4 Image (d) shows a depth image with artifacts, bright spots, Gaussian noise, and contour and blur perturbations.

[0118] Figure 5 Comparison data of the robustness of the robot motion control method provided in this application embodiment with other methods under simulated deep damage, such as... Figure 5 As shown, among the four types of damage, the method of this application exhibits the best overall performance. When the depth image is slightly damaged, the performance of removing the credible state determination or removing the dynamic attention allocation is close to that of the complete method. However, the gap gradually widens as the damage worsens. Under severe combined damage, the average forward progress of the complete method of this application is similar to that of the two ablation methods, but the completion rate of the complete method is higher. This indicates that when effective depth evidence is scarce, the complete method can complete the route more stably. Under normal and local degradation conditions where the depth damage ratio does not exceed 50%, the robot controlled by the method of this application can climb a 0.8 m high platform and leap over a 0.9 m wide ditch. Under global degradation conditions with no usable visual input, the robot still maintains the same climbing ability.

[0119] The ablation results show that the roles of image patch confidence state determination and dynamic attention allocation are complementary. After removing confidence state determination, image patches with an unconfidential confidence state will enter the terrain latent variable. The high progress but low completion rate indicates that failures occur more often in the later stages of the route. After removing dynamic attention allocation, there is a lack of explicit mechanism to adjust the fusion process based on visual reliability. Therefore, masking alone cannot cope with high-intensity damage. Frame-level binary switching either retains the damaged area or discards the valid area. Extreme parkour also lacks explicit reliability processing. Both methods degrade significantly under severe Gaussian noise and combined damage. These results verify that the method provided in this application can improve the robot's motion robustness under various types of depth image degradation.

[0120] The following is a real robot experiment: Obstacle course experiments under normal visual conditions: Parkour experiments were conducted on platforms of increasing height and ditches of increasing width. Each condition was tested independently 20 times. As the difficulty of the scene increased, the method of this application maintained a high success rate and remained stable even under the most difficult setting, indicating that the method of this application has agile movement capabilities under normal visual conditions. The advantages of the method of this application are particularly obvious on high platforms and wide ditches, which are superior to traditional methods. This improvement comes from the progressive single-stage training architecture proposed in this application, which avoids the performance loss in two-stage algorithms.

[0121] Local and global visual degradation experiments: Local degradation is constructed based on typical deployment failures, moving branches and leaves to block the camera's field of view, causing the right half of the point cloud to become completely ineffective; reflection artifacts are generated on the tile and glass surfaces in the staircase experiment; severe near-field depth failure near the high platform will trigger whole-frame rejection; global degradation is achieved by completely obscuring the robot's depth camera with a box, preventing it from providing effective visual information.

[0122] In experiments with locally degraded platforms and ravines, the performance was only slightly lower than under normal visual conditions. Under global degradation, the strategy of this application can still maintain a strong ability to climb high platforms.

[0123] Based on the method provided in this application, a real robot was controlled. According to the robot's response results, it can be found that when encountering local interference such as foliage occlusion and reflection from the stair surface, the robot can accurately identify and mask image blocks in the corresponding area that are considered untrusted. When the robot encounters severe near-field depth data corruption near a high platform, the whole-frame error assessment will switch the entire depth image to an untrusted whole-frame masked state. When the depth camera is completely blocked by external objects such as cardboard boxes, all visual input will be rejected, that is, the image block trust mask indicates that all image blocks are untrusted.

[0124] Long-distance outdoor deployment: Figure 6 The outdoor deployment route map provided in this application embodiment, Figure 6 The paper demonstrates an outdoor deployment of approximately 500 meters for 12 minutes. The same policy network can traverse irregularly shaped rocks 0.65 meters high without retraining, and cope with tall grass, wet soil, rubber surfaces, stairs, and changing lighting conditions, all of which were not included in the training course. The robot completes the entire heterogeneous route without manual switching of the controller, demonstrating that the method of this application can maintain agile movement and generalize to complex outdoor environments with zero samples.

[0125] The robot motion control device provided in the embodiments of this application is described below. The robot motion control device described below can be referred to in correspondence with the robot motion control method described above.

[0126] Figure 7This is a schematic diagram of the structure of the robot motion control device provided in the embodiments of this application, as shown below. Figure 7 As shown, the device includes an acquisition module 710, a determination module 720, a first encoding module 730, a second encoding module 740, and a control module 750.

[0127] The acquisition module 710 is used to acquire a depth image of the environment in which the robot to be controlled is currently located, as well as a body perception sequence within a preset time period. The determination module 720 is used to determine the trust status of each image block in the depth image and to mask the image blocks with a trust status of untrust in the depth image, thereby obtaining the image block trust mask of the masked image and the depth image; The first encoding module 730 is used to input the masked image, the body-aware sequence and the image patch trusted mask into the first encoder to obtain the terrain latent variables output by the first encoder; The second encoding module 740 is used to input the ontology-aware sequence into the second encoder to obtain the context latent variables output by the second encoder; The control module 750 is used to input terrain latent variables, context latent variables and the current body perception data into the policy network to obtain the control instructions output by the policy network. The control instructions are used to control the robot to be controlled to move. The first encoder, the second encoder, and the policy network are trained based on a teacher-student architecture.

[0128] Specifically, according to embodiments of this application, any number of these modules can be combined into one module, or any one of these modules can be split into multiple modules.

[0129] Alternatively, at least some of the functionality of one or more of these modules can be combined with at least some of the functionality of other modules and implemented in a single module.

[0130] According to embodiments of this application, at least one of these modules can be implemented at least partially as hardware circuitry, such as a Field Programmable Gate Array (FPGA), Programmable Logic Array (PLA), System-on-a-Chip, System-on-a-Substrate, System-on-Package, Application Specific Integrated Circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or as any one or a suitable combination of software, hardware, and firmware implementations.

[0131] Alternatively, at least one of these modules may be implemented at least partially as a computer program module that, when run, can perform a corresponding function.

[0132] It should be noted that the robot motion control device provided in this application embodiment can implement all the method steps implemented in the above robot motion control method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0133] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 8 As shown, the electronic device may include a processor 88, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 88, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 88 can call the computer program in the memory 830 to execute the above-described method.

[0134] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional modules and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0135] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the methods provided in the above embodiments.

[0136] On the other hand, embodiments of this application also provide a non-transitory computer-readable storage medium storing a computer program for causing a processor to execute the methods provided in the above embodiments.

[0137] The non-transitory computer-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0138] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A robot motion control method, characterized in that, include: Acquire depth images of the environment in which the robot to be controlled is currently located, as well as the body perception sequence within a preset time period; Determine the trust status of each image block in the depth image, and mask the image blocks with the trust status as untrusted in the depth image to obtain a masking image and a trust mask for the image blocks in the depth image; The masked image, the ontology-aware sequence, and the image patch trusted mask are input into the first encoder to obtain the terrain latent variables output by the first encoder; The ontology-aware sequence is input into the second encoder to obtain the context latent variables output by the second encoder; The terrain latent variables, the context latent variables, and the current body perception data are input into the policy network to obtain the control instructions output by the policy network. The control instructions are used to control the robot to be controlled to move. The first encoder, the second encoder, and the policy network are trained based on a teacher-student architecture.

2. The robot motion control method according to claim 1, characterized in that, Before acquiring the depth image of the environment in which the robot to be controlled is currently located, the method further includes: A reference image set is constructed based on multiple frames of depth images acquired by the robot under control when its vision is unobstructed. Each reference image in the reference image set is divided into multiple reference image blocks, and reference features of each reference image block are extracted; The reference features of the reference image blocks at the same image location in the reference image set are aggregated to obtain a block memory library; Determining the confidence state of each image patch in the depth image includes: The trusted state of each image block is determined based on the block memory.

3. The robot motion control method according to claim 2, characterized in that, Determining the trust status of each image block based on the block memory includes: Extract image block features for each of the image blocks; Determine the feature distance between each image block feature and the reference feature corresponding to the image block feature in the block memory; The image block whose feature distance is less than a preset distance threshold is marked as trustworthy; Image blocks whose feature distance is greater than or equal to the preset distance threshold are marked as untrustworthy.

4. The robot motion control method according to claim 1, characterized in that, Determining the confidence state of each image patch in the depth image includes: The depth image is reconstructed to obtain a reconstructed image; Obtain the reconstruction error between the reconstructed image and the depth image; If the reconstruction error is greater than a preset whole frame error threshold, the trust status of the depth image is marked as untrusted, so that the trust mask of the image block indicates that the trust status of all image blocks is untrusted. When the reconstruction error is less than or equal to the preset whole frame error threshold, the confidence state of each image block is determined based on the block memory, which stores the reference features of each image block.

5. The robot motion control method according to claim 1, characterized in that, The step of inputting the masked image, the ontology-aware sequence, and the image patch trusted mask into the first encoder to obtain the terrain latent variables output by the first encoder includes: The masking image, the ontology-aware sequence, and the image patch trusted mask are input into the first encoder, and the first encoder is controlled to perform the following steps: The masked image and the image block trusted mask are encoded into block visual features, and the ontology-aware sequence is encoded into motion features; The query vector is determined based on the motion features and the image patch confidence mask; The segmented visual features are concatenated with the motion features to obtain multimodal features; The query vector is controlled to perform multi-head attention weight allocation on the multimodal features based on the image patch trust mask; The multimodal features are fused based on the assigned attention weights to obtain the terrain latent variables.

6. The robot motion control method according to claim 5, characterized in that, The control of the query vector to perform multi-head attention weight allocation on the multimodal features based on the image patch trust mask includes: When the image patch trust mask indicates the existence of an image patch with a trust status, the query vector is controlled to allocate attention weights to the block visual features and the motion features in the multimodal features; When the image patch trust mask indicates that the trust status of all image patches is untrusted, the query vector is controlled to only allocate attention weights to the motion features in the multimodal features.

7. The robot motion control method according to claim 1, characterized in that, The training of the first encoder, the second encoder, and the policy network includes the following steps: Construct an initial unified network, which includes a teacher path, a student path, and an initial policy network shared by the teacher path and the student path, wherein the student path includes an initial first encoder and an initial second encoder; The privileged information in the sample simulation environment is input into the teacher pathway to obtain the teacher terrain latent variable and teacher context latent variable output by the teacher pathway. The privileged information includes the local terrain elevation map and physical dynamic parameters in the sample simulation environment. The sample masking image, the sample image patch trusted mask, and the sample ontology perception sequence are input into the student pathway to obtain the student terrain latent variable and the student context latent variable output by the student pathway. The data output from the student channel or the data output from the teacher channel is input into the initial policy network to obtain the control instructions output by the initial policy network. The network parameters of the initial unified network are optimized based on the data output from the student channel, the data output from the teacher channel, the control instructions, and the preset optimization objectives. After the network parameters are optimized, the teacher path is discarded, and the initial first encoder, the initial second encoder, and the initial policy network retained in the initial unified network are respectively used as the first encoder, the second encoder, and the policy network.

8. The robot motion control method according to claim 7, characterized in that, The optimization objectives include a distillation objective and a reinforcement learning objective. The optimization of the network parameters of the initial unified network based on the data output from the student pathway, the data output from the teacher pathway, the control instructions, and the preset optimization objectives includes: The network parameters of the initial unified network are optimized with the goal of approximating the teacher's terrain latent variable with the student's terrain latent variable and approximating the teacher's context latent variable with the student's context latent variable. Based on the interaction between the control command and the sample simulation environment, a control trajectory is generated, and the single-step reward corresponding to the control trajectory is obtained, so as to maximize the cumulative reward and optimize the network parameters for the reinforcement learning objective.

9. The robot motion control method according to claim 7, characterized in that, Also includes: Gradually increase the proportion of data output from the student channel to be input into the initial policy network, and gradually decrease the proportion of data output from the teacher channel to be input into the initial policy network, until all inputs to the initial policy network are switched to data output from the student channel.

10. A robot motion control device, characterized in that, include: The acquisition module is used to acquire depth images of the environment in which the robot to be controlled is currently located, as well as the body perception sequence within a preset time period. The determination module is used to determine the trust status of each image block in the depth image, and to mask the image blocks with the trust status as untrusted in the depth image to obtain a masking image and a trust mask for the image blocks in the depth image; The first encoding module is used to input the masking image, the body-aware sequence and the image patch trusted mask into the first encoder to obtain the terrain latent variable output by the first encoder; The second encoding module is used to input the ontology-aware sequence into the second encoder to obtain the context latent variables output by the second encoder; The control module is used to input the terrain latent variables, the context latent variables, and the current body perception data into the policy network to obtain the control instructions output by the policy network. The control instructions are used to control the robot to be controlled to move. The first encoder, the second encoder, and the policy network are trained based on a teacher-student architecture.

11. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the robot motion control method according to any one of claims 1 to 9 through the computer program.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the robot motion control method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot motion control method as described in any one of claims 1 to 9.