Robot motion control method, robot motion control device, and quadruped robot

Abstract

The application relates to a robot motion control method, a robot motion control device and a quadruped robot. The method comprises the following steps: extracting a motion feature vector of the robot according to a state vector sequence of the robot; extracting a visual feature vector according to a depth map sequence of the environment around the robot; obtaining a pre-constructed weight generation network, inputting the visual feature vector into the weight generation network, and outputting a weight coefficient of the visual feature vector; the weight coefficient is positively correlated with the complexity of the environment around the robot and the reliability of the visual feature vector; the weight generation network is a neural network representing the mapping relationship between the visual feature vector and the weight coefficient; performing weighted operation on the visual feature vector according to the weight coefficient, and combining the weighted visual feature vector with the motion feature vector to obtain a fusion feature vector; and converting the fusion feature vector into a motor instruction of the robot. The method can improve the motion stability of the quadruped robot.

Description

Technical Field

[0001] This application relates to the field of robot control, and in particular to robot motion control methods, robot motion control devices, and quadruped robots. Background Technology

[0002] Quadruped robots are gradually replacing human labor in safety inspections in industrial environments. In flammable and explosive environments such as petrochemical plants and coal mines, due to explosion-proof requirements, quadruped robots have thick shells, resulting in a mass and moment of inertia that are typically more than three times that of ordinary small and medium-sized robots. This leads to slower dynamic response and reduced adaptability. In these scenarios, quadruped robots are at greater risk of accidental falls in complex terrain. In situations with extremely high safety requirements, a fall could also pose a safety hazard to surrounding flammable and explosive areas.

[0003] In related technologies, robot motion control has evolved from traditional model control to data-driven reinforcement learning, resulting in significant improvements in stability under normal conditions. However, for quadruped robots with large mass and high rotational inertia, there is currently no method to ensure safe and stable operation on all terrains. No effective solution has yet been proposed to address the problem of low motion stability in quadruped robots within these technologies. Summary of the Invention

[0004] Therefore, it is necessary to provide a robot motion control method, a robot motion control device, and a quadruped robot that can solve the problem of low motion stability of quadruped robots, in order to address the above-mentioned technical problems.

[0005] Firstly, this embodiment provides a robot motion control method, the method comprising:

[0006] Collect the state vectors of the robot at the current moment and at historical moments to obtain a state vector sequence, and extract the motion feature vector of the robot at the current moment based on the state vector sequence;

[0007] Acquire and stack multiple frames of depth maps of the robot's surrounding environment at the current time and historical time to obtain a depth map sequence, and extract the visual feature vector of the robot's surrounding environment based on the depth map sequence;

[0008] A pre-constructed weight generation network is obtained, the visual feature vector is input into the weight generation network, and the weight coefficients of the visual feature vector are output; wherein, the weight coefficients are positively correlated with the complexity of the environment around the robot and the reliability of the visual feature vector, and the weight generation network is a neural network that represents the mapping relationship between the visual feature vector and the weight coefficients.

[0009] The visual feature vector is weighted according to the weight coefficients, and the weighted visual feature vector is combined with the motion feature vector to obtain the fused feature vector.

[0010] The fused feature vector is converted into motor commands for controlling the robot.

[0011] In some embodiments, the weight generation network includes a first neural network layer, a second neural network layer, and a third neural network layer; wherein,

[0012] The first neural network layer receives the visual feature vector;

[0013] The second neural network layer performs a linear transformation on the visual feature vector and uses a first activation function to perform a nonlinear transformation on the linearly transformed visual feature vector to extract the nonlinear features in the visual feature vector that correspond to the environmental complexity and the reliability of the visual feature vector.

[0014] The third neural network layer performs a linear transformation on the nonlinear features to obtain feature scalar values, and uses a second activation function to map the feature scalar values ​​to the target numerical range to obtain the weight coefficients.

[0015] In some embodiments, combining the weighted visual feature vector with the motion feature vector includes:

[0016] By concatenating the weighted visual feature vector and the motion feature vector, a joint feature vector is obtained;

[0017] A pre-constructed fusion mapping network is obtained, and the joint feature vector is input into the fusion mapping network to obtain the fusion feature vector; wherein, the fusion feature vector is a vector obtained by integrating the robot's visual feature vector and motion feature vector in the joint feature vector, and the fusion mapping network is a neural network that represents the mapping relationship between the joint feature vector and the fusion feature vector.

[0018] In some embodiments, the fusion mapping network includes a fourth neural network layer, a fifth neural network layer, and a sixth neural network layer; wherein,

[0019] The fourth neural network layer receives the joint feature vector;

[0020] The fifth neural network layer performs a linear transformation on the joint feature vector and uses a third activation function to perform a nonlinear transformation on the linearly transformed joint feature vector, so as to integrate the robot's visual feature vector and motion feature vector in the joint feature vector through the nonlinear transformation and output the first initial feature vector.

[0021] The sixth neural network layer maps the first initial feature vector to a specified dimension to obtain the fused feature vector.

[0022] In some embodiments, the motion feature vector of the robot at the current moment is extracted based on the state vector sequence, including:

[0023] Obtain a pre-built context-aided estimator;

[0024] The state vector sequence is input into the context-aided estimator, which extracts the context feature vector of the state vector sequence. The context-aided estimator is a neural network that represents the mapping relationship between the state vector sequence and the context feature vector. The context feature vector is used to indicate the robot's body motion features and visual terrain features.

[0025] The motion feature vector is obtained based on the context feature vector and the state vector at the current moment.

[0026] In some embodiments, visual feature vectors of the robot's surrounding environment are extracted from the depth map sequence, including:

[0027] The depth map sequence is convolved using a preset convolution kernel to obtain a second initial feature vector;

[0028] The second initial feature vector is downsampled to obtain a three-dimensional feature map;

[0029] The three-dimensional feature map is flattened into a one-dimensional vector, and the embedding dimension of the one-dimensional vector is compressed and mapped to a specified embedding dimension to obtain the visual feature vector.

[0030] In some embodiments, converting the fused feature vector into motor commands for controlling the robot includes:

[0031] The fused feature vector is converted into a joint motion offset vector;

[0032] The target joint position is obtained based on the default joint position and the joint motion offset vector;

[0033] Based on the difference between the target joint position and the current joint position, motor commands for moving the robot's joints are generated.

[0034] In some embodiments, the method further includes:

[0035] Reward items are constructed based on the robot's operating state in response to the motor commands. The reward items include at least one of the following: a speed tracking reward item based on the actual base speed and the target base speed; an energy efficiency reward item based on the actual joint torque and the target joint torque; a motion smoothness reward item based on the actual joint acceleration and the target joint acceleration; and a visual dependence reward item based on the weighting coefficient.

[0036] A reward function is constructed based on the reward item, and reinforcement learning processing is performed on the weight generation network, fusion mapping network, context-aided estimator, and policy neural network based on the reward function; wherein, the fusion mapping network is used to combine the adjusted visual feature vector and the motion feature vector to obtain a fused feature vector; the context-aided estimator is used to extract the motion feature vector of the robot at the current moment based on the state vector sequence, and the policy neural network is used to convert the fused feature vector into motor commands for controlling the robot.

[0037] Secondly, this embodiment provides a robot motion control device, the device comprising:

[0038] The motion feature acquisition module is used to collect the state vectors of the robot at the current moment and at historical moments, obtain a state vector sequence, and extract the motion feature vector of the robot at the current moment based on the state vector sequence;

[0039] The visual feature acquisition module is used to acquire and stack multiple frames of depth maps of the robot's surrounding environment at the current time and historical time to obtain a depth map sequence, and extract the visual feature vector of the robot's surrounding environment based on the depth map sequence.

[0040] The weight coefficient generation module is used to obtain a pre-constructed weight generation network, input the visual feature vector into the weight generation network, and output the weight coefficients of the visual feature vector; wherein, the weight coefficients are positively correlated with the complexity of the environment surrounding the robot and the reliability of the visual feature vector, and the weight generation network is a neural network that represents the mapping relationship between the visual feature vector and the weight coefficients.

[0041] The feature fusion module is used to perform a weighted operation on the visual feature vector according to the weight coefficient, and combine the weighted visual feature vector with the motion feature vector to obtain a fused feature vector;

[0042] The instruction generation module is used to convert the fused feature vector into motor instructions for controlling the robot.

[0043] Thirdly, this embodiment provides a quadruped robot, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the robot control method described in the first aspect above.

[0044] The aforementioned robot motion control method, robot motion control device, and quadruped robot extract motion feature vectors based on state vector sequences and visual feature vectors based on depth map sequences at the current and historical times. Weighting coefficients are output based on the reliability of the visual feature vectors displayed in the depth map sequences and the environmental complexity. These weighting coefficients adjust the fusion ratio between the visual feature vectors and motion feature vectors, achieving the effect of enabling visual guidance when the surrounding environment is complex and the visual feature vectors are reliable, and disabling visual guidance when the surrounding environment is complex and the visual feature vectors are unreliable. This improves the accuracy of robot motor commands generated based on fused feature vectors, enables adaptive and safe control of robot motion, and solves the problem of low motion stability in quadruped robots. Attached Figure Description

[0045] Figure 1 A flowchart illustrating an embodiment of the robot motion control method provided in this application;

[0046] Figure 2 A schematic diagram of the point cloud to depth map conversion process provided in this application;

[0047] Figure 3 A schematic diagram of the context-aided estimator provided in this application;

[0048] Figure 4 A flowchart illustrating Embodiment 2 of the robot motion control method provided in this application is shown.

[0049] Figure 5 This is a structural block diagram of the robot motion control device provided in this application;

[0050] Figure 6 This is a diagram showing the internal structure of the controller for the robot provided in this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0053] In one embodiment, such as Figure 1 The diagram shows a flowchart of a robot motion control method according to embodiment one, including the following steps:

[0054] Step 102: Collect the robot's state vectors at the current moment and at historical moments to obtain a state vector sequence, and extract the robot's motion feature vector at the current moment based on the state vector sequence.

[0055] The state vector of the acquisition robot includes at least one of the following: joint angles. Joint velocity Base posture (Quaternions), Gravity Projection Speed ​​command and joint commands For example, the state vectors of the past S time steps are collected to obtain a state vector sequence.

[0056] The motion feature vector includes robot body motion feature vectors that cannot be directly acquired at the current moment, such as robot body mass and forward speed vectors; the motion feature vector also includes visual terrain feature vectors; optionally, the motion feature vector may also include the acquired robot state vectors. Optionally, a nonlinear coding network is used to convert the high-dimensional state vector sequence to a low-dimensional space to achieve feature extraction of the state vector sequence, thus obtaining the motion feature vector at the current moment; in this process, the nonlinear coding network can use structures such as temporal convolution or recurrent neural networks to capture the body motion features and visual terrain features during the robot's state change process.

[0057] Step 104: Acquire and stack multiple frames of depth maps of the robot's surrounding environment at the current time and historical time to obtain a depth map sequence, and extract the visual feature vector of the robot's surrounding environment based on the depth map sequence.

[0058] The spatial range represented by the depth map can be a local environmental region in front of the robot, or it can refer to any other environmental region surrounding the robot. For example, the depth maps of the current moment and the past S-1 moments are stacked along the channel dimension to construct a multi-channel depth tensor. This yields a depth map sequence. The depth map sequence can capture the dynamic changes in terrain. Furthermore, using a CNN (Convolutional Neural Network) or a visual Transformer architecture, the differential features between frames in the depth map sequence can be extracted into visual feature vectors.

[0059] In the process of acquiring depth maps, traditional depth cameras can be used to directly collect and generate depth maps. However, considering that depth maps acquired by traditional cameras are highly susceptible to the influence of light, it would be very dangerous for robots to conduct inspections at night. Therefore, it is also possible to acquire environmental point clouds using radar and then convert the environmental point clouds into depth maps.

[0060] For example, after acquiring local environmental point clouds through radar, they are converted into depth maps with a resolution of 64×64. Figure 2 A flowchart illustrating the point cloud to depth map conversion process is provided, such as... Figure 2 As shown, after acquiring the radar point cloud, it is necessary to project it according to the depth camera characteristic parameters used in the target depth map, and then normalize the depth map. The specific process is as follows:

[0061] Radar acquires point clouds. The radar's point cloud output includes N three-dimensional spatial points. Determine camera characteristic parameters, including field of view. And the resolution (H, W), and then the origin of the pixel coordinates is obtained. and focal length :

[0062]

[0063]

[0064]

[0065]

[0066] Obtain the camera's characteristic parameter matrix K:

[0067]

[0068] Convert the point cloud into a depth map; specifically, project the point cloud coordinates (x, y, z) onto depth coordinates using a feature parameter matrix:

[0069]

[0070] Depth map normalization; specifically, normalizing coordinates and distances:

[0071]

[0072] in, Let (u,v) be the homogeneous pixel coordinates obtained after projection, (u,v) be the actual pixel coordinates, and w be the actual depth information of each pixel. It is the nearest depth threshold. It is the farthest depth threshold.

[0073] Step 106: Obtain the pre-constructed weight generation network, input the visual feature vector into the weight generation network, and output the weight coefficients of the visual feature vector; wherein, the weight coefficients are positively correlated with the complexity of the environment around the robot and the reliability of the visual feature vector, and the weight generation network is a neural network that represents the mapping relationship between the visual feature vector and the weight coefficients.

[0074] When the visual feature vector reflects a complex terrain with numerous cluttered obstacles and discontinuous terrain around the robot, the environmental complexity is judged to be high. When the visual feature vector reflects a clear image and an environment free of dust obstruction, the reliability of the visual feature vector is judged to be high. The higher the environmental complexity, the higher the reliability of the visual feature vector, and the larger the weight coefficient. When the visual feature vector reflects a continuous terrain around the robot without numerous obstacles, the environmental complexity is judged to be low. When the visual feature vector reflects excessive noise in the depth map and dust obstruction in the environment, the reliability of the visual feature vector can be judged to be low. The lower the environmental complexity, the lower the reliability of the visual feature vector, and the smaller the weight coefficient.

[0075] For example, the initial weight generation network can be trained using a labeled training sample set, where the labels are pre-annotated weight coefficients that match the environmental complexity and the reliability of the visual feature vectors. Through training, the weight generation network learns the mapping relationship from the feature space to the weight space, and quantizes the visual feature vectors into weight coefficients through weight matrix transformations and nonlinear activation functions within the neural network. Specific implementation methods for neural network training can be found in related technical documents and are not limited here.

[0076] Step 108: Perform weighted operations on the visual feature vectors according to the weight coefficients, and combine the weighted visual feature vectors with the motion feature vectors to obtain the fused feature vector.

[0077] Optionally, the visual feature vector is multiplied by a weighting coefficient, thereby adjusting the proportion of the visual feature vector in the fused feature vector through the weighting coefficient. Methods for obtaining the fused feature vector include: concatenating the weighted visual feature vector and the motion feature vector along the channel dimension; or, using a fully connected layer of a neural network to perform feature interaction between the visual feature vector and the motion feature vector.

[0078] Step 110: Convert the fused feature vector into motor commands for controlling the robot.

[0079] In this process, the fused feature vector is mapped to the target joint position or torque signal through an end-to-end policy neural network. The underlying driver then calculates the current and phase required for the motor to reach the target joint position or satisfy the torque signal, thereby realizing real-time driving of the robot actuator.

[0080] In the aforementioned robot motion control method, motion feature vectors are extracted from the state vector sequence, and visual feature vectors are extracted from the depth map sequence. Weighting coefficients are obtained based on the reliability of the visual feature vectors and the environmental complexity. These weighting coefficients adjust the fusion ratio between the visual and motion feature vectors, achieving the effect of enabling visual guidance when the surrounding environment is complex and the visual feature vectors are reliable, and disabling visual guidance when the surrounding environment is simple and the visual feature vectors are unreliable. This avoids the situation where, when controlling a robot based on proprioception, it cannot plan its landing points in advance when facing discontinuous terrain (such as deep ditches, high platforms, and cliffs), making it prone to missteps, falls, or getting stuck; and when controlling a robot based on visual perception, the robot's output movements oscillate frequently when noisy visual perception data is present, thus improving the motion stability of the quadruped robot.

[0081] In one embodiment, the weight generation network includes a first neural network layer, a second neural network layer, and a third neural network layer; wherein, the first neural network layer receives a visual feature vector; the second neural network layer performs a linear transformation on the visual feature vector and uses a first activation function to perform a nonlinear transformation on the linearly transformed visual feature vector to extract nonlinear features in the visual feature vector corresponding to environmental complexity and the reliability of the visual feature vector; the third neural network layer performs a linear transformation on the nonlinear features to obtain feature scalar values, and uses a second activation function to map the feature scalar values ​​to a target numerical range to obtain weight coefficients.

[0082] Optionally, the weight generation network can be viewed as a weight generator comprising a three-layer fully connected neural network. The first neural network layer corresponds to the input layer, which receives a visual feature vector with a dimension of 128. The second neural network layer corresponds to the hidden layer, containing 32 neurons, which performs a linear transformation on the visual feature vector and then a non-linear transformation using the ReLU activation function to extract non-linear features. The third neural network layer corresponds to the output layer, containing one neuron, which outputs a single scalar value through a linear transformation. This value is then mapped to the (0,1) interval through the Sigmoid activation function to generate the final weight coefficients.

[0083] Understandably, the number of neurons and the activation functions used in each layer of a neural network can be modified according to the specific application scenario or the required network depth. For example, in addition to using ReLU as the first activation function and Sigmoid as the second activation function, activation functions with similar functions, such as Tanh and Softplus, can also be used to adapt to different nonlinear mapping requirements or output ranges.

[0084] For example, the weight generator employs a lightweight MLP (Multi-Layer Perceptron) structure, receiving only visual feature vectors. Output feature scalar values After passing through the Sigmoid activation function The mapping is as follows:

[0085]

[0086]

[0087] The weight generator outputs a scalar weight coefficient. , A value of 0 indicates low environmental complexity and unreliable visual features, suggesting preparation for visual shielding. A value of 1 indicates a highly complex environment, where visual features are critical and reliable, and the visual features are ready to be activated.

[0088] In this embodiment, the semantics of the environment are understood based on the depth map features of the robot's surrounding environment, and the importance of visual features in the network input is dynamically determined, thereby adaptively adjusting the contribution of visual information.

[0089] In one embodiment, the combination of the weighted visual feature vector and the motion feature vector includes:

[0090] (1) By concatenating the weighted visual feature vector and the motion feature vector, a joint feature vector is obtained;

[0091] Specifically, using weighting coefficients The visual features are scaled to obtain the weighted modulated visual features. ; When the value is 0, the logical cutting off of visual information is achieved. Setting it to 1 fully enables visual information. The motion feature vector... With weighted modulated visual features The features are concatenated along the channel dimension to form a joint feature vector. Since the physical meanings of the robot's motion feature vector and visual feature vector are different, the integrity of the information in both the visual feature vector and the motion feature vector can be preserved through channel splicing.

[0092] (2) Obtain the pre-constructed fusion mapping network, input the joint feature vector into the fusion mapping network, and map to obtain the fusion feature vector; where the fusion feature vector is the vector obtained by integrating the robot visual feature vector and motion feature vector in the joint feature vector, and the fusion mapping network is a neural network that represents the mapping relationship between the joint feature vector and the fusion feature vector.

[0093] Optionally, The input is fed into a fusion mapping network, which remaps the high-dimensional joint feature vector to a fusion feature vector of a specified dimension. .

[0094] The fusion mapping network can be trained through end-to-end reinforcement learning. During the training phase, the joint feature vector is regarded as the environment state in reinforcement learning, and the fused feature vector is regarded as the decision action in reinforcement learning. The cumulative reward is obtained based on the stability of the robot's response to the fused feature vector, and the weight parameters of the fusion mapping network itself are optimized based on this reward. This allows the fusion mapping network to extract the robot's motion state and visual terrain information from the joint feature vector, that is, to achieve the fusion of visual feature vectors and motion feature vectors, thereby accurately outputting the corresponding fused feature vector.

[0095] Alternatively, the fusion mapping network can be viewed as a feature fusionist comprising a three-layer fully connected neural network, with weight coefficients... When the value is 0, the fusion mapping network relies only on Partial generation strategy, in When the value is 1, the fusion mapping network comprehensively utilizes both ontological motion features and visual terrain features. For ease of understanding, during the training process of the fusion mapping network: when the environment is flat or other continuous terrain and... When the value is 0, the visual feature components approach zero; at this time, the backpropagation gradient of the fusion mapping network mainly updates the... The relevant weight parameters allow the network to adapt to a pure ontology-based control strategy. This is especially important when the environment is complex, discontinuous terrain. When the value is 1, the visual feature components are fully input, and the network simultaneously utilizes... and The network optimizes its strategy by using a joint distribution of data to generate actions that overcome obstacles. This adaptive capability is not based on pre-defined rules, but rather on the spontaneous nature of the loss function minimized by the network on diverse terrain datasets.

[0096] Furthermore, in one embodiment, the fusion mapping network includes a fourth neural network layer, a fifth neural network layer, and a sixth neural network layer; wherein, the fourth neural network layer receives a joint feature vector; the fifth neural network layer performs a linear transformation on the joint feature vector and uses a third activation function to perform a nonlinear transformation on the linearly transformed joint feature vector, so as to integrate the robot visual feature vector and motion feature vector in the joint feature vector through nonlinear transformation, and output a first initial feature vector; the sixth neural network layer maps the first initial feature vector to a specified dimension to obtain a fused feature vector.

[0097] For example, the fourth neural network layer corresponds to the input layer, which is used to receive the concatenated joint feature vector. The fifth neural network layer, corresponding to the hidden layer, contains 256 neurons. It performs a linear transformation on the joint feature vector and then uses the ReLU activation function to non-linearly integrate the robot's visual and motion feature vectors, outputting the first initial feature vector. The sixth neural network layer, corresponding to the output layer, is used to map and compress the feature dimensions. Taking a 128-dimensional visual feature vector and a 65-dimensional motion feature vector as an example, the concatenated joint feature vector has a dimension of 193. This dimension can be compressed and mapped into a 128-dimensional fused feature vector using the sixth neural network layer. .

[0098] Here, 128 dimensions are the specified dimension, and the ReLU activation function is the third activation function. It should be understood that the number of neurons in the above neural network layer is merely an example and not a limitation. The specified dimension can be adjusted to other values ​​according to the processing requirements of subsequent motor instruction generation. Depending on the actual application requirements, the nonlinear mapping interval corresponding to the first initial feature vector can also be adjusted. Correspondingly, in addition to the ReLU activation function, the third activation function can also be set to other activation functions such as Sigmoid and Tanh.

[0099] In this embodiment, a fusion mapping network is used to achieve deep fusion of visual feature vectors and motion feature vectors, and the joint feature vector is mapped to a fusion feature vector of a specified dimension. This vector implicitly contains the robot's body feature state and surrounding environment information, thus obtaining the standard vector input required to generate motor commands.

[0100] In one embodiment, extracting the robot's motion feature vector at the current moment based on the state vector sequence includes: obtaining a pre-built context-aided estimator; inputting the state vector sequence into the context-aided estimator, and having the context-aided estimator extract the context feature vector of the state vector sequence; the context-aided estimator is a neural network that characterizes the mapping relationship between the state vector sequence and the context feature vector; the context feature vector is used to indicate the robot's ontological motion features and visual terrain features; and obtaining the motion feature vector based on the context feature vector and the state vector at the current moment.

[0101] The context-aided estimator, also known as the context-assisted estimator, is a neural network with a shared encoder architecture. It implicitly encodes state information that the robot cannot directly obtain based on historical state vector sequences, thus encoding a context feature vector. Furthermore, the context-aided estimator can output the robot's state information for future moments based on the context feature vector. Specifically, the context feature vector includes at least one of the following: an estimated value of the robot's motion velocity, an estimated value of its mass, and a terrain latent vector associated with the terrain where the robot is located. The motion feature vector includes the robot's state vector at the current moment and the robot's context feature vector at the current moment.

[0102] After training, the context-aided estimator can estimate robot state and terrain information. The training process is as follows: An initial context-aided estimator and training set are obtained; the state vector sequence from the training set is input into the initial context-aided estimator to obtain the predicted context feature vector and the future observations output based on the predicted context feature vector; a hybrid loss function is constructed based on the predicted context feature vector, the observations, the true context feature vector from the training set, and the true motion feature vector from the future time; the parameters of the context-aided estimator are adjusted according to the hybrid loss function to obtain the trained context-aided estimator. Based on the trained context-aided estimator, accurate prediction of the context feature vector can be achieved.

[0103] Specifically, the predicted context feature vector includes the robot's estimated motion velocity, estimated mass, and terrain latent vector. A target loss function is constructed based on the predicted context feature vector and the real context feature vectors in the training set. This includes: a velocity estimation loss function based on the estimated motion velocity and the robot's actual velocity; a mass estimation loss function based on the estimated mass and the robot's actual mass; and a terrain estimation loss function based on the robot's current state vector sequence, observations, the posterior distribution of the terrain latent vector, and the prior distribution of the terrain latent vector. The velocity estimation loss function, the mass estimation loss function, and the terrain estimation loss function are then integrated to obtain a hybrid loss function. The above loss functions are calculated using a context-aided estimator, thus enabling the estimation of relevant robot motion feature vectors solely through temporal ontological perception.

[0104] For ease of understanding, Figure 3 A schematic diagram of a context-aided estimator is provided, such as... Figure 3 As shown, the context-aided estimator possesses body mass estimation, velocity estimation, and autoencoding capabilities. Specifically, during training, it uses the state vectors from the past S time steps... The state vector sequence is constructed. And input the context-aided estimator, through multiple layers of mapping, to transform the state vector sequence The feature dimension is gradually compressed from 225 to 128, 64, and converges to 20, outputting a context feature vector, including: a 3×1 estimated motion velocity value. 1×1 mass estimate and 16×1 implicit vector Subsequently, the context feature vector is reconstructed through network layers, expanding its feature dimension from 20 to 64 and 128, outputting a 48-dimensional value corresponding to the robot's observation at the next time step. .

[0105] For example, the target loss function during network updates is calculated as follows:

[0106]

[0107] in, It is the hybrid loss function of the context-aided estimator. and These are the loss functions for velocity estimation, mass estimation, and terrain estimation, respectively. Each loss function is calculated using the following formula:

[0108]

[0109]

[0110]

[0111] This is the robot's actual speed value. This is the robot's actual mass value. These are the observations at the next time step reconstructed by the context-aided estimator. It is a local observation of a given time series. At that time, latent vector The posterior distribution, The latent vector parameterized by Gaussian distribution The prior distribution, It is a variational autoencoder in the context-aided estimator.

[0112] Obtain the trained context-aided estimator, input the state vector sequence sampled in step 102 above into the context-aided estimation network, and output the robot's motion feature vector. .

[0113] In this embodiment, a context-assisted estimator generates a state vector sequence containing potential feature representations of the robot's body state and the surrounding terrain, thereby obtaining a context feature vector. Based on the context feature vector and the current state vector, an accurate and comprehensive motion feature vector is obtained.

[0114] In one embodiment, the method of extracting visual feature vectors of the robot's surrounding environment based on a depth map sequence includes: convolving the depth map sequence with a preset convolution kernel to obtain a second initial feature vector; downsampling the second initial feature vector to obtain a three-dimensional feature map; flattening the three-dimensional feature map into a one-dimensional vector, and compressing and mapping the embedding dimension of the one-dimensional vector to a specified embedding dimension to obtain a visual feature vector.

[0115] For example, a 64×64 depth map is formed by stacking 5 depth map sequences. For example, after the depth map is formed into a 64×64×5 input tensor, the depth map sequence is convolved by a preset convolutional kernel in the convolutional feature extraction layer. Specifically, this convolutional feature extraction layer adopts a 3-level cascaded convolutional structure: each convolutional kernel is 3×3 in size, the stride is set to 2, and the number of output channels is set to 32, 64, and 128 respectively. After three levels of convolution, the spatial size is gradually downsampled from 64×64 to 8×8, and the final output is a 3D feature map with a dimension of 8×8×128. Subsequently, through a flattening and mapping layer, the above 3D feature map is flattened into a 1D vector with a length of 8192 (i.e., 8×8×128); then, it is compressed and mapped through two layers of fully connected neural network (MLP): the first layer maps 8192 dimensions to 256 dimensions, and the second layer maps 256 dimensions to 128 dimensions, to obtain the visual feature vector. .

[0116] In the above depth map sequence, the number of image frames, the preset convolutional kernel structure, and the dimension of the feature map are only examples and not limitations. For those skilled in the art, several modifications and improvements can be made without departing from the concept of this application. For example, other sizes of convolutional kernels can be selected, or the specified embedding dimension of the visual feature vector can be changed. These are all within the protection scope of this application.

[0117] In this embodiment, the convolution kernel slides across a depth map sequence containing multiple time channels, simultaneously capturing spatial geometric features such as the edges of the terrain around the robot, the height of steps, and the width of ravines, as well as temporal change information between frames, accurately capturing visual feature vectors.

[0118] In one embodiment, converting the fused feature vector into motor commands for controlling the robot includes: converting the fused feature vector into a joint motion offset vector; obtaining a target joint position based on a default joint position and the joint motion offset vector; and generating motor commands for moving the robot's joints based on the difference between the target joint position and the current joint position.

[0119] Specifically, a neural network can be used to convert the fused feature vector into a joint motion offset vector, and then generate corresponding motor commands based on the joint motion offset vector. Optionally, a pre-constructed policy neural network is obtained, which is a neural network representing the mapping relationship between the fused feature vector and the joint motion offset vector; the fused feature vector is input. The network then generates a joint motion offset vector. For example, the policy neural network is a three-layer fully connected neural network. The first layer includes 256 neurons and a ReLU activation function, the second layer includes 128 neurons and a ReLU activation function, and the third layer is the output layer, containing 12 neurons to output the joint motion offset vector. It is understood that the number of neurons and the selected activation function in each layer can be modified according to the specific application requirements.

[0120] The default joint position refers to a fixed joint angle of the robot. For example, the joint angle when the robot is standing can be used as the default joint position. The default joint position can also be configured to the joint angle when the robot is in other postures, without any restrictions.

[0121] Optionally, the target joint position is obtained based on the default joint position and the joint motion offset vector, including:

[0122]

[0123]

[0124] in, It is the target joint position. This is the default joint position. Pre-configured scaling factors are used to limit joint positions to a smaller range; It is the joint motion offset vector; and These are the proportional and derivative parameters preset in the PD controller during joint control. and These are the current position and current velocity of the joint, respectively. It refers to joint torque. The underlying execution of joint torque is completed by the motor, and the joint torque is controlled in a closed loop by a PD controller to generate motor commands and achieve precise torque output.

[0125] Furthermore, when issuing generator commands, at least one of the following safety constraints can be applied: joint angle limiting, maximum torque limiting, and when the IMU tilt angle is greater than a specified tilt angle threshold, the joint enters a high-damping mode. Combining these safety constraints with the difference between the target joint position and the current joint position, motor commands for moving the robot joint are generated. Among these safety constraints, the values ​​of various limiting parameters, the specified tilt angle threshold, and the joint damping value in the high-damping mode can all be configured as needed and are not restricted here.

[0126] In this embodiment, the abstract fusion feature vector is transformed into specific motor commands to enable the robot to operate.

[0127] Furthermore, in one embodiment, the fusion mapping network is used to combine the adjusted visual feature vector and the motion feature vector to obtain the fused feature vector; the context-aided estimator is used to extract the motion feature vector of the robot at the current moment based on the state vector sequence; and the policy neural network is used to convert the fused feature vector into motor commands for controlling the robot.

[0128] For example, Figure 4 A flowchart illustrating Embodiment 2 of the robot motion control method is provided, as follows: Figure 4 As shown, it includes:

[0129] Step 401: Collect the robot's current state vector. State vector sequence and depth map sequence .

[0130] Step 402: The context-aided estimator extracts the robot's context feature vector based on the state vector sequence. Integrate context feature vectors and the current state vector To obtain the motion feature vector The convolutional feature extraction layer (CNN) extracts 3D feature maps from the depth map sequence, and the flattening and mapping layer converts the 3D feature maps into visual feature vectors. .

[0131] Step 403: The weight generation network receives the visual feature vector and outputs the weight coefficients. The weighting coefficients are obtained by weighting the visual feature vectors. Combine it with the motion feature vector After concatenation, the input is fed into an MLP network, i.e., a feature fusion network, which outputs a fused feature vector. .

[0132] Step 404: Another MLP network, namely the policy neural network, receives the fused feature vector. Output joint motion offset vector ,according to Generate the motor commands required for robot control.

[0133] Furthermore, the robot motion control method also includes: constructing a reward term based on the robot's running state in response to motor commands; constructing a reward function based on the reward term; and performing reinforcement learning processing on the weight generation network, fusion mapping network, context-aided estimator, and policy neural network based on the reward function.

[0134] The reward items shall include at least one of the following: a speed tracking reward item based on the actual base speed and the target base speed; an energy efficiency reward item based on the actual joint torque and the target joint torque; a motion smoothness reward item based on the actual joint acceleration and the target joint acceleration; and a visual dependence reward item based on weighting coefficients. The actual base speed, actual joint torque, actual joint acceleration, and weighting coefficients are parameters generated in real time as the robot responds to motor commands; the target base speed, target joint torque, and target joint acceleration are pre-set parameters used to ensure stable robot operation.

[0135] Optionally, the reinforcement learning reward function is composed of a weighted average of sub-rewards from multiple reward items. Therefore, before applying the robot motion control method described in the above embodiments to a real environment, reinforcement learning guides the network to learn a robust and efficient gait. The weighting coefficients of each reward item can be configured according to requirements. The following describes the components of the reinforcement learning reward function. The reward items will be explained.

[0136] A reward system based on the actual base velocity and the target base velocity is used to achieve velocity tracking, encourage the robot to follow the desired velocity command, and reduce position errors. This includes:

[0137]

[0138]

[0139] in, and These are the linear velocity and angular velocity tracking reward functions, respectively. and This is the reward coefficient, with values ​​of 1.5 and 0.5. and These are the true values ​​of the linear velocity and angular velocity of the base. and These are the target linear velocity and angular velocity.

[0140] A reward system, constructed based on the actual joint torque and the target joint torque, is used to limit energy efficiency, penalize excessive motor current or torque, and encourage energy-saving gait.

[0141]

[0142] in, It is an energy efficiency reward function. It is the torque penalty coefficient, taken as... , It refers to joint torque.

[0143] A reward term, constructed based on the actual joint acceleration and the target joint acceleration, is used to limit the smoothness of movement, penalize abrupt changes in joint acceleration, and prevent mechanical shock.

[0144]

[0145] in, It is a smoothing reward function for actions. It is the joint acceleration penalty coefficient, taken as... , It is the joint acceleration value.

[0146] The reward term, constructed based on weighted coefficients, is used to limit the degree of visual dependence, punish excessive reliance on vision, and guide the strategy to complete the task relying solely on proprioception.

[0147]

[0148] in, It is a visually dependent reward function. It is the visual dependency penalty coefficient, taken as... , It is the weighting coefficient.

[0149] Optionally, the robot can be simulated and trained using the reinforcement learning method described above within a simulation environment. For example, the robot can be run in the isaacsim open-source simulation environment. To improve the robustness of the policy in real-world environments, domain randomization can be used during simulation to randomize the simulation environment. Specifically, this includes the following randomization parameters:

[0150] Dynamic parameters randomization: At the beginning of each training session, the robot base mass, center of mass position ±0.05m, motor torque constant ±10%, and joint friction coefficient are randomly increased within the range of 0 to 30% of the robot's own weight.

[0151] Randomization of ground properties: Randomly change the friction coefficient of the simulated ground from 0.3 to 1.2 to simulate scenarios such as ice, grass, and rubber.

[0152] Sensor noise injection: Gaussian noise is injected into the IMU data acquired by the robot IMU (Inertial Measurement Unit) to simulate vibration during operation; occlusion, missing or depth jump noise is randomly added to the depth map or state vector sequence to simulate water mist and dust interference during operation.

[0153] Delay randomization: In the control loop communication link, a communication delay of 0 to 5 frames is randomly variable.

[0154] In this embodiment, the parameters of each neural network are adjusted by reinforcement learning to improve the robustness of the motion control method in real-world environments.

[0155] Based on the same inventive concept, this application also provides a robot motion control device for implementing the robot motion control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more robot motion control device embodiments provided below can be found in the limitations of the robot motion control method described above, and will not be repeated here.

[0156] In one embodiment, such as Figure 5 As shown, a robot motion control device is provided, comprising:

[0157] The motion feature acquisition module 501 is used to collect the state vectors of the robot at the current moment and at historical moments, obtain the state vector sequence, and extract the motion feature vector of the robot at the current moment based on the state vector sequence.

[0158] The visual feature acquisition module 502 is used to acquire and stack multiple frames of depth maps of the robot's surrounding environment at the current time and historical time to obtain a depth map sequence, and extract the visual feature vector of the robot's surrounding environment based on the depth map sequence.

[0159] The weight coefficient generation module 503 is used to obtain a pre-constructed weight generation network, input the visual feature vector into the weight generation network, and output the weight coefficients of the visual feature vector; wherein, the weight coefficients are positively correlated with the complexity of the environment around the robot and the reliability of the visual feature vector, and the weight generation network is a neural network that represents the mapping relationship between the visual feature vector and the weight coefficients.

[0160] The feature fusion module 504 is used to perform weighted operations on the visual feature vector according to the weight coefficients, and combine the weighted visual feature vector with the motion feature vector to obtain the fused feature vector;

[0161] The instruction generation module 505 is used to convert the fused feature vector into motor instructions for controlling the robot;

[0162] The reinforcement training module 506 is used to construct reward items based on the robot's running state in response to motor commands. The reward items include at least one of the following: a speed tracking reward item based on the real base speed and the target base speed; an energy efficiency reward item based on the real joint torque and the target joint torque; a motion smoothness reward item based on the real joint acceleration and the target joint acceleration; and a visual dependence reward item based on weight coefficients. A reward function is constructed based on the reward items, and reinforcement learning processing is performed on the weight generation network, the fusion mapping network, the context-aided estimator, and the policy neural network based on the reward function. Among them, the fusion mapping network is used to combine the adjusted visual feature vector and the motion feature vector to obtain the fused feature vector; the context-aided estimator is used to extract the robot's motion feature vector at the current moment based on the state vector sequence; and the policy neural network is used to convert the fused feature vector into motor commands for controlling the robot.

[0163] In one embodiment, the weight generation network includes a first neural network layer, a second neural network layer, and a third neural network layer; wherein, the first neural network layer receives a visual feature vector; the second neural network layer performs a linear transformation on the visual feature vector and uses a first activation function to perform a nonlinear transformation on the linearly transformed visual feature vector to extract nonlinear features in the visual feature vector corresponding to environmental complexity and the reliability of the visual feature vector; the third neural network layer performs a linear transformation on the nonlinear features to obtain feature scalar values, and uses a second activation function to map the feature scalar values ​​to a target numerical range to obtain weight coefficients.

[0164] In one embodiment, the feature fusion module 504 combines the weighted visual feature vector and the motion feature vector, including: concatenating the weighted visual feature vector and the motion feature vector to obtain a joint feature vector; obtaining a pre-constructed fusion mapping network, inputting the joint feature vector into the fusion mapping network, and mapping to obtain a fused feature vector; wherein, the fused feature vector is a vector obtained by integrating the robot visual feature vector and the motion feature vector in the joint feature vector, and the fusion mapping network is a neural network that represents the mapping relationship between the joint feature vector and the fused feature vector.

[0165] Optionally, the fusion mapping network includes a fourth neural network layer, a fifth neural network layer, and a sixth neural network layer; wherein, the fourth neural network layer receives the joint feature vector; the fifth neural network layer performs a linear transformation on the joint feature vector and uses a third activation function to perform a nonlinear transformation on the linearly transformed joint feature vector, so as to integrate the robot's visual feature vector and motion feature vector in the joint feature vector through nonlinear transformation, and outputs a first initial feature vector; the sixth neural network layer maps the first initial feature vector to a specified dimension to obtain the fused feature vector.

[0166] In one embodiment, the motion feature acquisition module 501 extracts the motion feature vector of the robot at the current moment based on the state vector sequence, including: acquiring a pre-built context-aided estimator; inputting the state vector sequence into the context-aided estimator, and having the context-aided estimator extract the context feature vector of the state vector sequence; the context-aided estimator is a neural network that represents the mapping relationship between the state vector sequence and the context feature vector; the context feature vector is used to indicate the robot's body motion features and visual terrain features; and the motion feature vector is obtained based on the context feature vector and the state vector at the current moment.

[0167] In one embodiment, the visual feature acquisition module 502 extracts visual feature vectors of the robot's surrounding environment based on the depth map sequence, including: convolving the depth map sequence with a preset convolution kernel to obtain a second initial feature vector; downsampling the second initial feature vector to obtain a three-dimensional feature map; flattening the three-dimensional feature map into a one-dimensional vector, and compressing and mapping the embedding dimension of the one-dimensional vector to a specified embedding dimension to obtain a visual feature vector.

[0168] In one embodiment, the instruction generation module 505 converts the fused feature vector into motor instructions for controlling the robot, including: converting the fused feature vector into a joint motion offset vector; obtaining a target joint position based on the default joint position and the joint motion offset vector; and generating motor instructions for moving the robot joints based on the difference between the target joint position and the current joint position.

[0169] Each module in the aforementioned robot motion control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the robot's processor in hardware form or independent of it, or stored in the robot's memory in software form, so that the processor can call and execute the operations corresponding to each module.

[0170] In one embodiment, a quadruped robot is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0171] In one embodiment, a controller for a robot is provided, wherein the controller can, as follows: Figure 6As shown, the controller includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as state vectors and depth maps. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the controller implements a robot motion control method.

[0172] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the controller to which the present application is applied. A specific controller may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0173] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0174] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0175] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0176] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0177] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A robot motion control method, characterized in that, The method includes: Collect the state vectors of the robot at the current moment and at historical moments to obtain a state vector sequence, and extract the motion feature vector of the robot at the current moment based on the state vector sequence; Acquire and stack multiple frames of depth maps of the robot's surrounding environment at the current time and historical time to obtain a depth map sequence, and extract the visual feature vector of the robot's surrounding environment based on the depth map sequence; A pre-constructed weight generation network is obtained, the visual feature vector is input into the weight generation network, and the weight coefficients of the visual feature vector are output; wherein, the weight coefficients are positively correlated with the complexity of the environment around the robot and the reliability of the visual feature vector, and the weight generation network is a weight generator containing a three-layer fully connected neural network that represents the mapping relationship between the visual feature vector and the weight coefficients. The visual feature vector is weighted according to the weight coefficients, and the weighted visual feature vector is combined with the motion feature vector to obtain the fused feature vector. The fused feature vector is converted into motor commands for controlling the robot; Reward items are constructed based on the robot's operating state in response to the motor commands. The reward items include at least one of the following: a speed tracking reward item based on the actual base speed and the target base speed; an energy efficiency reward item based on the actual joint torque and the target joint torque; a motion smoothness reward item based on the actual joint acceleration and the target joint acceleration; and a visual dependence reward item based on the weighting coefficient. A reward function is constructed based on the reward item, and reinforcement learning processing is performed on the weight generation network, fusion mapping network, context-aided estimator, and policy neural network based on the reward function; wherein, the fusion mapping network is used to combine the adjusted visual feature vector and the motion feature vector to obtain a fused feature vector; the context-aided estimator is used to extract the motion feature vector of the robot at the current moment based on the state vector sequence, and the policy neural network is used to convert the fused feature vector into motor commands for controlling the robot.

2. The method according to claim 1, characterized in that, The weight generation network includes a first neural network layer, a second neural network layer, and a third neural network layer; wherein... The first neural network layer receives the visual feature vector; The second neural network layer performs a linear transformation on the visual feature vector and uses a first activation function to perform a nonlinear transformation on the linearly transformed visual feature vector to extract the nonlinear features in the visual feature vector that correspond to the environmental complexity and the reliability of the visual feature vector. The third neural network layer performs a linear transformation on the nonlinear features to obtain feature scalar values, and uses a second activation function to map the feature scalar values ​​to the target numerical range to obtain the weight coefficients.

3. The method according to claim 1, characterized in that, Combining the weighted visual feature vector with the motion feature vector includes: By concatenating the weighted visual feature vector and the motion feature vector, a joint feature vector is obtained; A pre-constructed fusion mapping network is obtained, and the joint feature vector is input into the fusion mapping network to obtain the fusion feature vector; wherein, the fusion feature vector is a vector obtained by integrating the robot's visual feature vector and motion feature vector in the joint feature vector, and the fusion mapping network is a neural network that represents the mapping relationship between the joint feature vector and the fusion feature vector.

4. The method according to claim 3, characterized in that, The fusion mapping network includes a fourth neural network layer, a fifth neural network layer, and a sixth neural network layer; wherein, The fourth neural network layer receives the joint feature vector; The fifth neural network layer performs a linear transformation on the joint feature vector and uses a third activation function to perform a nonlinear transformation on the linearly transformed joint feature vector, so as to integrate the robot's visual feature vector and motion feature vector in the joint feature vector through nonlinear transformation and output the first initial feature vector. The sixth neural network layer maps the first initial feature vector to a specified dimension to obtain the fused feature vector.

5. The method according to claim 1, characterized in that, Extracting the robot's motion feature vector at the current moment based on the state vector sequence includes: Obtain a pre-built context-aided estimator; The state vector sequence is input into the context-aided estimator, which extracts the context feature vector of the state vector sequence. The context-aided estimator is a neural network that represents the mapping relationship between the state vector sequence and the context feature vector. The context feature vector is used to indicate the robot's body motion features and visual terrain features. The motion feature vector is obtained based on the context feature vector and the state vector at the current moment.

6. The method according to claim 1, characterized in that, The visual feature vectors of the robot's surrounding environment are extracted based on the depth map sequence, including: The depth map sequence is convolved using a preset convolution kernel to obtain a second initial feature vector; The second initial feature vector is downsampled to obtain a three-dimensional feature map; The three-dimensional feature map is flattened into a one-dimensional vector, and the embedding dimension of the one-dimensional vector is compressed and mapped to a specified embedding dimension to obtain the visual feature vector.

7. The method according to claim 1, characterized in that, Converting the fused feature vector into motor commands for controlling the robot includes: The fused feature vector is converted into a joint motion offset vector; The target joint position is obtained based on the default joint position and the joint motion offset vector; Based on the difference between the target joint position and the current joint position, motor commands for moving the robot's joints are generated.

8. A robot motion control device, characterized in that, The device includes: The motion feature acquisition module is used to collect the state vectors of the robot at the current moment and at historical moments, obtain a state vector sequence, and extract the motion feature vector of the robot at the current moment based on the state vector sequence; The visual feature acquisition module is used to acquire and stack multiple frames of depth maps of the robot's surrounding environment at the current time and historical time to obtain a depth map sequence, and extract the visual feature vector of the robot's surrounding environment based on the depth map sequence. The weight coefficient generation module is used to obtain a pre-constructed weight generation network, input the visual feature vector into the weight generation network, and output the weight coefficients of the visual feature vector; wherein, the weight coefficients are positively correlated with the complexity of the environment surrounding the robot and the reliability of the visual feature vector; the weight generation network is a weight generator containing a three-layer fully connected neural network that represents the mapping relationship between the visual feature vector and the weight coefficients. The feature fusion module is used to perform a weighted operation on the visual feature vector according to the weight coefficient, and combine the weighted visual feature vector with the motion feature vector to obtain a fused feature vector; An instruction generation module is used to convert the fused feature vector into motor instructions for controlling the robot; The reinforcement training module is used to construct reward items based on the robot's running state in response to the motor commands. The reward items include at least one of the following: a speed tracking reward item based on the actual base speed and the target base speed; an energy efficiency reward item based on the actual joint torque and the target joint torque; a motion smoothness reward item based on the actual joint acceleration and the target joint acceleration; and a visual dependence reward item based on the weight coefficients. A reward function is constructed based on the reward items, and reinforcement learning processing is performed on the weight generation network, the fusion mapping network, the context-aided estimator, and the policy neural network based on the reward function. The fusion mapping network combines the adjusted visual feature vector with the motion feature vector to obtain a fused feature vector. The context-aided estimator extracts the robot's motion feature vector at the current moment based on the state vector sequence, and the policy neural network converts the fused feature vector into motor commands for controlling the robot.

9. A quadruped robot, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images