Motion control method, apparatus, device, medium and product

Abstract

The application provides a motion control method and device, computer equipment, computer readable storage medium and computer program product, and belongs to the motion control field. The motion control method comprises the following steps: obtaining body perception data and environment perception data of a target object at a current time, and a pre-trained strategy network, wherein the strategy network comprises an encoder, a gated recurrent unit and a multilayer perception machine; extracting a body feature vector based on the body perception data, and inputting the environment perception data into the encoder to extract a spatial feature vector; inputting the body feature vector and the spatial feature vector into the gated recurrent unit to obtain a comprehensive feature vector; inputting the comprehensive feature vector and the body feature vector into the multilayer perception machine to obtain a motion control instruction, wherein the motion control instruction is used for controlling the motion of the target object. The technical scheme of the embodiment of the application can enable the robot to realize efficient and safe autonomous motion planning and execution.

Description

Technical Field

[0001] This application relates to the field of motion control technology, and in particular to a motion control method, device, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of artificial intelligence and robotics, various types of robots (such as wheeled, tracked, and legged robots) have been widely used in many fields such as industry, medicine, and services.

[0003] However, robots currently have limited understanding of complex terrain environments, making it difficult to achieve safe and real-time autonomous movement planning and execution. The closed-loop accuracy from environmental perception to autonomous movement still needs to be improved.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the Invention

[0005] This application provides a motion control method, device, computer equipment, computer-readable storage medium, and computer program product. By combining real-time perception with historical observation through a gated loop unit, and then making motion decisions through a multilayer sensor, it can perform efficient and safe autonomous movement planning and execution based on a comprehensive and accurate analysis of the environment, thereby improving the closed-loop accuracy of environmental perception of autonomous movement.

[0006] In a first aspect, embodiments of this application provide a motion control method, the method comprising:

[0007] Acquire the ontology perception data and environment perception data of the target object at the current moment, as well as a pre-trained policy network, which includes an encoder, a gated recurrent unit, and a multilayer perceptron. Based on the ontology perception data, ontology feature vectors are extracted, and the environment perception data is input into the encoder to extract spatial feature vectors; The ontological feature vector and the spatial feature vector are input into the gated loop unit to obtain the comprehensive feature vector; The integrated feature vector and the ontological feature vector are input into a multilayer perceptron to obtain motion control instructions, which are used to control the motion of the target object.

[0008] In one embodiment, the environmental perception data includes a distant depth map and a near depth map; inputting the environmental perception data into the encoder to extract spatial feature vectors includes: If the distant depth map and the near depth map pass the verification, the distant depth map and the near depth map are preprocessed; wherein, the preprocessing includes at least one of downsampling, cropping, limiting and normalization; The preprocessed distant depth map and near depth map are input into the encoder to obtain the spatial feature vector.

[0009] In one embodiment, the ontological feature vector and the spatial feature vector are input into the gated loop unit to obtain a comprehensive feature vector, including: The ontology feature vector and the spatial feature vector are input into the gated loop unit so that the hidden state, the ontology feature vector and the spatial feature vector are fused by the gated loop unit to obtain the comprehensive feature vector; The hidden state is obtained by the gated loop unit based on the ontological feature vector and spatial feature vector at historical moments.

[0010] In one embodiment, the policy network is obtained through the following operation: Acquire a training dataset, a target terrain height map, an initial policy network, and an evaluation network. The training dataset includes multiple training samples, each of which includes training ontology perception data, training environment perception data, and target motion control commands. The terrain height map is used to characterize the three-dimensional terrain structure of the environment in which the target object is located. The initial policy network includes an initial encoder, an initial gated recurrent unit, and an initial multilayer perceptron. Based on the training ontology perception data, training ontology feature vectors are extracted. The training environment perception data is input into the initial encoder to extract training space feature vectors. The training ontology feature vectors and the training space feature vectors are input into the initial gated loop unit to obtain a training comprehensive feature vector through the initial gated loop unit. The training integrated feature vector and the training ontology feature vector are input into the initial multilayer perceptron to obtain the predicted motion control command through the initial multilayer perceptron; The target terrain height map, the training ontology perception data, the predicted motion control command, and the target motion control command are input into the evaluation network to obtain the prediction value through the evaluation network. The loss data is determined based on the predicted value, and the initial policy network and the evaluation network are trained based on the loss data to obtain the policy network.

[0011] In one embodiment, the operation of obtaining the policy network further includes: The trained integrated feature vector is input into the decoder to reconstruct the predicted terrain height map. Correspondingly, determining the loss data based on the predicted value includes: The loss data is determined based on the predicted value, the target terrain elevation map, and the predicted terrain elevation map.

[0012] In one embodiment, the trained integrated feature vector and the ontology feature vector are input into an initial multilayer perceptron to obtain predicted motion control commands through the initial multilayer perceptron, including: The training integrated feature vector and the ontology feature vector are input into the initial multilayer perceptron to obtain the predicted probability distribution of the motion control command set through the initial multilayer perceptron, and the predicted motion control command is obtained by sampling the predicted probability distribution; wherein, the motion control command set includes multiple candidate motion control commands that can be executed by the target object; Correspondingly, based on the predicted value, the target terrain elevation map, and the predicted terrain elevation map, the loss data is determined, including: The loss data is determined based on the predicted value, the target terrain elevation map, the predicted terrain elevation map, and the predicted probability distribution.

[0013] In one embodiment, the loss data is determined based on the predicted value, the target terrain height map, the predicted terrain height map, and the predicted probability distribution, including: Based on the predicted probability distribution and the predicted value, the first loss data is determined; Based on the predicted value, the second loss data is determined; Based on the difference between the target terrain elevation map and the predicted terrain elevation map, a third loss data is determined; Based on the predicted probability distribution, the fourth loss data is determined; The loss data is determined based on the first loss data, the second loss data, the third loss data, and the fourth loss data.

[0014] Secondly, embodiments of this application provide a motion control device, the device comprising: The acquisition module is used to acquire the ontology perception data and environment perception data of the target object at the current moment, as well as a pre-trained policy network, which includes an encoder, a gated recurrent unit, and a multilayer perceptron. The extraction module is used to extract ontological feature vectors based on the ontological perception data, and to input the environmental perception data into the encoder to extract spatial feature vectors. The fusion module is used to input the ontological feature vector and the spatial feature vector into the gated loop unit to obtain a comprehensive feature vector; The decision module is used to input the integrated feature vector and the ontological feature vector into the multilayer perceptron to obtain motion control instructions, which are used to control the motion of the target object.

[0015] Thirdly, embodiments of this application provide a computer device, including: At least one processor; and A memory that is communicatively connected to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method described above.

[0017] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0018] In some embodiments of this application, the technical solutions involve acquiring the target object's ontology perception data and environmental perception data at the current moment, as well as a pre-trained policy network. The policy network includes an encoder, a gating loop unit, and a multilayer perceptron. Ontology feature vectors are extracted, and spatial feature vectors are extracted based on the encoder. These ontology and spatial feature vectors are then input into the gating loop unit to obtain a comprehensive feature vector. The comprehensive feature vector and the ontology feature vector are then input into the multilayer perceptron to obtain motion control commands, thereby achieving motion control of the target object.

[0019] The embodiments of this application employing the above-mentioned technical solution may include the following advantages: by combining real-time perception with historical observation through a gated loop unit, and then making motion decisions through a multilayer sensor, efficient and safe autonomous movement planning and execution can be carried out based on a comprehensive and accurate analysis of the environment, thereby improving the closed-loop accuracy of environmental perception of autonomous movement. Attached Figure Description

[0020] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0021] Figure 1 A flowchart illustrating a motion control method according to Embodiment 1 of this application is shown schematically. Figure 2 A schematic diagram of the structure of the target object according to Embodiment 1 of this application is shown; Figure 3 A schematic view of the target object according to Embodiment 1 of this application is shown; Figure 4 This schematically illustrates another flowchart of the motion control method according to Embodiment 1 of this application; Figure 5 The diagram illustrates a preprocessing flowchart for a depth map according to Embodiment 1 of this application; Figure 6 A block diagram of a motion control device according to Embodiment 2 of this application is schematically shown; and Figure 7 A schematic diagram of the hardware architecture of a computer device according to Embodiment 3 of this application is shown. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0023] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0024] It should be noted that, in any stage of this application involving the collection, storage, use, transmission, and processing of data, each stage strictly adheres to the laws, regulations, industry standards, and regulatory requirements of the data source, usage location, and relevant countries and regions to ensure the legality and compliance of data activities. In the collection stage, the purpose, method, and scope of collection are clearly communicated to the data subject in a prominent manner. Collection is conducted only after obtaining the data subject's legal authorization, ensuring that the collection process follows the "minimum necessary" principle and does not exceed the scope of data collection. In the storage stage, storage periods are limited, and data is promptly deleted or anonymized / encrypted after the storage purpose is achieved. In the usage stage, a strict data security protection mechanism is implemented, using field-level desensitization technology and processing the original data according to preset desensitization rules. For different types of data, multiple desensitization strategies, such as data generalization, data anonymization, and data encryption, are employed to effectively mitigate the risk of sensitive information leakage and ensure that all data used is securely processed and desensitized, comprehensively protecting the rights and interests of data subjects and data security. In the transmission and processing stages, the confidentiality and security of data are ensured during transmission and processing.

[0025] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0026] To facilitate understanding of the technical solutions provided in the embodiments of this application by those skilled in the art, the relevant technologies are described below: With the development of artificial intelligence and robotics, various types of robots (such as wheeled, tracked, and legged robots) have been widely used in many fields such as industry, healthcare, and services. However, although these robots are equipped with sensors (such as cameras and radar) and artificial intelligence algorithms, they generally have limited understanding of complex terrain environments, making it difficult to achieve safe and real-time autonomous movement planning and execution. The closed-loop accuracy from environmental perception to autonomous movement still needs to be improved.

[0027] To address this, this application provides a motion control technical solution. This solution employs a reinforcement learning (RL) algorithm based on an actor-critic framework. During the training phase, on the one hand, it provides the critic network with privileged information (such as a real terrain height map) that the robot cannot directly access during the inference phase. Based on the real terrain height map, the critic network can more accurately evaluate the value of the actor network's current decisions, thereby guiding the actor network to learn better strategies. On the other hand, it reconstructs terrain height maps based on training samples. Subsequently, based on the difference between the reconstructed terrain height map and the real terrain height map, it guides the actor network to learn a more effective representation of terrain understanding. Through training, the actor network's environmental perception ability can be improved. During the inference phase, it can perform efficient, safe, and real-time autonomous movement planning and execution based on a comprehensive and accurate analysis of the environment, thereby improving the closed-loop accuracy from environmental perception to autonomous movement. Details are provided below.

[0028] The technical solutions of this application are described below through several embodiments. It should be understood that these embodiments can be implemented in many different forms and should not be construed as being limited to the embodiments set forth herein.

[0029] Example 1 Figure 1 A flowchart illustrating a motion control method according to Embodiment 1 of this application is shown schematically.

[0030] like Figure 1 As shown, the motion control method may include steps S100 to S106, wherein: Step S100: Obtain the ontology perception data and environment perception data of the target object at the current moment, as well as the pre-trained policy network, which includes an encoder, a gated recurrent unit, and a multilayer perceptron.

[0031] Step S102 involves extracting ontological feature vectors based on ontological perception data and inputting environmental perception data into the encoder to extract spatial feature vectors.

[0032] Step S104: Input the ontological feature vector and spatial feature vector into the gated loop unit to obtain the comprehensive feature vector.

[0033] Step S106: Input the integrated feature vector and the ontological feature vector into the multilayer perceptron to obtain motion control instructions, which are used to control the motion of the target object.

[0034] The motion control method provided in this embodiment acquires the target object's ontological perception data and environmental perception data at the current moment, as well as a pre-trained policy network. The policy network includes an encoder, a gated loop unit, and a multilayer perceptron. Ontological feature vectors are extracted, and spatial feature vectors are extracted based on the encoder. These ontological and spatial feature vectors are input into the gated loop unit to obtain a comprehensive feature vector. The comprehensive feature vector and the ontological feature vector are then input into the multilayer perceptron to obtain motion control commands, thereby achieving motion control of the target object. By combining real-time perception with historical observation through the gated loop unit, and then making motion decisions through the multilayer perceptron, efficient and safe autonomous movement planning and execution can be performed based on a comprehensive and accurate analysis of the environment, thus improving the closed-loop accuracy of environmental perception leading to autonomous movement.

[0035] The following combination Figure 1 The steps in steps S100 to S106 and other optional steps are described in detail.

[0036] Step S100 It acquires the ontology perception data and environment perception data of the target object at the current moment, as well as a pre-trained policy network, which includes an encoder, a gated recurrent unit, and a multilayer perceptron.

[0037] The target object can be various types of mobile platforms, such as wheeled, tracked, and legged platforms. Legged platforms can include bipedal, quadrupedal, and hexapedal configurations. The following will use a quadrupedal robot dog as an example to illustrate the motion control method provided in the embodiments of this application.

[0038] Acquire the target object's body perception data and environmental perception data at the current moment. The body perception data can include at least one of the target object's velocity, angle, angular velocity, turn angle, position, and attitude. The target object can acquire body perception data by configuring sensors such as wheeled odometry, inertial measurement units (IMUs), lidar, and cameras. Environmental perception data can include distant depth maps and near depth maps. Distant depth maps and near depth maps can be obtained by placing visual sensors (such as binocular depth cameras) on different parts of the target object.

[0039] like Figure 2As shown, an adapter can be installed on the back of the target quadruped robot dog, and then a support sheet metal can be fixed to the adapter. A camera bracket (which can be obtained through 3D printing) can be mounted on the front end of the support sheet metal. Binocular depth cameras are installed at different parts of the camera bracket. One binocular depth camera is used to observe the terrain in front of the target object and acquire a distant depth map, while the other binocular depth camera is used to observe the terrain of the target object's forelegs and acquire a near depth map. Figure 3 The diagram schematically illustrates the field of view (FOV) of two binocular depth cameras, showing the spatial range that each camera can perceive.

[0040] The target object can also acquire a pre-trained policy network, which may include an encoder, a gated recurrent unit (GRU), and a multilayer perceptron. The encoder converts the raw input data into a fixed-dimensional embedding representation (feature vector). The GRU handles temporal dependencies and maintains hidden states, giving the policy network memory capabilities. The multilayer perceptron makes motion decisions and outputs motion control commands to control the motion of the target object.

[0041] Step S102 Based on the ontology perception data, ontology feature vectors are extracted, and environmental perception data is input into the encoder to extract spatial feature vectors.

[0042] For example, ontological feature vectors can be extracted from ontological perception data through pre-set rules or pre-trained neural networks, and environmental perception data can be input into an encoder to extract spatial feature vectors. Figure 4 As shown, distant and near depth maps can be input into a depth map encoder to extract spatial feature vectors. In practical applications, binocular vision cameras generate depth maps based on the parallax of the left and right cameras. When the binocular vision camera is too close to the measured point, invalid data areas may exist, leading to errors in autonomous motion planning. To alleviate the above problems, embodiments of this application can preprocess the distant and near depth maps, and exemplary solutions will be provided below.

[0043] In one possible implementation, environmental perception data is input into an encoder to extract spatial feature vectors, including: preprocessing the distant depth map and the near depth map after the distant depth map and the near depth map have passed verification; wherein the preprocessing includes at least one of downsampling, cropping, limiting and normalization; and inputting the preprocessed distant depth map and the near depth map into the encoder to obtain spatial feature vectors.

[0044] For example, a binocular vision camera can perform left-right consistency verification on images captured by the left and right cameras. A first disparity value is calculated between each pixel in the left image and its corresponding pixel in the right image, and a second disparity value is calculated between each pixel in the right image and its corresponding pixel in the left image. Based on the first and second disparity values ​​for each pixel, the left-right consistency verification result can be determined. If the verification passes, the depth map is valid; if the verification fails, the depth map contains invalid data areas. If both the distant and near depth maps pass verification, they can be preprocessed separately.

[0045] Figure 5 The diagram illustrates the preprocessing steps for the depth map. First, assuming both the distant and near depth maps are valid, the message header, including timestamps and coordinate systems, can be initialized. Then, the original depth map can be downsampled, for example, reducing the original resolution to 640. A 480 image was downsampled to obtain a resolution of 64. A 48-bit image. The depth map obtained from the downsampling is cropped, for example, by removing 8 columns from each side, resulting in a 48-bit image. The image is 48 pixels deep. The depth values ​​of each pixel in the cropped image are subjected to limiting and normalization. For example, the distance depth map can be limited to [0.25, 5.0] meters, and the distance depth map can be limited to [0.25, 2.0] meters. The depth values ​​of each pixel in the limited depth map are then normalized, such as mapping them to the [-0.5, 0.5] range. The preprocessed distance and distance depth maps are then input into the encoder to obtain the spatial feature vector.

[0046] In this embodiment, invalid depth maps are first checked and filtered out. The depth maps that pass the check are then downsampled, cropped, limited, and normalized. This enables the encoder to extract spatial feature vectors that take into account both long-distance and short-distance environmental perception with extremely low computational power consumption. These spatial feature vectors can be used to characterize the terrain scanned by the target object at the current moment.

[0047] Step S104 The ontological feature vector and spatial feature vector are input into the gated loop unit to obtain the comprehensive feature vector.

[0048] like Figure 4 As shown, the ontological feature vector and spatial feature vector can be input into the gated recurrent unit. The gated recurrent unit can store historical information, which is based on the inputs at various times before the current time, i.e., the terrain previously scanned by the target object. The gated recurrent unit fuses the historical information with the ontological feature vector (i.e., the spatial feature vector) at the current time to obtain a comprehensive feature vector. An exemplary scheme is provided below.

[0049] In one possible implementation, the ontology feature vector and spatial feature vector are input into a gated loop unit to obtain a comprehensive feature vector. This includes: inputting the ontology feature vector and spatial feature vector into a gated loop unit to fuse the hidden state, ontology feature vector, and spatial feature vector through the gated loop unit to obtain a comprehensive feature vector; wherein the hidden state is obtained by the gated loop unit based on the ontology feature vector and spatial feature vector at a historical moment.

[0050] For example, ontology feature vectors and spatial feature vectors are input into a gated recurrent unit, which performs intelligent fusion based on reset and update mechanisms. The reset gate combines the two types of feature vectors currently input with the hidden state passed down from the previous time step and performs a nonlinear transformation to obtain an intermediate feature vector. This hidden state (historical information) includes the memory of all input ontology and spatial feature vectors from previous time steps. The update gate can be used to adjust the proportion of historical and current perceptions, ultimately outputting a comprehensive feature vector.

[0051] In this embodiment, a gated loop unit is introduced to encode historical observations into the hidden state, so that the target object can remember the terrain that was previously scanned, thus making up for the problem of limited field of view or occlusion in a single frame (single moment) depth map.

[0052] Step S106 The integrated feature vector and the ontological feature vector are input into the multilayer perceptron to obtain motion control commands, which are used to control the movement of the target object.

[0053] For example, by inputting the integrated feature vector and the ontology feature vector into a multilayer perceptron, the multilayer perceptron can generate matching motion control commands based on its understanding of the target object's current environment and its own state. Based on these motion control commands, the movement of the target object can be controlled.

[0054] In this embodiment, the depth map is compressed into a feature vector by an encoder, real-time perception is combined with historical observation by a gated loop unit, and motion decision is made by a multilayer perceptron. This enables efficient and safe autonomous movement planning and execution based on a comprehensive and accurate analysis of the environment, thereby improving the closed-loop accuracy of environmental perception of autonomous movement.

[0055] The above embodiments exemplify the inference phase of the policy network. The training process of the policy network will be further exemplified below.

[0056] In one possible implementation, the policy network can be trained through the following steps: Acquiring a training dataset, a target terrain height map, an initial policy network, and an evaluation network. The training dataset includes multiple training samples, each containing training ontology perception data, training environment perception data, and target motion control commands. The terrain height map represents the 3D terrain structure of the target object's environment. The initial policy network includes an initial encoder, an initial gated recurrent unit, and an initial multilayer perceptron. Based on the training ontology perception data, training ontology feature vectors are extracted. The training environment perception data is input into the initial encoder to extract training spatial feature vectors. The training ontology feature vectors and training spatial feature vectors are input into the initial gated recurrent unit to obtain a training composite feature vector. The training composite feature vector and training ontology feature vector are input into the initial multilayer perceptron to obtain predicted motion control commands. The target terrain height map, training ontology perception data, predicted motion control commands, and target motion control commands are input into the evaluation network to obtain predicted values. Based on the predicted values, loss data is determined, and the initial policy network and evaluation network are trained based on the loss data to obtain the policy network.

[0057] For example, a training dataset consisting of multiple training samples, a target terrain height map, an initial policy network, and an evaluation network can be obtained. Each training sample includes training ontology perception data, training environment perception data, and target motion control commands. The target motion control commands are the correct commands that the policy network must learn to predict given the training ontology perception data and training environment perception data. The target terrain height map can be privileged information (i.e., a real terrain height map) that the robot cannot directly obtain during the inference phase, and can be used to characterize the three-dimensional terrain structure of the robot's environment. The initial policy network can be used to predict the robot's motion control commands, and the evaluation network is used to evaluate the value of the motion control commands predicted by the initial policy network. The initial policy network may include an initial encoder, an initial gated recurrent unit, and an initial multilayer perceptron.

[0058] Training ontology feature vectors are extracted from training ontology perception data. Training environment perception data is input into the initial encoder to obtain a training comprehensive feature vector. The training ontology feature vector and the training comprehensive feature vector are then input into the initial multilayer perceptron. The initial multilayer perceptron can obtain the predicted probability distribution of the motion control command set, and the predicted motion control commands are obtained by sampling the predicted probability distribution. The motion control command set can include multiple candidate motion control commands executable by the target object.

[0059] like Figure 4As shown, the evaluation network can generate predicted values ​​based on privileged information (target terrain height map), ontological perception data, predicted motion control commands, and target motion control commands. The predicted value can represent the scoring result of the evaluation network on the predicted motion control commands.

[0060] Loss data can be determined based on predicted value. For example, the first loss data (policy cutoff loss) can be determined based on the predicted probability distribution and predicted value. CLIP The initial loss data can be used to guide the initial policy network to output actions (motion control commands) that yield higher rewards (scores). During training, the initial policy network needs to undergo multiple iterations to obtain the final deployed policy network. Iteration refers to updating the parameters of the initial policy network; the parameters before and after the update correspond to the old and new policies. Based on this, in each iteration, the training ontology feature vector and the training comprehensive feature vector can be input into the current initial policy network to obtain the predicted probability distribution of the new policy. Simultaneously, the predicted probability distribution of the old policy output from the previous iteration's initial policy network can be retained. Based on the predicted probability distributions of the old and new policies, the predicted value, and the preset truncation hyperparameters, the first loss data can be determined. The preset truncation hyperparameters can be used to limit the policy update magnitude to improve the stability of model training.

[0061] The second loss (value loss) can be determined based on the predicted value. value The second loss data can be used to guide the evaluation network to accurately assess the value of predicted motion control commands. The final loss data can be determined by a weighted sum of the first and second loss data. This loss data can be used to update the gradients of the initial policy network and the evaluation network, continuously training with the goal of minimizing the loss data, ultimately resulting in the policy network.

[0062] In this embodiment, during the training phase, the critic network (evaluation network) is provided with privileged information (real terrain height map) that the robot cannot directly obtain during the inference phase. Based on the real terrain height map, the critic network can more accurately evaluate the value of the current decision of the actor network (initial policy network), thereby guiding the actor network to learn a better strategy.

[0063] In one possible implementation, the operation of obtaining the policy network further includes: inputting the trained integrated feature vector into the decoder to reconstruct the predicted terrain height map through the decoder; correspondingly, determining loss data based on the prediction value includes: determining loss data based on the prediction value, the target terrain height map, and the predicted terrain height map.

[0064] For example, such as Figure 4As shown, the trained integrated feature vector can be input into a pre-defined decoder to reconstruct the terrain height map of the robot's surrounding environment from the latent vector output by the gated recurrent unit, i.e., the predicted terrain height map. Correspondingly, based on the predicted probability distribution and predicted value, the first loss data (policy truncation loss) can be determined. CLIP The second loss (value loss) can be determined based on the predicted value. value Based on the difference between the predicted terrain elevation map and the target terrain elevation map, the third loss data (reconstruction loss) is determined. mse The third loss data can be used to guide the intermediate hidden states of the initial policy network (initial encoder and initial gated recurrent unit) to better learn terrain information. The loss data is obtained by weighted summation of the first, second, and third loss data. The initial policy network and evaluation network are then trained based on the loss data to finally obtain the policy network.

[0065] For example, the third loss data can be obtained by calculating the mean square error between the predicted terrain height map and the target terrain height map. The calculation formula can be expressed as follows:

[0066] Where n can be the number of height measurement points, H pred H can be the estimated height vector, and H can be the actual height obtained in the simulation environment.

[0067] In this embodiment, a terrain height map is reconstructed based on the trained integrated feature vectors. Subsequently, the difference between the reconstructed terrain height map and the real terrain height map guides the strategy network to learn a more effective representation for terrain understanding.

[0068] In one possible implementation, training the integrated feature vector and the ontology feature vector are input into an initial multilayer perceptron to obtain predicted motion control commands. This can include: inputting the trained integrated feature vector and the trained ontology feature vector into the initial multilayer perceptron to obtain a predicted probability distribution of the motion control command set, and sampling the predicted probability distribution to obtain the predicted motion control commands; wherein the motion control command set includes multiple candidate motion control commands executable by the target object; correspondingly, loss data is determined based on the predicted value, the target terrain height map, and the predicted terrain height map, including: determining loss data based on the predicted value, the target terrain height map, the predicted terrain height map, and the predicted probability distribution.

[0069] For example, the trained integrated feature vector and ontology feature vector are input into an initial multilayer perceptron. The multilayer perceptron uses these two types of input feature vectors to gain a deep understanding of the terrain, thereby determining the predicted probability distribution of multiple candidate motion control commands executable by the target object. The predicted motion control command is then determined through sampling. Correspondingly, based on the predicted probability distribution and predicted value, the first loss data (policy truncation loss) can be determined. CLIP The second loss (value loss) can be determined based on the predicted value. value Based on the difference between the predicted terrain elevation map and the target terrain elevation map, the third loss data (reconstruction loss) is determined. mse The fourth loss data (entropy loss) can be determined based on the predicted probability distribution. entropy The fourth loss data can be used to encourage the robot to explore different actions. The final loss data can be determined by a weighted sum of the first, second, third, and fourth loss data. The calculation formula can be expressed as follows:

[0070] In this embodiment, training the initial policy network and the evaluation network by combining policy truncation loss, value loss, reconstruction loss and entropy loss can improve the policy network's environmental awareness, thereby enabling efficient, safe and real-time autonomous movement planning and execution based on accurate environmental information, and improving the closed-loop accuracy from environmental awareness to autonomous movement.

[0071] To make this application easier to understand, an exemplary application is provided below.

[0072] S1: Acquisition and dissemination of external environment perception data: The Orin airborne computing platform of the legged mobile platform can acquire dual-path depth maps provided by two depth cameras, perform a series of preprocessing steps such as cropping and limiting, and then publish the processed image data locally via DDS.

[0073] S2: Acquiring Body Perception Data: The body motion control computer of the legged mobile platform collects body perception data (angle, angular velocity, etc.) through sensors and sends it to Orin via network cable using DDS.

[0074] S3: Computation and Reasoning: After Orin subscribes to image data and ontology perception data, it runs a neural network controller (policy network) to perform inference tasks such as feature extraction and action command calculation internally.

[0075] S4: Send control signal: Orin sends the calculated control signal (motion control command) to the main body motion control computer again through DDS. The main body motion control computer drives the motor to complete the corresponding action, forming a complete control closed loop.

[0076] In this exemplary application, the legged mobile platform can accurately understand the surrounding environment and move autonomously within it efficiently and safely, forming a precise closed loop from "seeing" to "doing," enabling it to operate flexibly and safely in complex environments.

[0077] Example 2 Figure 6 The diagram schematically illustrates a motion control device according to Embodiment 2 of this application. This device can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of this application. The program module referred to in this embodiment is a series of computer program instruction segments capable of performing a specific function. The following description will specifically introduce the function of each program module in this embodiment. For example... Figure 6 As shown, the device 1000 may include: an acquisition module 1100, an extraction module 1200, a fusion module 1300, and a decision module 1400, wherein: The acquisition module 1100 is used to acquire the ontology perception data and environment perception data of the target object at the current moment, as well as a pre-trained policy network, which includes an encoder, a gated recurrent unit and a multilayer perceptron. The extraction module 1200 is used to extract ontological feature vectors based on ontological perception data, and to input environmental perception data into the encoder to extract spatial feature vectors. The fusion module 1300 is used to input the ontological feature vector and the spatial feature vector into the gated loop unit to obtain a comprehensive feature vector. The decision module 1400 is used to input the integrated feature vector and the ontological feature vector into the multilayer perceptron to obtain motion control instructions, which are used to control the motion of the target object.

[0078] As an optional embodiment, the environmental perception data includes a distant depth map and a near depth map; inputting the environmental perception data into the encoder to extract spatial feature vectors includes: If the distant depth map and the near depth map pass the verification, the distant depth map and the near depth map are preprocessed; wherein, the preprocessing includes at least one of downsampling, cropping, limiting and normalization; The preprocessed distant depth map and near depth map are input into the encoder to obtain the spatial feature vector.

[0079] As an optional embodiment, the ontology feature vector and the spatial feature vector are input into the gated loop unit to obtain a comprehensive feature vector, including: The ontology feature vector and the spatial feature vector are input into the gated loop unit so that the hidden state, the ontology feature vector and the spatial feature vector are fused by the gated loop unit to obtain the comprehensive feature vector; The hidden state is obtained by the gated loop unit based on the ontological feature vector and spatial feature vector at historical moments.

[0080] As an optional embodiment, the policy network is characterized by the following operation: Acquire a training dataset, a target terrain height map, an initial policy network, and an evaluation network. The training dataset includes multiple training samples, each of which includes training ontology perception data, training environment perception data, and target motion control commands. The terrain height map is used to characterize the three-dimensional terrain structure of the environment in which the target object is located. The initial policy network includes an initial encoder, an initial gated recurrent unit, and an initial multilayer perceptron. Based on the training ontology perception data, training ontology feature vectors are extracted. The training environment perception data is input into the initial encoder to extract training space feature vectors. The training ontology feature vectors and the training space feature vectors are input into the initial gated loop unit to obtain a training comprehensive feature vector through the initial gated loop unit. The training integrated feature vector and the training ontology feature vector are input into the initial multilayer perceptron to obtain the predicted motion control command through the initial multilayer perceptron; The target terrain height map, the training ontology perception data, the predicted motion control command, and the target motion control command are input into the evaluation network to obtain the prediction value through the evaluation network. The loss data is determined based on the predicted value, and the initial policy network and the evaluation network are trained based on the loss data to obtain the policy network.

[0081] As an optional embodiment, the operation of obtaining the policy network further includes: The trained integrated feature vector is input into the decoder to reconstruct the predicted terrain height map. Correspondingly, determining the loss data based on the predicted value includes: The loss data is determined based on the predicted value, the target terrain elevation map, and the predicted terrain elevation map.

[0082] As an optional embodiment, the trained integrated feature vector and the ontology feature vector are input into an initial multilayer perceptron to obtain predicted motion control commands through the initial multilayer perceptron, including: The training integrated feature vector and the ontology feature vector are input into the initial multilayer perceptron to obtain the predicted probability distribution of the motion control command set through the initial multilayer perceptron, and the predicted motion control command is obtained by sampling the predicted probability distribution; wherein, the motion control command set includes multiple candidate motion control commands that can be executed by the target object; Correspondingly, based on the predicted value, the target terrain elevation map, and the predicted terrain elevation map, the loss data is determined, including: The loss data is determined based on the predicted value, the target terrain elevation map, the predicted terrain elevation map, and the predicted probability distribution.

[0083] As an optional embodiment, the loss data is determined based on the predicted value, the target terrain elevation map, the predicted terrain elevation map, and the predicted probability distribution, including: Based on the predicted probability distribution and the predicted value, the first loss data is determined; Based on the predicted value, the second loss data is determined; Based on the difference between the target terrain elevation map and the predicted terrain elevation map, a third loss data is determined; Based on the predicted probability distribution, the fourth loss data is determined; The loss data is determined based on the first loss data, the second loss data, the third loss data, and the fourth loss data.

[0084] Example 3 Figure 7 This illustration schematically depicts the hardware architecture of a computer device 10000 suitable for implementing a motion control method according to Embodiment 3 of this application. In some embodiments, the computer device 10000 may be a terminal device such as a smartphone, wearable device, tablet computer, personal computer, vehicle terminal, game console, virtual device, workbench, digital assistant, set-top box, or robot. In other embodiments, the computer device 10000 may be a rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc. Figure 7 As shown, the computer device 10000 includes, but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate and be linked with each other via a system bus. Wherein: The memory 10010 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of a computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is typically used to store the operating system and various application software installed on the computer device 10000, such as program code for motion control methods. Furthermore, the memory 10010 can also be used to temporarily store various types of data that have been output or will be output.

[0085] In some embodiments, processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other chip. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data interaction or communication with computer device 10000. In this embodiment, processor 10020 is used to run program code stored in memory 10010 or process data.

[0086] Network interface 10030 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 10000 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0087] It should be pointed out that, Figure 7 Only computer devices with components 10010-10030 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0088] In this embodiment, the motion control method stored in memory 10010 can be further divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiments of this application.

[0089] Example 4 This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the motion control method in the embodiments.

[0090] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Of course, the computer-readable storage medium may include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the motion control method in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0091] Example 5 This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in the above embodiments.

[0092] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computer devices. They can be centralized on a single computer device or distributed across a network of multiple computer devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computer device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0093] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A motion control method, characterized in that, The method includes: Acquire the ontology perception data and environment perception data of the target object at the current moment, as well as a pre-trained policy network, which includes an encoder, a gated recurrent unit, and a multilayer perceptron. Based on the ontology perception data, ontology feature vectors are extracted, and the environment perception data is input into the encoder to extract spatial feature vectors; The ontological feature vector and the spatial feature vector are input into the gated loop unit to obtain the comprehensive feature vector; The integrated feature vector and the ontological feature vector are input into a multilayer perceptron to obtain motion control instructions, which are used to control the motion of the target object.

2. The method according to claim 1, characterized in that, The environmental perception data includes a distant depth map and a near depth map; the environmental perception data is input into the encoder to extract spatial feature vectors, including: If the distant depth map and the near depth map pass the verification, the distant depth map and the near depth map are preprocessed; wherein, the preprocessing includes at least one of downsampling, cropping, limiting and normalization; The preprocessed distant depth map and near depth map are input into the encoder to obtain the spatial feature vector.

3. The method according to claim 1, characterized in that, The ontological feature vector and the spatial feature vector are input into the gated loop unit to obtain a comprehensive feature vector, including: The ontology feature vector and the spatial feature vector are input into the gated loop unit so that the hidden state, the ontology feature vector and the spatial feature vector are fused by the gated loop unit to obtain the comprehensive feature vector; The hidden state is obtained by the gated loop unit based on the ontological feature vector and spatial feature vector at historical moments.

4. The method according to any one of claims 1-3, characterized in that, The policy network is obtained through the following operations: Acquire a training dataset, a target terrain height map, an initial policy network, and an evaluation network. The training dataset includes multiple training samples, each of which includes training ontology perception data, training environment perception data, and target motion control commands. The terrain height map is used to characterize the three-dimensional terrain structure of the environment in which the target object is located. The initial policy network includes an initial encoder, an initial gated recurrent unit, and an initial multilayer perceptron. Based on the training ontology perception data, training ontology feature vectors are extracted. The training environment perception data is input into the initial encoder to extract training space feature vectors. The training ontology feature vectors and the training space feature vectors are input into the initial gated loop unit to obtain a training comprehensive feature vector through the initial gated loop unit. The training integrated feature vector and the training ontology feature vector are input into the initial multilayer perceptron to obtain the predicted motion control command through the initial multilayer perceptron; The target terrain height map, the training ontology perception data, the predicted motion control command, and the target motion control command are input into the evaluation network to obtain the prediction value through the evaluation network. The loss data is determined based on the predicted value, and the initial policy network and the evaluation network are trained based on the loss data to obtain the policy network.

5. The method according to claim 4, characterized in that, The operation of obtaining the policy network also includes: The trained integrated feature vector is input into the decoder to reconstruct the predicted terrain height map. Correspondingly, determining the loss data based on the predicted value includes: The loss data is determined based on the predicted value, the target terrain elevation map, and the predicted terrain elevation map.

6. The method according to claim 5, characterized in that, The trained integrated feature vector and the ontology feature vector are input into an initial multilayer perceptron to obtain predicted motion control commands through the initial multilayer perceptron, including: The training integrated feature vector and the ontology feature vector are input into the initial multilayer perceptron to obtain the predicted probability distribution of the motion control command set through the initial multilayer perceptron, and the predicted motion control command is obtained by sampling the predicted probability distribution; wherein, the motion control command set includes multiple candidate motion control commands that can be executed by the target object; Correspondingly, based on the predicted value, the target terrain elevation map, and the predicted terrain elevation map, the loss data is determined, including: The loss data is determined based on the predicted value, the target terrain elevation map, the predicted terrain elevation map, and the predicted probability distribution.

7. The method according to claim 6, characterized in that, Based on the predicted value, the target terrain elevation map, the predicted terrain elevation map, and the predicted probability distribution, the loss data is determined, including: Based on the predicted probability distribution and the predicted value, the first loss data is determined; Based on the predicted value, the second loss data is determined; Based on the difference between the target terrain elevation map and the predicted terrain elevation map, a third loss data is determined; Based on the predicted probability distribution, the fourth loss data is determined; The loss data is determined based on the first loss data, the second loss data, the third loss data, and the fourth loss data.

8. A motion control device, characterized in that, The device includes: The acquisition module is used to acquire the ontology perception data and environment perception data of the target object at the current moment, as well as a pre-trained policy network, which includes an encoder, a gated recurrent unit, and a multilayer perceptron. The extraction module is used to extract ontological feature vectors based on the ontological perception data, and to input the environmental perception data into the encoder to extract spatial feature vectors. The fusion module is used to input the ontological feature vector and the spatial feature vector into the gated loop unit to obtain a comprehensive feature vector; The decision module is used to input the integrated feature vector and the ontological feature vector into the multilayer perceptron to obtain motion control instructions, which are used to control the motion of the target object.

9. A computer device, characterized in that, include: At least one processor; and A memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7.

Images