Robot motion control method and device, electronic equipment, storage medium and product

Abstract

This invention provides a robot motion control method, device, electronic device, storage medium, and product, relating to the field of robot control technology. It includes: using a trained robot motion control network to control the robot to switch gait based on movement speed and environmental terrain; the robot motion control network is trained based on motion capture data and depth images of various animals at different speeds, with the depth images obtained by a depth camera capturing images of different 3D terrain environments. This invention, through reinforcement learning training using motion capture data of various animals at different speeds, can achieve automatic dynamic gait switching based on speed thresholds, improving the naturalness of movement; by fusing depth images and motion capture data for learning training, it can achieve synergistic optimization of environmental perception and movement strategies, enhancing adaptability to complex terrain and significantly improving the robot's foot placement accuracy and body stability in different complex terrain environments.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a robot motion control method, device, electronic device, storage medium, and product. Background Technology

[0002] With the continuous development of technology, using reinforcement learning to mimic animal motion capture data is the mainstream technology for training legged robots to generate natural gait.

[0003] However, existing robot motion control methods generally suffer from the limitation of using only one motion mode. Current technologies typically use motion data of a single animal at a specific speed for training (such as imitating a dog's walking), which can easily lead to the robot's gait becoming fixed. When task requirements and task scenarios change, the robot struggles to achieve adaptive gait switching.

[0004] Therefore, how to achieve adaptive switching of robot gait when task requirements and task scenarios change, so as to improve the naturalness of robot movement and adaptability in different movement scenarios, is an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a robot motion control method, device, electronic device, storage medium, and product to address the shortcomings of existing technologies that rely solely on the motion data of a single animal at a specific speed for training, which can easily lead to robot gait fixation and make it difficult for the robot to adaptively switch gait when task requirements and scenarios change. The invention achieves dynamic automatic gait switching based on speed thresholds and collaborative optimization of environmental perception and motion strategies.

[0006] This invention provides a robot motion control method, comprising: A pre-trained robot motion control network is used to control the robot to switch gait modes based on movement speed and environmental terrain; The robot motion control network is trained on a reinforcement learning network based on motion capture data and depth images of various animals at different speeds. The depth images are captured by a depth camera for different three-dimensional terrain environments.

[0007] According to a robot motion control method provided by the present invention, the robot motion control network is trained in the following manner: Based on motion capture data of various animals at different speeds, biomimetic motion data is constructed. The biomimetic motion data is mapped onto the robot simulation model to generate motion trajectories for key parts. Based on the motion trajectory and depth images of the key parts, the reinforcement learning network is trained to obtain the robot motion control network.

[0008] According to a robot motion control method provided by the present invention, the step of training a reinforcement learning network based on the motion trajectory and depth image of the key parts to obtain the robot motion control network includes: The current state data of the robot simulation model is input into the policy network in the reinforcement learning network to obtain the joint angle data at the current moment; the current state data includes the current state data of the robot body and the depth image captured in real time by the depth camera integrated on the robot simulation model for the three-dimensional terrain environment. In a three-dimensional terrain environment, the robot simulation model is controlled to execute the joint angle data at the current moment and obtain the state data at the next moment. Based on the current state data of the robot body and the motion trajectory of the key parts, calculate the instantaneous reward; Based on the current state data, the current joint angle data, the next state data, the immediate reward, and the value function output by the evaluation network in the reinforcement learning network, the reinforcement learning network is iteratively optimized. After the iteration is completed, the final policy network is obtained, and the robot motion control network is determined based on the final policy network.

[0009] According to a robot motion control method provided by the present invention, the motion trajectory of the key part includes at least one of joint reference linear velocity and joint reference angular velocity, and the current state data of the robot body includes at least one of joint current linear velocity and joint current angular velocity; the step of calculating an instantaneous reward based on the current state data of the robot body and the motion trajectory of the key part includes: Based on the joint reference linear velocity and the joint's current linear velocity, determine that the linear velocity follows a reward; Based on the joint reference angular velocity and the joint's current angular velocity, the angular velocity is determined to follow the reward; Based on the linear velocity compliance reward and the angular velocity compliance reward, determine the control command compliance reward; The current joint angle data and the previous joint angle data of the robot simulation model are input into the discrimination network. The discrimination network scores the robot's current action based on the motion trajectory of the key parts, and obtains a biomimetic similarity reward. The immediate reward is calculated based on the control command compliance reward and the biomimetic similarity reward.

[0010] According to a robot motion control method provided by the present invention, the step of training a reinforcement learning network based on the motion trajectory and depth image of the key parts to obtain the robot motion control network includes: Obtain a set of simulated training environments containing multiple difficulty levels for complex terrains; The robot simulation model is initialized in a target simulation training environment with the lowest level of complex terrain training difficulty. The reinforcement learning network is trained based on the motion trajectory of the key parts and the depth images captured in real time by the depth camera integrated on the robot simulation model in the target simulation training environment. Based on the performance of the robot simulation model in the target simulation training environment, the difficulty level of the complex terrain training is adjusted until the performance reaches the preset performance standard, thus obtaining the robot motion control network.

[0011] According to a robot motion control method provided by the present invention, during the training of a reinforcement learning network, the method further includes: Domain randomization is applied to the target simulation training environment, and noise is injected into the state data observations obtained from the target simulation training environment.

[0012] The present invention also provides a robot motion control device, comprising: The control module is used to control the robot to switch gait based on movement speed and environmental terrain using a trained robot motion control network; The robot motion control network is trained on a reinforcement learning network based on motion capture data and depth images of various animals at different speeds. The depth images are captured by a depth camera for different three-dimensional terrain environments.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the robot motion control method described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robot motion control method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the robot motion control method as described above.

[0016] The robot motion control method, device, electronic device, storage medium, and product provided by this invention train a reinforcement learning network by collecting motion capture data of various animals at different speeds, enabling the robot to achieve automatic dynamic gait switching based on speed thresholds and improve the naturalness of movement. By training the reinforcement learning network based on motion capture data of various animals at different speeds and depth images obtained by depth cameras for different three-dimensional terrain environments, the collaborative optimization of environmental perception and motion strategy can be achieved, thereby enhancing the adaptability to complex terrain and significantly improving the accuracy of foot landing and body stability of the robot in different complex terrain environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts of the robot motion control method provided in the embodiments of the present invention.

[0019] Figure 2 This is the second flowchart of the robot motion control method provided in the embodiments of the present invention.

[0020] Figure 3 This is a schematic diagram of the robot motion control device provided in an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] In the description of embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Those skilled in the art will understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0024] Figure 1 This is one of the flowcharts illustrating the robot motion control method provided in this embodiment of the invention. (Refer to...) Figure 1 This invention provides a robot motion control method, which may specifically include the following steps: Step 101: Using a trained robot motion control network, the robot is controlled to switch gait based on movement speed and environmental terrain; wherein, the robot motion control network is trained on a reinforcement learning network based on motion capture data and depth images of various animals at different speeds, and the depth images are captured by a depth camera for different three-dimensional terrain environments.

[0025] It should be noted that the execution subject of the robot motion control method provided in this embodiment of the invention can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., while a non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This embodiment of the invention does not specifically limit these. The following embodiments of the invention describe the execution subject as a robot.

[0026] In this embodiment of the invention, motion capture data of various animals at different speeds can be collected using a high-precision motion capture system based on the movement characteristics of different real animals (e.g., motion capture data of a cat crawling forward and motion capture data of a dog running). This data is then used to train a reinforcement learning network to obtain a robot motion control network, enabling the robot to adaptively switch between different gaits based on different movement speeds. The motion capture data may include motion data such as gait cycle, joint angles, and foot trajectory.

[0027] Compared to existing technologies that rely solely on motion capture data from a single animal and cannot dynamically adjust gait, this invention collects motion capture data from multiple animals at different speeds and designs a dynamic gait automatic switching mechanism based on speed thresholds. For example, in low-speed scenarios, the robot is controlled to adopt a cat-like crawling gait (small joint movement amplitude, soft foot contact), significantly reducing motion noise; in high-speed scenarios, the robot switches to a dog-like running gait (large joint swing amplitude, high stride frequency), improving motion efficiency and biomimetic similarity; and in complex terrain scenarios, it switches to an animal jumping posture, etc. This invention dynamically integrates multiple biomimetic motion data, enabling adaptive gait switching and improving the robot's motion naturalness and noise control.

[0028] It should be noted that different robot motion control networks can be trained using motion capture data from animals of different shapes and sizes. For example, a quadruped robot can be trained using reinforcement learning based on motion capture data from various real animals, such as a cat crawling or a dog running, enabling it to naturally mimic these movements. Similarly, an underwater robot can be trained using motion capture data from various real animals, such as a snake wriggling or a fish wagging its tail, enabling it to naturally mimic these movements. This invention does not impose specific limitations.

[0029] In some embodiments, during the training of the reinforcement learning network, a robot simulation model can be constructed, and a depth camera simulation module can be integrated on the robot simulation model to simulate depth image input in real-world scenarios. The reinforcement learning network is trained based on motion capture data and depth images of various animals at different speeds, thereby achieving interaction between visual perception and robot motion control. In practical applications, the robot device can carry a depth camera, which can acquire depth images of the surrounding environment and terrain. Through a trained robot motion control algorithm, the robot can adaptively switch its gait based on movement speed and environmental terrain.

[0030] In some embodiments, reinforcement learning networks can be trained in different three-dimensional terrain simulation environments, thereby enabling robots to adaptively switch gait based on different terrain environments in practical applications.

[0031] This invention trains a reinforcement learning network by collecting motion capture data of various animals at different speeds, enabling the robot to automatically switch dynamic gait based on speed thresholds and improve the naturalness of movement. By training the reinforcement learning network based on motion capture data of various animals at different speeds and depth images captured by a depth camera for different 3D terrain environments, the collaborative optimization of environmental perception and motion strategy can be achieved, thereby enhancing adaptability to complex terrain and significantly improving the accuracy of foot landing and body stability of the robot in different complex terrain environments.

[0032] Based on any of the above embodiments, the robot motion control network can be trained through the following steps: Step S11: Construct biomimetic motion data based on motion capture data of various animals at different speeds; Step S12: Map the biomimetic motion data onto the robot simulation model to generate motion trajectories for key parts; Step S13: Based on the motion trajectory and depth image of the key parts, train the reinforcement learning network to obtain the robot motion control network.

[0033] In some embodiments, motion capture data of various motion modalities such as walking, running, and jumping at different speeds can be collected using a high-precision motion capture system, based on the movement characteristics of different real animals (such as a cat crawling, a dog running, and an antelope jumping). Then, the motion capture data can be processed by filtering, normalization, and noise reduction to form structured biomimetic motion data. Filtering, normalization, and noise reduction help ensure smooth and reliable data, enabling the robot to realistically reflect biological movement.

[0034] This invention collects motion capture data of various animals (such as cats and dogs) at different speeds to construct a multimodal bionic database, covering multiple movement modes such as low-speed silent gait, high-speed running gait, and jumping. It can realize dynamic fusion of multi-animal motion capture data and an adaptive gait switching mechanism.

[0035] In some embodiments, a URDF (Unified Robot Description Format) simulation model of the robot can be constructed, with precise configuration of equipment parameters such as body mass, joint drive parameters, and contact mechanical properties. The robot simulation model can integrate a depth camera simulation module to simulate depth image input in real-world scenarios, thereby enabling interaction between visual perception and motion control.

[0036] In some embodiments, motion redirection technology can be used to map the acquired motion capture data to a robot simulation model. Through joint mapping and inverse kinematics calculation, the motion adaptation problem caused by the difference between biological morphology and mechanical structure can be solved, and motion trajectories of key parts can be generated. Based on the motion trajectories of key parts and depth images, reinforcement learning networks can be trained.

[0037] For example, considering the quadruped robot's limited 12 degrees of freedom and differences in joint characteristics, a keypoint-plus-inverse kinematics approach can be used for motion relocalization. Specifically, the motion capture data can be scaled to match the quadruped robot's proportions; then, the keypoint positions and necessary rotational information can be extracted from the source motion capture data, and the inverse kinematics equations can be solved to obtain the joint angle values ​​for each joint; finally, the motion can be further optimized by trimming distances exceeding the quadruped robot's leg extension range and fine-tuning the posture, thereby ensuring that the final motion applied to the quadruped robot is natural and stable.

[0038] This invention constructs biomimetic motion data based on motion capture data of various animals at different speeds, maps the biomimetic motion data to a robot simulation model, generates motion trajectories of key parts, and trains a reinforcement learning network based on the motion trajectories and depth images of key parts. This can solve the motion adaptation problem caused by the difference between biological morphology and mechanical structure, and ensure that the final motion applied to the quadruped robot is natural and stable.

[0039] Based on any of the above embodiments, training the reinforcement learning network based on the motion trajectory and depth image of the key parts to obtain the robot motion control network may specifically include: Step S131: Input the current state data of the robot simulation model into the policy network in the reinforcement learning network to obtain the joint angle data at the current moment; the current state data includes the current state data of the robot body and the depth image captured in real time by the depth camera integrated on the robot simulation model for the three-dimensional terrain environment. Step S132: In the three-dimensional terrain environment, control the robot simulation model to execute the joint angle data at the current moment and obtain the state data at the next moment; Step S133: Calculate the instant reward based on the current state data of the robot body and the motion trajectory of the key parts; Step S134: Based on the current state data, the current joint angle data, the next state data, the immediate reward, and the value function output by the evaluation network in the reinforcement learning network, the reinforcement learning network is iteratively optimized. After the iteration is completed, the final policy network is obtained, and the robot motion control network is determined based on the final policy network.

[0040] In this embodiment of the invention, taking a quadruped robot as an example, the motion control of the quadruped robot can be modeled as a Markov Decision Process (MDP), and its core elements can be defined as follows: The state space S can include the current motion state of the quadruped robot and its interaction information with the environment. Specifically, the state can include the robot's joint angles, joint angular velocities, body angles, body acceleration, historical information, depth images, etc.

[0041] Motion space A: This space defines the joint angles of the robot's 12 joints. The joint angle information of these 12 joints is sent to the proportional-derivative controller (PD) to enable the robot to perform the corresponding actions.

[0042] State transition function P(s) t+1 |s t ,a t ): Describes the state s at the current moment. t Next, execute action a t Then, how does the system transition to the next state s? t+1 This process is simulated in a physical simulation environment, reflecting the robot's dynamic characteristics and its interaction with the environment.

[0043] In some embodiments, the current state data of the robot body may include data such as joint angles, joint angular velocities, joint linear velocities, body angles, body acceleration, and historical information of the robot at the current moment.

[0044] In some embodiments, the three-dimensional terrain environment may be a simulated three-dimensional terrain environment constructed for robot motion control training.

[0045] In this embodiment of the invention, the reinforcement learning network may include a policy network and an evaluation network. After inputting the current state data of the robot simulation model into the policy network to obtain the current joint angle data, the robot simulation model can be controlled in a 3D terrain environment to execute the current joint angle data. The next state data fed back from the physical simulation environment is simulated through a state transition function. Based on the current state data of the robot body and the motion trajectory of key parts, the immediate reward is calculated. Finally, based on the current state data, the current joint angle data, the next state data, the immediate reward, and the value function output by the evaluation network in the reinforcement learning network, the reinforcement learning network is iteratively optimized to obtain the final policy network after iteration.

[0046] In some embodiments, a robot motion control network can be deployed on the robot based on the parameters of the final policy network, thereby controlling the robot to adaptively switch between different gaits based on movement speed and environmental terrain.

[0047] Compared to existing technologies that rely on phased training (teacher network → student network) leading to biases, this invention proposes an end-to-end bionic-vision joint training framework. It acquires 3D environmental information in real-time using a depth camera, directly fusing this information with bionic motion data and inputting it into a reinforcement learning network for training. This achieves end-to-end fusion of bionic data and depth vision, enabling coordinated optimization of environmental perception and motion strategies. This significantly improves the robot's foot placement accuracy and body stability in complex terrains. This invention, through end-to-end fusion of bionic data and depth vision, enhances the robot's adaptability to complex terrain.

[0048] Based on any of the above embodiments, the motion trajectory of the key part includes at least one of joint reference linear velocity and joint reference angular velocity, and the current state data of the robot body includes at least one of joint current linear velocity and joint current angular velocity; the calculation of the instantaneous reward based on the current state data of the robot body and the motion trajectory of the key part may specifically include: Step S1331: Based on the joint reference linear velocity and the joint current linear velocity, determine that the linear velocity follows the reward. Step S1332: Based on the joint reference angular velocity and the joint's current angular velocity, determine whether the angular velocity follows the reward. Step S1333: Determine the control command compliance reward based on the linear velocity compliance reward and the angular velocity compliance reward; Step S1334: Input the current joint angle data and the previous joint angle data of the robot simulation model into the discrimination network. The discrimination network scores the robot's current action based on the motion trajectory of the key parts to obtain a biomimetic similarity reward. Step S1335: Calculate the immediate reward based on the control command compliance reward and the biomimetic similarity reward.

[0049] In this embodiment of the invention, a multi-objective reinforcement learning framework can be designed, and a biomimetic similarity reward can be added on the basis of ordinary reinforcement learning motion control, thereby simultaneously optimizing the robot's flat ground motion control and biomimetic motion fitting ability.

[0050] In some embodiments, the reward function of a reinforcement learning network can be defined as follows: ; ; in, To control the rewards for following instructions, For linear velocity to follow the reward, For angular velocity to follow the reward, Rewards for biomimetic similarity For instant rewards, , These are the weight values ​​for control instruction compliance rewards and biomimetic similarity rewards, respectively.

[0051] In some embodiments, the motion trajectory of a critical part may include motion trajectory reference parameters of at least one joint constituting the critical part at multiple consecutive moments. These motion trajectory reference parameters may include linear velocity, angular velocity, etc. A linear velocity compliance reward can be determined based on the joint reference linear velocity in the motion trajectory of the critical part and the joint's current linear velocity, and an angular velocity compliance reward can be determined based on the joint reference angular velocity in the motion trajectory of the critical part and the joint's current angular velocity.

[0052] In some embodiments, the biomimetic similarity reward can be defined as follows: ; Where q t-1 q t These can represent the robot's joint angle data at the current and previous moments, respectively; π sim It is a discriminant network that is trained synchronously during the training process. It is used to score the robot's current action, so that the biomimetic similarity reward is continuously increased during training. That is, during the training process, the robot's movement style is made closer to the motion capture data.

[0053] To enable quadruped robots to mimic various gait characteristics from slow walking to fast running, this invention employs a dynamic gait automatic switching mechanism based on speed thresholds. This mechanism can fit different gaits of animal movement at different speeds through biomimetic similarity rewards, facilitating adaptive gait switching and improving the naturalness of robot movement and noise control. For example, it can fit a crawling gait at speeds below 0.2 m / s and a running gait at speeds between 1.5 m / s and 2 m / s.

[0054] Based on any of the above embodiments, training the reinforcement learning network based on the motion trajectory and depth image of the key parts to obtain the robot motion control network may specifically include: Step S21: Obtain a set of simulation training environments containing multiple difficulty levels for complex terrain training. Step S22: Initialize the robot simulation model in the target simulation training environment with the lowest level of complex terrain training difficulty, and train the reinforcement learning network based on the motion trajectory of the key parts and the depth images captured in real time by the depth camera integrated on the robot simulation model in the target simulation training environment. Step S23: Based on the performance of the robot simulation model in the target simulation training environment, adjust the difficulty level of the complex terrain training until the performance reaches the preset performance standard, and obtain the robot motion control network.

[0055] In this embodiment of the invention, after the basic motion strategy training is completed, the complexity of the training environment can be gradually increased by introducing a course learning mechanism, thereby enhancing the robot's generalization ability and dynamic adaptability in varied and complex terrain.

[0056] In some embodiments, multiple progressive training difficulty levels can be designed for each type of complex terrain, thereby constructing a set of simulation training environments containing multiple complex terrain training difficulty levels. The robot simulation model can initially learn in the simplest terrain environment (i.e., the lowest complex terrain training difficulty level). Once it can stably complete the target task, the training difficulty level of the complex terrain can be gradually increased. If the performance of the robot simulation model at the current difficulty level does not meet expectations, the training difficulty can be appropriately reduced to ensure learning stability until the robot's performance reaches the preset performance standard, thus obtaining the robot motion control network and enhancing the robot's generalization ability and dynamic adaptability in varied and complex terrains.

[0057] In some embodiments, during training on complex terrain, both visual perception and robot body state can be used simultaneously. The visual perception component can preprocess and encode features from depth images acquired by a depth camera. Its update frequency is relatively low compared to the body state, allowing it to capture global structural changes in the terrain and reducing computational resource requirements during deployment. Body state data, on the other hand, can include information such as joint angles, joint angular velocities, body angles, body acceleration, and historical actions. Its encoding and update frequency is high, ensuring that motion control responds sufficiently sensitively to the body's dynamic state.

[0058] In some embodiments, the biomimetic features of the 3D environment information acquired in real time by the depth camera and the motion capture data can be synchronously input into the reinforcement learning network. This allows for automatic gait switching based on terrain (walking, running, climbing, etc.) by combining real-time speed and environmental results, in addition to speed-based automatic gait switching. Through this collaborative processing of multimodal perception information at different time scales, the reinforcement learning strategy can fully perceive complex terrain and changes in body state while ensuring real-time performance, thereby improving the robot's foot placement accuracy and body stability.

[0059] Based on any of the above embodiments, during the training of the reinforcement learning network, the method may further include: applying domain randomization processing to the target simulation training environment and injecting noise into the state data observations obtained from the target simulation training environment.

[0060] In some embodiments, in order to reduce the difference between the simulation environment and the real environment during deployment, domain randomization can be used and noise can be added to the state space during training, so that hardware deployment can be carried out directly.

[0061] Compared to existing technologies that rely on Experience-Driven Actuator Models (EAMs) and multi-loss function optimization but lack environmental awareness, this invention uses a depth camera to perceive 3D terrain information in real time and combines this with domain randomization techniques to significantly reduce the perceptual discrepancy between simulation and reality. In real, complex terrain, the robot can dynamically adjust its foot placement and joint torques based on visual information, avoiding motion failures caused by inaccurate simulation modeling. This visual perception-based simulation-reality transfer optimization improves the robustness of deployment in real-world environments.

[0062] Figure 2 This is the second flowchart illustrating the robot motion control method provided in this embodiment of the invention. (Refer to...) Figure 2 In one specific embodiment, the present invention can first collect and process motion capture data of various animals at different speeds to obtain biomimetic motion data; then, a quadruped robot simulation model can be constructed, and motion redirection technology can be used to map the biomimetic motion data to the robot simulation model to generate motion trajectories of key parts; subsequently, the motion control of the quadruped robot can be modeled as a Markov decision process, and the robot's flat-ground motion control ability can be trained through reinforcement learning. At the same time, motion capture data can be fitted, and complex terrain can be added to train the robot's ability to jump obstacles, climb stairs, etc., thereby enhancing the robot's generalization ability and dynamic adaptability in varied terrains; furthermore, domain randomization is used and noise is added to the state space during the training process, and the hardware can be directly deployed after training is completed.

[0063] This invention, based on visual perception and biomimetic data fusion technology, achieves synergistic optimization of biomimetic motion strategies and adaptability to complex terrain through an end-to-end joint training framework. It extensively collects motion capture data from various animals at different speeds, uniformly redirects the motion, and maps motion capture data from animals of different sizes to a quadruped robot. This ensures the quadruped robot can mimic the various gait characteristics of these animals, from slow walking to fast running. Specifically, in low-speed motion scenarios, the robot can mimic the slow walking gait of animals, which not only reduces the impact of noise but also improves motion stability. In high-speed motion, the robot can simulate the running posture of animals, thereby enhancing its dynamic performance and biomimetic realism. This invention also employs a phased training strategy. First, basic walking ability is trained using flat terrain. After the basic walking ability training is completed, complex terrain such as obstacle crossing and stairs are added. Combined with visual perception provided by a depth camera, complex walking ability is trained. Domain randomization technology is used to achieve algorithm transfer to a real robot.

[0064] In summary, the embodiments of the present invention have the following technical advantages: (1) By introducing a depth camera to acquire three-dimensional environmental information in real time, and combining it with bionic motion data to optimize vision-motion collaborative control, the robot’s ability to autonomously avoid obstacles and plan paths in dynamic scenes is enhanced; motion capture data of different animals are applied at different speeds to achieve gaits such as crawling silently at low speeds and running at high speeds.

[0065] (2) By combining biomimetic motion data with deep reinforcement learning, a multimodal reward mechanism driven by visual perception (such as terrain feature matching reward and foot landing point stability reward) is designed to improve the generalization ability of motion strategies in real environment.

[0066] (3) Abandoning the traditional step-by-step training model of teacher-student networks, a single-stage end-to-end joint training framework is proposed. By directly fusing biomimetic motion data with depth camera perception information into the reinforcement learning network, a biomimetic-vision joint encoder is designed to generate a control strategy that combines biomimetic motion characteristics and environmental adaptability in one step. This method eliminates the knowledge transfer bias from the teacher network to the student network. At the same time, by dynamically fusing biomimetic gait templates (such as animal foot trajectory and joint coordination) with real-time visual information, the strategy is directly optimized in real-world scenarios, significantly improving the robustness and terrain adaptability of motion control.

[0067] The robot motion control device provided by the present invention is described below. The robot motion control device described below can be referred to in correspondence with the robot motion control method described above.

[0068] Figure 3 This is a schematic diagram of the robot motion control device provided in an embodiment of the present invention. (Refer to...) Figure 3 This invention provides a robot motion control device, which may specifically include the following modules: The control module 310 is used to control the robot to switch gait based on movement speed and environmental terrain using a trained robot motion control network. The robot motion control network is trained on a reinforcement learning network based on motion capture data and depth images of various animals at different speeds. The depth images are captured by a depth camera for different three-dimensional terrain environments.

[0069] Based on any of the above embodiments, the robot motion control network is trained through the following modules: The module is used to construct biomimetic motion data based on motion capture data of various animals at different speeds. The generation module is used to map the biomimetic motion data onto the robot simulation model and generate motion trajectories for key parts. The training module is used to train the reinforcement learning network based on the motion trajectory and depth image of the key parts, so as to obtain the robot motion control network.

[0070] Based on any of the above embodiments, the training module includes: The state data input submodule is used to input the current state data of the robot simulation model into the policy network in the reinforcement learning network to obtain the joint angle data at the current moment; the current state data includes the current state data of the robot body and the depth image captured in real time by the depth camera integrated on the robot simulation model for the three-dimensional terrain environment. The execution control submodule is used to control the robot simulation model to execute the joint angle data at the current moment and obtain the state data at the next moment in a three-dimensional terrain environment. The reward calculation submodule is used to calculate the instant reward based on the current state data of the robot body and the motion trajectory of the key parts; The iterative optimization submodule is used to iteratively optimize the reinforcement learning network based on the current state data, the current joint angle data, the next state data, the immediate reward, and the value function output by the evaluation network in the reinforcement learning network. After the iteration is completed, the final policy network is obtained, and the robot motion control network is determined based on the final policy network.

[0071] Based on any of the above embodiments, the motion trajectory of the key parts includes at least one of joint reference linear velocity and joint reference angular velocity, and the current state data of the robot body includes at least one of joint current linear velocity and joint current angular velocity; the reward calculation submodule includes: The linear velocity follow-reward determination unit is used to determine the linear velocity follow-reward based on the joint reference linear velocity and the current linear velocity of the joint. An angular velocity follow-reward determination unit is used to determine the angular velocity follow-reward based on the joint reference angular velocity and the joint's current angular velocity. A control command compliance reward determination unit is used to determine the control command compliance reward based on the linear velocity compliance reward and the angular velocity compliance reward; The biomimetic similarity reward determination unit is used to input the current joint angle data and the previous joint angle data of the robot simulation model into the discrimination network, and the discrimination network scores the robot's current action based on the motion trajectory of the key parts to obtain the biomimetic similarity reward. An instant reward calculation unit is used to calculate an instant reward based on the control instruction compliance reward and the biomimetic similarity reward.

[0072] Based on any of the above embodiments, the training module includes: The training environment set acquisition submodule is used to acquire a set of simulation training environments containing multiple difficulty levels of complex terrains; The initialization training submodule is used to initialize the robot simulation model in a target simulation training environment with the lowest level of complex terrain training difficulty. Based on the motion trajectory of the key parts and the depth images captured in real time by the depth camera integrated on the robot simulation model in the target simulation training environment, the reinforcement learning network is trained. The training difficulty adjustment submodule is used to adjust the training difficulty level of complex terrain based on the performance of the robot simulation model in the target simulation training environment until the performance reaches the preset performance standard, thereby obtaining the robot motion control network.

[0073] Based on any of the above embodiments, during the training of the reinforcement learning network, the apparatus further includes: The randomization processing module is used to apply domain randomization processing to the target simulation training environment and inject noise into the state data observations obtained from the target simulation training environment.

[0074] This invention trains a reinforcement learning network by collecting motion capture data of various animals at different speeds, enabling the robot to automatically switch dynamic gait based on speed thresholds and improve the naturalness of movement. By training the reinforcement learning network based on motion capture data of various animals at different speeds and depth images captured by a depth camera for different 3D terrain environments, the collaborative optimization of environmental perception and motion strategy can be achieved, thereby enhancing adaptability to complex terrain and significantly improving the accuracy of foot landing and body stability of the robot in different complex terrain environments.

[0075] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440. The processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions from the memory 430 to execute a robot motion control method. This method includes using a trained robot motion control network to control the robot to switch gait patterns based on movement speed and environmental terrain. The robot motion control network is trained on a reinforcement learning network based on motion capture data and depth images of various animals at different speeds. The depth images are captured by a depth camera for different three-dimensional terrain environments.

[0076] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0077] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the robot motion control method provided by the above methods. The method includes: using a trained robot motion control network to control the robot to switch gait based on motion speed and environmental terrain; wherein the robot motion control network is obtained by training a reinforcement learning network based on motion capture data and depth images of various animals at different speeds, and the depth images are captured by a depth camera for different three-dimensional terrain environments.

[0078] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the robot motion control method provided by the above methods. The method includes: using a trained robot motion control network to control the robot to switch gait based on motion speed and environmental terrain; wherein the robot motion control network is obtained by training a reinforcement learning network based on motion capture data and depth images of various animals at different speeds, and the depth images are obtained by a depth camera for different three-dimensional terrain environments.

[0079] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0080] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot motion control method, characterized in that, include: A pre-trained robot motion control network is used to control the robot to switch gait modes based on movement speed and environmental terrain; The robot motion control network is trained on a reinforcement learning network based on motion capture data and depth images of various animals at different speeds. The depth images are captured by a depth camera for different three-dimensional terrain environments.

2. The robot motion control method according to claim 1, characterized by, The robot motion control network was trained in the following way: Based on motion capture data of various animals at different speeds, biomimetic motion data is constructed. The biomimetic motion data is mapped onto the robot simulation model to generate motion trajectories for key parts. Based on the motion trajectory and depth images of the key parts, the reinforcement learning network is trained to obtain the robot motion control network.

3. The robot motion control method according to claim 2, wherein, The process of training a reinforcement learning network based on the motion trajectory and depth image of the key parts to obtain the robot motion control network includes: The current state data of the robot simulation model is input into the policy network in the reinforcement learning network to obtain the joint angle data at the current moment; the current state data includes the current state data of the robot body and the depth image captured in real time by the depth camera integrated on the robot simulation model for the three-dimensional terrain environment. In a three-dimensional terrain environment, the robot simulation model is controlled to execute the joint angle data at the current moment and obtain the state data at the next moment. Based on the current state data of the robot body and the motion trajectory of the key parts, calculate the instantaneous reward; Based on the current state data, the current joint angle data, the next state data, the immediate reward, and the value function output by the evaluation network in the reinforcement learning network, the reinforcement learning network is iteratively optimized. After the iteration is completed, the final policy network is obtained, and the robot motion control network is determined based on the final policy network.

4. The robot motion control method according to claim 3, characterized by, The motion trajectory of the key parts includes at least one of the joint reference linear velocity and the joint reference angular velocity, and the current state data of the robot body includes at least one of the joint current linear velocity and the joint current angular velocity. The calculation of the instant reward based on the current state data of the robot body and the motion trajectory of the key parts includes: Based on the joint reference linear velocity and the joint's current linear velocity, determine that the linear velocity follows a reward; Based on the joint reference angular velocity and the joint's current angular velocity, the angular velocity is determined to follow the reward; Based on the linear velocity compliance reward and the angular velocity compliance reward, determine the control command compliance reward; The current joint angle data and the previous joint angle data of the robot simulation model are input into the discrimination network. The discrimination network scores the robot's current action based on the motion trajectory of the key parts, and obtains a biomimetic similarity reward. The immediate reward is calculated based on the control command compliance reward and the biomimetic similarity reward.

5. The robot motion control method according to claim 2, wherein, The process of training a reinforcement learning network based on the motion trajectory and depth image of the key parts to obtain the robot motion control network includes: Obtain a set of simulated training environments containing multiple difficulty levels for complex terrains; The robot simulation model is initialized in a target simulation training environment with the lowest level of complex terrain training difficulty. The reinforcement learning network is trained based on the motion trajectory of the key parts and the depth images captured in real time by the depth camera integrated on the robot simulation model in the target simulation training environment. Based on the performance of the robot simulation model in the target simulation training environment, the difficulty level of the complex terrain training is adjusted until the performance reaches the preset performance standard, thus obtaining the robot motion control network.

6. The robot motion control method according to claim 5, wherein, The method further includes the following steps during the training of the reinforcement learning network: Domain randomization is applied to the target simulation training environment, and noise is injected into the state data observations obtained from the target simulation training environment.

7. A robot motion control apparatus characterized by comprising: include: The control module is used to control the robot to switch gait based on movement speed and environmental terrain using a trained robot motion control network; The robot motion control network is trained on a reinforcement learning network based on motion capture data and depth images of various animals at different speeds. The depth images are captured by a depth camera for different three-dimensional terrain environments.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the robot motion control method as described in any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot motion control method as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot motion control method as described in any one of claims 1 to 6.

Images