Interaction system of quadruped robot, planning control method and medium

Abstract

The invention relates to the technical field of robots, in particular to an interactive system of a quadruped robot, a planning control method and a medium, and the system comprises a simulation training system which is used for simulating one or more interactive simulation scenes of the quadruped robot and iteratively training a planning control strategy of the quadruped robot in the one or more interactive simulation scenes, the planning control strategy is optimized until training is completed, and the optimized planning control strategy is obtained; one or more quadruped robots are deployed with the optimized planning control strategy, and each quadruped robot executes an interaction task in a real interaction scene based on the optimized planning control strategy. Therefore, the problems that a traditional control algorithm is difficult to achieve design and development of high-complexity actions, cannot adapt to different physical environments, and is poor in generalization and interactivity and the like in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the field of robotics, and in particular to an interactive system, a planning and control method, and a medium for a quadruped robot. Background Art

[0002] For a long time, human control over machines has been based on manually designed and constructed rule commands, which can be applied to relatively simple task scenarios. However, with the complexity of task scenarios, the complexity of the required rule control system has also greatly increased, becoming too cumbersome and difficult to adapt to unstructured and changing environments. For example, the quadruped robot motion control planning focused on in this application is different from the wheeled robot that can achieve normal movement by simply performing differential control of the motor. The legged robot involves motion posture problems and needs to solve the control parameters required for each joint motor to achieve different actions. Even simple actions require extremely tedious and time-consuming design and testing processes. When faced with highly dynamic and complex tasks such as target tracking, the development of motion planning and control has always been a problem to be solved.

[0003] Reinforcement learning combined with virtual simulation provides a new and effective solution to this problem. Reinforcement learning enables intelligent agents to continuously optimize and iterate their strategies by constantly interacting with the environment with the expected reward as the goal. It has achieved amazing results in solving many tasks in artificially modeled scenarios. For example, in competitive games such as chess, it can achieve a competitive level far beyond that of humans through parallel multi-agent interactive training.

[0004] In related technologies, most reinforcement learning control strategies can only complete relatively basic stepping tasks. When it comes to high-level tasks such as target tracking, such as kicking a ball, it is difficult to parse and map complex task objectives into reward models. There are still differences between the virtual simulation environment and the real physical environment. In low-level basic motion posture control tasks, parameters such as motor position and torque can be manually adjusted. However, when the task becomes complicated, it is impossible to adapt to the differences between the real physical environment and the simulation environment to achieve better results, and it is impossible to make the strategy adapt to a variety of environments with different physical conditions. Summary of the invention

[0005] The present application provides an interactive system, a planning and control method, a quadruped robot and a storage medium for a quadruped robot, so as to solve the problems in the related art that traditional control algorithms are difficult to realize the design and development of highly complex actions, cannot adapt to different physical environments, have poor generalization and interactivity, and the like.

[0006] The first aspect of the present application provides an interactive system for a quadruped robot, comprising the following steps: a simulation training system for simulating one or more interactive simulation scenarios of the quadruped robot, iteratively training the planning and control strategy of the quadruped robot in the one or more interactive simulation scenarios to optimize the planning and control strategy until the training is completed to obtain the optimized planning and control strategy; deploying one or more quadruped robots with the optimized planning and control strategy, wherein each quadruped robot performs an interactive task in a real interactive scenario based on the optimized planning and control strategy.

[0007] Optionally, the simulation training system includes: an environment construction module, which is used to randomly simulate environmental parameters and / or the quadruped robot's own parameters to generate interactive simulation scenes suitable for different interactive tasks, and simulate the simulation target of the interactive task in the interactive simulation scene, and set the motion mode of the simulation target; a training module, which is used to iteratively train the planning control strategy through a reinforcement learning strategy. In each iterative training process, the planning control strategy is used to control the simulation target to perform the interactive task in the interactive simulation scene, and the planning control strategy is adjusted according to the result of each iterative training until the iterative training stop condition is met to obtain the optimized planning control strategy.

[0008] Optionally, the environmental parameters include one or more of friction coefficient, ground contact elasticity coefficient and terrain, the self parameters include joint damping and / or body load weight of the quadruped robot, and the motion mode includes one or more of static mode, alternating reciprocating motion mode and random motion mode.

[0009] Optionally, the simulation training system also includes: an action space, which is used to determine the action parameters of the control layer and the decision layer, wherein the control layer is used to select different action parameters to adjust the control mode of the quadruped robot; the decision layer controls the body movement of the quadruped robot by transmitting a triplet to the control layer; an observation state space, which observes the states of the control layer and the decision layer by setting state space characteristics, wherein the control layer learns timing characteristics based on historical data and adjusts the planning control strategy according to different state sequences; the decision layer adds the characteristics of the yaw error and distance between the virtual target and the target position; a reward function of the control layer, wherein the reward function includes a decision reward and a control reward, the control reward is used to quantify the degree of fit between the quadruped robot and the speed control command, and the decision reward is used to quantify the posture optimization-related indicators of the quadruped robot.

[0010] Optionally, the triplet includes a forward velocity, a tangential velocity, and a counterclockwise turning velocity around a vertical ground axis in the quadruped robot's own world coordinate system.

[0011] Optionally, the degree of fit includes linear velocity deviations in the x and y directions and angular velocity deviations in the z-axis direction, and the posture optimization-related indicators include one or more of motion amplitude, basic height, super-mechanical limit penalty, torque limit, foot air time, collision penalty and interruption penalty.

[0012] Optionally, the quadruped robot includes: a visual perception system, used to perceive the environmental information around the quadruped robot and identify the task objectives in the environmental information; a strategy execution system, deployed with the optimized planning control strategy, planning interactive actions according to the position of the quadruped robot and the position of the task objectives, and generating control instructions, and using the control instructions to control the quadruped robot to perform the interactive actions.

[0013] The second aspect of the present application provides a planning and control method for a quadruped robot, wherein the quadruped robot is deployed with an optimized planning and control strategy generated by the interactive system of the quadruped robot as described in the above embodiment, wherein the method comprises the following steps: sensing environmental information around the quadruped robot; identifying the task objectives of the interactive tasks in the environmental information; using the optimized planning and control strategy to plan the interactive action of the quadruped robot from the current position to the task target position, and generating control instructions, and using the control instructions to control the quadruped robot to perform the interactive action.

[0014] The third aspect of the present application provides a quadruped robot, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the planning and control method of the quadruped robot as described in the above embodiment.

[0015] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the planning and control method of the quadruped robot as described in the above embodiment.

[0016] Therefore, this application has at least the following beneficial effects:

[0017] The embodiment of the present application utilizes a variety of parallel simulation environments to quickly optimize and iterate the planning and control strategies to adapt to the complex planning and control in different environments, and deploys them on a quadruped robot for verification and testing, so that the quadruped robot has the ability to track highly dynamic targets. The modular design of the quadruped robot's control system and the plasticity of the training system enable it to be expanded and developed according to different needs in the future, so that it has the ability to understand and execute complex tasks and meet the needs of generalization and interactivity.

[0018] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0020] Figure 1 A schematic diagram of an interactive system of a quadruped robot provided according to an embodiment of the present application;

[0021] Figure 2 A framework diagram of a reinforcement learning strategy training system provided according to an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a real scene rendered for simulation environment training according to an embodiment of the present application;

[0023] Figure 4 A diagram of a real machine network execution architecture provided according to an embodiment of the present application;

[0024] Figure 5 A demonstration diagram of interactively kicking a ball while chasing the ball provided in an embodiment of the present application;

[0025] Figure 6 A flowchart of a planning and control method for a quadruped robot provided according to an embodiment of the present application;

[0026] Figure 7 Schematic diagram of the structure of a quadruped robot provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0028] At present, the problems that reinforcement learning focuses on solving are mainly concentrated in artificially constructed virtual task scenarios, such as the aforementioned Go and video games. The relevant influencing variables involved in the solution space can be set and controlled manually, and can be reliably observed, calculated and transmitted. When it comes to problems in the real physical world, the influencing factors that need to be considered include not only the electromechanical and sensor characteristics of the intelligent body's own physical carrier, but also the gravity, friction, elasticity and other factors of the real physical world; and for a system with low stability such as a quadruped robot, the hardware loss cost of directly conducting reinforcement learning exploration training in a real scene is also unbearable, and the advantage of rapid iteration of parallel computing will also be lost.

[0029] In order to solve this problem, a sim-to-real reinforcement learning training deployment framework was proposed. First, a virtual simulation environment was built in batches on the computer for parallel training. Then, when deployed on the real machine, optimization and adjustment were made based on the effect of the pre-training strategy and the environment.

[0030] In related technologies, the design of traditional control algorithms makes it difficult to achieve the design and development of highly complex actions, and often cannot meet the generalization requirements, including not only adapting to different physical terrain environments, but also scenarios that require interaction with people; and current reinforcement learning solutions often remain at the development level of simple stepping motion control, do not have the ability to execute complex tasks, and are difficult to meet the needs of interactivity.

[0031] If we want to build a quadruped robot with strong embodied intelligence, we need to have the ability to understand and execute complex tasks, and be able to cooperate and interact with multiple machines and multiple people in multiple scenarios. This application solves the above problems in the following ways:

[0032] (1) The network model is divided into two parts. The lower-level control model is responsible for outputting the specific parameters of the four-legged twelve-motor system to control motion postures such as stepping and kicking. The upper-level model is responsible for perception and decision-making, and implements tasks through the motion control service interface provided by the reasonable lower-level model. Through modular design, the complexity of system design can be reduced, flexibility can be improved, replacement and reuse can be facilitated, the difficulty of development and debugging can be reduced, and other functional modules can be connected for expansion.

[0033] (2) Introduce a large amount of randomization and historical data input to improve network adaptability. In the virtual simulation environment, a large number of parameters are randomized, not only to account for the differences in physical parameters such as friction elastic coefficient that may exist in different environments, but also to take into account the differences in the motor parameters and joint damping coefficients of the quadruped robot itself that may be affected by the process and aging; and some historical data are sent back to the network model to expand the model input from a certain time node state to a time series state. In this way, the impact of historical actions and adjustment strategies can be dynamically learned, so that it can better adapt to different working scenarios in different environments and physical body states, greatly reducing the cost of migrating from the simulation environment to the real environment.

[0034] (3) A system for interactive soccer kicking between a quadruped robot and a human has been developed on a real machine. The quadruped robot is able to detect and identify balls, and pursue dynamic balls. It can also adjust the competitive level of the sport by setting the mode, thus achieving interactive kicking that matches the human kicking rhythm and competitive pursuit of quickly reaching the goal.

[0035] The following describes the interactive system, planning and control method, quadruped robot and storage medium of the quadruped robot of the embodiment of the present application with reference to the accompanying drawings. Specifically, Figure 1 A schematic diagram of an interactive system of a quadruped robot provided in an embodiment of the present application.

[0036] like Figure 1 As shown, the quadruped robot interaction system 10 includes: a simulation training system 100 and a quadruped robot 200 .

[0037] Among them, the simulation training system 100 is used to simulate one or more interactive simulation scenarios of the quadruped robot 200, and iteratively train the planning and control strategy of the quadruped robot 200 in one or more interactive simulation scenarios to optimize the planning and control strategy until the training is completed to obtain the optimized planning and control strategy; the quadruped robot 200 is deployed with the optimized planning and control strategy, wherein each quadruped robot 200 performs interactive tasks in a real interactive scenario based on the optimized planning and control strategy.

[0038] It can be understood that the embodiment of the present application simulates an interactive simulation scenario of four groups of robots based on a simulation training system, and iteratively trains the planning and control strategy of the quadruped robot in the scenario to optimize the planning and control strategy. The quadruped robot performs interactive tasks in a real interactive scenario based on the optimized planning and control strategy, thereby enabling the quadruped robot to interact with multiple machines and multiple people in multiple scenarios, have the ability to understand and execute complex tasks, and meet the requirements of generalization and interactivity.

[0039] It should be noted that the reinforcement learning algorithm of the present application selects the PPO (Proximal Policy Optimization) algorithm for iterative training.

[0040] In the embodiment of the present application, the simulation training system 100 includes: an environment construction module and a training module.

[0041] Among them, the environment construction module is used to randomly simulate the environmental parameters and / or the quadruped robot's own parameters, generate interactive simulation scenes suitable for different interactive tasks, simulate the simulation targets of the interactive tasks in the interactive simulation scenes, and set the motion mode of the simulation targets; the training module is used to iteratively train the planning control strategy through the reinforcement learning strategy. In each iterative training process, the planning control strategy is used to control the simulation targets to perform interactive tasks in the interactive simulation scene, and the planning control strategy is adjusted according to the results of each iterative training until the iterative training stop conditions are met to obtain the optimized planning control strategy.

[0042] Among them, the environmental parameters include one or more of the friction coefficient, the ground contact elasticity coefficient and the terrain, the self parameters include the joint damping and / or the body load weight of the quadruped robot, and the motion mode includes one or more of the static mode, the alternating reciprocating motion mode and the random motion mode.

[0043] It can be understood that the environment construction module in the embodiment of the present application is used to generate interactive simulation scenes for different interactive tasks, and can simulate the simulation targets of the interactive tasks and set the motion mode of the simulation targets. The training module is used to iteratively train the planning control strategy through the reinforcement learning strategy, and adjust the strategy during the iteration process until the iterative training stop condition is met to obtain the optimized planning control strategy, so as to facilitate the subsequent implementation of complex planning control of different environments in the simulation training system based on the optimized planning control strategy.

[0044] It should be noted that during the construction of the virtual simulation environment, the friction coefficient, ground contact elastic coefficient, joint damping, fuselage load weight and terrain are mainly randomized to simulate a variety of different task conditions as much as possible; the tracking target is abstracted as a sphere in the virtual environment, with three different motion modes: stationary, alternating reciprocating motion and random motion, simulating chasing the sphere and intercepting the pass in the real world. As the robot's proficiency in chasing the target increases, the ball speed will gradually increase.

[0045] In an embodiment of the present application, the simulation training system also includes: an action space, an observation state space, and a reward function of a control layer.

[0046] Among them, the action space is used to determine the action parameters of the control layer and the decision layer, wherein the control layer is used to select different action parameters to adjust the control mode of the quadruped robot; the decision layer controls the body movement of the quadruped robot by transmitting triplets to the control layer; the observation state space observes the states of the control layer and the decision layer by setting the state space characteristics, wherein the control layer learns the time series characteristics according to the historical data, and adjusts the planning control strategy according to different state sequences; the decision layer adds the characteristics of the yaw error and distance between the virtual target and the target position; the reward function of the control layer, wherein the reward function includes the decision reward and the control reward, the control reward is used to quantify the degree of fit between the quadruped robot and the speed control command, and the decision reward is used to quantify the posture optimization related indicators of the quadruped robot.

[0047] The triplet includes the forward velocity, tangential velocity, and counterclockwise turning velocity around the vertical ground axis in the quadruped robot's own world coordinate system.

[0048] Among them, the degree of fit includes the linear velocity deviation in the x and y directions and the angular velocity deviation in the z-axis direction. The posture optimization related indicators include one or more of the movement amplitude, basic height, super-mechanical limit penalty, torque limit, foot air time, collision penalty and interruption penalty.

[0049] It can be understood that the embodiment of the present application determines the action parameters of the control layer and the decision layer based on the action space, so as to facilitate the subsequent control of the body movement through the decision layer, and observes the states of the control layer and the decision layer by setting the state space characteristics through the observation state space, so as to facilitate the subsequent adjustment of the planning control strategy, and calculates the degree of fit of the speed control command and the posture optimization related indicators through the reward function of the control layer, so as to facilitate the subsequent control of the quadruped robot.

[0050] Specifically, 1) Action space: First, the action parameters of the control layer network and the decision layer network need to be determined; the control layer selects the action parameters of the twelve motors of the quadruped robot, and can adjust different control modes to express different parameters, such as target position, speed or torque. This system uses the target position through experiments; and the decision layer transmits (Command x ,Command y ,Command z ) triples, which respectively represent the control of the fuselage motion by the forward (x) velocity in its own world coordinate system, the tangential velocity (y) and the counterclockwise turning velocity around the vertical ground axis (z).

[0051] 2) Observation state space: Set the observation and control network by setting the following state space characteristics:

[0052]

[0053] Among them, G represents the three-dimensional gravity vector, C represents the control parameter triplet, P, V, A represent the position, speed, and action of the twelve motors including the past 5 frames of historical records; the addition of historical data allows the network to learn implicit temporal features to adjust strategies for different state sequences; and the decision network adds features for the ball chasing target (H, D), which represent the yaw error (Heading error) and distance (Distance) from the target position respectively.

[0054] 3) Reward function: The reward function of the control layer network mainly calculates two aspects of rewards. One is the degree of fit with the speed control command (r lin vel , r ang vel ), including linear velocity deviation in the x and y directions and angular velocity deviation in the z-axis direction; the second is attitude optimization related indicators (r a , r h , r l , r t , r f , r c , r ter ), including range of motion, base height, penalty for exceeding mechanical limits, torque limit, time the foot is in the air, collision penalty, and interruption penalty.

[0055] In the embodiment of the present application, the quadruped robot 200 includes: a visual perception system and a strategy execution system.

[0056] Among them, the visual perception system is used to perceive the environmental information around the quadruped robot and identify the task objectives in the environmental information; the strategy execution system is deployed with an optimized planning and control strategy, which plans interactive actions according to the position of the quadruped robot and the position of the task objective, and generates control instructions, which are used to control the quadruped robot to perform interactive actions.

[0057] It can be understood that the visual perception system of the embodiment of the present application is used to perceive the environmental information around the quadruped robot and identify the task objectives in the environmental information; the strategy execution system plans interactive actions according to the position of the quadruped robot and the position of the task objectives, and generates control instructions, and uses the control instructions to control the quadruped robot to perform interactive actions, so that the quadruped robot can understand and perform complex tasks to meet interactivity and generalization.

[0058] Specifically, 1) Visual perception system: It is mainly divided into depth perception module and target recognition module. The depth perception module draws on the Unimatch network structure and is developed based on the binocular camera to achieve a low-cost, lightweight, high-precision depth estimator that can be deployed on edge devices. No lidar is required, only a cheap low-resolution binocular camera is needed to achieve centimeter-level accuracy within 3m and a high frame rate. The target recognition part develops a lightweight and efficient detector based on specific ball targets, which mainly uses HSV color block filtering + contour detection to identify specified spheres. This module can be replaced with a specified target detector as needed.

[0059] 2) Strategy execution system: Due to the size and cost of the quadruped robot and the limited computing power of the platform, the original model inference speed based on python and pytorch training could not meet the real-time control requirements of the robot; the model was migrated to the NVIDIA TENSORRT inference framework, the network structure was optimized and adjusted, and quantization was performed for parallel GPU (graphics processing unit) calculations. Finally, after testing, efficient inference can be achieved on Nvidia NX.

[0060] According to the interactive system of a quadruped robot proposed in the embodiment of the present application, an interactive simulation scene of four groups of robots is simulated based on a simulation training system, and the planning and control strategy of the quadruped robot is iteratively trained in the scene to optimize the planning and control strategy. The quadruped robot performs interactive tasks in a real interactive scene based on the optimized planning and control strategy, thereby enabling the quadruped robot to interact with multiple machines and multiple people in multiple scenarios, having the ability to understand and execute complex tasks, and meeting the requirements of generalization and interactivity.

[0061] The following will be combined Figures 2 to 5 The interactive system of the quadruped robot is described in detail as follows:

[0062] The present application embodiment takes a quadruped robot kicking ball interactive system as an example to illustrate, wherein the quadruped robot kicking ball interactive system architecture mainly includes a simulation training system and a real machine execution system, which are described as follows:

[0063] 1. Build a framework for the quadruped robot reinforcement learning strategy training system

[0064] 1) First, the reinforcement learning algorithm selects the PPO algorithm for iterative training, adopts the Actor-Critic architecture, and can perform importance sampling on the data to achieve efficient on-policy (the action strategy and the target strategy are the same strategy) learning.

[0065] 2) Action Space: First, the action parameters of the control layer network and the decision layer network need to be determined. The control layer selects the action parameters of the twelve motors of the quadruped robot. Different control modes can be adjusted to show different parameters, such as target position, speed or torque. This system uses the target position after experiments. The decision layer transmits (Command) to the control layer. x ,Command y ,Command z ) triples, which respectively represent the control of the fuselage motion by the forward (x) velocity in its own world coordinate system, the tangential velocity (y) and the counterclockwise turning velocity around the vertical ground axis (z).

[0066] 3) State Space Observation: Set the observation and control network by setting the following state space characteristics:

[0067]

[0068] They represent G three-dimensional gravity vector, C control parameter triplet, P, V, A twelve motor positions, speeds, and actions including the past five frames of historical records; the addition of historical data allows the network to learn implicit temporal features to adjust strategies for different state sequences; and the decision network adds features for the ball chasing target (H, D), which represent Heading error (yaw error from the target position) and Distance (distance from the target position), respectively.

[0069] 4) Reward function: The control layer network reward function mainly calculates two aspects of rewards. One is the degree of fit with the speed control command (r lin vel , r ang vel ), including linear velocity deviation in the x and y directions and angular velocity deviation in the z-axis direction; second, attitude optimization related indicators (r a , r h , r l , r t , r f , r c , r ter ), including range of motion, base height, penalty for exceeding mechanical limits, torque limit, time the foot is in the air, collision penalty, and interruption penalty.

[0070] 5) Environment construction: During the construction of the virtual simulation environment, the friction coefficient, ground contact elastic coefficient, joint damping, fuselage load weight and terrain are mainly randomized to simulate a variety of different task conditions. The tracking target is abstracted as a sphere in the virtual environment, with three different movement modes: static, alternating reciprocating motion and random motion, simulating chasing the sphere in the real world, that is, intercepting the pass. As the robot's proficiency in chasing the target increases, the ball speed will gradually increase.

[0071] 2. Real machine task execution system

[0072] 1) Visual perception system: It is mainly divided into depth perception module and target recognition module. The depth perception module draws on the Unimatch network structure and is developed based on binocular cameras to achieve low-cost, lightweight, high-precision depth estimator that can be deployed on edge devices. No lidar is required, only cheap low-resolution binocular cameras are needed to achieve centimeter-level accuracy within 3m and have a high frame rate. The target recognition part develops a lightweight and efficient detector based on specific ball targets, mainly using HSV color block filtering + contour detection to identify the specified sphere. This module can be replaced with a specified target detector as needed.

[0073] 2) Strategy execution system: Due to the size and cost of the quadruped robot and the limited computing power of the platform, the original model inference speed trained based on python and pytorch could not meet the real-time control requirements of the robot. The model was migrated to the NVIDIA TENSORRT inference framework, the network structure was optimized and adjusted, and quantization was performed for parallel GPU computing. Finally, after testing, efficient inference can be achieved on Nvidia NX.

[0074] In summary, the multi-layer reinforcement learning network structure proposed in this scheme can well solve the design contradiction between decision-making rewards and control rewards, and can use the highly diverse parallel simulation environment to quickly optimize and iterate the strategy to achieve complex planning and control that can adapt to different environments; and it is quantitatively deployed on a quadruped robot for verification testing, so that the quadruped robot has the ability to track highly dynamic targets, can chase balls, and interact with humans to play football; the modular design of the quadruped robot control system and the plasticity of the training system enable it to be expanded and developed according to different needs in the future, so as to achieve an embodied intelligent system with high interactive experience, high intelligent performance, and high complex functions.

[0075] Next, the planning and control method of a quadruped robot proposed in accordance with an embodiment of the present application will be described with reference to the accompanying drawings.

[0076] Figure 6 It is a flow chart of the planning and control method of the quadruped robot according to an embodiment of the present application.

[0077] like Figure 6 As shown, the planning and control method of the quadruped robot is deployed with an optimized planning and control strategy generated by the interactive system of the quadruped robot as in the above embodiment, wherein the method comprises the following steps:

[0078] In step S101, the environmental information around the quadruped robot is sensed.

[0079] It can be understood that the embodiments of the present application sense the environmental information around the quadruped robot so as to identify the task objectives of the interactive tasks in the environmental information.

[0080] It should be noted that the embodiment of the present application can perceive the environmental information around the quadruped robot through the binocular camera of the visual perception system.

[0081] In step S102, the task target of the interactive task in the environment information is identified.

[0082] It can be understood that the embodiment of the present application identifies the task target of the interactive task in the environmental information, so as to facilitate the subsequent planning of the interactive action from the current position to the task target position based on the optimized planning control strategy.

[0083] In step S103, the optimized planning control strategy is used to plan the interactive action of the quadruped robot from the current position to the task target position, and a control instruction is generated, and the control instruction is used to control the quadruped robot to perform the interactive action.

[0084] It can be understood that the embodiment of the present application uses the optimized planning control strategy to plan the interactive actions of the quadruped robot from the current position to the task target position, and generates control instructions, and uses the control instructions to control the quadruped robot to perform interactive actions, so that the quadruped robot can understand and perform complex tasks to meet interactivity and generalization.

[0085] According to the planning and control method of a quadruped robot proposed in the embodiment of the present application, by sensing the environmental information around the quadruped robot, identifying the task objectives of the interactive tasks in the environmental information, and using the optimized planning and control strategy to plan the interactive actions of the quadruped robot from the current position to the task target position, and generate control instructions, the control instructions are used to control the quadruped robot to perform interactive actions, so that the quadruped robot can understand and perform complex tasks to meet interactivity and generalization.

[0086] Figure 7 A schematic diagram of the structure of a quadruped robot provided in an embodiment of the present application. The quadruped robot may include:

[0087] A memory 701 , a processor 702 , and a computer program stored in the memory 701 and executable on the processor 702 .

[0088] When the processor 702 executes the program, the planning and control method of the quadruped robot provided in the above embodiment is implemented.

[0089] Furthermore, the quadruped robot also includes:

[0090] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0091] The memory 701 is used to store computer programs that can be executed on the processor 702 .

[0092] The memory 701 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0093] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0094] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0095] The processor 702 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0096] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the planning and control method of the quadruped robot as described above.

[0097] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0098] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0099] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0100] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0101] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

Claims

1. An interactive system for a quadruped robot, characterized in that: include: A simulation training system, used to simulate one or more interactive simulation scenarios of a quadruped robot, and iteratively train the planning and control strategy of the quadruped robot in the one or more interactive simulation scenarios to optimize the planning and control strategy until the training is completed to obtain the optimized planning and control strategy; One or more quadruped robots are deployed with the optimized planning and control strategy, wherein each quadruped robot performs an interactive task in a real interactive scene based on the optimized planning and control strategy.

2. The interactive system of a quadruped robot according to claim 1, characterized in that: The simulation training system comprises: An environment construction module, used for randomly simulating environmental parameters and / or the quadruped robot's own parameters, generating interactive simulation scenes suitable for different interactive tasks, simulating simulation targets of the interactive tasks in the interactive simulation scenes, and setting motion modes of the simulation targets; A training module is used to iteratively train the planning control strategy through a reinforcement learning strategy. In each iterative training process, the planning control strategy is used to control the simulation target to perform interactive tasks in the interactive simulation scenario, and the planning control strategy is adjusted according to the results of each iterative training until the iterative training stop condition is met, thereby obtaining the optimized planning control strategy.

3. The interactive system of a quadruped robot according to claim 2, characterized in that: The environmental parameters include one or more of the friction coefficient, the ground contact elasticity coefficient and the terrain, the self parameters include the joint damping and / or the body load weight of the quadruped robot, and the motion mode includes one or more of the static mode, the alternating reciprocating motion mode and the random motion mode.

4. The interactive system of a quadruped robot according to any one of claims 1 to 3, characterized in that: The simulation training system also includes: An action space is used to determine action parameters of a control layer and a decision layer, wherein the control layer is used to select different action parameters to adjust the control mode of the quadruped robot; the decision layer controls the body movement of the quadruped robot by transmitting a triplet to the control layer; Observe the state space, and observe the states of the control layer and the decision layer by setting the state space characteristics, wherein the control layer learns the time series characteristics according to the historical data, and adjusts the planning control strategy according to different state sequences; the decision layer adds the characteristics of the yaw error and distance between the virtual target and the target position; The reward function of the control layer includes a decision reward and a control reward, wherein the control reward is used to quantify the degree of fit between the quadruped robot and the speed control command, and the decision reward is used to quantify the posture optimization-related indicators of the quadruped robot.

5. The interactive system of a quadruped robot according to claim 4, characterized in that: The triplet includes the forward velocity, the tangential velocity and the counterclockwise turning velocity around the vertical ground axis in the quadruped robot's own world coordinate system.

6. The interactive system of a quadruped robot according to claim 4, characterized in that: The degree of fit includes linear velocity deviation in the x and y directions and angular velocity deviation in the z-axis direction, and the posture optimization related indicators include one or more of movement amplitude, basic height, super-mechanical limit penalty, torque limit, foot air time, collision penalty and interruption penalty.

7. The interactive system of a quadruped robot according to claim 1, characterized in that: The quadruped robot comprises: A visual perception system, used to perceive the environmental information around the quadruped robot and identify the task target in the environmental information; The strategy execution system is deployed with the optimized planning control strategy, plans interactive actions according to the position of the quadruped robot and the position of the task target, generates control instructions, and uses the control instructions to control the quadruped robot to perform the interactive actions.

8. A planning and control method for a quadruped robot, characterized in that: The quadruped robot is deployed with an optimized planning control strategy generated by the interactive system of the quadruped robot according to any one of claims 1 to 7, wherein the method comprises the following steps: Perceiving environmental information around the quadruped robot; Identifying a task goal of an interactive task in the environmental information; The optimized planning control strategy is used to plan the interactive action of the quadruped robot from the current position to the task target position, and a control instruction is generated. The control instruction is used to control the quadruped robot to perform the interactive action.

9. A quadruped robot, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the planning and control method of the quadruped robot as claimed in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the planning and control method of a quadruped robot as claimed in claim 8.

Images