Control method and apparatus for legged robot, and legged robot and medium

Abstract

The present application relates to the field of robot control. Disclosed are a control method and apparatus for a legged robot, and a legged robot and a medium. The method comprises: acquiring body sensing information and external sensing information, wherein the body sensing information is used for representing the motion state of the body of a legged robot, and the external sensing information is used for representing environmental information around the legged robot; inputting the body sensing information and the external sensing information into a deep neural network, so as to obtain a first prediction residual that is output by the deep neural network; adjusting a trajectory generation parameter of a foot trajectory generator on the basis of the first prediction residual; and on the basis of a first joint motion parameter that is output by the parameter-adjusted foot trajectory generator, controlling the motion state of the legged robot. By using the solution provided in the embodiments of the present application, the flexibility and stability of a legged robot when moving in a complex environment are improved.

Description

Control method and device of legged robot, legged robot and medium

[0001] This application claims priority to Chinese patent application No. 202310859169.1, filed on July 12, 2023, entitled “Control method, device, legged robot and medium for a legged robot”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The embodiments of the present application relate to the field of robot control, and in particular to a control method and device for a legged robot, a legged robot, and a medium. Background Art

[0003] Legged robots are widely used in scenarios such as expedition rescue, industrial production, and medical assistance. In various usage scenarios, legged robots also face higher requirements for flexibility and stability.

[0004] In related technologies, legged robots can obtain proprioception information and perform motion control based on the proprioception information combined with a foot trajectory generator, so that the legged robots can perform motion control under unknown terrain conditions and maintain the robustness of the motion.

[0005] However, legged robots can only handle some relatively simple terrains, such as muddy ground. For complex terrains, such as plum blossom piles and terrain with gaps, their terrain adaptability is poor, which leads to poor flexibility and stability of legged robots when moving in complex environments.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a control method, device, legged robot, and medium for a legged robot, which can improve the flexibility and stability of the legged robot when moving in complex environments. The technical solution is as follows:

[0008] In one aspect, an embodiment of the present application provides a control method for a legged robot, the method being performed by the legged robot, the method comprising:

[0009] Acquiring proprioceptive information and external perception information, wherein the proprioceptive information is used to characterize the motion state of the legged robot body, and the external perception information is used to characterize environmental information surrounding the legged robot;

[0010] Inputting the proprioception information and the external perception information into a deep neural network to obtain a first prediction residual output by the deep neural network, wherein the first prediction residual is used to correct a trajectory generation parameter of a foot trajectory generator;

[0011] adjusting the trajectory generation parameters of the foot trajectory generator based on the first prediction residual;

[0012] The motion state of the footed robot is controlled based on the first joint motion parameter output by the foot trajectory generator after parameter adjustment.

[0013] On the other hand, an embodiment of the present application provides a control device for a legged robot, the device comprising:

[0014] An information acquisition module, configured to acquire proprioceptive information and external perception information, wherein the proprioceptive information is used to characterize the motion state of the legged robot body, and the external perception information is used to characterize the environmental information surrounding the legged robot;

[0015] a prediction module, configured to input the proprioception information and the external perception information into a deep neural network to obtain a first prediction residual output by the deep neural network, wherein the first prediction residual is used to correct a trajectory generation parameter of a foot trajectory generator;

[0016] a parameter adjustment module, configured to adjust the trajectory generation parameter of the foot trajectory generator based on the first prediction residual;

[0017] A control module is used to control the motion state of the legged robot based on the joint motion parameters output by the foot trajectory generator after parameter adjustment.

[0018] On the other hand, an embodiment of the present application provides a legged robot, which includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the control method of the legged robot as described in the above aspects.

[0019] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one computer instruction is stored. The at least one computer instruction is loaded and executed by a processor to implement the control method of the legged robot as described in the above aspects.

[0020] In another aspect, an embodiment of the present application provides a computer program product, comprising computer instructions stored in a computer-readable storage medium. A processor of a legged robot reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the legged robot to perform the legged robot control method provided in the above aspects.

[0021] In an embodiment of the present application, a parameterized foot trajectory generator is introduced, which supports adjustment of trajectory generation parameters through a deep neural network. Because the deep neural network combines the proprioceptive information that characterizes the motion state of the legged robot body and the external perception information that characterizes the environment surrounding the legged robot when adjusting the parameters of the foot trajectory generator, compared to the foot trajectory generator with fixed parameters, the parameterized foot trajectory generator can generate foot trajectories that are more in line with the current environment. Furthermore, controlling the footed robot based on the joint motion parameters output by the parameter-adjusted foot trajectory generator can improve the flexibility and stability of the footed robot in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0023] FIG2 is a control framework diagram of a legged robot provided by an exemplary embodiment of the present application;

[0024] FIG3 is a flow chart of a control method for a legged robot provided by an exemplary embodiment of the present application;

[0025] FIG4 is a flow chart of a control process of a legged robot provided by another exemplary embodiment of the present application;

[0026] FIG5 is a schematic diagram of a control process of a legged robot provided by an exemplary embodiment of the present application;

[0027] FIG6 is a flowchart of a deep neural network training process provided by an exemplary embodiment of the present application;

[0028] FIG7 is a schematic diagram of a control process of a legged robot provided by an exemplary embodiment of the present application;

[0029] FIG8 is a schematic diagram of a simulation environment provided by an exemplary embodiment of the present application;

[0030] FIG9 is a schematic diagram of a comparison of leg lift heights provided by an exemplary embodiment of the present application;

[0031] FIG10 is a schematic diagram of joint angle changes provided by an exemplary embodiment of the present application;

[0032] FIG11 is a structural block diagram of a control device for a legged robot provided by an exemplary embodiment of the present application;

[0033] FIG12 shows a schematic structural diagram of a legged robot provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0035] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0036] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0037] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning through demonstration.

[0038] Deep Neural Network (DNN) is a technology in the field of machine learning. A deep neural network is a multi-layer unsupervised neural network that can use the output features of the previous layer as the input of the next layer for feature learning. Through layer-by-layer feature mapping, the features of existing spatial samples are mapped to another feature space, so as to learn better feature expressions for existing inputs.

[0039] Long Short-Term Memory (LSTM) is a recurrent neural network suitable for processing and predicting important events in time series with very long intervals and delays. LSTM has a variety of applications in science and technology, including language translation, robot control, image analysis, document summarization, speech recognition, and image recognition.

[0040] The technical solution of this application mainly relates to robotics technology in artificial intelligence technology, mainly to robot intelligent control.

[0041] A robot is a mechatronic device that utilizes mechanical transmission and modern microelectronics to mimic certain human or animal skills. Developed on the basis of electronics, mechanics, and information technology, it is an automated machine that possesses some of the same intelligent capabilities as humans or living things, such as perception, planning, movement, and coordination. It is a highly flexible automated machine.

[0042] Trajectory generator: An algorithmic module used in robotics and automated control to generate predetermined trajectories. The purpose of a trajectory generator is to calculate the precise spatial path that the end effector (e.g., a manipulator or leg) of a robotic arm, robot, robot dog, or other automated device should follow when performing a task. This path includes not only position information but also dynamic characteristics such as velocity and acceleration. The foot trajectory generator in the embodiment of this application is used to generate the motion trajectory of the foot of a legged robot.

[0043] The trajectory generator can use a joint space planning approach to describe the foot trajectory using a function of joint angles, or a Cartesian space planning approach to describe the foot trajectory using a function of Cartesian position and posture with respect to time. In the embodiment of the present application, the foot trajectory generator uses a joint space planning approach to describe the foot trajectory by outputting joint motion parameters, thereby achieving motion state control of the legged robot.

[0044] The foot trajectory generator can control the characteristics of the generated trajectory based on the trajectory generation parameters, thereby controlling the motion state of the robot. Among them, the trajectory generation parameters may include cadence parameters, stride parameters, gait height parameters (leg lift height parameters), etc. In the embodiment of the present application, the trajectory generation parameters of the foot trajectory generator support parameterization, and the trajectory generation parameters are parameterized by a deep neural network based on the motion state of the foot robot body and the surrounding environment.

[0045] FIG1 shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application. The implementation environment includes a legged robot 110 and a plum blossom pile terrain 120. The legged robot 110 is a quadruped robot, and the legged robot 110 includes a base 111, four legs 112 arranged on the base 111, and a number of joints corresponding to the four legs. The legged robot 110 obtains proprioceptive information representing the motion state of the body, and simultaneously obtains external perception information representing the environmental information around the footed robot 110 (plum blossom pile terrain 120). The footed robot 110 performs motion control through proprioceptive information and external perception information, and flexibly avoids risky gap areas during movement, so that the footed robot 110 can move stably in the plum blossom pile terrain 120.

[0046] Please refer to Figure 2, which shows a control framework diagram of a legged robot provided by an exemplary embodiment of the present application. In one example, as shown in Figure 2, a control scenario of a quadruped robot is used as an example. The implementation environment of this solution may include a legged robot 210 and a control device 220 (optional).

[0047] In some embodiments, as shown in FIG2 , a legged robot 210 includes a body 201 (also referred to as a base or chassis) and a leg mechanism 202. A controller for the legged robot 210 is located within the body 201. The body 201 (the controller within the body) issues commands to the leg mechanism 202 to control its movements.

[0048] The leg mechanism 202 is provided with multiple joints, meaning that the leg mechanism 202 is a multi-segment leg structure. Each joint may be provided with one or more joint motors. For example, one leg mechanism in the leg mechanism 202 includes joints 203 and 204. Joint 203 is provided with one joint motor for controlling thigh movement, while joint 204 is provided with two joint motors for controlling calf movement.

[0049] The control device 220 may include, but is not limited to, a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc. Alternatively, the control device 220 may be a server. The control device 220 may be used to control the legged robot 210.

[0050] The legged robot 210 and the control device 220 can communicate via a network, such as a wireless network, a wired network, etc.

[0051] Exemplarily, after obtaining the proprioception information and external perception information of the legged robot 210 at the current moment, the control device 220 can predict the joint motion parameters of the legged robot 210 at the next moment based on the proprioception information and external perception information, and control the movement of the legged robot 210 based on the predicted joint motion parameters, so that the legged robot 210 can perform actions accurately and efficiently.

[0052] In some embodiments, this process can also be completed independently by the legged robot. The legged robot obtains proprioception information and external perception information, and then inputs the proprioception information and external perception information into a deep neural network after vectorized feature processing. The deep neural network outputs a first prediction residual, which is used to correct the trajectory generation parameters of the foot trajectory generator of the legged robot. The trajectory generation parameters corresponding to the trajectory generator may include a baseline cadence and a baseline leg lift height. After the parameters are adjusted, the foot trajectory generator outputs the joint motion parameters at the next moment, and the legged robot controls the motion state of the legged robot based on the joint motion parameters.

[0053] The embodiments of the present application do not limit the execution entity of the control of the legged robot. For the convenience of description, the following embodiments are described using the legged robot as an example of the execution entity.

[0054] Figure 3 shows a flow chart of a control method for a legged robot provided by an exemplary embodiment of the present application. This embodiment is described by taking the method executed by a legged robot as an example, and the method includes the following steps.

[0055] Step 301: Acquire proprioceptive information and external perception information. The proprioceptive information is used to characterize the motion state of the legged robot body, and the external perception information is used to characterize the environmental information around the legged robot.

[0056] In some embodiments, the legged robot periodically acquires proprioception information. This proprioception information may include, but is not limited to, the legged robot's body velocity information, body rotation angle information, joint angular velocity information, joint angular acceleration information, and direction commands. The frequency of acquiring proprioception information may match the trajectory generation period of the foot trajectory generator.

[0057] The body velocity information can include the legged robot's velocity information in the x, y, and z axes, where the z axis is perpendicular to the legged robot, the x axis is the width of the legged robot's base, and the y axis is the length of the legged robot's base. Correspondingly, the body rotation information includes the angle information of the legged robot's rotation around the x, y, and z axes, respectively.

[0058] Optionally, the body rotation angle information can be expressed using Euler angles, and the body velocity information and body rotation angle information can be collected by an inertial measurement unit (IMU) mounted on the legged robot. The legged robot can obtain joint angular velocity and joint angular acceleration from encoders corresponding to each joint. Furthermore, the legged robot can receive directional commands from the user, which indicate the direction of movement of the legged robot.

[0059] In some embodiments, the legged robot periodically acquires external sensory information. This external sensory information is used to characterize the environmental information surrounding the legged robot. The external sensory information includes, but is not limited to, terrain information surrounding the legged robot, including, but not limited to, terrain height information and geological information. By acquiring terrain height information, the legged robot can sense whether the surrounding terrain is continuous or has gaps. By acquiring geological information, the robot can sense whether the surrounding terrain is suitable for advancement (truthfully) or unsuitable for advancement (e.g., a puddle). The frequency of acquiring external sensory information can match the trajectory generation cycle of the foot trajectory generator.

[0060] External perception information can be collected in a variety of ways, for example, by visual sensors arranged on the legged robot, or by using LiDAR (Light Detection and Ranging) sensors to collect terrain information.

[0061] Of course, in other possible embodiments, the external environment information may also include access information, which is used to indicate whether the area around the legged robot can be entered; or, the external environment information may also include meteorological information, which is used to indicate the meteorological conditions around the legged robot, such as rain, snow, temperature, humidity, etc. The embodiments of the present application do not limit the specific content of the external environment information.

[0062] In one possible implementation, external perception information can be collected through motion capture technology. Multiple cameras are set up in the external environment of the legged robot to collect a terrain map, and the relative coordinates between the position of the legged robot and the terrain map are obtained. The legged robot receives the coordinates to generate a terrain height map, and uses the terrain height map as external perception information.

[0063] In step 302 , the proprioception information and the external perception information are input into a deep neural network to obtain a first prediction residual output by the deep neural network. The first prediction residual is used to correct the trajectory generation parameters of the foot trajectory generator.

[0064] In some embodiments, a legged robot includes a trajectory generator (TG) for each leg of the legged robot, which is used to generate the trajectory of each leg of the legged robot at the next moment. The TG provides prior knowledge to the controller of the legged robot, allowing the legged robot to perform motion control based on the reference information output by the TG. Optionally, the legged robot can also use independent policies modulating trajectory generators (PMTG) for each leg.

[0065] The foot trajectory generator in the embodiment of the present application is not a foot trajectory generator with fixed parameters, but a foot trajectory generator that supports parameterization. That is, the trajectory generation parameters used to generate the foot trajectory in the foot trajectory generator support dynamic adjustment, thereby improving adaptability to different environments.

[0066] In some embodiments, the trajectory generation parameters of the foot trajectory generator are parameterized by a deep neural network.

[0067] In some embodiments, the trajectory generation parameters of the foot trajectory generator include at least one of a cadence parameter, a stride parameter, and a leg lift height parameter (gait height parameter). The cadence parameter is used to control the cadence of the legged robot, the stride parameter is used to control the stride of the legged robot, and the leg lift height parameter is used to control the leg lift height of the legged robot.

[0068] In some embodiments, due to changes in the legged robot's proprioceptive and external perception information, the motion state of the legged robot at the next moment predicted by the foot trajectory generator deviates from the actual motion state of the legged robot at the next moment. Therefore, the legged robot inputs the proprioceptive and external perception information into a deep neural network, which predicts the deviation between the actual motion state and the predicted motion state. Based on this deviation, a first prediction residual is determined, allowing the legged robot to subsequently adjust the trajectory generation parameters of the foot trajectory generator based on the first prediction residual. In some embodiments, before using the deep neural network for residual prediction, the proprioceptive and external perception information need to be vectorized to obtain a proprioceptive vector and an external perception vector. The local perception vector and the external perception vector are then fused, and the fusion result is input into the deep neural network.

[0069] Step 303: Adjust the trajectory generation parameters of the foot trajectory generator based on the first prediction residual.

[0070] In some embodiments, the legged robot adjusts the trajectory generation parameters of the foot trajectory generator according to the first residual, so that the foot trajectory generator can more accurately predict the motion state of the legged robot at the next moment, thereby outputting more accurate reference information (joint motion parameters).

[0071] Optionally, the legged robot adjusts all or part of the trajectory generation parameters based on the first prediction residual. For example, the legged robot adjusts the cadence parameter based on the first prediction residual, or adjusts the leg lift height parameter based on the first prediction residual, or adjusts both the cadence and leg lift height parameters based on the first prediction residual.

[0072] Step 304 : Control the motion state of the footed robot based on the joint motion parameters output by the foot trajectory generator after parameter adjustment.

[0073] In some embodiments, after the legged robot adjusts the parameters of the foot trajectory generator based on the first prediction residual, the foot trajectory generator can output more accurate joint motion parameters at the next moment, so that the legged robot can control the foot motion state according to the joint motion parameters.

[0074] Optionally, the controller of the legged robot determines the motor output (such as the motor output torque) of the motor at each joint based on the joint motion parameters output by the foot trajectory generator, thereby driving the joint motion through the motor to make the legged robot move.

[0075] In summary, in the embodiment of the present application, a parameterized foot trajectory generator is introduced, which supports the adjustment of trajectory generation parameters through a deep neural network. Since the deep neural network combines the proprioceptive information that characterizes the motion state of the foot robot body and the external perception information that characterizes the environment surrounding the foot robot when adjusting the parameters of the foot trajectory generator, compared with the foot trajectory generator with fixed parameters, the foot trajectory generator that supports parameterization can generate foot trajectories that are more in line with the current environment. Furthermore, controlling the foot robot based on the joint motion parameters output by the foot trajectory generator after parameter adjustment can improve the flexibility and stability of the foot robot in complex environments.

[0076] In one possible design, in addition to outputting a first prediction residual for adjusting trajectory generation parameters, the deep neural network also outputs a second prediction residual for correcting joint motion parameters. When controlling the motion state of the legged robot based on the joint motion parameters output by the foot trajectory generator after the parameters have been adjusted, the legged robot corrects the joint motion parameters output by the foot trajectory generator based on the second prediction residual, thereby controlling the motion state of the legged robot based on the corrected joint motion parameters.

[0077] Figure 4 shows a flow chart of a control process of a legged robot provided by another exemplary embodiment of the present application. This embodiment is described by taking the method executed by a legged robot as an example, and the method includes the following steps.

[0078] Step 401: Use the received movement direction instruction, historical motion parameters of the legged robot, and historical prediction residuals as proprioception information, where the historical prediction residuals include the historical first prediction residuals and the historical second prediction residuals output by the deep neural network.

[0079] In some embodiments, the deep neural network is an LSTM network, so the legged robot can input the corresponding historical motion state information into the LSTM network, and predict the motion state at the next moment based on the historical motion state through the LSTM network, and output a first prediction residual. The legged robot then adjusts the trajectory generation parameters of the trajectory generator according to the first prediction residual, so that it outputs more accurate joint motion parameters at the next moment, and the legged robot then performs motion control according to the joint motion parameters.

[0080] Optionally, the historical motion state information includes historical motion parameters of the legged robot.

[0081] In some embodiments, in addition to historical motion state information as input, movement direction instructions and historical prediction residuals (prediction residuals output by the deep neural network in the historical period) can also be used as input.

[0082] In some embodiments, the movement direction instruction is a direction instruction issued by a user and received by the legged robot, which is used to indicate the movement direction of the legged robot; the historical motion parameters of the legged robot are the joint motion parameters generated by the legged robot at a historical moment; the historical prediction residual may include a historical first prediction residual (used to adjust the trajectory generation parameters), or, include a historical first prediction residual and a historical second prediction residual (used to correct the joint motion parameters).

[0083] In some embodiments, the first prediction residual includes a cadence residual and a leg-lift height residual, which are used to correct the cadence parameter and leg-lift height parameter of the foot trajectory generator, respectively. Accordingly, the historical first prediction residual includes the historical cadence residual and the historical leg-lift height residual. The second prediction residual is used to correct the joint motion parameters output by the foot trajectory generator. Accordingly, the historical second prediction residual is the second prediction residual corresponding to a historical moment in the footed robot, and is used to correct the historical joint motion parameters output by the foot trajectory generator.

[0084] Step 402: Acquire a first terrain height map around the foot of the legged robot; and use the first terrain height map as external perception information.

[0085] In some embodiments, the external perception information reflects the terrain state around the legged robot. The legged robot can generate a first terrain height map based on the height of the terrain around it and use the first terrain height map as the external perception information. The first terrain height map can be obtained using the following method:

[0086] Taking the foot of the legged robot as the center, the ground height is sampled based on at least two sampling radii to obtain the heights of at least two ground sampling points.

[0087] In a possible implementation, for each leg of the legged robot, the legged robot samples the ground height on circles of different radii with the foot corresponding to each leg as the center, to obtain heights of multiple ground sampling points.

[0088] The height of the ground sampling point can be expressed as p ij , i represents the i-th leg of the legged robot, and j represents the j-th sampling point. If each leg of the legged robot samples n points, then j = {1, 2, ..., n}. After determining the height of the ground sampling point, the legged robot can compare the ground sampling point height with the height of the foot to obtain information about the legged robot's external perception state.

[0089] For example, for a quadruped robot, with the quadruped as the center of the circle, each leg samples the ground height with a radius of 0.5m and 1m, and samples once at each sampling radius. Then each leg corresponds to two ground sampling points (n=2), and the four legs are sampled a total of 8 times. Accordingly, 8 ground sampling point heights can be obtained.

[0090] Furthermore, the legged robot generates a first terrain height map based on the difference between the height of the foot and the height of the ground sampling point.

[0091] In one possible implementation, the legged robot calculates a difference between the foot height of each leg and the height of a ground sampling point corresponding to each leg, and generates a first terrain height map based on the difference and the position coordinates of the ground sampling points. This first terrain height map can represent the difference between the current foot height of the legged robot and the height of the surrounding terrain near the foot.

[0092] For example, for a quadruped robot, with the quadruped as the center, ground height samples are taken for each leg at radii of 0.5m and 1m. Each leg is sampled once within each sampling radius, resulting in two ground sampling points (n=2) for each leg. The heights of the two ground sampling points are then subtracted from the height of the foot corresponding to that leg. A first terrain height map is generated based on the height differences and the positions of the ground sampling points. For example, the left front leg has a foot height of 0.3m, and the two ground sampling points corresponding to the left front leg have heights of 0.4m and 0.3m. Therefore, the height difference between the left front leg foot height and the ground sampling point height is 0.1m and 0m. This height difference, combined with the positions of the ground sampling points corresponding to the left front leg, generates terrain height information around the left front leg. This is repeated for the remaining three legs, generating a first terrain height map based on the terrain height information around each leg.

[0093] In other embodiments, the legged robot obtains a second terrain height map of a specific shape area below the reference position; and uses the second terrain height map as external perception information.

[0094] Optionally, to improve the effectiveness of the generated terrain height map, the reference position can be located in the middle of the legged robot, and the specific shape area can be larger than the legged robot's projected area. For example, the reference position can be located in the middle of the legged robot's body, and the specific shape area can be a circular area with a diameter greater than the length of the legged robot. This embodiment of the application defines the specific location of the reference position and the specific form and size of the specific shape area.

[0095] In some embodiments, a collection component is provided at the reference position of the legged robot, which is used to sample the height of a specific shaped area below to obtain a second terrain height map. The collection component can be a ranging component (such as a ToF ranging component, an infrared ranging component, a LiDAR sensor) or a camera component.

[0096] In one possible implementation, a legged robot obtains a reference position of its main body and, with the reference position as the center, obtains a second terrain height map of a specific shaped area. For example, for a quadruped robot, a second terrain height map of a circular area with a radius of 1 meter is obtained with the reference position of the main body as the center, or a second terrain height map of a square area with a side length of 1 meter is obtained with the reference position of the main body as the center.

[0097] Of course, in addition to the two methods of obtaining terrain height maps mentioned above, the legged robot can also obtain terrain height maps in other ways, such as obtaining a terrain height map within a predetermined range corresponding to the position coordinates from a pre-constructed environmental terrain height map based on the position coordinates of the legged robot. This embodiment does not limit this.

[0098] In step 403, the proprioceptive information and the external perception information are input into a deep neural network to obtain a first prediction residual and a second prediction residual output by the deep neural network. The first prediction residual includes a cadence residual and a leg lift height residual. The first residual is used to correct the trajectory generation parameters of the foot trajectory generator, and the second prediction residual is used to correct the joint motion parameters.

[0099] In some embodiments, the legged robot inputs the received movement direction instructions, the historical motion parameters of the legged robot, and the historical prediction residuals as proprioception information into the LSTM network. At the same time, the legged robot inputs the first terrain height map (or the second terrain height map) as external perception information into the deep neural network, so that the LSTM network predicts the motion state of the legged robot at the next moment based on the historical motion state of the legged robot and the external environment state, and outputs the first prediction residual. The cadence residual and leg lift height residual in the first prediction residual are used to correct the cadence parameters and leg lift height parameters of the foot trajectory generator.

[0100] Step 404: Correct the reference cadence based on the cadence residual to obtain an adjusted cadence parameter.

[0101] In some embodiments, the legged robot has a pre-set baseline cadence. The foot trajectory generator uses the cadence residual output by the LSTM network to correct the baseline cadence to obtain an adjusted cadence parameter. This allows the foot trajectory generator, after parameter adjustment, to more accurately predict the motion state of the legged robot at the next moment based on the dynamic cadence and output more accurate joint motion parameters. The cadence parameter can be the sum of the baseline cadence and the cadence residual, and the cadence residual can be positive or negative.

[0102] In a possible application scenario, let the base step frequency be f base , the step frequency residual output by the LSTM network is δf, then the adjusted step frequency parameter f can be obtained t is: f t =f base +δf

[0103] Step 405: Correct the reference leg-lifting height based on the leg-lifting height residual to obtain an adjusted leg-lifting height parameter.

[0104] In some embodiments, the legged robot has a pre-set reference leg lift height. The foot trajectory generator corrects the reference leg lift height using the leg lift height residual output by the LSTM network to obtain an adjusted leg lift height parameter. This allows the foot trajectory generator, after parameter adjustment, to more accurately predict the motion state of the legged robot at the next moment based on the dynamic leg lift height and output more accurate joint motion parameters. The leg lift height parameter can be the sum of the reference leg lift height and the leg lift height residual, and the leg lift height residual can be positive or negative.

[0105] In a possible application scenario, let the baseline leg-lifting height be H, and the leg-lifting height residual output by the LSTM network be δh, then the adjusted leg-lifting height parameter H can be obtained. t is: H t =H+δh

[0106] It should be noted that there is no strict execution order between the above steps 404 and 405.

[0107] Step 406 : Acquire the joint motion parameters output by the foot trajectory generator after parameter adjustment.

[0108] In some embodiments, the foot trajectory generator can output relatively accurate joint motion parameters after adjusting the cadence parameter and the leg lift height parameter. The joint motion parameters can be the torque of each joint of the legged robot. In one possible implementation, the joint motion parameters can be obtained by the following method:

[0109] The legged robot obtains the plantar position of each leg based on the adjusted cadence and foot lift height parameters of the foot trajectory generator and the motion state of each leg. Inverse kinematics (IK) is then used to calculate the target motor position corresponding to the plantar position. Finally, the proportional-integral-derivative (PID) controller is used to calculate the torque corresponding to each joint and use the torque as the joint motion parameter.

[0110] In a possible implementation scenario, the legged robot is a quadruped robot, and the motion state of each leg is represented by a phase p∈[0,2π]. When the legged robot adopts a trotting gait, the phase p of the left front leg is given. LF In the case of , the remaining three legs (right front leg p RF 、Left hind leg LR 、Right hind leg RR )’s phase: p RR =p LF and p RF =p LR =p LF +π

[0111] The phase of the left front leg can be calculated based on the phase of the left front leg, the step frequency and the unit time step of the leg robot at the previous moment by the following formula: t =p t-1 +f t T

[0112] Where p t Indicates the phase of the left front leg at the current moment, p t-1represents the left front leg phase at the current moment, f t represents the step frequency, and T is the unit time step. The unit time step can be calculated based on the frequency of the deep neural network output signal. For example, a deep neural network outputs 50 signals per second, that is, it sends a signal every 0.02 seconds (20ms), so the unit time step is 0.02 seconds (20ms).

[0113] After obtaining the phase of each leg of the legged robot, the phase is regularized for subsequent calculation of the plantar position corresponding to each leg. q is the leg phase after regularization:

[0114]

[0115] To obtain the joint motion parameters, it is necessary to first calculate the plantar position corresponding to each leg of the legged robot, and then use inverse kinematics to calculate the target motor position corresponding to the plantar position, so as to perform PID to obtain the joint motion parameters (torque). Among them, the plantar position corresponding to each leg is equal to the reference plantar position plus the plantar position residual. Since the reference plantar position of each leg is the default value, it is only necessary to calculate the plantar position residual corresponding to each leg. The plantar position residual corresponding to each leg can be calculated by the following formula: l =-(2q 3 -3q 2 +1)+0.5 d x =c x *d l d y =c y *d l t=2p i / π

[0116] In the above formula, d x , d y , d z To find the solution, d x , d y , d z Respectively represent the residual position of the plantar position of each leg of the leg robot in the x-axis direction, y-axis direction and z-axis direction, pi represents the phase of the i-th leg, q represents the regularized phase, t is similar to q, which is an intermediate quantity for further processing of the phase of the i-th leg for subsequent calculations, and d l ∈[-0.5,0.5] is the intermediate value of leg phase normalization, solving d x with d y Two intermediate quantities c are also required x and c y Participate in calculation, c x and c yThey represent the displacements generated by the footed robot moving in the x-axis and y-axis directions according to the moving direction instruction, respectively. x and c y It can be calculated by the following formula: x =Lcosθ c y =Lsinθ

[0117] Where L is a known quantity representing the reference step length of the legged robot, and θ is the angle between the moving direction indicated by the moving direction command and the current moving direction of the legged robot.

[0118] Through the above calculation process, the legged robot can obtain the sole position (d x +X,d y +Y,d z +Z), where X, Y, and Z are the reference sole positions of each leg of the legged robot in the x-axis, y-axis, and z-axis directions, respectively.

[0119] The foot trajectory generator outputs the foot sole position of each leg of the legged robot as (d x +X,d y +Y,d z +Z), inverse kinematics is used to calculate the target motor position corresponding to the plantar position, and then the torque corresponding to each joint is obtained through PID. The foot-type robot uses the torque corresponding to each joint as the joint motion parameter.

[0120] Step 407: Modify the joint motion parameters based on the second prediction residual.

[0121] In some embodiments, after the legged robot inputs the proprioception information and the external perception information into the LSTM network, while obtaining the first prediction residual output by the deep neural network, the legged robot can also obtain a second prediction residual, wherein the second prediction residual can be the joint motion residual corresponding to each joint of the legged robot. The legged robot corrects the joint motion parameters output by the foot trajectory generator based on the joint motion residual, so that the predicted joint motion state represented by the corrected joint motion parameters is closer to the actual joint motion state. In a possible implementation, the LSTM network can be a deep neural network obtained through reinforcement learning training. Thus, the legged robot can obtain an action based on the LSTM network by setting a strategy and an observation state, and use the action as the output of the LSTM network. Therefore, the legged robot can set the action space corresponding to the action according to demand, thereby determining the output of the LSTM network.

[0122] For example, a quadruped robot with four groups of robots each having three joints is constructed. The action space is configured to include fourteen dimensions of action. The first two dimensions represent the residuals of cadence and leg lift height, while the remaining twelve dimensions represent the joint motion residuals corresponding to the twelve joints of the legged robot. The first two dimensions are used as the first prediction residuals output by the LSTM network, while the remaining twelve dimensions are used as the second prediction residuals output by the LSTM network. The second prediction residuals represent the joint motion residuals corresponding to each joint of the legged robot. Based on these joint motion residuals, the legged robot corrects the joint motion parameters of the twelve joints output by the foot trajectory generator.

[0123] In some embodiments, the deep neural network may output only the first prediction residual, without outputting the second prediction residual for use in correcting the joint parameters output by the foot trajectory generator; this embodiment is not limited thereto. The legged robot may adjust the trajectory generation parameters of the foot trajectory generator using the obtained first prediction residual, and control the motion state of the legged robot based on the joint motion parameters output by the foot trajectory generator after the parameter adjustments.

[0124] Step 408 : Control the motion state of the legged robot based on the corrected joint motion parameters.

[0125] In some embodiments, the joint motion parameters may be the joint torques corresponding to the joints of the legged robot. The legged robot may determine the joint motor positions through the joint motion parameters (joint torques), thereby adjusting the positions of the joint motors and controlling the motion state of the legged robot.

[0126] Please refer to Figure 5, which shows a schematic diagram of the control process of the legged robot in the above embodiment. Figure 5 includes a legged robot 510, a foot trajectory controller 520, and a deep neural network 530. The legged robot in the figure takes a quadruped robot as an example. The legged robot 510 obtains proprioception information 540 and external perception information 550, and inputs the proprioception information 540 and external perception information 550 into the deep neural network 530 after vectorized feature processing. The deep neural network 530 outputs a first prediction residual, which is used to adjust the baseline cadence and baseline leg lift height of the foot trajectory generator 520. After the parameters are adjusted, the foot trajectory generator 520 outputs the joint motion parameters at the next moment, and the legged robot 510 performs motion control based on the joint motion parameters at the next moment.

[0127] In summary, in the embodiments of the present application, an LSTM network is used as a deep neural network. The legged robot first uses the received movement direction command, the legged robot's historical motion parameters, and historical prediction residuals as proprioceptive information. At the same time, the legged robot obtains a terrain height map around the foot and uses the first terrain height map as external perception information, allowing the legged robot to perceive the state of the body and the external environment at the same time. The legged robot then inputs the proprioceptive perception information and external perception information into the LSTM network to obtain the first prediction residual and the second prediction residual output by the deep neural network. Based on the cadence residual and leg lift height residual in the first prediction residual, the legged robot adjusts the cadence parameters and leg lift height parameters in the foot trajectory generator, thereby obtaining more accurate joint motion parameters output by the foot trajectory generator compared to before the parameter adjustment. Then, the legged robot corrects the joint motion parameters again based on the second prediction residual output by the LSTM network, so that the predicted joint motion state represented by the corrected joint motion parameters is closer to the actual joint motion state. The legged robot can control the motion state of the legged robot according to the corrected joint motion parameters. Since proprioception information and external perception information are introduced in the motion control process, the legged robot can adapt to different environments and move forward steadily in different environments, thereby improving the flexibility and stability of the legged robot when moving in complex environments.

[0128] During the motion control process of the legged robot, the legged robot inputs proprioception information and external perception information into the deep neural network, and obtains a first prediction residual through the deep neural network prediction. The more accurate the network model parameters corresponding to the deep neural network are, the more accurate the first prediction residual output by it will be. Therefore, the legged robot can obtain sample proprioception information and sample external perception information in advance, train the deep neural network through reinforcement learning, and continuously optimize the deep neural network so that the accuracy of the first prediction residual output by the deep neural network is higher, thereby improving the accuracy of the parameter adjustment of the foot trajectory generator. As shown in Figure 6, it shows a flowchart of the deep neural network training process provided by an exemplary embodiment of the present application. The training process includes the following steps:

[0129] Step 601: Acquire sample body perception information and sample external perception information.

[0130] The process of obtaining the sample body perception information and the sample external perception information may refer to the above-mentioned process of obtaining the body perception information and the external perception information, which will not be described in detail in this embodiment.

[0131] In step 602, the sample ontological perception information and the sample external perception information are input into a deep neural network to obtain a sample prediction residual output by the deep neural network.

[0132] In some embodiments, the sample prediction residual may include a first sample prediction residual, or include the first sample prediction residual and a second sample prediction residual, wherein the first sample prediction residual is used to adjust the trajectory generation parameters, and the second sample prediction residual is used to correct the joint motion parameters.

[0133] Step 603: Adjust the trajectory generation parameters of the foot trajectory generator based on the sample prediction residual.

[0134] In some embodiments, the legged robot adjusts trajectory generation parameters of a foot trajectory generator based on the first sample prediction residual.

[0135] Step 604 : Control the motion state of the footed robot based on the sample joint motion parameters output by the foot trajectory generator after parameter adjustment.

[0136] In some embodiments, when the sample prediction residual only includes the first sample prediction residual, the foot robot controls the foot movement based on the sample joint motion parameters output by the foot trajectory generator; when the sample prediction residual also includes the second sample prediction residual, the foot robot uses the second sample prediction residual to correct the sample joint motion parameters, thereby controlling the foot movement based on the corrected sample joint motion parameters.

[0137] Step 605: Determine an exercise reward based on the exercise state.

[0138] In some embodiments, the motion of a legged robot (i.e., the process of training a deep neural network) can be viewed as a partially observable Markov decision process (POMDP), expressed as <S,A,R,P,γ>, where S and A represent the state and action space, respectively. R represents the reward function, P is the state transition probability, and γ∈(0,1) is the reward discount factor.

[0139] A legged robot takes an action a in its current motion state and receives a scalar reward r, then moves to the next motion state s. Combining the state transition probability distribution and the reward discount coefficient, the discounted reward of this process can be obtained. The overall goal of training is to find an optimal strategy to maximize the future discounted reward.

[0140] During training, if the legged robot's action a receives a positive reward, the robot's tendency to perform that action in the future will be strengthened. If the legged robot's action a receives a negative reward (penalty), the robot's tendency to perform that action will be weakened. During the robot's movement, the robot continuously feeds rewards to the deep neural network based on its motion state, allowing the deep neural network to dynamically adjust its parameters based on the rewards, thereby optimizing the deep neural network. Therefore, establishing the motion rewards and their corresponding reward functions is particularly important during training.

[0141] In some embodiments, the legged robot may determine the motion reward based on the motion state by at least one of the following methods:

[0142] 1. The motion reward includes at least one of an in-command speed reward and an out-of-command speed reward, wherein the in-command speed reward and the out-of-command speed reward are used to encourage the legged robot to move in a desired direction and speed. The process of determining the motion reward based on the motion state may include the following steps:

[0143] In some embodiments, the legged robot obtains its body speed and the moving direction indicated by the moving direction instruction; when the body speed is greater than the lower limit of the expected speed and less than the upper limit of the expected speed, the default reward value is determined as the first speed reward value; and the first speed reward value is used as the speed reward within the instruction.

[0144] In one possible embodiment, the legged robot's body speed can be split into a speed component in the command direction (command direction speed) and a speed component in the non-command direction (non-command direction speed) based on the movement direction command. It is hoped that the legged robot can move in the desired direction indicated by the command, and when the command direction speed of the body speed is within the desired speed range, the legged robot's movement speed is considered to meet the requirements. The reward corresponding to the legged robot's current action (the first speed reward value) is a positive reward, thereby encouraging the legged robot to move in the desired direction and at the desired speed. The default reward value can be a value set in advance by the legged robot, for example, the default reward value is set to 1.

[0145] In some embodiments, when the body speed is less than the expected speed lower limit, a second speed reward value is determined based on the body speed, the moving direction and the expected speed lower limit; the second speed reward value is used as the in-command speed reward, and the second speed reward value is less than the default reward value.

[0146] In one possible implementation, the body speed of the legged robot can be divided into the command direction speed and the non-command direction speed according to the movement direction instruction. We hope that the legged robot can move in the desired direction indicated by the instruction. The movement of the legged robot in the non-command direction is not encouraged. Therefore, when the command direction speed in the body speed exceeds the expected speed range (less than the lower limit of the expected speed), it means that the movement speed of the legged robot in the desired movement direction does not meet the requirements (too slow). The second reward value corresponding to the action of the legged robot at the current moment is a reverse reward (penalty) to punish the legged robot for moving too slowly in the desired direction.

[0147] Optionally, the second reward value is determined based on the body speed, the moving direction and the lower limit of the expected speed, and the second speed reward value is less than the default reward value.

[0148] In some embodiments, when the main body speed is greater than the expected speed upper limit, a third speed reward value is determined based on the main body speed, the moving direction and the expected speed upper limit; the third speed reward value is used as the in-command speed reward, and the third speed reward value is less than the default reward value; the out-of-command speed reward is determined based on the main body speed and the moving direction, and the out-of-command speed reward is negatively correlated with the main body speed.

[0149] In one possible implementation, similar to the above embodiment, when the command direction speed in the body speed exceeds the expected speed range (greater than the expected speed upper limit), it means that the movement speed of the legged robot in the expected movement direction does not meet the requirements (too fast), and the third reward value corresponding to the action of the legged robot at the current moment is a reverse reward (penalty) to punish the legged robot for moving too fast in the expected direction.

[0150] Optionally, the third reward value is determined based on the body speed, the moving direction and the expected speed upper limit, and the third speed reward value is less than the default reward value.

[0151] In a possible application scenario, the first reward value is 1, the second reward value and the third reward value can be calculated by the following formula, where r v represents the speed reward within the instruction, the expected speed range (v l ,v h ), v l is the lower limit of the expected speed, v h is the upper limit of the expected speed, c t =[cosθ,sinθ,0],θ∈[0,2π] is the moving direction instruction, θ is the angle between the instruction moving direction and the current moving direction of the legged robot, That is, it represents the component velocity in the command direction of the legged robot's body velocity (command direction velocity).

[0152] In one possible implementation, since movement of the legged robot in non-command directions is discouraged, and the non-command direction speed in the body velocity represents movement of the legged robot in a direction other than the command direction, the non-command reward corresponding to the legged robot's current non-command direction action is a negative reward (penalty) to penalize the legged robot for movement in an undesired direction. Furthermore, the farther the legged robot moves (i.e., the greater the non-command direction speed), the greater the penalty fed back to the deep neural network. This means that the non-command direction speed and the non-command speed reward are negatively correlated. Since the non-command direction speed is the component speed of the body velocity in the non-command direction, the non-command speed reward determined by the legged robot based on the body velocity and movement direction is also negatively correlated with the component speed of the body velocity outside the movement direction.

[0153] In a possible application scenario, the out-of-command speed reward can be calculated by the following formula, where r vo Represents the speed bonus outside the command, and the moving direction command is represented by c t =[cosθ,sinθ,0],θ∈[0,2π], θ is the angle between the command moving direction and the current moving direction of the legged robot, Represents the component velocity in the non-command direction (non-command direction velocity) of the legged robot's body velocity.

[0154] 2. The motion reward includes an energy reward, wherein the energy reward is used to encourage the legged robot to reduce energy consumption during motion. The process of determining the motion reward based on the motion state may include the following steps:

[0155] The joint torque and joint angular velocity of the legged robot are obtained; the joint motion power is determined based on the joint torque and joint angular velocity; and the energy reward is determined based on the joint motion power, and the energy reward is negatively correlated with the joint motion power.

[0156] In one possible implementation, the energy consumption of a legged robot is positively correlated with the joint motion power of the legged robot. Joint motion power is related to the torque and angular velocity of each joint of the legged robot. We want the legged robot to maintain stable motion while reducing energy consumption. Therefore, the energy reward is an inverse reward (penalty). That is, the more energy the legged robot consumes, the greater the penalty value fed back to the deep neural network. Therefore, the energy reward is negatively correlated with joint motion power.

[0157] In a possible application scenario, the energy reward can be calculated by the following formula, where r tau represents the energy reward, τ t represents the torque of each joint, q v，tRepresents the joint angular velocity of each joint.

[0158] 3. The motion reward includes a foot terrain reward, which is used to encourage the legged robot to avoid dangerous terrain. The process of determining the motion reward based on the motion state may include the following steps:

[0159] In one possible embodiment, a leg of a legged robot can have two states during movement: a swing state and a stance state. The legged robot obtains a foot terrain height; and determines the difference between the maximum and minimum foot terrain heights corresponding to the leg as a terrain height difference.

[0160] In some embodiments, when a leg of the legged robot is in a raised state, or when a leg of the legged robot is in a bottomed state and the terrain height difference is greater than a risky terrain height threshold, a first value is determined as a foot terrain reward. The foot terrain reward is calculated once for each leg of the legged robot.

[0161] Among them, when the leg of the legged robot is in the raised state, it means that the leg has not yet stepped into the risky terrain (such as a terrain gap), so the first value is determined to be the foot terrain reward, and the first value is the default value set for the legged robot.

[0162] When the leg of the legged robot is in the bottoming state, if the terrain height difference is greater than the risk terrain height threshold at this time, it means that the terrain stepped on by the leg is safe terrain. The legged robot also determines the first value as the foot terrain reward and feeds it back to the deep neural network, representing an action to encourage the leg of the legged robot to step into safe terrain.

[0163] In one possible implementation, when the leg of the legged robot is in a bottoming state and the terrain height difference is less than a risk terrain height threshold, a second value is determined as the foot terrain reward, and the second value is less than the first value.

[0164] When the leg of the legged robot is in a bottoming state and the terrain height difference is less than the risk terrain height threshold, it means that the leg has stepped into risky terrain (such as a terrain gap). Therefore, in the foot terrain reward fed back by the legged robot to the deep neural network, the foot terrain reward corresponding to the second value is less than the foot terrain reward corresponding to the first value, which means that the leg of the legged robot is not encouraged to step into risky terrain.

[0165] In one possible implementation, the foot terrain reward can be set according to the following formula: terrain,i Represents the i-th leg of the legged robot (f i ) corresponding to the foot terrain reward, max(zij )-min(z ij ) represents the i-th leg of the legged robot (f i ) is the difference between the maximum and minimum values ​​of the foot terrain height corresponding to (terrain height difference), H thre Represents the risk terrain height threshold.

[0166] 4. The motion reward includes a leg-lifting height reward, which is used to encourage the leg-lifting robot to lower the leg-lifting height. The process of determining the motion reward based on the motion state may include the following steps:

[0167] In one possible implementation, each leg of the legged robot calculates a leg lift height reward.

[0168] In some embodiments, the legged robot first obtains the leg-lifting height of each leg, the foot terrain height corresponding to each leg, and the sample leg-lifting height residual in the sample prediction residual, and then determines the difference between the maximum and minimum values ​​of the foot terrain height corresponding to a certain leg as the terrain height difference corresponding to the leg; the leg-lifting height difference of the leg can be determined based on the leg-lifting height, the sample leg-lifting height residual, the terrain height difference, and the leg-lifting height threshold.

[0169] When the leg-lifting height difference is greater than 0, the leg-lifting robot determines the leg-lifting height reward based on the leg-lifting height difference. The leg-lifting height reward is negatively correlated with the leg-lifting height difference, that is, the greater the leg-lifting height difference, the smaller the leg-lifting height reward (because lifting the leg too high increases power consumption).

[0170] In a possible implementation, when the leg-lifting height difference of a certain leg of the leg-type robot is less than or equal to 0, the leg-type robot determines the leg-lifting height reward of the leg to be 0.

[0171] In one possible implementation, the leg lift height reward can be set according to the following formula: height,i Represents the i-th leg of the legged robot (f i ), H represents the baseline leg-lifting height of the leg-lifting robot, δh is the residual error of the sample leg-lifting height, ΔH is the difference between the maximum and minimum values ​​of the terrain height (terrain height difference), F thre Represents the leg raising height threshold. height,i =-max(H+δh-ΔH-F thre ,0)

[0172] 5. The motion reward includes a smoothness reward, which is used to encourage the legged robot to have a smooth gait. The process of determining the motion reward based on the motion state may include the following steps:

[0173] In one possible implementation, the legged robot determines the difference in joint angles between adjacent moments in time. A smoothness bonus is then determined based on this difference in joint angles, with the smoothness bonus being negatively correlated with the difference in joint angles. A smaller difference in joint angles between adjacent moments indicates a smaller amplitude of motion, meaning the robot's movements are smoother.

[0174] In one possible implementation, the smoothing reward can be set as follows: smooth represents the smooth reward of the legged robot, q t represents the joint angle of the legged robot at the current moment, q t-1 Represents the joint angles of the legged robot at the previous moment.

[0175] Step 606: Train the deep neural network based on the motion reward.

[0176] In one possible implementation, a reinforcement learning approach is used to train a deep neural network based on motion rewards.

[0177] In some embodiments, a legged robot trains a neural network through reinforcement learning based on at least one of an on-command speed reward, an off-command speed reward, an energy reward, a foot terrain reward, a leg lift height reward, and a smoothness reward. The legged robot determines the reward value for each of these five types of rewards based on its current motion state and feeds the reward value back to the deep neural network, which optimizes network parameters using a policy gradient algorithm such as the Proximal Policy Optimization (PPO) algorithm, thereby outputting more accurate prediction residuals.

[0178] In some embodiments, the legged robot can train a deep neural network through reinforcement learning based on the above five types of motion rewards (in-command speed reward, out-of-command speed reward, energy reward, foot terrain reward, leg lift height reward and smoothness reward).

[0179] Legged robots can also train deep neural networks through reinforcement learning based on some types of motion rewards. For example, legged robots train neural networks through reinforcement learning based on in-command speed rewards, out-of-command speed rewards, and foot terrain rewards. The legged robots determine the reward values ​​corresponding to the in-command speed rewards, out-of-command speed rewards, and foot terrain rewards based on the current motion state, and feed the reward values ​​back to the deep neural network, so that the deep neural network optimizes the network parameters based on policy gradient algorithms such as the PPO algorithm, thereby outputting more accurate prediction residuals.

[0180] Optionally, when training a deep neural network based on at least two types of motion rewards, reward weights can be set for different types of motion rewards based on the motion requirements of the legged robot, and based on the reward weights, at least two types of motion rewards can be weightedly calculated, thereby training the deep neural network based on weighted motion rewards.

[0181] For example, a high weight can be set for the foot terrain reward to improve the operating safety of the legged robot in complex environments; a high weight can be set for the ability reward to reduce the operating power consumption of the legged robot. The embodiments of this application do not limit the specific method of setting the reward weight.

[0182] Please refer to Figure 7, which shows a schematic diagram of the control process of the legged robot in the above embodiment. Figure 7 includes a legged robot 710, a foot trajectory controller 720 and a deep neural network 730. The figure shows a legged robot taking a quadruped robot as an example. The legged robot 710 obtains proprioception information 740 and external perception information 750, and inputs the proprioception information 740 and the external perception information 750 into the deep neural network 730 after vectorized feature processing. The deep neural network 730 outputs a first prediction residual and a second prediction residual. The first prediction residual is used to correct the trajectory generation parameters of the foot trajectory generator 720 of the legged robot 710, and the second prediction residual is used to correct the joint motion parameters. The trajectory generation parameters corresponding to the foot trajectory generator 720 may include a reference step frequency and a reference leg lift height. After the parameters are adjusted, the foot trajectory generator 720 outputs the joint motion parameters at the next moment. The legged robot 710 corrects the joint motion parameters based on the second prediction residual, and controls the motion state of the legged robot based on the joint motion parameters. The legged robot 710 feeds back rewards to the deep neural network 730 based on the motion state, so that the deep neural network 730 is trained through reinforcement learning based on the rewards.

[0183] To summarize, in an embodiment of the present application, the legged robot obtains sample proprioception information and sample external perception information, so that the deep neural network predicts the motion state of the legged robot based on the sample proprioception information and the sample external perception information, and outputs a sample prediction residual. The legged robot then adjusts the parameters of the foot trajectory generator according to the sample prediction residual, and controls the motion state using the sample joint motion parameters output by the foot trajectory generator. The legged robot determines a motion reward based on the motion state, and trains the deep neural network according to reinforcement learning through the motion reward, so that the trained deep neural network can output a more accurate prediction residual.

[0184] In a possible application scenario, Figure 8 shows a schematic diagram of a simulation environment provided by an exemplary embodiment of the present application. A legged robot conducts a simulation experiment based on the control method of the legged robot to obtain simulation results, thereby verifying whether the legged robot adopting the control method can adapt to various complex environments during movement. At the same time, by comparing the simulation results of the legged robot that does not adopt the control method in the simulation environment, it is determined whether the flexibility and stability of the legged robot adopting the control method when moving in a complex environment are improved.

[0185] Among them, (a) in Figure 8 is the simulation environment of block terrain, (b) in Figure 8 is the simulation environment of stepping stone terrain, (c) in Figure 8 is the simulation environment of stair terrain, and (d) in Figure 8 is the simulation environment of plum blossom pile terrain.

[0186] The footed robot adopts three control schemes to walk in the above four different simulation environments, and obtains corresponding simulation results for comparison. Among them, the first control scheme is the control scheme provided in the embodiment of the present application. The footed robot adopts the control method in the embodiment of the present application for motion control, and moves forward in the simulation environment of block terrain, stair terrain, stepping stone terrain and plum blossom pile terrain, and the distance the footed robot moves forward in each simulation environment is used as the simulation result corresponding to each simulation environment. The second control scheme is a fixed TG scheme. The footed robot can obtain proprioception information and external perception information at the same time, but the trajectory generation parameters of the foot trajectory generator (TG) of the footed robot are fixed and will not be adjusted based on the motion state of the footed robot. Accordingly, the footed robot adopting the second control scheme also moves forward in the simulation environment of block terrain, stair terrain, stepping stone terrain and plum blossom pile terrain, and the distance the footed robot moves forward in each simulation environment is used as the simulation result corresponding to each simulation environment. The third control scheme is a non-perception control scheme. The legged robot can only obtain proprioceptive perception information but not external perception information. The trajectory generation parameters of the foot trajectory generator (TG) of the legged robot can be adjusted based on the motion state of the legged robot. Accordingly, the legged robot using the third control scheme also advances in the simulation environments of block terrain, stair terrain, stepping stone terrain and plum blossom pile terrain, and the distance advanced by the legged robot in each simulation environment is used as the simulation result corresponding to each simulation environment.

[0187] Regarding external perception information, in the simulation experiment, both the legged robots of the present application scheme and the fixed TG scheme can obtain external perception information, wherein the external perception information is a terrain height map generated by the legged robot. With each leg of the legged robot as the center of the circle, circles with radii of 0 cm, 3 cm, 6 cm, and 10 cm are used to sample the terrain height respectively. The circles with radii of 0 cm, 3 cm, 6 cm, and 10 cm sample 1, 6, 8, and 10 point heights respectively as terrain sampling points. The legged robot generates a terrain height map according to the heights of the terrain sampling points and uses the terrain height map as external perception information.

[0188] Regarding proprioception, in the simulation experiments, the legged robots of all three schemes were able to obtain internal perception information, including movement direction commands. To determine whether the legged robots in the three schemes were capable of walking in any direction under different simulation environments, six different direction commands were randomly selected from [0, π / 4, -π / 4, -π, -5π / 4, -3π / 4] as movement direction commands, where 0 represents forward walking and -π represents backward walking. Three random experiments were conducted using these selected movement direction commands. The mean and standard deviation of the maximum travel distance of the legged robots in these three random experiments were calculated and used as the simulation results.

[0189] Due to the map size limitation of the simulation environment, the maximum forward distance of the legged robot is 5m under ideal conditions. The final results are shown in Table 1.

[0190] Table 1

[0191] As can be seen from Table 1, in each simulation environment, the footed robot of the present application has the best overall performance. In both the block terrain and stair terrain environments, the forward distance exceeds 4.9m. At the same time, in the two complex terrain environments with terrain gaps, the stepping stone terrain and the plum blossom pile terrain, the footed robot can also advance more than 4.9m, and the variance corresponding to the forward distance in the four scenarios is small, indicating that the footed robot can advance relatively stably in each instruction direction. Compared with the present application, the fixed TG solution and the non-perception solution, the footed robot has poor flexibility in the two complex terrain environments with terrain gaps, the stepping stone terrain and the plum blossom pile terrain, and the forward distance is small (less than 2m), and the variance corresponding to the forward distance is large, indicating that the footed robots of these two solutions have large differences in the forward situation in each instruction direction and poor forward stability. It can be seen that obtaining external perception information and adjusting the parameters of the foot trajectory generator during the movement of the legged robot have a great impact on the motion control of the legged robot in a complex environment. The present application scheme obtains external perception information while obtaining proprioception information, and adjusts the parameters of the foot trajectory generator based on the motion state of the legged robot, so that the legged robot can avoid risky terrain and move in the desired direction.

[0192] In a possible application scenario, as shown in FIG9 , which is a schematic diagram of a comparison of leg lift heights provided by an exemplary embodiment of the present application, a preferred control method for a legged robot can support the legged robot in adjusting the leg lift height according to the height of the terrain, thereby reducing the energy consumption of the legged robot, rather than raising the leg as high as possible in all terrains. To evaluate whether a legged robot using the control method of the present application can adjust the leg lift height according to the height of the terrain and reduce energy consumption, the foot trajectories generated by the legged robot using the solution of the present application were compared with the foot trajectories generated by the legged robot using the existing technology solution. The tests were conducted in different terrain environments (staircase terrain), and the comparison results are shown in FIG9 .

[0193] Figure 9 shows a legged robot tested on three terrains: a 3cm staircase, a 7.5cm staircase, and a 12cm staircase. The foot trajectories generated by the three control schemes for the legged robot in these terrains are compared. The first control scheme is the control scheme provided in an embodiment of the present application (ours), the second control scheme is a fixed-height TG scheme, and the third control scheme is a visionless scheme.

[0194] From Figure 9, we can see that the leg-lifting height of the foot-type robot of the present application solution can be adjusted according to the different stair step heights, while the leg-lifting height of the foot-type robot of other solutions is significantly higher than the leg-lifting height of the foot-type robot of this solution when facing stair terrain with lower step heights (3cm, 7.5cm). In addition, the fourth figure of Figure 9 shows the leg-lifting height parameter (H+δh) output by the foot trajectory generator of the foot-type robot in the present application solution, and the parameter adjustment by the deep neural network in stair terrain with different step heights. It can be seen from the figure that the foot trajectory generator can adjust the output leg-lifting height parameter according to the terrain height. In particular, for stair terrain with a height of 3cm, the leg-lifting height parameter output by the foot trajectory generator can allow the foot-type robot to reduce the foot-lifting height to about 3 centimeters. These results show that the foot-type robot using the control method of the present application can adjust the leg-lifting height according to the terrain height, effectively reducing energy consumption.

[0195] In a possible application scenario, as shown in Figure 10, Figure 10 is a schematic diagram of the joint angle changes provided by an exemplary embodiment of the present application; in order to evaluate whether the legged robot adopting the control method of the present application can avoid risky terrain in complex environments and improve the flexibility and stability of the legged robot when moving in complex environments, the legged robot adopting the solution of the present application will be tested in two terrain environments to observe the changes in the joint angles of the legged robot in different terrain environments. The changes in the joint angles of the legged robot in the two terrain environments are shown in Figure 10.

[0196] Both test environments involved stepping stones, but the gaps were set differently. In the first (a), the gap widths were set to 8.5cm, 17.0cm, and 25.5cm, yielding the joint variations (b). In the second (c), the gap widths were set to 6.8cm, 20.4cm, and 13.6cm, yielding the joint variations (d). Regarding the joint variations, the joint angles (θ / rad) corresponding to the three joints (joint 1, joint 2, and joint 3) of the left front leg and the right front leg were measured in both environments.

[0197] From Figure 10, it can be seen that the legged robot using the control method of the present application can safely cross gaps of different widths. At the same time, in two terrain environments, the angles of the second joint of the left front leg and the second joint of the right front leg can be adjusted according to the different terrain gap widths. As the terrain gap width increases, the joint angle change also increases.

[0198] FIG11 is a block diagram of a control device for a legged robot provided by an exemplary embodiment of the present application, the device comprising:

[0199] An information acquisition module 1101 is configured to acquire proprioceptive information and external perception information, wherein the proprioceptive information is used to characterize the motion state of the legged robot body, and the external perception information is used to characterize the environmental information surrounding the legged robot;

[0200] a prediction module 1102, configured to input the proprioception information and the external perception information into a deep neural network to obtain a first prediction residual output by the deep neural network, wherein the first prediction residual is used to correct a trajectory generation parameter of a foot trajectory generator;

[0201] a parameter adjustment module 1103, configured to adjust the trajectory generation parameters of the foot trajectory generator based on the first prediction residual;

[0202] The control module 1104 is configured to control the motion state of the legged robot based on the joint motion parameters output by the foot trajectory generator after parameter adjustment.

[0203] Optionally, the first prediction residual includes a cadence residual and a leg lift height residual, and the trajectory generation parameters of the foot trajectory generator include a cadence parameter and a leg lift height parameter. The parameter adjustment module 1103 is configured to:

[0204] Correcting the reference cadence based on the cadence residual to obtain an adjusted cadence parameter;

[0205] The reference leg-lifting height is corrected based on the leg-lifting height residual to obtain an adjusted leg-lifting height parameter.

[0206] Optionally, the output of the deep neural network further includes a second prediction residual, which is used to correct the joint motion parameters output by the foot trajectory generator. The control module 1104 is used to:

[0207] acquiring the joint motion parameters output by the foot trajectory generator after parameter adjustment;

[0208] modifying the joint motion parameter based on the second prediction residual;

[0209] The motion state of the legged robot is controlled based on the corrected joint motion parameters.

[0210] Optionally, the deep neural network is an LSTM network, and the information acquisition module 1101 is used to:

[0211] The received movement direction instruction, the historical motion parameters of the legged robot, and the historical prediction residuals are used as the proprioception information, and the historical prediction residuals include the historical first prediction residuals and the historical second prediction residuals of the output of the deep neural network.

[0212] Optionally, the information acquisition module 1101 is used to:

[0213] Acquire a first terrain height map around the foot of the legged robot; and use the first terrain height map as the external perception information;

[0214] or,

[0215] Acquire a second terrain height map of a specific shape area below the reference position of the legged robot; and use the second terrain height map as the external perception information.

[0216] Optionally, the information acquisition module 1101 is used to:

[0217] Taking the foot of the legged robot as the center, sampling the ground height based on at least two sampling radii to obtain the heights of at least two ground sampling points;

[0218] The first terrain height map is generated based on the difference between the foot height and the ground sampling point height.

[0219] Optionally, the device further includes a network training module, configured to:

[0220] Acquire sample ontological perception information and sample external perception information;

[0221] Inputting the sample ontological perception information and the sample external perception information into the deep neural network to obtain a sample prediction residual output by the deep neural network;

[0222] adjusting parameters of the foot trajectory generator based on the sample prediction residual;

[0223] controlling the motion state of the footed robot based on the sample joint motion parameters output by the foot trajectory generator after parameter adjustment;

[0224] determining an exercise reward based on the exercise state;

[0225] The deep neural network is trained based on the motion reward.

[0226] Optionally, the motion reward includes at least one of an in-command speed reward and an out-of-command speed reward, wherein the in-command speed reward and the out-of-command speed reward are used to encourage the legged robot to move in a desired direction and speed. The network training module is used to:

[0227] Obtaining the body speed of the legged robot and the moving direction indicated by the moving direction instruction;

[0228] When the speed of the main body is greater than the lower limit of the expected speed and less than the upper limit of the expected speed, the default reward value is used to determine a first speed reward value; and the first speed reward value is used as the speed reward within the instruction;

[0229] or,

[0230] When the body speed is less than the expected speed lower limit, determining a second speed reward value based on the body speed, the moving direction, and the expected speed lower limit; using the second speed reward value as the in-command speed reward, wherein the second speed reward value is less than the default reward value;

[0231] or,

[0232] When the main body speed is greater than the expected speed upper limit, determining a third speed reward value based on the main body speed, the moving direction, and the expected speed upper limit; using the third speed reward value as the in-command speed reward, wherein the third speed reward value is less than the default reward value;

[0233] The out-of-command speed bonus is determined based on the main body speed and the moving direction, and the out-of-command speed bonus is negatively correlated with the component speed of the main body speed outside the moving direction.

[0234] Optionally, the motion reward includes an energy reward, wherein the energy reward is used to encourage the legged robot to reduce energy consumption during motion. The network training module is used to:

[0235] The determining of the exercise reward based on the exercise state includes:

[0236] Obtaining joint torque and joint angular velocity of the legged robot;

[0237] determining a joint motion power based on the joint torque and the joint angular velocity;

[0238] The energy reward is determined based on the joint movement power, and the energy reward is negatively correlated with the joint movement power.

[0239] Optionally, the motion reward includes a foot terrain reward, which is used to encourage the legged robot to avoid dangerous terrain. The network training module is used to:

[0240] Obtaining the height of the foot terrain of the legged robot;

[0241] determining a difference between a maximum value and a minimum value of the foot terrain height as a terrain height difference;

[0242] In a case where the user is in a leg-lifting state or the terrain height difference is greater than a risk terrain height threshold, determining a first value as the foot terrain reward;

[0243] or,

[0244] In a bottoming-out state and when the terrain height difference is less than the risk terrain height threshold, a second value is determined as the foot terrain reward, the second value being less than the first value.

[0245] Optional, leg lift height reward, which is used to encourage the leg-lifting robot to lower the leg lift height. The network training module is used to:

[0246] Obtaining the leg-lifting height of the leg-type robot, the foot terrain height, and the sample leg-lifting height residual in the sample prediction residual;

[0247] determining a difference between a maximum value and a minimum value of the foot terrain height as a terrain height difference;

[0248] Determine the leg-lifting height difference based on the leg-lifting height, the sample leg-lifting height residual, the terrain height difference, and the leg-lifting height threshold;

[0249] When the leg-lifting height difference is greater than 0, determining the leg-lifting height reward based on the leg-lifting height difference, and the leg-lifting height reward is negatively correlated with the leg-lifting height difference;

[0250] When the leg-lifting height difference is less than or equal to 0, the leg-lifting height reward is determined to be 0.

[0251] Optional, smooth reward, which is used to encourage the legged robot to have smooth gait. The network training module is used to:

[0252] Determining the difference in joint angles of the legged robot at adjacent moments;

[0253] The smoothness reward is determined based on the joint angle difference, and the smoothness reward is negatively correlated with the joint angle difference.

[0254] In summary, in the application embodiment, a parameterized foot trajectory generator is introduced, which supports the adjustment of trajectory generation parameters through a deep neural network. Since the deep neural network combines the proprioceptive information that characterizes the motion state of the legged robot body and the external perception information that characterizes the surrounding environment of the legged robot when adjusting the parameters of the foot trajectory generator, compared with the foot trajectory generator with fixed parameters, the foot trajectory generator that supports parameterization can generate foot trajectories that are more in line with the current environment. Furthermore, controlling the footed robot based on the joint motion parameters output by the foot trajectory generator after parameter adjustment can improve the flexibility and stability of the footed robot in complex environments.

[0255] It should be noted that the apparatus provided in the above embodiments is merely exemplified by the division of the above functional modules. In actual applications, the above functions can be distributed among different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The implementation process is detailed in the method embodiments and will not be repeated here.

[0256] FIG12 shows a block diagram of a legged robot 1200 according to an exemplary embodiment of the present application. The legged robot can be implemented as the server in the above-mentioned solution of the present application. The legged robot 1200 includes a central processing unit (CPU) 1201, a system memory 1204 including a random access memory (RAM) 1202 and a read-only memory (ROM) 1203, and a system bus 1205 connecting the system memory 1204 and the central processing unit 1201. The legged robot 1200 also includes a mass storage device 1206 for storing an operating system 1209, application programs 1210, and other program modules 1211.

[0257] The mass storage device 1206 is connected to the central processing unit 1201 via a mass storage controller (not shown) connected to the system bus 1205. The mass storage device 1206 and its associated computer-readable media provide non-volatile storage for the legged robot 1200. In other words, the mass storage device 1206 may include a computer-readable medium (not shown) such as a hard disk or a Compact Disc Read-Only Memory (CD-ROM) drive.

[0258] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electronically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, tape cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media are not limited to the above-mentioned ones. The above-mentioned system memory 1204 and mass storage device 1206 can be collectively referred to as memory.

[0259] According to various embodiments of the present disclosure, the legged robot 1200 can also be connected to a remote computer on a network such as the Internet for operation. That is, the legged robot 1200 can be connected to a network 1208 via a network interface unit 1207 connected to the system bus 1205. Alternatively, the network interface unit 1207 can be used to connect to other types of networks or remote computer systems (not shown).

[0260] The memory further includes at least one computer program, which is stored in the memory. The central processing unit 1201 implements the control method of the legged robot shown in the above embodiments by executing the at least one computer program.

[0261] An embodiment of the present application also provides a legged robot, which includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the control method of the legged robot provided by the above-mentioned method embodiments.

[0262] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one computer instruction. The at least one computer instruction is loaded and executed by a processor to implement the control method of the legged robot provided by the above-mentioned method embodiments.

[0263] An embodiment of the present application also provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; the computer instructions are read and executed from the computer-readable storage medium by a processor of the legged robot, so that the legged robot executes the control method of the legged robot provided by the above-mentioned method embodiments.

[0264] An embodiment of the present application further provides a chip, which includes a programmable logic circuit or a program, and a device equipped with the chip is used to implement the control method of the legged robot as described above.

[0265] It is understandable that in the specific implementation methods of this application, the data involved, historical data, and portraits and other data related to object data processing related to the identity or characteristics of the object, when the above embodiments of this application are applied to specific products or technologies, need to obtain the object's permission or consent, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0266] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0267] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A control method for a legged robot, the method being performed by the legged robot, the method comprising: Acquire proprioceptive information and external perception information, wherein the proprioceptive information is used to characterize the motion state of the legged robot body, and the external perception information is used to characterize the environmental information around the legged robot; Inputting the proprioceptive information and the external perception information into a deep neural network to obtain a first prediction residual output by the deep neural network, wherein the first prediction residual is used to correct a trajectory generation parameter of a foot trajectory generator; adjusting the trajectory generation parameters of the foot trajectory generator based on the first prediction residual; The motion state of the foot robot is controlled based on the joint motion parameters output by the foot trajectory generator after parameter adjustment.

2. The method according to claim 1, wherein: The first prediction residual includes a step frequency residual and a leg lift height residual, and the trajectory generation parameters of the foot trajectory generator include a step frequency parameter and a leg lift height parameter; The step of adjusting the trajectory generation parameter of the foot trajectory generator based on the first prediction residual comprises: Correcting the reference step frequency based on the step frequency residual to obtain an adjusted step frequency parameter; The reference leg-lifting height is corrected based on the leg-lifting height residual to obtain an adjusted leg-lifting height parameter.

3. The method according to claim 1 or 2, wherein: The output of the deep neural network also includes a second prediction residual, and the second prediction residual is used to correct the joint motion parameters output by the foot trajectory generator; The method of controlling the motion state of the footed robot based on the joint motion parameters output by the foot trajectory generator after parameter adjustment includes: Acquire the joint motion parameters output by the foot trajectory generator after parameter adjustment; modifying the joint motion parameter based on the second prediction residual; The motion state of the legged robot is controlled based on the corrected joint motion parameters.

4. The method according to claim 3, wherein: The deep neural network is an LSTM network, and the acquiring of ontology perception information includes: The received moving direction instruction, the historical motion parameters of the legged robot and the historical prediction residuals are used as the proprioception information, and the historical prediction residuals include the historical first prediction residuals and the historical second prediction residuals of the output of the deep neural network.

5. The method according to any one of claims 1 to 4, wherein: The obtaining of external perception information includes: Acquire a first terrain height map around the foot of the legged robot; and use the first terrain height map as the external perception information; or, Acquire a second terrain height map of a specific shape area below the reference position of the legged robot; and use the second terrain height map as the external perception information.

6. The method according to claim 5, wherein: The step of obtaining a first terrain height map around the foot of the footed robot comprises: Taking the foot of the legged robot as the center, sampling the ground height based on at least two sampling radii to obtain at least two ground sampling point heights; The first terrain height map is generated based on the difference between the height of the foot and the height of the ground sampling point.

7. The method according to any one of claims 1 to 6, wherein: The method further comprises: Acquire sample body perception information and sample external perception information; The sample body perception information and the sample external perception information are input into the deep neural network to obtain the deep The sample prediction residual output by the neural network; Adjusting parameters of the foot trajectory generator based on the sample prediction residual; Controlling the motion state of the footed robot based on the sample joint motion parameters output by the foot trajectory generator after parameter adjustment; determining an exercise reward based on the exercise state; The deep neural network is trained based on the motion reward.

8. The method according to claim 7, wherein: The motion reward includes at least one of an in-command speed reward and an out-of-command speed reward, wherein the in-command speed reward and the out-of-command speed reward are used to encourage the legged robot to move in a desired direction and speed; Determining an in-command speed bonus based on the motion state includes: Acquire the body speed of the legged robot and the moving direction indicated by the moving direction instruction; When the speed of the main body is greater than the lower limit of the expected speed and less than the upper limit of the expected speed, the default reward value is used to determine the first speed reward value; and the first speed reward value is used as the speed reward within the instruction; or, When the body speed is less than the expected speed lower limit, determining a second speed reward value based on the body speed, the moving direction and the expected speed lower limit; using the second speed reward value as the speed reward within the instruction, wherein the second speed reward value is less than the default reward value; or, In the case where the body speed is greater than the expected speed upper limit, determining a third speed reward value based on the body speed, the moving direction and the expected speed upper limit; using the third speed reward value as the speed reward within the instruction, the third speed reward value being less than the default reward value; Determining an out-of-command speed reward based on the motion state includes: The out-of-command speed reward is determined based on the main body speed and the moving direction, and the out-of-command speed reward is negatively correlated with the component speed of the main body speed outside the moving direction.

9. The method according to claim 7 or 8, wherein: The movement reward includes an energy reward, and the energy reward is used to encourage the legged robot to reduce energy consumption during movement; The determining of the exercise reward based on the exercise state comprises: Obtaining joint torque and joint angular velocity of the legged robot; determining a joint motion power based on the joint torque and the joint angular velocity; The energy reward is determined based on the joint movement power, and the energy reward is negatively correlated with the joint movement power.

10. The method according to any one of claims 7 to 9, wherein: The motion reward includes a foot terrain reward, and the foot terrain reward is used to encourage the footed robot to avoid risky terrain; The determining of the exercise reward based on the exercise state comprises: Obtaining the height of the foot terrain of the legged robot; Determine the difference between the maximum value and the minimum value of the foot terrain height as the terrain height difference; In a case where the leg is in a raised state, or in a bottoming state and the terrain height difference is greater than a risk terrain height threshold, determining a first value as the foot terrain reward; or, In a bottoming out state and when the terrain height difference is less than the risk terrain height threshold, a second value is determined as the foot terrain reward, and the second value is less than the first value.

11. The method according to any one of claims 7 to 10, wherein: The movement reward includes a leg-lifting height reward, and the leg-lifting height reward is used to encourage the leg-lifting robot to lower the leg-lifting height; The determining of the exercise reward based on the exercise state comprises: Obtaining the leg-lifting height of the leg-type robot, the foot terrain height, and the sample leg-lifting height residual in the sample prediction residual; Determine the difference between the maximum value and the minimum value of the foot terrain height as the terrain height difference; Determine a leg-lifting height difference based on the leg-lifting height, the sample leg-lifting height residual, the terrain height difference, and the leg-lifting height threshold; When the leg-lifting height difference is greater than 0, the leg-lifting height reward is determined based on the leg-lifting height difference, and the leg-lifting height reward is negatively correlated with the leg-lifting height difference; When the leg-lifting height difference is less than or equal to 0, the leg-lifting height reward is determined to be 0.

12. The method according to any one of claims 7 to 11, wherein: The motion reward includes a smoothness reward, and the smoothness reward is used to encourage the legged robot to have a smooth gait; The determining of the exercise reward based on the exercise state comprises: Determining the difference in joint angles of the legged robot at adjacent moments; The smoothness reward is determined based on the joint angle difference, and the smoothness reward is negatively correlated with the joint angle difference.

13. A control device for a legged robot, the device comprising: An information acquisition module, used to acquire proprioceptive information and external perception information, wherein the proprioceptive information is used to characterize the motion state of the legged robot body, and the external perception information is used to characterize the environmental information around the legged robot; A prediction module, used for inputting the proprioceptive information and the external perception information into a deep neural network to obtain a first prediction residual output by the deep neural network, wherein the first prediction residual is used to correct a trajectory generation parameter of a foot trajectory generator; a parameter adjustment module, configured to adjust the trajectory generation parameter of the foot trajectory generator based on the first prediction residual; A control module is used to control the motion state of the footed robot based on the joint motion parameters output by the foot trajectory generator after parameter adjustment.

14. A legged robot, comprising a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the control method of the legged robot as described in any one of claims 1 to 12.

15. A computer-readable storage medium, wherein at least one computer instruction is stored in the computer-readable storage medium, and the at least one computer instruction is loaded and executed by a processor to implement the control method of the legged robot according to any one of claims 1 to 12.

16. A computer program product, comprising computer instructions, wherein the computer instructions are stored in a computer-readable storage medium; a processor of a legged robot reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the legged robot implements the control method of the legged robot as described in any one of claims 1 to 12.

Images