Terrain-aware pace planning system

By generating body and pace paths in robot data processing hardware, the problem of slow navigation and easy collision in constrained environments is solved, and the robot's fast and balanced navigation in constrained environments is achieved.

CN118915800APending Publication Date: 2024-11-08BOSTON DYNAMICS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410932052.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2019-02-28
Filing Date
2019-08-15
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The prior art is difficult to implement real-time navigation and pace planning of robotic equipment in a constrained environment, resulting in slow navigation and prone to collision with obstacles or losing balance.

Method used

By receiving image data in robot data processing hardware, generating body-obstacle maps, ground height maps and pace-obstacle maps, computers generate robots' body paths and pace paths in the environment, ensuring that the robot avoids obstacles and maintains balance .

Benefits of technology

The robot is able to quickly and efficiently navigate in a constrained environment, avoiding contact with obstacles while maintaining balance and speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915800A_ABST
    Figure CN118915800A_ABST
Patent Text Reader

Abstract

A method includes receiving, at data processing hardware of a robot, sensor data associated with an environment of the robot from at least one sensor; generating, by the data processing hardware, an obstacle map based on the sensor data, the obstacle map indicating obstacles; determining, by the data processing hardware, a first pace path of movement of the robot from the first position to the second position according to the first rhythm of the robot, the first pace path indicating one or more first pace positions; generating, by the data processing hardware, a second pace path for movement of the robot from the first position to a second position based on the first pace path and the obstacle map, the second pace path indicating one or more second pace positions; and indicating, by the data processing hardware, navigation according to the second-step path.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the invention patent application with the application date of August 15, 2019, application number 201980078255.0, and invention name “Terrain-aware pace planning system”. Technical Field

[0002] The present disclosure relates to planning a sequence of steps in the presence of constraints, particularly constraints imposed by terrain. Background Art

[0003] Robotic devices are increasingly being used in constrained or cluttered environments to perform a variety of tasks or functions. These robotic devices may need to navigate through these constrained environments without stepping on or crashing into obstacles. As these robotic devices become more common, real-time navigation and step planning are required to avoid contact with obstacles while maintaining balance and speed. Summary of the invention

[0004] One aspect of the present disclosure provides a method for planning a sequence of steps in the presence of constraints. The method includes receiving image data of an environment surrounding the robot from at least one image sensor at the data processing hardware of the robot. The robot includes a body and legs. The method also includes generating a body-obstacle map, a ground height map, and a step-obstacle map based on the image data by the data processing hardware. The method also includes generating a body path for the movement of the robot's body when maneuvering in the environment based on the body-obstacle map by the data processing hardware, and generating a step path for the robot's legs when maneuvering in the environment based on the body path, the body-obstacle map, the ground height map, and the step-obstacle map by the data processing hardware.

[0005] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, the image data includes three-dimensional point cloud data captured by a three-dimensional volumetric image sensor. At least one image sensor may include one or more of a stereo camera, a scanning light detection and ranging (LIDAR) sensor, or a scanning laser detection and ranging (LADAR) sensor.

[0006] In some examples, the method includes identifying spatial occupancy in an environment based on image data by data processing hardware, and generating a three-dimensional spatial occupancy map by the data processing hardware based on the identified spatial occupancy in the environment. Generating a body-obstacle map, a ground height map, and a step-obstacle map based on image data may include: generating a body-obstacle map based on the three-dimensional spatial occupancy map; generating a ground height map based on the three-dimensional spatial occupancy map; and generating a step-obstacle map based on the ground height map. The ground height map identifies the height of the ground at each location near the robot, and the step-obstacle map identifies step-free areas in the environment that the robot should not enter. Optionally, generating a body-obstacle map includes generating a two-dimensional body-obstacle map based on the three-dimensional spatial occupancy map.

[0007] In some examples, the three-dimensional space occupancy map may include a voxel map with voxels, each voxel representing the three-dimensional space of the environment. Each voxel can be classified as ground, obstacle or other. In addition, the device may include filtering the three-dimensional space occupancy map by data processing hardware to generate a body-obstacle map. In some embodiments, the body path is generated based on the body-free area specified in the body-obstacle map. In some examples, the step path is generated based on the nominal step pattern and step constraints of the nominal gait of the robot. The step constraint may include at least one of the following: a threshold range of the pressure offset center of each leg in contact with the ground, the pressure offset center indicating the acceptable amount of robot weight distribution of each leg at each step; whether the step path causes the leg to step into the step-free area of ​​the obstacle map; whether the step path causes the robot's body to enter the body obstacle; whether the step path causes the robot's self-collision; or the space margin around any step-free area of ​​the obstacle map. In addition, the step constraint may include a soft constraint or a hard constraint. In some embodiments, generating the leg step path of the robot includes refining the generated body path.

[0008] Another aspect of the present disclosure provides a robot. The robot includes: a body; legs, which are connected to the body and configured to maneuver around the environment robot; data processing hardware that communicates with the legs; and memory hardware that communicates with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include receiving image data of the environment around the robot from at least one image sensor. The operations also include generating a three-dimensional space occupancy map based on the identified space occupancy in the environment and generating a two-dimensional body-obstacle map based on the three-dimensional space occupancy map. The operations also include generating a body path for the movement of the robot's body when maneuvering in the environment based on the body-obstacle map and generating a step path for the robot's legs when maneuvering in the environment based on the body path, the body-obstacle map, the ground height map, and the step-obstacle map.

[0009] This aspect may include one or more of the following optional features. In some embodiments, the image data includes three-dimensional point cloud data captured by a three-dimensional volumetric image sensor. In some examples, at least one image sensor includes one or more of a stereo camera, a scanning light detection and ranging (LIDAR) sensor, or a scanning laser detection and ranging (LADAR) sensor.

[0010] In some examples, the operation includes identifying spatial occupancy in the environment based on the image data and generating a three-dimensional spatial occupancy map based on the identified spatial occupancy in the environment. Generating a body-obstacle map, a ground height map, and a step-obstacle map based on the image data may include generating a body-obstacle map based on the three-dimensional spatial occupancy map, generating a ground height map based on the three-dimensional spatial occupancy map, and generating a step-obstacle map based on the ground height map. The ground height map identifies the ground height at each location near the robot, and the step-obstacle map identifies a step-free area in the environment that the robot should not enter. Optionally, generating a body-obstacle map includes generating a two-dimensional body-obstacle map based on the three-dimensional spatial occupancy map.

[0011] The three-dimensional space occupancy map may include a voxel map with voxels, each voxel representing the three-dimensional space of the environment. Each voxel can be classified as ground, obstacle or other. In some examples, the operation also includes filtering the three-dimensional space occupancy map to generate a body-obstacle map. The body path can be based on the body-free area specified in the body-obstacle map, and the step path can be based on the nominal step pattern and step constraints of the nominal gait of the robot. In some embodiments, the step constraint includes at least one of the following: a threshold range of the pressure offset center of each leg in contact with the ground, the pressure offset center indicates the acceptable amount of robot weight distribution of each leg at each step; whether the step path causes the leg to step into the step-free area of ​​the obstacle map; whether the step path causes the robot's body to enter the body obstacle; whether the step path causes the robot's self-collision; or the space margin around any step-free area of ​​the obstacle map. The step constraint may include a soft constraint or a hard constraint. In some embodiments, generating the step path of the robot's legs includes refining the generated body path.

[0012] Another aspect of the present disclosure provides a method, which includes: at data processing hardware of a robot, receiving sensor data associated with the robot's environment from at least one sensor; generating an obstacle map based on the sensor data by the data processing hardware, the obstacle map indicating obstacles; determining by the data processing hardware a first step path for the robot's movement from a first position to a second position based on a first rhythm of the robot, the first step path indicating one or more first step positions; generating by the data processing hardware a second step path for the robot's movement from the first position to the second position based on the first step path and the obstacle map, the second step path indicating one or more second step positions; and navigating by the data processing hardware according to the second step path indication.

[0013] Another aspect of the present disclosure provides a robot, comprising: at least one sensor; data processing hardware; and memory hardware, communicating with the data processing hardware, the memory hardware storing instructions, wherein the instructions are executed by the data processing hardware so that the data processing hardware: receives sensor data associated with the robot's environment from at least one sensor; generates an obstacle map based on the sensor data, the obstacle map indicating obstacles; determines a first step path of the robot's movement from a first position to a second position based on a first rhythm of the robot, the first step path indicating one or more first step positions; generates a second step path of the robot's movement from the first position to the second position based on the first step path and the obstacle map, the second step path indicating one or more second step positions; and indicates navigation according to the second step path.

[0014] Another aspect of the present disclosure provides a computing system, comprising: data processing hardware; and memory hardware, communicating with the data processing hardware, the memory hardware storing instructions, wherein the instructions are executed by the data processing hardware so that the data processing hardware: receives sensor data associated with an environment of a robot from at least one sensor; generates an obstacle map based on the sensor data, the obstacle map indicating obstacles; determines a first step path of the robot moving from a first position to a second position according to a first rhythm of the robot, the first step path indicating one or more first step positions; generates a second step path of the robot moving from the first position to the second position based on the first step path and the obstacle map, the second step path indicating one or more second step positions; and indicates navigation according to the second step path.

[0015] Another aspect of the present disclosure provides a method, including: obtaining, by data processing hardware of the robot, a body-obstacle map that identifies one or more first obstacles in the robot's environment; obtaining, by the data processing hardware, a step-obstacle map that identifies one or more second obstacles in the environment; using the body-obstacle map by the data processing hardware to generate a body path for movement of the robot's body from a starting point to an end point of the body path; using the body path by the data processing hardware to generate a step path that defines one or more step positioning of the robot's legs; and determining, by the data processing hardware, an adjusted step path using the step-obstacle map.

[0016] Another aspect of the present disclosure provides a robot comprising: a body; two or more legs, the two or more legs coupled to the body and configured to maneuver the robot around an environment of the robot; data processing hardware; and memory hardware, communicating with the data processing hardware, the memory hardware storing instructions, wherein the instructions are executed by the data processing hardware so that the data processing hardware: obtains a body-obstacle map for identifying one or more first obstacles in the environment; obtains a step-obstacle map for identifying one or more second obstacles in the environment; using the body-obstacle map, generates a body path for movement of the body from a starting point of the body path to an end point of the body path; using the body path, generates a step path that defines one or more step positioning of a leg among the two or more legs; and using the step-obstacle map, determines an adjusted step path.

[0017] The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic diagram of an example system for planning a sequence of steps in the presence of constraints.

[0019] Figure 2A is an isometric view of a volumetric 3D map of voxels.

[0020] Figure 2B is a perspective view of the environment including the stairs.

[0021] Figure 2C yes Figure 2A Example body-obstacle map of the environment.

[0022] Figure 2D yes Figure 2A An example no-step map of an environment.

[0023] Figure 3 yes Figure 1 A schematic diagram of example components of a control system of the system.

[0024] Figure 4 is a flow chart of an example method for generating a final pace plan.

[0025] Figure 5 is a schematic diagram of an example body path overlaid on an example body-obstacle map.

[0026] Figure 6 is a schematic diagram of step positions associated with a fast pace for following a body path overlaid on an example no-step map.

[0027] Figure 7 is a schematic diagram of the locations of steps associated with slow cadences overlaid on an example no-step map.

[0028] Figure 8 is a schematic diagram of the pace locations associated with a moderate pace overlaid on an example no-pace map.

[0029] Fig. 9 is the final step plan of step positions associated with the selected gait overlaid on the example step-free map.

[0030] Fig.10 is a flow chart of an example method for terrain and constraint planning for pace planning.

[0031] Fig.11 is a flow chart of another example method for terrain and constraint planning for pace planning.

[0032] Fig.12 is a flow chart of another example method for terrain and constraint planning for pace planning.

[0033] Fig.13 is a schematic diagram of an example computing device that can be used to implement the systems and methods described herein.

[0034] The same reference symbols in different drawings denote the same elements. DETAILED DESCRIPTION

[0035] As legged robotic devices (also referred to as "robots") become more common, there is an increasing need for robots to navigate environments that are constrained in a variety of ways. For example, a robot may need to navigate a cluttered room with large and small objects scattered on the floor. Or, as another example, a robot may need to negotiate stairs. Typically, navigating in such environments is a slow and difficult process, which causes legged robots to often stop, collide with objects, and / or become unbalanced. Embodiments herein are directed to systems and methods for generating terrain and constraint plans for step plans in real time, thereby allowing legged robotic devices to quickly and efficiently navigate constrained environments while maintaining smoothness and balance.

[0036] refer to Figure 1 , a robot or robotic device 10 includes a body 11 having two or more legs 12, and executes a step planning system 100 for enabling the robot 10 to navigate in a constrained environment 8. Each leg 12 is connected to the body 11 and may have an upper portion 14 and a lower portion 16 separated by a leg joint 18. The lower portion 16 of each leg 12 terminates in a foot 19. The foot 19 of each leg is optional, and the end of the lower portion of one or more legs 12 may be connected to a wheel. The robot 10 has a vertical gravity axis Vg along the direction of gravity and a center of mass CM, which is the point where the weighted relative position sum of the distributed mass of the robot 10 is zero. The robot 10 also has a posture P based on CM relative to the vertical gravity axis Vg (i.e., a fixed reference frame relative to gravity) to define a specific posture or posture taken by the robot 10. The posture of the robot 10 can be defined by the orientation or angular position of the robot 10 in space. Movement of the legs 12 relative to the body 11 changes the attitude of the robot 10 (ie, a combination of the robot's CM position and the attitude or orientation of the robot 10 ).

[0037] In some embodiments, the robot 10 also includes one or more accessories, such as an articulated arm 20 disposed on the body 11 and configured to move relative to the body 11. The articulated arm 20 can have five or more degrees of freedom. In addition, the articulated arm 20 can be interchangeably referred to as a manipulator arm or simply an accessory. In the example shown, the articulated arm 20 includes two parts 22, 24 that can rotate relative to each other and the body 11; however, the articulated arm 20 can include more or fewer parts without departing from the scope of the present disclosure. The first part 22 can be separated from the second part 24 by an articulated arm joint 26. An end effector 28 (which can be interchangeably referred to as a manipulator head 28) can be connected to the distal end of the second part 24 of the articulated arm 20, and can include one or more actuators 29 for grasping / grasping an object.

[0038] The robot 10 also includes a vision system 30 having at least one imaging sensor or camera 31, each sensor or camera 31 capturing image data or sensor data of an environment 8 surrounding the robot 10 at a viewing angle 32 and within a field of view 34. The vision system 30 can be configured to move the field of view 34 in any direction by adjusting the viewing angle 32 or by panning and / or tilting (independently or via the robot 10) the camera 31. Alternatively, the vision system 30 can include multiple sensors or cameras 31 so that the vision system 30 captures a substantially 360-degree field of view around the robot 10. The vision system 30 provides image data or sensor data 17 derived from the image data captured by the camera or sensor 31 to the data processing hardware 36 of the robot 10. The data processing hardware 36 is in digital communication with the memory hardware 38, and in some embodiments, can be a remote system. The remote system can be a single computer, multiple computers, or a distributed system (e.g., a cloud environment) with scalable / elastic computing resources and / or storage resources. The step planning system 100 of the robot 10 is executed on the data processing hardware 36. In the example shown, the step planning system 100 includes a perception system 110 that receives image or sensor data 17 from the vision system 30 and generates one or more maps 112, 114, 116 indicating obstacles in the surrounding environment 8. The step planning system 100 also includes a control system 300 that receives the maps 112, 114, 116 generated by the perception system 110 and generates a body path or trajectory 510 ( Figure 5 ), and generates a step path or step plan 350 using the body path 510. Using the step plan 350, the robot 10 maneuvers through the environment 8 in accordance with the step plan 350 by placing the foot 19 or the distal end of the leg 12 at the position indicated by the step plan 350. In some embodiments, at least a portion of the step planning system 100 is executed on a remote device that communicates with the robot 10. For example, the perception system 110 can be executed on the remote device to generate one or more maps 112, 114, 116, and the control system 300 executed on the robot 10 can receive the maps 112, 114, 116 from the remote device. Here, the control system 300 can generate the body path 510 and the step path 350. Optionally, the entire walking planning system 100 can be executed on the remote device, and the remote device can control / instruct the robot 10 to maneuver the environment 8 based on the body path 510 and the step path 350.

[0039] In some embodiments, the camera 31 of the vision system 30 includes one or more stereo cameras (e.g., one or more RGBD stereo cameras). In other examples, the vision system 30 includes one or more radar sensors, such as a scanning light detection and ranging (LIDAR) sensor, a scanning laser detection and ranging (LADAR) sensor, a light scanner, a time-of-flight sensor, or any other three-dimensional (3D) volumetric image sensor (or any combination of such sensors). In some embodiments, the vision system 30 identifies spatial occupancy in the environment 8 based on captured image or sensor data 17. The perception system 110 may use the image data 17 captured by the vision system 30 to generate a 3D point cloud. A point cloud is a set of data points representing the surface of objects in the environment 8 surrounding the robot 10. Based on the point cloud, the perception system 110 may generate a 3D spatial occupancy map 200 ( Figure 2A In some examples, perception system 110 generates voxels 210, 212 ( Figure 2A ). Each voxel 210, 212 (i.e., a cube) represents the 3D space of the environment. The size of each voxel 210, 212 depends on the fidelity of the perception system 110 and the processing power of the vision system 30 and the data processing hardware 36. For example, the robot 10 can generate a voxel map 200 (i.e., a 3D occupancy map) of the environment 8 (e.g., a few meters in each direction) surrounding the robot 10, where each voxel 210, 212 is a 3 cm cube. For each voxel, the perception system 110 can store various statistical data.

[0040] In some embodiments, the perception system 110 classifies each voxel 210, 212 containing an object (e.g., using a classification algorithm, such as a linear classifier, a decision tree, a neural network, dedicated logic, etc.) as ground 9, an obstacle, or other. When the perception system 110 determines that the robot 10 can step on the point or space represented by the voxel 210, 212, the perception system 110 classifies the voxel 210 as ground 9. For example, the robot 10 can classify the surface of a sidewalk or a step as ground 9. When the perception system 110 determines that the robot 10 cannot step on the point or space represented by the voxel 210, 212, the perception system 110 classifies the voxel 212 as an obstacle. For example, the perception system 110 classifies objects that are too high for the robot's legs to reach or objects that would cause the robot 10 to lose stability (i.e., balance) if stepped on as obstacles. The third classification, i.e., other, can be used for voxels 210, 212 that the robot 10 can safely ignore or neglect. For example, the perception system 110 classifies an object far above the robot 10 or an object far away from the robot 10 as others. Figure 2AAn example of a simple voxel map 200, 200a is shown, which includes a plane of ground voxels 210, 210a-n and a set of obstacle voxels 212, 212a-n (ie, chairs).

[0041] Using the volumetric 3D map 200 including the classified voxels 210, 212, the perception system 110 generates a body-obstacle map 112. In some embodiments, the body-obstacle map 112 represents a two-dimensional (2D) map that labels or illustrates "forbidden areas" or "body-free areas" of the body 11 of the robot 10. That is, the body-obstacle map 112 is a 2D map that marks each location (i.e., a pixel of the map 112, each pixel representing a column of space in the environment 8 of the robot 10) as a location where the body 11 of the robot 10 can safely travel through or where the body 11 of the robot 10 can not safely travel through. The body-obstacle map 112 may include a grid of cells (e.g., pixels), where each cell of the grid contains a Boolean value (e.g., the body can enter or the body cannot enter). For example, referring to Figure 2B , view 201 shows an environment 8 including stairs with railings. When the robot 10 ascends or descends the stairs, the railings will act as a barrier to the body 11 of the robot 10 (ie, the railings are at a height that will contact the body 11). Figure 2C Shown Figure 2B The body-obstacle map 112 of the 2D image of the stairs (i.e., the plan view of the stairs). Figure 2C In the figure, the illegal body area (eg, obstacle pixel) 212 (ie, prohibited area) represents an area that the body of the robot 10 cannot or should not enter (eg, stair railings, walls, large obstacles, etc.).

[0042] refer to Figure 1 and 2D , the perception system 110 also uses the volumetric 3D map 200 (or the ground height map 116, as discussed in more detail below) to generate a step-obstacle map 114. In some examples, the step-obstacle map 114 represents a 2D plan view showing prohibited or "no step" areas 213 for steps of the legs 12 of the robot 10. That is, the step-obstacle map 114 is similar to the body-obstacle map 112, however, the prohibited areas 213 instead represent areas where the steps of the robot 10 (i.e., the distal ends of the feet 19 or legs 12) should not "touch the ground." That is, although the foot 19 or the distal ends of the legs 12 can cross the prohibited areas 213, the foot 19 may not complete a step within the area 213. The step-obstacle map 114 may include a grid of cells (e.g., pixels), where each cell of the grid contains a Boolean value (e.g., a step or no step).

[0043] Different considerations may be used to generate the step-obstacle map 114 and the body-obstacle map 112, which may result in some obstacles being classified as body obstacles, step obstacles, body and step obstacles, or neither. For example, the legs 12 of the robot 10 support the body 11 at a certain distance above the ground 9, so that the body 11 can safely avoid obstacles near the ground 9. The step-obstacle map 114 may also consider aspects such as how high the robot 10 can walk with the legs 12. In addition, in some examples, the knees (i.e., leg joints 18) of the robot 10 may extend outward in front of or behind the feet 19, thereby limiting where the feet 19 can be placed (e.g., the knees may hit a sufficiently high obstacle before the feet 19 can be lifted and placed on the obstacle). Therefore, the prohibited area 213 may include an area on the ground 9 where there are no obstacles, but due to the geometry and posture of the robot 10, the robot 10 passing through the prohibited area 213 may cause the body 11 of the robot 10 to contact an obstacle above the ground 9.

[0044] Typically, obstacles that are classified as body obstacles are also classified as step obstacles, but the reverse is not necessarily true, as a step obstacle may not be classified as a body obstacle (e.g., an obstacle is high enough to cause problems when stepping but low enough that the robot 10's body 11 does not contact the obstacle). In some cases, a body obstacle may not be classified as a step obstacle. For example, a table can be a body obstacle, but the robot 10 can walk under the table. The perception system 110 can classify a body obstacle as a large step obstacle because it is not feasible to place a foot directly next to the body obstacle (i.e., increase the size of the body obstacle).

[0045] In some embodiments, the perception system 110 classifies large areas of gait obstacles as body obstacles. For example, if an area of ​​the environment 8 contains a particularly dense number of gait obstacles such that traversing the area would be difficult, the perception system 110 may classify the entire area as a body obstacle, even though the height of the obstacles does not affect the body of the robot 10 to obtain a better final gait plan 350 (as discussed in more detail below). In some embodiments, the perception system 110 classifies an area as a body obstacle to ensure that the robot 10 does not enter a particular area for reasons other than collision with an object. For example, a user may wish to direct the robot 10 in a certain direction or along a certain path.

[0046] Figure 2D Pace - Obstacle Map 114 Representatives Figure 2B Stairs. Figure 2CLike the body-obstacle map 112 of the robot 10, the step-obstacle map 114 depicts areas 213 that the perception system 110 determines are not safe or valid for the robot 10 to walk on, as well as safe or valid areas 210. For example, the ground 9 in front of the stair case and each individual step are marked as valid in the step-obstacle map 114.

[0047] Back to Figure 1 In some embodiments, the perception system 110 also generates a ground height map 116 from the 3D volume map 200. The ground height map 116 identifies the height of the ground 9 at each location near the robot 10. That is, similar to a topographic map, the ground height map 116 is a 2D map that records the height of the ground 9 at each location in the horizontal plane relative to a reference point or height. In some examples, the ground height map 116 only shows the height of the ground 9, without showing any surface above the ground 9. That is, the ground height map 116 can mark the height of the ground 9 under the table, rather than the height of the table surface. The ground height map 116 can be used to help generate a step-obstacle map 114 (e.g., determine when the ground is too high or too steep to pass safely and should be marked as a step obstacle). The perception system 110 generates the ground height map 116, for example, by determining the height of the voxels 210 classified as the ground 9 in each column of the 3D map. The step-obstacle map 114 can be generated from the ground height map 116. The perception system optionally processes both the body-obstacle map 112 and the step-obstacle map 114 as signed distance fields (ie, using a signed distance function).

[0048] Reference now Figure 3 , the control system 300 of the step planning system 100 receives the maps (body-obstacle map 112, step-obstacle map 114, and ground height map 116) from the perception system 110 and generates a step plan 350 (i.e., a map of locations where the robot 10 places its feet 19) for use by the robot 10 to navigate the environment 8. In some embodiments, the control system 300 includes a body path generator 310 and a constrained step planner 320.

[0049] The body path generator 310 receives the body-obstacle map 112 and the location 311 to which the robot 10 is to navigate (i.e., where the robot 10 intends to go) from the perception system 110. The body path generator 310 then generates a body trajectory 510 (i.e., a path to be followed by the body 11 of the robot 10) that avoids obstacles in the body-obstacle map 112 (i.e., where the robot 10 intends to go) as the robot 10 maneuvers in the environment 8. Figure 5 ) marked with physical obstacles 520( Figure 5). The body path generator 310 generates a body trajectory or body path 510 using a non-resource-intensive method or algorithm (e.g., a potential field method, a rapidly exploring random tree, and / or a trajectory optimizer). For example, using a potential field method, a simplified model of the object 11 (e.g., momentum is not considered, and only position is considered to plan velocity) is used to quickly generate a planar trajectory representing an approximate path 510 for the robot 10 to traverse. The planar trajectory may include the horizontal motion of the CM and the yaw of the robot 10. Although not necessarily optimal, the body trajectory 510 quickly provides a good approximation of the path, which provides an ideal starting point for further path optimization. It is worth noting that the control system 300 generates the body trajectory 510 without using the step-obstacle map 114, so the body path 510 does not provide where the robot 10 should walk when following the body trajectory 510.

[0050] Continue to refer Figure 3 , the constrained step planner 320 receives the body trajectory 510 from the body path generator 310 as a starting point for generating a final constrained step position (e.g., a step plan 350). In some examples, the constrained step planner 320 includes a gait determiner 330 that first determines a gait timing 332 that provides a nominal step position for the robot 10. That is, the gait determiner 330 determines which gait (e.g., a slow walk, a fast walk, a trot, etc.) provides a nominal step position relative to the step-obstacle map 114 ( Figure 6 ) in the step obstacle 620 ( Figure 6 ). Optionally, the gait determiner 330 is separate from the constrained step planner 320. In some embodiments, the gait determiner 330 provides the determined gait timing 332 to the step solver 340. As described in more detail below, the step solver 340 receives the gait timing 332 and one or more constraints 342, 342a–n. The step solver 340 applies the constraints 342 to the nominal step position of the determined gait timing 332 and solves the optimized step plan 350. As described in more detail below, in some embodiments, the constraints 342 include a center of pressure (CoP) offset constraint 342a, a body prohibition constraint 342b, a step prohibition constraint 342c, a self-collision constraint 342d, a prohibition margin constraint 342e, and a balance constraint 342f. In addition to or in place of one or more constraints 342a-342f, the constraints 342 may also include one or more other constraints.

[0051] In some embodiments, the constrained step planner 320 receives various other information. For example, the constrained step planner 320 may receive the current position and speed of the CM of the robot 10, foot contact and take-off information (e.g., timing), and swing foot position and / or speed. The constrained step planner 320 may also receive the body-obstacle map 112. In some embodiments, the constrained step planner 320 adjusts or refines the body path trajectory 510. The adjustment may be small. For example, the constrained step planner 320 may take into account the sway of the body 11 when traversing the environment 8 (which is not considered in the simplified body path trajectory 510). In some cases, the adjustment may be large. For example, the simplified body trajectory 510 may be physically impossible (e.g., including infinite acceleration), or may be difficult to solve once the gait timing 332 is determined. In some embodiments, the constrained step planner 320 only adjusts the translation of the body 11 of the robot 10 instead of the yaw trajectory, while in other embodiments, the translation and yaw of the body 11 are adjusted.

[0052] Figure 4 An example flow chart 400 depicting the process flow of the step planning system 100 is shown. At step 402, the perception system 110 creates a body-obstacle map 112, and at step 404, the control system 300 uses the body-obstacle map 112 to generate a body trajectory or body path 510. The perception system 110 also creates a step-obstacle map 114 at step 406, and at step 408, the control system 300 selects a gait timing 332 via the gait determiner 330 of the constrained step planner 320 using the planar body path 510 and the step-obstacle map 114. At 410, the step solver 340 of the constrained step planner 320 uses the selected gait timing 332, the body-obstacle map 112, and the step-obstacle map 114 to solve the final step plan 350 (i.e., the location where the robot 10 places its feet 19).

[0053] Figure 5A schematic diagram 500 depicting a body path 510 for navigating around a body obstacle 520 is shown. For example, the flowchart 400 depicts the control system 300 using the body-obstacle map 112 generated in step 402 to generate the body path 510 in step 404. Using, for example, a potential field method, the body path generator 310 of the control system 300 draws the body path 510 from point A to point B to navigate around one or more body obstacles 520. The body obstacle 520 may also be referred to as a body obstacle zone 520, wherein if the body 11 of the robot 10 passes through / enters the body obstacle zone 520, the body 11 will contact one or more obstacles. That is, the area defined by the body obstacle zone 520 does not indicate the body obstacle itself, but rather indicates an area that the body 11 of the robot 10 is not allowed to enter because the body 11 will contact one or more obstacles. The body path generator 310 ensures the validity of the body path 510 by generating a simulated body 530 of the robot 10 traveling along the path 510. For example, when the simulated body 530 does not contact any body obstacle area 520, a valid path is generated.

[0054] As previously described, after receiving the step-obstacle map 114 from the perception system 110, the gait determiner selects a gait to generate a nominal step position. Figure 6-8 In some implementations, the gait determiner 330 of the constrained step planner 320 analyzes multiple potential gaits to find an optimal nominal step position. Figure 6 A schematic diagram 600 is shown depicting a step position 630 associated with a fast cadence for following a body path 510 plotted on a step-obstacle map 114. For example, the flowchart 400 depicts the control system 300, via the gait determiner 330 of the constrained step planner 320, using the planar body path 510 generated at step 404 and the step-obstacle map 114 generated at step 406 to select a gait timing 332 having a fast cadence for the step position 630 at step 408. As used herein, the terms "foot position(s)" and "step position" may be used interchangeably.

[0055] The gait determiner 330 starts with the body path 510 plotted on the step-obstacle map 114 and overlays a selected cadence (i.e., where the robot 10 would walk if the body 11 followed the body path 510 and the legs 12 moved at the selected cadence). In the example shown, the body path 510 may intersect one or more step obstacles 620, but not the body obstacles 520 (this is pre-ensured by the body path generator 310). Each step position 630, 630a-n is plotted and evaluated. In some embodiments, the gait determiner 330 generates a score that reflects the quality of the step position 630 of the current simulated gait timing. Due to the number of small collisions between the step position 630 and the step obstacle 620 (e.g., locations where the step position 630 overlaps the step obstacle 620), Figure 6 The score for a fast pace of 620 may be relatively low. The score may be affected by the number of collisions with the step obstacles 620 and the severity of the collisions. The gait determiner 330 may emphasize the distance that the step position 630 must move to avoid the obstacles 620. For example, a step position 630 that slightly collides with three obstacles 620 may be better than a step position 630 that severely collides with a single obstacle 620.

[0056] Figure 7 A schematic diagram 700 is shown that depicts step positions 630 associated with a slow cadence for following a body path 510 plotted on a step-obstacle map 114. Figure 6 The fast-paced pace associated with the position 630, Figure 7 The slow cadence of shows multiple step positions 630 within or contacting the step obstacle 620, resulting in a non-ideal score. On the other hand, Figure 8 The schematic diagram 800 depicts the current and the moderate cadence (i.e., the cadence of the body path 510 plotted on the step-obstacle map 114) for following the body path 510 plotted on the step-obstacle map 114. Figure 6 The fast pace is slower, but better than Figure 7 The medium rhythm has the lowest number of collisions between the step position 630 and the step obstacle 620, so the highest score can be obtained among the slow, medium and fast rhythms. Although only three rhythms are illustrated, it should be understood that the gait determiner 330 can evaluate any number of gait timings before selecting a specific rhythm. The score assigned to each analyzed gait timing can reflect the amount of optimization required to meet a given constraint. The more constraints violated by the nominal step position 630 (such as collisions with the step obstacle 620), the more optimization may be required. In addition to the step obstacle and body obstacle constraints, the score can reflect other constraints (such as speed constraints). For example, for some tasks or environments, a slow rhythm may be more weighted than a fast rhythm.

[0057] Reference now Fig. 9 Once the gait determiner selects the gait timing 332 and the nominal step position 630, the gait solver 340 of the constrained step planner 320 solves the final step plan 350 (eg, step 410 of flowchart 400). Fig. 9 A schematic diagram 900 is shown, which depicts a final step position 630 associated with a selected gait timing 332 (e.g., cadence) for following a body path 510 plotted on an obstacle map 114. The step solver 340 accepts a plurality of constraints 342 (i.e., variables) that the step solver 340 considers when solving. In some examples, each constraint 342 is a "hard" constraint or a "soft" constraint. A hard constraint is a constraint 342 that the step solver 340 cannot violate and still have a successful step plan 350. For example, avoiding a step obstacle 620 (e.g., the edge of a fall) can be marked as a hard constraint because stepping on a particular step obstacle 620 may result in a catastrophic outcome (e.g., the edge of a fall). A soft constraint is a constraint 342 that the step solver 340 will attempt to satisfy, but may violate if a successful step plan 350 is required. For example, the step solver 340 may have a constraint 342 (i.e., a "margin" constraint 342e) that does not enter a threshold distance of a step obstacle 620. Although maintaining a threshold distance is ideal, the pace solver 340 may intrude upon boundaries if necessary (e.g., to ensure compliance with hard constraints). Soft constraints may be weighted. That is, each constraint 342 may be assigned a particular weight or "softness" that allows the pace solver 340 to determine which constraint to violate first if all constraints cannot be satisfied. For example, if the pace solver 340 has a margin constraint 342e and a balance constraint 342f (i.e., a requirement to maintain balance of the robot 10), the balance constraint 342f may be weighted more (e.g., "harder") than the margin constraint 342e, because maintaining balance may be more important than maintaining a margin from the pace obstacle 620. Therefore, the pace solver 340 may choose to violate the margin constraint 342e first, or to a greater extent than the balance constraint 342f.

[0058] Constraint 342 can be a (hard or soft) equality constraint (e.g., x=5) or an inequality constraint (e.g., x<=5). In some embodiments, the pace solver 340 also strives to minimize the cost while obeying (or attempting to obey) constraint 342. The cost may be equivalent to a soft equality constraint. That is, in some cases, a soft constraint may be considered by the solver 340 as a cost to be minimized. Some constraints (e.g., balance constraint 342f) may be considered as costs or soft constraints. For example, if there is no constraint that x is greater than 5, adding the example cost and the example inequality constraint will have the same effect (assuming equal weights). However, if x is less than 5 without the constraint, adding the cost (or equality constraint) will cause x to be closer to 5, but the inequality constraint will have no effect. In some embodiments, the pace solver 340 uses costs and soft inequality constraints instead of hard constraints or equality constraints.

[0059] The pace solver 340 can solve any number of constraints 342. As previously described ( Figure 3 ), the step solver 340 may have constraints 342b, 342c for avoiding step / body obstacle areas, a margin constraint 342e for maintaining a threshold distance from the step obstacle 620, and a balance constraint 342f for maintaining balance and / or stability. In other examples, the step solver 340 may receive a pressure offset center constraint 342a, which includes a threshold range of pressure offset centers for the legs 12 in contact with the ground 9. The pressure offset center may indicate an acceptable amount of robot 10 weight distribution for each leg 12 at each step (i.e., the weight distribution between the legs 12 in contact with the ground 9). That is, the pressure offset center constraint 342a ensures that the percentage of the weight of the robot 10 applied to the robot step is valid. For example, when both feet are in contact with the ground 9, the step solver 340 may be constrained to not apply a 120% vertical force to the first foot (e.g., 20% more than the entire weight of the robot 10) and a -20% vertical force to the second foot, because such expertise is not possible. In another example, the step solver 340 can receive a self-collision constraint 342d. That is, the constraint 342d ensures that the step solver 340 attempts to not cause the robot 10 to collide with itself (i.e., place the first foot 19 where the second foot 19 is already located). The constraints 342 can be predetermined before navigation. The constraints 342 can also be modified, added, or removed during navigation. In some examples, the constraints 342 are received from a source external to the control system (e.g., a user or manufacturer of the robot 10), while in other examples, the step planning system 100 autonomously generates the constraints 342 based on data received from sensors of the robot 10.

[0060] In order to satisfy the constraints 342 assigned to the step solver 340, the step solver 340 may adjust each step position 630. Fig. 9 , the step obstacle avoidance constraint 342c can "push" or otherwise adjust the step position 630 away from the obstacle. For example, due to the prohibition boundary 620 (which is generated in response to the prohibition constraint 342c), the step position 630a is moved to the step position 630b. When the step solver 340 modifies the position of the step position 630 from the original nominal step position, the adjustment can cascade or ripple into changes in other step positions 630. For example, if Fig. 9 As shown, in response to the step solver 340 previously adjusting the step position 630a, the step position 630c may be moved to the step position 630d in order to maintain balance as the robot 10 moves along the body trajectory 510. When the step solver 340 completes the final constrained step position plan 350, the robot 10 may then begin walking, placing its feet relative to the determined step position 630. During walking, the robot 10 may continuously rerun or regenerate the step plan 350 based on the latest map 112, 114, 116 (with the same or updated constraints 342) received from the perception system 110, and in response, adjust or change the step plan 350 appropriately.

[0061] Ideally, the pace solver 340 uses quadratic programming so that the pace solver 340 can solve the pace plan 350 in real time. Quadratic programming uses linear constraints to quickly solve optimization problems. That is, in some examples, the pace solver 340 minimizes or maximizes a quadratic function of several variables that are linearly constrained. Still referring to Fig. 9 To linearly constrain the amorphous shaped obstacle region, the pace solver 340 may draw a series of straight lines 920 to approximate the shape of the obstacle. In some examples, the pace solver 340 applies linear constraints only to the relevant portions of the obstacle. In other examples, the pace solver 340 initially solves the pace plan 350 without any constraints 342, and then iteratively adds constraints 342 and regenerates intermediate pace plans until the pace solver 340 achieves an optimized final pace plan 350.

[0062] The step solver 340 can start by solving a convex constraint. The solver 340 can then use the solution from the convex constraint to iteratively determine the best linear approximation for the non-convex constraint. For example, based on the current position and speed of the swing foot 19, the known time before contact (i.e., between the foot 19 and the ground 9), and the maximum acceleration of the foot 19, the solver 340 can determine a rectangular area where the foot 19 can touch the ground. Similarly, other shapes can approximate other areas. For example, because each leg 12 has a maximum length, the foot 19 touches the ground and does not occur too far from the hips. The area can be represented as an octagon. The foot 19 leaving the ground can be similarly approximated to the foot 19 touching the ground, but a rectangle (as opposed to an octagon) can be used instead. The standing leg 12 can have a trapezoidal boundary to prevent self-collision.

[0063] Thus, the step planning system 100 of the robot 10 separates the approximation and determination of the body path 510 from the determination of the precise step plan 350. By first quickly approximating the body trajectory 510, the control system 300 generates a reasonable first-pass solution that can be used to quickly optimize the precise final step plan 350, which would otherwise be computationally inefficient. Therefore, as the robot 10 maneuvers in the environment 8, the step plan 350 can be regenerated at a high frequency (e.g., 300 Hz) to enable real-time navigation. The perception system 110 can operate at a frequency different from that of the control system. That is, a new map can be provided to the control system 300 at a rate different from (e.g., slower) than the rate at which the control system 300 determines the step plan 350. The high frequency regeneration of the control system 300 allows the robot 10 to quickly adapt to new perception data (e.g., a newly detected object), quickly react to surprising dynamics (e.g., maintain balance after being pushed or collided), or respond to new requirements (e.g., increase or decrease speed).

[0064] Fig.10 1 is a flow chart of an example method 1000 for terrain and constraint planning step planning. The flow chart begins at operation 1002, where image data 17 of an environment 8 surrounding the robot 10 is received from at least one image sensor 31 at the data processing hardware 36 of the robot 10. The image sensor 31 may include one or more of a stereo camera, a scanning light detection and ranging (LIDAR) sensor, or a scanning laser detection and ranging (LADAR) sensor. In some embodiments, the image data 17 includes three-dimensional point cloud data captured by a three-dimensional volumetric image sensor. The robot 10 includes a body 11 and legs 12. At step 1004, the method 1000 includes generating, by the data processing hardware 36, a body-obstacle map 112, a step-obstacle map 114, and a ground height map 116 based on the image data 17.

[0065] At step 1006, the method 1000 includes generating, by the data processing hardware 36, a body path 510 for movement of the body 11 of the robot 10 when maneuvering in the environment 8 based on the body-obstacle map 112. At step 1008, the method 1000 includes generating, by the data processing hardware 36, a step path 350 for the leg 12 of the robot 10 when maneuvering in the environment 8 based on the body path 510, the body-obstacle map 112, the step-obstacle map 114, and the ground height map 116.

[0066] Fig.11 1 is a flow chart of another example method 1100 for terrain and constraint planning step planning. The flow chart begins at operation 1102, where image data 17 of an environment 8 surrounding the robot 10 is received from at least one image sensor 31 at the data processing hardware 36 of the robot 10. The image sensor 31 may include one or more of a stereo camera, a scanning light detection and ranging (LIDAR) sensor, or a scanning laser detection and ranging (LADAR) sensor. In some embodiments, the image data 17 includes three-dimensional point cloud data captured by a three-dimensional volumetric image sensor. The robot 10 includes a body 11 and legs 12. At step 1104, the method 1100 includes identifying, by the data processing hardware 36, spatial occupancy in the environment 8 based on the image data 17. At step 1106, the method 1100 includes generating, by the data processing hardware 36, a three-dimensional spatial occupancy map 200 based on identifying spatial occupancy in the environment 8. In some examples, the three-dimensional spatial occupancy map 200 includes a voxel map 200, 200a having voxels 212, each voxel 212 representing a three-dimensional space of the environment 8. Each voxel 212 may be classified as ground 9, obstacle, or other. At step 1108, the method 1100 includes generating, by the data processing hardware 36, a two-dimensional body-obstacle map 112 based on the three-dimensional spatial occupancy map 200. At step 1110, the method 1100 includes generating, by the data processing hardware 36, a ground height map 116 based on the three-dimensional spatial occupancy map 200. The ground height map 116 identifies the height of the ground 9 at each location near the robot 10.

[0067] At step 1112, the method 1100 includes generating, by the data processing hardware 36, a step-obstacle map 114 based on the ground height map 116, the step-obstacle map 114 identifying no-step regions 213 in the environment 8 that the robot 10 should not step into. At step 1114, the method 1100 includes generating, by the data processing hardware 36, a body path 510 for movement of the body 11 of the robot 10 when maneuvering the robot 10 in the environment based on the two-dimensional body-obstacle map 112. In some examples, the body path 510 is based on the no-body regions specified in the two-dimensional body-obstacle map 112. At step 1116, the method 1100 includes generating, by the data processing hardware 36, a step path 350 for movement of the legs 12 of the robot 10 when maneuvering the robot 10 in the environment 8 based on the body path 510, the body-obstacle map 112, the step-obstacle map 114, and the ground height map 116. The step path 350 may be based on a nominal step pattern of a nominal gait of the robot 10 and the step constraints 342. In some embodiments, generating a step path 350 for a leg 12 of the robot 10 includes refining the generated body path 510. In some embodiments, the step constraints 342 include at least one of: a threshold range of a pressure excursion center for each leg 12 in contact with the ground, the pressure excursion center indicating an acceptable amount of robot weight distribution for each leg 12 at each step; whether the step path 350 causes a leg to step into a no-step region 213 of a step-obstacle map 114; whether the step path 350 causes the body 11 of the robot 10 to enter a body obstacle 520; whether the step path 350 causes a self-collision of the robot 10; or a spatial margin around any no-step region 213 of the step-obstacle map 114. Optionally, the step constraints 342 include soft constraints or weighted constraints.

[0068] In some implementations, the method 1100 includes filtering, by the data processing hardware 36, the three-dimensional spatial occupancy map 200 to generate the two-dimensional body-obstacle map 112. The filtering may fill in gaps around incompletely observed obstacles and / or remove spurious data from the map 112.

[0069] Fig.12 1 is a flow chart of another example method 1200 for terrain and constraint planning step planning. The flow chart begins at operation 1202, where a two-dimensional body-obstacle map 112, a step-obstacle map 114, and a ground height map 116 are received at the data processing hardware 36 of the robot 10. Here, the data processing hardware 36 of the robot 10 can obtain the maps 112, 114, 116 from a remote device in communication with the data processing hardware. For example, the remote device can obtain the maps 112, 114, 116 from the vision system 30 ( Figure 1 ) receiving image data 17( Figure 1), and generate maps 112, 114, 116 based on the image data 17, as shown in the above reference Figure 1 The ground height map 116 identifies the height of the ground 9 at each location near the robot 10. The step-obstacle map 114 identifies places in the environment 8 that the robot 10 should not step into. At step 1204, the method 1200 includes generating, by the data processing hardware 36, a body path 510 for movement of the body 11 of the robot 10 when maneuvering the robot 10 in the environment based on the two-dimensional body-obstacle map 112. At step 1206, the method 1200 includes generating, by the data processing hardware 36, a step path 350 for movement of the legs 12 of the robot 10 when maneuvering the robot 10 in the environment 8 based on the body path 510, the body-obstacle map 112, the ground height map 116, and the step-obstacle map 114.

[0070] Fig.13 1 is a schematic diagram of an example computing device 1300 (e.g., data processing hardware 36 and memory hardware 38) that can be used to implement the systems and methods described in this document. The components shown here, their connections and relationships, and their functions are merely exemplary and are not meant to limit the implementation of the inventions described and / or claimed in this document.

[0071] The computing device 1300 includes a processor 1310 (e.g., data processing hardware 36), a memory 1320 (e.g., memory hardware 38), a storage device 1330, a high-speed interface / controller 1340 connected to the memory 1320 and a high-speed expansion port 1350, and a low-speed interface / controller 1360 connected to a low-speed bus 1370 and the storage device 1330. Each component 1310, 1320, 1330, 1340, 1350, and 1360 are interconnected using various buses and can be installed on a common motherboard or in other appropriate ways. The processor 1310 can process instructions for execution within the computing device 1300, including instructions stored in the memory 1320 or on the storage device 1330, to display graphical information of a graphical user interface on an external input / output device (such as a display 1380 connected to the high-speed interface 1340). In other embodiments, multiple processors and / or multiple buses and multiple memories and multiple types of memories can be used appropriately. Furthermore, multiple computing devices 1300 may be connected, with each device providing portions of the necessary operations (eg, as a server bank, a group of blade servers, or a multi-processor system).

[0072] Memory 1320 stores information non-temporarily within computing device 1300. Memory 1320 may be a computer-readable medium, a volatile storage unit, or a non-volatile storage unit. Non-temporary memory 1320 may be a physical device used to temporarily or permanently store programs (e.g., instruction sequences) or data (e.g., program state information) for use by computing device 1300. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electrically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0073] Storage device 1330 can provide mass storage for computing device 1300. In some embodiments, storage device 1330 is a computer readable medium. In various embodiments, storage device 1330 can be a floppy disk device, a hard disk device, an optical disk device or a tape device, a flash memory or other similar solid-state storage device, or a device array, including a device in a storage area network or other configuration. In other embodiments, a computer program product is tangibly contained in an information carrier. The computer program product includes instructions that perform one or more methods when executed, such as the above method. The information carrier is a computer or machine readable medium, such as a memory on memory 1320, storage device 1330, or processor 1310.

[0074] The high-speed controller 1340 manages bandwidth-intensive operations of the computing device 1300, while the low-speed controller 1360 manages less bandwidth-intensive operations. This division of responsibilities is exemplary only. In some embodiments, the high-speed controller 1340 is coupled to the memory 1320 and the high-speed expansion port 1350, which can accept various expansion cards (not shown). In some embodiments, the low-speed controller 1360 is coupled to the storage device 1330 and the low-speed expansion port 1390. The low-speed expansion port 1390 can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), and can be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or a router, for example, through a network adapter.

[0075] Various implementations of the systems and techniques described herein can be realized in digital electronic and / or optical circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system that includes at least one programmable processor, which may be special purpose or general purpose, coupled to receive data and instructions from and send data and instructions to a storage system, at least one input device, and at least one output device.

[0076] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0077] The processes and logic flows described in this specification can be performed by one or more programmable processors (also referred to as data processing hardware), which execute one or more computer programs to perform functions by operating on input data and generating outputs. These processes and logic flows can also be performed by dedicated logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits). For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include or be operably connected to one or more large-capacity storage devices (such as disks, magneto-optical disks, or optical disks) for storing data, to receive data from it or to transmit data to it, or both. However, a computer does not require such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and storage devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0078] Many embodiments have been described. However, it will be appreciated that various modifications may be made without departing from the spirit and scope of the present disclosure. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A method, include: receiving, at data processing hardware of the robot, sensor data associated with an environment of the robot from at least one sensor; generating an obstacle map based on the sensor data by data processing hardware, the obstacle map indicating obstacles; Determining, by the data processing hardware, a first step path for movement of the robot from the first position to the second position based on the first cadence of the robot, the first step path indicating one or more first step positions; Generating, by the data processing hardware, a second step path for movement of the robot from the first position to the second position based on the first step path and the obstacle map, the second step path indicating one or more second step positions; as well as The data processing hardware directs navigation according to the second step path.

2. The method according to claim 1, in, The second step path is generated according to a second rhythm of the robot.

3. The method according to claim 1, in, The one or more first step positions are adjusted to obtain the one or more second step positions.

4. The method according to claim 1, in, The one or more first step locations are within a threshold distance of an obstacle, and wherein the one or more second step locations are outside the threshold distance of the obstacle.

5. The method according to claim 1, in, The one or more first step locations are located within a threshold distance of a first number of obstacles of the obstacle map, and wherein the one or more second step locations are located within a threshold distance of a second number of obstacles of the obstacle map, wherein the first number of obstacles is greater than the second number of obstacles.

6. The method according to claim 1, in, The first step path is associated with a first weight distribution of the robot, and wherein the second step path is associated with a second weight distribution of the robot.

7. The method according to claim 1, in, The first step path is associated with a first measure of balance of the robot, and wherein the second step path is associated with a second measure of balance of the robot.

8. The method according to claim 1, in, The obstacles include obstacles associated with a set of stairs.

9. The method according to claim 1, in, Each of the one or more first step positions and the one or more second step positions indicates a respective position for placing a respective distal end of a respective leg of the robot.

10. The method according to claim 1, in, Generating the second step path includes: A first step position of the one or more first step positions is adjusted to obtain a second step position of the one or more second step positions.

11. The method according to claim 1, in, Generating the second step path includes: adjusting a first step position of the one or more first step positions to obtain a second step position of the one or more second step positions; and A third step location of the one or more second step locations is identified based on the second step location.

12. The method according to claim 1, in, Generating the second step path includes: adjusting a first step position of the one or more first step positions to obtain a second step position of the one or more second step positions; and A third step position of the one or more first step positions is adjusted based on the second step position to obtain a fourth step position of the one or more second step positions.

13. The method according to claim 1, further comprising: include: receiving additional sensor data associated with the environment; updating the obstacle map based on the additional sensor data to obtain an updated obstacle map; updating the second step path based on the updated obstacle map to obtain an updated second step path, the updated second step path indicating one or more third step positions; as well as Navigate according to the updated second step path instructions.

14. The method according to claim 1, further comprising: include: Simulate navigation based on the first step path.

15. The method according to claim 1, further comprising: include: A first rhythm is selected from two or more rhythms associated with the robot.

16. The method according to claim 1, further comprising: include: A portion of the sensor data is classified as corresponding to an obstacle, wherein generating the obstacle map is based on classifying the portion of the sensor data as corresponding to an obstacle.

17. The method according to claim 1, in, The robot further comprises four legs, wherein each of the four legs comprises a respective distal end, wherein navigating according to the second step path indication comprises: The instruction is to place a distal end of one of the four legs at a step position in the one or more second step positions.

18. The method according to claim 1, in, The obstacle map is a body obstacle map, wherein the obstacles are obstacles of the robot's body, and the method further comprises: A step obstacle map is generated based on the sensor data, the step obstacle map indicating obstacles at distal ends of the legs of the robot, wherein generating a second step path is based on the body obstacle map and the step obstacle map.

19. A robot, include: at least one sensor; Data processing hardware; as well as Memory hardware in communication with the data processing hardware, the memory hardware storing instructions, wherein execution of the instructions by the data processing hardware causes the data processing hardware to: receiving sensor data associated with the robot's environment from the at least one sensor; generating an obstacle map based on the sensor data, the obstacle map indicating obstacles; Determining a first step path of movement of the robot from the first position to the second position according to a first rhythm of the robot, the first step path indicating one or more first step positions; Based on the first step path and the obstacle map, generating a second step path of movement of the robot from the first position to the second position, the second step path indicating one or more second step positions; and Navigate according to the second step path instructions.

20. The robot according to claim 19, in, The second step path is generated according to a second rhythm of the robot.

21. The robot according to claim 19, in, The one or more first step positions are adjusted to obtain the one or more second step positions.

22. The robot according to claim 19, in, The one or more first step locations are within a threshold distance of an obstacle, and wherein the one or more second step locations are outside the threshold distance of the obstacle.

23. The robot according to claim 19, in, The one or more first step locations are located within a threshold distance of a first number of obstacles of the obstacle map, and wherein the one or more second step locations are located within a threshold distance of a second number of obstacles of the obstacle map, wherein the first number of obstacles is greater than the second number of obstacles.

24. The robot according to claim 19, in, The first step path is associated with a first weight distribution of the robot, and wherein the second step path is associated with a second weight distribution of the robot.

25. The robot according to claim 19, in, The first step path is associated with a first measure of balance of the robot, and wherein the second step path is associated with a second measure of balance of the robot.

26. The robot according to claim 19, in, The obstacles include obstacles associated with a set of stairs.

27. The robot according to claim 19, in, Each of the one or more first step positions and the one or more second step positions indicates a respective position for placing a respective distal end of a respective leg of the robot.

28. The robot according to claim 19, in, To generate a second step path, execution of the instructions by the data processing hardware further causes the data processing hardware to: A first step position of the one or more first step positions is adjusted to obtain a second step position of the one or more second step positions.

29. The robot according to claim 19, in, To generate a second step path, execution of the instructions by the data processing hardware further causes the data processing hardware to: adjusting a first step position of the one or more first step positions to obtain a second step position of the one or more second step positions; and A third step location of the one or more second step locations is identified based on the second step location.

30. The robot according to claim 19, in, To generate a second step path, execution of the instructions by the data processing hardware further causes the data processing hardware to: adjusting a first step position of the one or more first step positions to obtain a second step position of the one or more second step positions; as well as A third step position of the one or more first step positions is adjusted based on the second step position to obtain a fourth step position of the one or more second step positions.

31. The robot according to claim 19, in, Execution of the instructions by the data processing hardware further causes the data processing hardware to: receiving additional sensor data associated with the environment; updating the obstacle map based on the additional sensor data to obtain an updated obstacle map; updating the second step path based on the updated obstacle map to obtain an updated second step path, wherein the updated second step path indicates one or more third step positions; as well as Navigate according to the updated second step path instructions.

32. The robot according to claim 19, in, Execution of the instructions by the data processing hardware further causes the data processing hardware to: Simulate navigation based on the first step path.

33. The robot according to claim 19, in, Execution of the instructions by the data processing hardware further causes the data processing hardware to: A first rhythm is selected from two or more rhythms associated with the robot.

34. The robot according to claim 19, in, Execution of the instructions by the data processing hardware further causes the data processing hardware to: A portion of the sensor data is classified as corresponding to an obstacle, wherein generating the obstacle map is based on classifying the portion of the sensor data as corresponding to an obstacle.

35. The robot according to claim 19, in, The robot further comprises four legs, wherein each of the four legs comprises a respective distal end, wherein, to navigate according to the second step path indication, execution of the instructions by the data processing hardware further causes the data processing hardware to: The instruction is to place a distal end of one of the four legs at a step position in the one or more second step positions.

36. The robot according to claim 19, in, The obstacle map is a body obstacle map, wherein the obstacles are obstacles of a robot's body, wherein execution of the instructions by the data processing hardware further causes the data processing hardware to: A step obstacle map is generated based on the sensor data, the step obstacle map indicating obstacles at distal ends of the legs of the robot, wherein generating a second step path is based on the body obstacle map and the step obstacle map.

37. A computing system, include: Data processing hardware; as well as Memory hardware in communication with the data processing hardware, the memory hardware storing instructions, wherein execution of the instructions by the data processing hardware causes the data processing hardware to: receiving sensor data associated with an environment of the robot from at least one sensor; generating an obstacle map based on the sensor data, the obstacle map indicating obstacles; Determining a first step path of movement of the robot from the first position to the second position according to a first rhythm of the robot, the first step path indicating one or more first step positions; Based on the first step path and the obstacle map, generating a second step path of movement of the robot from the first position to the second position, the second step path indicating one or more second step positions; and Navigate according to the second step path instructions.

38. A method, include: obtaining, by data processing hardware of the robot, a body-obstacle map identifying one or more first obstacles in an environment of the robot; obtaining, by the data processing hardware, a step-obstacle map identifying one or more second obstacles in the environment; generating, by data processing hardware, a body path for movement of the robot's body from a body path start point to a body path end point using the body-obstacle map; generating, by data processing hardware, a step path defining one or more step positions of a leg of the robot using the body path; and The adjusted step path is determined by data processing hardware using the step-obstacle map.

39. The method according to claim 38, in, The generated step path includes: Based on the step-obstacle map, the gait pattern is determined according to the robot's rhythm.

40. The method according to claim 39, in, Determining gait patterns involves: A gait pattern is selected from two or more gait patterns, each gait pattern of the two or more gait patterns being associated with a corresponding cadence of the robot.

41. The method according to claim 40, in, Each of the two or more gait patterns is weighted based on a corresponding step positioning of a leg, and wherein selecting the gait pattern is based on the corresponding step positioning of the leg of each of the two or more gait patterns.

42. The method according to claim 38, in, Determining the adjusted step path is based on one or more constraints.

43. The method according to claim 42, in, The one or more constraints include: One or more hard constraints; and One or more soft constraints, wherein determining the adjusted pace path is further based on one or more costs of violating the one or more soft constraints.

44. The method according to claim 43, in: The adjusted pace path violates at least one of the one or more soft constraints; and A cost of violating at least one of the one or more soft constraints is less than a threshold cost.

45. The method according to claim 43, in: The adjusted pace path violates at least two of the one or more soft constraints; and A sum of costs of violating at least two of the one or more soft constraints is less than a threshold cost.

46. ​​The method according to claim 42, in, The one or more constraints include: Weight distribution constraints; Pace zone constraints; body area restraints; A self-collision constraint; or Residue constraints.

47. The method according to claim 38, further comprising: include: Use the adjusted stride path to determine the adjusted body path.

48. A robot, include: Body; two or more legs coupled to the body and configured to maneuver the robot around the robot's environment; Data processing hardware; as well as Memory hardware in communication with the data processing hardware, the memory hardware storing instructions, wherein execution of the instructions by the data processing hardware causes the data processing hardware to: obtaining a body-obstacle map identifying one or more first obstacles in the environment; Obtaining a pace-obstacle map identifying one or more second obstacles in the environment; generating a body path for movement of the body from a body path start point to a body path end point using the body-obstacle map; Using the body path, generating a step path defining one or more step positions of a leg of the two or more legs; and Use the pace-obstacle map to determine the adjusted pace path.

49. The robot according to claim 48, in, To generate a step path, execution of the instructions by the data processing hardware further causes the data processing hardware to: Based on the step-obstacle map, the gait pattern is determined according to the robot's rhythm.

50. The robot according to claim 49, in, To determine a gait pattern, execution of the instructions by the data processing hardware further causes the data processing hardware to: A gait pattern is selected from two or more gait patterns, each gait pattern of the two or more gait patterns being associated with a corresponding cadence of the robot.

51. The robot according to claim 50, in, Each of the two or more gait patterns is weighted based on a corresponding step position of a leg, and wherein, to select a gait pattern, execution of the instructions by the data processing hardware further causes the data processing hardware to: A gait pattern is selected based on a corresponding step position of the leg for each of the two or more gait patterns.

52. The robot according to claim 48, in, To determine the adjusted pace path, execution of the instructions by the data processing hardware further causes the data processing hardware to determine the adjusted pace path based on one or more constraints.

53. The robot according to claim 52, in, The one or more constraints include: One or more hard constraints; and One or more soft constraints, wherein execution of the instructions by the data processing hardware to determine the adjusted pace path further causes the data processing hardware to determine the adjusted pace path further based on one or more costs of violating the one or more soft constraints.

54. The robot according to claim 53, in: The adjusted pace path violates at least one of the one or more soft constraints; and A cost of violating at least one of the one or more soft constraints is less than a threshold cost.

55. The robot according to claim 53, in: The adjusted pace path violates at least two of the one or more soft constraints; and A sum of costs of violating at least two of the one or more soft constraints is less than a threshold cost.

56. The robot according to claim 52, in, The one or more constraints include: Weight distribution constraints; Pace zone constraints; body area restraints; A self-collision constraint; or Residue constraints.

57. The robot according to claim 48, in, Execution of the instructions by the data processing hardware further causes the data processing hardware to: Use the adjusted stride path to determine the adjusted body path.