Method and System for Obstacle Avoidance in Robot Path Planning Using a Depth Sensor

Abstract

This teaching relates to methods, systems, media, and implementations for robotic path planning. Depth data of obstacles acquired by one or more depth sensors deployed in a 3D robotic workspace and represented with respect to a sensor coordinate system is transformed into depth data with respect to a robotic coordinate system. The 3D robotic workspace is discretized to generate 3D grid points representing the discretized 3D robotic workspace. Based on the depth data with respect to the robotic coordinate system, binary values are assigned to at least some of the 3D grid points to generate a binary representation of the obstacles present in the 3D robotic workspace. With respect to one or more sensing points associated with a part of the robot, it is determined whether that part is going to collide with any obstacle. Based on this determination, a path is planned for the robot to move along the path while avoiding any obstacle.

Description

[0001] Cross - reference to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 986,132, filed on Mar. 6, 2020, the entire content of which is incorporated herein by reference. Background Technical Field

[0004] This disclosure relates to robotic path planning with obstacle avoidance. Specifically, this disclosure relates to path planning with obstacle avoidance using depth sensors. Background Art

[0006] Robotic path planning is finding a trajectory of a robot's motion from an initial position to a target position. For a robotic arm with multiple joints, for each move, there are as many possible moves as the number of joints. Generally, the continuous space for joint angle measurement can be discretized into a high - dimensional grid. Path planning is finding a path from the initial position to the target position in the discrete space while avoiding obstacles.

[0007] One of the challenging problems in path planning is accurately modeling obstacles. Current methods typically model obstacles by simple geometric shapes such as cubes, spheres, or cylinders. However, real - world scenarios are usually more complex. Modeling the scenarios of a robot's working environment based on simple shapes is not an easy task in itself. This is a time - consuming process, involving a large amount of manual work or high computational burden. In a dynamic environment such as an operating room, the position of the patient and the positioning of medical devices may change over time. In such an environment, it becomes very important to correctly model the environment in real - time.

[0008] Some recently developed depth sensors such as the Microsoft Azure Kinect can be used to model a robot's working environment in real - time. For example, the Microsoft Azure Kinect depth sensor can use an infrared camera to provide depth measurements for each pixel. Through appropriate calibration between the camera coordinate system and the robot coordinate system, the robot's working environment can be modeled as a set of three - dimensional (3D) points. This set of 3D points can be called a point cloud, where the number of unstructured points is one million or more. Directly utilizing such a large number of 3D points in robotic path planning can be computationally prohibitive. For example, directly checking whether the robotic arm may collide with these one million points would be very time - consuming. Therefore, there is a great need for a method that can effectively perform point - cloud - based path planning for obstacle avoidance. Summary of the Invention

[0009] The teachings disclosed herein relate to methods, systems, and programming related to robots. More specifically, the teachings relate to methods, systems, and programming related to robot path planning.

[0010] In one example, a method is implemented on at least one processor, a memory, and a communication platform capable of connecting to a network for robot path planning. Depth data of obstacles acquired by one or more depth sensors deployed in a 3D robot workspace and represented with respect to a sensor coordinate system is transformed into depth data with respect to a robot coordinate system. The 3D robot workspace is discretized to generate a set of 3D grid points representing the discretized 3D robot workspace. Based on the depth data with respect to the robot coordinate system, binarized values are assigned to at least some of the 3D grid points in the set to generate a binarized representation of the obstacles present in the 3D robot workspace. With respect to one or more sensing points associated with a part of the robot, it is determined whether that part is to collide with any obstacle in the 3D robot workspace. Based on the result of this determination, a path is planned for the robot to move along the path while avoiding any obstacles.

[0011] In different examples, a robot path planning system is disclosed, which includes a coordinate transformer, a workspace discretization unit, a workspace binarization unit, a collision determination unit, and a path planning unit. The coordinate transformer is configured to transform depth data of obstacles acquired by one or more depth sensors deployed in a 3D robot workspace and represented with respect to a sensor coordinate system into depth data with respect to a robot coordinate system. The workspace discretization unit is configured to discretize the 3D robot workspace to generate a set of 3D grid points representing the discretized 3D robot workspace. The workspace binarization unit is configured to assign binarized values to at least some of the 3D grid points in the set based on the depth data with respect to the robot coordinate system to generate a binarized representation of the obstacles present in the 3D robot workspace. The collision determination unit is configured to determine, with respect to one or more sensing points associated with a part of the robot, whether that part is to collide with any obstacle in the 3D robot workspace. The path planning unit is configured to plan a path for the robot based on the result of this determination to move along the path while avoiding any obstacles.

[0012] Other concepts relate to software for implementing the teachings. According to this concept, a software product includes at least one machine-readable non-transitory medium and information carried by the medium. The information carried by the medium can be executable program code data, parameters associated with the executable program code, and / or information related to a user, a request, content, or other additional information.

[0013] In one example, a machine-readable, non-transitory, and tangible medium has information for robot path planning recorded thereon. When this information is read by a machine, the machine is caused to perform the following steps. Depth data of obstacles acquired by one or more depth sensors deployed in a 3D robot workspace and represented with respect to a sensor coordinate system is transformed into depth data with respect to a robot coordinate system. The 3D robot workspace is discretized to generate a set of 3D grid points representing the discretized 3D robot workspace. Based on the depth data with respect to the robot coordinate system, binary values are assigned to at least some of the grid points in the set of 3D grid points to generate a binary representation of the obstacles present in the 3D robot workspace. With respect to one or more sensing points associated with a part of the robot, it is determined whether that part is to collide with any obstacles in the 3D robot workspace. Based on the result of this determination, a path is planned for the robot to move along the path while avoiding any obstacles.

[0014] Additional advantages and novel features will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the drawings, or may be learned by the production or operation of examples. The advantages of the present teachings may be realized and obtained by practice or use of various aspects of the methods, tools, and combinations set forth in the detailed examples discussed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Aspects of the present disclosure described herein are also described in terms of exemplary embodiments. These exemplary embodiments are described in detail with reference to the drawings. These embodiments are non-limiting exemplary embodiments, where like reference numerals represent similar structures throughout several views of the drawings, and where:

[0016] Figure 1 Depicts an exemplary high-level system diagram for robot path planning using depth sensors according to an embodiment of the present teachings;

[0017] Figure 2 Illustrates an exemplary flowchart for path planning through depth sensors according to an embodiment of the present teachings;

[0018] Figure 3 Illustrates an exemplary flowchart for workspace discretization, depth binarization, and robot exclusion according to an embodiment of the present teachings;

[0019] Figure 4 Shows an exemplary process for dynamic distance map calculation according to an embodiment of the present teachings;

[0020] Figures 5A - 5CShows an example of a distance map calculated at different scales according to an embodiment of the present teachings;

[0021] Figure 6 Illustrates how to detect collisions through a distance map according to an embodiment of the present teachings;

[0022] Figure 7 Depicts collision detection using sensing points without a distance map according to an embodiment of the present teachings;

[0023] Figure 8 Is a schematic diagram of an exemplary mobile device architecture that can be used to implement a dedicated system for implementing the present teachings; and

[0024] Figure 9 Depicts the architecture of a computer that can be used to implement a dedicated system for implementing the present teachings. Detailed Description of the Invention

[0025] In the following detailed description, many specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. However, it will be apparent to those skilled in the art that the present teachings may be practiced without such details. In other instances, well-known methods, procedures, components, and / or circuits have been described at a relatively high level without detail in order to avoid unnecessarily obscuring aspects of the present teachings.

[0026] The present disclosure is directed to a method and system for robotic path planning while avoiding obstacles. Specifically, the present disclosure is directed to a system and method for planning the path of a robotic arm having multiple joints while avoiding obstacles present in a scene and modeled using a depth sensor. As referred to herein, a robotic arm is the arm of a robot having multiple segments.

[0027] Figure 1 Shows an exemplary system diagram 100 for facilitating robotic path planning through a depth sensor according to an embodiment of the present teachings. System 100 includes a sensor-robot calibration unit 105, a depth data acquisition unit 106, a depth data transformation unit 108, a workspace specification unit 110, a robotic workspace discretization unit 111, a robot exclusion unit 112, a depth data binarization unit 113, an allowed obstacle distance specification unit 114, a dynamic resolution distance map generation unit 115, a robotic sensing point generation unit 118, a distance-based collision detection unit 120, and a path optimization unit 122.

[0028] One or more depth sensors 102 can be used to model the working environment of the robot 104. The depth sensors can include, but are not limited to, laser scanners, infrared time-of-flight sensors, LiDAR sensors, or vision-based stereo cameras. The sensor-robot calibration unit 105 can perform a coordinate transformation from coordinates in the sensor space to coordinates in the robot space. The depth data acquisition unit 106 can generate a point cloud including a set of 3D points representing the depth information of the robot working environment. Such 3D points represented as 3D coordinates in the sensor space can be transformed by the depth data transformation unit 108 into 3D coordinates in the robot coordinate system. The robot working space specifying unit 110 can specify (e.g., specify in terms of size and span) a certain range of the robot working space with respect to the robot coordinate system. The specified 3D working space can be discretized by the robot working space discretization unit 111. With respect to the discretized 3D working space, the transformed 3D coordinates (depth data represented in the robot coordinate system) in the working space can be binarized by the depth data binarization unit 113. The robot exclusion unit 112 can exclude the robot arm from any discretized position where the discretized depth data exists. The obstacle distance specifying unit can specify for the robot the minimum allowable distance from an obstacle. The dynamic resolution distance map generation unit 115 can generate for each point in the working space the closest distance to an obstacle. This results in a distance map 116. The robot sensing point generation unit 118 can generate a set of points at which to perform an inspection to see if the robot collides with any obstacles. The distance-based collision detection unit 120 can be set to perform a collision check with respect to obstacles. The result of such a collision check can be used by the path optimization unit 122, which can identify the optimal path from the starting position to the target position of the robot arm while avoiding collisions with any obstacles present in the working environment.

[0029] Figure 2FIG. illustrates an exemplary flowchart of robot path planning for obstacle avoidance by using a depth sensor according to an embodiment of the present teachings. At step 202, a rigid transformation may be performed that transforms the coordinates of depth data representing the robot's working environment in sensor space into coordinates in robot space. Such a transformation may employ any existing calibration method known in the art. At step 204, one or more depth sensors may be used to acquire depth data of the robot's working environment. The depth data from such depth sensors may be combined to provide 360-degree coverage of the robot's working environment. This may be necessary because each depth sensor may have a limited field of view of the depth sensor. All objects present in the robot's working environment (such as tables, patients on an operating table, nearby medical equipment) may be considered obstacles. The goal is to prevent the robot arm from colliding with any obstacles while the robot arm is moving. At step 206, the extent of the robot's workspace may be defined, for example, using the bounding box of the workspace to specify the extent based on the minimum and maximum values of the X-Y-Z coordinates in the robot coordinate system. This extent may define the area in which the robot may need to avoid colliding with obstacles. At step 208, the depth data represented as 3D coordinates in sensor space from the depth sensors is transformed into 3D coordinates in the robot coordinate system. In some embodiments, data outside the workspace may not be used for collision avoidance or may be discarded, for example, to save computational costs. At step 210, the continuous robot workspace may be discretized to generate a set of 3D grids. At step 211, the transformed depth data or 3D coordinates in the robot workspace may be used to generate a binary value of 0 or 1 for each grid in the 3D grid. Any 3D grid into which the 3D transformed coordinates fall is assigned a value of 1. Any 3D grid into which no 3D transformed coordinates fall is assigned a value of 0. In this way, any 3D grid with a value of 1 is occupied by an obstacle. Any 3D grid with a value of zero represents open space. At step 212, the 3D grid occupied by the ego robot may be assigned a uniquely defined value or may be excluded from being assigned a value of 0. At step 214, the minimum allowable obstacle distance may be specified, which defines the minimum distance between the robot and any obstacle. Such a minimum allowable obstacle distance may be defined based on application requirements (such as for safety reasons).

[0030] At step 216, a distance map can be generated based on the discretized space with the binarized grid and the minimum allowable obstacle distance. The distance map can define the distance from each 0-value grid to the nearest 1-value grid (which corresponds to a point on the obstacle surface). To increase speed for real-time obstacle avoidance, the distance map can be generated at a dynamic resolution for enhanced computational speed. At step 218, a set of sensing points can be generated for the robotic arm. These sensing points can be used to sense the distance of the robotic arm to the obstacle. These sensing points can correspond to points along the robotic joint axes; for example, they can be on the surface of the joints; they can be outside the joint surfaces. At step 220, such sensing points can be used to detect whether there is a collision with an obstacle at any robotic position. At step 222, the collision detection module at 220 can be used to identify the best path between the starting robotic position and the target robotic position. Any known or future path planning method can be applied for this purpose at step 222 based on the distance map and the minimum allowable obstacle distance.

[0031] Figure 3Illustrated is an exemplary workflow of steps 210, 211, and 212 according to one embodiment of the present teachings. At step 301, a number of resolutions may be specified for the discretization at step 210. The number of resolutions may be used to divide a specified span of the workspace to generate a 3D grid at step 302. For example, if the X-axis span of the workspace is 1000 mm and the resolution is 10, then the X-axis span may be discretized into 100 3D grids. The number of resolutions is determined based on the accuracy requirements of an application where collision detection may be based. The robotic workspace may be specified manually. It may also be determined automatically. For example, a depth camera may be used to detect a region of interest, such as a patient on an operating table, via, for example, image processing methods. The detected region of interest may be used to determine the boundaries of the robotic workspace. Alternatively, one or more magnetic tracking sensors may be placed on the patient's body. The positions of these sensors may be used to determine the boundaries of the workspace. With the discretization along each axis, the workspace may be discretized into a set of 3D grids. At step 304, it is determined whether a value of 0 or 1 is to be assigned to each 3D grid in the workspace. As discussed herein, this decision may be made based on the transformed 3D coordinates in the robotic coordinate system. First, the transformed 3D data points (or coordinates) may be used to find the nearest grid. The depth data in the camera coordinate system is indexed according to the rows and columns of the image plane, while the transformed data points in the robotic coordinate system may be indexed according to the grid indices in the x-y plane of the workspace. Assume that (i, j, k) is a grid point representing a point on the surface of an obstacle in the robotic coordinate system, where i and j represent the grid point indices along the x and y axes, and k represents the index along the z axis. The robotic coordinate system may be chosen such that the x-y plane is parallel to the floor. Then the z value may be referred to as the "height", and the k index may be referred to as the "height index". Then, all grid points having the same x and y indices but a height index less than or equal to k may be assigned to 1. All other grid points may be assigned to the value 0. That is, all 1-value grid points are on or inside the obstacle, while 0-value grid points are in the air. At step 306, the robotic arm may be projected onto the x-y plane of the grid (also referred to as the base plane). At step 308, a value of 0 is assigned to all grid points whose projection on the base plane overlaps with the robotic projection. In this way, the robot itself may be excluded from the depth data obtained from the depth sensor. That is, the robot itself may not be considered an obstacle.

[0032] Figure 4 Illustrated is an exemplary process for dynamic distance map calculation for step 216. At step 402, the grid points in the workspace may be subsampled at multiple scales in all directions. For example, for scale 2, one out of every two grid points may be sampled on all 3 axes. Figures 5A - 5CIllustrates an example of subsampling for a two-dimensional grid. Figure 5A The original grid is shown; Figure 5B is Figure 5A a subsample at a scale of 2 of Figure 5C is Figure 5A a subsample at a scale of 4 of . By subsampling, the number of grid points can be reduced according to each scale.

[0033] At step 404, a maximum-to-compute distance can be defined for each scale based on the minimum allowable obstacle distance specified at step 214. This is the maximum distance calculated from the obstacles in the distance map for each scale. A smaller maximum-to-compute distance can be assigned for smaller scales. In this way, the computational burden can be greatly reduced. This maximum-to-compute distance can be defined in such a way that for smaller scales, it is taken as a smaller percentage of the minimum allowable obstacle distance. For example, in the case of 3 scales, for scale 1, it can be taken as, for example, 20% of the minimum allowable obstacle distance; for scale 2, it can be taken as, for example, 60% of the minimum allowable obstacle distance; for scale 3, it can be taken as, for example, 120% of the minimum allowable obstacle distance. This is illustrated in Figures 5A - 5C In Figure 5A 502 is an obstacle (grid point with value 1), and 504 is the surface of the obstacle. The minimum allowable obstacle distance is indicated by 503, which is an equal distance line from the obstacle surface 504. 506 indicates the equal distance line of the maximum-to-compute distance for scale 1. Only the grid points between 504 and 506 may require distance calculation, which is a number much smaller than the number of grid points between 503 and 504. Figure 5B Shows the grid at scale 2, where 510 indicates the maximum-to-compute distance at scale 2, and 508 is the obstacle surface at scale 2. Only the points between 510 and 508 may require distance calculation. Figure 5C Shows the grid at scale 3, where 514 indicates the maximum-to-compute distance at scale 3, and 512 is the obstacle surface at scale 3. Only the points between 514 and 512 may require distance calculation.

[0034] At step 408, distance maps at all scales can be merged. This merging can be based on the following principle. At any particular grid point at the finest scale, if the distance value at the coarsest scale is within the maximum distance to be calculated from the obstacle surface, then that distance value is selected. Otherwise, the distance value can be selected as the distance value of the nearest grid point at a higher scale. In this way, the grid points that are at the farthest distance from the obstacle surface can use the distance calculated at the largest scale, while the grid points closest to the obstacle surface can use the distance calculated at the smallest scale.

[0035] Figure 6 A 2D example of step 218 for generating the robot sensing points and step 220 for collision detection is shown. In Figure 6 , 602 is the obstacle, 604 is the line of the minimum allowable obstacle distance, and 606 is the joint of the robot arm, which is in the shape of a 3D cylinder. It should be noted that this illustration is made only for one joint. For a multi-joint robot, each joint will be checked for collision in the same way as the following example. First, the center line 608 of the joint can be determined. On the center line, a set of equally spaced sensing points 610, 612,..., 614 can be defined. The distance between adjacent points can be taken to be the same as the radius of the cross-sectional circle of the cylinder. The points on the center line can be called sensing points. These sensing points can be used to determine whether the joint has fallen into the prohibited area, i.e., the area between the lines 604 and 603. First, the values of the distance map at the sensing points are read. If any of these values plus the aforementioned radius is less than the minimum allowable obstacle distance, then the joint collides with the obstacle. In Figure 6 , the sensing points 610 and 612 are within the prohibited area, while 606 is not within the prohibited area. Then, the joint as shown is determined to collide with the obstacle, and this joint position should be excluded in the path planning module.

[0036] Figure 7Another example of using a set of sensing points for collision checking. In the figure, 702 is an obstacle, 704 is the surface of the obstacle, and 706 is the joint of the robotic arm. For multiple joints, each joint will be independently checked for collisions in a similar manner. The set of sensing points is shown by lines 708 and 710. These points are distributed on lines emanating from the joint surface. These lines can be referred to as sensing lines. The length of the sensing line can be set to be the same as the minimum allowable obstacle distance. The sensing points on the sensing line may not be equally spaced. They can be arranged in such a way that they are closer to each other at positions closer to the joint; when they are farther from the joint, they are more separated (as can be seen in the figure). On each sensing line, starting from the sensing point closest to the joint, each sensing point is checked to see if it is in contact with the obstacle surface 704. If not, the next sensing point is checked. Otherwise, the check stops. If any of these sensing lines contacts the obstacle, the joint is determined to be in collision with the obstacle. The check for contact can be simply performed by evaluating the closest grid point: if the grid point has a value of 1 (i.e., on or inside the obstacle), then it is in collision with the obstacle. Otherwise, it is not in collision with the obstacle. If none of the sensing lines contact the obstacle, the joint is determined not to be in collision with the obstacle. In the figure, sensing line 708 is in contact with the obstacle, while 710 is not. Note that in this collision detection mode, it may not be necessary to calculate the distance map.

[0037] Figure 8 A schematic diagram of an exemplary mobile device architecture that can be used to implement a dedicated system for implementing the present teachings according to various embodiments. In this example, the user device on which the present teachings can be implemented corresponds to mobile device 800, including but not limited to smartphones, tablets, music players, handheld game consoles, global positioning system (GPS) receivers, and wearable computing devices (e.g., glasses, watches, etc.) or in any other form. Mobile device 800 may include one or more central processing units (“CPUs”) 840, one or more graphics processing units (“GPUs”) 830, a display 820, a memory 860, a communication platform 810 (such as a wireless communication module), a storage device 890, and one or more input / output (I / O) devices 850. Any other suitable components, including but not limited to a system bus or controller (not shown), may also be included in mobile device 800. As Figure 8As shown, a mobile operating system 870 (e.g., iOS, Android, Windows Phone, etc.) and one or more applications 880 can be loaded from a storage device 890 into a memory 860 for execution by a CPU 840. The application 880 can include a browser or any other suitable mobile application for managing a machine learning system in accordance with the teachings herein on the mobile device 800. User interaction, if any, can be implemented via an I / O device 850 and provided to various components connected via a network(s).

[0038] To implement the various modules, units, and their functions described in this disclosure, a computer hardware platform can be used as the hardware platform(s) for one or more of the elements described herein. The hardware elements, operating systems, and programming languages of such computers are conventional in nature, and it is assumed that those skilled in the art are sufficiently familiar with them to adapt those techniques to the appropriate settings described herein. A computer with user interface elements can be used to implement a personal computer (PC) or other type of workstation or terminal device, but if appropriately programmed, the computer can also act as a server. It is believed that those skilled in the art are familiar with the structure, programming, and general operation of such computer devices, and thus the drawings should be self-explanatory.

[0039] Figure 9 is a schematic diagram of an exemplary computer system architecture according to various embodiments of the teachings herein. A functional block diagram illustration of such a specialized system incorporating the teachings herein has a hardware platform including user interface elements. The computer 900 can be a general-purpose computer or a specialized computer. Both can be used to implement the specialized system of the teachings herein. The computer 900 can be used to implement any of the components(s) described herein. For example, the teachings herein can be implemented on a computer such as the computer 900 via its hardware, software program, firmware, or a combination thereof. Although only one such computer is shown, for convenience, the computer functions related to the teachings herein described herein can be implemented in a distributed manner on multiple similar platforms to distribute the processing load.

[0040] For example, computer 900 may include a communication port 950 connected to or from a network to facilitate data communication. Computer 900 also includes a central processing unit (CPU) 920 in the form of one or more processors for executing program instructions. The exemplary computer platform may also include an internal communication bus 910, different forms of program storage and data storage (e.g., disk 970, read-only memory (ROM) 930, or random access memory (RAM) 940) for various data files to be processed and / or communicated by computer 900, and program instructions that may be executed by CPU 920. Computer 900 may also include I / O components 960 that support an input / output stream between the computer and other components therein, such as user interface elements 980. Computer 900 may also receive programming and data via network communication.

[0041] Thus, aspects of the present teachings (one or more) as described above may be embodied in programming. The program aspects of the technology may be thought of as a "product" or "article of manufacture" typically in the form of executable code and / or associated data carried or embodied in a machine-readable medium. Tangible non-transitory "storage" type media include any or all memories or other storage for a computer, processor, etc., or associated modules thereof, such as various semiconductor memories, tape drives, disk drives, etc., which may provide storage for software programming at any time.

[0042] All or part of the software can sometimes be communicated via a network such as the Internet or various other telecommunications networks. For example, such communication can enable the loading of software from one computer or processor to another, e.g., from a server or host computer of a motion planning system of a robot to a computing environment or a hardware platform of other systems that implement a computing environment or similar functions related to path planning (one or more). Thus, another type of medium that can carry software elements includes light waves, radio waves, and electromagnetic waves, such as those used across physical interfaces between local devices, over wired and optical landlines, and via various air links. Physical elements that carry such waves, such as wired or wireless links, optical links, etc., can also be considered media that carry software. As used herein, unless limited to tangible "storage" media, the term computer or machine "readable medium" refers to any medium that participates in providing instructions to a processor for execution.

[0043] Thus, a machine-readable medium can take many forms, including but not limited to tangible storage media, carrier media, or physical transmission media. Non-volatile storage media includes, for example, optical or magnetic disks, such as any storage device in any computer(s), which can be used to implement the system or any of its components as shown. Volatile storage media includes dynamic memory, such as the main memory of such a computer platform. Tangible transmission media includes coaxial cables; copper wire and fiber optics, including the wires that form a bus within a computer system. Carrier transmission media can take the form of electrical or electromagnetic signals, or acoustic or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Thus, common forms of computer-readable media include, for example: floppy disks, flexible disks, hard disks, magnetic tape, any other magnetic media, CD-ROM, DVD or DVD-ROM, any other optical media, punched cards, paper tape, any other physical storage media with patterns of holes, RAM, PROM, and EPROM, FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, a cable or link transporting such a carrier wave, or any other medium from which a computer can read to turn into code and / or data. Many of these forms of computer-readable media may involve carrying one or more sequences of one or more instructions to a physical processor for execution.

[0044] Those skilled in the art will recognize that this teaching is subject to various modifications and / or enhancements. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a software-only solution - for example, installed on an existing server. Additionally, as disclosed herein, the motion planning system of a robot can be implemented as firmware, a firmware / software combination, a firmware / hardware combination, or a hardware / firmware / software combination.

[0045] While what has been described above is considered to constitute this teaching and / or other examples, it should be understood that various modifications can be made thereto and the subject matter disclosed herein can be implemented in various forms and examples, and the teaching can be applied to many applications, only some of which have been described herein. The following claims are intended to claim any and all applications, modifications, and variations that fall within the true scope of this teaching.

Claims

1. A method implemented on at least one processor, a memory, and a communication platform capable of connecting to a network for robot path planning, the method comprises: transforming depth data of obstacles acquired by one or more depth sensors deployed in a 3D robot workspace and represented with respect to a sensor coordinate system into depth data with respect to a robot coordinate system; discretizing the 3D robot workspace to generate a set of 3D grid points representing the discretized 3D robot workspace; assigning binary values to at least some of the set of 3D grid points based on the depth data with respect to the robot coordinate system to generate a binary representation of the obstacles present in the 3D robot workspace; determining, with respect to one or more equally spaced points located on a part of the robot, whether the part is to collide with any of the obstacles in the 3D robot workspace, wherein the determination further includes generating a distance map by subsampling the set of 3D grid points at multiple scales; at each of the multiple scales: assigning a corresponding maximum distance to be calculated from the obstacles, and generating a distance submap for that scale based on the corresponding maximum distance to be calculated from the obstacles; and merging the distance submaps for the multiple scales; and planning a path for the robot based on the result of the determination to move along the path while avoiding any of the obstacles.

2. The method according to claim 1, wherein, the step of assigning binary values includes: for each 3D grid point in the set of 3D grid points, when it is determined that at least some of the depth data in the depth data with respect to the robot coordinate system matches the 3D grid point, assigning a first value indicating the presence of an obstacle to the 3D grid point; when it is determined that no depth data with respect to the robot coordinate system matches the 3D grid point, assigning a second value indicating the absence of an obstacle to the 3D grid point; and when the projection of the 3D grid point on the base plane of the robot coordinate system overlaps with the projection of the robot on the base plane, assigning the second value to the 3D grid point.

3. The method according to claim 2, wherein, the one or more points are along the center line of the part, and the determination step further includes: for each of the one or more points, reading a distance value associated with the point from the distance map, obtaining the sum of the distance value and the radius of the part, and evaluating whether the part is to collide with any of the obstacles by comparing the sum with an allowed obstacle distance.

4. The method according to claim 1, wherein, the corresponding maximum distance to be calculated is determined based on an allowed obstacle distance and an adjustment factor, where the adjustment factor is determined based on the levels of the multiple scales.

5. The method according to claim 1, further comprises: Calculate a sensor-robot transformation between the sensor coordinate system and the robot coordinate system via calibration, wherein the steps of the transformation are performed based on the sensor-robot transformation.

6. A machine-readable and non-transitory medium having information recorded thereon, wherein, when read by a machine, the information causes the machine to perform the following operations: Transform depth data of an obstacle acquired by one or more depth sensors deployed in a 3D robot workspace and represented with respect to a sensor coordinate system into depth data with respect to a robot coordinate system; Discretize the 3D robot workspace to generate a set of 3D grid points representing the discretized 3D robot workspace; Based on the depth data with respect to the robot coordinate system, assign binary values to at least some of the 3D grid points in the set of 3D grid points to generate a binary representation of the obstacle present in the 3D robot workspace; With respect to one or more equally spaced points located on a part of the robot, determine whether the part is to collide with any of the obstacles in the 3D robot workspace, wherein the determination further includes generating a distance map by: Subsampling the set of 3D grid points at multiple scales; At each of the multiple scales: Assign a corresponding maximum distance to be calculated from the obstacle, and Based on the corresponding maximum distance to be calculated from the obstacle, generate a distance sub-map for that scale; And Merge the distance sub-maps for the multiple scales; And Based on the result of the determination, plan a path for the robot to move along the path while avoiding any of the obstacles.

7. The medium according to claim 6, wherein, The step of assigning binary values includes: For each 3D grid point in the set of 3D grid points, When it is determined that at least some of the depth data in the depth data with respect to the robot coordinate system matches the 3D grid point, assign a first value indicating the presence of an obstacle to the 3D grid point; When it is determined that no depth data with respect to the robot coordinate system matches the 3D grid point, assign a second value indicating the absence of an obstacle to the 3D grid point; and When the projection of the 3D grid point on the base plane of the robot coordinate system overlaps with the projection of the robot on the base plane, assign the second value to the 3D grid point.

8. The medium according to claim 7, wherein, The one or more points are along the center line of the part, and the determination step further includes: For each of the one or more points, Read the distance value associated with the point from the distance map, Obtain the sum of the distance value and the radius of the part, and Evaluate whether the part is to collide with any of the obstacles by comparing the sum with an allowed obstacle distance.

9. The medium according to claim 7, wherein, The corresponding maximum distance to be calculated is determined based on an allowed obstacle distance and an adjustment factor, where The adjustment factor is determined based on the levels of the multiple scales.

10. The medium as claimed in claim 6, further comprising: calculating a sensor-robot transformation between the sensor coordinate system and the robot coordinate system via calibration, wherein the transformation steps are performed based on the sensor-robot transformation.

11. A system for a robot path planning system, comprising: a coordinate transformer configured to transform depth data of an obstacle acquired by one or more depth sensors deployed in a 3D robot workspace and represented with respect to a sensor coordinate system into depth data with respect to a robot coordinate system; a workspace discretization unit configured to discretize the 3D robot workspace to generate a set of 3D grid points representing the discretized 3D robot workspace; a workspace binarization unit configured to assign a binary value to at least some of the set of 3D grid points based on the depth data with respect to the robot coordinate system to generate a binary representation of the obstacle present in the 3D robot workspace; a collision determination unit configured to determine, with respect to one or more equally spaced points located on a part of the robot, whether the part is to collide with any of the obstacles in the 3D robot workspace, the collision determination unit generating a distance map by: subsampling the set of 3D grid points at multiple scales; at each of the multiple scales: assigning a corresponding maximum distance to be calculated from the obstacle, and generating a distance submap for that scale based on the corresponding maximum distance to be calculated from the obstacle; and merging the distance submaps for the multiple scales; and a path planning unit configured to plan a path for the robot to move along the path while avoiding any of the obstacles based on the determined result.

12. The system as claimed in claim 11, wherein, the workspace binarization unit performs the following steps for each 3D grid point in the set of 3D grid points: when it is determined that at least some of the depth data in the depth data with respect to the robot coordinate system matches the 3D grid point, assigning a first value indicating the presence of an obstacle to the 3D grid point; when it is determined that no depth data with respect to the robot coordinate system matches the 3D grid point, assigning a second value indicating the absence of an obstacle to the 3D grid point; and when the projection of the 3D grid point on the base plane of the robot coordinate system overlaps with the projection of the robot on the base plane, assigning the second value to the 3D grid point.

13. The system as claimed in claim 12, wherein, the one or more points are along the center line of the part, and the collision determination unit further performs: for each of the one or more points, reading a distance value associated with the point from the distance map, obtaining the sum of the distance value and the radius of the part, and evaluating whether the part is to collide with any of the obstacles by comparing the sum with an allowed obstacle distance.

14. The system as claimed in claim 12, Among them, the corresponding maximum distance to be calculated is determined based on the allowable obstacle distance and an adjustment factor, where the adjustment factor is determined based on the levels of the plurality of ratios.

Images