Decentralized traffic-aware navigation planning for mobile robots

Abstract

The method includes storing an occupancy map for a facility, the occupancy map defining the current occupancy for each of several regions within the facility for the current time value, and defining the future occupancy for each of at least a subset of the regions for later time values; generating a path from the current orientation of a mobile robot to a target orientation within the facility based on the current and future occupancy; capturing sensor data representing the vicinity of the mobile robot in response to the mobile robot executing the path; detecting obstacles from the sensor data; and transmitting occupancy data to generate an updated occupancy map, the occupancy data including updated future occupancy for at least one of the subsets of the regions, the occupancy data including obstacle data indicating the location of obstacles, and path data defining the path.

Description

Technical Field

[0001] In facilities such as warehouses, manufacturing facilities, medical facilities, etc., for example, autonomous or semi-autonomous mobile robots may be deployed to transfer articles within the associated facility. To navigate the facility, the mobile robot captures sensor data (such as images, etc.) and detects obstacles within the sensor data. The mobile robot may then generate a path, for example, towards a target location, taking into account any detected obstacles. However, in a facility housing multiple such robots, the paths generated by the individual robots may collide with each other.

Summary of the Invention

[0002] The accompanying drawings, in which like reference numerals refer to the same or functionally similar elements throughout the different views, are incorporated herein and form a part of this specification, along with the following detailed description, and serve to further illustrate embodiments of the concepts including the invention claimed and to explain various principles and advantages of those embodiments.

Brief Description of the Drawings

[0003] [Figure 1] It is a view of an article handling mobile robot deployed in a facility.

[0004] [Figure 2] It is a view of a specific component of the mobile robot of FIG. 1.

[0005] [Figure 3] It is a flowchart showing a method of traffic situation consideration type navigation plan.

[0006] [Figure 4] It is a view showing an exemplary occupancy map used in the method of FIG. 3.

[0007] [Figure 5]This figure shows an example of executing block 310 of the method shown in Figure 3.

[0008] [Figure 6] This figure shows examples of the execution of blocks 315, 320, 325, and 330 of the method shown in Figure 3.

[0009] [Figure 7] This figure shows an example of how blocks 305, 310, and 315 are executed by another mobile robot.

[0010] [Figure 8] This figure shows an example of executing block 335 of the method shown in Figure 3.

[0011] [Figure 9] This figure shows an example of executing blocks 340 and 345 of the method shown in Figure 3.

[0012] [Figure 10] This figure shows another example of executing blocks 335 and 340 of the method shown in Figure 3. [Modes for carrying out the invention]

[0013] Those skilled in the art will understand that elements in drawings are shown for the sake of brevity and clarity and are not necessarily drawn to an accurate scale. For example, the dimensions of some elements in drawings may be exaggerated relative to others to help improve the understanding of embodiments of the invention.

[0014] Components of the apparatus and methods are represented in the drawings by conventional symbols where appropriate, and only specific details relevant to understanding embodiments of the invention are shown so as not to obscure this disclosure with details that would be readily apparent to those skilled in the art who benefit from the description herein.

[0015] Examples disclosed herein relate to a method for storing an occupancy map for a facility, the occupancy map comprising: storing (i) defining a current occupancy for each of several regions within the facility corresponding to a current time value; and (ii) defining a future occupancy for each of at least a subset of the regions corresponding to a time value after the current time value; generating a path from the current pose of a mobile robot in the facility to a target pose of the mobile robot based on the current and future occupancies; capturing sensor data representing the vicinity of the mobile robot in response to the mobile robot executing the path; detecting obstacles from the sensor data; and transmitting occupancy data from the mobile robot to generate an updated occupancy map, the occupancy data comprising (i) obstacle data indicating the location of obstacles; and (ii) path data defining the path.

[0016] Additional examples of disclosures herein include a memory for storing an occupancy map for a facility, the occupancy map defining (i) a current occupancy for each of several regions within the facility corresponding to a current time value, and (ii) a future occupancy for each of at least a subset of the regions corresponding to a time value after the current time value; and a processor configured to generate a path from the current orientation to a target orientation of the mobile robot in the facility based on the current and future occupancies, capture sensor data representing the vicinity of the mobile robot in response to the execution of the path, detect obstacles from the sensor data, and transmit occupancy data for generating an updated occupancy map including updated future occupancies for at least one of the subsets of the regions, the occupancy data including (i) obstacle data indicating the location of obstacles, and (ii) path data defining the path.

[0017] Further examples of the disclosure herein relate to a method for generating a path from the current orientation of a mobile robot in a facility to a target location based on an occupancy map of the facility, wherein the occupancy map defines the current occupancy for each of several areas within the facility; detecting obstacles from sensor data captured via the mobile robot's sensors during the execution of the path; storing the detected locations of the obstacles; selecting detected obstacles having detected locations outside the sensor's field of view in response to determining that the path is blocked; generating an observation path so that the observation locations of the selected obstacles are within the sensor's field of view; capturing further sensor data during the execution of the observation path and determining from the further sensor data whether the detected locations remain obstructed; and modifying the execution of the path based on whether the detected locations remain obstructed.

[0018] Further examples disclosed herein include a mobile robot comprising a drive assembly, sensors, and a processor configured to generate a path from the current orientation of a mobile robot to a target position in a facility based on an occupancy map of the facility, wherein the occupancy map defines the current occupancy for each of several areas within the facility; generate; control the drive assembly to execute the path; detect obstacles from sensor data captured via the sensors while the path is being executed; store the detected locations of obstacles; select detected obstacles having detected locations outside the sensor's field of view in response to determining that the path is blocked; generate an observation path so that the observed locations of the selected obstacles are within the sensor's field of view; capture further sensor data while the observation path is being executed and determine from the further sensor data whether the detected locations remain obstructed; and modify the path execution based on whether the detected locations remain obstructed.

[0019] FIG. 1 shows the interior of a facility 100 such as a warehouse, manufacturing facility, medical facility, etc. The facility 100 includes a plurality of support structures 104 that hold articles 108. In the illustrated example, the support structures 104 include shelf modules arranged as a set that forms, for example, aisles 112-1 and 112-2 (collectively referred to as aisle 112 and generically referred to as aisles 112. Similar nomenclature is used herein for other components). As shown in FIG. 1, the support structure 104 in the form of a shelf module includes a support surface 116 that supports the article 108. In other examples, the support structure 104 may include pegboards, containers, etc.

[0020] In other examples, the facility 100 may include fewer or more aisles 112 than shown in FIG. 1. In the illustrated example, the aisle 112 is formed by a set of eight support structures 104 (four on each side of each aisle 112). However, the facility 100 may have a variety of other aisle layouts. As is apparent, each aisle 112 is an open-ended space bounded on both sides by support structures 104. The aisle 112 may be for the movement of people, vehicles, etc. In yet further examples, the facility 100 may not include an aisle 112 and instead may include an assembly line, etc.

[0021] The article 108 may be processed according to various processes depending on the nature of the facility 100. In some examples, the facility 100 is a shipping facility, a distribution facility, etc., and the article 108 may be placed on the support structure 104 for storage and later retrieved for shipment from the facility. The placement of the article 108 on and / or retrieval from the support structure may be performed or assisted by mobile robots 120-1, 120-2 deployed in the facility 100. Based on, for example, the size and / or layout of the facility 100, more robots 120 than the robots 120-1 and 120-2 shown in FIG. 1 may be deployed in the facility 100. The components of the robot 120 will be discussed in more detail below. Generally, each robot 120 in the facility 100 is configured to transfer the article 108 within the facility 100.

[0022] Each robot 120 may be configured to track its posture (e.g., position and orientation) within the facility 100 according to, for example, a coordinate system 124 pre-established in the facility 100. The robot 120 may autonomously navigate within the facility 100, for example, moving to a position assigned to the robot 120 to receive and / or deposit the article 108. The article 108 may be deposited within or on the robot 120 and removed from the robot 120 by a human operator and / or a mechanical device such as a robot arm deployed in the facility 100. The positions where each robot 120 navigates may be assigned to the robot 120 by the central server 128. That is, the server 128 is configured to assign tasks to the robot 120. Each task may include either or both of one or more positions to move to and one or more operations to perform at those positions. For example, the server 128 may assign a task to a given robot 120 to move to a specific position defined in the coordinate system 124 and wait to receive one or more articles 108 at that position.

[0023] Tasks may be assigned to the robot 120 by message exchange between the server 128 and the robot 120, for example, via a suitable combination of a local area network and a wide area network. The server 128 may be deployed within the facility 100 or may be deployed away from the facility 100. In some examples, the server 128 is configured to assign tasks to the robots 120 in multiple facilities and may not be physically located in any of the individual facilities.

[0024] The server 128 includes a processor 132, such as one or more central processing units (CPUs), graphics processing units (GPUs), or dedicated hardware controllers such as application-specific integrated circuits (ASICs). The processor 132 is communicatively coupled to a non-temporary computer-readable medium, such as memory 136, which may be a preferred combination of volatile and non-volatile memory elements. The processor 132 is also coupled to a communication interface 140, such as a transceiver (e.g., an Ethernet controller), which enables the server 128 to communicate with other computing devices, such as the mobile robot 120. The memory 136 may store a number of computer-readable instructions that can be executed by the processor 132, such as an application 144 that, when executed by the processor 132, configures the processor 132 to manage certain aspects of the operation of the mobile robot 120, including task allocation and provision of occupancy data, as discussed below.

[0025] To navigate to a given location within facility 100 (for example, a target location assigned to the mobile robot 120 by server 128), the mobile robot 120 may be configured to capture sensor data representing at least a portion of the physical environment around the robot 120 (i.e., the area surrounding the robot 120). The robot 120 may then be configured to detect nearby obstacles from the sensor data and navigate to avoid or away from the obstacles as necessary.

[0026] As will be apparent to those skilled in the art, facility 100 may contain a variety of obstacles. For example, as can be seen in Figure 1, obstacles that the mobile robot 120 may need to avoid and navigate around include the support structure 104, people such as the worker 148, and mobile equipment such as forklifts and other mobile robots. Obstacles that the mobile robot 120 may encounter while moving around facility 100 may also include stationary obstacles such as boxes and pallets. In other words, obstacles that the mobile robot 120 may encounter during navigation may include permanent or semi-permanent stationary obstacles such as the support structure 104, as well as temporary stationary obstacles such as boxes, and mobile obstacles such as other robots 120 and the worker 148.

[0027] The mobile robot 120 may be configured to generate a path from its current orientation to a target position based on an occupancy map that indicates whether various areas of facility 100 are currently occupied or not. Such an occupancy map does not have to simply show whether an area is occupied or not in a binary manner. For example, the occupancy map may indicate the likelihood of an area being occupied based, for example, how recently the mobile robot 120 has observed that area. Even in a system that uses an occupancy map that shows the probability of interference rather than a binary index as described above, the occupancy map may only provide information that defines the current occupancy of each area within facility 100. Such an occupancy map may not reflect the future positions of moving obstacles such as other robots 120, and therefore, two or more robots 120 may plan paths that cross substantially the same area of ​​facility 100 substantially simultaneously. This is because the area is not occupied when those paths are generated. Overlapping paths in both space and time can cause congestion and / or trigger collision avoidance mechanisms within the robot 120, thereby reducing the efficiency with which the robot 120 moves around the facility 100.

[0028] Therefore, as discussed below, the robot 120 and server 128 are configured to perform additional functions to enable traffic-situation-aware route planning in the robot 120. In other words, the functions performed by the robot 120 and server 128 in facility 100 facilitate the generation of routes in each robot 120 that take into account not only the current occupancy of various areas within facility 100 but also the future occupancy of specific areas. Thus, the resulting routes can be traffic-situation-aware in that the routes are generated to avoid congestion. Such route generation can also be at least partially decentralized in that route generation can be performed by the robot 120 itself, rather than by the server 128, which could place a heavy computational load on the server 128.

[0029] Before discussing in more detail the functionality performed by the robot 120 and server 128, we will discuss specific components of the robot 120 with reference to Figure 2. As shown in Figure 2, each robot 120 includes a chassis 200 that supports various other components of the robot 120. In particular, the chassis 200 supports a drive assembly 204, such as one or more electric motors that drive a set of wheels, tracks, etc. The drive assembly 204 may include one or more sensors, such as a wheel odometer, an inertial measurement unit (IMU), etc.

[0030] The chassis 200 also supports receptacles, shelves, etc., for supporting the items 108 during transport. For example, the robot 120 may include selectable combinations of receptacles 212. In the illustrated example, the chassis 200 supports a rack 208 which includes rails or other structural mechanisms configured to support the receptacles 212 at a variable height above the chassis 200. Thus, the receptacles 212 can be installed on and removed from the rack 208, thereby allowing the robot 120 to support different combinations of receptacles 212.

[0031] The robot 120 may include output devices such as a display 216. In the illustrated example, the display 216 is mounted above the rack 208, but it will be apparent that in other examples the display 216 may be located elsewhere on the robot 120. The display 216 may include a built-in touchscreen or other input devices. In some examples the robot 120 may include other output devices in addition to or instead of the display 216. For example the robot 120 may include one or more speakers, light-emitting elements such as a strip of light-emitting diodes (LEDs) along the rack 208, etc.

[0032] The robot 120's chassis 200 also supports a variety of other components, including a processor 220, such as a dedicated hardware controller, for example, one or more central processing units (CPUs), graphics processing units (GPUs), or application-specific integrated circuits (ASICs). The processor 220 is communicatively coupled to a non-temporary computer-readable medium, such as memory 224, such as a preferred combination of volatile and non-volatile memory elements. The processor 220 is also coupled to a communication interface 228, such as a wireless transceiver, which enables the robot 120 to communicate with other computing devices, such as a server 128 and other robots 120.

[0033] Memory 224 stores various data used for autonomous or semi-autonomous navigation, including applications 232 that can be executed by the processor 220 to perform navigation functions and other task execution functions. In some examples, the above functions may be performed by multiple separate applications stored in memory 224.

[0034] The chassis 200 may support one or more sensors 240, such as cameras and / or depth sensors (e.g., lidar, depth camera, time-of-flight camera, etc.), coupled with the processor 220. The sensors 240 are configured to capture images and / or depth data representing at least a portion of the physical environment of the robot 120. The data captured by the sensors 240 may be used by the processor 220 for navigation purposes, such as route planning and obstacle avoidance, and in some examples to update a map of the facility.

[0035] Each sensor 240 has its own field of view (FOV). For example, the first FOV 242a corresponds to a laser scanner, such as a lidar sensor, mounted on the front face of the chassis 200. The FOV 242a may be substantially two-dimensional, for example, extending forward in a substantially horizontal plane. The second FOV 242b corresponds to a camera (e.g., a depth camera, a color camera, etc.) similarly mounted on the front face of the chassis 200. As is obvious, a variety of other optical sensors, each having its own FOV 242, may be mounted on the chassis 200 and / or rack 208.

[0036] The power-consuming components of the robot 120 may be supplied with such power from a battery 244, which is implemented as one or more rechargeable batteries housed in the chassis 200 and rechargeable via a charging port (not shown) or other suitable charging interface.

[0037] Looking at Figure 3, we see Method 300 for a decentralized traffic-sensing navigation planning system. Method 300 is described below, along with an example of its implementation at Facility 100. In particular, as shown in Figure 3, certain blocks of Method 300 are executed by the mobile robot 120, for example, by the execution of application 232 by processor 220. Other blocks of Method 300 are performed by server 128, for example, by the execution of application 144 by processor 132. In some cases, which are exemplified below, functions performed by server 128 in Figure 3 may be performed by robot 120, and vice versa, in order to optimize the computational load on either or both robot 120 and server 128.

[0038] In block 305, the mobile robot 120 is configured to obtain an occupancy map for facility 100. The occupancy map may be obtained, for example, from server 128, which stores and updates a central copy of the occupancy map for distribution to robot 120. Updating and providing the occupancy map to robot 120 is an iterative process, as can be understood from the discussion below. The occupancy map may be obtained in block 305, for example, via a request from robot 120 to server 128, or by pushing the occupancy map from server 128 to robot 120 (for example, in response to an update of the occupancy map performed by server 128). The occupancy map may initially be obtained entirely from server 128. If robot 120 has a local copy of the occupancy map (for example, previously received from server 128), the occupancy map may be obtained in block 305 in the form of a record of changes from a previous occupancy map, in order to reduce the amount of data sent from server 128 to robot 120.

[0039] The occupancy map may be a simplified representation of facility 100, for example, by dividing facility 100 into multiple regions and defining the current occupancy for each such region. The current occupancy indicates whether the region is currently occupied (e.g., at the time the current occupancy was last updated) by a permanent obstacle, such as a support structure 104, or by a moving obstacle, such as another robot 120. The occupancy map may also define future occupancy for specific regions of facility 100, as discussed below. The future occupancy, in contrast to the current occupancy, indicates whether the corresponding region will be occupied at a specific future point in time. In other words, some regions of the occupancy map may contain two or more occupancies, for example, one current occupancy and one or more future occupancies.

[0040] Looking at Figure 4, two exemplary occupancy map structures are shown. In particular, the first occupancy map 400 includes multiple cells 404 arranged, for example, in a grid, representing the area of ​​facility 100. That is, facility 100 may be divided into cells 404 with any suitable resolution (various other cell sizes may be adopted to balance accuracy and computational load, e.g., 5cm × 5cm). The cells 404 shown in Figure 4 are only a portion of the complete occupancy map corresponding to a portion of passage 112-2 (shown by a dashed line). The complete occupancy map may cover the entire facility 100.

[0041] Each cell 404 contains a current occupancy value, which in the illustrated example is a cost value between 0 and 100. A value of 0 may indicate an empty space to which the robot 120 can move, while a value of 100 may indicate an occupied space to which the robot 120 cannot move. A value between 0 and 100 may be used to indicate, for example, the possibility that a cell is occupied (for example, the current occupancy value does not have to be binary). A value closer to 100 indicates an increased confidence that the corresponding cell 404 is occupied. A specific occupancy value may be determined from a map of facility 100 showing the locations of permanent or semi-permanent obstacles, such as support structures 104. For example, a cell 404 corresponding to a support structure 104 may always be assigned a current occupancy value of 100. Other cells 404 may be updated over time in response to obstacle detection by the robot 120.

[0042] Figure 4 also shows another structure of the occupancy map, which is in the form of a lattice 408 containing multiple nodes 412 joined by edges 416. Each node 412 represents an area within facility 100, and each edge 416 represents a path segment between nodes 412 to which the robot 120 can move. The edges are associated with a current occupancy value using a scale between 0 and 100, as described above, for example with respect to cell 404. For example, in the sample portion of the lattice 408 shown in Figure 4, the seven fully visible edges 416 correspond to the empty portion of passage 112-2 and therefore have a current occupancy value of 0. The lattice 408 can reduce memory and / or computational requirements by excluding nodes 412 from areas of facility 100 that contain permanent or semi-permanent structures such as support structures 104. As discussed below, at least some of the edges 416 of the lattice 408 may contain future occupancy values. In the example shown in Figure 4, neither cell 404 nor edge 416 contains a future occupancy value.

[0043] Returning to Figure 3, in block 310, the robot 120 is configured to obtain a target position in facility 100 and generate a path from the robot 120's current orientation to the target position. The target position may be obtained from server 128, for example, as described above. In some examples, the target position may be generated locally by the robot 120. Turning to Figure 5, an example of block 310 in operation is shown.

[0044] As shown in Figure 5, robot 120-1 receives a command from server 128 to move from its current orientation (e.g., between passages 112-1 and 112-2) to a target position 500. In response to receiving the target position 500, robot 120-1 is configured to generate a path 504 from its current orientation to the target position 500, optimizing the path 504 for the minimum travel distance, for example. Path 504 is generated based on the occupancy map received in block 305. More generally, any path generation performed by robot 120-1 is based on the most recently received version of the occupancy map. In the illustrated example, the occupancy map does not include any future occupancy values ​​and further indicates that passage 112-2 is empty. Therefore, path 504 extends from robot 120-1's current orientation to the entrance of passage 112-2 and along passage 112-2 to the target position 500.

[0045] Referring again to Figure 3, in block 315, robot 120-1 is configured to execute the path generated in block 310. Executing path 504 involves controlling the drive assembly 204 to move along path 504 while tracking the current orientation of the mobile robot 120-1, and capturing sensor data representing the robot 120-1's surroundings within FOV 242 using sensor 240. In other words, while moving along path 504, processor 220 is configured to periodically update the robot 120-1's currently tracked orientation in coordinate system 124 (higher and lower frequencies may be used, for example, at a frequency of approximately 30 Hz). Processor 220 is also configured to periodically control sensor 240 to capture sensor data (higher and lower frequencies may be used, for example, at a frequency of approximately 30 Hz) and to detect obstacles from the sensor data.

[0046] The processor 220 is further configured to store the location of any obstacle detected from sensor data in block 315 by detecting faces and / or edges in, for example, point cloud data. For example, the processor 220 may store one or more sets of coordinates in coordinate system 124 that indicate the location of the detected obstacle. The stored location of the obstacle detected from sensor data may be referred to as the observation location.

[0047] In block 325, the processor 220 may be configured to transmit occupancy data, for example, to the server 128 (or, in some examples, directly to another mobile robot 120). The occupancy data includes obstacle data indicating the observed positions of any obstacles detected during the execution of the path 504, such as the coordinates described above. The occupancy data may optionally include path data defining the path 504 itself, such as a set of attitudes defined in a coordinate system, accompanied by velocity data (e.g., the expected travel speed of robot 120-1 in each attitude along the path 504).

[0048] Blocks 310, 315, and 320 are shown as being performed sequentially, but the execution of route 504, obstacle detection, and provision of route and obstacle data to server 128 may be performed iteratively and in combinations different from those shown in Figure 3. For example, processor 220 may be configured to transmit the route data described above in response to route generation in block 310, even before the start of route execution. Robot 120-1 may be configured to transmit obstacle data in substantially real time in response to detecting each obstacle in block 315 during the execution of route 504. In another example, server 128 may request route and / or obstacle data from robot 120, and therefore the transmission of occupancy data may be performed in block 320 in response to such a request.

[0049] In block 325, server 128 is configured to receive occupancy data from robot 120-1, and in block 330, server 128 is configured to generate an updated occupancy map according to the occupancy data. For example, server 128 may be configured to update any edges or cells corresponding to obstacles observed by robot 120-1, for example, by increasing the current cost values ​​associated with those edges or cells. Server 128 may be configured to set or update future cost values ​​associated with areas traversed by path 504. For example, server 128 may determine the expected position of mobile robot 120-1 in facility 100 for each of a set of future time intervals, according to attitude and velocity data provided by mobile robot 120-1. For each of future time intervals, server 128 may set a corresponding future cost value in association with the area containing the expected position of mobile robot 120-1 for that time interval.

[0050] Looking at Figure 6, examples of the execution of blocks 315, 320, 325, and 330 are shown. For example, robot 120-1, having entered passage 112-2, detects obstacles 600 such as boxes and pallets in passage 112-2 from sensor data captured by sensor 240. Robot 120-1 may be configured to send data indicating the observed location of the obstacles 600, along with data defining the path 504, to server 128. Server 128 is then configured to update the occupancy map based on the occupancy data from robot 120-1. A portion 604 of the initial occupancy map with a cost of 0 (indicating empty space) is shown in Figure 6. An updated portion 608 of the occupancy map corresponding to the same portion of passage 112-2, shown as a dashed line in Figure 6, is also shown following the execution of block 330.

[0051] In the updated portion 608 of the occupancy map, a particular edge 416 includes cost data 612 that defines both a current cost value and at least one future cost value. For example, cost data 612-1 includes a current cost of 0 (e.g., on a scale of 0 indicating definitely available space to 10 indicating definitely occupied space) and three future cost values. The future cost values ​​may indicate the likelihood that the corresponding space will be occupied at a given future point in time, e.g., 5 seconds, 10 seconds, and 15 seconds into the future. In other examples, these time intervals do not have to be equal. For example, the future cost values ​​may correspond to future points in time at increasing intervals from the current point in time (e.g., 2 seconds, 5 seconds, 9 seconds, 15 seconds, etc.).

[0052] Cost data 612-1 shows the increased cost associated with the base edge 416 at 15 seconds into the future in the illustrated example. Two further exemplary cost data sets 612-2 and 612-3 show the increased costs associated with their respective edges 416 at 20 seconds into the future. As is clear, the increased costs shown at the future time points mentioned above correspond to the time points when the robot 120 is expected to move across the base edge 416. Also, as seen in Figure 6, both the current and future cost values ​​may be selected by the server 128 based on either or both the confidence level associated with obstacle detection and the length of time between the current time point and the future time point corresponding to the future cost.

[0053] For example, the current cost associated with obstacle 600 (having a value of 9) may be selected based, for example, on the positioning confidence of robot 120-1 at the time obstacle 600 is detected (for example, the higher the positioning confidence, the greater the cost value associated with the detected obstacle). Furthermore, the future costs obtained from path 504 may decrease as the period becomes even further into the future, so that the future costs in cost data 612-2 and 612-3 (having a value of 7) are lower than the future costs in cost data 612-1 (having a value of 8).

[0054] As described above, the updated occupancy map may be sent to the mobile robot 120-1 in response to a request from robot 120, or it may be pushed to robot 120 by server 128. In a separate instance of method 300 in robot 120-2, for example, robot 120-2 may receive an updated map (including the updated portion 608 shown in Figure 6) obtained from path and obstacle data provided by robot 120-1.

[0055] In block 310, robot 120-2 may be configured to receive a target position, for example, target position 700 as shown in Figure 7, from server 128. As is clear from Figure 7, the shortest path between robot 120-2's current orientation and target position 700 is through passage 112-2. However, from the updated occupancy map, robot 120-2 may determine that movement through passage 112-2 will cause robot 120-2 to be positioned near obstacle 600 at approximately the same time that robot 120-1 is near obstacle 600. In other words, if robot 120-2 plans a path through passage 112-2, robots 120-1 and 120-2 may interfere with each other. Therefore, robot 120-2 may generate a path 704 to target position 700 that moves outside of passage 112-2 in block 310. Even though path 708 is longer than the path to the target position 700 via passage 112-2, path 708 may allow robot 120-2 to reach the target position 700 in less time than it would via passage 112-2. As will be apparent to those skilled in the art, path 708 may be generated locally in robot 120-2, taking into account both obstacle 600 (not observed by robot 120-2 itself) and path 504 of robot 120-1. Furthermore, robot 120-2 does not need to be aware that the path robot 120-1 travels through passage 112-2 may explicitly interfere with it. Instead, the path generation process in robot 120-2 simply takes into account indications from the occupancy map that a portion of passage 112-2 may be obstructed in the future. The nature of the obstruction does not need to be indicated in the occupancy map.

[0056] More generally, the provision of obstacle and path data from the mobile robot 120 to the server 128, and the iterative updating of the occupancy map by the server 128 based on such obstacle and path data, enables the robot 120 to plan and execute a path through the facility 100 that benefits from obstacle detection performed by other robots 120. Path generation may incorporate the future costs described above to reduce congestion, without requiring path generation by the server 128. As is evident, in some cases, the server 128 may receive observations of the same obstacle from more than one robot 120. In such cases, the server 128 may update the occupancy map to maintain the observations at maximum detection confidence or to average the observation area, etc.

[0057] Returning to Figure 3, while executing the path generated in block 310, the mobile robot 120 may determine in block 335 whether its progress along the generated path is blocked. For example, robot 120-1 may determine whether its progress along path 504 is blocked. The determination in block 335 may be positive, for example, if an obstacle is detected in path 504 and it is not easy to navigate around the obstacle. For example, looking at Figure 8, robot 120-1 is shown as continuing to move along path 504 toward the target position 700. However, an obstacle 800 is observed in path 504 that prevents robot 120-1 from continuing to move along path 504. Furthermore, with the presence of obstacle 600, there is not enough space between obstacles 600 and 800 for robot 120-1 to fit around obstacle 800, so robot 120-1 cannot simply move around obstacle 800. Therefore, robot 120-1 is blocked, and the determination in block 335 is positive. If the determination in block 335 is negative, robot 120-1 continues executing path 504 in block 315.

[0058] Referring again to Figure 3, in block 340, robot 120-1 is configured to select unobservable obstacles from among obstacles detected by the continuous execution of block 315 and tracked, for example, in memory 224. Unobservable obstacles are obstacles previously detected from sensor data, and their observed position no longer falls within the FOV 242 of sensor 240 due to the movement of robot 120-1 after the obstacle detection. Unobservable obstacles may be retained in memory 224 until the observed position of the obstacle again falls within the FOV 242 of at least one of the sensors 240, and / or until a predetermined period of time has elapsed (regardless of whether the position of the obstacle has been observed again). Therefore, unobservable obstacles may be tracked in the robot 120's memory 224, but may not exist at their previously observed position.

[0059] Therefore, in block 340, robot 120-1 is configured to select unobservable obstacles that contribute to the blockage state of robot 120-1. After selecting one or more unobservable obstacles, robot 120-1 may then plan an observation path so that the observation position of the selected obstacles is within the FOV 242 and determine whether the observation position of the selected obstacles remains occupied.

[0060] The selection of unobservable obstacles in block 340 may involve filtering tracked obstacles to exclude any obstacles currently stored in memory 224 within the FOV 242. Thus, referring to Figure 8, obstacle 800 is excluded by filtering. Stationary, permanent, or semi-permanent obstacles, such as support structures 104, may also be excluded by filtering in block 340. This is because such obstacles are not expected to move and can therefore be assumed to exist even without direct observation. Certain other obstacles, such as other mobile robots 120, may also be excluded from selection in block 340 by filtering, even if such robots 120 are outside the FOV 242. For example, each robot 120 may be configured to broadcast its current attitude and direction of movement via short-range communication, such as Bluetooth. Thus, there is little need to directly observe other mobile robots 120 when attempting to clear an obstructed path.

[0061] In some examples, the robot 120 may be constrained to move along a virtual lane in the passage 112, which has been previously defined and labeled on a map, for example, by the server 128. If it is necessary for the robot 120 to remain on such a lane, the robot 120 may filter out any obstacles that are not on such a lane. This is because determining that previously observed obstacles outside the lane no longer exist does not contribute to releasing the robot 120's constraints.

[0062] In block 340, robot 120-1 has selected at least one unobservable obstacle (e.g., obstacle 600 in this example). In block 345, robot 120-1 is configured to generate an observation path. The observation path is generated to move the FOV 242 of at least one of the sensors 242 to encompass the observation position of the selected obstacle. For example, the observation path may be an in-situ rotation in a direction and angle selected to move the FOV 242 toward the observation position of obstacle 600. Figure 9 shows the execution of an observation path, which includes a rotation of approximately 50 degrees to the left, as shown in the lower part of Figure 9, from the obstructed orientation shown in the upper part of Figure 9. This rotation brings the previous observation position of obstacle 600 (shown by the dashed line) into the FOV 242. Robot 120 may, for example, select a rotation direction that minimizes the rotation angle required to bring the observation position of obstacle 600 into the FOV 242.

[0063] Following block 345, robot 120-1 may return to block 335 to determine whether path 504 remains blocked. In the example shown in Figure 9, the obstacle 600 is no longer present, and therefore robot 120 can navigate between the obstacle 800 and the support structure 104 to move toward the target position 504, so the determination in block 335 is negative.

[0064] If path 504 remains blocked in block 335 (for example, if an obstacle 600 remains), the execution of blocks 335, 340, and 345 may be repeated until, for example, the path can be unblocked, or until there are no further unobservable obstacles to select and observe. If there are no unobservable obstacles remaining, robot 120 may wait for a predetermined period of time and monitor obstacle 800 to determine whether or not it has moved. If obstacle 800 has not moved, robot 120 may generate a new path to the target position 500, for example, by discarding path 504. The new path may, for example, return along passage 112-2 and travel outside passage 112-2 to the target position 504.

[0065] In addition to or instead of the rotation described above, a variety of other observation paths may be employed. For example, the observation path may include translational motion, for instance, moving backward from the initial posture of the mobile robot 120 to bring an obstacle located near the chassis 200 and below the FOV 242 into the FOV 242.

[0066] In a further example, the processor 220 may be configured to rank unobservable obstacles in block 340 by, for example, generating a score for each of the unobservable obstacles. The processor 220 may be configured to select any unobservable obstacles that have a score above a certain threshold. Looking at Figure 10, for example, obstacle 600 and another obstacle 1000 are shown, both of which were previously observed by robot 120-1 as it moved along the path toward obstacle 800. In block 340, robot 120-1 may assign a score to each of obstacles 600 and 1000 based on either or both of the obstacle size (larger obstacles receive higher scores) and the distance of the obstacles from robot 120-1 (smaller distances receive higher scores). Robot 120-1 may then be configured in block 340 to select only the unobservable obstacles that have a score above a certain threshold. Alternatively, robot 120-1 may select only the obstacle with the highest score, for example, so that in the first run of block 340, obstacle 1000 is not selected in block 340 due to its longer distance from robot 120-1.

[0067] Therefore, by implementing method 300, robot 120 can perform path generation locally, benefiting from the knowledge of paths and obstacles generated and observed by other robots 120, thereby enabling decentralized path planning that still takes traffic conditions into account. Furthermore, implementing method 300 can facilitate the unblocking of the paths described above while minimizing the observation operations used to determine whether previously observed obstacles still exist.

[0068] The above specification describes specific embodiments. However, those skilled in the art will understand that various modifications and changes can be made without departing from the scope of the invention as described in the following claims. Accordingly, the specification and drawings should be considered illustrative rather than restrictive, and all such modifications are intended to be within the scope of this teaching.

[0069] Benefits, advantages, solutions to problems, and any elements that may produce or make more prominent any benefit, advantage, or solution should not be construed as material, necessary, or essential features or elements of any or all of the claims. The present invention is defined solely by the appended claims and all equivalents of those claims at the time of publication, including any amendments made during the pendency of this application.

[0070] Furthermore, in this document, relational terms such as "first and second," "upper and lower," etc., may be used simply to distinguish one entity or action from another, without necessarily requiring or suggesting any actual relationship or order between such entities or actions. Terms such as "equip," "have," "possess," "include," "contain," "contain," or any other variation thereof are intended to cover non-exclusive inclusion, and therefore, a process, method, article, or apparatus that equips, has, includes, or contains a set of elements may include not only those elements but also other elements that are not expressly listed or that are specific to such process, method, article, or apparatus. The elements followed by "equip," "have," "include," or "contain" do not, unless further restricted, exclude the presence of additional identical elements in the process, method, article, or apparatus that equips, has, includes, or contains that element. The term "one" is defined as one or more unless otherwise expressly stated herein. Terms such as “substantially,” “essentially,” “approximately,” “about,” or any other variation thereof are defined as close to what is understood by those skilled in the art, and in one non-limiting embodiment, such terms are defined as within 10%, in another embodiment within 5%, in another embodiment within 1%, and in another embodiment within 0.5%. As used herein, the term “combined” is defined as being connected, but not necessarily directly and not necessarily mechanically. A device or structure “configured” in a particular way is configured at least in that way, but may be configured in ways not listed.

[0071] Certain expressions may be used herein to enumerate combinations of elements. Examples of such expressions include: “at least one of A, B, and C,” “one or more of A, B, and C,” “at least one of A, B, or C,” and “one or more of A, B, or C.” Unless otherwise specified, the above expressions encompass any combination of A and / or B and / or C.

[0072] It will be understood that some embodiments may comprise one or more dedicated processors (or “processing devices”), such as microprocessors, digital signal processors, custom processors, and field-programmable gate arrays (FPGAs), and specific stored program instructions (including both software and firmware) that control one or more processors to perform some, most, or all of the functions of the method and / or apparatus described herein in conjunction with certain non-processor circuits. Alternatively, some or all of the functions may be performed by a state machine without stored program instructions, or in one or more application-specific integrated circuits (ASICs) in which each function or some combination of certain functions is performed as custom logic. Of course, a combination of the two approaches may also be used.

[0073] Furthermore, embodiments may be implemented as computer-readable storage media storing computer-readable code for programming a computer (e.g., comprising a processor) to perform the methods described and claimed herein. Examples of such computer-readable storage media include, but are not limited to, hard disks, CD-ROMs, optical storage devices, magnetic storage devices, ROMs (read-only memory), PROMs (programmable read-only memory), EPROMs (erasable programmable read-only memory), EEPROMs (electrically erasable programmable read-only memory), and flash memory. Furthermore, those skilled in the art will anticipate that, guided by the concepts and principles disclosed herein, it will be readily possible to generate such software instructions and programs and ICs with minimal experimentation, motivated, for example, by available time, current technology, and economic considerations, despite potentially significant effort and numerous design changes.

[0074] The disclosure summary is provided to enable readers to quickly grasp the nature of the technical disclosure. It is presented with the understanding that it is not to be used to interpret or limit the scope or meaning of the claims. In addition, it is found that in the detailed description above, various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the claimed embodiments require more features than are explicitly described in each claim. Rather, as reflected in the following claims, the subject matter of the invention consists of fewer features than all the features of a single disclosed embodiment. Thus, the following claims are incorporated herein into the detailed description, and each claim itself constitutes separately claimed subject matter.

[0075] Further examples are as follows:

[0076] In one further example, the method comprises generating a path from the current orientation of a mobile robot in a facility to a target location based on an occupancy map of the facility, wherein the occupancy map defines the current occupancy for each of several areas within the facility; detecting obstacles from sensor data captured via the mobile robot's sensors during the execution of the path; storing the detected location of the obstacles; selecting detected obstacles whose detected location is outside the sensor's field of view in response to determining that the path is blocked; generating an observation path so that the observation location of the selected obstacles is within the sensor's field of view; capturing further sensor data during the execution of the observation path and determining from the further sensor data whether the detected location remains obstructed; and modifying the execution of the path based on whether the detected location remains obstructed.

[0077] In one example, modifying the path involves generating an updated path that travels through the detected location in response to determining that the detected location is not obstructed.

[0078] In one example, generating an observation path involves selecting a direction of rotation for the mobile robot such that it minimizes the length of the arc of rotation required to position the observation location of the selected obstacle within the sensor's field of view.

[0079] In one example, selecting detected obstacles includes selecting a subset of detected obstacles whose detection location is outside the sensor's field of view, generating a score for each of the subsets of obstacles, and selecting the detected obstacles based on the scores.

[0080] In one example, generating a score involves determining the size of the detected obstacles in a subset, and determining the distance from the detected obstacles to the mobile robot.

[0081] In one example, the occupancy map further defines the future occupancy for each subset of the domain.

[0082] In one example, the occupancy map defines multiple nodes, each corresponding to one of the regions, and multiple edges extending between each node, where the current and future occupancy for a given region includes the cost associated with the node corresponding to the given region.

[0083] In one example, the occupancy map defines a grid of cells, each corresponding to one of the regions, and the current and future occupancy for a given region includes the cost associated with the cell corresponding to the given region.

[0084] In one example, the occupancy map includes multiple future occupancy values ​​and corresponding consecutive future time values ​​for at least one region.

[0085] In another example, the mobile robot comprises a drive assembly, sensors, and a processor configured to generate a path from the mobile robot's current orientation within a facility to a target location based on an occupancy map of the facility, wherein the occupancy map defines the current occupancy for each of several areas within the facility; to control the drive assembly to execute the path; to detect obstacles from sensor data captured via the sensors during the execution of the path; to store the detected locations of obstacles; to select detected obstacles having their detected locations outside the sensor's field of view in response to determining that the path is blocked; to generate an observation path so that the observation location of the selected obstacles is within the sensor's field of view; to capture further sensor data during the execution of the observation path and determine from the further sensor data whether the detected location remains obstructed; and to modify the execution of the path based on whether the detected location remains obstructed. In one example, the processor is configured to modify the path by generating an updated path that travels through the detected location in response to determining that the detected location is not obstructed. In one example, the processor is configured to generate an observation path by selecting a rotation direction for the mobile robot such that it minimizes the length of the arc of rotation required to position the observation location of a selected obstacle within the sensor's field of view. In another example, the processor is configured to select detected obstacles by selecting a subset of detected obstacles whose detection location is outside the sensor's field of view, generating a score for each of the subsets of obstacles, and selecting the detected obstacles based on the scores. In yet another example, the processor is configured to generate a score by determining the size of the detected obstacles in the subset and determining the distance from the detected obstacles to the mobile robot. In yet another example, the occupancy map further defines future occupancy for each of the subsets of the region.In one example, the occupancy map defines multiple nodes, each corresponding to one of the regions, and multiple edges extending between each node, where the current and future occupancy for a given region includes the cost associated with the node corresponding to the given region. In another example, the occupancy map defines a grid of cells, each corresponding to one of the regions, where the current and future occupancy for a given region includes the cost associated with the cell corresponding to the given region. In yet another example, the occupancy map includes multiple future occupancy and corresponding consecutive future time values ​​for at least one region.

Claims

1. The purpose is to store an occupancy map of a facility, wherein the occupancy map is (i) For each of the multiple areas within the facility, define the current occupancy level corresponding to the current time value, (ii) For each of at least a subset of the region, define a future occupancy corresponding to a time value after the current time value: Memorizing and, Based on the current occupancy and the future occupancy, a path is generated from the current posture of the mobile robot in the facility to the target posture of the mobile robot. In response to the execution of the path by the mobile robot, sensor data representing the vicinity of the mobile robot is captured, The detection of obstacles from the aforementioned sensor data, Transmitting occupancy data for generating an updated occupancy map, which includes updated future occupancy for at least one of the subsets of the region, wherein the occupancy data includes (i) obstacle data indicating the location of the obstacles, and (ii) path data defining the path. Methods that include...

2. The occupancy map defines multiple nodes, each corresponding to one of the regions, and multiple edges extending between each node. The current and future occupancy of a given region includes the cost associated with the node corresponding to the given region. The method according to claim 1.

3. The occupancy map defines a grid of cells, each corresponding to one of the regions. The current and future occupancy of a given region includes the cost associated with the cell corresponding to the given region. The method according to claim 1.

4. The aforementioned future occupancy rate indicates the future presence of another mobile robot in the corresponding region. The method according to claim 1.

5. Determining a first region corresponding to the aforementioned obstacle data, To update the current occupancy rate of the first area determined above, Determining a second region corresponding to the aforementioned route data, To update the future occupancy rate of the second area determined above, The method according to claim 1, further comprising:

6. To transmit the updated occupancy map, which includes the updated current occupancy and the updated future occupancy, to each of the multiple mobile robots. The method according to claim 5, further comprising:

7. The aforementioned current occupancy rate includes the cost value, The aforementioned future occupancy includes a cost value and a future time value. The method according to claim 1.

8. The occupancy map includes, for at least one region, multiple future occupancy values ​​and corresponding consecutive future time values. The method according to claim 7.

9. The detection location of the aforementioned obstacle, and the detection locations of one or more further detected obstacles are stored. In response to determining that the aforementioned path is blocked, the system selects a detected obstacle whose detection position is outside the sensor's field of view, The observation path is generated so that the observation position of the selected obstacle is within the sensor's field of view, During the execution of the aforementioned observation path, additional sensor data is captured, and it is determined from the additional sensor data whether or not the detection position remains obstructed. Based on whether the detection location remains obstructed, the execution of the path is modified. The method according to claim 1, further comprising:

10. Modifying the aforementioned route In response to determining that the detection location is not obstructed, an updated path is generated that travels through the detection location. The method according to claim 9, including the method described in claim 9.

11. Generating the aforementioned observation path means The rotation direction of the mobile robot is selected so as to minimize the rotation angle required to position the observed location of the selected obstacle within the sensor's field of view. The method according to claim 9, including the method described in claim 9.

12. A memory for storing an occupancy map of a facility, wherein the occupancy map is: (i) For each of the multiple areas within the facility, define the current occupancy level corresponding to the current time value, (ii) For each of at least a subset of the region, define a future occupancy corresponding to a time value after the current time value: Memory and It is a processor, Based on the current occupancy and the future occupancy, generate a path from the current posture of the mobile robot in the facility to the target posture. In response to the execution of the aforementioned path, sensor data representing the vicinity of the mobile robot is captured. To detect obstacles from the aforementioned sensor data, and Transmitting occupancy data for generating an updated occupancy map, which includes updated future occupancy for at least one of the subsets of the region, wherein the occupancy data includes (i) obstacle data indicating the location of the obstacles, and (ii) path data defining the path. Configured to perform, Processor and A mobile robot equipped with [specific features / equipment].

13. The occupancy map defines multiple nodes, each corresponding to one of the regions, and multiple edges extending between each node. The current and future occupancy of a given region includes the cost associated with the node corresponding to the given region. The mobile robot according to claim 12.

14. The occupancy map defines a grid of cells, each corresponding to one of the regions. The current and future occupancy of a given region includes the cost associated with the cell corresponding to the given region. The mobile robot according to claim 12.

15. The aforementioned future occupancy rate indicates the future presence of another mobile robot in the corresponding region. The mobile robot according to claim 12.

16. The aforementioned processor, The detection location of the aforementioned obstacle, and the detection locations of one or more further detected obstacles are stored. In response to determining that the aforementioned path is blocked, the system selects a detected obstacle whose detection position is outside the sensor's field of view, The observation path is generated so that the observation position of the selected obstacle is within the sensor's field of view, During the execution of the aforementioned observation path, additional sensor data is captured, and it is determined from the additional sensor data whether or not the detection position remains obstructed. Based on whether the detection location remains obstructed, the execution of the path is modified. Further configured to perform, The mobile robot according to claim 12.

Images