Detection of obstacle motion state for mobile robot

Abstract

The method includes the steps of: using sensors of a mobile robot to capture sensor data representative of the mobile robot's physical environment; detecting an obstacle in the physical environment from the sensor data; in response to detecting the obstacle, determining from the sensor data whether the obstacle exhibits a predetermined attribute; assigning a first operating state or a second operating state to the obstacle in accordance with the determination; selecting a navigation constraint based on the assigned operating state; and controlling a movement assembly of the mobile robot to navigate the physical environment based on the selected navigation constraint.

Description

[Background technology]

[0001] Autonomous or semi-autonomous mobile robots may be deployed within facilities such as warehouses, manufacturing facilities, medical facilities, etc., to, for example, transport items within the facility. To navigate within the facility, the mobile robot captures sensor data (e.g., images, etc.) and detects obstacles within the sensor data. The mobile robot may generate a path toward, for example, a target location, taking the detected obstacles into account. Various obstacles may exist within the facility, including stationary and moving obstacles. Summary of the Invention

[0002] The accompanying drawings, together with the following detailed description, which are incorporated in and form a part of this specification, serve to further explain embodiments of the concepts comprising the claimed invention(s) and to explain various principles and advantages of those embodiments, in which like reference numbers indicate identical or functionally similar elements throughout the different views. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a schematic diagram of a mobile robot for transporting goods deployed within a facility.

[0004] [Figure 2] FIG. 2 is a schematic diagram of certain components of the mobile robot of FIG.

[0005] [Figure 3] FIG. 3 is a flowchart showing a method for detecting the motion state of an obstacle.

[0006] [Figure 4] FIG. 4 is a schematic diagram illustrating an exemplary implementation of block 305 of the method of FIG.

[0007] [Figure 5]FIG. 5 is a schematic diagram illustrating an exemplary implementation of block 310 of the method of FIG.

[0008] [Figure 6] FIG. 6 illustrates an exemplary implementation of blocks 320, 325 of the method of FIG.

[0009] [Figure 7] FIG. 7 illustrates another exemplary implementation of blocks 320, 325, and 330 of the method of FIG.

[0010] [Figure 8] FIG. 8 illustrates an exemplary implementation of block 345 of the method of FIG.

[0011] [Figure 9] FIG. 9 illustrates another exemplary implementation of block 345 of the method of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] Those skilled in the art will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.

[0013] Components of the apparatus and methods are designated by conventional numerals in the drawings, where appropriate, and the drawings show only those specific details relevant to understanding the embodiments of the invention so as not to obscure the detailed disclosure, which disclosure will be readily apparent to those skilled in the art having the benefit of the descriptions herein.

[0014] In one embodiment, to optimize navigation around an obstacle, the robot identifies whether the obstacle is moving or stationary and implements a corresponding navigation approach.

[0015] Embodiments disclosed herein are directed to a method comprising: using sensors of a mobile robot to capture sensor data representative of a physical environment of the mobile robot; detecting an obstacle in the physical environment from the sensor data; in response to detecting the obstacle, determining from the sensor data whether the obstacle exhibits a predetermined attribute; assigning a motion state to the obstacle in accordance with the determination; selecting a navigation constraint based on the assigned motion state; and controlling a movement assembly of the mobile robot to navigate the physical environment based on the selected navigation constraint.

[0016] Additional embodiments disclosed herein are directed to a computing device including a sensor and a processor, wherein the processor is configured to: use the sensor to capture sensor data representative of a physical environment of a mobile robot; detect an obstacle in the physical environment from the sensor data; in response to detecting the obstacle, determine from the sensor data whether the obstacle exhibits a predetermined attribute; assign a motion state to the obstacle in accordance with the determination; select a navigation constraint based on the assigned motion state; and control a movement assembly of the mobile robot to navigate the physical environment based on the selected navigation constraint.

[0017] FIG. 1 illustrates the interior of a facility 100, such as a warehouse, manufacturing facility, medical facility, or the like. The facility 100 includes a plurality of support structures 104 that hold items 108. In the illustrated example, the support structures 104 include shelving modules arranged in sets to form, for example, aisles 112-1, 112-2 (collectively referred to as aisles 112 and generally as aisle 112; similar nomenclature is used for other components herein). As shown in FIG. 1 , the support structures 104 in the form of shelving modules include support surfaces 116 that support the items 108. In other examples, the support structures 104 may include pegboards, bins, or the like.

[0018] In other embodiments, facility 100 may include fewer aisles 112 than are shown in FIG. 1 , or may include more aisles 112 than are shown in FIG. 1 . In the illustrated embodiment, aisles 112 are formed by sets of eight support structures 104 (four on each side), although a facility may have a variety of other aisle layouts. As will be apparent, each aisle 112 is an open-ended space bounded on both sides by support structures 104. The aisles 112 may be traversed by people, vehicles, and the like. In yet other embodiments, facility 100 need not include aisles 112, but instead may include an assembly line, or the like.

[0019] The items 108 may be processed according to various processes, depending on the nature of the facility 100. In some examples, the facility 100 is a shipping facility, a distribution facility, etc., and the items 108 may be placed on a support structure 104 for storage and then removed from the facility for shipping. The placement of the items 108 on the support structure and / or the removal of the items 108 from the support structure may be performed or assisted by a mobile robot 120. For example, based on the size and / or layout of the facility 100, a greater number of robots 120 than those shown in FIG. 1 may be deployed within the facility 100. The components of the robot 120 are described in more detail below. In general, each robot 120 within the facility 100 is configured to transport the items 108 within the facility 100.

[0020] The robot 120 may be configured to track its pose (e.g., position and orientation) within the facility 100, for example, within a coordinate system 124 pre-established within the facility 100. The robot 120 may autonomously navigate (move) within the facility 100, for example, to locations assigned to the robot 120, where it may pick up and / or deposit items 108. The items 108 may be placed in or on the robot 120 and retrieved from the robot 120 by human workers and / or mechanized devices, such as a robotic arm, deployed within the facility 100. The locations to which each robot 120 travels may be assigned to the robot 120 by a central server 128. That is, the server 128 is configured to assign tasks to the robot 120. Each task may include one or more locations to travel to and / or one or more actions to perform at those locations. For example, the server 128 may assign the robot 120 the task of traveling to a location defined by the coordinate system 124 and waiting to receive one or more items 108 at that location.

[0021] Tasks may be assigned to robots 120 by exchanging messages between server 128 and robots 120, for example, via a suitable combination of local area networks and wide area networks. Server 128 may be located at facility 100 or may be remotely located away from facility 100. In some embodiments, server 128 is configured to assign tasks to robots 120 at multiple facilities and need not be physically located at any of the individual facilities.

[0022] To navigate to a given location within facility 100 (e.g., a target location assigned to mobile robot 120 by server 128), mobile robot 120 may be configured to acquire sensor data representative of at least a portion of the robot's physical environment (i.e., the robot's surroundings). From the sensor data, robot 120 may be configured to detect obstacles in its vicinity and, if necessary, navigate around or away from the obstacles.

[0023] As will be apparent to those skilled in the art, the facility 100 may include a wide variety of obstacles. For example, as shown in FIG. 1 , obstacles that the mobile robot 120 must avoid include a support structure 104, humans, such as a worker 132, and mobile equipment, such as a forklift 136. The forklift 136 has a chassis that supports a movable component, such as a set of tines or blades 140, a cab 144, and an indicator light or beacon 148 configured to illuminate when the forklift 136 is powered on. Obstacles may also include other mobile robots, boxes, pallets, etc. The mobile robot 120 may be configured to employ different navigation constraints when moving near different obstacles. For example, when moving along a static (i.e., stationary) obstacle such as the support structure 104, the mobile robot 120 may plan a navigation path that falls within a certain threshold distance (e.g., 10 cm) and may travel such a path at a certain maximum speed (e.g., 2 m / s). When the robot 120 moves near a moving obstacle, such as a worker 132, it may plan a navigation path that maintains a greater distance (e.g., 2 m) from the worker 132 and may move at a lower maximum speed (e.g., 1 m per second).

[0024] Using different navigation constraints for different obstacles may, for example, reflect increased uncertainty in the future motion of a moving obstacle. For example, the mobile robot 120 may be configured to classify each detected obstacle as static (stationary) or dynamic (moving) and apply a distinct set of navigation constraints to each category. Although, certain obstacles, such as the forklift 136, may be stationary when first observed by the mobile robot 120 and thus classified as static, they may subsequently move quickly, requiring evasive action by either or both the mobile robot 120 and the operator of the forklift 136 (e.g., operator 132) to avoid a collision.

[0025] Thus, the mobile robot 120 is configured not only to detect obstacles and determine the type of such obstacle (e.g., distinguish between the worker 132, the forklift 136, and the support structure 104), as described in more detail below, but also to assign motion states to particular obstacles. For example, in the case of a forklift 136, a motion state indicates whether the forklift 136 is currently being operated, e.g., by the worker 132, regardless of whether the forklift 136 is moving. The mobile robot 120 can use the assigned motion states to select optimized navigation constraints. For example, the forklift 136 can be treated as a dynamic obstacle even when it is not moving when observed by the mobile robot 120.

[0026] Before describing the functions implemented by the robots 120 in more detail, certain components of the robots 120 will be described with reference to Figure 2. As shown in Figure 2, each robot 120 includes a chassis 200 that supports various other components of the robot 120. In particular, the chassis 200 supports a movement assembly 204, such as one or more electric motors that drive a set of wheels, tracks, etc. The movement assembly 204 may include one or more sensors, such as wheel odometers, an inertial measurement unit (IMU), etc.

[0027] The chassis 200 also supports receptacles, shelves, etc. for supporting the items 108 during transport. For example, the robot 120 may include a combination of selectable receptacles 212. In the illustrated example, the chassis 200 supports racks 208, which may include, for example, rails or other structural features configured to support receptacles 212 at various heights above the chassis 200. Thus, the receptacles 212 may be attached to and detached from the racks 208, allowing different combinations of receptacles 212 to be supported by the robot 120.

[0028] The robot 120 may also include an output device such as a display 216. In the illustrated example, the display 216 is mounted above the rack 208, although it should be apparent that in other embodiments, the display 216 may be located elsewhere on the robot 120. The display 216 may, in some embodiments, include an integrated touchscreen or other input device. The robot 120 may also include other output devices in addition to or instead of the display 216. For example, the robot 120 may include one or more speakers, lights such as a strip of light-emitting diodes (LEDs) along the rack 208, etc.

[0029] The chassis 200 of the robot 120 also supports various other components, including a processor 220 (e.g., one or more central processing units (CPUs), graphics processing units (GPUs), or dedicated hardware controllers such as application specific integrated circuits (ASICs)). The processor 220 is communicatively coupled to a non-transitory computer-readable medium such as memory 224 (e.g., a suitable combination of volatile and non-volatile memory elements). The processor 220 is also coupled to a communication interface 228, such as a wireless transceiver, that allows the robot 120 to communicate with other computing devices, such as a server 128 or other robots 120.

[0030] Memory 224 stores various data used for autonomous or semi-autonomous navigation, including applications 232 executable by processor 220 to implement navigation functions and other task performance functions. In some embodiments, the aforementioned functions may be implemented via multiple different applications stored in memory 224.

[0031] Chassis 200 may also support sensors 240, such as one or more cameras and / or depth sensors (e.g., LiDAR, depth cameras, time-of-flight cameras) coupled to processor 220. The sensors 240 are configured to capture image and / or depth data indicative of at least a portion of the physical environment of robot 120. Data captured by sensors 240 may be used by processor 220 for navigation purposes, such as path planning and obstacle avoidance, and in some embodiments, for updating maps of the facility.

[0032] Each sensor 240 has a field of view (FOV). For example, a first FOV 242a corresponds to a laser scanner, such as a lidar sensor, disposed on the front of the chassis 200. The FOV 242a may be substantially two-dimensional, e.g., extending forward in a substantially horizontal plane. A second FOV 242b corresponds to a camera (e.g., a depth camera, a color camera, etc.) mounted on the front of the chassis 200. As will be apparent, various other optical sensors with their own FOVs 242 may be disposed on the chassis 200 and / or rack 208.

[0033] Components of the robot 120 that consume power may be supplied with such power from batteries 244 (e.g., implemented as one or more rechargeable batteries housed within the chassis 200 and rechargeable via a fill port (not shown) or other suitable charging interface).

[0034] 3, a method 300 of obstacle motion state detection is shown. Method 300 is described below with respect to its exemplary implementation within facility 100. In particular, as shown in FIG. 3, the blocks of method 300 are performed by mobile robot 120, e.g., via execution of application 232 by processor 220. In other examples, certain blocks of method 300 may be performed by server 128, e.g., to reduce the computational load on processor 220.

[0035] In block 305, the mobile robot 120 is configured to capture sensor data, for example, by activating one or more sensors 240. For example, as shown in the overhead view of FIG. 4, the mobile robot 120 may activate one or more cameras defining a field of view (FOV) 242b to capture images of a portion of the robot's 120 surroundings, such as a series of images at an appropriate frequency (e.g., 30 Hz). In particular, the FOV 242b is oriented forward, in the direction the robot 120 is moving. As seen in FIG. 4, the FOV 242b includes a portion of the support structure 104 as well as the forklift 136 that is outside the aisle 112-1 in which the robot 120 is currently located. The robot 120 may activate multiple sensors, including sensors of different types, in block 305. As will be apparent, the object represented in the sensor data can be located in the coordinate system 124 using the current position and pose of the robot 120 and the relative position of the object with respect to the robot 120 derived from the sensor data.

[0036] 3, in block 310, the robot 120 is configured to detect obstacles in the sensor data captured in block 305 and classify the detected obstacles. Obstacle detection and classification includes detecting the location of obstacles in the coordinate system 124 from the sensor data and the robot's current position and pose, and determining the type of each detected obstacle.

[0037] Referring to FIG. 5 , an image 500 captured in block 305 is shown, e.g., with the robot 120 in the orientation shown in FIG. 4 . Thus, a portion 504 of the support structure 104 and the forklift 136 are visible in the image 500. The processor 220 may perform various operations to detect and classify obstacles in the image 500. For example, the processor 220 may be configured to execute one or more classification models, such as a trained convolutional neural network (CNN), to obtain the location (e.g., in the form of a bounding box) and type of obstacle in the image 500. Such a classifier may be configured, for example, to generate a boundary surrounding each obstacle and generate a score or other suitable confidence measure for each boundary. The score generated for each boundary may indicate the likelihood that the boundary contains each obstacle among multiple obstacle types that the classifier is trained to recognize. For example, a classifier configured to recognize a forklift, a human, and a support structure may generate three scores for each boundary and select the highest score as the detected obstacle type.

[0038] As shown in the lower portion of Figure 5, processor 220 may generate boundary 508 surrounding forklift 136 and boundary 512 surrounding portion 504 of support structure 104. Processor 220 may also detect obstacle types 516, 520 as contents of boundaries 508, 512 (e.g., selected from a plurality of predetermined obstacle types on which the aforementioned classifier was trained). For example, obstacle type 516 "forklift" is detected for boundary 508, and obstacle type 520 "shelf" is detected for boundary 512. Processor 220 may also determine the speed of a detected obstacle, such as forklift 136, from the sensor data captured in block 305. In the example shown in Figures 4 and 5, forklift 136 is stationary (i.e., has a speed of zero).

[0039] 3, processor 220 is configured to perform a series of evaluations contained within boundary (dashed area) 312 for each obstacle detected in block 310. That is, the blocks of method 300 within boundary (dashed area) 312 are repeated for each detected obstacle, and when no detected obstacles remain to be evaluated, the method proceeds (to block 345, described further below).

[0040] In block 315, the processor 220 is configured to determine whether to detect the motion state of the obstacle detected and classified in block 310. Certain obstacles, such as the support structure 104, do not need to be evaluated to detect their motion state. For example, the support structure 104 may always be treated as a static obstacle for navigation purposes (i.e., the navigation constraints applied to the support structure 104 may be constant). As yet another example, an obstacle, such as the worker 132, may always be treated as a dynamic obstacle and therefore may not require evaluation of its motion state. As described below, the motion state of an obstacle may be used to determine navigation constraints to apply when navigating near the obstacle. For example, when the robot 120 determines that the forklift 136 is in an operational state, the robot 120 may treat the forklift 136 as a dynamic (i.e., moving) obstacle, even if the forklift 136 is currently stationary.

[0041] The determination in block 315 may be made according to whether the obstacle is currently moving and according to attribute definitions maintained by the robot 120, such as in memory 224. The obstacle's movement may be detected from a series of images or other sensor data captured in block 305 and expressed as a vector (e.g., position, direction, and velocity) in coordinate system 124. Detecting the motion state of the obstacle is used to determine whether the obstacle should be treated as dynamic (i.e., moving) even if the obstacle is currently stationary. For example, in the case of a forklift 136, detecting that the forklift 136 is in operation (not idle) may lead to treating the forklift 136 as a dynamic obstacle even if the forklift 136 is currently stationary. One result is that if the forklift 136 is moving, motion state detection may be bypassed. More generally, for a currently moving obstacle, the determination in block 315 is negative (no). For stationary obstacles, the determination in block 315 is further based on the attribute definitions discussed above.

[0042] Referring to FIG. 6 , multiple associations 600 between attribute definitions and obstacle types are shown. Because the obstacle types “Shelf” and “Human” have no attributes associated with them, the determination in block 315 for those obstacle types is negative. On the other hand, the obstacle type “Forklift” is associated with the attributes “Worker Present” and “Beacon On,” and the determination in block 315 is positive (yes) for the detected forklift 136 in image 500. That is, a stationary forklift detected in the sensor data from block 305 is subject to additional evaluation before selecting a navigation constraint. The operating state of the detected forklift 136 in image 500 is initially idle (e.g., because the forklift 136 is stationary), but if the forklift 136 exhibits one or more of the attributes shown in FIG. 6 , the operating state may be updated.

[0043] After a positive determination is made at block 315 for one or more obstacles detected at block 310, processor 220 is configured to retrieve, for example, from memory 224, definitions of attributes to be evaluated for the associated obstacles. The attribute definitions may specify processing operations to be performed on sensor data corresponding to particular obstacles to determine whether those obstacles exhibit particular attributes. In this example, processor 220 is configured to retrieve the definitions of the "worker present" and "beacon on" attributes shown in FIG. 6.

[0044] The worker presence attribute definition may, for example, define a classification model configured to distinguish between a forklift with a human worker in the cab 144 and a forklift with an empty cab 144. The worker presence attribute definition may, in another example, also define a threshold distance between the forklift 136 and a human detected in block 310, such that a human detected closer to the forklift 136 than the threshold may be considered to be the operator of the forklift 136. In other words, the worker presence attribute definition may define one or more aspects of determining whether a human is physically associated with the forklift 136.

[0045] The attribute definition for beacon on may include criteria employed to determine whether the beacon 148 is illuminated, which may include, for example, intensity and / or color thresholds to be evaluated for the portion of the image 500 within the boundary 508.

[0046] Next, processor 220 is configured to determine, in block 325, whether a given attribute obtained in block 320 is indicated by the corresponding obstacle detected in block 310. In the example of Figure 6, i.e., during the first execution of block 325, processor 220 is configured to determine whether an operator is present in forklift 136, for example, by providing the portion of image 500 within boundary 508 to the classifier described above. As can be seen in Figures 5 and 6, if no operator is present in forklift 136, result 604 of the evaluation in block 325 is negative. Thus, processor 220 is configured to bypass block 330 (leaving the operational state of the forklift in a non-operating state) and determine, in block 335, whether any attributes remain to be evaluated.

[0047] In this example, the determination at block 335 is affirmative because the beacon-on attribute remains to be evaluated. Therefore, processor 220 returns to block 325 to determine whether the next attribute is indicated by the associated obstacle. The determination of whether beacon 148 is illuminated generates result 608, which is obtained, for example, by searching for a high-intensity area, a specific color area, etc., within boundary 508. Result 608, shown in FIG. 6, indicates that no illuminated beacon was detected. Therefore, the determination at block 325 is negative. In this example, no attributes from block 320 remain to be evaluated for boundary 508, so the determination at block 335 is negative. Therefore, processor 220 proceeds to block 340. The operating state of forklift 136 detected in image 500 remains in the aforementioned idle state because it was not updated via block 330.

[0048] In block 340, processor 220 may be configured to set an obstacle category for the obstacle based on a combination of the classification (class) and motion state from block 310, or based on a combination of the classification (class), motion state, and operational state evaluated via blocks 320 through 335. The obstacle category may be selected from, for example, a static category and a dynamic category. As previously described, the static category is generally applied to obstacles that are stationary, and the dynamic category is generally applied to obstacles that are moving. However, processor 220 may apply the dynamic category to a static obstacle when the obstacle is in an operational state rather than an idle state (i.e., when the obstacle exhibits at least some of the attributes obtained in block 320). In this exemplary implementation of block 340, processor 220 applies the static category to forklift 136 because the operational state of forklift 136 remains “idle.”

[0049] Setting an obstacle category may be implemented by storing a category label associated with the obstacle, which is subsequently used in the navigation process to select a navigation constraint when traveling near the obstacle, for example, in block 345. In other examples, when applicable, block 340 may be omitted and navigation constraints may be selected based on a combination of the obstacle class (obtained from block 310) and the operational state (maneuvering state) obtained from blocks 320-335.

[0050] After processing the forklift 136 detected in the image 500, the processor 220 is configured to repeat the previous portion of the method 300 within the boundary 512 of the shelf portion 504 detected in the image 500. As previously described, the determination at block 315 for the shelf portion 504 is negative, and at block 340 the processor 220 is configured to classify the shelf portion 504 as a static obstacle.

[0051] In block 345, after processing and classifying each obstacle detected in block 310, processor 220 is configured to set navigation constraints based on the detected obstacle(s) and generate a path for moving facility 100 toward a target location assigned by server 128, for example.

[0052] Before describing the performance of block 345, a further example of motion state detection is shown in FIG. 7. In FIG. 7, the robot 120 stores attribute associations 700 that include additional attribute associations for a forklift obstacle type. In particular, the forklift obstacle type is associated with the two attributes described above and further associated with a "blade raised" attribute. The "blade raised" attribute is exhibited by a forklift 136 when the blade 140 is at least a threshold distance above the ground (e.g., the floor of the facility 100). Evaluating whether the forklift 136 exhibits the "blade raised" attribute may include providing a portion 704 of the image captured in block 305 to a further classifier, for example, trained to distinguish between a forklift with a lowered blade 140 and a forklift with a raised blade 140.

[0053] As can also be seen in Figure 7, portion 704 of the image indicates that a worker 708 is present within the cab 144 of the forklift 136. After obtaining the three attributes described above in block 325, processor 220 may be configured to first determine whether the forklift 136 exhibits the "worker present" attribute by providing portion 704 of the image to a classifier, such as that described above in connection with Figure 5, to generate result 712 indicating that a worker is present. That is, the forklift 136 is determined to exhibit the "worker present" attribute. Accordingly, processor 220 proceeds to block 330 and updates the operational status of the forklift 136.

[0054] Updating the operational state in block 330 may be implemented in various ways. For example, if any of the attributes evaluated via block 325 are indicated by the forklift 136, the operational state may be updated from "idle" to "operating." In another example, updating the operational state in block 330 may include incrementing the score of each of the indicated attributes. For example, each attribute may be associated with a score component, and when the aggregated score meets a threshold, the operational state may be updated from "idle" to "operating." In the example shown in FIG. 7, the "operator presence" attribute contributes a value of "6" (716) to such score (it is understood that various scoring mechanisms and values ​​may be employed). In this example, the threshold score for changing the operational state to "operating" is "8." Therefore, the operational state remains "idle."

[0055] Following a negative determination at block 335, processor 220 may determine at the next instance of block 325 whether the forklift exhibits the "blade raised" attribute. As previously discussed, the evaluation at block 325 may include providing image portion 704 to a classifier configured to return a result 720 indicating whether blade 140 is raised or lowered. In the example of FIG. 7, result 720 indicates that the blade is lowered (i.e., forklift 136 does not exhibit the "blade raised" attribute). Therefore, no adjustment to the operational state score is made.

[0056] In further execution of block 325, processor 220 may determine whether forklift 136 exhibits the "beacon on" attribute. In the example of FIG. 7, beacon 148 is illuminated, and processor 220 identifies an increased intensity region and / or predetermined color region 724 in image portion 704. Thus, result 728 of the determination in block 325 indicates that forklift 136 exhibits the "beacon on" attribute. Accordingly, in block 330, processor 220 may be configured to update the score using a predetermined value (732) associated with the "beacon on" attribute (e.g., "3" in this example). Thus, the total score associated with the motion state ("9") exceeds the aforementioned threshold, and motion state 736 is updated from "idle" to "moving" in block 330. Thus, in block 340, forklift 136 is classified as a moving obstacle, even though it is currently stationary.

[0057] In block 345, the processor 220 is configured to select navigation constraints to, for example, generate a path toward the goal location. The navigation constraints include constraints associated with the obstacles detected and classified in block 310. The navigation constraints may include, for example, a threshold distance that the obstacle must be maintained within during navigation (i.e., the robot 120 must remain at least the threshold distance from the obstacle). The navigation constraints may also include a maximum speed that the robot 120 should adopt when it is near an obstacle.

[0058] The navigation constraints may be selected from a table or other mapping that associates obstacle classes and operational states with constraints in block 345. For example, memory 224 may store a first set of navigation constraints associated with a forklift obstacle class for use when forklift 136 is in an idle state and a second set of navigation constraints associated with the forklift obstacle class for use when forklift 136 is in an operational state. In other examples, the navigation constraints may be stored in association with categories described in connection with block 340, such as a first set of constraints used for static obstacles and a second set of constraints used for dynamic obstacles.

[0059] In some examples, navigation constraints, such as the aforementioned separation distances, need not be explicitly stored. For example, processor 220 may be configured to maintain an occupancy grid, lattice, etc., representing facility 100 (or a portion thereof). Navigation constraints may be implemented by assigning an occupancy value (e.g., a cost) to each cell of the occupancy grid or each edge connecting nodes in the lattice. The path generation in block 345 may be configured to travel to the target location while minimizing cumulative cost; thus, distance and / or speed constraints may be implemented by assigning higher or lower costs to cells or edges at obstacle locations.

[0060] 8, an exemplary performance of block 345 is shown for a scenario in which the forklift 136 is idle, as described in connection with FIG. 6. As seen in FIG. 8, the selected navigation constraints include a separation distance represented by a boundary 800 that surrounds the forklift 136 and a separation distance represented by a boundary 804 that surrounds a portion of the support structure 104. As will be apparent, the boundaries 800, 804 do not need to be explicitly stored by the robot 120 (e.g., as a set of coordinates in the coordinate system 124). Instead, these boundaries may be derived from occupancy costs, edge costs, etc., that are assigned based on the categories assigned to the forklift 136 and the support structure 104 in block 340.

[0061] The robot 120 can be configured to generate a path 808 to a goal location 812 according to navigation constraints (e.g., boundaries 800, 804). As can be seen in Figure 8, there is enough unoccupied space between the boundaries 800, 804 to allow the robot 120 to pass through.

[0062] 9, another exemplary performance of block 345 is shown for a scenario in which the forklift 136 is in operation, as described in connection with FIG. 7. In FIG. 9, a boundary 900 surrounding the forklift 136 implements a larger separation distance from the forklift 136, preventing the robot 120 from planning a path that moves close to the forklift 136 as in FIG. 8. As can be seen, there is not enough space between the boundaries 900, 804 to allow the robot 120 to pass. The robot 120 may plan a path 904, for example, toward the end of aisle 112-1, and then may pause, for example, to wait for the forklift 136 to move before planning a further path to the target location 812.

[0063] Various other attributes may be used to evaluate the operational state of the forklift 136 or other obstacles. Further exemplary attributes include the presence or absence of running lights different from the beacon 148. Evaluation of operational states may be extended to other obstacles in addition to or instead of the forklift. For example, the processor 220 may be configured to evaluate the operational state of a cherry picker, e.g., using some or all of the attributes described above. Further exemplary attributes that may be used to evaluate the operational state of a cherry picker include determining whether the boom of the cherry picker is raised (where a raised boom indicates an operational state rather than an idle state). In a further example, the operational state of another mobile robot 120 may be evaluated via method 300, e.g., by detecting the presence or absence of illuminated running lights.

[0064] The foregoing specification describes particular embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of the invention as set forth in the claims. Accordingly, the specification and drawings are to be understood in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present teachings.

[0065] Benefits, advantages, solutions to problems, and any elements that may cause or make more noticeable any benefit, advantage, or solution should not be construed as critical, necessary, or essential features or elements of any or all of the claims. The present invention is defined solely by the appended claims, including any amendments made during the pendency of this application and all equivalents of those claims as issued.

[0066] Furthermore, in this document, related terms such as first and second, above and below, etc. may be used only to distinguish one entity or operation from another and may not necessarily require or imply an actual relationship or ordering between such entities or operations. The terms "comprises," "comprising," "has," "having," "include," "including," "contains," "containing," or any other variations thereof, are intended to cover a non-exclusive inclusion. A description of a process, method, article, or apparatus as comprising, having, including, or containing a list of elements does not include only those elements, but may include other elements not expressly listed and other elements inherent in such process, method, article, or apparatus. An element preceded by "comprises ... a," "has ...," "includes ... a," or "contains ... a" does not, without further constraints, preclude the presence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, or contains the element. The terms "a" and "an" are defined as one or more, unless expressly stated otherwise. The terms "substantially," "essentially," "approximately," "about," and any other variations thereof are defined as close as understood by one of ordinary skill in the art, and in one non-limiting embodiment, such terms are defined as within 10%, in another embodiment within 5%, in another embodiment within 1%, and in another embodiment within 0.5%. The term "coupled," as used herein, is defined as connected, but not necessarily directly, and not necessarily mechanically. A device or structure "configured" in a certain way is configured in at least that way, but may also be configured in ways not listed.

[0067] Certain phrases may be used herein to enumerate (list) combinations of elements. Examples of such phrases include "at least one of A, B, and C," "one or more of A, B, and C," "at least one of A, B, or C," and "one or more of A, B, or C." Unless expressly stated otherwise, the foregoing phrases include any combination of A and / or B and / or C.

[0068] It will be understood that some embodiments may consist of one or more special-purpose processors (or "processing devices"), such as microprocessors, digital signal processors, customized processors, field programmable gate arrays (FPGAs), etc., in combination with specific non-processor circuitry and unique stored program instructions (including both software and firmware) that control the one or more processors to implement some, most, or all of the functionality of the methods and / or apparatus described herein. Alternatively, some or all of the functionality may be implemented by a state machine without stored program instructions, or by one or more application-specific integrated circuits (ASICs) in which each function or a particular combination of functions is implemented as custom logic. Of course, a combination of the two approaches may also be used.

[0069] Furthermore, embodiments may be implemented as a computer-readable storage medium having computer-readable code stored thereon for programming a computer (e.g., including a processor) to perform the methods described and claimed herein. Examples of such computer-readable storage media include, but are not limited to, hard disks, CD-ROMs, optical storage devices, magnetic storage devices, ROMs (read-only memories), PROMs (programmable read-only memories), EPROMs (erasable programmable read-only memories), EEPROMs (electrically erasable programmable read-only memories), flash memories, etc. Furthermore, it is expected that those skilled in the art will be readily able to generate such software instructions, programs, and ICs with minimal experimentation when guided by the concepts and principles disclosed herein, despite significant effort and numerous design choices, motivated by, for example, available time, current technology, and economic considerations.

[0070] This Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. It may also be noted that in the foregoing Detailed Description, various features are grouped together in various embodiments for the purpose of facilitating this disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. The following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as separately claimed subject matter.

Claims

1. capturing sensor data representative of the mobile robot's physical environment using sensors of the mobile robot; detecting obstacles in the physical environment from the sensor data; responsive to detecting the obstacle, determining from the sensor data whether the obstacle exhibits a predetermined attribute; assigning an operational state to the obstacle in accordance with the determination; selecting a navigation constraint based on the assigned operating state; controlling a locomotion assembly of the mobile robot to navigate the physical environment based on the selected navigation constraints; A method comprising:

2. Detecting the obstacle includes detecting a location of the obstacle and a type of the obstacle selected from a plurality of obstacle types.

2. The method of claim 1 .

3. storing in a memory of the mobile robot an association between the definition of the predetermined attribute and the detected type of the obstacle.

3. The method of claim 2, further comprising:

4. obtaining the definition of the predetermined attribute according to the detected type of the obstacle before determining whether the obstacle exhibits the predetermined attribute.

4. The method of claim 3, further comprising:

5. The step of assigning an operational state to the obstacle comprises: assigning a first operational state when the obstacle exhibits the predetermined attribute; assigning a second operational state when the obstacle does not exhibit the predetermined attribute; 2. The method of claim 1, comprising:

6. determining from the sensor data whether the obstacle exhibits a further predetermined attribute; Further provided with The step of assigning an operational state to the obstacle is further dependent on determining whether the obstacle exhibits a further predetermined attribute.

2. The method of claim 1 .

7. The navigation constraints include a maximum movement speed of the mobile robot.

2. The method of claim 1 .

8. The navigation constraints define the distance from the obstacles that should be maintained during navigation of the physical environment.

2. The method of claim 1 .

9. Determining whether the obstacle exhibits a predetermined attribute includes determining whether a human is physically associated with the obstacle.

2. The method of claim 1 .

10. Determining whether the obstacle exhibits a predetermined attribute includes determining whether a movable component of the obstacle is in a predetermined position relative to the obstacle.

2. The method of claim 1 .

11. Determining whether the obstacle exhibits a predetermined attribute includes determining whether a light emitter of the obstacle is activated.

2. The method of claim 1 .

12. A sensor, a processor; 1. A computing device comprising: The processor: using said sensors to capture sensor data representative of a physical environment of the mobile robot; Detecting obstacles in the physical environment from the sensor data; In response to detecting the obstacle, determining from the sensor data whether the obstacle exhibits a predetermined attribute; assigning an operational state to the obstacle in accordance with the determination; selecting a navigation constraint based on the assigned operational state; Controlling a movement assembly of the mobile robot to navigate the physical environment based on the selected navigation constraints. It is configured as follows:

1. A computing device comprising:

13. The processor is configured to detect the obstacle by detecting a location of the obstacle and a type of the obstacle selected from a plurality of obstacle types. The computing device of claim 12 .

14. a memory for storing an association between the predetermined attribute definition and the detected type of the obstacle; 14. The computing device of claim 13, further comprising:

15. The processor is further configured to obtain the definition of the predetermined attribute according to the detected type of the obstacle before determining whether the obstacle exhibits the predetermined attribute.

15. The computing device of claim 14.

16. The processor is further configured to assign a motion state to the obstacle by assigning a first motion state when the obstacle exhibits the predetermined attribute and assigning a second motion state when the obstacle does not exhibit the predetermined attribute. The computing device of claim 12 .

17. The processor is further configured to determine from the sensor data whether the obstacle exhibits a further predetermined attribute and assign an operational state to the obstacle according to a determination of whether the obstacle exhibits the further predetermined attribute. The computing device of claim 12 .

18. The navigation constraints include a maximum movement speed of the mobile robot. The computing device of claim 12 .

19. The navigation constraints define the distance from the obstacles that should be maintained during navigation of the physical environment. The computing device of claim 12 .

20. The processor is further configured to determine whether the obstacle exhibits a predetermined attribute by determining whether a human is physically associated with the obstacle. The computing device of claim 12 .

21. The processor is further configured to determine whether the obstacle exhibits a predetermined attribute by determining whether a movable component of the obstacle is in a predetermined position relative to the obstacle. The computing device of claim 12 .

22. The processor is further configured to determine whether the obstacle exhibits a predetermined attribute by determining whether an illuminator of the obstacle is activated. The computing device of claim 12 .

Images