Storage system and robotic picking method

Abstract

A mobile manipulator robot (200) for retrieving inventory items from a storage system (100) includes a body (202), a wheel assembly, a sensor (262) for determining the robot's position within the storage system, an interface configured to transmit processor-readable data to a remote processor and operator interface and receive processor-executable instructions from the remote processor and operator interface, an imaging device for acquiring images of the inventory items, a picking manipulator (206), first and second pneumatic gripping elements for gripping the inventory items, and a coupler (222) configured to mate with a valve (150) for accessing a pneumatic supply for operating at least one of the first or second pneumatic gripping elements. The robot is configured to transition the valve from a closed state to an open state to selectively place one of the first or second pneumatic gripping elements in communication with the pneumatic supply.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. Patent Application No. 16 / 804,251, filed February 28, 2020, and claims the benefit of U.S. Provisional Patent Application No. 62 / 961,390, filed January 15, 2020, and U.S. Provisional Patent Application No. 62 / 879,843, filed July 29, 2019, the disclosures of which are each incorporated herein by reference.

[0002] The present disclosure relates generally to storage systems and inventory retrieval methods, and more particularly to storage systems and mobile manipulator robots for retrieving inventory items from storage systems. [Background technology]

[0003] Warehouses, or distribution fulfillment centers, require systems that allow for the efficient storage and retrieval of large numbers of diverse products. Traditionally, inventory items are stored in bins and arranged on rows of shelves on either side of aisles. Each bin or bin holds multiple items of one or more product types. The aisles provide access between shelves for an operator or robot to navigate the aisles and retrieve items. It is well understood that aisles reduce the storage density of a system. In other words, the amount of space actually used for product storage (e.g., on shelves) is relatively small compared to the amount of space required for the entire storage system. Because warehouse space is often scarce and expensive, alternative storage systems that maximize storage space are desired.

[0004] In one alternative approach that offers significant improvements in storage density, containers are stacked on top of each other and arranged in adjacent rows, i.e., there are no aisles between adjacent rows of stacked containers. Thus, more containers, and therefore more inventory, can be stored in a given space.

[0005] Various methods have been contemplated for retrieving inventory from stacked bins. For example, U.S. Patent No. 10,189,641 discloses a system in which bins are stacked and arranged in multiple rows beneath a grid. A vehicle equipped with a lifting device navigates the grid and lifts the desired bin. The bin is then transported down the port to a picking / sorting zone, where an operator or robot picks individual products from the bin and sorts them into one or more order bins. To minimize unnecessary transport of bins, each bin is typically transported to the picking / sorting zone only after receiving multiple orders for a particular product.

[0006] Despite the increased storage density offered by known stacked storage systems, various drawbacks remain. For example, order fulfillment times are often long, especially for infrequently ordered products, as containers are prioritized for retrieval as a function of the number of products of one type ordered. Furthermore, vehicles must navigate long distances while driving bins back and forth to transport ports (which takes a significant amount of time and consumes significant battery power). Furthermore, the required picking / sorting zones reduce the warehouse's overall storage density, further increasing complexity and cost. While the throughput of a stacked storage system can be increased by adding vehicles to the grid (or by modifying the system to add container transport ports), there are limitations to the number of vehicles that can operate on the grid after the grid becomes overly congested with vehicles, reducing the system's throughput due to gridlock. Summary of the Invention

[0007] According to a first aspect of the present disclosure, a high-density storage structure is provided. The storage structure includes a support member configured to accommodate a plurality of containers, a first set of parallel rails for supporting a mobile manipulator robot, and a fluid supply line having a plurality of valves disposed within the supply line. Each valve has a closed state in which the supply line is fluidly isolated from an external environment and an open state in which the supply line is in fluid communication with the external environment such that a mobile manipulator robot traversing the first set of parallel rails can receive a fluid supply from the fluid supply line.

[0008] According to another aspect of the present disclosure, a mobile manipulator robot for retrieving inventory from a storage structure is provided. The robot may include a body having an interface configured to transmit processor-readable data to a central processor and receive processor-executable instructions from the central processor, a mobile assembly coupled to the body, a coupler selectively mateable with a port to receive a fluid supply from a supply line, and a picking arm connected to the body. The picking arm may be coupled to a first pneumatic gripping tool configured to grip an inventory item.

[0009] According to yet another aspect of the present disclosure, there is provided a method for controlling a mobile manipulator robot to retrieve a product from a bin located within a storage structure, the method may include moving the mobile manipulator robot to a picking position on a first set of parallel rails of the storage structure, identifying a gripping area located on the product based at least in part on image data obtained by a sensor attached to the mobile manipulator robot, adjusting a picking arm equipped with a pneumatic gripping tool to a gripping orientation, and gripping the product using the pneumatic gripping tool. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic perspective view of a frame structure for accommodating a plurality of stacked containers according to the prior art; [Figure 2]FIG. 2 is a schematic plan view of a portion of the frame structure of FIG. 1. [Figure 3A] 3A and 3B are schematic perspective rear and front views, respectively, of a prior art load handling device for use with the frame structure shown in FIGS. 1 and 2; [Figure 3B] 3A and 3B are schematic perspective rear and front views, respectively, of a prior art load handling device for use with the frame structure shown in FIGS. 1 and 2; [Figure 3C] FIG. 4 is a schematic perspective view showing a container being lifted by the handling device of FIGS. 3A and 3B. [Figure 4] 3A-3C are schematic perspective views of the frame structure of FIG. 1 with a plurality of load handling devices installed on the frame structure of FIGS. [Figure 5] 5 is a schematic perspective view of the frame structure of FIG. 4 showing an excavation operation for retrieving a target vessel from a stack of vessels. [Figure 6A] FIG. 1 is a schematic diagram of a robotic system including a storage structure for accommodating multiple stacked containers, according to one embodiment of the present disclosure. [Figure 6B] FIG. 6B is a schematic perspective view of the storage structure of FIG. 6A. [Figure 6C] FIG. 1 is a schematic perspective view of two storage structures positioned on top of each other, according to one embodiment of the present disclosure. [Figure 6D] FIG. 6C is a schematic side view of an excavation robot performing an excavation operation within the storage structure of FIG. 6B. [Figure 7A] FIG. 1 is a perspective view of a rail showing a channel extending through the rail and a conduit extending from the channel to the surface of the rail. [Figure 7B] FIG. 7B is an enlarged view of a portion of the rail of FIG. 7A. [Figure 8A] 7B is a cross-sectional view of a valve disposed within the conduit of FIG. 7A. [Figure 8B] FIG. 8B is an enlarged view of the valve of FIG. 8A. [Figure 9A] FIG. 6C is a schematic perspective view of a mobile manipulator robot including a picking arm with a pneumatic gripping tool and a tool holder mounted on top of the storage structure of FIG. 6B. [Figure 9B] FIG. 9B is an enlarged view of a portion of the mobile manipulator robot of FIG. 9A. [Figure 9C] 9B is a flowchart illustrating an exemplary method for determining a gripping pose for a picking arm of the mobile manipulator robot of FIG. 9A. [Figure 9D] FIG. 1 is a schematic diagram of multiple product items disposed within a container. [Figure 9E] FIG. 9E is a schematic diagram showing the gripping area of ​​the product item of FIG. 9D. [Figure 9F] FIG. 9B is a perspective view of a first set of pneumatic gripping tools stored in the tool holder of FIG. 9A. [Figure 9G] FIG. 9B is a perspective view of a second set of pneumatic gripping tools stored in the tool holder of FIG. 9A. [Figure 10A] FIG. 9B is a top view showing the mobility assembly of the robot of FIG. 9A. [Figure 10B] FIG. 10B is a schematic diagram of a strut mechanism that assists in the rotation of the wheels of the mobility assembly of FIG. 10A. [Figure 11] FIG. 9B is a schematic cross-sectional view of a coupler of the robot of FIG. 9A. [Figure 12A] FIG. 9B is a perspective view of the picking arm of the robot of FIG. 9A. [Figure 12B] FIG. 12B is a side view of a portion of the picking arm of FIG. 12A. [Figure 12C] 1 is a schematic cross-sectional view of an order bin and target container holding inventory items of different sizes. [Figure 13A] 12A and 12B. FIG. 12B is a cross-sectional view showing the connection between the pneumatic gripping tool of FIG. 9A and the picking arm of FIGS. [Figure 13B] FIG. 13 is a schematic perspective view showing the coupling between the picking arm of FIGS. 12A and 12B and an alternative pneumatic gripping tool. [Figure 13C] FIG. 10 is a schematic diagram showing two pneumatic supply lines of the mobile manipulator robot of FIG. 9 that can be coupled to several example pneumatic tools. [Figure 14A] 12 is a schematic cross-sectional view illustrating the coupling between the coupler of FIG. 11 and the conduit of FIG. 7A. [Figure 14B] 12 is a schematic cross-sectional view illustrating the coupling between the coupler of FIG. 11 and the conduit of FIG. 7A. [Figure 15] 13B is a flowchart illustrating a method of gripping a product item using the picking arm and pneumatic gripping tool of FIG. 13A. [Figure 16A] FIG. 10 is a schematic perspective view of a mobile manipulator robot including a container retrieval device having a hoist plate according to another embodiment of the present disclosure. [Figure 16B] FIG. 16B is a schematic perspective view of the hoist plate of FIG. 16A. [Figure 16C] FIG. 10 is a schematic perspective view of a hoist plate including a plurality of suction cups according to another embodiment of the present disclosure. [Figure 16D] FIG. 10 is a schematic perspective view of a hoist plate including a retractable and movable picking arm according to yet another embodiment of the present disclosure. [Figure 16E] FIG. 10 is a schematic perspective view of a hoist plate including a retractable and movable picking arm according to yet another embodiment of the present disclosure. [Figure 16F] FIG. 1 is a schematic perspective view of two storage structures arranged side by side, showing a mobile manipulator robot traversing the side of the storage structures. [Figure 17] FIG. 16B is a schematic diagram of an alternative pneumatic system for use with the mobile manipulator robot of FIG. 9A or the mobile manipulator robot of FIG. 16A. [Figure 18] FIG. 18 is a cross-sectional view of a modified gripping tool for use with the alternative pneumatic system of FIG. 17. [Figure 19] FIG. 1 is a partial perspective view of a modified storage structure including a gantry frame supporting a robotic picking arm equipped with a pneumatic gripping tool. [Figure 20] FIG. 10 is a schematic diagram of another modified storage structure including an assembly positioned above the storage structure and a pneumatic supply line extending from the assembly toward the storage structure. [Figure 21]16B is a flowchart illustrating an exemplary method of controlling the movement of the mobile manipulator robot of FIG. 9A or the mobile manipulator robot of FIG. 16A using a computing system. [Figure 22] 16B is a flowchart illustrating an exemplary method of controlling the movement of the mobile manipulator robot of FIG. 9A or the mobile manipulator robot of FIG. 16A using a movement interface. [Figure 23] FIG. 1 is a schematic perspective view showing a mobile manipulator robot traversing a warehouse floor and picking inventory items from shelves. [Figure 24] FIG. 16B is a schematic perspective view of the mobile manipulator robot of FIG. 16A performing an excavating operation. [Figure 25] FIG. 16B is a schematic top view of the multiple mobile manipulator robot of FIG. 16A, including one or more container retrieval devices. [Figure 26] 1 is a flowchart illustrating an example of an order fulfillment process. [Figure 27] 1 is a flowchart illustrating an example of an order fulfillment process. DETAILED DESCRIPTION OF THE INVENTION

[0011] As used herein, for example, orientation terms such as "vertical" and "horizontal," or relative terms such as "up," "upward," "down," and "downward," when used to describe the orientation or relative position of particular features of a storage structure or mobile manipulator robot, refer to the orientation or relative position of the features within a normal gravitational reference frame when the storage structure is positioned with its bottom resting on a surface. Also, as used herein, the terms "substantially," "generally," and "about" are intended to mean that small deviations from absolute values ​​are included within the scope of the terms so modified.

[0012] 1 and 2 show a prior art storage structure for efficiently storing a plurality of stackable containers 10, also known as bins. The containers 10 are stacked on top of each other to form a stack 12 and arranged on a frame structure 14. Each container 10 typically holds multiple product items (not shown). The product items in each container 10 may be the same or may be different product types.

[0013] The frame structure 14 includes a plurality of vertical members 16 that support a first set of parallel horizontal members 18 extending in a first direction (e.g., the X direction) and a second set of parallel horizontal members 20 extending in a second direction (e.g., the Y direction). The horizontal members 18 and horizontal members 20 form a plurality of horizontal grid spaces in which the stacks 12 are housed. The frame structure 14 is thus constructed to prevent horizontal movement of the stacks 12 of bins 10 and to guide vertical movement of the bins.

[0014] The top level of the frame structure 14 includes rails 22 arranged in a grid pattern across the tops of the horizontal members 18 and 20. With further reference to FIGS. 3A-3C and 4, the rails 22 support a plurality of robotic load handling devices 30. A first set of parallel rails 22a guides movement of the load handling devices 30 in a first direction (e.g., the X direction) across the top of the frame structure 14, and a second set of parallel rails 22b, disposed perpendicular to the first set of parallel rails, guides movement of the load handling devices in a second direction (e.g., the Y direction) across the top of the frame structure. In this manner, the rails 22 enable the load handling devices 30 to move laterally in two directions (the X direction and the Y direction) across the top of the frame structure 14, such that the load handling devices can be moved to a position above any one of the stacks 12 of bins 10.

[0015] Each load handling device 30 includes a vehicle 32 with a first set of wheels 34, consisting of a pair of wheels at the front of the vehicle and a pair of wheels at the rear of the vehicle, positioned to engage two adjacent rails of the first set of parallel rails 22a. Similarly, a second set of wheels 36, consisting of a pair of wheels on each side of the vehicle, is positioned to engage two adjacent rails of the second set of parallel rails 22b. Each set of wheels 34, 36 can be raised or lowered so that either the first set of wheels 34 or the second set of wheels 36 engages each set of parallel rails 22a, 22b, depending on the desired direction of travel of the vehicle 32.

[0016] When the first set of wheels 34 engages the first set of parallel rails 22a and the second set of wheels 36 is lifted off the second set of parallel rails 22b, the first set of wheels can be driven by a drive mechanism (not shown) housed on the vehicle 32 to move the load handling device 30 in the X direction. To move the load handling device 30 in the Y direction, the first set of wheels 34 is lifted off the first set of parallel rails 22a and the second set of wheels 36 is lowered into engagement with the second set of parallel rails 22b. A drive mechanism (not shown) associated with the second set of wheels 36 can then be used to drive the second set of wheels in the Y direction.

[0017] The load handling device 30 also includes a crane device 40 having a cantilever arm 42 extending laterally from the top of the vehicle 32. A gripper plate 44 is suspended from the cantilever arm 42 by a cable 46 connected to a winding mechanism (not shown) housed within the vehicle 32. The cable 46 can therefore be wound onto or unwound from the cantilever arm 42 to adjust the gripper plate 44 in the Z direction relative to the vehicle 32.

[0018] Gripper plate 44 is adapted to engage the top of bin 10. For example, gripper plate 44 may include pins (not shown) that mate with corresponding holes (not shown) in a rim forming the top surface of bin 10, and sliding clips (not shown) engageable with the rim to grip the bin. The clips are driven into engagement with bin 10 by a suitable drive mechanism housed within gripper plate 44, and may be powered and controlled by signals carried via cable 46 or a separate control cable (not shown).

[0019] To remove a bin 10 from the top of the stack 12, the handling device 30 is moved in the X and Y directions as needed so that the gripper plate 44 is positioned above the stack where the desired bin is located. The gripper plate 44 is then lowered to engage the bin 10 at the top of the stack 12, as shown in FIG. 3C. After the clip engages and secures the bin 10, the gripper plate 44, and subsequently the bin, can then be pulled upward by winding the cable 46. At the peak of its vertical movement, the bin 10 is housed under the cantilever arm 42 and held on the rail 22. In this manner, the handling device 30 can transport the bin 10 to another location. The cable 46 is long enough to allow the handling device 30 to retrieve and place the bin 10 at any depth within the stack 12, including at floor level. The vehicle 32 is heavy enough to counteract the weight of the bin 10 and maintain stability during the lifting process. Much of the weight of the vehicle 32 is due to the large and heavy batteries required to power and operate the drive mechanisms for the wheels 34,36.

[0020] Known storage structures, such as that shown in FIG. 4, may include multiple material handling devices 30 operating simultaneously to increase system throughput. The storage structure shown in FIG. 4 includes two ports 24, or shafts, for moving bins 10 into and out of the storage structure. Additional conveyor systems (not shown) may be associated with each port 24. In this manner, bins 10 transported to a port 24 by a material handling device 30 are subsequently transferred to a picking / sorting station (not shown), where the products contained in the bins are picked and sorted into individual orders. Similarly, bins 10 may be moved by a conveyor system to a port 24 from an external location, such as a bin filling station (not shown), and transported by a material handling device 30 to a stack 12 to replenish the storage structure.

[0021] If a bin that is not located at the top of the stack 12 (a "target bin") needs to be retrieved, the bin 10a above (a "non-target bin") (e.g., a bin located between the target bin 10b and the rail 22) must first be moved to allow the material handling device 30 access to the target bin. This action is called "digging."

[0022] 5 illustrates a known excavation operation in which one of the handling devices 30 sequentially lifts each non-target bin 10a from a stack 12 of bins 10, including a target bin 10b. Each of the non-target bins 10a can be placed in a temporary location on top of another stack 12. After each of the non-target bins 10a is removed, the target bin 10b can be extracted from the frame 14 by the handling device 30 and transported to the port 24. After the target bin 10b is extracted, the non-target bin 10a can be returned to the original stack 12, restoring the original order of the stack minus the target bin.

[0023] Each of the material handling devices 30 can operate under the control of a central computer. Individual bins 10 in the system are tracked so that the appropriate bins can be retrieved, transported, and replaced as needed. For example, during an excavation operation, the temporary location of each of the non-target bins 10a is recorded so that the non-target bins can be replaced in the stack in a specific order.

[0024] The systems shown in Figures 1-5 allow for high-density product storage, requiring the transport of entire bins of product back and forth between the stack and the picking / sorting zone. During this time, products cannot be picked and sorted into new orders, reducing overall system throughput. To minimize bin transport, target bins 10b are typically retrieved and transported to the picking / sorting station only after multiple orders have been placed for a given type of product item. While this approach reduces bin transport, it often results in longer-than-desired order fulfillment times, especially when the order contains one or more products that consumers rarely order. For this reason, "piece-picking" inventory from known frame structures 14 has been attempted. For example, U.S. Patent Applications 2018 / 0319590 and 2018 / 0346243 disclose robots equipped with picking arms for picking individual items from bins arranged in a frame structure. Nevertheless, the picking robots and systems disclosed in U.S. Patent Nos. 2018 / 0319590 and 2018 / 0346243 are not robust enough to handle the picking of a wide variety of products.

[0025] On the other hand, the present disclosure provides a robot having a picking manipulator (sometimes referred to herein as a “picking arm”) that can be coupled to a gripping tool for grasping various products and placing the products into one of multiple order containers. Until now, a major obstacle to developing robotic picking arms has been the inability of picking arms to consistently grasp products of various sizes, shapes, weights, materials, surface textures, densities, mass distributions, stiffness, and fragility. While picking arms equipped with pneumatic gripping tools have been considered as one potential solution for grasping a wide variety of products, these gripping tools require a wide range of suction forces and flow rates that can only be generated by large vacuum pumps and / or compressors (e.g., small vacuum pumps / compressors can only provide adequate suction for a very narrow range of items). However, oversized pneumatic compressors and / or vacuum pumps are prohibitively large for a material handling device 30 or a similarly sized vehicle. In other words, the material handling device 30 cannot carry a large pneumatic compressor and / or vacuum pump within its vehicle body 32. Increasing the size of the body 32 so that the load handling device 30 can carry an oversized pneumatic compressor and / or vacuum pump requires resizing the footprint of the body to consume a large amount of grid space. As a result, fewer load handling devices can occupy the grid at one time, reducing system throughput. For this reason, robots with pneumatic gripping tools are typically confined to warehouse floors and are often fixed to fixed bases.

[0026] The present disclosure provides a robotic system including a storage structure with a pressurized air supply system and a compact mobile manipulator robot selectively coupleable to the pressurized air supply system, enabling the mobile manipulator robot to grasp inventory items with its pneumatic gripping tool. As a result, the robot can grasp a wide variety of products while traversing the storage structure and support large payloads during grasping. The mobile manipulator robot's ability to quickly and efficiently grasp a wide variety of inventory items is further enhanced by the robot's ability to quickly switch between two or more pneumatic gripping tools and request grasping assistance from a teleoperator when the robot is unable to grasp an item autonomously during edge-case scenarios (or when predicted control commands have high uncertainty or low reliability). Thus, the mobile manipulator robot can continue normal operation with minimal downtime or interruption. These improvements, among other advantages, are discussed in further detail in this disclosure.

[0027] FIG. 6A is a schematic diagram of a robotic system 100 according to one embodiment of the present disclosure. A robot, such as a mobile manipulator robot 200 (sometimes referred to herein as a “manipulator robot” or “robot”), can be housed in a storage system 101, such as a warehouse or other fulfillment center (hereinafter “warehouse”), and can perform the task of picking inventory items contained within a storage structure 114. The robot 200 can operate in one of two modes: an autonomous mode by executing autonomous control instructions, or a remote operation mode in which the control instructions are manually piloted (e.g., directly controlled) by an operator. While the term “control instructions” (whether autonomous or piloted) is described herein primarily as instructions for grasping an item, it should be understood that the term may also refer to various other robotic tasks, such as recognizing an inventory item, placing or releasing a grasped item (e.g., in a particular location or orientation), or other robotic tasks that facilitate order fulfillment. In one embodiment, the robot 200 can be a machine-learning robot capable of executing autonomous or piloted control instructions.

[0028] The robotic system 100 includes one or more operator interfaces 102, at least one of which may be located at a remote site outside the warehouse 101, one or more processor-based computer systems 103, each of which is communicatively coupled via one or more network or non-network communication channels 104 and one or more storage devices 105 that store, for example, machine learning grasp pose prediction algorithms used to predict grasp poses for the manipulator robot 200 to execute and grasp inventory items. While the storage devices 105 are shown as separate from the computer system 103, in at least some implementations, the storage devices may be integral parts or components of the computer system (e.g., memory such as RAM, ROM, flash, registers, hard disk drives, solid-state drives). As used herein, the term "remote processor" or "remote computer" refers to a processor that communicates with and is located remotely from the hardware of the referenced robot, and may include, for example, one or more processors or a single central processor for coordinating and automating fulfillment tasks between robots. Meanwhile, when the term "on-board" is used herein, this term means that a component is carried by the referenced robot. For example, "on-board processor" means that the processor is located within the hardware of the referenced robot. When the general terms "processor" or "computer" are used herein, the terms may refer to any remote processor, any on-board processor, or a combination thereof, unless otherwise specified.

[0029] The operator interface 102 includes one or more input devices and one or more output devices for receiving control commands from an operator. The one or more user interface devices 102 may be, for example, a personal computer, a tablet, a (smart) phone, a wearable computer, etc. Exemplary input devices include a keyboard, a mouse, a touchscreen display, a display (e.g., an LCD or OLED screen), a controller, a joystick, etc. In this regard, the teleoperator can input synchronous (real-time) or asynchronous (scheduled or cue) control commands, which may be, for example, click-point control commands, 3D mouse control commands, click-drag control commands, keyboard or arrow key control commands, and / or image-captured hand or body control commands. Exemplary output devices include, but are not limited to, a display (e.g., an LCD or OLED screen), a head-mounted display, a speaker, and / or a haptic feedback controller (e.g., a vibration element, a piezoelectric actuator, a rumble, kinesthetic, a rumble motor). Thus, operator interface 102 may be utilized by an operator to observe robotic picking, e.g., aspects of manipulator robot 200 and / or inventory stored within storage structure 114. The operator may view or see a representation of manipulator robot 200 performing one or more tasks, such as grasping an item, by reviewing one or more still and / or video images of manipulator robot 200 and / or its environment. These images and / or videos may be played and / or displayed in real time. If manipulator robot 200 fails to perform a task autonomously, the operator may utilize operator interface 102 to instruct the robot to grasp a product item and / or release the product item into a desired order receptacle.Although the operator interface 102 is primarily described herein in connection with assisting the robot 200 in performing grasping tasks, it will be understood that the interface can be used at any time (including before a grasping attempt fails) to enable a teleoperator to manually control the robot and perform operational tasks, including picking, rearranging, packing or repacking one or more items, picking up dropped items, manipulating items in inventory bins, or other order fulfillment tasks, including performing inventory audits, replenishment tasks, system checks, product identification, and / or overriding other autonomous control instructions.

[0030] The computer system 103 coordinates the operation of the robotic system 100. The computer system 103 may be a processor-based computer system. The processor may be any logic processing device, such as one or more microprocessors, central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), application specific integrated circuits (ASICs), programmable gate arrays (PGAs), programmed logic units (PLUs), etc. In some implementations, the computer system 103 may include a control subsystem that includes at least one processor.

[0031] Examples of suitable network or non-network communication channels 104 include wire-based network or non-network communication channels, optical-based network or non-network communication channels, wireless (i.e., radio and / or microwave frequency) network or non-network communication channels, or a combination of wired, optical, and / or wireless network or non-network communication channels.

[0032] Mobile manipulator robot 200 includes an interface for sending and / or receiving processor-readable data or processor-executable instructions to computer 103 via communication channel 104. In this way, computer 103 can predict a grasping pose (e.g., the position and / or orientation and / or posture of the robot's picking arm) and send control commands to manipulator robot 200 to execute the predicted grasping pose and grasp a product item. If a control command fails to perform a task (e.g., grasp an item) or if the remote computer determines that the predicted control command is unlikely to succeed, the system autonomously requests operator intervention, allowing robot 200 to be controlled remotely from a local or remote location.

[0033] As described in more detail below, the system allows a teleoperator to remotely control the manipulator robot 200 and move the robot into various grasping (or manipulation) poses to train the machine learning system to more accurately predict future autonomous robot control commands.

[0034] 6A shows two robots 200 located within a single warehouse, it will be understood that the system may include a single robot or any number of robots located within a single warehouse, or one or more robots located within multiple warehouses. Thus, the robotic system is advantageously configured to allow one or more operators to remotely pilot or control the multiple manipulator robots 200 via one or more operator interfaces 102 from a site local to the warehouse in which the robots are contained or remotely located.

[0035] A storage structure 114, such as that shown in FIG. 6B, is configured to efficiently store stackable containers 110, also referred to herein as bins. Each bin 110 is configured to hold multiple product items (not shown), which may be identical or of various product types. Examples of product types include household goods, apparel, appliances, beauty products, groceries, or other products that may be stored and shipped from a warehouse. Products can be arranged within the storage structure 114 in several ways to optimize picking / packing. For example, products can be arranged based on product type (e.g., similar products are grouped together), the rate at which products need to be filled, the environment in which the items need to be stored (e.g., temperature), the number of times a product is traditionally sold in a particular period of time, the size of the item, items that are commonly purchased together, etc.

[0036] In situations where a product requires specific storage conditions (e.g., temperature or humidity), such as food products, the container 110 can be packed with dry ice with a similar mechanism to adjust the storage conditions for the specific product type. Alternatively, the storage structure 114 can be constructed to include one or more separate, insulated refrigerated or frozen areas. Each refrigerated or frozen area can rely on cryogenic cooling to achieve the desired temperature, or can utilize a separate refrigeration system formed, for example, from a condenser, compressor, and evaporator configured to circulate gas through the system to refrigerate and / or freeze the insulated area. Product items, such as food products, can be stored in the container 110 and placed within the storage structure 114 in either the frozen area, the refrigerated area, and / or at room temperature based on the storage requirements of the product type. In some cases, these frozen / refrigerated areas can be located at lower levels of the storage structure 114. Food products can naturally be slotted near or far from the frozen and refrigerated areas based on the requirements of their individual temperatures and storage environments. This configuration also separates the robots located at the top of the storage structure 114 from the frozen / refrigerated areas. Nevertheless, if the robot or a portion thereof needs to access a frozen or refrigerated area, the robot may include heating components to regulate the temperature of its electronics and other systems.

[0037] The container 110 preferably has an open end through which products can be retrieved. The open end of the container 110 can be an open top or an open side. The bottom of the container 110 can have an inwardly tapered interior surface, which allows inventory products to easily roll and / or slide toward the center of the container and away from the sidewalls of the container, facilitating picking. In some cases, the bottom of the container 110 can include a sliding, pivoting, or bomb bay door to facilitate dumping of inventory items from the container to another container or other location. The bottom of the container 110 can also be designed to nest inside or against a rim forming the top surface of another container to prevent lateral movement of the containers relative to each other when stacked. Thus, the storage structure 114 need not include significantly fewer support members than some or corresponding frame structures 14. As a result, the storage structure 114 is less expensive to manufacture and can be installed more quickly than the frame structure 14.

[0038] Nevertheless, the storage structure 114 may include vertical members 116 supporting a first set of horizontal members 118 extending in a first direction (e.g., the X direction) and a second set of horizontal members 120 extending in a second direction (e.g., the Y direction). The horizontal members 118 and 120 form a plurality of horizontal spaces for accommodating the stacks 112. The horizontal spaces are constructed to prevent lateral movement of the stacks of bins 110. The storage structure 114 may further include one or more ports 121 or shafts for transferring bins into and out of the storage structure. A conveyor belt or shuttle system (not shown) may be associated with each port 121 to transport the bins 110 to an external location. For example, bins containing product for shipment may be transported down the port 121 to an external location for further packaging and / or shipping, while empty bins may be transported down the port to a bin filling station (not shown) for refilling and then up the port to one of the stacks 112 to refill the storage structure.

[0039] The top level of the storage structure 114 may include a first set of rails 122 extending in a first direction (e.g., the X-direction) and / or a second set of rails 124 extending in a second direction (e.g., the Y-direction). In embodiments in which the storage structure 114 includes the first set of rails 122 and the second set of rails 124, the combination of the first set of rails and the second set of rails forms a horizontally oriented grid 126 having a plurality of grid spaces 127. The rails 122, 124 enable one or more robots to move around the grid 126 above the stacks 112 of bins 110. At least one of the vertical member 116, the horizontal member 118, the horizontal member 120, or the rails 122, 124 may define a channel for transporting a fluid, such as compressed air, to a robot mounted on the grid 126, as discussed in further detail below.

[0040] As shown in FIG. 6C , multiple similarly constructed storage structures 114 with shallower stacks (e.g., fewer containers per stack) can be stacked on top of each other to reduce the time it takes to excavate a target container 110b (e.g., a container storing a desired product). This increases the throughput of the system. In such a scenario, each storage structure 114, or level, is spaced apart from adjacent levels with sufficient clearance between each level to allow one or more robots to move around their respective grids 126. One or more elevators and / or ramps with inclined and / or slanted rails 129 (in the Z direction) can be provided between the grids 126 of adjacent storage structures 114 to allow robots to move between levels as needed.

[0041] 6B , one or more sides of the storage structure 114 may additionally or alternatively include a second set of rails 124 extending in a second direction (e.g., the Y direction) and / or a third set of rails 125 extending in a third direction (e.g., the Z direction). In embodiments in which the storage structure 114 includes the second set of rails 124 and the third set of rails 125, the combination of the second set of rails 124 and the third set of rails 125 forms a vertically oriented grid 126 having a plurality of grid spaces 127. The manipulator robot 200 can traverse the vertical grid 126 to extract and pick bins 110 from the extracted bins stored on shelves, racks, or stacks on the side of the storage structure 114. When the term "grid" is used herein without an orientation modifier (e.g., vertical or horizontal), the term can refer to any grid structure formed by a combination of rails 122, 124, 125, regardless of whether the grid is horizontally or vertically oriented.

[0042] It is also envisioned (not shown) that multiple similarly constructed storage structures 114 may be positioned laterally adjacent to one another to increase storage capacity. In such a scenario, each storage structure 114 is positioned away from adjacent storage structures with sufficient space between adjacent storage structures to allow the robot 200 to traverse around each vertically oriented grid 126 and access the containers 110 housed in any of the adjacent storage structures.

[0043] As shown in FIGS. 7A and 7B , each of the rails 122, 124, 125 forming the grid 126 may be extruded or otherwise formed from a highly conductive metal, such as aluminum. A power source P may be coupled to the grid 126 to supply voltage to the rails 122, 124, 125, which in turn may selectively provide voltage to the robot 200 to recharge the robot's small batteries or super / ultracapacitors and / or directly power the robot's various drive mechanisms. Power may be transferred from the grid 126 to the robot 200 in one of several ways. For example, the grid 126 may have a single polarity, such as a negative charge, while a structure or ceiling (not shown) above the grid is positively charged (or vice versa). In this embodiment, the robot 200 may include an antenna 219 (shown in FIG. 9A ) that contacts the positively charged structure or ceiling above the grid 126 and completes a circuit between the opposite polarities. In another configuration, adjacent rails of at least one set of parallel rails 122, 124, and 125 can have opposite polarities such that conductive brushes (e.g., contact elements) of the robot 200 complete a circuit when the robot 200 is positioned on the adjacent parallel rails. For example, a first rail of the parallel rails 122 can have a positive polarity, while an adjacent rail of the parallel rails 122 can have a negative polarity. In this manner, the robot 200 need not include a large on-board battery associated with the load handling device 30. As a result, the robot 200 is less bulky and more maneuverable than a corresponding load handling device 30.

[0044] The rails 122, 124, 125 may include a double U-channel or profiled track having a top surface 128, an outer surface 130, an inner surface 132, and drive surfaces 136a, 136b (collectively "drive surfaces 136"). In this manner, two robots can traverse a single rail 122, 124, 125, increasing the number of robots that can operate on the grid 126 at any given time. For example, a first robot supported by drive surface 136a can pass a second robot supported by drive surface 136b. The top surfaces 128, outer surface 130, and inner surface 132 of the rails 122, 124, 125 may be anodized or painted with a non-conductive coating to prevent the robots or storage structure 114 from shorting out and minimize the risk of electrocution. In other words, the drive surfaces 136 of the rails 122, 124, 125 may be the only surfaces of the rails that remain at least partially or completely charged (except for the ends or small portions of the rail ends that are not anodized for the purpose of transmitting power along the rails of the grid).

[0045] The storage structure 114 further includes a fluid supply system 138 configured to supply a fluid, such as compressed air, to the robot 200 when the robot is mounted on the rails 122, 124, 125. The fluid supply system 138 thus eliminates the need for the robot 200 to carry a bulky on-board air compressor or vacuum generator to operate its pneumatic gripping tool 248 (FIG. 9A) and grip inventory items stored in the container 110. The fluid supply system 138 includes a fluid source S and a supply line 140. The fluid source S may be a compressor, such as a pneumatic compressor, for supplying compressed air to the supply line 140. Alternatively, the fluid source S may be a vacuum pump or vacuum generator.

[0046] Although the supply lines 140 are primarily described and illustrated herein as extending through the rails 122, 124, 125 of the grid 126, it will be understood that the supply lines may alternatively be formed by or extend at least partially through channels in the vertical members 116, horizontal members 118, or horizontal members 120, and may be attached to or otherwise coupled to the outer surface of at least one of the rails and frame structure or otherwise form the frame of the storage structure 114 in close proximity to the rails, so long as the fluid supply is accessible to the manipulator robot 200 when the robot is positioned on the grid.

[0047] As shown in FIG. 7A , the supply line 140 may include a series of channels 142, conduits 144, and ports 146. The channels 142 may extend along the entire length of the rails 122, 124, and 125, and preferably may be embedded within the lower portion of the u-channel so that the channels extend continuously along the length of each rail without interruption at the intersections of the rails 122 and 124 or the rails 124 and 125. Multiple conduits 144 may extend between the channels 142 and ports 146 located on the surface of each rail. In a preferred embodiment, at least one of the rails 122, 124, and 125 surrounding each grid space 127 has a conduit 144. Thus, the grid 126 can supply fluids, such as compressed air, to the robot 200 regardless of the robot's position on the grid.

[0048] 8A and 8B , multiple valves 150 may be disposed within supply line 140, for example, within conduit 144 of rails 122, 124, 125, or within a channel formed by vertical member 116, horizontal member 118, and / or horizontal member 120. Each valve 150 is transitionable between a closed state, in which compressed air is contained within supply line 140, and an open state, in which the supply line is in fluid communication with the environment so that compressed air may be supplied to manipulator robot 200. Each valve 150 may include a biasing member 152, such as a spring, and a plug 154 coupled to the spring to seal port 146. When spring 152 is in a neutral or unbiased state, spring 152 biases the plug against port 146, sealing off the compressed air within supply line 140. Alternative valves may be used to seal off the compressed air within supply line 140. For example, the valve may be constructed as any passively or actively actuated valve capable of transitioning between a closed state and an open state, such as an electrohydraulic servovalve.

[0049] 8B , the rails 122, 124, 125 of the grid 126 may define a cavity 143 aligned with the longitudinal axis of the conduit 144. The cavity 143 may include a tapered edge extending from the top surface 128 of the rails 122, 124, 125 toward the port 146. A magnet 157 or other ferrous material may surround the port 146 to magnetically couple the robot 200 to the grid 126 during transfer of compressed air from the supply line 140 to the robot 200. To seal the connection between the robot 200 and the grid 126, a gasket such as an O-ring 155 may be provided around the port 146 and / or anywhere else surrounding the valve 150 to prevent compressed air from leaking from the supply line 140. The compressed air of the supply system 138 can be selectively accessed by the mobile manipulator robot 200 to provide the necessary suction, allowing the manipulator robot to piece-pick inventory items ranging in size, shape, weight, material, surface texture, density, mass distribution, stiffness, and fragility.

[0050] 9A and 9B, manipulator robot 200 includes a car body 202, a mobility assembly 204 configured to guide movement of the car body along rails 122, 124, and 125, and a picking manipulator 206, also referred to herein as a "picking arm." Manipulator robot 200 also includes a communications interface for transmitting and receiving data between the manipulator robot and remote computer 103 and / or between the manipulator robot and operator interface 102. The data may include information obtained from positioning sensors, generally related to the position of the manipulator robot relative to storage structure 114 or warehouse 101, allowing remote computer 103 to control the robot's movement around grid 126 or warehouse. The position sensors may be a global positioning system, a local / indoor positioning system, a local feature positioning system, or a combination thereof. The global positioning system may be a GPS system. Local or indoor positioning systems are indoor positioning systems that use various technologies such as optical, radio, magnetic or acoustic signals to measure distance or time of flight to nearby anchor nodes (nodes with known fixed locations such as WiFi / LiFi access points, Bluetooth beacons or ultra-wideband beacons, magnetic positioning, or dead reckoning) to actively locate mobile devices or tags and provide surrounding location or environmental context.The local feature positioning system, on the other hand, may be any of a conductive, capacitive, infrared (IR) or other sensor used to detect features within the warehouse, such as a sensor that detects and counts the intersection of rails or grid spaces, a magnetic sensor designed to detect magnets or ferrous materials within the grid 126, an imager for reading barcodes or AR / QR codes on bins 110, rails or other structures (which can then be relayed to the remote processor 103 to determine the position of the mobile manipulator robot), an imager capable of performing simultaneous localization and mapping (SLAM), an encoder on the mobile assembly 204 to measure distance traveled, magnetic, NFC, RFID, or any one of other types of positioning sensors on some of the mobile robots and / or within the grid described herein, so long as the remote computer is able to determine the position of and control the position of the individual mobile robots. The data may also include data obtained from sensors associated with inventory (hereinafter "inventory data") (e.g., location, dimensions, shape, weight, material, porosity, surface texture, color, density, mass distribution, stiffness, fragility, etc.) to assist the computer or remote operator in distinguishing between different products within a container and / or predicting gripping poses for gripping product items.

[0051] The carbody 202 may be formed from four sidewalls 208a, 208b, 208c, 208d (collectively "sidewalls 208"), an open bottom end 210, and an open top end 212. The sidewalls 208 are preferably sized such that the carbody 202 has a footprint of a single grid space 127. In other words, when the robot 200 is positioned on the horizontal grid 126, two opposing sidewalls (e.g., 208a, 208c) are positioned on two adjacent rails 122 extending in the X direction, while the other two opposing sidewalls (e.g., 208b, 208d) are positioned on two adjacent rails 124 extending in the Y direction. In other embodiments, the carbody 202 of the robot 200 may have a footprint larger than a single grid space 127. The open bottom end 210 and open top end 212 of the vehicle body 202 allow the picking arm 206 to extend through the vehicle body to grasp product contained in a target bin 110b that may be located directly below the body (e.g., a bin located at the top of a stack of bins aligned with the vehicle body in the Z direction). Alternatively, the picking arm 206 can be used to pick product contained in a target bin located laterally adjacent to the vehicle body 202, as shown in FIG. 9A.

[0052] One or more of the side walls 208 of the car body 202 may optionally include a pivotable digging plate (not shown) for digging into the stacks 112 and pulling a target bin to the top of a particular stack and / or transporting the bin for replenishment purposes. The digging plate may be pivotable between a collapsed state, in which the digging plate is flush against a respective inner or outer surface of one of the side walls 208 of the car body 202, and an operational state, in which the digging plate extends vertically, radially away from the respective side wall of the car body. The digging plate may be similar to the gripper plate 44 of the material handling device 30 in that the digging plate is configured to be lowered in the Z direction to engage any of the bins 110 positioned in the stacks 112. Like the gripper plate 44, the digging plate may be adapted to pull the bin 110 upward by reeling in a cable of sufficient length to retrieve a target bin positioned at any depth within the stack 112. However, the robot 200 need not include a digging plate or another mechanism for digging a container from the stack 112. System 100 may instead rely on a combination of manipulator robot 200 and a separate robot specially adapted to perform excavation tasks. The excavation robot may be the known load handling device 30 or excavation robot 205 (FIGS. 6C and 6D).

[0053] With particular reference to FIG. 6D , the excavation robot 205 can include a body having a container receiving cavity and an excavator 207 extendable beneath the body. The excavator 207 can be a scissor lift or can include a series of telescoping beams or other compact linear actuators with a long stroke. In this manner, the excavator 207 can reach beneath the grid 126 to lift a single container 110, or multiple containers (e.g., a target bin 110b and each of the non-target bins 110a above the target bin), through the receiving cavity and onto the grid in a single lift. Alternatively, the excavator 207 can be located on a single exterior surface of the excavation robot 200 and include a latching device, such as a hook, for engaging one or more sides of the container 110. In this manner, the excavation robot 205 can reach under the grid 126 and lift a single container 110, or multiple containers (e.g., target bin 110b and non-target bins 110a above the target bins) above the grid and to the side of the excavation robot (e.g., without lifting the containers through the excavation robot's container-receiving cavity). The excavator 207 of the excavation robot 205 can be electrically, pneumatically, or otherwise actuated.

[0054] The interior surface of the sidewalls 208 of the robot 200 may also include latches, hooks, digging plates, or other mechanisms (not shown) for coupling order bins 210a, 214b (collectively "order bins 214") within the body 202 of the manipulator robot so that the combination of the piece-picking robot and one or more order bins has a footprint of approximately one grid space 127. Alternatively, latches, hooks, digging plates, or other mechanisms may be positioned on the exterior surface of one or more sidewalls 208 of the body 202 to couple one or more order bins 214 around the body, as shown in Figures 9A and 9B.

[0055] Each order bin 214 can accommodate one or more orders. If a single order bin accommodates multiple orders, the bin can be divided to separate the multiple orders within the single bin, or all items from the multiple orders can be mixed and undivided. For example, order bin 214a can accommodate an order from a first consumer, and order bin 214b can accommodate an order from a second consumer. Thus, after the robot 200 picks a product from target bin 110b, the product can be placed directly into the order bin corresponding to the order of the consumer who purchased the product. In one embodiment, the bottom end of the order bin 214 can include a sliding, pivoting, or bomb bay door to facilitate dumping of the items into another container, area, or downport 121 for further sorting or processing. However, it will be understood that the piece-picking robot 200 need not carry order bins 214. Instead, the piece-picking robot 200 can be used only to grasp the product, which can then be placed in an order bin 214 (not shown) to be carried by a "transport robot" (e.g., a robot performing the task of carrying the order bin). In this way, both the manipulator robot 200 and the transport robot can move along the grid 126 and make contact at specific picking or transfer locations.

[0056] 9B , the robot 200 further includes one or more sensors 262, such as an RGB or RGB-D camera, a video recorder, a light detection and ranging (LIDAR), or the like, directed to acquire photographs, point clouds, videos, or the like (generally referred to herein as “images” or “image(s)”) of the product items stored within the receptacle 110. While the sensor 262 is shown coupled to the picking arm 206, the sensor may instead be coupled to the body 202 or the grasping tool 248 of the robot 200 (as shown in FIG. 12B ). The images may be transmitted to the processor 103 via a network or non-network communication channel 104 and, in some cases, may be further relayed to the operator interface 102. In this manner, the processor 103 can implicitly or explicitly analyze the images and then execute machine learning algorithms located in the storage device 105 to predict a gripping pose to grasp a desired product item before transmitting a gripping pose control command to the robot 200 via the communication channel 104, which, when executed by the robot, causes the robot's picking arm 206 to approach and grasp the item. While a gripping pose may refer to a single pose, grasping an item often requires a set of poses executed in succession. As used herein, the term "grasping pose" may refer to a single pose or a set of poses executed in succession. Images are preferably acquired continuously as the robot 200 traverses the grid 126 and transmitted to the remote computer 103. In this manner, the remote computer 103 can determine the gripping pose of the picking arm 206 of the robot 200, or the picking arm of another manipulator robot, before the manipulator robot reaches the picking position, thus increasing the throughput of the robotic system 100.

[0057] 9C is a flowchart illustrating a method 400 for autonomously determining a gripping pose. The process for determining a gripping pose may begin at block 402 with a command from the processor 103 directing the sensor 262 to acquire an image of inventory located within the target container 110b.

[0058] The image may then be transmitted to the processor 103 via a network or non-network communication channel 104 in block 404. Upon receiving the image, the processor 103 may analyze the image and inventory data of the items stored in the target container 110b in block 406.

[0059] Based on the inventory data, the processor 103 may execute one or more grasp posture detection algorithms (which may be neural networks or machine learning algorithms stored in the storage device 105) to predict one or more candidate grasp postures at block 408. The processor 103 may then implement a policy utilizing one or more metrics, checks, and filters to select one or more of the predicted candidate grasp postures for the robot 200 to sequentially execute or add to its queue at block 410. The processor 103 then produces, creates, or generates a signal including processor-readable information representative of the selected grasp posture and transmits the signal to the robot 200 via the communication channel 104 at block 412. However, it will be appreciated that the robot 200 may not rely on remote computing and communications, but may instead execute some or all of the grasp model on an on-board computer.

[0060] As shown in Figures 9D and 9E, the sensor 262 and the gripping model can work in concert to identify a gripping region 414 for a product item, defined as a specific area on the product item or the entire product item's packaging that is likely to be successfully grasped by the manipulator robot 200. The gripping region 414 can be a relatively non-porous, flat surface area of ​​the product item and / or the product packaging if the gripping tool 248 utilizes suction, an antipodal surface if the gripping tool includes finger-like gripping elements, or an uneven surface or edge if the gripping tool is a universal jamming gripper, or other geometric characteristics that facilitate handling by a particular type of gripper capable of picking and handling items with specific geometric, material, and surface characteristics. Figure 9D illustrates different types of product items within a target container 110b. Figure 9E illustrates the identification of the gripping region 414 for a product item positioned within the region of the target container.

[0061] 10A , the mobile assembly 204 is configured to guide the movement of the car body 202 along the rails 122, 124, 125 and position the robot 200 above or laterally adjacent to the target bin 110b (e.g., the bin containing the product to be picked). The mobile assembly 204 may include a plurality of wheels 216, a motor 218, and one or more transmissions (belts or linkages) 220 operably coupling each wheel to the motor. The wheels 216 may be configured as smooth outer surfaces (e.g., cylindrical, disc, or spherical) or gears and may be formed of any material, such as rubber, metal, or plastic, so long as the wheels are capable of guiding the movement of the car body 202 and positioning the robot 200 along the rails 122, 124, 125.

[0062] Each of the wheels 216 may include a direct drive (not shown) or semi-direct drive (not shown) actuator in its hub with a magnetic encoder, a hub motor (not shown), and a gear-drive actuator (not shown) or belt-drive actuator (not shown) for rotating the wheel 216 and moving the carbody 202 along the rails 122, 124, 125 on which the wheel is disposed. The mobility assembly 204 may include four wheels 216, one wheel disposed at or adjacent each corner of the carbody 202. The orientation of the wheels 216 is controlled by a motor 218 and a transmission 220. More specifically, the transmission 220 couples the motor 218 directly or indirectly to each of the wheels 216 such that rotation of the motor simultaneously rotates / pivots the orientation of each of the wheels 216 between a first orientation in which each wheel is oriented, for example, along the rail 122, and a second orientation (e.g., 90 degrees) in which the wheel is aligned with the rail 124. Thus, the four wheels 216 can be used to guide the movement of the carbody 202 in two directions, e.g., along the rails 122 (e.g., the X direction) and along the rails 124 (e.g., the Y direction). The transmission 220 can also simultaneously pivot the wheels 216 less than or more than 90 degrees to orient the wheels and precisely control the movement of the robot 200 in any direction when the robot is not positioned on the grid 126. Thus, the robot 200 need not include a second set of wheels or a separate drive mechanism for lifting and releasing the second set of wheels each time the robot drives along a different rail, as is the case with the known load handling device 30. Nevertheless, it will be understood that the robot 200 may instead be constructed with two separate sets of wheels and drive mechanisms as described above with respect to the load handling device 30. In one embodiment, the wheels 216 may include magnets or electromagnets configured to act in concert with magnets or electromagnets in the rails 122, 124, 125 to slightly levitate and propel the robot along the rails.

[0063] 10B, the manipulator robot 200 may further include a support mechanism 237 connected to the body 202 and used to support the mobile assembly 204 from a drive surface. The support mechanism 237 includes one or more linear or rotary actuators 239 designed to move one or more stands 241 in the z-direction relative to the body 202 of the robot 200. The linear actuator 239 includes a housing 243 coupled to the body 202, preferably on the inner surface of one or more side walls 208, and a plunger 245 retractable toward and extendable away from the housing. The plunger 245 is connected to the stand 241, which may extend continuously or discontinuously around the inner surface of the side wall 208 adjacent the bottom end 210 of the body 202. As plunger 245 extends away from housing 243, stand 241 is moved downward and contacts a drive surface, such as the rails of grid 126, positioning the stand below grid 126 of wheels 216, which in turn transfers the load of manipulator robot 200 from the wheels to the stand. In this regard, because wheels 216 are suspended or floating above the drive surface, the orientation of the wheels can be quickly and easily pivoted by motor 218 and transmission 220, as described above. Plunger 245 can then be retracted to lift stand 241 off the drive surface, thereby re-engaging the wheels with the drive surface and allowing the manipulator robot to move.

[0064] The mobile assembly 204, or body 202, of the manipulator robot 200 may further include one or more electric brushes or conductive elements 221 (shown in FIG. 14B ) for engaging the inner drive surfaces 136 a, 136 b of the rails 122, 124, 125 and transferring charge from the rails to a relatively small on-board battery or super / ultracapacitor and then to the robot's drive motor or gear-driven actuator. As a result, the robot 200 can charge its battery or super / ultracapacitor while the robot traverses the grid 126. Thus, system throughput is increased because the robot 200 does not have to be removed from the grid 126 and / or paused to charge or replace its battery or super / ultracapacitor. The relatively small on-board battery or super / ultracapacitor also makes the robot 200 lighter, faster, and safer than a corresponding load handling device 30. Additionally, small batteries or super / ultracapacitors can temporarily power the drive motors and / or gear-driven actuators to drive the wheels 216 even when the robot 200 is removed from and driven from the grid 126. For example, the robot 200 can be driven in any direction across the warehouse floor to navigate the robot between the grids 126 and / or to other areas of the warehouse so that the robot can assist with other fulfillment tasks such as replenishment, picking / sorting inventory from shelves or containers (such as bins, totes, or other structures that hold inventory), e.g., at a picking / sorting station, and / or packing the picked / sorted inventory.

[0065] 11 , the robot 200 further includes a pneumatic coupler 222 adapted to receive a fluid, such as compressed air, from the fluid supply system 138. The coupler 222 is preferably extendable from a position within the sidewall 208 of the carbody 202 to a position outside the sidewall of the carbody in a manner that allows the coupler to selectively engage and disengage with the valve 150. When the coupler 222 is positioned within the carbody 202 of the robot 200, the coupler does not interfere with other structural features of other robots or the storage system positioned on the grid 126. The coupler 222 may be a generally hollow tube sized to be positioned within the cavity 143 of the rails 122, 124, 125. The mating end of the coupler 222 may be tapered and / or include a self-alignment or misalignment handling device to assist in positioning the coupler 222 within the cavity 143. The mating end of coupler 222 may also include an O-ring (not shown), a magnet 223 for magnetically engaging magnet 157 or ferrous material disposed about port 146, and a device 224 for transitioning valve 150 between a closed and an open state. Device 224 may be, for example, a mechanical member adapted to push plug 154 into conduit 144 (away from port 146) or any other device for electrically, magnetically, mechanically, or otherwise transitioning valve 150 or another valve between a closed and an open state. For example, a similarly constructed coupler may include one or more conductive pads for supplying power to and actuating an electrohydraulic servovalve.

[0066] The robot 200 may optionally carry a small air tank 266 (FIG. 9A) for storing compressed air. In some embodiments, the air tank 266 may be smaller than 20 cubic feet. The air tank 266 of the robot 200 is in selective communication with the coupler 222. In this manner, the robot 200 does not need to access the compressed air of the supply system 138 every time the robot desires to grasp a product item. The robot 200 instead relies on the compressed air stored in the air tank 266 to pick inventory items for a limited time and only needs to couple to the supply system 138 when the robot desires to replenish the air tank. As a result, the robot 200 can temporarily operate the picking arm 206 on the grid without coupling to the supply system 138 and / or when the robot is driven off the grid to assist with other grasping and sorting tasks.

[0067] 12A and 12B show an exemplary embodiment of a picking arm 206 coupled to a pneumatic gripping tool 248. The picking arm 206 is movable with several degrees of freedom to position the pneumatic gripping tool 248 relative to inventory stored anywhere within the receptacle 110 and has a long stroke (Z-direction) that enables the robot 200 to lift items of any size from the receptacle and place the items in the order bin 214. For example, the picking arm 206 may include at least six degrees of freedom. In one non-limiting example, the picking arm 206 includes three Cartesian degrees of freedom, a fourth degree of freedom in yaw, and fifth and sixth degrees of freedom for increasing linear stroke in the pitch and roll or z-direction. In the exemplary embodiment, the picking arm 206 may include a base member 226, one or more horizontal extensions 228, a vertical extension 230, and a positioning arm 232 configured to removably secure the pneumatic gripping tool 248. The positioning arm 232 may be a relatively thin tube having a smaller diameter than the gripping tool 248. This allows the positioning arm 232 to freely position the gripping tool 248 within the vessel 110 without interference from other items or partitions disposed within the vessel. The positioning arm 232 may be coupled to the vertical extension 230 via a coupling mechanism 233, which allows the positioning arm to move along a "first linear path," such as a track, that extends along the length of the vertical extension. One or more fluid lines 253 (FIG. 13B) are disposed within the positioning arm 232 to fluidly couple the gripping tool 248 and the coupler 222 (FIG. 11). If multiple fluid lines 253 are utilized, the fluid lines may be independent of one another.

[0068] The base member 226 is attached to the carbody 202 and can extend above the open top end 212 of the carbody. The base member 226 can include a "second linear path" 227, such as a track, extending along the length of the base member. The horizontal extension 228 can be coupled to the base member 226 in a manner that allows the horizontal member to move along the second linear path 227 to vertically position the vertical extension 230. The horizontal extension 228 is also rotationally coupled to the base member 226, the vertical extension 230 via joint 236, an actuator, and a motor (not shown), allowing the pneumatic gripping tool to be positioned with several degrees of freedom relative to the product item. In an exemplary embodiment, the actuator can have a magnetic encoder with a diametrically polarized magnet coupled to the motor rotor. The motor can be in the form of a brushless motor and have a diameter greater than its length. Alternatively, the picking arm 206 can be pneumatically or hydraulically actuated and utilize an actuatable valve to control a hydraulic or pneumatic rotary or linear actuator that controls the attitude of the pneumatic gripping tool 248. As will be further explained with reference to FIG. 12C, in a preferred embodiment, the combination of the first and second linear paths is greater than or equal to two times the height of the vessel 110, and preferably greater than or equal to three times the height of the vessel.

[0069] 12C is a schematic cross-sectional view showing a target container 110b holding a first relatively small item 238 (height) and a second relatively large item 240 (height) that is approximately the height of the container. The long-stroke picking arm 206 of the robot 200 can pick both item 238 and item 240 from the target container 110b (if the target container is located at the top of the stack 112, just below the grid 126) and place the items in the order bin 214. For example, when picking item 238 from the bottom of the target container 110b, the gripping tool 248 must first be lowered a distance equal to the height of the robot body (shown in FIG. 9A) and then lowered to approximately the height of the target container 110b (e.g., a distance equal to about twice the height of the target container). Thus, to pick item 238, horizontal extension 228 is moved to the bottom of first linear path 227 and positioning arm 232 is moved downward relative to vertical extension 230 and to the bottom of the second linear path, allowing gripping tool 248 to contact and grasp relatively small item 238. After item 238 is grasped, positioning arm 232 can be moved upward and along first linear path 227 and horizontal extension can move upward along second linear path 227, allowing the first item to be placed in order bin 214.

[0070] On the other hand, to grasp a relatively large second item 240 and place the second item in order bin 214, the grasping tool must be raised high enough to allow the bottom of the second item to pass the top of the order bin (e.g., grasping tool 248 must be positioned a distance approximately equal to the height of the container from the top of the order bin). Thus, after the relatively large second item 240 is grasped by picking arm 206, positioning arm 232 may be retracted upward relative to vertical member 230 along the first linear path, and the horizontal member may move toward the top of second linear path 227 to allow the bottom of second item 240 to pass the top of order bin 214. Thus, to grasp and place a relatively small item, such as item 238, and a relatively tall item, such as item 240, the stroke (Z direction) of picking arm 206 must be at least two times the height of the container, and preferably three times the height. Although the stroke length can be achieved in a single linear path, by dividing the stroke length into two or more linear paths, the picking arm 206 can be made more compact and have a smaller vertical profile.

[0071] It will be appreciated that further increasing the z-stroke of the picking arm 206 will enable the picking arm to reach below the grid 126 and pick an item from a target container 110b that is at the top of the stack but below the top level, in the same manner as above, as long as a non-target container 110a is not above the target container.

[0072] In a preferred embodiment, as shown in FIG. 12B , the picking arm 206 also includes a spring 257 (and / or a back-drivable actuator, or a force-controlled actuator such as a quasi-direct drive, direct drive, or the like, a series elastic actuator, or a geared actuator with torque sensing that exhibits active compliance and acts as a virtual spring) or an adaptive gripping tool 248 that can be used to provide passive or active compliance, sense collisions, and assist in performing manipulation tasks such as picking or high-density packaging. The spring 257 (and / or a back-drivable actuator or a force-controlled actuator) can be provided between the pneumatic gripping tool 248 and the positioning arm 232 and / or in the coupling mechanism 233 that couples the positioning arm 232 and the vertical extension 230. Thus, if the gripping tool 248 pushes against a product or infrastructure of the storage structure with too much force, the gripping tool or positioning arm 232 will recoil, preventing damage to the picking arm 206 and / or the product. Compliance can also better position the gripping tool 248 relative to the item being grasped.

[0073] It is understood that the picking arm 206 may be alternatively constructed and / or include fewer or additional components, so long as the pneumatic gripping tool is positionable with multiple degrees of freedom to grip inventory items stored within the target receptacle 110b. For example, the picking arm 206 may also include a load cell or force-torque sensor to measure the payload of the gripped item and / or sense the external force applied to the gripping tool. In this manner, the robot 200 can instantly determine and / or verify the identity of the gripped item to pick and tightly pack the inventory items.

[0074] As described above, the gripping tool 248 is in fluid communication with the coupler 222 and, therefore, in selective communication with the fluid source S. In embodiments in which the fluid source S is a pneumatic compressor providing compressed air, the robot 200 can include one or more air ejectors, air aspirators, Venturi pumps 244 ( FIG. 11 ) or similar devices (hereinafter “Venturi pumps”) to use the compressed air from the supply system 138 to generate a vacuum or suction force. Referring to FIG. 13C , an example pneumatic circuit is shown in which one or more “bypass valves” can be provided between the coupler 222 and the gripping tool 248. The bypass valves are controlled by one or more valve actuators and are movable between three states: a closed state, a first open state, and a second open state. In the closed state, the bypass valves prevent compressed air from passing to the gripping tool 248. In a first open state, the bypass valve allows compressed air to flow from the coupler 222 through the venturi pump 244 to the gripping tool 248. Thus, when the bypass valve is in the first open state, the valve allows compressed air to flow through the venturi pump 244, allowing the venturi pump to generate suction to actuate a gripping tool 248 that relies on suction for gripping. In a second open state, the bypass valve allows compressed air to flow from the coupler 222 to the gripping tool 248, but diverts the compressed air around the venturi pump 244. Thus, when the bypass valve is in the second open state, the compressed air bypasses the venturi pump, and the robot 200 utilizes the compressed air to actuate a pneumatic gripping tool 248, such as a clamp and / or one of the other tool elements described below. Compressed air may also be utilized to blow or sweep air away from the gripping tool 248 to rearrange inventory items within the container 110 and / or rearrange inventory items within the order bin 214 to facilitate packing. An additional valve ("variable valve"), such as a throttle regulator, may be provided upstream of the bypass valve (e.g., between the coupler 222 and the "bypass valve") to precisely regulate the flow of air to the bypass valve and, therefore, the gripping tool 248.The variable valves and bypass valves may be solenoid valves and may be selectively actuated by drivers or relays controlled by a processor.

[0075] Referring to FIG. 13A , the positioning arm 232 includes a magnet 246, such as a ring magnet 246, or other magnet configuration that magnetically couples a gripping tool 248 to the positioning arm. The gripping tool 248 can be any pneumatically operated tool for gripping an item. For example, the gripping tool 248 can be a suction cup having a sidewall 251 formed of a resilient material, such as rubber, with a bellows 250 and a groove 249 disposed on the bellows. Thus, the sidewall 251 of the gripping tool 248 is adapted to compress when the gripping tool engages an object. The gripping tool 248 can further include a lip 252 formed of a resilient material, which can also be rubber, such that the lip of the gripping tool is adapted to deform to the surface of the product it engages and form a seal. A ring magnet 254 can be provided on the gripping tool 248 to attract the magnet 246 of the positioning arm 232 and magnetically couple the gripping tool to the positioning arm. A gasket such as an O-ring 256 can be provided on the gripping tool 248 to seal the connection between the positioning arm 232 and the gripping tool. In some embodiments, the gripping tool 248 can further have a groove (not shown) that cooperates with a protruding feature (not shown) on the positioning arm 232 to prevent rotational and axial movement of the gripping tool relative to the positioning arm when the gripping tool is coupled to the positioning arm. In other embodiments, the gripping tool 248 can be coupled to the positioning arm 232 via a mechanical connection such as a push / pull connection, a snap-fit ​​connection, a hook-in-slot connection, a tab-in-slot connection, a twist / lock connection, or any combination of a male / female mechanical connection.

[0076] Turning now to FIG. 13B , the gripping tool 248 can include one or more elements 247 to assist in performing its gripping task. As used herein, the term “tool” refers to any device that is attached to or connectable to the picking arm 206 of the robot 200 and designed to perform a fulfillment task, such as gripping an item, packing an item, cutting a box, etc. In contrast, the term “element” refers to a particular aspect of the entire tool. For example, the gripping tool 248 can have one or more gripping elements, such as a suction cup and / or fingers, for gripping an item. In this example, the suction cup is an element, and each finger is an individual element that forms the entire tool. As shown in FIG. 13A , the entire tool 248 can be formed from a single element (e.g., a suction cup). It will be understood that in other examples, the tool 248 can include multiple elements, such as suction cups, fingers, and / or other elements, to complete a fulfillment task. As used herein, the term "gripping tool" means that the tool includes at least one gripping element designed primarily to grip an item, but does not exclude the tool from having additional elements designed primarily to complete other performance tasks. Additionally, the terms "pneumatic tool" and "pneumatic element" mean that the tool or element is pneumatically actuated.

[0077] The exemplary gripping tool 248 may include suction cups and / or clamps (not shown) with multiple pneumatically actuated fingers. The fingers can be used in combination with or separately from the suction cups to grip products. In some embodiments, the fingers themselves may include suction cups. In other embodiments, as shown in FIG. 13B, the gripping tool 248 may include multiple suction cups arranged on a single gripping tool. Multiple suction cups can be arranged in an array to grip large, heavy inventory items in several discrete locations, thereby providing a more stable grip than a single suction cup. Suction cups can also be arranged to grip multiple items at once. In further embodiments, other gripping elements may be utilized. By way of example, these gripping elements may include a universal jamming gripper, a foam vacuum gripper, pneumatically inflatable fingers, variable stiffness fingers, pressure-actuated fingers, pneumatically actuated linkage or piston-driven grippers with rigid or compliant fingers, or other pneumatically or vacuum-actuated (positive or negative pressure) gripper elements. The gripping tool 248 can also include conductive target pads and push pins on the gripping side of the gripping tool (or vice versa) to provide power and communication signals to the gripping tool's internal sensors and / or actuators, or to electrically supplement the pneumatic gripping. In other embodiments, the tool need not include a gripping element for gripping an item. Instead, the tool may include elements such as a knife, a pneumatic rotary cutting tool (shown in FIG. 9G), or another pneumatically actuated cutting tool for cutting open boxes. Tools can include pneumatically actuated elements, including, but not limited to, air caulk guns, spray guns, air chisels and punches, air cutoff tools, air drills, air files, air grinders and sanders, air guns, air hammers, air nailers, air nibblers and shears, air riveters, air routers, air scarifiers, air screwdrivers, air staplers, air tapping tools, air powered saws, and air powered ratchets and wrenches.

[0078] As shown in FIG. 13B , the positioning arm 232 of the picking arm 206 can include multiple individual fluid lines 253 that can be individually coupled to one or more elements 247 on a single tool 248. For example, if the picking arm 206 of the robot 200 is coupled to a gripping tool having a single element, such as a single suction cup, each of the fluid lines 253 can be in communication with the single suction cup. On the other hand, if the picking arm 206 of the robot 200 is coupled to a gripping tool having multiple elements, such as multiple suction cups, each fluid line 253 can be in communication with a respective suction cup, allowing the suction cups to be independently actuated. Each fluid line 253 can have a venturi pump 244, a bypass valve, and one or more variable valves associated with the fluid line 253 to control the suction or compressed air force, as described above with reference to FIG. 13C . As shown in FIGS. 13B and 13C , multiple fluid lines 253 can provide a pneumatic supply to each element on a multi-element tool for actuation or gripping purposes. For example, a variable stiffness finger may be in fluid communication with two fluid lines to independently control finger position and finger stiffness. Alternatively, multiple fluid lines 253 may be in fluid communication with a single element tool, such as a tool with a single suction cup, to provide a higher flow rate to the tool.

[0079] Any one of the above pneumatic elements, or a combination thereof, may be used to grasp one or more objects at a time, pack grasped objects, replace a robot's battery pack, actuate a bin's bomb bay door, lift and attach order bins to containers, cut and seal boxes, manipulate items in order bins, for example, by nudging, blowing, or knocking items over, or perform other tasks that facilitate order fulfillment.

[0080] 9A, 9F, and 9G, the carbody 202 of the robot 200 can include a tool holder 258 for holding multiple tools 248. The tool holder 258 can include multiple retainers 260a, 260b (collectively "retainers 260"), such as arcuate or rectangular notches for receiving grooves 249 of the gripping tool 248, or holding areas, such as cups for receiving the bottom or sidewall 251 of the gripping tool. In this manner, multiple different tools 248 (e.g., tools having different tool elements and / or multiples or configurations of tool elements, or suction cups having lips of different sizes, materials, shapes, configurations, or orientations) can be interchangeably coupled to the positioning arm 232 and tool holder 258 when not in use, allowing the picking arm 206 to select a particular gripping tool based on the size, shape, material, or weight of the product for which the robot is to perform a gripping task. In some embodiments, the tool holder 258 can be magnetic to help secure the tool 248. The tool holder 258 may alternatively or additionally include a mating member for securing the tool 248 within the retainer 260 via a snap-fit ​​connection. Each one of the tools 248 may include an RFID, AR tag, calibrated weight, QR code, or similar identifier that can be identified by a sensor or load cell of the picking arm 206. In this manner, the robot 200 can determine if a tool 248 is secured to the picking arm 206 and verify that the secured tool is the desired tool.

[0081] In other embodiments, tool holders 258 may be provided in dedicated "tool holder stations" on or adjacent to specific areas of grid 126, or other areas within warehouse 101. Thus, robot 200 can drive to a tool holder station to exchange individual tools, or to exchange one tool holder 258 for an entirely different tool holder with a different set of tools. In this way, robot 200 can exchange tool holders based on its next set of tasks, so that the robot does not have to carry each tool it may be instructed to use.

[0082] 9B , one or more sensors 264, such as a scanner, can be located on the body 202 or picking arm 206 of the robot 200 to scan the picked product and determine and / or verify which order bin 214 the picked product should be placed in. The scanner's scanning field can be expanded by placing mirrors on the interior surface of the sidewall 208. Sensor 264 or another sensor can be used to acquire images or data of the grasped product after it has been picked (and before it is placed in the order bin 214) to identify and / or determine the item's size and dimensions. This information can be transmitted to the remote computer 103, which can then instruct the manipulator robot 200 to place the grasped item in a specific location in one of the order bins 214 and / or in a specific orientation to facilitate high-density packing. In some embodiments, the body 202 may also include features such as shelves or ledges where a grasped item can be temporarily placed and then re-grasped from a different orientation, facilitating packing based on the characteristics of the grasped item and other items that subsequently need to be placed in that container. Alternatively, a gripping tool 248 having multiple gripping elements 247 with one or more gimbal degrees of freedom between the elements may be operated to adjust the gripping elements relative to one another to re-grasp a previously grasped item in a desired orientation. It will be appreciated that two or more gripping tools 248 provided on two or more robots 200 may also be used to grasp, re-grasp, pack, or accomplish any other manipulation task of one or more items (or containers).

[0083] Next, we will describe the use of robotic system 100 to piece-pick individual product items from a container 110. The robot 200 can use its picking arm 206 to grasp one or more order bins 214, which can be attached to its own body 202 or the body of another robot. Alternatively, the order bins 214 can be attached to the robot 200 by a drilling plate or another device on the robot, or external to the robot, or with the assistance of an operator. The robot 200 can then be autonomously positioned on the grid 126 and operated under the control of a remote computer 103, which continuously records the respective positions of the robot, the container 110, and the products contained therein. The remote computer 103 can be further designed to efficiently control the movement of the robot 200 and can use a series of safety checks on teleoperator instructions and autonomous commands to prevent the robots and robotic systems described herein from colliding with each other as they move through a warehouse.

[0084] Upon receiving one or more orders, the computer assigns the orders to one or more manipulator robots 200 based on each robot's current order quantity and the location of the products included in the orders. If products are placed under one or more non-target bins 110a, the robot 200, or a separate excavating robot 205 located nearby, can pull the target bin 110b to the top of the stack 112. For example, the excavating robot 205 can position itself on the stack 112 containing the target bin 110b. The excavating robot 205 can then extend its excavator 207 below the excavating robot between the vertical member 116 and the stack 112 (on one or both sides of the stack) to grasp each of the target bins 110b and each of the non-target bins 110a positioned between the target bin and the grid 126. Each of the grasped bins can then be lifted, for example, so that the non-target bins are lifted through the excavating robot's receiving cavity and the target bins are placed within the receiving cavity. The excavation robot 205 can then drive to a position on another stack 112 that is missing a single container and release the target bin 110b at the top of that stack so that the manipulator robot 200 can pick an item from the target bin. In releasing the target bin 110b, the excavation robot 205 can either release only the target bin (e.g., never release the non-target bins 110a), or release the target and non-target bins and stack the non-target bins on top of the target bins so that the bottom non-target bin is placed into the excavation robot's receiving cavity and the other non-target containers are stacked on top of the excavation robot's receiving cavity, again and again securing all the non-target bins and returning to the original stack, placing the non-target bins in the original stack in their original order minus the target bin.

[0085] Next, with target bin 110b at the top of stack 112, remote computer 103 autonomously directs assigned robot 200 to a first location on grid 126 located above or adjacent to the target bin. Mobility assembly 204 enables robot 200 to navigate rails 122, 124, 125 and move to a desired location on grid 126. Robot 200 then transitions valve 150 to its open state to receive pressurized air and pick up from target bin 110b.

[0086] 14A, coupler 222 is positioned within cavity 143 such that magnets 223 of the coupler engage magnets 157 surrounding port 146. Insertion of coupler 222 into cavity 143 may be aided by the tapered edges of the coupler and the cavity. In this way, if coupler 222 is slightly misaligned with respect to port 146, the coupler's tapered edges will slide over the cavity's tapered edges, guiding the coupler into proper alignment with the port.

[0087] Alignment can also be assisted by a magnetic connection between the magnet 223 of the coupler 222 and the magnet 157 or ferrous material surrounding the port 146. The magnetic connection also secures the coupler 222 within the cavity 143 and helps counter the upward force of compressed air that occurs when the device 224 compresses the plug 154 into the conduit 144 (away from the top surface 128 of the rails 122, 124, 125) while the valve 150 transitions to an open configuration, allowing air pressure to flow around the plug and into the coupler. If the valve is an electrohydraulic servovalve, the coupler can be engaged similarly to the valve, with the conductive target pad providing power to electrically transition the valve from a closed to an open state. Alternatively, the electrohydraulic valve can be transitioned by voltage received from the grid 126 upon receiving a signal from the robot 200 or remote computer 103.

[0088] With the fluid supply system 138 coupled to the robot 200, the robot can use compressed air immediately for gripping and / or store it in an air tank 266 for later use. In embodiments in which the pneumatic gripping tools 248 rely on suction to grip objects, one or more venturi pumps 244 can use the compressed air provided by the pressurized air source S to generate suction to operate the gripping tools 248.

[0089] Upon arrival at the desired grid space 127, the picking arm 206 and pneumatic gripping tool 248 may be immediately positioned in a gripping position as directed by the remote computer 103, as described above with reference to FIG. 9C, or as directed by a teleoperator.

[0090] A method for grasping a product item will now be described with further reference to FIG. 15 and flowchart 500. If the robot 200 has not predetermined a grasping posture before the robot 200 is in a picking position, the method begins at 502 with a command from the processor 103 instructing the sensor 262 to acquire an image of the inventory located in the target container 110b. After the manipulator robot 200 receives a selected grasping posture signal, the robot executes a signal at 504, causing the picking arm 206 to execute the selected gasping posture. That is, the grasping tool 248 approaches the product item as instructed by the processor 103 and contacts the grasping area 414 of the product item. After the grasping attempt, one of the sensors, such as a pressure sensor (shown in FIG. 13C), characterizes the grasp at 508 as either successful or unsuccessful. That is, if the picking arm 206 of the robot 200 is able to successfully grasp and remove the product item from the target receptacle 110b, the pressure sensor characterizes the grasp as successful and sends a successful grasp signal to the processor 103 via the communication channel 104. On the other hand, if the picking arm 206 of the robot 200 is unable to remove the item from the receptacle, or if the picking arm drops the item before the processor 103 instructs the robot 200 to release the item, the pressure sensor characterizes the grasp as unsuccessful and sends a failed grasp signal to the processor via the communication channel 104. When the grasp is characterized as unsuccessful, the processor 103 can either: (1) immediately signal the teleoperator interface 102 at 510a to request operator intervention, or (2) determine a new or modified grasp pose at 510b and attempt to autonomously pick up the product item based on the new or modified grasp pose. If the processor 103 chooses to autonomously determine the grasp pose, with reference to FIG. 9C, the above steps may be repeated until the grasp is characterized as successful at 512 or until operator intervention is requested at 510a.

[0091] FIG. 21 is a flowchart 2100 illustrating a high-level overview of an exemplary method for controlling the operation of a mobile piece-picking robot, such as piece-picking robot 200, by a computing system, such as computing system 103, and an operator interface, such as operator interface 102, when operator intervention is requested at 510a. As shown in block 2102, computing system 103 receives performance data from the robot. The performance data may include inventory data related to inventory items stored in bins within storage structure 114 for which the robot performs a manipulation or grasping task. The performance data may also include requests for assistance from the robot indicating that the robot needs assistance to manipulate or otherwise grasp an object, or other performance statistics received from the robot, such as grasp success rate, number of consecutive failed grasps, etc. Computing system 103 then sends a notification to operator interface 102, as shown in block 2104. The notification may be sent in response to receiving the performance data. The notification corresponds to the performance data and may include inventory data or other information related to the inventory items for which the robot performs a manipulation or grasping task. The computing system receives a control command, as shown in block 2106. The control command may be received from an operator interface and may include at least one of a partial pose of a gripping element of the robot, an identified manipulation or gripping area on the inventory item, or a selection of a gripping element to be used by the robot to manipulate and / or grip the inventory item. The computing system then forwards the control command to the robot to perform the manipulation or gripping of the inventory item, as shown in block 2108.

[0092] 22 is a flowchart 2200 illustrating a high-level overview of an exemplary method for controlling the operation of a mobile piece-picking robot, such as piece-picking robot 200, by a system, such as operator system 102. As shown in block 2202, the system outputs one or more images of an inventory item. The system receives control instructions for the robot based on the one or more images, as shown in block 2104. The system then forwards the control instructions to the manipulator robot, as shown in block 2106.

[0093] At a more detailed level, when the processor 103 signals for intervention, the signal may be sent directly or indirectly to the operator interface 102. In situations where the operator interface 102 is communicatively coupled to multiple manipulator robots 200, each of the robots may be indirectly coupled to the operator interface 102 through a "broker." The broker, which may be part of the processor 103 or a separate processor, performs the task of ordering help requests from each robot in the operator interface's queue. The broker may run an algorithm that determines a "need help score" to prioritize the queue, or it may directly connect teleoperators to specific robots based on the "need help score" generated by the robot. The algorithm may be based on several factors, including the number of pre-grasp failures, the time elapsed since the start of the task, the difficulty of the task, the level of accuracy required, the product / SKU to be operated on, the task to be performed (e.g., picking, packing, inventory audit, correcting other errors), etc.

[0094] When a help request signal is received by the operator interface 102, the operator can remotely control the picking arm 206 of the robot 200 and instruct the picking arm to execute a specified gripping pose to grasp the product item. Specifically, the operator can view the item on an output device (e.g., a display) of the operator interface 102 and directly control the picking arm 206 of the robot 200 to grasp the item in a gripping region 414 by manipulating an input device of the operator interface. In some cases, the operator can also prompt the picking arm 206 to grasp the product item in combination with an automated motion sequence calculated by the motion planner. In this manner, the operator can simply select a pixel on the image feed that represents the gripping region 414, while the processor 103 autonomously determines and instructs the robot 200 to execute the selected gripping pose as described above with reference to FIG. 9C.

[0095] The pressure or other sensor can then characterize the grasp as either successful or unsuccessful, as described above at 508. An operator can additionally or alternatively perform the same characterization. If the sensor (or operator) determines the grasp was successful, the grasp data (such as the grasp pose, grasp area, grasp tool, inventory data, and other sensor or robot information) used to grasp the product item can be saved for future use in storage device 105 at 514. Thus, robot 200 can learn to infer or predict new grasp poses to improve automation of the grasping process.

[0096] No single gripping tool can optimally handle a wide variety of inventory. Therefore, the robot 200 may autonomously decide to switch gripping tools or may be instructed by a teleoperator. The gripping tool 248 may be selected based on the type of task or product type (which may be determined by the remote computer through inventory tracking of the product type in each bin), analysis of image data, and / or the results of historical data related to successful picks of that product or similarly constructed products. More specifically, the remote computer 103 or operator may instruct the manipulator robot 200 to couple a particular gripping tool 248 to a picking arm 206 that can engage the item's gripping area 414 with minimal leakage between the gripping tool and the item's surface.

[0097] 9F and 13A, when the robot 200 or teleoperator determines that it is desirable to switch gripping tools 248, the robot moves the picking arm 206 to position the groove 249 and / or sidewall 251 of the gripping tool attached to the picking arm within one of the retainers 260 of the tool holder 258. The picking arm 206 can then be retracted or moved upward to decouple the magnet 246 of the picking arm from the magnet 254 of the gripping tool 248. The picking arm 206 can then be positioned over another gripping tool disposed within the tool holder 258 to magnetically couple the picking arm to the other gripping tool before moving the picking arm laterally to slide the coupled gripping tool out of its respective retainer 260. However, it will be appreciated that other mechanical mechanisms for exchanging gripping tools, such as push-pull or twist-lock connections, may also be utilized, and the picking arm 206 of the robot 200 may be operated in any manner that facilitates disconnection of a first gripping tool in the tool holder 258 and connection of a second gripping tool in the tool holder.

[0098] When the gripping tool 248 contacts a product item, the gripping tool's lip 252 deforms and conforms to the product's surface as suction is applied to grip the product. Furthermore, the compliance of the gripping tool 248 and / or picking arm 206 compensates for inaccuracies in the sensing system or gripping algorithm to position the gripping tool for a better gripping position upon contact with the product. Then, with the product gripped, the picking arm 206 lifts the target product from the container 110 and, optionally, places or vibrates / rotates the product in front of the scanner 264 to scan an identifier, such as a barcode or RFID, placed on the target product to verify that the correct product has been gripped and / or notifies the picking arm about the order bin 214 from which the product should be released. During this time, one of the sensors may further collect data related to the product's size and dimensions and transmit this information to the remote computer 103 via the communication channel 104.

[0099] In some cases, the remote computer 103 can autonomously direct, or a teleoperator can manually direct, the picking arm 206 to release or place the grasped item in a specific location and / or orientation within the order bin 214. The grasping tool 248 and / or other elements of the grasping tool can then be used to push, blow, or otherwise manipulate the product into a specific location or orientation within the bin 214. In this manner, subsequently picked items can be efficiently packed within the order bin 214, allowing smaller order bins to be utilized. This increases the total amount of order bins that can be transported by a single robot, which in turn increases system throughput. While this disclosure primarily describes a processor (remote or onboard) configured to implicitly or explicitly analyze images, predict grasp poses, and determine grasp areas or desired grasp elements / tools, it should be understood that the processor may be further configured to implicitly or explicitly analyze images of the order receptacle 214 to determine packing poses, desired packing areas within the order bin, or desired packing tools to facilitate high-density packing. Similar algorithms and analyses can be used to assist in the performance of other manipulation tasks. Finally, these images and / or teleoperator commands responsive to the images can be stored in storage device 105 as associated with a particular manipulation task for future use. Thus, robot 200 can learn to infer or predict how to perform a manipulation task (e.g., grasping or packing).

[0100] After the robot 200 sequentially picks each of the products corresponding to a particular order, the order bin 214 may be transported from the storage structure 114, for example, via shaft 121 and associated conveyor belt, for additional processing, sorting, packaging, and / or transportation. When the robot 200 is tasked with picking orders for multiple consumers at once, the robot 200 need not pick all of the products associated with a first consumer's order before beginning to pick an order for a second consumer. Indeed, the remote computer instructs the robot 200 to pick items based on the product's storage location in an order that facilitates tight packing of the items, regardless of the consumer who ordered the products.

[0101] 16A shows a mobile manipulator robot 600 (hereinafter sometimes referred to as a "manipulator robot" or "robot") according to another embodiment of the present disclosure. Manipulator robot 600 may include any of the functionality described above in connection with robot 200, as well as any of the additional functionality described below. Common functionality between manipulator robot 200 and manipulator robot 600 will not be described again in detail below. Instead, when such functionality is described in connection with manipulator robot 600, the functionality will simply be referenced by the corresponding 600-series numeral. For example, manipulator robot 600 includes a mobile assembly 604 and a picking arm 606, which may be configured as described above with respect to mobile assembly 202 and picking arm 206, respectively, in connection with manipulator robot 200.

[0102] The manipulator robot 600 includes a carbody 602 that can be formed from four sidewalls 608. The carbody 602 of the manipulator robot 600 can have an open or closed bottom end 610 and an open or closed top end 612. The sidewalls 608 are preferably sized so that the carbody 602 has a footprint of a single grid space 127. In other words, when the robot 600 is placed on a horizontal grid 126, two opposing sidewalls are positioned on two adjacent rails 122 extending in the X direction, while the other two opposing sidewalls are positioned on two adjacent rails 124 extending in the Y direction. In other embodiments, the carbody 602 of the robot 600 can have a footprint larger than a single grid space 127. For example, the carbody 602 of the robot 600 can have a footprint equal to a 1x2 grid space, a 2x2 grid space, or a 3x3 grid space. Hardware and other components can be stored within the carbody cavity. For example, a cavity in the body 602 may house a small air tank and / or a relatively small battery 667 or a super / ultra capacitor and / or a heating element.

[0103] The picking arm 606 of the robot 600 may be configured to engage and release the battery 667 from the conductive contacts 669. In this regard, when the battery 667 is low, the picking arm 606 can disconnect the battery from the conductive contacts 669 and place the battery in a charging station (not shown). The picking arm 606 can then grab a charged battery from the charging station (not shown) and bring the charged battery into contact with the conductive contacts 669 to transfer power from the charged battery or super / ultracapacitor to the robot's various drive mechanisms. Battery packs can also be "swapped" or replaced without the use of the picking arm 606. For example, a battery pack can be replaced by moving the robot 200 in a first direction to safely engage the battery with a "battery exchange port" (not shown), maneuvering the robot relative to the battery exchange port in a manner that disengages the battery from the conductive contacts and allows the robot to be driven away from the battery exchange port without the battery. In one example, after the battery 667 engages with the battery exchange port, the plunger of the support mechanism can be extended, which in turn lifts the body of the robot and disengages the battery from its conductive contacts 669. Alternatively, the battery pack can be replaced using a battery exchange mechanism onboard or external to the robot.

[0104] One or more container retrieval devices 668 may be permanently attached to or removably coupled to the vehicle body 602. In other words, the manipulator robot 600 can autonomously add or remove container retrieval devices as needed, and can carry zero to four container retrieval devices at any time (as shown in FIG. 25 ). In this regard, when the manipulator robot 600 includes multiple container retrieval devices, the container retrieval devices can be used to carry multiple order bins simultaneously, perform multiple digging operations simultaneously (e.g., including returning a previously extracted container to a stack), and / or perform any combination of the foregoing.

[0105] The container retrieval device 668 includes a pair of opposing support arms 670 attached to or coupleable to the vehicle body 602 and a hoist plate 672 designed to engage and secure a container 110. With further reference to FIG. 16B , the hoist plate 672 has an open side and defines an opening 674 extending through its upper and lower surfaces to allow the picking arm 606 access to the interior of a container secured by the hoist plate. The opening 674 is preferably slightly larger than the outer periphery of the container 110 to allow the hoist plate 672 to slide around the stack 112 of containers 110. In this regard, as the hoist plate 672 is lowered along the stack 112 of containers 110, the stack of containers autonomously aligns the hoist plate laterally relative to the containers.

[0106] The hoist plate 672 may be coupled to and suspended from the support arm 670 by a cable 676 connected to a take-up mechanism 678, such as a spool, hoist, or winch, of the container retrieval device 668. The cable 676 may be wound, unwound, or wound onto the support arm 670 to adjust the hoist plate 672 in the z-direction relative to the support arm. An encoder 680 may be coupled to the spool or take-up mechanism to measure the distance the hoist plate 672 moves in the z-direction. The spool or take-up mechanism may also include a torque sensor 682 for measuring the weight of the container 110 supported by the hoist plate 672 or for detecting when the container 110 is in contact with the stack 112. Alternatively, the cable 676 or support arm 670 may include a load cell, force sensor, strain gauge, or other sensor configured to detect the weight of the container. As a result, an inventory audit can be performed autonomously while the container is being lifted or held by hoist plate 672 to determine or verify the number of product items that have been removed from the container, or to determine when a container is low on a particular product type and needs to be refilled. Similarly, sensors can be used to ensure that manipulator robot 600 does not attempt to lift one or more containers with a total load greater than it can handle.

[0107] Hoist plate 672 is adapted to engage with the top surface and / or one or more sides of container 110 to grip the container. For example, hoist plate 672 may include slidable or pivotable hooks 686 engageable with the rim of container 110 and / or engagement features 689, such as openings formed in the rim or sides of the container (shown in FIG. 24 ). The hooks are driven into engagement with container 110 by a suitable drive mechanism housed within plate 672, which may be powered and controlled by signals carried via cable 676, a separate control cable (not shown), or wirelessly. In one example, cable 676 may be made from a conductive metal strip to transmit power and electrical signals between hoist plate 672 and support arm 670.

[0108] The container retrieval device 668 may further include a sensor 688, such as a camera, depth imager, or similar device, for aligning the hoist plate with the top of the container 110. The sensor may use markers, such as AR tags or barcodes, on the container 110, or may otherwise use features of the container itself to facilitate proper alignment. The sensor 688 is preferably located on the hoist plate 672, but may also be located on the support arm 670. In addition to facilitating alignment, the camera may continuously capture images of adjacent grid spaces and then retrieve inventory stored in adjacent storage containers as the manipulator robot 600 traverses the grid 126. These images may then be transmitted to the remote processor 103 via the network 104 to assist in inventory audits or to predict one of the manipulator robot's grasping poses before it reaches the target container 110b to increase system throughput. Additionally, the sensor 688 may be used to continuously track items in the order bins 214 supported by the hoist plate 672. In this manner, when inventory items related to multiple orders are contained within a single undivided order bin 214, the items can be continuously tracked so that the processor knows which items are associated with which orders, allowing the items to be later sorted into individual orders without having to scan each product item.

[0109] In embodiments in which the container retrieval device 668 is removably coupleable to the carbody 602 of the manipulator robot 600, the manipulator robot can autonomously (upon receiving a control command from the processor 103 or the operator interface 102) exchange one container retrieval device having a first hoist plate for another container retrieval device having a hoist plate of a different configuration. Each container retrieval device has its own set of motors, actuators, sensors, processor, circuitry, battery, and power system and can be coupled to the carbody 602 of the robot 600 using an electromechanical interface configured to transmit mechanical loads and electrical communications. For example, the container retrieval device 668 with hoist plate 672 shown in FIGS. 16A and 16B can be exchanged for the container retrieval device 690 with hoist plate 692 shown in FIG. 16C or the container retrieval device 694 with hoist plate 696 shown in FIGS. 16D and 16E. Hoist plate 692 includes an array of suction cups (instead of hooks 686 of hoist plate 672), each of which may be formed similarly to the suction cups shown and described with reference to FIG. 13C, to engage and lift boxes, cartons, etc.

[0110] Hoist plate 696 may be similar to hoist plate 668 and may further include one or more suction cups attached to the plate by extendable and retractable arms 698. The arms 698 may also be movable laterally around the hoist plate in the X and Y directions. It will be appreciated that when hoist plate 696 is lowered and arms 698 are extended (as shown in FIG. 16D ), the suction cups can be freely positioned with multiple degrees of freedom within a target container 110 b located at any depth within storage structure 114 to grasp and pick individual items from the target container, as long as no non-target container 110 a is located above the target container. The arms 698 may be retracted as shown in FIG. 16E (e.g., upward in the z direction) to avoid interference with hoist plate 696 engaging a container 110.

[0111] Any of the above hoist plates may be equipped with additional sensors (e.g., temperature sensors, thermal cameras, humidity sensors, etc.) to monitor storage conditions within the various sections of the storage structure 114 and ensure that the sections are properly adjusted based on the product type stored in that section.

[0112] The body 602 of the manipulator robot 600 can include a payload management system designed to transfer payloads from one or more container retrieval devices to the body chassis. The payload management system may rely on one or more actuators that are non-backdrivable or utilize mechanical brakes to allow the actuators to be de-powered while holding and transporting the bin. Alternatively, a separate non-backdrivable container engagement mechanism on the body of the robot can be used to engage the container while it is being held by the hoist mechanism.

[0113] Container retrieval device 668, or any of the other container retrieval devices mentioned herein, may also be replaced with a container retrieval device 768 including a pair of opposing arms 770 designed to directly engage and secure container 110 (shown in FIG. 16F). Of course, container retrieval device 768 may instead be permanently attached to the manipulator robot. Each arm 770 may extend laterally away from the body and may include a hook or latch 772 at its distal end, which may be rotatable or pivotable so that the hook surrounds and engages target container 110b. In this manner, arms 770 are designed to extend and retract horizontally to directly engage container 110b and extract the target container from a shelf located within the warehouse (e.g., the side of storage structure 114 or any other shelf located within the warehouse), as long as the container is located laterally adjacent to the body of the robot shown in FIG. 16F.

[0114] Because manipulator robot 600 is operated as described above with respect to robot 200, use of manipulator robot 600 will now be described only with respect to container retrieval device 668. To retrieve a target container 110b from stack 112, manipulator robot 600 moves around grid 126 to position container retrieval device 668 on the stack including target container 110b. Hoist plate 672 can then be lowered by rewind cable 676 so that each non-target container 110a (if any) passes through opening 674 until the hoist plate is positioned adjacent to a non-target container located exactly one level above target container 110b. Next, with hoist plate 672 at the appropriate height, hook 686 can be slid toward a non-target container located exactly one level above target container 110b, engaging engagement feature 689 or other feature on the non-target container to secure the container to the hoist plate.

[0115] Next, each of the non-target containers 110a positioned above the target container 110b can be lifted by the hoist plate 672 by winding the cable upward until the bottom non-target container (the container secured to the hoist plate) is positioned between the support arms 670. The manipulator robot 600 can then be driven to a position above any of the other stacks and release each of the non-target bins it is carrying. Because the hoist plate 672 is three-sided (e.g., has an open side), the manipulator robot 600 can release all of the non-target containers 110a it is carrying and move the released non-target containers, even if one or more containers are located above the grid 126. This would not be possible if the hoist plate 672 were completely enclosed. It will be understood that any of the hoist plates or excavation equipment described herein can have an open side similar to the hoist plate 672. After the manipulator robot 600 releases the stack of non-target containers 110a, the manipulator robot (or another robot) can retrieve the target container 110b from the stack. The manipulator robot 600 can either pick the item from the target container 110b directly into an order bin secured by another container retrieval device 668 held by the robot, or place the target bin 110b on top of another stack, before picking from the target bin and placing the grasped item into an order bin that the robot is carrying or is otherwise nearby.

[0116] Alternatively, the manipulator robot 600 can extract each of the target bin 110b and non-target bin 110a in a single lift. Single-lift extraction can be accomplished by lowering the hoist plate 672 around each of the non-target bins, securing the hoist plate's latches 686 to the target bins, and raising the hoist plate until the target bins are held between the support arms 670 (e.g., in the container-receiving cavities of the container retrieval device 668), and the non-target bins are positioned above the container retrieval device. The manipulator robot 600 can then position its container retrieval device 668 above another stack of containers missing exactly one container before lowering and releasing all of the containers, so that the target container 110b is positioned directly below the grid 126 (e.g., at the top level where it can be picked), and each of the non-target containers is positioned above the target container, stacked on the grid. Hoist plate 672 can then grasp each of the non-target containers 110a above and move them to any other stack, which again does not need to be a stack missing containers equal to or greater than the number of non-target containers secured by manipulator robot 600. In other words, manipulator robot 600 can optionally release each of the non-target containers simultaneously, even if one or more containers are positioned on the grid, because the open side of hoist plate 672 allows manipulator robot 600 to drive away from non-target containers 110a after being released to perform other tasks. If manipulator robot 600 has multiple container retrieval devices 668, the container retrieval devices can perform digging operations independently or simultaneously (as shown in FIG. 24).

[0117] In a variant embodiment, the manipulator robot can include any and all of the features of the manipulator robot 200 and the manipulator 600, but with details of its pneumatic system, as described below. The pneumatic system 300 of the variant robot is shown schematically in FIG. 17. In this variant, the robot does not rely on air pressure from a storage structure; instead, the robot can have a modified pneumatic system coupled to the robot's body. The pneumatic system can include a dual-layer vacuum with a first vacuum 302 and a second vacuum 304 selectively communicating with a single layer or single suction cup or a gripping tool, such as a modified gripping tool 348 (FIG. 18). The first vacuum 302 can be a vacuum with a high flow rate (capable of displacing a large amount of air per minute), while the second vacuum 304 can be a powerful vacuum generator capable of generating a larger pressure differential with atmospheric pressure (thereby increasing the payload or force that can be held by the suction cup). The pneumatic system 300 further includes two valves 306a, 306b (collectively “valves 306”), e.g., servo valves, which can be switched between an open state and a closed state to control communication between the first and second vacuums and the gripping tool 248 or modified gripping tool 348.

[0118] 18 , the modified gripping tool 348 includes a first suction cup 308 and a second suction cup 310, which may be concentrically arranged within the first suction cup. The first suction cup 308 and the second suction cup 310 are otherwise generally formed as described above with respect to the suction cups of the gripping tool 248 and therefore will not be described in detail again. The only difference is that the modified gripping tool 348 has dual suction cups, as opposed to the single suction cup of the gripping tool 248. A first vacuum 302, or high-flow vacuum, can be selectively communicated with the first suction cup 308, while a second vacuum 304, or high-pressure vacuum, can be selectively communicated with the second suction cup 310. It is emphasized that any of the battery exchange mechanisms described with respect to manipulator robot 200 and manipulator robot 600 may be incorporated into a transforming robot with pneumatic system 300, even the single or double layer vacuum systems (or on-board compressor systems) described above, to use significantly more batteries than manipulator robot 200 or manipulator robot 600, which access an air pressure supply from an external source.

[0119] The use of pneumatic system 300 will be described only with respect to the grasping task, as the transforming robot may be operated in other manners as described above with respect to robot 200 and / or robot 600. Prior to grasping a product, valve 306a may be transitioned to its open position to provide high-flow vacuum suction to first suction cup 308 as the lip of the first suction cup correspondingly deforms to conform to the surface of the target product. After the initial seal is initiated, valve 306b may be transitioned to its open state to enable a high-pressure vacuum line for vacuum 304. The high-pressure suction allows the picking arm to support a larger payload than would be possible with high-flow vacuum alone. In this manner, a more secure grasp may be provided. Of course, both valves 306a and 306b may be set to their open positions during the initial grasping of the target product and until the robot desires to release the target product. Alternatively, valves 306a and 306b may be switched back and forth and between open, closed, and partially closed states to achieve the desired grasp of the target product.

[0120] By utilizing two relatively small vacuum sources, a high flow vacuum and a high pressure vacuum, to respectively create an initial seal and firmly grip the product, the physical size of the vacuum is reduced, eliminating the need for dramatic modifications to the robot's body. In this manner, the transforming robot can be used to piece-pick products stored within the storage structure 114 or frame structure 14 discussed with respect to the prior art.

[0121] FIG. 19 is a perspective view of a modified robotic system 100′ configured to efficiently store multiple stacked containers 110′. The modified robotic system 100′ includes all of the above-described features of system 100 and additional features described below. For example, additional rails 122′, 124′, 125′ may extend above grid 126 (supporting manipulator robot 200, manipulator robot 600, and / or excavator robot 205) and, alone or in combination with additional support members, form a gantry frame. This supports one or more robot picking arms 206′ equipped with pneumatic gripping tools 248′ in a manner that allows the picking arms 206′ to move around the gantry frame and pick inventory pieces from the containers 110′. In this manner, compressed air can flow through rails 122′, 124′, 125′ and / or the additional rails to the pneumatic gripping tools 248′ of the picking arms 206′ to grip products from the containers 110′. Rails 122', 124', 125', or additional support members disposed above the grid, may also include one or more valves similar to valve 150, which are accessible to manipulator robot 200 or manipulator robot 600 (disposed on the grid) and allow the manipulator robot or manipulator robot 600 to selectively couple to a pneumatic supply system.

[0122] Alternatively, the robot's picking arm 206' can be fixed to a frame above the grid, and the excavator robot 205 or another bin-carrying robot can transport the target container 110' to the fixed picking arm, which can grab the desired item and place it in an order bin carried by a transporter robot. In this way, the container 110' does not have to be transported back and forth down the port from the picking / sorting station.

[0123] FIG. 20 is a perspective view of yet another alternative robotic system 100″ configured to efficiently store multiple stacked containers. Robotic system 100″ includes all of the above-described features of robotic system 100 as well as additional features described below. Manipulator robot 200, or another manipulator robot such as manipulator robot 600, may be positioned at a station on grid 126 (e.g., not moving), configured to move only within specific areas of the grid, or otherwise positioned or configured to move around the warehouse. These robots may be permanently or selectively coupled to supply lines 140″ that hang from a structure such as the ceiling of warehouse 101 or otherwise extend toward the surface of grid 126 or the warehouse floor to provide access to an air supply when the robot is positioned on the grid or off-grid within the warehouse, e.g., on the warehouse floor. In some embodiments, supply lines 140″ may be retracted via a drag chain cable carrier, cable reel retractor, or similar device, for example, to manage cable slack in the supply line. Supply line 140'' may further include a power cord or other mechanism to provide voltage to the manipulator robot when the robot is coupled to the supply line. A container handling robot, such as excavator robot 205 or robot 600, can transport containers 110 to manipulator robots located within specific areas of the grid before inventory is picked from the container and placed in other containers, such as order bins 214, as described above with respect to robotic system 100.

[0124] 23 , it will be understood that pressurized air may also be supplied to a manipulator robot, such as robot 200 or robot 600, when the manipulator robot is positioned away from grid 126 in other areas of the warehouse to facilitate other tasks, such as product manipulation and / or piece picking, and / or packing, unpacking, manufacturing, or production tasks. For example, a pressurized air supply line may be provided within the shelf and / or within the warehouse floor, and / or within a line or hose extending downward from structure above the robot toward the robot, as long as it is accessible to any coupler of the manipulator robot disclosed herein. In this way, when the robot is driven away from grid 126 to assist in other fulfillment tasks, such as picking inventory from shelves and / or packaging picked / sorted inventory at a picking / sorting station, the robot has access to an air pressure supply to operate its pneumatic tools. As a result, the same compact robot can be used in different areas of the warehouse to perform a variety of order fulfillment tasks, and can be driven in any direction across the warehouse floor without the need for large onboard air compressors or being permanently tethered to flexible supply lines that could become tangled with other supply lines during each robot's movements.

[0125] FIG. 26 is a flowchart 2600 illustrating steps in an exemplary order fulfillment process. As shown in block 2602, inventory arrives at a warehouse, such as warehouse 101, via a truck that may drive up to the warehouse's dock door. Containers are removed from the truck and placed on a conveyor in block 2604, either manually or using a robotic system in the truck. In block 2606, the conveyor or another operator moves the containers to an area within warehouse 101, and any of the robots described herein can use their container retrieval devices to pick the containers and transport them to a different desired area of ​​the warehouse, as represented by block 2608. To ship items from the warehouse, the reverse order is followed. In other words, the containers are transported by the robot's container retrieval device to a conveyor, which then transports them to a truck.

[0126] FIG. 27 is a flowchart 2700 illustrating steps in another exemplary order fulfillment process. As shown in block 2702, a truck delivers inventory to a warehouse, such as warehouse 101, arriving under a frame or “grid” structure similar to grid 126 described above. The truck may have a convertible top, a movable floor relative to the top, or a removable pallet, pod, or shipping container. In block 2704, the truck can remove its top, move its floor relative to the top, or otherwise expose its pallet, pod, or shipping container. Next, any of the robots having a container retrieval device described herein can drive the grid over the exposed container and, in block 2706, retrieve the container directly from the truck bed, pallet, pod, or shipping container. Once again, the reverse steps are performed to ship items from the warehouse. More specifically, the robot can deliver the container from the warehouse to the grid on the truck and then directly unload the container into the truck bed.

[0127] Summarizing the foregoing, a storage system for robotic picking includes a storage structure including a support member, a first set of parallel rails for supporting a mobile manipulator robot, a fluid supply line, and a plurality of valves disposed in the fluid supply line, each valve having a closed state in which the fluid supply line is isolated from an external environment and an open state in which the fluid supply line is in communication with the external environment such that the fluid supply line is configured to supply fluid to the mobile manipulator robot; and / or

[0128] The storage structure may further include a second set of parallel rails extending substantially perpendicular to the first set of parallel rails, the first set of parallel rails and the second set forming a grid having a plurality of grid spaces; and / or

[0129] If the grid is a first grid and the plurality of grid spaces is a first plurality of grid spaces, the storage structure may further include a third set of parallel rails and a fourth set of parallel rails extending substantially perpendicular to the third set of parallel rails, the third and fourth sets of parallel rails may form a second grid that is elevated above the first grid, the second grid having a plurality of second grid spaces. A ramp including a fifth set of rails may connect the first grid and the second grid, or an elevator may connect the first grid and the second grid. And / or

[0130] The storage structure may include a fluid source in fluid communication with the fluid supply line, which may be a pneumatic source; and / or

[0131] The fluid supply lines may be attached to the outer surfaces of the first set of parallel rails; and / or

[0132] The fluid supply line may include a channel embedded within and extending longitudinally of the first set of parallel rails, and a plurality of conduits may extend between the channel and respective ports disposed adjacent to the surfaces of the first set of parallel rails; and / or

[0133] At least one of the plurality of valves may be at least partially disposed within a respective one of the conduits; and / or

[0134] At least one of the plurality of valves may include a biasing member coupled to the plug, wherein the biasing member may bias the plug into the port when the valve is in a closed state; and / or

[0135] The storage system may further include a mobile manipulator robot for picking inventory items stored within the storage structure, the robot may include a body, a mobility assembly coupled to the body and configured to guide movement of the robot along the first set of parallel rails, a coupler sized and configured to mate with at least one of the plurality of valves and receive fluid from the fluid supply line, a picking arm with a pneumatic gripping tool for picking the inventory items; and / or

[0136] The first set of parallel rails may include a conductive metal configured to receive a voltage from a charged or grounded power source; and / or

[0137] Portions of the surface of the conductive metal may be anodized or otherwise coated to prevent the transmission of voltage through the coated surface.

[0138] A mobile manipulator robot for retrieving inventory is also provided, including a body having an interface configured to transmit processor-readable data to a central processor and receive processor-executable instructions from the central processor, a mobile assembly coupled to the body, a coupler mateable with a port for receiving a fluid supply from a fluid supply line, and a picking arm connected to the body, the picking arm coupled to a first pneumatic gripping tool configured to pick an inventory item; and / or

[0139] The robot may further include a tool holder attached to the body, the tool holder having a plurality of retainers; and / or

[0140] The robot may further include a second pneumatic gripping tool having a different size, configuration, or material than the first pneumatic gripping tool, and the first pneumatic gripping tool and the second pneumatic gripping tool may be interchangeably coupleable to the picking arm and receivable by a respective one of the plurality of retainers; and / or

[0141] The first pneumatic gripping tool may have additional tool elements; and / or

[0142] The robot may further include a venturi pump disposed downstream of the coupler and upstream of the first pneumatic gripping tool; and / or

[0143] The robot may further include a conductive contact configured to receive a voltage from the charged surface; and / or

[0144] The mobility assembly may include a plurality of wheels, a motor, and a transmission operatively coupling the motor to each of the plurality of wheels, the motor may be arranged to control the orientation of each wheel such that the wheels are simultaneously rotatable between a first orientation and a second orientation; and / or

[0145] The robot may further include sensors for collecting inventory data related to at least one of surface shape, surface texture, color, or porosity that may determine a gripping area of ​​the inventory item; and / or

[0146] The first pneumatic gripping tool may be a suction cup; and / or

[0147] The robot may further include an air tank coupled to the body, the air tank may be less than 20 cubic feet; and / or

[0148] The inventory may be stored in a container having a height, and the picking arm may have an end effector stroke in the vertical direction that is at least twice the height of the container; and / or

[0149] The picking arm may include a base member coupled to the body, a horizontal extension coupled to the base member, a vertical extension coupled to the horizontal extension, a positioning arm coupled to the vertical member and movable relative to the vertical extension, and at least one of a spring, a back-drivable actuator, a force-controlled actuator, or a compliant gripping element may be coupled to the positioning arm to provide passive or active compliance.

[0150] An order fulfillment system is also provided, the order fulfillment system including a mobile manipulator robot movable in two dimensions within a storage system for picking inventory items stored in bins or on shelves, the mobile manipulator robot including a body, a wheel assembly coupled to the body, the wheel assembly including a plurality of wheels and an actuator for moving the body within the storage system, a sensor for determining a position of the body relative to the storage system in which the body is disposed, an interface configured to wirelessly transmit processor-readable data to a remote processor and wirelessly receive processor-executable instructions from the remote processor, an imaging device for acquiring images of the inventory items, the imaging device configured to collect the inventory data, and a picking manipulator coupled to the body, the picking manipulator having at least three degrees of freedom; a first pneumatic gripping element and a second pneumatic gripping element coupleable to a picking manipulator, at least one of the first pneumatic gripping element or the second pneumatic gripping element being formed from a compliant material, and each of the first pneumatic gripping element and the second pneumatic gripping element being movable by a picking manipulator in a three-dimensional workspace to access and grip an inventory item stored in one of the bins or one of the shelves. a coupler having a mating end in fluid communication with at least one of the first pneumatic gripping element or the second pneumatic gripping element, the mating end of the coupler configured to selectively mate with the valve to access a pneumatic supply for operating at least one of the first pneumatic gripping element or the second pneumatic gripping element, the system configured to align the mating end of the coupler with the valve and transition the valve from a closed state to an open state to selectively place one of the first pneumatic gripping element or the second pneumatic gripping element in communication with the pneumatic supply; and / or

[0151] The first pneumatic gripping element and the second pneumatic gripping element may be on a picking manipulator; and / or

[0152] The picking manipulator may include a first pneumatic line in communication with the first gripping element and a second pneumatic line in communication with the second pneumatic gripping element, the second pneumatic line being separate from the first pneumatic line; and / or

[0153] The first pneumatic gripping element may be disposed on a first tool and the second pneumatic gripping element may be disposed on a second tool different from the first tool; and / or

[0154] The picking manipulator may include magnets for removably coupling the first tool and the second tool to the picking manipulator; and / or

[0155] The picking manipulator may be selectively and removably coupleable to the first tool and the second tool via a mechanical connection; and / or

[0156] The system may further include a tool holder with a first retainer and a second retainer for holding a first tool and a second tool, respectively, the first tool and the second tool being accessible to and interchangeably couplable to a picking manipulator of the robot; and / or

[0157] The tool holder can be coupled to the body of the mobile manipulator robot; and / or

[0158] At least one of the first pneumatic gripping element or the second pneumatic gripping element may be a suction cup; and / or

[0159] At least one of the first pneumatic gripping element or the second pneumatic gripping element may be a pneumatically actuated finger; and / or

[0160] The component coupling the picking manipulator or one of the first pneumatic gripping element or the second pneumatic gripping element to the picking manipulator may include a spring, a back-drivable actuator or a force-controlled actuator; and / or

[0161] The body may include a latching device configured to engage and move the container into a receiving cavity of the robot, the receiving cavity being open at the top; and / or

[0162] The processor-executable instructions may be control instructions including at least one of instructions to select a pneumatic gripping element for manipulating an inventory item, instructions to manipulate an inventory item, instructions to grip an inventory item, or instructions to pack an inventory item; and / or

[0163] The system may further include a teleoperator interface for generating control instructions; and / or

[0164] The wheel assembly may include a plurality of wheels configured to guide movement of the body along a first profiled track extending in a first direction; and / or

[0165] The plurality of wheels may be further configured to guide movement of the body along a second profiled track extending in a second direction substantially perpendicular to the first direction; and / or

[0166] The robot may further include a venturi pump disposed downstream of the coupler and upstream of at least one of the first pneumatic gripping element or the second pneumatic gripping element; and / or

[0167] The robot may further include a conductive contact for receiving a voltage from the charged surface; and / or

[0168] The system may further include one of the bin or shelf, the bin or shelf having a height, and the first pneumatic gripping element may be vertically movable by the mobile manipulator robot a distance of at least twice the height to grip the inventory item; and / or

[0169] The robot may be configured to retrieve inventory items stored in a grid-based storage system, the grid-based storage system including a frame arranged to accommodate a stack of vertical bins, and a grid arranged on the frame, the grid may include a first set of parallel rails extending in a first direction and a second set of parallel rails extending in a second direction substantially perpendicular to the first direction, such that the first set of parallel rails and the second set of parallel rails collectively define a grid space, the first set of parallel rails and the second set of parallel rails may have profiled tracks that guide movement of the wheel assemblies along the first set of parallel rails and the second set of parallel rails. a plurality of containers configured to perform the pneumatic gripping of the mobile manipulator robot, the plurality of containers being stackable on one another to form a plurality of vertical stacks, each vertical stack being positionable under a respective grid space; a pneumatic supply line coupleable to one of the frame or grid; an air pressure source in fluid communication with the pneumatic supply line; and a plurality of valves in fluid communication with the pneumatic supply line, each valve being capable of having a closed state in which the pneumatic supply line is isolated from an external environment and an open state in which the pneumatic supply line is in communication with the external environment and is configured to supply air to the mobile manipulator robot for operating at least one of the first pneumatic gripping element and the second pneumatic gripping element.

[0170] The robot may be configured to mechanically, magnetically, electrically, or wirelessly transition the valve from a closed state to an open state; and / or

[0171] The first set of parallel rails and the second set may include conductive metal configured to receive a voltage from a charged or grounded power source; and / or

[0172] The pneumatic supply line may include a channel embedded within and extending longitudinally of the first set of parallel rails, and a plurality of conduits may extend between the channel and respective ports disposed adjacent to the surfaces of the first set of parallel rails. Each of a plurality of valves may be at least partially disposed within a respective one of the conduits.

[0173] The channel may extend continuously for a distance of about two grid spaces or more; and / or

[0174] At least one of the plurality of valves may include a biasing member coupled to the plug, wherein the biasing member may bias the plug into the port when the valve is in a closed state; and / or

[0175] Each grid space may be defined by four rail sides and four corners disposed between adjacent rail sides, and at least one of the four rail sides or one of the four corners may include one of the plurality of valves; and / or

[0176] The air pressure supply lines may be attached to the outer surface of the frame or the outer surface of the grid; and / or

[0177] The pneumatic supply lines may be mounted on a surface that is spaced from the grid and accessible to the coupler.

[0178] An order fulfillment system is also provided, the order fulfillment system including a mobile manipulator robot movable within a warehouse to pick inventory items, the mobile manipulator robot including a body, a wheel assembly coupled to the body, the wheel assembly including a plurality of wheels and an actuator for moving the body within the warehouse, a sensor for determining a position of the body relative to the warehouse in which the body is located, an interface configured to wirelessly transmit processor-readable data to a remote processor and wirelessly receive processor-executable instructions from the remote processor, and an imaging device for acquiring images of the inventory items, the imaging device configured to collect inventory data. a picking manipulator coupled to the body, the picking manipulator having at least three degrees of freedom; a first pneumatic gripping tool formed from a compliant material and coupleable to the picking manipulator, the first pneumatic gripping tool being movable by the picking manipulator within a three-dimensional workspace to access and grasp inventory items; and a container retrieval device coupleable to the body and having a container receiving space, the container retrieval device including a hoist plate having an upper surface, a lower surface, and openings therethrough, the hoist plate being extendable vertically relative to the body to engage and move containers within the container receiving space.

[0179] The container retrieval device may include a set of latches or hooks for engaging the container, which may be slidable or pivotable relative to the hoist plate; and / or

[0180] The hoist plate may have an open side; and / or

[0181] The system may further include a second pneumatic gripping tool coupleable to the picking manipulator, the second pneumatic gripping tool being movable by the picking manipulator to access and grip the inventory item; and / or

[0182] The mobile manipulator robot may further include a tool holder with a first retainer and a second retainer for holding a respective one of the first pneumatic gripping tool and the second pneumatic gripping tool; and / or

[0183] The mobile manipulator robot may further include an on-board processor in communication with at least one of the imaging device or the picking manipulator, and the picking manipulator may be configured to interchangeably couple to the first pneumatic gripping tool and the second pneumatic gripping tool upon receiving processor-executable instructions from the remote processor or executing instructions from the on-board processor; and / or

[0184] The system may further include a storage structure and a plurality of containers for storing inventory items, the plurality of containers being arranged in a vertical stack within the storage structure; and / or

[0185] The system may further include a tool holder coupled to the storage structure, the tool holder having a plurality of retainers; and / or

[0186] The system may further include a grid disposed above the containers, the grid including a first set of parallel rails extending in a first direction and a second set of parallel rails extending in a second direction substantially perpendicular to the first direction, such that the first set of parallel rails and the second set collectively define a grid space. The first set of parallel rails and the second set of parallel rails may have profiled tracks for guiding movement of the wheel assembly along the first set of parallel rails and the second set of parallel rails, each vertical stack may be disposed below a respective grid space, and a container retrieval device may extend below the grid and within each grid space to engage and lift one of the plurality of containers in the receiving cavity. And / or

[0187] If one of the plurality of containers is a target container positioned below a plurality of non-target containers, the opening in the hoist plate can be sized to allow the hoist plate to slide around the perimeter of each of the non-target containers; and / or

[0188] The container retrieval device may be a first container retrieval device and may be disposed on a first side of the body for lifting a first container from the first stack. The mobile manipulator device may further include a second container retrieval device disposed on a second side of the body for lifting a second container from a second stack different from the first stack.

[0189] An order fulfillment system is also provided, the order fulfillment system comprising a storage structure in a warehouse including: a grid including a first set of parallel rails extending in a first direction and a second set of parallel rails extending in a second direction substantially perpendicular to the first direction, such that the first set of parallel rails and the second set collectively define a grid space; a stack of vertically arranged bins, the stack configured to be positioned below each of the grid spaces; and a mobile manipulator robot for picking an inventory item stored in one of the bins, the mobile manipulator robot including a body coupled to a wheel assembly, the wheel assembly including a plurality of wheels and an actuator for moving the body along the grid; a sensor for determining a position of the body relative to the grid on which the body is positioned; an interface configured to wirelessly transmit processor readable data to a remote processor and wirelessly receive processor executable instructions from the remote processor; and an imaging device for obtaining images of the inventory items. the imaging device is configured to collect inventory data; a picking manipulator coupled to the body, the picking manipulator having at least three degrees of freedom; and a first pneumatic gripping element and a second pneumatic gripping element coupleable to the picking manipulator, each of the first pneumatic gripping element and the second pneumatic gripping element being movable by the picking manipulator within a three-dimensional workspace for accessing an inventory item stored in one of the bins. and / or a first pneumatic gripping element and a second pneumatic gripping element, the first pneumatic gripping element including a first suction cup, conductive contacts for receiving power from an energy source, and an on-board processor in communication with at least one of the imaging device or the picking manipulator, the picking manipulator configured to engage one of the first pneumatic gripping element or the second pneumatic gripping element with an inventory item after receiving processor-executable instructions from the remote processor or executing instructions from the on-board processor.

[0190] The manipulator robot may further include an on-board compressor or vacuum for operating the first suction cup; and / or

[0191] The energy source may be the electrically charged surface of the grid; and / or

[0192] The energy source may be an on-board battery; and / or

[0193] The storage structure may further include a pneumatic supply line coupled to the storage structure and a plurality of valves in fluid communication with the pneumatic supply line, each valve being transitionable between a closed state and an open state, and the mobile manipulator robot may further include a coupler having a mating end in fluid communication with the suction cup, the mating end of the coupler being configured to selectively mate with one of the valves to access the pneumatic supply from the pneumatic supply line; and / or

[0194] The second pneumatic gripping element may include a second suction cup. The mobile manipulator robot may further include a first fluid line extending between the coupler and the first suction cup and a second fluid line extending between the coupler and the second suction cup. The air pressure or flow rate in the first fluid line may be independently controllable relative to the air pressure or flow rate in the second fluid line. And / or

[0195] The first suction cup and the second suction cup may be disposed on a single pneumatic gripping tool; and / or

[0196] The first suction cup may be disposed on a first pneumatic gripping tool and the second pneumatic gripping element may be disposed on a second pneumatic gripping tool that is different from the first pneumatic gripping tool; and / or

[0197] The robot may further include a first venturi pump in fluid communication with the first fluid line and a second venturi pump in fluid communication with the second fluid line; and / or

[0198] Also provided is a mobile manipulator robot for picking inventory items from bins stored in a storage structure having rails, the robot including: a body including an interface configured to send processor-readable data to a remote processor and receive processor-executable instructions from the remote processor; a wheel assembly coupled to the body, the wheel assembly including a plurality of wheels and an actuator for moving the body along the rails of the storage structure; an imaging sensor for acquiring images of the inventory items; a tool holder coupled to the body, the tool holder having a first retainer and a second retainer; a first tool positionable in the first retainer; and a second tool positionable in the second retainer; a picking arm connected to the body; and an on-board processor in communication with at least one of the wheel assembly, the imaging sensor, or the picking arm, the picking arm configured to interchangeably couple to the first tool and the second tool upon receiving instructions from one of the remote processor or the on-board processor.

[0199] The first tool may have a different size, configuration, or material than the second tool; and / or

[0200] The picking arm may include a positioning arm having a first pneumatic line and a second pneumatic line separate from the first pneumatic line; and / or

[0201] The first tool may include a single pneumatic gripping element; and / or

[0202] The first tool can be coupled to the positioning arm, and the first pneumatic line and the second pneumatic line can communicate with a single pneumatic gripping element; and / or

[0203] The second tool may include a pneumatic gripping tool having a first pneumatic gripping element and a second pneumatic gripping element; and / or

[0204] A second tool may be coupled to the positioning arm, a first pneumatic line may be in communication with the first pneumatic gripping element and separate from the second pneumatic gripping element, and a second pneumatic line may be in communication with the second pneumatic gripping element and separate from the first pneumatic gripping element; and / or

[0205] The robot may further include a first venturi pump in communication with the first pneumatic line and a second venturi pump in communication with the second pneumatic line; and / or

[0206] The tool holder may include at least one of a magnetic material or a compatible material for securing a first tool within the first retainer and a second tool within the second retainer; and / or

[0207] The picking arm may include magnets for removably coupling the first tool and the second tool to the picking arm; and / or

[0208] The picking arm may be selectively and removably coupleable to the first tool and the second tool via a non-magnetic push / pull or twist lock connection; and / or

[0209] The robot may further include a coupler mateable with the port to receive the fluid supply from the supply line.

[0210] Also provided is a mobile manipulator robot for picking inventory items from a bin having a height, the bins being stored under a grid including a first set of rails extending in a first direction and a second set of rails extending in a second direction perpendicular to the first set of rails, the robot including: a body including an interface configured to send processor-readable data to a remote processor and receive processor-executable instructions from the remote processor; a wheel assembly including a plurality of wheels coupled to the body and an actuator configured to move the body along the first and second sets of rails; an imaging sensor for acquiring images of the inventory items; and a picking arm connected to the body and selectively couplable to a gripping tool for picking the inventory items from the bin, the picking arm being movable such that the gripping tool has a vertical stroke that is at least twice the height of the bin; and / or

[0211] The gripping tool may be pneumatically actuated and formed of a compliant material; and / or

[0212] The stroke of the vertical gripping tool may be at least three times the height of the container; and / or

[0213] The picking arm may include a base member coupled to the body, a first extension coupled to the base member and movable along a first linear path relative to the base member, the first linear path having a vertical component and defining a first maximum vertical distance, and a positioning arm coupled to the first extension and movable along a second linear path relative to the first extension, the second linear path having a vertical component and defining a second maximum vertical distance; and / or

[0214] The combination of the first maximum vertical distance and the second maximum vertical distance may be equal to at least two times the height of the container; and / or

[0215] The robotic picking arm may further include a resilient element coupled to the positioning arm to provide passive compliance; and / or

[0216] The robot may further include at least one of a backdrivable actuator or a force-controlled actuator coupled to the positioning arm to provide active compliance; and / or

[0217] The gripping tool may include at least one of a suction cup, a foam vacuum gripper, a universal jamming gripper, or a plurality of pneumatically actuated fingers.

[0218] Also provided is a mobile manipulator robot for picking inventory items from bins stored in a storage structure having rails, the robot including: a body including an interface configured to send processor-readable data to a remote processor and receive processor-executable instructions from the remote processor; a wheel assembly coupled to the body, the wheel assembly including a plurality of wheels and an actuator for moving the body along the rails of the storage structure; an imaging sensor for acquiring images of the inventory items; a tool holder coupled to the body, the tool holder having a first retainer and a second retainer, a first tool positionable in the first retainer, and a second tool positionable in the second retainer; a picking arm connected to the body; and an on-board processor in communication with at least one of the wheel assembly, the imaging sensor, or the picking arm, the picking arm configured to interchangeably couple to the first tool and the second tool upon receiving instructions from one of the remote processor or the on-board processor.

[0219] The first tool may have a different size, configuration, or material than the second tool; and / or

[0220] The picking arm may include a positioning arm having a first pneumatic line and a second pneumatic line separate from the first pneumatic line; and / or

[0221] The first tool may include a single pneumatic gripping element; and / or

[0222] The first tool may be coupled to the positioning arm, and the first and second pneumatics may be in communication with a single pneumatic gripping element; and / or

[0223] The second tool may include a pneumatic gripping tool having a first pneumatic gripping element and a second pneumatic gripping element; and / or

[0224] A second tool may be coupled to the positioning arm, a first pneumatic line may be in communication with the first pneumatic gripping element and separate from the second pneumatic gripping element, and a second pneumatic line may be in communication with the second pneumatic gripping element and separate from the first pneumatic gripping element; and / or

[0225] The robot may further include a first venturi pump in communication with the first pneumatic line and a second venturi pump in communication with the second pneumatic line; and / or

[0226] The tool holder may further include at least one of a magnetic material or a compatible material for securing the first tool within the first retainer and the second tool within the second retainer; and / or

[0227] The picking arm may include magnets for removably coupling the first tool and the second tool to the picking arm; and / or

[0228] The picking arm may be selectively and removably coupleable to the first tool and the second tool via a non-magnetic push / pull or twist lock connection; and / or

[0229] The robot may further include a coupler mateable with the port to receive the fluid supply from the supply line.

[0230] Also provided is a mobile manipulator robot for picking inventory items from a bin having a height, the bins being stored under a grid including a first set of rails extending in a first direction and a second set of rails extending in a second direction perpendicular to the first set of rails, the robot including: a body including an interface configured to send processor-readable data to a remote processor and receive processor-executable instructions from the remote processor; a wheel assembly including a plurality of wheels coupled to the body and an actuator configured to move the body along the first and second sets of rails; an imaging sensor for acquiring images of the inventory items; and a picking arm connected to the body and selectively couplable to a gripping tool for picking the inventory items from the bin, the picking arm being movable such that the gripping tool has a vertical stroke that is at least twice the height of the bin; and / or

[0231] The gripping tool may be pneumatically actuated and formed of a compliant material; and / or

[0232] The stroke of the vertical gripping tool may be at least three times the height of the container; and / or

[0233] The picking arm may include a base member coupled to the body, a first extension coupled to the base member and movable along a first linear path relative to the base member, the first linear path having a vertical component and defining a first maximum vertical distance, and a positioning arm coupled to the first extension and movable along a second linear path relative to the first extension, the second linear path having a vertical component and defining a second maximum vertical distance; and / or

[0234] The combination of the first maximum vertical distance and the second maximum vertical distance may be equal to at least two times the height of the container; and / or

[0235] The robot may further include a resilient element coupled to the positioning arm to provide passive compliance; and / or

[0236] The robot may further include at least one of a backdrivable actuator or a force-controlled actuator coupled to the positioning arm to provide active compliance; and / or

[0237] The gripping tool may include at least one of a suction cup, a foam vacuum gripper, a universal jamming gripper, or a plurality of pneumatically actuated fingers.

[0238] Although the disclosure herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present disclosure. It is therefore to be understood that numerous modifications can be made to the exemplary embodiments and that other arrangements can be devised without departing from the spirit and scope of the present disclosure as defined by the appended claims. The technical concepts that can be understood from the above-described embodiments will be described below. [Aspect 1] A storage system for robotic picking, comprising: a storage structure configured to house a plurality of containers, the storage structure including a support member, a first set of parallel rails for supporting a mobile manipulator robot, and a fluid supply line; a plurality of valves disposed in the fluid supply line, each valve of the plurality of valves having a closed state in which the fluid supply line is isolated from an external environment and an open state in which the fluid supply line is in communication with the external environment, such that the fluid supply line is configured to supply fluid to the mobile manipulator robot; Including, the system. [Aspect 2] The system of aspect 1, wherein the storage structure further includes a second set of parallel rails extending substantially perpendicular to the first set of parallel rails, the first set of parallel rails and the second set of parallel rails forming a grid having a plurality of grid spaces. [Aspect 3] the grid is a first grid, the plurality of grid spaces is a first plurality of grid spaces; and a third set of parallel rails and a fourth set of parallel rails extending substantially perpendicular to the third set of parallel rails, the third set of parallel rails and the fourth set of parallel rails forming a second grid above the first grid, the second grid having a plurality of second grid spaces; a fifth set of rails connecting the first grid and the second grid, or a ramp including an elevator connecting the first grid and the second grid; 3. The system of claim 2, comprising: [Aspect 4] 10. The system of embodiment 1, further comprising a fluid source in fluid communication with the fluid supply line, the fluid source being a pneumatic source. [Aspect 5] 2. The system of embodiment 1, wherein the fluid supply lines are attached to an outer surface of the first set of parallel rails. [Aspect 6] The system of aspect 1, wherein the fluid supply line includes a channel embedded within the first set of parallel rails and extending longitudinally thereof, and a plurality of conduits extending between the channel and respective ports positioned adjacent to the surface of the first set of parallel rails. [Aspect 7] 7. The system of claim 6, wherein at least one of the plurality of valves is at least partially disposed within a respective one of the conduits. [Aspect 8] A system as described in aspect 7, wherein at least one of the plurality of valves includes a biasing member coupled to a plug, and when the valve is in the closed state, the biasing member biases the plug toward the port. [Aspect 9] said mobile manipulator robot for picking inventory items stored within said storage structure; The mobile manipulator robot comprises: The main body and a mobility assembly coupled to the body and configured to guide movement of the mobile manipulator robot along the first set of parallel rails; a coupler sized and configured to mate with at least one of the plurality of valves and receive fluid from the fluid supply line; a picking arm with a pneumatic gripping tool for picking inventory items; 7. The system of embodiment 6, comprising: [Aspect 10] 2. The system of embodiment 1, wherein the first set of parallel rails comprises a conductive metal configured to receive voltage from a charged or grounded power source. [Aspect 11] 11. The system of embodiment 10, wherein a portion of the surface of the conductive metal is anodized or otherwise coated to prevent transmission of the voltage through the coated surface. [Aspect 12] 1. A mobile manipulator robot for retrieving inventory, comprising: a body including an interface configured to transmit processor-readable data to a central processor and to receive processor-executable instructions from said central processor; a mobile assembly coupled to the body; a coupler mateable with the port for receiving a fluid supply from the fluid supply line; a picking arm connected to the body, the picking arm coupled to a first pneumatic gripping tool configured to pick an inventory item; A mobile manipulator robot, including: [Aspect 13] 13. The mobile manipulator robot of claim 12, further comprising a tool holder attached to the body, the tool holder having a plurality of retainers. [Aspect 14] A mobile manipulator robot as described in aspect 13, further comprising a second pneumatic gripping tool having a different size, configuration, or material than the first pneumatic gripping tool, each of the first pneumatic gripping tool and the second pneumatic gripping tool being interchangeably coupleable to the picking arm and receivable by a respective one of the plurality of retainers. [Aspect 15] A mobile manipulator robot as described in aspect 12, wherein the first pneumatic gripping tool has additional tool elements. [Aspect 16] 13. The mobile manipulator robot of embodiment 12, further comprising a venturi pump provided downstream of the coupler and upstream of the first pneumatic gripping tool. [Aspect 17] 13. The mobile manipulator robot of embodiment 12, further comprising a conductive contact for receiving a voltage from the charged surface. [Aspect 18] A mobile manipulator robot as described in aspect 12, wherein the mobile assembly includes a plurality of wheels, a motor, and a transmission operably coupling the motor to each of the plurality of wheels, the motor being arranged to control the orientation of each of the wheels, thereby allowing the wheels to simultaneously rotate between a first orientation and a second orientation. [Aspect 19] A mobile manipulator robot as described in aspect 12, further comprising a sensor for collecting inventory data related to at least one of surface shape, surface texture, color, or porosity that can determine a gripping area of ​​the inventory item. [Aspect 20] A mobile manipulator robot according to aspect 12, wherein the first pneumatic gripping tool is a suction cup. [Aspect 21] 13. The mobile manipulator robot of claim 12, further comprising an air tank coupled to the body, the air tank being less than 20 cubic feet. [Aspect 22] 13. The mobile manipulator robot of claim 12, wherein the inventory is stored in a container having a height, and the picking arm has an end effector stroke in a vertical direction that is at least twice the height of the container. [Aspect 23] The picking arm a base member coupled to the body; a horizontal extension coupled to the base member; a vertical extension coupled to the horizontal extension; a positioning arm coupled to the vertical member, the positioning arm being movable relative to the vertical extension; Including, 13. The mobile manipulator robot of embodiment 12, wherein at least one of a spring, a back-drivable actuator, a force-controlled actuator, or an adaptive gripping element is coupled to the positioning arm to provide passive or active compliance. [Aspect 24] 1. An order fulfillment system, comprising: a mobile manipulator robot movable in two dimensions within the storage system to pick inventory items stored in bins or on shelves; The mobile manipulator robot comprises: The main body and a wheel assembly coupled to the body, the wheel assembly including a plurality of wheels and an actuator for moving the body within the storage system; a sensor for determining the location of the body relative to the storage system in which the body is located; an interface configured to wirelessly transmit processor-readable data to a remote processor and wirelessly receive processor-executable instructions from the remote processor; an imaging device for capturing images of the inventory items, the imaging device configured to collect inventory data; a picking manipulator coupled to the body, the picking manipulator having at least three degrees of freedom; a first pneumatic gripping element and a second pneumatic gripping element coupleable to the picking manipulator, at least one of the first pneumatic gripping element or the second pneumatic gripping element being formed from a compliant material, each of the first pneumatic gripping element and the second pneumatic gripping element being movable by the picking manipulator within a three-dimensional workspace to access and grip the inventory items stored in one of the bins or one of the shelves; a coupler having a mating end in fluid communication with at least one of the first pneumatic gripping element or the second pneumatic gripping element, the mating end of the coupler configured to selectively mate with a valve to access a pneumatic supply for operating at least one of the first pneumatic gripping element or the second pneumatic gripping element; Including, an order fulfillment system configured to align the mating end of the coupler with the valve and transition the valve from a closed state to an open state to selectively place one of the first pneumatic gripping element or the second pneumatic gripping element in communication with the pneumatic supply. [Aspect 25] 25. The system of claim 24, wherein the first pneumatic gripping element and the second pneumatic gripping element are on the picking manipulator. [Aspect 26] 25. The system of claim 24, wherein the picking manipulator includes a first pneumatic line in communication with the first gripping element and a second pneumatic line in communication with the second pneumatic gripping element, the second pneumatic line being separated from the first pneumatic line. [Aspect 27] 25. The system of claim 24, wherein the first pneumatic gripping element is disposed on a first tool and the second pneumatic gripping element is disposed on a second tool different from the first tool. [Aspect 28] 28. The system of aspect 27, wherein the picking manipulator includes a magnet for removably coupling the first tool and the second tool to the picking manipulator. [Aspect 29] 28. The system of claim 27, wherein the picking manipulator is selectively and removably coupleable to the first tool and the second tool via a mechanical connection. [Aspect 30] The system of aspect 27 further includes a tool holder having a first retainer and a second retainer for holding the first tool and the second tool, respectively, wherein the first tool and the second tool are accessible to the picking manipulator of the mobile manipulator robot and are interchangeably coupleable to the picking manipulator. [Aspect 31] A system as described in aspect 30, wherein the tool holder is coupled to the body of the mobile manipulator robot. [Aspect 32] 25. The system of embodiment 24, wherein at least one of the first pneumatic gripping element or the second pneumatic gripping element is a suction cup. [Aspect 33] 25. The system of embodiment 24, wherein at least one of the first pneumatic gripping element or the second pneumatic gripping element is a pneumatically actuated finger. [Aspect 34] A system as described in aspect 24, wherein a component coupling one of the picking manipulator, the first pneumatic gripping element, or the second pneumatic gripping element to the picking manipulator includes a spring, a back-drivable actuator, or a force-controlled actuator. [Aspect 35] A system as described in aspect 24, wherein the body includes a latching device configured to engage and move a container into a receiving cavity of the mobile manipulator robot, the receiving cavity having an open top. [Aspect 36] 25. The system of claim 24, wherein the processor-executable instructions are control instructions including at least one of instructions for selecting a pneumatic gripping element for manipulating the inventory item, instructions for manipulating the inventory item, instructions for gripping the inventory item, or instructions for packaging the inventory item. [Aspect 37] 25. The system of embodiment 24, further comprising a teleoperator interface for generating the control instructions. [Aspect 38] Aspect 25. The system of aspect 24, wherein the wheel assembly includes a plurality of wheels configured to guide movement of the body along a first profiled track extending in a first direction. [Aspect 39] Aspect 39. The system of aspect 38, wherein the plurality of wheels are further configured to guide movement of the body along a second profiled track extending in a second direction substantially perpendicular to the first direction. [Aspect 40] 25. The system of aspect 24, wherein the mobile manipulator robot further includes a venturi pump positioned downstream of the coupler and upstream of at least one of the first pneumatic gripping element or the second pneumatic gripping element. [Aspect 41] 25. The system of embodiment 24, wherein the mobile manipulator robot further comprises a conductive contact for receiving a voltage from a charged surface. [Aspect 42] The system of aspect 24 further includes one of a container or a shelf, wherein the container or the shelf has a height, and the first pneumatic gripping element is vertically movable by the mobile manipulator robot a distance of at least twice the height to grip the inventory item. [Aspect 43] the mobile manipulator robot is configured to retrieve inventory items stored within a grid-based storage system; The grid-based storage system comprises: a frame arranged to accommodate a stack of vertical containers; a grid disposed on the frame, the grid including a first set of parallel rails extending in a first direction and a second set of parallel rails extending in a second direction substantially perpendicular to the first direction, such that the first set of parallel rails and the second set of parallel rails collectively define a grid space, the first set of parallel rails and the second set of parallel rails having profiled tracks for guiding movement of the wheel assembly along the first set of parallel rails and the second set of parallel rails; a plurality of containers configured to store the inventory items, the plurality of containers being stackable on one another to form a plurality of vertical stacks, each vertical stack being positionable under a respective grid space; an air pressure supply line coupled to one of the frame or the grid; an air pressure source in fluid communication with the air pressure supply line; a plurality of valves in fluid communication with the pneumatic supply line, each valve of the plurality of valves having a closed state in which the pneumatic supply line is isolated from an external environment and an open state in which the pneumatic supply line is in communication with the external environment and is configured to supply air to the mobile manipulator robot for operating at least one of the first pneumatic gripping element and the second pneumatic gripping element; 25. The system of claim 24, comprising: [Aspect 44] A system as described in aspect 43, wherein the mobile manipulator robot is configured to mechanically, magnetically, electrically, or wirelessly transition the plurality of valves from the closed state to the open state. [Aspect 45] A system as described in aspect 43, wherein the first set of parallel rails and the second set of parallel rails comprise conductive metal configured to receive voltage from a charged or grounded power source. [Aspect 46] A system as described in aspect 43, wherein the pneumatic supply line includes a channel embedded within and extending longitudinally of the first set of parallel rails, and a plurality of conduits extending between the channel and respective ports positioned adjacent to the surface of the first set of parallel rails, and each of the plurality of valves is at least partially positioned within a respective one of the conduits. [Aspect 47] 44. The system of embodiment 43, wherein the channel extends continuously for a distance of about two grid spaces or more. [Aspect 48] Aspect 44. The system of aspect 43, wherein at least one of the plurality of valves includes a biasing member coupled to a plug, and when the valve is in the closed state, the biasing member biases the plug toward the port. [Aspect 49] A system as described in aspect 43, wherein each grid space is defined by four rail sides and four corners disposed between adjacent rail sides, and at least one of the four rail sides or one of the four corners includes one of the plurality of valves. [Aspect 50] A system as described in aspect 43, wherein the air pressure supply line is attached to the outer surface of the frame or the outer surface of the grid. [Aspect 51] A system as described in aspect 43, wherein the air pressure supply line is spaced from the grid and attached to a surface accessible to the coupler. [Aspect 52] 1. An order fulfillment system, comprising: a mobile manipulator robot movable within a warehouse to pick inventory items; The mobile manipulator robot comprises: The main body and a wheel assembly coupled to the body, the wheel assembly including a plurality of wheels and an actuator for moving the body within the warehouse; a sensor for determining the location of the body relative to the warehouse in which the body is located; an interface configured to wirelessly transmit processor-readable data to a remote processor and wirelessly receive processor-executable instructions from the remote processor; an imaging device for acquiring images of the inventory items, the imaging device configured to collect inventory data; a picking manipulator coupled to the body, the picking manipulator having at least three degrees of freedom; a first pneumatic gripping tool formed from a compliant material and coupleable to the picking manipulator, the first pneumatic gripping tool being movable by the picking manipulator within the three-dimensional workspace to access and grasp inventory items; and a container retrieval device coupleable to the body and having a container receiving space, the container retrieval device including a hoist plate having an upper surface, a lower surface, and an opening extending through the upper surface and the lower surface, the hoist plate being extendable in a vertical direction relative to the body and adapted to engage and move a container within the container receiving space; an order fulfillment system, including: [Aspect 53] An order fulfillment system as described in aspect 52, wherein the container retrieval device includes a set of latches or hooks for engaging with the container, the latches or hooks being slidable or pivotable relative to the hoist plate. [Aspect 54] 53. The order fulfillment system of claim 52, wherein the hoist plate has an open side. [Aspect 55] 53. The order fulfillment system of claim 52, further comprising a second pneumatic gripping tool coupleable to the picking manipulator, the second pneumatic gripping tool being movable by the picking manipulator to access and grip the inventory item. [Aspect 56] An order fulfillment system as described in aspect 55, wherein the mobile manipulator robot further includes a tool holder having a first retainer and a second retainer for holding one of the first pneumatic gripping tool and the second pneumatic gripping tool, respectively. [Aspect 57] 56. The order fulfillment system of claim 55, wherein the mobile manipulator robot further includes an onboard processor in communication with at least one of the imaging device or the picking manipulator, and the picking manipulator is configured to interchangeably couple to the first pneumatic gripping tool and the second pneumatic gripping tool upon receiving the processor-executable instructions from the remote processor or executing instructions from the onboard processor. [Aspect 58] 53. The order fulfillment system of claim 52, further comprising a storage structure and a plurality of containers for storing inventory items, the plurality of containers being arranged in a vertical stack within the storage structure. [Aspect 59] 59. The order fulfillment system of claim 58, further comprising a tool holder coupled to the storage structure, the tool holder having a plurality of retainers. [Aspect 60] further comprising a grid disposed above the container, the grid including a first set of parallel rails extending in a first direction and a second set of parallel rails extending in a second direction substantially perpendicular to the first direction, such that the first set of parallel rails and the second set of parallel rails collectively define grid spaces, the first set of parallel rails and the second set of parallel rails having profiled tracks for guiding movement of the wheel assembly along the first set of parallel rails and the second set of parallel rails, each vertical stack being disposed below a respective grid space; An order fulfillment system as described in aspect 58, wherein the container retrieval device can extend below the grid and within each grid space and can engage and lift one of the plurality of containers in the receiving cavity. [Aspect 61] An order fulfillment system as described in aspect 60, wherein one of the plurality of containers is a target container positioned below a plurality of non-target containers, and the opening in the hoist plate is sized so that the hoist plate can slide around each of the non-target containers. [Aspect 62] the container retrieval device is a first container retrieval device and is positioned on a first side of the body for lifting a first container from a first stack; An order fulfillment system as described in aspect 60, wherein the mobile manipulator device further includes a second container retrieval device positioned on a second side of the body for lifting a second container from a second stack different from the first stack. [Aspect 63] 1. A warehouse including a storage structure, a grid including a first set of parallel rails extending in a first direction and a second set of parallel rails extending in a second direction substantially perpendicular to the first direction, such that the first set of parallel rails and the second set of parallel rails collectively define a grid space; a stack of vertically oriented containers, the stack configured to be positioned beneath each of the grid spaces; and a warehouse, a mobile manipulator robot for picking an inventory item stored in one of said bins, a body coupled to a wheel assembly, the wheel assembly including a plurality of wheels and an actuator for moving the body along the grid; a sensor for determining the position of the body relative to the grid on which the body is disposed; an interface configured to wirelessly transmit processor-readable data to a remote processor and wirelessly receive processor-executable instructions from the remote processor; an imaging device for acquiring images of the inventory items, the imaging device configured to collect inventory data; a picking manipulator coupled to the body, the picking manipulator having at least three degrees of freedom; a first pneumatic gripping element and a second pneumatic gripping element coupleable to the picking manipulator, each of the first pneumatic gripping element and the second pneumatic gripping element being movable by the picking manipulator within a three-dimensional workspace to access and grip the inventory items stored in one of the bins, the first pneumatic gripping element including a first suction cup; a mobile manipulator robot, a conductive contact for receiving power from an energy source; an on-board processor in communication with at least one of the imaging device or the picking manipulator, the picking manipulator configured to engage one of the first pneumatic gripping element or the second pneumatic gripping element with an inventory item after receiving the processor-executable instructions from the remote processor or after executing instructions from the on-board processor; an order fulfillment system, including: [Aspect 64] 64. The order fulfillment system of embodiment 63, wherein the mobile manipulator robot further includes an onboard compressor or vacuum source for operating the first suction cup. [Aspect 65] 64. The order fulfillment system of claim 63, wherein the energy source is an electrically charged surface of the grid. [Aspect 66] 64. The order fulfillment system of claim 63, wherein the energy source is an on-board battery. [Aspect 67] a pneumatic supply line coupled to the storage structure; a plurality of valves in fluid communication with the pneumatic supply line, each valve of the plurality of valves being transitionable between a closed state and an open state, the mobile manipulator robot further including a coupler having a mating end in fluid communication with the suction cup, the mating end of the coupler configured to selectively mate with one of the plurality of valves to access a pneumatic supply from the pneumatic supply line; 64. The order fulfillment system of embodiment 63, further comprising: [Aspect 68] An order fulfillment system as described in aspect 63, wherein the second pneumatic gripping element includes a second suction cup, and the mobile manipulator robot further includes a first fluid line extending between the coupler and the first suction cup and a second fluid line extending between the coupler and the second suction cup, and wherein the air pressure or flow rate in the first fluid line is controllable independently of the air pressure or flow rate in the second fluid line. [Aspect 69] An order fulfillment system as described in embodiment 68, wherein the first suction cup and the second suction cup are disposed on a single pneumatic gripping tool. [Aspect 70] An order fulfillment system as described in aspect 68, wherein the first suction cup is positioned on a first pneumatic gripping tool and the second pneumatic gripping element is positioned on a second pneumatic gripping tool different from the first pneumatic gripping tool. [Aspect 71] 69. The order fulfillment system of embodiment 68, further comprising: a first venturi pump in fluid communication with the first fluid line; and a second venturi pump in fluid communication with the second fluid line. [Aspect 72] 1. A mobile manipulator robot for picking inventory items from bins stored in a storage structure having rails, the robot comprising: a body including an interface configured to transmit processor-readable data to a remote processor and to receive processor-executable instructions from said remote processor; a wheel assembly coupled to the body, the wheel assembly including a plurality of wheels and an actuator for moving the body along the rails of the storage structure; an imaging sensor for capturing images of the inventory items; a tool holder coupled to the body, the tool holder having a first retainer and a second retainer; a first tool positionable within the first retainer; a second tool positionable within the second retainer; a picking arm connected to the main body; an on-board processor in communication with at least one of the wheel assembly, the imaging sensor, or the picking arm, the picking arm configured to interchangeably couple to the first tool and the second tool upon receiving instructions from one of the remote processor or the on-board processor; A mobile manipulator robot, including: [Aspect 73] A mobile manipulator robot as described in aspect 72, wherein the first tool has a different size, configuration, or material than the second tool. [Aspect 74] A mobile manipulator robot as described in aspect 72, wherein the picking arm includes a positioning arm having a first pneumatic line and a second pneumatic line separated from the first pneumatic line. [Aspect 75] A mobile manipulator robot as described in aspect 74, wherein the first tool includes a single pneumatic gripping element. [Aspect 76] A mobile manipulator robot as described in aspect 75, wherein when the first tool is coupled to the positioning arm, the first pneumatic line and the second pneumatic line are in communication with the single pneumatic gripping element. [Aspect 77] A mobile manipulator robot as described in embodiment 74, wherein the second tool includes a pneumatic gripping tool having a first pneumatic gripping element and a second pneumatic gripping element. [Aspect 78] A mobile manipulator robot as described in aspect 77, wherein when the second tool is coupled to the positioning arm, the first pneumatic line is in communication with the first pneumatic gripping element and is isolated from the second pneumatic gripping element, and the second pneumatic line is in communication with the second pneumatic gripping element and is isolated from the first pneumatic gripping element. [Aspect 79] 75. The mobile manipulator robot of embodiment 74, further comprising: a first venturi pump in communication with the first pneumatic line; and a second venturi pump in communication with the second pneumatic line. [Aspect 80] A mobile manipulator robot as described in aspect 72, wherein the tool holder includes at least one of a magnetic material or a compatible material for securing the first tool within the first retainer and the second tool within the second retainer. [Aspect 81] A mobile manipulator robot as described in aspect 72, wherein the picking arm includes a magnet for removably coupling the first tool and the second tool to the picking arm. [Aspect 82] A mobile manipulator robot as described in aspect 72, wherein the picking arm is selectively and removably coupleable to the first tool and the second tool via a non-magnetic push / pull connection or a twist lock connection. [Aspect 83] 73. The mobile manipulator robot of embodiment 72, further comprising a coupler mateable with the port to receive the fluid supply from the supply line. [Aspect 84] 1. A mobile manipulator robot for picking inventory items from a bin having a height, the bin being stored under a grid including a first set of rails extending in a first direction and a second set of rails extending in a second direction perpendicular to the first set of rails; The mobile manipulator robot comprises: a body including an interface configured to transmit processor-readable data to a remote processor and to receive processor-executable instructions from said remote processor; a wheel assembly including a plurality of wheels coupled to the body and an actuator configured to move the body along the first set of rails and the second set of rails; an imaging sensor for capturing images of the inventory items; a picking arm connected to the body and selectively couplable to a gripping tool for picking an inventory item from the container, the picking arm being movable such that the gripping tool has a vertical stroke that is at least twice the height of the container; A mobile manipulator robot, including: [Aspect 85] A mobile manipulator robot as described in aspect 84, wherein the gripping tool is pneumatically actuated and formed of a compliant material. [Aspect 86] A mobile manipulator robot as described in aspect 84, wherein the stroke of the gripping tool in the vertical direction is at least three times the height of the container. [Aspect 87] The picking arm, a base member coupled to the body; a first extension coupled to the base member and movable relative to the base member along a first linear path, the first linear path having a vertical component and defining a first maximum vertical distance; a positioning arm coupled to the first extension and movable relative to the first extension along a second linear path, the second linear path having a vertical component and defining a second maximum vertical distance; 85. The mobile manipulator robot of embodiment 84, comprising: [Aspect 88] A mobile manipulator robot as described in aspect 87, wherein a combination of the first maximum vertical distance and the second maximum vertical distance is equal to at least twice the height of the container. [Aspect 89] 88. The mobile manipulator robot of embodiment 87, further comprising an elastic element coupled to the positioning arm to provide passive compliance. [Aspect 90] 88. The mobile manipulator robot of embodiment 87, further comprising at least one of a back-drivable actuator or a force-controlled actuator coupled to the positioning arm to provide active compliance. [Aspect 91] A mobile manipulator robot as described in embodiment 84, wherein the gripping tool includes at least one of a suction cup, a foam vacuum gripper, a universal jamming gripper, or multiple pneumatically actuated fingers. [Aspect 92] 1. A mobile manipulator robot for picking inventory items from bins stored in a storage structure having rails, the robot comprising: a body including an interface configured to transmit processor-readable data to a remote processor and to receive processor-executable instructions from said remote processor; a wheel assembly coupled to the body, the wheel assembly including a plurality of wheels and an actuator for moving the body along the rails of the storage structure; an imaging sensor for capturing images of the inventory items; a tool holder coupled to the body, the tool holder having a first retainer and a second retainer; a first tool positionable within the first retainer; a second tool positionable within the second retainer; a picking arm connected to the main body; an on-board processor in communication with at least one of the wheel assembly, the imaging sensor, or the picking arm, the picking arm configured to interchangeably couple to the first tool and the second tool upon receiving instructions from one of the remote processor or the on-board processor. [Aspect 93] A mobile manipulator robot as described in aspect 92, wherein the first tool has a different size, configuration, or material than the second tool. [Aspect 94] A mobile manipulator robot as described in aspect 92, wherein the picking arm includes a positioning arm having a first pneumatic line and a second pneumatic line separated from the first pneumatic line. [Aspect 95] A mobile manipulator robot as described in aspect 94, wherein the first tool includes a single pneumatic gripping element. [Aspect 96] A mobile manipulator robot as described in aspect 95, wherein when the first tool is coupled to the positioning arm, the first pneumatic line and the second pneumatic line are in communication with the single pneumatic gripping element. [Aspect 97] A mobile manipulator robot as described in aspect 94, wherein the second tool includes a pneumatic gripping tool having a first pneumatic gripping element and a second pneumatic gripping element. [Aspect 98] A mobile manipulator robot as described in aspect 97, wherein when the second tool is coupled to the positioning arm, the first pneumatic line is in communication with the first pneumatic gripping element and is isolated from the second pneumatic gripping element, and the second pneumatic line is in communication with the second pneumatic gripping element and is isolated from the first pneumatic gripping element. [Aspect 99] 95. The mobile manipulator robot of embodiment 94, further comprising a first venturi pump in communication with the first pneumatic line and a second venturi pump in communication with the second pneumatic line. [Aspect 100] A mobile manipulator robot as described in aspect 92, wherein the tool holder includes at least one of a magnetic material or a compatible material for securing the first tool within the first retainer and the second tool within the second retainer. [Aspect 101] A mobile manipulator robot as described in aspect 92, wherein the picking arm includes a magnet for removably coupling the first tool and the second tool to the picking arm. [Aspect 102] A mobile manipulator robot as described in aspect 92, wherein the picking arm is selectively and removably coupleable to the first tool and the second tool via a non-magnetic push / pull connection or a twist lock connection. [Aspect 103] 93. The mobile manipulator robot of embodiment 92, further comprising a coupler mateable with a port to receive the fluid supply from a supply line. [Aspect 104] 1. A mobile manipulator robot for picking inventory items from tall bins, the bins being stored under a grid including a first set of rails extending in a first direction and a second set of rails extending in a second direction perpendicular to the first set of rails; The mobile manipulator robot comprises: a body including an interface configured to transmit processor-readable data to a remote processor and to receive processor-executable instructions from said remote processor; a wheel assembly including a plurality of wheels coupled to the body and an actuator configured to move the body along the first set of rails and the second set of rails; an imaging sensor for capturing images of the inventory items; a picking arm connected to the body and selectively couplable to a gripping tool for picking an inventory item from the container, the picking arm being movable such that the gripping tool has a vertical stroke that is at least twice the height of the container; and A mobile manipulator robot, including: [Aspect 105] A mobile manipulator robot as described in aspect 104, wherein the gripping tool is pneumatically actuated and formed of a compliant material. [Aspect 106] A mobile manipulator robot as described in aspect 104, wherein the stroke of the gripping tool in the vertical direction is at least three times the height of the container. [Aspect 107] The picking arm a base member coupled to the body; a first extension coupled to the base member and movable relative to the base member along a first linear path, the first linear path having a vertical component and defining a first maximum vertical distance; a positioning arm coupled to the first extension and movable relative to the first extension along a second linear path, the second linear path having a vertical component and defining a second maximum vertical distance; 105. The mobile manipulator robot of embodiment 104, comprising: [Aspect 108] A mobile manipulator robot as described in aspect 107, wherein a combination of the first maximum vertical distance and the second maximum vertical distance is equal to at least twice the height of the container. [Aspect 109] A mobile manipulator robot as described in embodiment 107, further comprising an elastic element coupled to the positioning arm to provide passive compliance. [Aspect 110] A mobile manipulator robot as described in embodiment 107, further comprising at least one of a back-drivable actuator and a force control actuator coupled to the positioning arm to provide active compliance. [Aspect 111] A mobile manipulator robot as described in aspect 104, wherein the gripping tool includes at least one of a suction cup, a foam vacuum gripper, a universal jamming gripper, and a plurality of pneumatically actuated fingers. [Industrial Applicability]

[0239] The systems and methods described herein enable efficient storage and retrieval of inventory items within a warehouse or distribution fulfillment center.

Claims

1. A mobile manipulator robot (200; 600) for retrieving inventory, comprising: a body (202) including an interface configured to transmit processor-readable data to a remote processor and to receive processor-executable instructions from said remote processor; a tool holder (258) attached to the body and having a plurality of retainers (260); a translation assembly (204) coupled to the body (202) to translate the body (202) in two dimensions; a sensor for determining the location of the body (202) relative to a fulfillment center or landmarks within the fulfillment center; an imaging device (262) for capturing images of the inventory items; a picking manipulator (206) coupled to the body (202) and having at least three degrees of freedom; a first pneumatic gripping tool (248) movable by said picking manipulator (206) and adapted to grip said inventory item; a coupler (222) having a mating end in fluid communication with the first pneumatic gripping tool (248), the mating end configured to access a supply of air pressure from an external air pressure source; A mobile manipulator robot, including:

2. 2. The mobile manipulator robot of claim 1, further comprising a second pneumatic gripping tool (248) having a different size, configuration, or material than the first pneumatic gripping tool (248), each of the first pneumatic gripping tool (248) and the second pneumatic gripping tool (248) being interchangeably coupleable to the picking manipulator (206) and receivable by a respective one of the plurality of retainers (260).

3. 2. The mobile manipulator robot of claim 1, wherein said first pneumatic gripping tool (248) includes a first gripping element and a second gripping element, and said picking manipulator (206) includes a first pneumatic line in communication with said first gripping element and a second pneumatic line in communication with said second gripping element, said second pneumatic line being separate from said first pneumatic line.

4. The mobile manipulator robot of claim 1, further comprising a venturi pump (244) provided downstream of said coupler (222) and upstream of said first pneumatic gripping tool (248).

5. The mobile manipulator robot of claim 1, wherein said first pneumatic gripping tool (248) is a suction cup.

6. The mobile manipulator robot of claim 1, further comprising an air tank (266) coupled to said body (202), said air tank (266) being less than 20 cubic feet.

7. A mobile manipulator robot (200; 600) for retrieving inventory, comprising: a body (202) including an interface configured to transmit processor-readable data to a remote processor and to receive processor-executable instructions from said remote processor; a mobility assembly (204) coupled to the body (202), the mobility assembly (204) including a plurality of wheels (216) configured to guide movement of the body (202) along a first profiled track extending in a first direction and along a second profiled track extending in a second direction substantially perpendicular to the first direction; an imaging device (262) for capturing images of the inventory items; a picking manipulator (206) coupled to the body (202) and having at least three degrees of freedom; a first pneumatic gripping tool (248) movable by said picking manipulator (206) and adapted to grip said inventory item; a coupler (222) having a mating end in fluid communication with the first pneumatic gripping tool (248), the mating end configured to access a supply of air pressure from an external air pressure source; a container retrieval device (668) including a hoist plate (672) designed to engage and secure a container, said hoist plate (672) suspended by a cable connected to a winding mechanism that moves said hoist plate (672) vertically; A mobile manipulator robot, including:

8. 10. The mobile manipulator robot of claim 1, wherein said mobile manipulator robot is configured to transition a valve (150) from a closed state to an open state mechanically, magnetically, electrically, or wirelessly.

9. The mobile manipulator robot of claim 1 , wherein the coupler (222) includes a self-alignment or misalignment handling device to assist in positioning the coupler (222) on the valve (150).

10. the mobile manipulator robot is configured to retrieve inventory items stored within a grid-based storage system; The grid-based storage system comprises: a frame including a vertical member (116) supporting a first set of parallel rails (122) extending in a first direction and a second set of parallel rails (124) extending in a second direction substantially perpendicular to the first direction, such that the first set of parallel rails and the second set of parallel rails collectively form a grid (126) defining a plurality of grid spaces, the first set of parallel rails (122) and the second set of parallel rails (124) having profiled tracks for guiding movement of the mobile assembly (204) along the first set of parallel rails (122) and the second set of parallel rails (124); a plurality of containers configured to store the inventory items, the plurality of containers being stackable on one another to form a plurality of vertical stacks (112), each vertical stack positioned below a respective grid space; a pneumatic supply line (140) coupled to one of the frame or the grid; an air pressure source in fluid communication with the air pressure supply line (140); a plurality of valves (150) in fluid communication with the pneumatic supply line (140), each valve of the plurality of valves (150) having a closed state in which the pneumatic supply line is isolated from an external environment and an open state in which the pneumatic supply line is in communication with the external environment to supply air pressure to the mobile manipulator robot (200; 600) to operate the first pneumatic gripping tool (248); 10. The mobile manipulator robot of claim 1, comprising:

11. A storage structure (114) configured to house a plurality of containers (110), comprising: a vertical member (116) supporting a first set of parallel rails (122) and a second set of parallel rails (124) substantially perpendicular to the first set of parallel rails, the first set of parallel rails (122) and the second set of parallel rails (124) forming a grid (126) defining a plurality of grid spaces, the grid arranged to support a mobile manipulator robot having a pneumatic gripping tool (248) and a coupler (222) fluidly coupled to the pneumatic gripping tool (248); a pneumatic supply line (140) configured to be accessible to the mobile manipulator robot when the mobile manipulator robot is positioned on the grid; a plurality of valves (150) disposed in the pneumatic supply line (140), the valves configured to engage the couplers (222) and transition from a closed state to an open state with the engaged couplers (222), the closed state isolating the pneumatic supply line (140) from an external environment, while the open state allows the pneumatic supply line to communicate with the external environment to supply air pressure to the mobile manipulator robot; Storage structures, including:

12. The storage structure of claim 11, further comprising an air pressure source (S) in fluid communication with the air pressure supply line (140).

13. The storage structure of claim 11, wherein the air pressure supply line (140) is at least partially disposed within the grid (126).

14. 12. The storage structure of claim 11, wherein the pneumatic supply line (140) includes a channel (142) embedded within and extending longitudinally of the first set of parallel rails (122), and a plurality of conduits (144) extending between the channel (142) and respective ports disposed adjacent surfaces of the first set of parallel rails (122).

15. 15. The storage structure of claim 14, wherein the channel (142) extends continuously for a distance of about two grid spaces or more.

16. The storage structure of claim 14, wherein at least one of the plurality of valves (150) is at least partially disposed within a respective one of the conduits (144).

17. 15. The storage structure of claim 14, wherein at least one of the plurality of valves includes a biasing member coupled to a plug, the biasing member biasing the plug toward the port when the valve is in the closed state.

18. The storage structure of claim 11, wherein the air pressure supply lines (140) are attached to an exterior surface of the first set of parallel rails (122).

19. 1. A system comprising:

1. A mobile manipulator robot for retrieving inventory, comprising: a body (202) including an interface configured to transmit processor-readable data to a remote processor and to receive processor-executable instructions from said remote processor; a mobile assembly (204) coupled to the body (202), the mobile assembly (204) including a plurality of wheels (216) configured to guide movement of the body (202) along a first profiled track extending in a first direction and along a second profiled track extending in a second direction substantially perpendicular to the first direction; a picking manipulator (206) coupled to the body (202) and having at least three degrees of freedom; a first pneumatic gripping tool (248) movable by said picking manipulator (206) for gripping an inventory item; and a coupler (222) having a mating end in fluid communication with the first pneumatic gripping tool (248); a mobile manipulator robot, A storage structure according to claim 11; Including, the mobile manipulator robot is configured to retrieve the inventory items stored within the storage structure; the mating end of the coupler is configured to engage the valve (150) to transition the valve (150) from the closed state to the open state, selectively placing the first pneumatic gripping tool (248) in pneumatic communication with the pneumatic supply line.

Images