Managing robotic manipulators in workflows

Abstract

A robotic system includes an autonomous order picker lift truck configured to move within a commercial environment that includes commercial products supported by donor pallets; a robotic manipulator assembly positioned on a platform of the autonomous order picker lift truck; and a control system communicably coupled to the robotic manipulator assembly and the autonomous order picker lift truck. The control system is configured to perform operations including commanding the autonomous order picker lift truck to move to a particular donor pallet or shelf that is mounted on a rack assembly at or above a support surface of the commercial environment; commanding the autonomous order picker lift truck to move the platform to the particular donor pallet or shelf; and commanding the robotic manipulator assembly to pick a commercial product or a case from the particular donor pallet or shelf and place it on an order pallet or a receiving pallet.

Description

Attorney Docket No.: 248-0023W01MANAGING ROBOTIC MANIPULATORS IN WORKFLOWSTECHNICAL FIELD

[0001] The present disclosure describes systems and methods associated with managing one or more robots in workflows, such as workflows that involve integrating one or more robotic manipulator assemblies with other autonomous mobile robots in warehouse workflows to operate at various vertical levels of case picking.BACKGROUND

[0002] Conventional workflows that include human workers, such as warehouse workflows, can often use certain autonomous robots to replace one, some, or all of the human workers. For example, in a conventional person-to-goods case pick at height process, a human leverages an order picker lift truck or equivalent case pick at height equipment to move through aisles of pallet racking that contain donor pallets at different heights and pick cases or boxes from the donor pallets to either build an order pallet or a receiving pallet of cases or boxes that will be shipped individually or in quantities less than an order pallet. Autonomous order picker lift trucks can be used instead of conventional human-piloted order picker lift trucks to eliminate the need for a human to drive conventional order picker lift trucks up and down aisles to reach donor pallets at any height in aisles, but this does not eliminate the needs for a human to build a pallet on the autonomous order picker, whether an order pallet or a pallet of cases or boxes that will be shipped individually or in quantities less than an order pallet.SUMMARY

[0003] In an example implementation, a robotic system includes at least one autonomous order picker lift truck configured to move within a commercial environment that includes a plurality of commercial products supported by a plurality of donor pallets; a robotic manipulator assembly positioned on a vertically-moveable platform of the autonomous order picker lift truck; and a control system communicably coupled to the robotic manipulator assembly and the one or more autonomous order picker lift trucks. The control system is configured to perform operations including commanding the at least one autonomous order picker lift truck to move to a particular donor pallet or shelf that is mounted on a rack assembly at or above a support surface of the commercial environment; commanding the autonomous order picker lift truck to move theAttorney Docket No.: 248-0023W01 vertically-moveable platform to a vertical height at or near the particular donor pallet or shelf; and commanding the robotic manipulator assembly to pick a commercial product or a case from the particular donor pallet or shelf and place the picked commercial product or case on an order pallet or a receiving pallet.

[0004] In an aspect combinable with the example implementation, the at least one autonomous order picker lift truck is at least one of an autonomous mobile robot (AMRs) or an autonomous guided vehicle (AGV).

[0005] In another aspect combinable with one, some, or all of the previous aspects, the operation of commanding the robotic manipulator assembly to pick the commercial product or the case from the particular donor pallet or shelf and place the picked commercial product or case on the order pallet or the receiving pallet includes commanding the robotic manipulator assembly to autonomously pick the commercial product or the case from the particular donor pallet or shelf and place the picked commercial product or case on the order pallet or the receiving pallet.

[0006] In another aspect combinable with one, some, or all of the previous aspects, the operations include commanding the robotic manipulator assembly to operate an end effector of the robotic manipulator assembly to grasp, hold, or lift the commercial product or the case from the particular donor pallet or shelf.

[0007] In another aspect combinable with one, some, or all of the previous aspects, the operations include commanding the robotic manipulator assembly to operate an adjustable appendage coupled to the end effector to move the end effector proximate to the particular donor pallet.

[0008] In another aspect combinable with one, some, or all of the previous aspects, the operations include commanding the at least one autonomous order picker lift truck to move to another particular donor pallet that is mounted on another rack assembly at or above the support surface of the commercial environment; commanding the autonomous order picker lift truck to move the vertically-moveable platform to a vertical height at or near the another particular donor pallet; and commanding the robotic manipulator assembly to pick another commercial product or another case from the another particular donor pallet and place the picked another commercial product or another case on the order pallet or the receiving pallet.

[0009] In another aspect combinable with one, some, or all of the previous aspects, the at least one commercial product and the another commercial product are different.Attorney Docket No.: 248-0023W01

[0010] In another aspect combinable with one, some, or all of the previous aspects, the operations include commanding a second autonomous order picker lift truck to move a humanoid robot that is positioned on a vertically-moveable platform of the another autonomous order picker lift truck to be adjacent a second particular donor pallet that is mounted on the rack assembly at or above the support surface of the commercial environment; commanding the second autonomous order picker lift truck to move the humanoid robot to a second vertical height at or near the second particular donor pallet; and commanding the humanoid robot to pick a second commercial product or case from the second particular donor pallet and place the picked second commercial product or case to a second order pallet or receiving pallet.

[0011] In another aspect combinable with one, some, or all of the previous aspects, at least a portion of the control system is a part of the humanoid robot.

[0012] In another aspect combinable with one, some, or all of the previous aspects, all of the control system is a part of at least one of the humanoid robot.

[0013] In another aspect combinable with one, some, or all of the previous aspects, at least a portion of the control system is in a head of at least one of the one or more humanoid robots.

[0014] In another aspect combinable with one, some, or all of the previous aspects, the humanoid robot includes a torso and a lower body.

[0015] In another aspect combinable with one, some, or all of the previous aspects, the torso includes two arm appendages, two hand appendages, and a head, and the lower body includes two leg appendages and two feet appendages.

[0016] In another aspect combinable with one, some, or all of the previous aspects, the operations include determining a path of the autonomous order picker lift truck through the commercial environment based on combination of commercial products specified for the order pallet or receiving pallet; and determining a picking order of the commercial products specified in the order pallet based on one or more heights within the rack assembly at which the commercial products are located.

[0017] In another aspect combinable with one, some, or all of the previous aspects, the operation of determining the path includes determining a first commercial product of the combination of commercial products based on at least one of the one or more heights, a weight, or a size of the first commercial product; and determining a second commercial product of theAttorney Docket No.: 248-0023W01 combination of commercial products based on at least one of the one or more heights, a weight, or a size of the second commercial product.

[0018] In another example implementation, a computer-implemented method includes registering, with the control system, at least one autonomous order picker lift truck configured to move within a commercial environment that includes a plurality of commercial products supported by a plurality of donor pallets; registering, with the control system, a robotic manipulator assembly positioned on a vertically-moveable platform of the autonomous order picker lift truck; commanding, with the control system, the at least one autonomous order picker lift truck to move to a particular donor pallet that is mounted on a rack assembly at or above a support surface of the commercial environment; commanding, with the control system, the autonomous order picker lift truck to move the vertically-moveable platform to a vertical height at or near the particular donor pallet; and commanding, with the control system, the robotic manipulator assembly to pick a commercial product or a case from the particular donor pallet and place the picked commercial product or case on an order pallet or a receiving pallet.

[0019] In an aspect combinable with the example implementation, the at least one autonomous order picker lift truck is at least one of an autonomous mobile robot (AMRs) or an autonomous guided vehicle (AGV).

[0020] In another aspect combinable with one, some, or all of the previous aspects, commanding, with the control system, the robotic manipulator assembly to pick the commercial product or the case from the particular donor pallet and place the picked commercial product or case on the order pallet or the receiving pallet includes commanding, with the control system, the robotic manipulator assembly to autonomously pick the commercial product or the case from the particular donor pallet and place the picked commercial product or case on the order pallet or the receiving pallet.

[0021] Another aspect combinable with one, some, or all of the previous aspects includes commanding, with the control system, the robotic manipulator assembly to operate an end effector of the robotic manipulator assembly to grasp, hold, or lift the commercial product or the case from the particular donor pallet.

[0022] Another aspect combinable with one, some, or all of the previous aspects includes commanding, with the control system, the robotic manipulator assembly to operate an adjustableAttorney Docket No.: 248-0023W01 appendage coupled to the end effector to move the end effector proximate to the particular donor pallet.

[0023] Another aspect combinable with one, some, or all of the previous aspects includes commanding, with the control system, the at least one autonomous order picker lift truck to move to another particular donor pallet that is mounted on another rack assembly at or above the support surface of the commercial environment; commanding, with the control system, the autonomous order picker lift truck to move the vertically-moveable platform to a vertical height at or near the another particular donor pallet; and commanding, with the control system, the robotic manipulator assembly to pick another commercial product or another case from the another particular donor pallet and place the picked another commercial product or another case on the order pallet or the receiving pallet.

[0024] In another aspect combinable with one, some, or all of the previous aspects, the at least one commercial product and the another commercial product are different.

[0025] Another aspect combinable with one, some, or all of the previous aspects includes commanding, with the control system, a second autonomous order picker lift truck to move a humanoid robot that is positioned on a vertically-moveable platform of the another autonomous order picker lift truck to be adjacent a second particular donor pallet that is mounted on the rack assembly at or above the support surface of the commercial environment; commanding, with the control system, the second autonomous order picker lift truck to move the humanoid robot to a second vertical height at or near the second particular donor pallet; and commanding, with the control system, the humanoid robot to pick a second commercial product or case from the second particular donor pallet and place the picked second commercial product or case to a second order pallet or receiving pallet.

[0026] In another aspect combinable with one, some, or all of the previous aspects, at least a portion of the control system is a part of the humanoid robot.

[0027] In another aspect combinable with one, some, or all of the previous aspects, all of the control system is a part of at least one of the humanoid robot.

[0028] In another aspect combinable with one, some, or all of the previous aspects, at least a portion of the control system is in a head of at least one of the one or more humanoid robots.

[0029] In another aspect combinable with one, some, or all of the previous aspects, the humanoid robot includes a torso and a lower body.Attorney Docket No.: 248-0023W01

[0030] In another aspect combinable with one, some, or all of the previous aspects, the torso includes two arm appendages, two hand appendages, and a head, and the lower body includes two leg appendages and two feet appendages.

[0031] Another aspect combinable with one, some, or all of the previous aspects includes determining, with the control system, a path of the autonomous order picker lift truck through the commercial environment based on a combination of commercial products specified for the order pallet or receiving pallet; and determining, with the control system, a picking order of the commercial products specified in the order pallet based on one or more heights within the rack assembly at which the commercial products are located.

[0032] In another aspect combinable with one, some, or all of the previous aspects, determining the path includes determining, with the control system, a first commercial product of the combination of commercial products based on at least one of the one or more heights, a weight, or a size of the first commercial product; and determining, with the control system, a second commercial product of the combination of commercial products based on at least one of the one or more heights, a weight, or a size of the second commercial product.

[0033] Implementations of systems and methods according to the present disclosure can include one, some, or all of the following features. For example, implementations according to the present disclosure can eliminate or reduce a need for humans to mount lift trucks to reach product at height, and lift in a warehouse workflow. Also, implementations according to the present disclosure can help prevent or avoid injuries of humans that work at height in a case picking process. As another example, implementations that utilize robotic manipulator assemblies and / or humanoid robots for case pick-to-pallet workflows can help reduce labor turnover and overall labor costs as well as result in a reduction of workers’ compensation claims and time away from work due to injury to picking heavy cases from donor pallets and placing these heavy cases on order pallets, as well as picking cases at a height above a finished floor.

[0034] The details of one or more implementations of the subject matter described in this disclosure are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.Attorney Docket No.: 248-0023W01BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG. 1 is a schematic illustration of an example implementation of a workflow in a commercial environment that includes autonomous mobile robots, robotic manipulator assemblies, and (optionally) humanoid robots according to the present disclosure.

[0036] FIGS. 2A and 2B are example implementations of a humanoid robot according to the present disclosure.

[0037] FIGS. 3A and 3B are schematic illustrations of views of an autonomous mobile robot and a vertically elevated robotic manipulator assembly in a case picking workflow according to the present disclosure.

[0038] FIG. 4 shows a schematic drawing of a control system that can be used in the workflow of FIG. 1 according to the present disclosure.DETAILED DESCRIPTION

[0039] FIG. l is a schematic illustration of an example implementation of a workflow in a commercial environment 100 that includes an autonomous order picker lift trucks (either based on an autonomous mobile robot or an autonomous guided vehicle) and robotic manipulator assemblies, and in some aspects, optionally humanoid robots, according to the present disclosure. For example, commercial environment 100 can be a warehouse (in any industry) or other facility (construction warehouse, shipping center, packaging center, warehouse, distribution or fulfillment center, manufacturing facility or otherwise) in which one or more commercial products (i.e., products sold or otherwise transferred in commerce) are gathered and stored (at least transiently) and arranged in combinations for sale, shipping, further distribution channels, or packaging. The workflow illustrated and described in FIG. 1 can represent a case pick at height process that integrates robotic manipulator assemblies 130 and autonomous order picker lift trucks 108a-n to solve existing issues with existing case pick at height processes regardless of if those processes leverage humans along with conventional order picker lift trucks. Optionally, the workflow illustrated and described in FIG. 1 can also integrate humanoid robots 112a-n into the workflow. The example workflow that integrates robotic manipulator assemblies 130 (and optionally, humanoid robots 112a-n) and autonomous order picker lift trucks 108a-n can eliminate or reduce the need for humans to pick cases at height.Attorney Docket No.: 54248-0023 WO 1

[0040] In particular, in example implementations, a case picking process can include operations performed at different heights (or distances) above a support surface (e.g., the floor, mezzanine, or otherwise) of a commercial environment. Thus, the cases may need to be picked from different vertical levels of pallet racking. In conventional case picking workflows in which humans perform the case picking operations, a human leverages an order picker lift truck to move through aisles of pallet racking that contain donor pallets at different heights and pick cases or boxes from the donor pallets to either build an order pallet or a receiving pallet of cases or boxes that will be shipped individually or in quantities less than an order pallet. However, such conventional systems also suffer from problems associated with the use of human case pickers, including injuries (or worse) to the human pickers that operate at heights (dangerous or otherwise) above the support surface.

[0041] Autonomous order picker lift trucks, which can be extensions or developments of autonomous reach trucks or autonomous VNA (very narrow aisle) trucks, can be used instead of human-controlled order picker lift trucks to eliminate the need for a human to drive the order picker to and from an aisle as well as control both horizontal and vertical operation of the order picker. But this does not eliminate the need for a human to pick cases (at height) and place on a donor or receiving pallet.

[0042] Example implementations according to the present disclosure provide for a case picking at height process that integrates one or more robotic manipulator assemblies (and optionally, humanoid robots such as full bi-ped humanoid robots or the torso of the humanoid robot) along with autonomous non-humanoid robots, such as autonomous order picker lift trucks, to provide a better solution than existing person-to-goods case picking at height processes.

[0043] For example, with a case picking at height process that integrates robotic manipulator assemblies (and optionally humanoid robots) and autonomous order picker lift trucks, a robotic manipulator assembly can be autonomously transported to the correct donor pallet by the autonomous order picker lift truck. In some aspects, a Warehouse Management System (WMS) or Warehouse Execution System (WES) directs the autonomous order picker lift truck with a correct sequence of donor pallets to pick cases in a correct order so that the cases can be layered correctly on a pallet (heavy cases on the bottom or departmental-based loading as examples).

[0044] A WMS or WES directs the robotic manipulator assembly (and, in some aspects, humanoid robots) to pick a specific number of cases from a donor pallet and place on the pallet onAttorney Docket No.: 248-0023W01 the autonomous order picker lift truck. Either the WMS or WES software, or the software platform associated with the robotic manipulator assembly, the humanoid robot, or both, will direct the proper placement of each case on the order pallet to create the correct layers on the order or receiving pallet. After picking and placing the appropriate number of cases in the correct location(s) on the order or receiving pallet, the autonomous order picker lift truck repositions itself and the robotic manipulator assembly to the next pick location where the process repeats across several pick locations until the order or receiving pallet is completed. Example implementations of a picking at height process that integrates robotic manipulator assemblies (and optionally, humanoid robots) with autonomous order picker lift trucks fully automates the case picking at height process and eliminates the dangerous work of humans picking at height while also ensuring that cases are layered in the proper order on order or receiving pallets.

[0045] As shown in FIG. 1, commercial environment 100 (such as a warehouse) includes or encloses an occupiable volume 104 in a building structure 102. The volume 104 can store donor pallets 106a-n (with “n” representing a variable number according to the present disclosure), for example, on a ground surface, pallet racking, or shelving. Each donor pallet 106a-n can enclose, contain, or otherwise support one or more commercial products. In some aspects, each donor pallet 106a-n supports a particular, unique commercial product (e.g., in a large quantity). For example, as shown, donor pallet 106a supports commercial products 105, while donor pallet 106n supports commercial products 107. There can be as many donor pallets 106a-n and commercial products as desired or necessary.

[0046] In this example, one or more autonomous order picker lift trucks 108a-n are positioned in the volume 104 and operable to move throughout the volume 104 to receive commercial products (105, 107, or others) onto order pallets 114a-n. In this example, one or more of autonomous order picker lift trucks 108a-n includes an order pallet or otherwise have an order pallet 114a-n placed onto the autonomous order picker lift trucks (either by its own actions or by a human operator). Each order pallet 114a-n can have a specified or predetermined quantity and type of commercial products to be placed on the order pallet according to a commercial transaction that has been (or will be) completed. As shown in this example, each autonomous order picker lift trucks 108a-n includes a safety field 110, which represents a volume surrounding the particular autonomous order picker lift trucks into which other objects (such as humanoid robots 112a-n) should not enter (e.g., for safety or other reasons).Attorney Docket No.: 54248-0023 WO 1

[0047] In the present disclosure, one or more of the autonomous order picker lift trucks 108a-n can be an autonomous mobile robot (AMRs). As another example, one or more of the autonomous robots 108a-n can be an autonomous guided vehicle (AGV). Most likely, an autonomous order picker lift trucks will be a variant of an autonomous reach truck or autonomous VNA reach truck. In some aspects, AMRs can be distinguished from AGVs based on, for example, a level of autonomy of movement within the volume 104. For example, an AMR can be capable of full autonomous movement through the volume 104 such as by mapping the volume 104 and freely moving through the volume 104 based on the mapping. An AGV, however, may, in some aspects, move through the volume 104 on a guide wire that is positioned in the volume 104 (e.g., within a floor). Thus, the AGV may follow the guide wire through aisles of the volume 104 but, when not in the aisles, will wait on any object around it to move instead of navigating on a different path.

[0048] One or more of the autonomous robots 108a-108n are autonomous order picker lift trucks, which can autonomously (at least partially) operate to move through the volume 104. In some aspects, as explained more fully herein, a robotic manipulator assembly 130 can be mounted or positioned on (e.g., on a platform of) autonomous order picker lift truck 108e and be raised above a support surface (such as the floor of the building structure 102) to pick commercial product 105 from a case or multiple cases (or specific products within a case) on an elevated donor pallet 106n (e.g., positioned on a rack structure above the floor). Once picked, the lift truck 108d can return the robotic manipulator assembly 130 to the support surface, autonomously move the robotic manipulator assembly 130 to another vertical level in the volume 104 (e.g., a higher level) to pick additional cases, autonomously move the robotic manipulator assembly 130 to another location in the volume 104 (e.g., at the same vertical level), or other operation.

[0049] In this example workflow, the commercial environment 100 can include a Warehouse Management System (WMS) 999 that can, in some aspects, include, interface with, or incorporate a Warehouse Execution System (WES), a Warehouse Control System (WCS), and / or WES (or pick path optimization) functionality. Other functionality can be included with or interface with the WMS 999, such as cubing (or so-called “Tetris”) functionality, which allows the WMS 999 to instruct the robotic manipulator assembly 130 (which can be one of many robotic manipulator assemblies 120 in the environment 100) how and in what order to place product on one or more pallets (e.g., based on size and / or weight of the product, an order of removal of theAttorney Docket No.: 248-0023W01 product from the pallet such as last on-first off, an arrangement of the unloaded product within a store or other commercial enterprise, or a combination thereof).

[0050] The WMS 999, WES or WCS can be, for example, a microprocessor based control system that controls the operations of the autonomous robots 108a-n, the robotic manipulator assembly(s) 130, and, when included, humanoid robots 112a-n, according to software instructions executable by the WMS 999, WES or WCS. In some aspects, the WMS 999, WES or WCS controls operations of autonomous robots 108a-n, robotic manipulator assembly(s) 130, and (optionally) the humanoid robots 112a-n to move commercial products (105, 107, and others) from donor pallets 106a-n to order pallets 114a-n (e.g., in specified quantities and in a specified order of loading) in order to fulfill a commercial transaction or otherwise. In some aspects, WMS 999, WES or WCS can be a physically separate control system that communicates (e.g., wired or wirelessly) with the autonomous robots 108a-n, the robotic manipulator assembly(s) 130, or (optionally) humanoid robots 112a-n. Alternatively, some or all of the functionality (e.g., processing capability, memory storage, communications, software instructions) can be located in one or more of the humanoid robots 112a-n (such as, for example, within a head or torso of a humanoid robot). Thus, in some aspects, one or more of the humanoid robots 112a-n can act as the WMS 999 to control the autonomous robots 108a-n, the robotic manipulator assembly(s), and the optional humanoid robots 112a-n.

[0051] In some aspects of an example workflow of commercial environment 100, the WMS 999, WES or WCS can identify or register all of the autonomous robots 109a-n and robotic manipulator assemblies 103 (as well as, when used, humanoid robots 112a-n) within the volume 104 (e.g., in order to determine which of the robotic manipulator assemblies and / or robots are activated or operable). The WMS 999, WES or WCS can then communicate with autonomous robot 108c and direct autonomous robot 108c toward the donor pallets 106a (e.g., in some cases, subsequent to picking up an empty order pallet 114a-n) in order to load an empty order pallet 114c with commercial products (105, 107, or others) to fulfill a specified transaction. In some aspects, the WMS 999, WES or WCS directs the autonomous robot 108c toward a particular donor pallet 106a-n (and subsequently to other donor pallets 106a-n in a specific order) based on a size or weight (or both) of the commercial product supported on the particular donor pallets 106a-n.

[0052] For example, in loading the empty order pallet 114c, it may be beneficial or advantageous to load commercial product heaviest (or largest) to lightest (or smallest) accordingAttorney Docket No.: 248-0023W01 to cubing functionality built into or interfaced with the WMS 999, WES or WCS. In such aspects, lighter commercial product may not be crushed or damaged by later-loaded and heavier commercial product. In some instances, the WMS can provide one or more autonomous robot 108a-n and / or one or more robotic manipulator assemblies 130 one or more tasks. In some aspects, such as with the inclusion of cubing functionality, the WMS 999, WES or WCS can instruct, e.g., the robotic manipulator assemblies 130 and / or the humanoid robots 112a-n to pick multiple cases and place them on a pallet, with a location specificity of placement on the pallet varying depending on the aforementioned criteria.

[0053] In the case of operations by a humanoid robot, after the autonomous robot 108c is directed to a particular donor pallet, such as donor pallet 106a to pick up commercial product 105, the WMS 999 controls the humanoid robot 112a at the donor pallet 106a to pick a specific number of commercial product 105 from the donor pallet 106a and place the picked commercial product 105 onto the order pallet 114c. In some aspects, the WMS 999 directs the humanoid robot 112a to remain close to the donor pallet 106a until the autonomous robot 108c has stopped at a specific location to ensure that the humanoid robot 112a does not trigger the safety field 110 of the autonomous robot 108c. In some aspects, the WMS 999 or controller of the humanoid robot 112a (e.g., in the head or torso of the humanoid robot 112a) directs a proper placement of each commercial product 105 on the order pallet 114c to create the correct layers of product on the order pallet 114c. After picking and placing the appropriate number of commercial product 105 in the correct location(s) on the order pallet 114c, the humanoid robot 112a repositions itself close to the donor pallet 106a (e.g., at pallet racking or shelving) so that the autonomous robot 108c can leave the area around the donor pallet 106a without its safety field 110 being triggered.

[0054] The above-described operations can be repeated at one or more additional donor pallets 106a-n. For example, another autonomous robot 108d can next be directed to donor pallet 106n to pick up commercial product 107. If the order pallet 114d is complete, the autonomous robot 108c can move the order pallet 114d toward a location in the volume 104 in which the order pallet 114c can be shipped or otherwise packaged (e.g., as shown with autonomous robot 108b moving a completed order pallet 114b).

[0055] Turning now to FIGS. 3A and 3B, these figures are schematic illustrations of views of an autonomous mobile robot and a vertically elevated robotic manipulator assembly in a case picking workflow, such as the environment 100 described with reference to FIG. 1. FIG. 3A showsAttorney Docket No.: 248-0023W01 a side view of the autonomous robot 108e with the robotic manipulator assembly 130 positioned on a platform 111 of the autonomous robot 108e. In this figure, a donor pallet 106b is mounted on or supported by forks 115 that extend from the platform 111 (much like a forklift). The robotic manipulator assembly 130 is mounted on or supported by the platform 111 as shown.

[0056] In this example, the robotic manipulator assembly 130 includes a base 132 and a segmented appendage 134 that extends from the base 132. An end effector 136 (such as a claw, hand, pincer, suction effector, or otherwise) is coupled to an end of the segmented appendage 134 opposite the base 132. In some aspects, the base 132 can rotate (e.g., about the Y axis, shown) to rotate the segmented appendage 134 in an arc (e.g., 360°) so as to move the end effector 136 in the arc as well.

[0057] Segmented appendage 134, in some aspects, comprises an articulated “arm” appendage that includes two or more hinged, pivotable, and / or rotatable connections (e.g., analogous to elbows, shoulders, and / or wrists of a human arm). As such, the segmented appendage 134 can be operated to move in three dimensions from the base 132 (e.g., x, y, and z directions).

[0058] FIG. 3B shows a front view of the 108e with the robotic manipulator assembly 130 positioned on the platform 111 of the autonomous robot 108e above a support surface 301 (e.g., floor) of an environment. As shown in this example, the autonomous robot 108e is positioned adjacent a product rack assembly 140 that includes multiple shelves 143 above the support surface 301. In this example, the platform 111 is raised so as to raise the robotic manipulator assembly 130 to a level above the surface 301 adjacent a top most shelf 143; however, the platform 111 of the autonomous robot 108e can be adjusted to any appropriate height, as well as level with the support surface 301, so as to position the robotic manipulator assembly 130 at any particular shelf 143 of the rack assembly 140 (including the support surface 301, itself).

[0059] In the example operations (e.g., autonomously or semi-autonomously as conducted by the WMS 999) shown in FIGS. 3 A and 3B (which can be implemented in the environment 100), the autonomous robot 108e can be operated or moved within the environment 100 (and on support surface 301) to be adjacent or near the rack assembly 140. If needed, the platform 111 can be operated to move, e.g., vertically upward, to raise the robotic manipulator assembly 130 to a level at or near a commercial product 105 that has previously been selected to be picked to the donor pallet 106b. Once at the level, the robotic manipulator assembly 130 can be operated to pick up the commercial product 105 with the end effector 136 and place the commercial product 105 onAttorney Docket No.: 248-0023W01 the donor pallet 106b. The robotic manipulator assembly 130 can be operated to select and pick additional commercial product 105, or the autonomous robot 108e can be operated to move to another portion of the rack assembly 140 (vertically or along the support surface 301) or another rack assembly 140.

[0060] Optionally, one or more humanoid robots 112a-n are positioned in the volume 104, such as proximate to (e.g., within 1 ft., 2 ft., 3 ft., 4 ft., 5-10 ft., or otherwise) the donor pallets 106a-n. In some aspects, there can be a one-to-one ratio of humanoid robots 112a-n to donor pallets 106a-n. Alternatively, there can be more humanoid robots 112a-n than donor pallets 106a- n (e.g., more than one humanoid robot 112a-n can be positioned adjacent or assigned to a particular donor pallet 106a-n). Alternatively, there can be fewer humanoid robots 112a-n than donor pallets 106a-n (e.g., a particular humanoid robot 112a-n can be positioned adjacent or assigned to more than one donor pallet 106a-n). Generally, each humanoid robot 112a-n is operable to move to remove one or more commercial products from a particular donor pallet 106a-n (or more than one donor pallet) and, as described herein, place the removed commercial product(s) onto an order pallet.

[0061] Turning briefly to FIGS. 2A and 2B, these figures illustrate example implementations of a humanoid robot 112a according to the present disclosure. As shown, the humanoid robot 112a can include a torso and lower body, with appendages (e.g., legs, feet, arms, hands, head) that mimic or otherwise resemble and functional similarly to the corresponding human body appendages. The humanoid robot 112a can include multifunction movement, such as: walking, squatting, bending at waist, kneeling, torso rotation, head rotation, lifting (from ground or another support surface such as a lift truck or mezzanine, to torso height, above head), carrying (e.g., commercial product 105 as shown in FIG. 2B) as well as other movements that mimic human natural movement). In addition, in some aspects, the humanoid robot 112a can include visual image sensing and recognition (e.g., built into the head or otherwise), radar, or lidar to detect objects within its path. As such, the humanoid robot 112a can be distinguished from the autonomous robots 108a-n, which can have movement capability (e.g., through wheels, tracks, rollers, or other non-humanlike apparatus) but do not include humanlike appendages. Example autonomous robots 108a-n can be, for example robots made by Fetch Robotics, Mobile Industrial Robots, OTTO Robotics or otherwise.Attorney Docket No.: 248-0023W01

[0062] FIG. 4 shows a schematic drawing of a control system 400 that can be used in the example workflow of FIG. 1 according to the present disclosure. For example, all or parts of the control system (or controller) 400 can be used for the operations described previously, for example as or as part of the Warehouse Management System (WMS) 999. Some or all of the example control system 400 (or WMS 900 generally) can be implemented as cloud-based system and / or service, alone or in combination with other portions of the example control system 400 that can be implemented at the commercial environment 100. The controller 400 is intended to include various forms of digital computers, such as printed circuit boards (PCB), processors, digital circuitry, or otherwise. Additionally, the system can include portable storage media, such as, Universal Serial Bus (USB) flash drives. For example, the USB flash drives may store operating systems and other applications. The USB flash drives can include input / output components, such as a wireless transmitter or USB connector that may be inserted into a USB port of another computing device.

[0063] The controller 400 includes a processor 410, a memory 420, a storage device 430, and an input / output device 440. Each of the components 410, 420, 430, and 440 are interconnected using a system bus 450. The processor 410 is capable of processing instructions for execution within the controller 400. The processor may be designed using any of a number of architectures. For example, the processor 410 may be a CISC (Complex Instruction Set Computers) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimal Instruction Set Computer) processor.

[0064] In one implementation, the processor 410 is a single-threaded processor. In another implementation, the processor 410 is a multi-threaded processor. The processor 410 is capable of processing instructions stored in the memory 420 or on the storage device 430 to display graphical information for a user interface on the input / output device 440.

[0065] The memory 420 stores information within the control system 400. In one implementation, the memory 420 is a computer-readable medium. In one implementation, the memory 420 is a volatile memory unit. In another implementation, the memory 420 is a nonvolatile memory unit.

[0066] The storage device 430 is capable of providing mass storage for the controller 400. In one implementation, the storage device 430 is a computer-readable medium. In various differentAttorney Docket No.: 248-0023W01 implementations, the storage device 430 may be a floppy disk device, a hard disk device, an optical disk device, a tape device, flash memory, a solid state device (SSD), or a combination thereof.

[0067] The input / output device 440 provides input / output operations for the controller 400.In one implementation, the input / output device 440 includes a keyboard and / or pointing device. In another implementation, the input / output device 440 includes a display unit for displaying graphical user interfaces.

[0068] The features described can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The apparatus can be implemented in a computer program product tangibly embodied in an information carrier, for example, in a machine-readable storage device for execution by a programmable processor; and method steps can be performed by a programmable processor executing a program of instructions to perform functions of the described implementations by operating on input data and generating output. The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0069] Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of multiple processors of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer will also include, or be operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memoryAttorney Docket No.: 248-0023W01 devices, such as EPROM, EEPROM, solid state drives (SSDs), and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD- ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).

[0070] To provide for interaction with a user, the features can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) or LED (light-emitting diode) monitor for displaying information to the user and a keyboard and a pointing device such as a mouse or a trackball by which the user can provide input to the computer. Additionally, such activities can be implemented via touchscreen flat-panel displays and other appropriate mechanisms.

[0071] The features can be implemented in a control system that includes a back-end component, such as a data server, or that includes a middleware component, such as an application server or an Internet server, or that includes a front-end component, such as a client computer having a graphical user interface or an Internet browser, or any combination of them. The components of the system can be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), peer-to-peer networks (having ad-hoc or static members), grid computing infrastructures, and the Internet.

[0072] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular implementations of particular inventions. Certain features that are described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a sub combination.

[0073] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular orderAttorney Docket No.: 248-0023W01 shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0074] A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. For example, example operations, methods, or processes described herein may include more steps or fewer steps than those described. Further, the steps in such example operations, methods, or processes may be performed in different successions than that described or illustrated in the figures. Accordingly, other implementations are within the scope of the following claims.

Claims

Attorney Docket No.: 248-0023W01WHAT IS CLAIMED IS:

1. A robotic system, comprising: at least one autonomous order picker lift truck configured to move within a commercial environment that includes a plurality of commercial products supported by a plurality of donor pallets; a robotic manipulator assembly positioned on a vertically-moveable platform of the autonomous order picker lift truck; and a control system communicably coupled to the robotic manipulator assembly and the one or more autonomous order picker lift trucks, the control system configured to perform operations comprising: commanding the at least one autonomous order picker lift truck to move to a particular donor pallet or shelf that is mounted on a rack assembly at or above a support surface of the commercial environment; commanding the autonomous order picker lift truck to move the vertically- moveable platform to a vertical height at or near the particular donor pallet or shelf; and commanding the robotic manipulator assembly to pick a commercial product or a case from the particular donor pallet or shelf and place the picked commercial product or case on an order pallet or a receiving pallet.

2. The robotic system of claim 1, wherein the at least one autonomous order picker lift truck is at least one of an autonomous mobile robot (AMRs) or an autonomous guided vehicle (AGV).

3. The robotic system of either one of claims 1 or 2, wherein the operation of commanding the robotic manipulator assembly to pick the commercial product or the case from the particular donor pallet or shelf and place the picked commercial product or case on the order pallet or the receiving pallet comprises: commanding the robotic manipulator assembly to autonomously pick the commercial product or the case from the particular donor pallet or shelf and place the picked commercial product or case on the order pallet or the receiving pallet.Attorney Docket No.: 248-0023W014. The robotic system of either one of claims 1 or 2, wherein the operations comprise commanding the robotic manipulator assembly to operate an end effector of the robotic manipulator assembly to grasp, hold, or lift the commercial product or the case from the particular donor pallet or shelf.

5. The robotic system of claim 4, wherein the operations comprise commanding the robotic manipulator assembly to operate an adjustable appendage coupled to the end effector to move the end effector proximate to the particular donor pallet.

6. The robotic system of any one of the previous claims, wherein the operations comprise: commanding the at least one autonomous order picker lift truck to move to another particular donor pallet that is mounted on another rack assembly at or above the support surface of the commercial environment; commanding the autonomous order picker lift truck to move the vertically-moveable platform to a vertical height at or near the another particular donor pallet; and commanding the robotic manipulator assembly to pick another commercial product or another case from the another particular donor pallet and place the picked another commercial product or another case on the order pallet or the receiving pallet.

7. The robotic system of claim 6, wherein the at least one commercial product and the another commercial product are different.Attorney Docket No.: 248-0023W018. The robotic system of any one of the previous claims, wherein the operations comprise: commanding a second autonomous order picker lift truck to move a humanoid robot that is positioned on a vertically-moveable platform of the another autonomous order picker lift truck to be adjacent a second particular donor pallet that is mounted on the rack assembly at or above the support surface of the commercial environment; commanding the second autonomous order picker lift truck to move the humanoid robot to a second vertical height at or near the second particular donor pallet; and commanding the humanoid robot to pick a second commercial product or case from the second particular donor pallet and place the picked second commercial product or case to a second order pallet or receiving pallet.

9. The robotic system of claim 8, wherein at least a portion of the control system is a part of the humanoid robot.

10. The robotic system of claim 9, wherein all of the control system is a part of at least one of the humanoid robot.

11. The robotic system of either one of claims 9 or 10, wherein at least a portion of the control system is in a head of at least one of the one or more humanoid robots.

12. The robotic system of any one of the previous claims 8-11, wherein the humanoid robot comprises a torso and a lower body.

13. The robotic system of claim 12, wherein the torso comprises two arm appendages, two hand appendages, and a head, and the lower body comprises two leg appendages and two feet appendages.Attorney Docket No.: 248-0023W0114. The robotic system of any one of the previous claims, wherein the operations comprise: determining a path of the autonomous order picker lift truck through the commercial environment based on a combination of commercial products specified for the order pallet or receiving pallet; and determining a picking order of the commercial products specified in the order pallet based on one or more heights within the rack assembly at which the commercial products are located.

15. The robotic system of claim 14, wherein the operation of determining the path comprises: determining a first commercial product of the combination of commercial products based on at least one of the one or more heights, a weight, or a size of the first commercial product; and determining a second commercial product of the combination of commercial products based on at least one of the one or more heights, a weight, or a size of the second commercial product.

16. A computer-implemented method, comprising: registering, with the control system, at least one autonomous order picker lift truck configured to move within a commercial environment that includes a plurality of commercial products supported by a plurality of donor pallets; registering, with the control system, a robotic manipulator assembly positioned on a vertically-moveable platform of the autonomous order picker lift truck; commanding, with the control system, the at least one autonomous order picker lift truck to move to a particular donor pallet that is mounted on a rack assembly at or above a support surface of the commercial environment; commanding, with the control system, the autonomous order picker lift truck to move the vertically-moveable platform to a vertical height at or near the particular donor pallet; and commanding, with the control system, the robotic manipulator assembly to pick a commercial product or a case from the particular donor pallet and place the picked commercial product or case on an order pallet or a receiving pallet.Attorney Docket No.: 248-0023W0117. The computer-implemented method of claim 16, wherein the at least one autonomous order picker lift truck is at least one of an autonomous mobile robot (AMRs) or an autonomous guided vehicle (AGV).

18. The computer-implemented method of either one of claims 16 or 17, wherein commanding, with the control system, the robotic manipulator assembly to pick the commercial product or the case from the particular donor pallet and place the picked commercial product or case on the order pallet or the receiving pallet comprises: commanding, with the control system, the robotic manipulator assembly to autonomously pick the commercial product or the case from the particular donor pallet and place the picked commercial product or case on the order pallet or the receiving pallet.

19. The computer-implemented method of either one of claims 16 or 17, comprising commanding, with the control system, the robotic manipulator assembly to operate an end effector of the robotic manipulator assembly to grasp, hold, or lift the commercial product or the case from the particular donor pallet.

20. The computer-implemented method of claim 19, comprising commanding, with the control system, the robotic manipulator assembly to operate an adjustable appendage coupled to the end effector to move the end effector proximate to the particular donor pallet.

21. The computer-implemented method of any one of the previous claims 16-20, comprising: commanding, with the control system, the at least one autonomous order picker lift truck to move to another particular donor pallet that is mounted on another rack assembly at or above the support surface of the commercial environment; commanding, with the control system, the autonomous order picker lift truck to move the vertically-moveable platform to a vertical height at or near the another particular donor pallet; and commanding, with the control system, the robotic manipulator assembly to pick another commercial product or another case from the another particular donor pallet and place the picked another commercial product or another case on the order pallet or the receiving pallet.Attorney Docket No.: 248-0023W0122. The computer-implemented method of claim 21, wherein the at least one commercial product and the another commercial product are different.

23. The computer-implemented method of any one of the previous claims 16-22, comprising: commanding, with the control system, a second autonomous order picker lift truck to move a humanoid robot that is positioned on a vertically-moveable platform of the another autonomous order picker lift truck to be adjacent a second particular donor pallet that is mounted on the rack assembly at or above the support surface of the commercial environment; commanding, with the control system, the second autonomous order picker lift truck to move the humanoid robot to a second vertical height at or near the second particular donor pallet; and commanding, with the control system, the humanoid robot to pick a second commercial product or case from the second particular donor pallet and place the picked second commercial product or case to a second order pallet or receiving pallet.

24. The computer-implemented method of claim 23, wherein at least a portion of the control system is a part of the humanoid robot.

25. The computer-implemented method of claim 24, wherein all of the control system is a part of at least one of the humanoid robot.

26. The computer-implemented method of either one of claims 24 or 25, wherein at least a portion of the control system is in a head of at least one of the one or more humanoid robots.

27. The computer-implemented method of any one of the previous claims 23-26, wherein the humanoid robot comprises a torso and a lower body.

28. The computer-implemented method of claim 27, wherein the torso comprises two arm appendages, two hand appendages, and a head, and the lower body comprises two leg appendages and two feet appendages.Attorney Docket No.: 248-0023W0129. The computer-implemented method of any one of the previous claims 16-28, comprising: determining, with the control system, a path of the autonomous order picker lift truck through the commercial environment based on a combination of commercial products specified for the order pallet or receiving pallet; and determining, with the control system, a picking order of the commercial products specified in the order pallet based on one or more heights within the rack assembly at which the commercial products are located.

30. The computer-implemented method of claim 29, wherein determining the path comprises: determining, with the control system, a first commercial product of the combination of commercial products based on at least one of the one or more heights, a weight, or a size of the first commercial product; and determining, with the control system, a second commercial product of the combination of commercial products based on at least one of the one or more heights, a weight, or a size of the second commercial product.

Images