Integrating humanoid robots with autonomous vehicles in workflows

Abstract

A robotic system includes humanoid robots configured to move within a commercial environment that includes totes that hold commercial product; at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle configured to couple to and move carts through the commercial environment; and a control system communicably coupled to the humanoid robots and the autonomous tugger, autonomous mobile robot, or automated guided vehicle. The control system performs operations including commanding the autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the carts to a first location in the commercial environment; commanding the humanoid robots at the first location to perform at least one of: pick at least one tote to leave at the first location or place at least one tote from the first location on the carts; and commanding the autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the from the first location to a second location in the commercial environment.

Description

INTEGRATING HUMANOID ROBOTS WITH AUTONOMOUS VEHICLES IN WORKFLOWSTECHNICAL FIELD

[0001] The present disclosure describes systems and methods associated with integrating humanoid robots with autonomous tuggers in workflows, such as autonomous mobile robots (AMRs) or automated guided vehicles (AGVs) that are used in workflows.BACKGROUND

[0002] In many manufacturing processes, such as workcell delivery, lineside delivery, machine tending, work-in-progress (WIP), and “supermarket” (stocking area) replenishment of carts, humans drive tuggers that pull carts through a manufacturing plant and in-between distribution centers collocated with manufacturing plants, and move totes to and from the carts that are being pulled behind the tugger. Further, autonomous tuggers, whether automated guided vehicles or autonomous mobile robots, can be used instead of manually driven tuggers to move carts of totes through a manufacturing plant and even between a collocated distribution center or warehouse and a manufacturing plant before starting the delivery process. Since there is no longer a human driving the tugger now that it is autonomous, there is also not a human to move carts to and from the carts pulled by the autonomous tugger.SUMMARY

[0003] In an example implementation, a robotic system includes one or more humanoid robots configured to move within a commercial environment that includes a plurality of totes configured to hold at least one of commercial product or material; at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle configured to couple to and move one or more carts through the commercial environment, each of the one or more carts configured to support at least a portion of the totes; and a control system communicably coupled to the one or more humanoid robots and the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle. The control system is configured to perform operations including commanding the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts to a first location in the commercial environment; commanding the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts; and commanding the at least one autonomous tugger, autonomous mobile robot, or automatedguided vehicle to move the one or more carts from the first location to a second location in the commercial environment.

[0004] In an aspect combinable with the example implementation, the operations include commanding the one or more humanoid robots to move away from a travel path of the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle between the first location and the second location prior to commanding the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts from the first location to the second location.

[0005] In another aspect combinable with any of the previous aspects, the operations include commanding the one or more humanoid robots at the first location to pick a specific number of totes from the one or more carts to leave at the first location.

[0006] In another aspect combinable with any of the previous aspects, the operations include commanding the one or more humanoid robots at the first location to place a specific number of totes from the first location on the one or more carts.

[0007] In another aspect combinable with any of the previous aspects, the one or more humanoid robots at the first location are configured to place a specific number of totes from the first location on the one or more carts based on a visual inspection of the one or more carts by the one or more humanoid robots.

[0008] In another aspect combinable with any of the previous aspects, the visual inspection of the one or more carts by the one or more humanoid robots includes a visual inspection of a type of the commercial product or material supported in the one or more totes to be placed from the first location on the one or more carts.

[0009] In another aspect combinable with any of the previous aspects, the one or more humanoid robots at the first location are configured to pick a specific number of totes from the one or more carts to leave at the first location based on a visual inspection of the one or more carts by the one or more humanoid robots.

[0010] In another aspect combinable with any of the previous aspects, the visual inspection of the one or more carts by the one or more humanoid robots includes a visual inspection of a t pe of the commercial product or material supported in the one or more totes to be picked from the one or more carts and left at the first location.

[0011] In another aspect combinable with any of the previous aspects, the first location includes a manufacturing line or workcell and the second location includes a source of the material.

[0012] In another aspect combinable with any of the previous aspects, the operation of commanding the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts includes commanding the one or more humanoid robots at the first location to pick at least one tote that holds material from the one or more carts to leave at the first location; and commanding the one or more humanoid robots at the first location to place at least one empty tote from the first location on the one or more carts.

[0013] In another aspect combinable with any of the previous aspects, the first location includes a manufacturing line or workcell and the second location includes a source of the material.

[0014] In another aspect combinable with any of the previous aspects, the operation of commanding the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts includes commanding the one or more humanoid robots at the first location to pick at least one tote that holds material from the one or more carts to leave at the first location; and commanding the one or more humanoid robots at the first location to place at least one tote that holds commercial product from the first location on the one or more carts.

[0015] In another example implementation, a computer-implemented method includes registering, with a control system, one or more humanoid robots within a commercial environment that includes a plurality' of commercial product and material; registering, with the control system, at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle configured to couple to and move one or more carts through the commercial environment, each of the one or more carts configured to support a plurality of totes configured to hold at least one of commercial product or material; commanding, with the control system, the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts to a first location in the commercial environment; commanding, with the control system, the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts; and commanding, with the control system, the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts from the first location to a second location in the commercial environment.

[0016] An aspect combinable with the example implementation includes commanding, with the control system, the one or more humanoid robots to move away from a travel path of the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle between the first location and the second location prior to commanding the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts from the first location to the second location.

[0017] Another aspect combinable with any of the previous aspects includes commanding, with the control system, the one or more humanoid robots at the first location to pick a specific number of totes from the one or more carts to leave at the first location.

[0018] Another aspect combinable with any of the previous aspects includes commanding, with the control system, the one or more humanoid robots at the first location to place a specific number of totes from the first location on the one or more carts.

[0019] In another aspect combinable with any of the previous aspects, the one or more humanoid robots at the first location are configured to place a specific number of totes from the first location on the one or more carts based on a visual inspection of the one or more carts by the one or more humanoid robots.

[0020] In another aspect combinable with any of the previous aspects, the visual inspection of the one or more carts by the one or more humanoid robots includes a visual inspection of a type of the commercial product or material supported in the one or more totes to be placed from the first location on the one or more carts.

[0021] In another aspect combinable with any of the previous aspects, the one or more humanoid robots at the first location are configured to pick a specific number of totes from the one or more carts to leave at the first location based on a visual inspection of the one or more carts by the one or more humanoid robots.

[0022] In another aspect combinable with any of the previous aspects, the visual inspection of the one or more carts by the one or more humanoid robots includes a visual inspection of a type of the commercial product or material supported in the one or more totes to be picked from the one or more carts and left at the first location.

[0023] In another aspect combinable with any of the previous aspects, the first location includes a manufacturing line or w orked I and the second location includes a source of the material.

[0024] In another aspect combinable with any of the previous aspects, commanding, with the control system, the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, orplace at least one tote from the first location on the one or more carts includes commanding, with the control system, the one or more humanoid robots at the first location to pick at least one tote that holds material from the one or more carts to leave at the first location; and commanding, with the control system, the one or more humanoid robots at the first location to place at least one empty7tote from the first location on the one or more carts.

[0025] In another aspect combinable with any of the previous aspects, the first location includes a manufacturing line or workcell and the second location includes a source of the material.

[0026] In another aspect combinable with any of the previous aspects, commanding, with the control system, the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts includes commanding, with the control system, the one or more humanoid robots at the first location to pick at least one tote that holds material from the one or more carts to leave at the first location; and commanding, with the control system, the one or more humanoid robots at the first location to place at least one tote that holds commercial product from the first location on the one or more carts.

[0027] Implementations of systems and methods according to the present disclosure can include one, some, or all of the following features. For example, implementations according to the present disclosure can provide for a fully integrated workflow of humanoid robots and autonomous tuggers, whether automated guided vehicles (AGVs) or Autonomous Mobile Robots (AMRs) to enable companies to fully autonomize tote-based manufacturing processes such as workcell delivery, lineside delivery7, machine tending, work-in-progress (WIP), and “supermarket” (stocking area) replenishment of carts by a humanoid robot moving totes to and from the carts that are being pulled behind an autonomous tugger, AMR or AGV. This can significantly reduce the amount of labor needed for these tasks in manufacturing plants so that manufacturing companies can redeploy scarce labor to skilled positions.

[0028] The details of one or more implementations of the subject matter described in this disclosure are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0029] FIG. 1 is a schematic illustration of an example implementation of a workflow in a commercial environment that integrates humanoid robots with one or more autonomous tuggers, autonomous mobile robots, and / or automated guided vehicles according to the present disclosure.

[0030] FIGS. 2 A and 2B are example implementations of a humanoid robot according to the present disclosure.

[0031] FIGS. 3A and 3B are schematic illustrations of another example implementation of a workflow in a commercial environment that integrates humanoid robots with one or more autonomous tuggers, autonomous mobile robots, and / or automated guided vehicles according to the present disclosure.

[0032] FIGS. 4A-4C are schematic illustrations of another example implementation of a workflow in a commercial environment that integrates humanoid robots with one or more autonomous tuggers, autonomous mobile robots, and / or automated guided vehicles according to the present disclosure.

[0033] FIG. 5 shows a schematic drawing of a control system that can be used in the example workflows that integrate humanoid robots with one or more autonomous tuggers, autonomous mobile robots, and / or automated guided vehicles according to the present disclosure.DETAILED DESCRIPTION

[0034] Integrating humanoid robots to move totes (or boxes or other containers) to and from carts pulled by autonomous tuggers, whether automated guided vehicles (AGVs) or Autonomous Mobile Robots (AMRs), enables companies to fully autonomize tote-based manufacturing processes such as workcell delivery, lineside delivery, machine tending, workin-progress (WIP), and “supermarket” (stocking area) replenishment of carts by a humanoid robot moving totes to and from the carts that are being pulled behind an autonomous tugger, AMR or AGV.

[0035] There are several examples of workflows in which one or more humanoid robots can be integrated into a workflow that includes one or more autonomous tuggers, AGVs, and / or AMRs (each of which pulling one or more carts that carry totes of material, empty totes, or both). For example, an example workflow is a workcell delivery or lineside delivery process. In such workflows, humanoid robots can be integrated to transfer totes of material from one or more carts pulled by autonomous tugger(s), AGV(s), or AMR(s) to a line or workcell andtransfer empty totes from the line or workcell back to the cart(s). In this example, the humanoid robot can travel with the autonomous tugger, but more preferably remains in the area of the workcells or manufacturing lines and services one or more workcells or lines.

[0036] As another example workflow, humanoid robots can be integrated into a machine tending workflow. Like the workcell delivery and lineside delivery7process, one or more humanoid robots transfer totes of material from a line or workcell to one or more carts pulled by autonomous tugger(s). AGV(s) or AMR(s), and transfer empty totes from the cart(s) to the line or workcell. In this additional example and unlike the process where a human is driving a tugger, the humanoid robot may travel with the autonomous tugger, but more preferably remains in the area of the workcells or manufacturing lines and sen-ices one or more workcells or lines.

[0037] As another example workflow, humanoid robots can be integrated into a workin-progress or WIP workflow. In this example, humanoid robots can transfer totes of newly manufactured material by one line or workcell from that line or workcell to one or more carts pulled by autonomous tugger(s). AGV(s) or AMR(s). The autonomous tugger, whether AMR or AGV, drives to a new line or workcell, where other humanoid robots can transfer totes of newly manufactured material from a previous line or workcell from the cart(s) to a new line or workcell. In this example, humanoid robots transfer empty totes from the cart(s) to the first line or workcell in this process and also transfer empty totes from the second line or workcell in this process to the cart(s). As with the other examples and unlike a process where a human is driving a tugger, the humanoid robot may travel with the autonomous tugger, but more preferably remains in the area of the workcells or manufacturing lines and sendees one or more workcells or lines.

[0038] FIG. 1 is a schematic illustration of an example implementation of a workflow in a commercial environment 100 that includes an autonomous tugger, autonomous mobile robot or an automated guided vehicle) that pull carts with integrated humanoid robots according to the present disclosure. For example, commercial environment 100 can represent be “supermarket” workflow. In a supermarket workflow, for example, the environment 100 can include or be a stocking area of totes, where one or more autonomous tuggers (or autonomous mobile robot or automated guided vehicles) with a cart train drive in and humanoid robots transfer empty totes from the carts to an empty tote area and replenish the carts by transferring totes of new material from the supermarket area to the autonomous tugger carts. In some aspects, a supermarket or stocking area of totes can be in a manufacturing plant or in a distribution center or warehouse that is adjacent to the manufacturing plant. In the latter case,the train of carts pulled by an autonomous tugger, autonomous mobile robot or an automated guided vehicle is driven from the distribution center or warehouse into the manufacturing area where the delivery process begins. Humanoid robots remain in the supermarket or stocking area of totes, thereby servicing one or more areas where carts in autonomous tugger trains are replenished.

[0039] The workflow illustrated and described in FIG. 1 can represent such a supermarket workflow that integrates humanoid robots 112a and 1 12b (among others) and autonomous tuggers 108a-n to solve existing issues with existing case pick at height processes regardless of if those processes leverage humans along with conventional order picker lift trucks. The example workflow integrates humanoid robots 112a and 112b and autonomous tuggers 108a and 108b (among others) that pull or otherwise move one or more carts 114 that support or hold empty totes 107 or full totes 110. In the present disclosure, the term “tote” can refer to conventional totes as used in processes such as “supermarket” processes, workcell deliver}' or lineside delivery processes, machine tending processes, as well as other forms of containers (whether open ended or enclosed) such as boxes, crates, baskets, or otherwise.

[0040] In some aspects, one or more of the autonomous tuggers 108a and 108b are automated guided vehicles (AGVs) or autonomous mobile robots (AMRs) (and the present disclosure contemplates that the three types of autonomous machines can be used, e.g., interchangeably). In some aspects, autonomous tuggers based on AMRs can be distinguished from autonomous tuggers based on AGVs by. for example, a level of autonomy of movement within a volume 104 of a building structure 102 of the commercial environment. For example, an autonomous tugger based on an AMR can be capable of full autonomous movement through the volume 104 such as by mapping the volume 104 and freely moving through the volume 104 based on the mapping to pull one or more carts 114. An autonomous tugger based on an AGV, however, may, in some aspects, move through the volume 104 on a guide wire 113 that is positioned in the volume 104 (e.g., within a floor). Thus, the AGV may follow the guide wire 113 through aisles of the volume 104 but, when not in the aisles, will wait on any object around it to move instead of navigating on a different path.

[0041] As shown in FIG. 1, commercial environment 100 (such as a warehouse) includes or encloses the occupiable volume 104 in the building structure 102. The volume 104 can include empty tote area 106 (which can be one or multiple areas or enclosures) as well as full tote area 109 (which can be one or multiple areas or enclosures). Generally, empty tote area 106 stores (at least transiently) one or more empty totes 107, e.g., totes that do not hold or enclose any commercial product or material (which can be used to manufacture or form acommercial product). Full tote area 109 stores (at least transiently) one or more full totes 110, e.g., totes that hold or enclose commercial product or material (which can be used to manufacture or form a commercial product).

[0042] As shown in this example, one or more humanoid robots 112a and 112b are positioned in the volume 104, such as proximate to (e.g., within 1 ft., 2 ft., 3 ft., 4 ft., 5-10 ft., or otherwise) the respective tote areas 106 and 109. In some aspects, there can be a one-to-one ratio of humanoid robots 112a or 112b to a particular tote area. Alternatively, there can be more humanoid robots 112a or 112b than particular tote areas 106 or 109.

[0043] Generally, each humanoid robot 112a is operable to move, e.g., to remove one or more empty totes 107 from carts 114 that are then moved to an area near or adjacent the empty tote area 106 by the autonomous tugger 108a. The humanoid robot 112a can be commanded, e.g., by MES (manufacturing execution system) 999 (or autonomously operate), to remain out of a path of movement of the autonomous tugger 108a until the autonomous tugger 108a (and carts 114) is stationary at or near the empty tote area 106. At that time, the humanoid robot 112a can be commanded (or autonomously operate) to move empty totes 107 from the carts 114 to the empty tote area 106. Once the operation of moving the empty totes 107 to the empty tote area 106 is completed, the humanoid robot 112a can be commanded (or autonomously operate) to move out of a path of movement of the autonomous tugger 108a until the autonomous tugger 108a (and carts 114) has left the area at or near the empty tote area 106.

[0044] Likewise, each humanoid robot 112b is operable to move, e.g.. to remove one or more full totes 1 10 from full tote area 109 and place it on carts 114 that are moved to an area near or adjacent the full tote area 109 by the autonomous tugger 108b. The humanoid robot 112b can be commanded, e.g., by MES 999 (or autonomously operate), to remain out of a path of movement of the autonomous tugger 108b until the autonomous tugger 108b is stationary at or near the full tote area 109. At that time, the humanoid robot 112b can be commanded (or autonomously operate) to move full totes 110 from full tote area 109 to the carts 114. Once the operation of moving the full totes 110 to the carts 114 is completed, the humanoid robot 112b can be commanded (or autonomously operate) to move out of a path of movement of the autonomous tugger 108b until the autonomous tugger 108b (and carts 114) has left the area at or near the full tote area 109.

[0045] Turning briefly, to FIGS. 2A and 2B, these figures illustrate example implementations of a humanoid robot 112a (which can be the same as or similar to humanoid robot 112b) according to the present disclosure. As shown, the humanoid robot 112a can include atorso and lower body, with appendages (e.g., legs, feet, arms, hands, head) that mimicor otherwise resemble and functional similarly to the corresponding human body appendages. The humanoid robot 112a can include multifunction movement, such as: walking, squatting, bending at waist, kneeling, torso rotation, head rotation, lifting (from ground or another support surface such as a lift truck or mezzanine, to torso height, above head), carrying (e.g., empty tote 107 as shown in FIG. 2B) as well as other movements that mimic human natural movement). In addition, in some aspects, the humanoid robot 112a can include visual image sensing and recognition (e.g., built into the head or otherwise), radar, or lidar to detect objects within its path.

[0046] In this example workflow7, the commercial environment 100 can include the Manufacturing Execution System (MES) 999 that can interface with an ERP (enterprise resource planning) system. The MES 999 can be, for example, a microprocessor based control system that controls the operations of the humanoid robots 112a and 112b and autonomous tuggers 108a and 108b according to software instructions executable by the MES 999. In some aspects, the MES 999 controls operations of the humanoid robots 112a and 112b and autonomous tuggers 108a and 108b to move totes (107, 110, and others) to / from carts 114 and tote areas as described(e.g., in specified quantities and in a specified order of loading) in order to fulfill a commercial workflow or otherwise. In some aspects, MES 999 can be a physically separate control system that communicates (e.g., wired or wirelessly) with the humanoid robots 112a and 112b and autonomous tuggers 108a and 108b. Alternatively, some or all of the functionality (e.g., processing capability, memory storage, communications, software instructions) can be located in one or more of the humanoid robots 1 12a and 112b (such as, for example, within a head or torso of a humanoid robot). Thus, in some aspects, one or more of the humanoid robots 112a and 112b can act as the MES 999 to control the humanoid robots 112a and 112b and autonomous tuggers 108a and 108b.

[0047] In some aspects of an example workflow of commercial environment 100, the MES 999 can identify or register all of the humanoid robots 112a and 1 12b and autonomous tuggers 108a and 108b within the volume 104 (e.g., in order to determine which of the robots are activated or operable). The MES 999 can then communicate with autonomous tuggers 108a and 108b and direct them toward particular tote areas 106 and 109 to move the carts 114 to be loaded or unloaded. The above-described operations can be repeated at one or more tote areas 106 and 109 (e.g., at multiple tote areas 106 and multiple tote areas 109) within the volume 104.

[0048] FIGS. 3A and 3B are schematic illustrations of another example implementation of a workflow 300 in a commercial environment that integrates humanoid robots with one ormore autonomous tuggers, autonomous mobile robots and / or automated guided vehicles according to the present disclosure. For example, workflow 300 can be a machine tending process. In such a workflow 300, one or more humanoid robots 312 can be integrated to transfer totes 310 of material from one or more carts 308 pulled by an autonomous tugger, AGV, or AMR (not shown) to a machine line 302 or workcell 304. Humanoid robot 312 can also transfer empty totes 306 from the line 302 or workcell 304 back to the cart(s) 308. In this example, the humanoid robot 312 is operated (e.g., by an MES or autonomously, or a combination thereof) to remain at or near the line 302 and / or workcell 304 rather than traveling with an autonomous tugger. However, as the autonomous tugger (or AGV or AMR) and carts 308 (collectively, the “train"’) are moved to and away from the line 302 and / or workcell 304, the humanoid robot 312 can be operated (e.g.. by an MES or autonomously, or a combination thereof) to move away from the movement paths of the train.

[0049] FIGS. 4A-4C are schematic illustrations of another example implementation of a workflow 400 in a commercial environment that integrates humanoid robots with one or more autonomous tuggers, autonomous mobile robots and / or automated guided vehicles according to the present disclosure. For example, workflow 400 can be a workcell delivery or hneside delivery process. As shown in these figures, the example humanoid robot 402 comprises a torso 404 (e.g., with two arms and head) that is coupled to a hinge lift 406, which in turn is coupled to a mobile base 408 (in this example, a wheeled base). The torso 404 provides humanoid robot 402 with full, upper body movement capabilities as available to. e.g., humanoid robot 1 12a and 1 12b. At a "‘waist” of the torso 404, the hinge lift 406 provides for vertical raising and lowering of the torso 404 to allow the humanoid robot 402 to reach various distances away from a floor 401 of the workflow environment. For instance, in FIG. 4 A, the hinge lift 406 is completely retracted, allowing the torso 404 to reach one or more totes 412 on a lower (or lowest) shelf of carts 410 (pulled by an autonomous tugger, not shown). In FIG. 4B, the hinge lift 406 is in a first extended position (e.g., between a fully retracted and fully extended position), allowing the torso 404 to place one or more totes 412 at a workcell 414. In FIG. 4B, the hinge lift 406 is in a fully extended position, the torso 404 to reach one or more totes 412 on a higher (or highest) shelf of carts 410.

[0050] As shown in workflow 400, one or more humanoid robots 402 can be integrated to transfer totes 412 of material from one or more carts 410 pulled by an autonomous tugger or AGV to one or more workcells 414. Humanoid robot 402 can also transfer empty totes from the workcell 414 back to the cart(s) 410. In this example, the humanoid robot 402 is operated (e.g., by an MES or autonomously, or a combination thereof) to remain at or near the workcell414 rather than traveling with an autonomous tugger. However, as the autonomous tugger (or AGV or AMR) and carts 410 (collectively, the ‘"train”) are moved to and away from the workcell 414, the humanoid robot 402 can be operated (e.g., by an MES or autonomously, or a combination thereof) to move away from the movement paths of the train.

[0051] FIG. 5 shows a schematic drawing of a control system 500 that can be used in the example workflows of FIGS. 1, 3A-3B, and 4A-4C according to the present disclosure. For example, all or parts of the control system (or controller) 500 can be used for the operations described previously, for example as or as part of the Manufacturing Execution System (MES) 999. Some or all of the example control system 500 (or MES 900 generally) can be implemented as cloud-based system and / or service, alone or in combination with other portions of the example control system 500 that can be implemented at the commercial environment 100. The controller 500 is intended to include various forms of digital computers, such as printed circuit boards (PCB), processors, digital circuitry, or otherwise. Additionally, the system can include portable storage media, such as, Universal Serial Bus (USB) flash drives. For example, the USB flash drives may store operating systems and other applications. The USB flash drives can include input / output components, such as a wireless transmitter or USB connector that may be inserted into a USB port of another computing device.

[0052] The controller 500 includes a processor 510, a memory’ 520, a storage device 530, and an input / output device 540. Each of the components 510, 520, 530, and 540 are interconnected using a system bus 550. The processor 510 is capable of processing instructions for execution within the controller 500. The processor may be designed using any of a number of architectures. For example, the processor 510 may be a CISC (Complex Instruction Set Computers) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimal Instruction Set Computer) processor.

[0053] In one implementation, the processor 510 is a single-threaded processor. In another implementation, the processor 510 is a multi -threaded processor. The processor 510 is capable of processing instructions stored in the memory' 520 or on the storage device 530 to display graphical information for a user interface on the input / output device 540.

[0054] The memory 520 stores information within the control system 500. In one implementation, the memory' 520 is a computer-readable medium. In one implementation, the memory7520 is a volatile memory unit. In another implementation, the memory7520 is a nonvolatile memory unit.

[0055] The storage device 530 is capable of providing mass storage for the controller 500. In one implementation, the storage device 530 is a computer-readable medium. In variousdifferent implementations, the storage device 530 may be a floppy disk device, a hard disk device, an optical disk device, a tape device, flash memory, a solid state device (SSD), or a combination thereof.

[0056] The input / output device 540 provides input / output operations for the controller 500. In one implementation, the input / output device 540 includes a keyboard and / or pointing device. In another implementation, the input / output device 540 includes a display unit for displaying graphical user interfaces.

[0057] The features described can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The apparatus can be implemented in a computer program product tangibly embodied in an information carrier, for example, in a machine-readable storage device for execution by a programmable processor; and method steps can be performed by a programmable processor executing a program of instructions to perform functions of the described implementations by operating on input data and generating output. The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0058] Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of multiple processors of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer will also include, or be operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM. EEPROM, solid state drives (SSDs), and flash memory devices; magnetic disks such as internal hard disks andremovable disks; magneto-optical disks: and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by. or incorporated in, ASICs (application-specific integrated circuits).

[0059] To provide for interaction with a user, the features can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) or LED (light-emitting diode) monitor for displaying information to the user and a keyboard and a pointing device such as a mouse or a trackball by which the user can provide input to the computer. Additionally, such activities can be implemented via touchscreen flatpanel displays and other appropriate mechanisms.

[0060] The features can be implemented in a control system that includes a back-end component, such as a data server, or that includes a middleware component, such as an application server or an Internet server, or that includes a front-end component, such as a client computer having a graphical user interface or an Internet browser, or any combination of them. The components of the system can be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include a local area network (‘'LAN’’), a wide area network ('‘WAN’’), peer-to-peer networks (having ad-hoc or static members), grid computing infrastructures, and the Internet.

[0061] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular implementations of particular inventions. Certain features that are described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0062] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations,and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0063] A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. For example, example operations, methods, or processes described herein may include more steps or fewer steps than those described. Further, the steps in such example operations, methods, or processes may be performed in different successions than that described or illustrated in the figures. Accordingly, other implementations are within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. A robotic system, comprising: one or more humanoid robots configured to move within a commercial environment that includes a plurality of totes configured to hold at least one of commercial product or material; at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle configured to couple to and move one or more carts through the commercial environment, each of the one or more carts configured to support at least a portion of the totes; and a control system communicably coupled to the one or more humanoid robots and the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle, the control system configured to perform operations comprising: commanding the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts to a first location in the commercial environment; commanding the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts; and commanding the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts from the first location to a second location in the commercial environment.

2. The robotic system of claim 1, wherein the operations comprise commanding the one or more humanoid robots to move away from a travel path of the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle between the first location and the second location prior to commanding the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts from the first location to the second location.

3. The robotic system of either one of claims 1 or 2, wherein the operations comprise commanding the one or more humanoid robots at the first location to pick a specific number of totes from the one or more carts to leave at the first location.

4. The robotic system of any one of the previous claims, wherein the operations comprise commanding the one or more humanoid robots at the first location to place a specific number of totes from the first location on the one or more carts.

5. The robotic system of either one of claims 1 or 2, wherein the one or more humanoid robots at the first location are configured to place a specific number of totes from the first location on the one or more carts based on a visual inspection of the one or more carts by the one or more humanoid robots.

6. The robotic system of claim 5, wherein the visual inspection of the one or more carts by the one or more humanoid robots comprises a visual inspection of a type of the commercial product or material supported in the one or more totes to be placed from the first location on the one or more carts.

7. The robotic system of claim 5, wherein the one or more humanoid robots at the first location are configured to pick a specific number of totes from the one or more carts to leave at the first location based on a visual inspection of the one or more carts by the one or more humanoid robots.

8. The robotic system of claim 7, wherein the visual inspection of the one or more carts by the one or more humanoid robots comprises a visual inspection of a type of the commercial product or material supported in the one or more totes to be picked from the one or more carts and left at the first location.

9. The robotic system of any one of the previous claims, wherein the first location comprises a manufacturing line or workcell and the second location comprises a source of the material, and the operation of commanding the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts comprises: commanding the one or more humanoid robots at the first location to pick at least one tote that holds material from the one or more carts to leave at the first location; and commanding the one or more humanoid robots at the first location to place at least one empty tote from the first location on the one or more carts.

10. The robotic system of any one of the previous claims 1-8, wherein the first location comprises a manufacturing line or workcell and the second location comprises a source of the material, and the operation of commanding the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts comprises: commanding the one or more humanoid robots at the first location to pick at least one tote that holds material from the one or more carts to leave at the first location; and commanding the one or more humanoid robots at the first location to place at least one tote that holds commercial product from the first location on the one or more carts.

11. A computer-implemented method, comprising: registering, with a control system, one or more humanoid robots within a commercial environment that includes a plurality of commercial product and material; registering, with the control system, at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle configured to couple to and move one or more carts through the commercial environment, each of the one or more carts configured to support a plurality7of totes configured to hold at least one of commercial product or material; commanding, with the control system, the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts to a first location in the commercial environment; commanding, with the control system, the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts; and commanding, with the control system, the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts from the first location to a second location in the commercial environment.

12. The computer-implemented method of claim 11, comprising commanding, w ith the control system, the one or more humanoid robots to move away from a travel path of the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle between the first location and the second location prior to commanding the at least one autonomous tugger, autonomous mobile robot, or automated guided vehicle to move the one or more carts from the first location to the second location.

13. The computer-implemented method of either one of claims 11 or 12, comprising commanding, with the control system, the one or more humanoid robots at the first location to pick a specific number of totes from the one or more carts to leave at the first location.

14. The computer-implemented method of any one of the previous claims 11-13, comprising commanding, with the control system, the one or more humanoid robots at the first location to place a specific number of totes from the first location on the one or more carts.

15. The computer-implemented method of either one of claims 11 or 12, wherein the one or more humanoid robots at the first location are configured to place a specific number of totes from the first location on the one or more carts based on a visual inspection of the one or more carts by the one or more humanoid robots.

16. The computer-implemented method of claim 15, wherein the visual inspection of the one or more carts by the one or more humanoid robots comprises a visual inspection of a type of the commercial product or material supported in the one or more totes to be placed from the first location on the one or more carts.

17. The computer-implemented method of claim 15, wherein the one or more humanoid robots at the first location are configured to pick a specific number of totes from the one or more carts to leave at the first location based on a visual inspection of the one or more carts by the one or more humanoid robots.

18. The computer-implemented method of claim 17, wherein the visual inspection of the one or more carts by the one or more humanoid robots comprises a visual inspection of a type of the commercial product or material supported in the one or more totes to be picked from the one or more carts and left at the first location.

19. The computer-implemented method of any one of the previous claims 11-18, wherein the first location comprises a manufacturing line or workcell and the second location comprises a source of the material, and commanding, with the control system, the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts comprises: commanding, with the control system, the one or more humanoid robots at the first location to pick at least one tote that holds material from the one or more carts to leave at the first location; and commanding, with the control system, the one or more humanoid robots at the first location to place at least one empty tote from the first location on the one or more carts.

20. The computer-implemented method of any one of the previous claims 11-18, wherein the first location comprises a manufacturing line or workcell and the second location comprises a source of the material, and commanding, with the control system, the one or more humanoid robots at the first location to perform at least one of: pick at least one tote from the one or more carts to leave at the first location, or place at least one tote from the first location on the one or more carts comprises: commanding, with the control system, the one or more humanoid robots at the first location to pick at least one tote that holds material from the one or more carts to leave at the first location; and commanding, with the control system, the one or more humanoid robots at the first location to place at least one tote that holds commercial product from the first location on the one or more carts.

Images