Robot system with object handling mechanism for loading and unloading cargo carriers

Abstract

The robotic system may include a chassis operably coupled to a proximal conveyor, a first segment including a first segment conveyor extending along the length of the first segment, and a gripper including a distal conveyor extending along the length of the gripper. The robotic system may further include a controller configured to operate the chassis, conveyor, segment, gripper, or a combination thereof to remove and transport objects from a cargo loading structure such as a cargo container.

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 64 / 453,167, filed on Mar. 20, 2023, which is hereby incorporated by reference in its entirety.

[0002] Technical Field The present disclosure generally relates to robotic systems, and more particularly, to systems, processes, and techniques for object - handling mechanisms. Embodiments herein may relate to robotic systems for loading and / or unloading cargo carriers (e.g., shipping containers, trailers, box trucks, etc.).

Background Art

[0003] Due to constantly improving performance and falling costs, many robots (e.g., machines configured to automatically / autonomously perform physical actions) are currently widely used in a variety of different fields. For example, robots can be used to perform various tasks (e.g., manipulate or transfer objects through space) in manufacturing and / or assembly, packaging and / or wrapping, transportation and / or shipping, etc. When performing tasks, robots can reproduce human actions, thereby replacing or reducing the human involvement that would otherwise be required to perform dangerous or repetitive tasks.

[0004] The present disclosure generally relates to robotic systems, and more particularly, to systems, processes, and techniques for object - handling mechanisms. Embodiments herein may relate to robotic systems for loading and / or unloading cargo carriers (e.g., shipping containers, trailers, box trucks, etc.). However, despite technological advancements, in many cases, robots lack the sophistication required to reproduce the human interactions necessary to perform larger - scale and / or more complex tasks, such as transferring objects to / from cargo carriers. Accordingly, there is still a need for improved techniques and systems for managing the operations and / or interactions between robots.

[0005] The embodiments of the invention for carrying out the present technology are described and explained by using the accompanying drawings. [Brief explanation of the drawing]

[0006] [Figure 1] This illustrates an exemplary environment in which a robotic system with a cooperative transport mechanism can operate. [Figure 2] Block diagram showing a robotic system in one or more embodiments. [Figure 3] This is a perspective view of a robot system according to an embodiment of this technology. [Figure 4] Figure 3 is an enlarged side view of the robot system showing the operation of the support legs according to an embodiment of this technology. [Figure 5] Figure 3 is a side view of the robot system showing the vertical operation of a segment according to an embodiment of this technology. [Figure 6] Figure 3 is a top view of the robot system showing the horizontal movement of a segment according to an embodiment of this technology. [Figure 7] A and B are side views of the robot system in Figure 3, illustrating the vertical movement of the segment relative to the cargo carrier according to an embodiment of this technology. [Figure 8] This is a schematic side view of a robot system according to one or more embodiments. [Figure 9] This is a schematic top view of the robot system in the first state shown in Figure 8. [Figure 10] This is a schematic top view of the robot system in the second state shown in Figure 8. [Figure 11] This is a schematic diagram showing a robotic system located inside a cargo carrier according to one or more embodiments. [Figure 12A] This shows a robotic system in a first state of the process of unloading a cargo carrier, according to one or more embodiments. [Figure 12B] Figure 12A shows a robotic system in a second state of the process of unloading a cargo carrier, according to one or more embodiments. [Figure 12C] Figure 12A shows a robotic system in a second state of the process of unloading a cargo carrier, according to one or more embodiments. [Figure 12D] Figure 12A is a perspective view showing a robotic system in a second state of the process of unloading a cargo carrier, according to one or more embodiments. [Figure 12E] Figure 12A shows a robotic system in a third state of the process of unloading a cargo carrier, according to one or more embodiments. [Figure 12F] Figure 12A shows a robotic system in a fourth state of the process of unloading a cargo carrier, according to one or more embodiments. [Figure 13] This flowchart illustrates the process of operating a robotic system according to one or more embodiments. [Figure 14] This flowchart illustrates the process of operating a robotic system according to one or more embodiments. [Figure 15] This is a schematic side view of a gripper for a robotic system according to one or more embodiments. [Figure 16] Figure 15 is a schematic diagram of the top view of the gripper. [Figure 17A] This is a schematic diagram showing a first state of the process of operating a robot system according to one or more embodiments. [Figure 17B] This is a schematic diagram showing a second state of the process of operating a robot system according to one or more embodiments. [Figure 17C] This is a schematic diagram showing a third state of the process of operating a robot system according to one or more embodiments. [Figure 17D] This is a schematic diagram showing a fourth state of the process of operating a robot system according to one or more embodiments. [Figure 17E] This is a schematic diagram showing a fifth state of the process of operating a robot system according to one or more embodiments. [Figure 17F] This is a schematic diagram showing a sixth state of the process of operating a robot system according to one or more embodiments. [Figure 18] This flowchart illustrates the process of operating a robotic system according to one or more embodiments. [Figure 19]Perspective view of a robot system according to an embodiment of the present technology. [Figure 20] Enlarged side view of the robot system of FIG. 19 according to an embodiment of the present technology. [Figure 21] Perspective view of the robot system of FIG. 19 on the floor of a warehouse according to an embodiment of the present technology. [Figure 22] Enlarged side view of the robot system of FIG. 19 showing the operation of the support legs according to an embodiment of the present technology. [Figure 23] Enlarged side view of the robot system of FIG. 19 showing the operation of the support legs according to an embodiment of the present technology. [Figure 24] Enlarged perspective view of the front wheel of the robot system of FIG. on the floor of a warehouse according to an embodiment of the present technology. [Figure 25] Enlarged perspective view of the rear support leg of the robot system of FIG. 19 according to an embodiment of the present technology. [Figure 26] Perspective view of a chassis joint for a robot system according to one or more embodiments. [Figure 27] Flow diagram showing a process for operating a robot system according to one or more embodiments. [Figure 28] Front view of a robot system and a chassis joint in a first state according to one or more embodiments. [Figure 29] Front view of the robot system and the chassis joint of FIG. 28 in a second state. [Figure 30] Flow diagram showing a process for operating a robot system according to one or more embodiments. [[ID=3!]] [Figure 31] Partial schematic isometric view of a robot system configured according to some embodiments of the present technology. [Figure 32A] Partial schematic upper side view of an end effector configured according to some embodiments of the present technology. [Figure 32B] Partial schematic lower side view of an end effector configured according to some embodiments of the present technology. [[ID=!3]] [Figure 33A]Figure 32A is a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of the present technology. [Figure 33B] Figure 32A is a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of the present technology. [Figure 33C] Figure 32A is a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of the present technology. [Figure 33D] Figure 32A is a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of the present technology. [Figure 33E] Figure 32A is a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of the present technology. [Figure 33F] Figure 32A is a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of the present technology. [Figure 34] This is a partial schematic top side view of an end effector configured according to several embodiments of the present technology. [Figure 35] This is a partial schematic side view of a gripping component for an end effector configured according to several embodiments of the present technology. [Figure 36A] Figure 34 shows a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of this technology. [Figure 36B] Figure 34 shows a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of this technology. [Figure 36C]Figure 34 shows a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of this technology. [Figure 36D] Figure 34 shows a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of this technology. [Figure 36E] Figure 34 shows a partial schematic side view of an end effector of the type shown in the figure, at various stages of the process for picking up an object according to some embodiments of this technology. [Figure 37] This is a flowchart of the process for picking up a target object according to several embodiments of this technology. [Figure 38] A and B are partial schematic upper side views showing additional features in the terminal region of an end effector configured according to some embodiments of the present technology. [Figure 39A] These are a partial schematic top view and a schematic upper side view, respectively, of an end effector configured according to several embodiments of the present technology. [Figure 39B] These are a partial schematic top view and a schematic upper side view, respectively, of an end effector configured according to several embodiments of the present technology. [Figure 40] This is a partial schematic upper side view of a distal joint for a robotic system configured according to several embodiments of this technology. [Figure 41] This is a partial schematic lower side view of a distal joint for a robotic system configured according to several embodiments of this technology. [Figure 42] Figures A and B are partial schematic side views of a distal joint for a robotic system configured according to several embodiments of this technology. [Figure 43A] This is a partial schematic top view of a distal joint for a robotic system configured according to several embodiments of this technology. [Figure 43B]This is a partial schematic top view of a distal joint for a robotic system configured according to several embodiments of this technology. [Figure 43C] This is a partial schematic top view of a distal joint for a robotic system configured according to several embodiments of this technology. [Figure 43D] This is a partial schematic bottom view of a distal joint for a robotic system configured according to several embodiments of this technology. [Figure 44A] These are partial schematic side views of a distal joint of the type shown in Figures 43A to 43C, configured according to several embodiments of the present technology. [Figure 44B] These are partial schematic side views of a distal joint of the type shown in Figures 43A to 43C, configured according to several embodiments of the present technology. [Figure 44C] These are partial schematic side views of a distal joint of the type shown in Figures 43A to 43C, configured according to several embodiments of the present technology. [Figure 45] Figure 40 is a partial schematic top side view of a connection management feature in a distal joint of the type shown, according to several embodiments of the present technology. [Figure 46] Figure 45 shows a partial schematic cross-sectional view of a connection management feature of the type shown, according to several embodiments of this technology. [Figure 47] This is a partial schematic isometric view of a drive component for a gripping component configured according to several embodiments of the present technology. [Figure 48] This is a partial schematic isometric view of a branching component of a drive component configured according to several embodiments of this technology. [Figure 49] This is a partial schematic isometric drawing showing additional details regarding the drive components for gripping components according to some embodiments of the present technology. [Figure 50] Various images illustrating the visual processing of object arrangement according to one or more embodiments are shown. [Figure 51] The following are various images illustrating the visual processing of unrecognized objects after object removal, according to one or more embodiments. [Figure 52] Various images are shown illustrating visual processing for verifying unrecognized objects according to one or more embodiments. [Figure 53] This shows various images illustrating target selection for unrecognized objects according to one or more embodiments. [Figure 54] Images A and B show grip calculations for objects rotated according to one or more embodiments. [Figure 55] A top view of an environment showing the alignment of rotated, unrecognized objects according to one or more embodiments. [Figure 56] This is a top view of an environment for demonstrating object gripping calculations according to one or more embodiments. [Figure 57] This is a flowchart of a method for picking up an object according to some embodiments of this technology. [Figure 58A] This is an illustrative diagram of a support detection process for an unrecognized object according to one or more embodiments. [Figure 58B] This is an illustrative diagram of a support detection process for an unrecognized object according to one or more embodiments. [Figure 58C] This is an illustrative diagram of a support detection process for an unrecognized object according to one or more embodiments. [Figure 58D] This is an illustrative diagram of a support detection process for an unrecognized object according to one or more embodiments. [Figure 58E] This is an illustrative diagram of a support detection process for an unrecognized object according to one or more embodiments. [Figure 59] This is a flowchart of a method for detecting a new object from an unrecognized area, according to some embodiments of the present technology. [Figure 60A] This is an illustrative diagram of the object selection rules according to one or more embodiments of this technology. [Figure 60B] This is an illustrative diagram of the object selection rules according to one or more embodiments of this technology. [Figure 60C] This is an illustrative diagram of the object selection rules according to one or more embodiments of this technology. [Figure 60D] This is an illustrative diagram of the object selection rules according to one or more embodiments of this technology. [Figure 61] This is a flowchart of a method for evaluating selection criteria for picking up objects according to some embodiments of the present technology. [Figure 62] This is a front view of an environment for demonstrating support and grip calculations for an unrecognized object according to one or more embodiments. [Figure 63] This is a flowchart illustrating a method for deriving a stable gripping posture for transporting an object according to some embodiments of this technology. [Figure 64] Figures A to B are illustrative diagrams of support object verification for an object transfer process according to one or more embodiments. [Figure 65] This is a flowchart of a method for verifying the spatial conditions for picking up an object according to some embodiments of this technology. [Figure 66] This is a flowchart of a method for monitoring real-time performance for picking up objects according to some embodiments of this technology.

[0007] The art described herein will become more apparent to those skilled in the art by examining the embodiments for carrying out the invention in conjunction with the drawings. Embodiments or representations illustrating aspects of the invention are shown as examples, and the same reference may indicate similar elements. Although the drawings show various embodiments for illustrative purposes, those skilled in the art will recognize that alternative embodiments can be used without departing from the principles of the art. Thus, although certain embodiments are shown in the drawings, the art is open to various modifications. [Modes for carrying out the invention]

[0008] The disclosed technology includes methods, apparatus, and systems for robotic handling of objects. Specifically, according to some embodiments herein, the disclosed technology includes, but is not limited to, methods, apparatus, and systems for robotic loading and unloading of cargo carriers, including transport containers, trailers, cargo beds, and box trucks. Conventional processes for loading and unloading cargo carriers are extremely laborious. Typically, cargo carriers are loaded or unloaded through manual labor or using human-operated tools (e.g., pallet jacks). Thus, this process is time-consuming and expensive, and such processes require repetitive and physically very strenuous work. Previous attempts to automate parts of the loading or unloading process have certain disadvantages that have prevented them from being widely adopted.

[0009] Many existing robotic systems cannot compensate for the variability of packaging patterns and object sizes within cargo carriers, such as in the handling of mixed stockkeeping units (SKUs). For example, cargo carriers packed with irregularly sized boxes often cannot be automatically (i.e., without human input / effort) removed in a regular or repeating pattern. Here, we introduce a robotic system configured to automatically / autonomously load / unload cargo carriers packed with objects of irregular size and orientation, such as mixed SKU boxes. As further described herein, the robotic system may use a vision system to reliably recognize irregularly sized objects and, based on that recognition, control end-of-arm tools (EOATs), including grippers.

[0010] Furthermore, many existing robotic systems require the replacement or adjustment of existing infrastructure in loading / unloading areas of warehouses or other distribution centers (e.g., truck bays). Often, existing warehouses have conveyor systems for moving objects through the warehouse. Typically, objects are removed from such conveyors and manually placed into cargo carriers for loading. Conversely, objects may be manually placed onto conveyors after being manually removed from cargo carriers for unloading. Conventional automated devanning / loading solutions often require the adjustment of existing warehouse systems (e.g., conveyors) for the corresponding interfaces. Thus, while existing infrastructure exists within warehouses or other distribution centers, the gaps between cargo carriers and their infrastructure are currently filled manually or require physical adjustment. Existing robotic systems may require the replacement or removal of such pre-existing infrastructure, increasing the cost and time required to implement the robotic system. As further described herein, robotic systems may include chassis configured to integrate with existing infrastructure within warehouses or other distribution centers. In this way, the robotic systems according to the embodiments herein can be adapted to existing infrastructure within a warehouse or distribution center in some embodiments.

[0011] In some embodiments, the robotic system may be configured to load or unload cargo carriers automatically or semi-automatically. In some embodiments, the robotic system may use computer vision and other sensors to control the operation of various components of the robotic system. In some embodiments, the robotic system may include a gripper comprising at least one suction cup and at least one conveyor. The at least one suction cup may be configured to grip an object when a vacuum is applied to at least one suction cup, and the conveyor may be configured to move the object proximal after it has been gripped by at least one suction cup. The robotic system may also include one or more sensors configured to acquire information (e.g., two-dimensional (2D) and / or three-dimensional (3D) image data) including a plurality of objects stored within the cargo carrier (e.g., within the crown and / or front surface of the cargo carrier). For example, the sensor may include (1) one or more cameras configured to acquire one or more visual spectral images of an object in a cargo carrier, (2) one or more distance sensors (e.g., light detection and ranging (lidar) sensors) configured to measure the distance from one or more distance sensors to multiple objects, or a combination thereof.

[0012] Many conventional computer vision approaches are computationally intensive and prone to errors in dynamic and variable environments. For example, in the case of boxes, boxes may have different colors, labels, orientations, sizes, etc., making it difficult for computer vision alone to reliably identify the boundaries of boxes within a cargo container. Therefore, in some embodiments, a robotic system may include a local controller configured to receive both image information and information from one or more distance sensors in order to more consistently identify objects within a cargo carrier for removal by the robotic system in a less computationally intensive manner. The local controller may include one or more processors and memory. The local controller may receive image information from at least one vision sensor that images multiple objects. Based on the image, the local controller may identify a minimum viable region (MVR) corresponding to a first object among the multiple objects. The MVR may be an area of ​​the image corresponding to a single object with high confidence. In other words, when an area in an image does not sufficiently match a known object in the master data, the MVR can represent a portion of the unrecognized image area that (1) has a sufficient probability (e.g., by a given threshold) of belonging to a single object, and / or (2) corresponds to the smallest operable or graspable area. In some cases, the MVR may be assigned based on the smallest known dimensions of an object in the cargo carrier. In other embodiments, the MVR may be assigned by one or more computer vision algorithms with error limits. The MVR may be smaller than the size of an object among multiple objects. After assigning the MVR, the controller may instruct the gripper to grasp the unrecognized object using the corresponding MVR, for example, by applying a vacuum to at least one suction cup and making contact and grasping with the MVR. After the first object has been grasped, the controller may further instruct the gripper to lift the first object to create a gap or separation between the grasped object and the object below. The controller may then receive from one or more sensors (e.g., sensors in the EOAT) indicating the area below the MVR.Based on these sensor outputs, the controller can identify the lower boundary of the lifted object. Similarly, the controller can also acquire multiple distance measurements in the horizontal direction. Based on these sensor outputs, the controller can identify the side boundary of the object. Using the identified boundary, the controller can update the dimensions (e.g., width and height) and / or actual edges of an object that was not previously recognized, and the object may be removed from multiple objects. The controller can then proceed to operate on a different / new object based on the remaining image by removing the area defined by the MVR and / or the updated edges from the previously acquired image (e.g., from a different system image sensor). In this way, the operation of the robot system is obtained by acquiring a single image from the first sensor of all objects to be removed, and the operation can continue by acquiring a new image for each removed object and subtracting the area from the original image without processing. Such a configuration may be particularly effective when objects are arranged in multiple vertical layers, as objects behind previously removed objects cannot be mistakenly identified as the next to be removed.

[0013] Systems and methods for robotic systems having a cooperative transport mechanism are described herein. Robotic systems configured according to several embodiments (e.g., integrated systems of devices, each performing one or more designated tasks) autonomously perform an integrated task by coordinating the actions of multiple units (e.g., robots).

[0014] Some details describing structures or processes, which are well known and often associated with robotic systems and subsystems, but which could unnecessarily obscure some important aspects of the disclosed technology, are omitted in the following description for clarity. Furthermore, while the following disclosure shows several embodiments of different aspects of the technology, some other embodiments may have different configurations or components than those described in this section. Thus, the disclosed technology may have other embodiments that have additional elements or lack some of the elements described below.

[0015] term Many embodiments or aspects of the Disclosure described below may take the form of computer-executable instructions or controller-executable instructions, which include routines executed by a programmable computer or controller. Those skilled in the art will understand that the disclosed technology may be implemented on computer or controller systems other than those shown and described below. The technology described herein may be embodied in a dedicated computer or data processor that is specifically programmed, configured, or constructed to execute one or more of the computer-executable instructions described below. Thus, the terms “computer” and “controller” as used herein refer to any data processor and may include Internet devices and handheld devices (including palmtop computers, wearable computers, mobile phones or mobile phones, multiprocessor systems, processor-based or programmable home appliances, network computers, minicomputers, etc.). Information processed by these computers and controllers may be presented on any suitable display medium, including liquid crystal displays (LCDs). Instructions for executing computer-executable tasks or controller-executable tasks may be stored on any suitable computer-readable medium, including hardware, firmware, or a combination of hardware and firmware. The instructions can be contained in any suitable memory device, including, for example, a flash drive, a USB device, and / or other suitable media.

[0016] The following sections provide numerous specific details to enable a full understanding of the technology disclosed herein. In other embodiments, the technology described herein can be practiced without these specific details. In other cases, well-known features (e.g., specific functions or routines) are not described in detail to avoid unnecessarily obscuring this disclosure. References to “embodiments,” “one embodiment,” etc., in the modes for carrying out this invention mean that the specific features, structures, materials, or properties described are included in at least one embodiment of this disclosure. Therefore, not all occurrences of such phrases in this specification necessarily refer to the same embodiment. On the other hand, such references are not necessarily mutually exclusive. Furthermore, specific features, structures, materials, or properties can be combined in any preferred manner in one or more embodiments. It should be understood that the various embodiments shown in the figures are merely illustrative and are not necessarily drawn to scale.

[0017] Any reference in this disclosure to “one embodiment” or “several embodiments” means that the particular features, functions, structures, or characteristics described are included in at least one embodiment. Where such phrases appear, they do not necessarily refer to the same embodiment, nor do they necessarily refer to mutually exclusive alternative embodiments.

[0018] Unless otherwise explicitly required by the context, the terms “comprise,” “comprising,” and “comprised of” should be interpreted in a comprehensive sense, not an exclusive or comprehensive sense; that is, “including, but not limited to.” The term “based on” should also be interpreted in a symbiotic sense; therefore, the term “based on” is intended to mean “based on, at least partially.”

[0019] The terms “coupled” and “connected,” along with their derivatives, may be used herein to describe structural relationships between constituent elements. It should be understood that these terms are not intended to be synonymous with one another. Rather, in certain embodiments, “connected” may be used to indicate that two or more elements are in direct contact with one another. Unless otherwise indicated in the context, the term “coupled” may be used to indicate that two or more elements are in direct or indirect contact with one another (with other intervening elements between them), or that two or more elements are linked or interacting with one another (for example, in a causal relationship such as the transmission / reception of a signal or a function call), or both.

[0020] The term "module" can broadly refer to software, firmware, hardware, or a combination thereof. A module is typically a functional component that generates one or more outputs based on one or more inputs. A computer program may contain or utilize one or more modules. For example, a computer program may utilize multiple modules, each responsible for completing different tasks, or it may utilize a single module responsible for completing all tasks.

[0021] When used to refer to a list of multiple items, the word "or" is intended to cover all of the following interpretations: any of the items in the list, all of the items in the list, and any combination of the items in the list.

[0022] Embodiments of the present disclosure are described in detail herein with reference to the accompanying drawings. Through several figures in which exemplary embodiments are shown, similar figures represent similar elements. However, embodiments of the claims can be embodied in many different forms and should not be construed as being limited to embodiments described herein. The examples described herein are non-limiting examples and, among other possible examples, are merely examples.

[0023] Throughout this specification, multiple examples (e.g., “610”) may implement components, operations, or structures (e.g., “610a”) described as single examples. Furthermore, multiple examples (e.g., “610”) collectively refer to sets of components, operations, or structures (e.g., “610a”) described as single examples. Unless otherwise indicated, the description of a single component (e.g., “610a”) is equivalent to a similarly numbered component (e.g., “610b”). These and other aspects, features, and embodiments may be expressed as methods, apparatus, systems, components, program products, means, or steps for performing a function, and in other ways. These and other aspects, features, and embodiments will become apparent from the following description, including the claims.

[0024] For ease of reference, robot systems and their components may be described herein with reference to the top and bottom, upper and lower, upward and downward, and / or horizontal, xy-plane, vertical, or z-plane, with respect to the spatial orientation of the embodiments shown in the drawings. However, it should be understood that robot systems and their components can be moved to and used in different spatial orientations without changing the structure and / or function of the embodiments disclosed in this Art.

[0025] Furthermore, embodiments of this specification may refer to various translational and rotational degrees of freedom. "Translation" may refer to a linear change in position along an axis. "Rotation" may refer to an angular change in orientation along an axis. "Orientation" may refer to a combination of position and orientation within a reference frame. The degrees of freedom described herein may refer to various reference frames, including global reference frames (e.g., referring to the direction of gravity) or local reference frames (e.g., referring to local orientations or dimensions such as longitudinal dimensions, referring to a cargo carrier, referring to the vertical plane of an object within a cargo carrier, or referring to the local environment of a robotic system). Rotational degrees of freedom may be called "roll," "pitch," and "yaw," and may be based on local reference frames, such as with respect to the longitudinal plane and / or cross-section of various components of a robotic unit (e.g., the longitudinal plane and / or cross-section of a chassis). For example, “roll” may refer to rotation around a longitudinal axis that is at least generally parallel to the longitudinal plane of the chassis; “pitch” may refer to rotation around a transverse axis perpendicular to the longitudinal axis that is at least generally parallel to the cross-section of the chassis; and “yaw” may refer to rotation around a second transverse axis that is perpendicular to both the longitudinal and transverse axes, and / or perpendicular to both the longitudinal plane and cross-section of the chassis and / or gripper. In some embodiments, the longitudinal axis may be aligned with proximal and distal directions. In some cases, “proximal” may refer to the direction away from the cargo carrier, and “distal” may refer to the direction toward the cargo carrier. Overview of an exemplary robotic system

[0026] Figure 1 shows an exemplary environment in which a robot system 100 having a cooperative transport mechanism may operate. The robot system 100 includes and / or can communicate with one or more units (e.g., robots) configured to perform one or more tasks. Embodiments of the cooperative transport mechanism can be implemented or carried out by various units.

[0027] In the example shown in Figure 1, the robot system 100 may include an endpoint unit 102 such as a truck loader / unloader, a transfer unit 104 (e.g., a palletizing robot and / or piece picker robot), a transport unit 106, a storage interface unit 108, or a combination thereof, at a warehouse or distribution / shipping hub. Each unit within the robot system 100 can be configured to perform one or more tasks. Tasks can be combined in sequence to perform actions that achieve a goal, such as unloading objects from a cargo carrier (e.g., a truck, cargo container, or van) and storing the objects in a warehouse, or unloading objects from a storage location and preparing the objects for shipment (e.g., by loading them into cargo). In some embodiments, a task may include placing objects in a target location (e.g., on top of a conveyor or inside a cargo carrier). As will be described in detail below, the robot system 100 may derive individual placement locations / orientations, calculate corresponding motion plans, or a combination thereof, for loading and / or unloading objects. Each unit can be configured to perform a set of actions to accomplish a task (for example, manipulating one or more of its components).

[0028] In some embodiments, a task may involve manipulating an object 112 (e.g., one of the packages, boxes, cases, cages, pallets, etc., corresponding to the task being performed) from a start / transfer location 114 to a task / transfer location 116 (e.g., moving and / or reorienting it). For example, an endpoint unit 102 (e.g., a devanning robot) may be configured to transfer the object 112 from a location on a carrier (e.g., a truck) to a location on a conveyor. Alternatively, a transfer unit 104 may be configured to transfer the object 112 from one location (e.g., a conveyor, pallet, or container) to another location (e.g., a pallet, container, etc.). In another example, a transfer unit 104 (e.g., a palletizing robot) may be configured to transfer the object 112 from a source location (e.g., a pallet, a pickup area, and / or a conveyor) to a destination pallet. Once the operation is complete, the transport unit 106 (e.g., a conveyor, an automated guided vehicle (AGV), a shelf transport robot, etc.) can transport the object 112 from the area associated with the transport unit 104 to the area associated with the storage interface unit 108, and the storage interface unit 108 can transport the object 112 from the transport unit 104 to the storage location (e.g., a location on a shelf) (e.g., by moving the pallet carrying the object 112).

[0029] For illustrative purposes, the robotic system 100 is described in the context of a packaging center and / or warehouse, but it should be understood that the robotic system 100 can be configured to perform tasks in other environments / for other purposes such as manufacturing, assembly, storage / inventory, medical, and / or other types of automation. It should also be understood that the robotic system 100 may include other units such as manipulators, service robots, and modular robots, which are not shown in Figure 1. For example, in some embodiments, the robotic system 100 may include a depalletizing unit for transferring objects from a cage cart or pallet to a conveyor or other pallet, a container switching unit for transferring objects from one container to another, a packaging unit for wrapping / boxing objects, a sorting unit for grouping objects according to one or more of its characteristics, a piece picking unit for manipulating objects differently according to one or more of its characteristics (e.g., for sorting, grouping, and / or transferring), or a combination thereof.

[0030] Figure 2 is a block diagram showing a robot system 100 according to one or more embodiments. In some embodiments, for example, the robot system 100 (e.g., in one or more of the units and / or robots described above) may include electronic / electrical devices such as one or more processors 202, one or more storage devices 204 (e.g., non-temporary memory), one or more communication devices 206, one or more input / output devices 208, one or more actuator devices 212, one or more transport motors 214, one or more sensors 216, or a combination thereof. Various devices can be coupled to one another via wired and / or wireless connections (e.g., system communication path 218). For example, the robot system 100 may include buses such as a system bus, Peripheral Component Interconnect (PCI) bus or PCI-Express bus, Hypertransport or Industry Standard Architecture (ISA) bus, Small Computer System Interface (SCSI) bus, Universal Serial Bus (USB), IIC (I2C) bus, or IEEE (Institute of Electrical and Electronics Engineers) Standard 1394 bus (also known as "FireWire"). Furthermore, for example, the robot system 100 may include bridges, adapters, processors, or other signal-related devices to provide wired connections between devices. Wireless connections may be based on, for example, cellular communication protocols (e.g., 3G, 4G, LTE, 5G, etc.), wireless local area network (LAN) protocols (e.g., Wireless Fidelity (Wi-Fi)), peer-to-peer or device-to-device communication protocols (e.g., Bluetooth, Near Field Communication (NFC), etc.), Internet of Things (IoT) protocols (e.g., NB-IoT, LTE-M, etc.), and / or other wireless communication protocols.

[0031] The processor 202 may include a data processor (e.g., a central processing unit (CPU), a dedicated computer, a graphics processing unit (GPU), and / or an onboard server) configured to execute instructions (e.g., software instructions) stored in a memory device 204 (e.g., computer memory). In some embodiments, the processor 202 may be included in a separate / standalone controller operably coupled to other electronic / electrical devices shown in Figure 2 and / or the robotic unit shown in Figure 1. The processor 202 may implement program instructions for controlling / interfacing with other devices, thereby causing the robotic system 100 to perform actions, tasks, and / or operations.

[0032] The storage device 204 may include a non-temporary computer-readable medium on which program instructions (e.g., software 210) are stored. Some examples of the storage device 204 include volatile memory (e.g., cache and / or random access memory (RAM)) and / or non-volatile memory (e.g., flash memory and / or magnetic disk drives). Other embodiments of the storage device 204 may include portable memory and / or cloud storage devices.

[0033] In some embodiments, a storage device 204 can be used to further store processing results and / or predetermined data / thresholds and provide access to them. For example, the storage device 204 can store master data 252, which includes descriptions of objects (e.g., boxes, cases, and / or products) that can be operated by the robot system 100. In one or more embodiments, the master data 252 may include dimensions, shape (e.g., templates for potential poses and / or computer-generated models for recognizing objects in different poses), color scheme, image, identification information (e.g., barcodes, Quick Response (QR) codes®, logos, etc., and / or their expected locations), expected weight, other physical / visual characteristics, or a combination thereof, of the objects expected to be operated by the robot system 100. In some embodiments, the master data 252 may include operation-related information about the objects, such as the center of gravity (CoM) position of each object, expected sensor measurements corresponding to one or more actions / operations (e.g., force, torque, pressure, and / or contact measurements), or a combination thereof.

[0034] The communication device 206 may include circuitry configured to communicate with external or remote devices over a network. For example, the communication device 206 may include a receiver, transmitter, modulator / demodulator (modem), signal detector, signal encoder / decoder, connector port, network card, etc. The communication device 206 may be configured to transmit, receive, and / or process electrical signals according to one or more communication protocols (e.g., Internet Protocol (IP), wireless communication protocols, etc.). In some embodiments, the robot system 100 may use the communication device 206 to exchange information between units of the robot system 100 and / or to exchange information with systems or devices outside the robot system 100 (e.g., for reporting, data collection, analysis, and / or troubleshooting purposes).

[0035] The input / output device 208 may include a user interface device configured to transmit information to and / or receive information from a human operator. For example, the input / output device 208 may include a display 250 and / or other output devices (e.g., a speaker, a tactile circuit, or a tactile feedback device) to transmit information to a human operator. The input / output device 208 may also include control or receiving devices such as a keyboard, mouse, touchscreen, microphone, user interface (UI) sensors (e.g., a camera for receiving motor commands), or a wearable input device. In some embodiments, the robot system 100 may use the input / output device 208 to interact with a human operator when performing actions, tasks, operations, or a combination thereof.

[0036] The robot system 100 may include physical or structural members (e.g., robot manipulator arms) connected by joints for motion (e.g., rotational displacement and / or translational displacement). The structural members and joints may form a kinetic chain configured to operate end effectors (e.g., grippers and / or EOATs) configured to perform one or more tasks (e.g., gripping, rotation, welding, etc.) depending on the application / operation of the robot system 100. The robot system 100 may include actuation devices 212 (e.g., motors, actuators, wires, artificial muscles, electroactive polymers, etc.) configured to drive or operate (e.g., displace and / or reorient) the structural members around or at the corresponding joints. In some embodiments, the robot system 100 may include transport motors 214 configured to transport the corresponding units / chassis to various locations.

[0037] The robot system 100 may include sensors 216 configured to acquire information used for performing tasks such as manipulating structural members and / or transporting robot units. Sensors 216 may include devices configured to detect or measure one or more physical properties of the robot system 100 (e.g., the condition, state, and / or location of one or more structural members / their joints) and / or one or more physical properties of the surrounding environment. Some embodiments of sensors 216 may include accelerometers, gyroscopes, force sensors, strain gauges, tactile sensors, torque sensors, position encoders, cross sensors, and the like.

[0038] In some embodiments, for example, sensor 216 may include one or more visual sensors 222 configured to detect the surrounding environment (e.g., visual cameras and / or infrared cameras, 2D and / or 3D imaging cameras, LiDAR, or distance measuring devices such as radar). The visual sensors 222 can generate a representation of the detected environment, such as digital images and / or point clouds, which can be processed via machine / computer vision (e.g., for automated inspection, robot guidance, or other robotic applications). As will be described in more detail below, the robotic system 100 can process the digital images and / or point clouds (e.g., via processor 202) to identify the object 112 in Figure 1, the starting location 114 in Figure 1, the task location 116 in Figure 1, the posture of the object 112, a confidence scale for the starting location 114 and / or posture, or a combination thereof.

[0039] To manipulate the target object 112, the robot system 100 can acquire and analyze image data of a designated area (e.g., a pickup location such as inside a truck or on a conveyor belt) to identify the target object 112 and its starting location 114. Similarly, the robot system 100 can acquire and analyze image data of another designated area (e.g., a transfer location for placing an object on a conveyor, a location for placing an object in a container, or a location on a pallet for stacking purposes) to identify the task location 116. For example, the vision sensor 222 may include one or more cameras configured to generate image data of the pickup area and / or one or more cameras configured to generate image data of the task area (e.g., the transfer area). Based on the image data, the robot system 100 can determine the starting location 114, the task location 116, the associated orientation, and / or other processing results, as described below.

[0040] In some embodiments, for example, the sensor 216 may include a system sensor 224 (e.g., a position encoder, a potentiometer, etc.) configured to detect the position of structural members (e.g., robot arms and / or end effectors) and / or corresponding joints of the robot system 100. The robot system 100 can use the position sensor 224 to track the position and / or orientation of the structural members and / or joints during task execution. Furthermore, the system sensor 224 may include sensors such as cross sensors configured to track the location / movement of transported objects.

[0041] Overview of an exemplary endpoint interface system Figure 3 is a perspective view of a robot system 300 according to an embodiment of the present technology. The robot system 300 may be an example of the robot system 100 (e.g., endpoint unit 102) shown in and described in relation to Figure 1. In the embodiment shown, the robot system 300 includes a chassis 302, a conveyor arm or first segment 304 ("first segment") coupled to the chassis 302 and extending toward the distal portion 301a of the robot system 300, a second segment 321 coupled to the chassis 302 and extending toward the proximal portion 301b of the robot system 300, and a gripper 306 coupled to the first segment 304 at the distal portion 301a. As described above with respect to robot system 100, robot system 300 can be configured to perform one or more tasks to perform operations that achieve objectives such as unloading objects from a cargo carrier (truck or van) and storing the objects in a warehouse, or unloading objects from a storage location and preparing them for shipment (e.g., loading objects onto a cargo carrier). For example, in some embodiments, robot system 300 can be positioned such that the second segment 321 is adjacent to a warehouse conveyor (e.g., a conveyor that is previously / already in place within the operating environment). The object can then be transported toward or toward the gripper 306 along a path formed by the warehouse conveyor, the second segment 321, the chassis 302, and the first segment 304. As will be described in more detail herein, the chassis 302, the first segment 304, the second segment 321, and the gripper 306 can be actuated to various positions and / or angular positions, or otherwise operated, so that objects can be transported or transferred between the warehouse and the cargo carrier in a desired efficient manner.

[0042] The robot system 300 may also include support legs 310 coupled to the chassis 302, one or more controllers 338 supported by the chassis 302, a first joint roller 309 coupled between the first segment 304 and the gripper 306, and a second joint roller 337 coupled between the first segment 304 and the second segment 321. The chassis 302, the first segment 304, the second segment 321, the sensor arm 330, the support legs 310, and / or other components of the robot system 300 may be made from metal (e.g., aluminum, stainless steel), plastic, and / or other suitable materials.

[0043] The chassis 302 may include a frame structure supporting a first segment 304, a second segment 321, a controller 338, and / or one or more sensor arms 330 coupled to the chassis 302. In the embodiment shown, two sensor arms 330 each extend vertically on either side of the first segment 304. An upper sensor 324 (e.g., an upper vision sensor) and a lower sensor 325 (e.g., a lower vision sensor) are coupled to each sensor arm 330 along the vertical direction and are positioned to generally face the distal portion 301a.

[0044] The first segment 304 is coupled to the chassis 302 so as to extend in a cantilever manner toward the distal portion 301a. The first segment 304 supports a first conveyor 305 (e.g., a conveyor belt) that extends along and / or around the first segment 304. Similarly, the second segment 321 is coupled to the chassis 302 so as to extend in a cantilever manner toward the proximal portion 301b of the robot system 300. The second segment 321 supports a second conveyor 322 (e.g., a conveyor belt) that extends along and / or around the second segment 321. In some embodiments, one or more actuators 336 (e.g., motors) configured to move the first and second conveyors 305, 322 are coupled to the chassis 302. In some embodiments, the actuator is located elsewhere (for example, housed in or coupled to the first and / or second segments 304, 321). The actuator 336 can also be operated to rotate the first segment 304 around a first axis A1 and / or a second axis A2. As shown in Figure 3, the first axis A1 may be generally perpendicular to the cross-section of the chassis 302 (for example, the second plane P2 shown in Figures 4 and 8), while the second axis A2 may be generally parallel to the cross-section of the chassis 302. In other words, the first axis A1 may lie in a first plane that is generally perpendicular to the second plane containing the second axis A2. In some embodiments, the actuator 336 can also pivot the second joint roller 337 around the first and second axes A1, A2, or around different axes. The movement and / or rotation of the first segment 304 relative to the chassis 302 will be described in further detail below with reference to Figures 5 to 7B.

[0045] As described above, the gripper 306 can be coupled with a joint roller 309 positioned between it and the first segment 304 so as to extend from the first segment 304 to the distal portion 301a. In some embodiments, the gripper 306 is configured to use vacuum to grip an object and selectively release the object. The gripper 306 may include a suction cup 340 (and / or any other suitable gripping element such as a magnetic component, a mechanical gripping component, etc., which may generally be referred to as “gripper element,” “gripping element,” etc.), and / or a distal conveyor 342. The suction cup 340 can grip an object with air pressure so that the suction cup 340 can carry the object and then place it on the distal conveyor 342 which transports the object in the proximal direction.

[0046] In some embodiments, one or more actuators 308 (e.g., motors) are configured to rotate the gripper 306 and / or the first joint roller 309 relative to the first segment 304 around a third axis A3 and / or a fourth axis A4. As shown in Figure 3 and described in more detail below with reference to Figures 40-44C, the third axis A3 may be generally parallel to the longitudinal plane of the gripper 306 (e.g., the third plane P3 shown in Figure 43A), while the fourth axis A4 may be generally perpendicular to the longitudinal plane of the gripper 306. Furthermore or alternatively, the third axis A3 may be generally perpendicular to the cross-section of the gripper 306 (e.g., the fourth plane P4 shown in Figure 42A), while the fourth axis A4 may be generally parallel to the cross-section of the gripper 306. In other words, the third axis A3 may lie in a third plane that is generally orthogonal to a fourth plane containing the fourth axis A4. In some embodiments, as will be described in more detail below, the robot system 300 can maintain a cross-section of the gripper 306 that is generally parallel to the cross-section of the chassis 302, such that (for example, a rotation around the second axis A2 is filled by a counter-rotation around the fourth axis A4). As a result, for example, in some embodiments, the third axis A3 may be generally orthogonal to the cross-section of the chassis 302, and / or the fourth axis A4 may be generally parallel to the cross-section of the chassis 302.

[0047] In some embodiments, the actuator 308 is configured to operate the suction cup 340 and / or the distal conveyor 342. In some embodiments, the actuator 308 is coupled to the first segment 304, the first joint roller 309, and / or the gripper 306. The movement and / or rotation of the gripper 306 relative to the components of the second segment 304 and the gripper 306 are described in further detail below.

[0048] In the embodiments shown, two support legs 310 are rotatably coupled to the chassis 302 around pivots 316 located on either side of the chassis 302. Wheels 312 are mounted on the distal portions of each support leg 310. The chassis 302 also supports actuators 314 (e.g., linear actuators, motors) operably coupled to the support legs 310. In some embodiments, the robot system 300 includes fewer or more support legs 310, and / or support legs 310 configured in different positions and / or orientations. In some embodiments, the wheels 312 can be motorized to move the chassis 302, and thus the rest of the robot system 300, along a linear direction L1. The operation of the actuators 314 is described in further detail below with respect to Figure 4.

[0049] The controller 338 can be operably coupled (e.g., wired or wirelessly) to control the actuators 308, 336, and 314. In some embodiments, the controller 338 is positioned to offset moments exerted on the chassis 302, for example, by a cantilevered first segment 304. In some embodiments, the robot system 100 includes counterweights coupled to the chassis 302 to offset such moments.

[0050] As an exemplary example, the robot system 300 can be configured to interface with and operate between (1) a cargo carrier positioned on or around the distal section 301a and (2) a conveyor pre-installed in a truck bay positioned on or around the proximal section 301b. The support legs 310 can enable the chassis 302 and / or the second segment 321 to be positioned on and / or overlapped with the existing object handling components. For example, the support legs 310 can be adjacent to or next to the peripheral surface of the warehouse conveyor, and the chassis 302 and / or the second segment 321 can be positioned on and / or partially overlapping the end of the warehouse conveyor.

[0051] Based on the relative arrangement described above, the robot system 300 can automatically transfer objects between the cargo carrier and the warehouse conveyor. Using an exemplary devanning process, the robot system 300 can use the first segment 304 to position the EOAT (e.g., gripper 306) adjacent to or in front of objects located within / stacked in the cargo carrier in a continuous / repeated manner. Using the EOAT, the robot system 300 can (1) grasp the objects and first remove them from the cargo carrier, and (2) place / release the grasped objects onto the first joint roller 309 and / or the first conveyor 305. The robot system 300 can transfer objects from the EOAT to the warehouse conveyor by operating a series of connected rollers and conveyors, such as the first joint roller 309, the first conveyor 306, the second joint roller 337, and the second conveyor 322.

[0052] When transporting an object, the robot system 300 can analyze sensor information such as one or more image data (e.g., 2D and / or 3D data) and other observed characteristics of the object. For example, in a mixed SKU environment, objects of different types and sizes can be stacked on top of each other, adjacent to each other. The coplanar surfaces (e.g., fronts) of the stacked objects can form walls or vertical surfaces that extend at least partially across the width and / or height inside the cargo carrier. The robot system 300 can initially detect objects inside the cargo carrier using 2D and / or 3D image data from the visual sensors 324 and / or 325. The detection operation may include identifying edges, calculating and evaluating the dimensions of the edges or the dimensions between edges, and evaluating surface textures such as visual characteristics including codes, numbers, letters, shapes, drawings, etc., that identify the object or its contents. The robot system 300 can compare the sensor output and / or derived data to the attributes of known or expected objects shown in the master data 252 in Figure 2. If the compared attributes match, the robot system 300 can detect the object depicted in the image data by determining the type or identifier of the object and its estimated real-world location (e.g., the peripheral edge of the object). The robot system 300 can perform additional operations and analyses to verify the detection or related data, and / or when a portion of the image cannot match the attributes shown in the master data 252, and the corresponding portion may indicate an unrecognized object. Details regarding the operations of the robot system 300 and the corresponding details are described below.

[0053] Figure 4 is an enlarged side view of the robot system 300 showing the operation of the support leg 310 according to an embodiment of the present technology. In the embodiment shown, the robot system 300 is positioned on top of a conveyor segment 320, which may already be present in a warehouse or other work site. Specifically, the chassis 302 is positioned at or on the distal end of the conveyor segment 320 so that the first segment 304 can rotate relative to the chassis 302 without contacting the conveyor segment 320. The conveyor segment 320 may be on a support surface or floor 372 (or other surface) within the warehouse. The robot system 300 can position one or more components to compensate for uneven floors, sloped floors, and other environments, providing an acceptable range of positions for transporting objects. For example, the robot system 300 can level the chassis 302 by, for example, moving the chassis 302 relative to the floor 372. The chassis 302 may then be oriented generally horizontally so that the conveyor belt of the robot system 300 is within the target range of positions for transporting objects (for example, the cross-section of the chassis 302 may be generally horizontal).

[0054] In the embodiments shown, one end of the actuator 314 is rotatably coupled to the chassis 302 via a hinge 315. The other end of the actuator 314 is coupled to the support leg 310 via a hinge or bearing 313 so that the actuator 314 and the support end 310 can rotate relative to each other. During operation, the actuator 314 can be controlled (for example, via a controller 338 shown in Figure 2) to move the support leg 310 between a first state (shown by a solid line in Figure 4) and a second state (shown by a dotted line in Figure 4). In the first state, the support leg 310 is pulled toward the hinge 315 by the actuator 314 or otherwise positioned so that the wheel 312 is at a height above the floor 372. In some embodiments, the first state shown corresponds to the maximum vertical distance (e.g., height) to which the wheel 312 can be lifted relative to the floor 372, defined by distance D1. Distance D1 may be at least 140 millimeters (mm), 160 mm, 180 mm, 200 mm, 220 mm, or in the range of 140 to 220 mm. In the second state, the support leg 310 is pushed away from the hinge 315 by the actuator 314 or otherwise positioned so that the wheel 312 is below floor 372. In some embodiments, the second state shown corresponds to the maximum vertical line to which the wheel 312 can be lowered relative to floor 372, defined by distance D2. Distance D2 may be at least 290 mm, 310 mm, 330 mm, 350 mm, 370 mm, or in the range of 290 to 370 mm.

[0055] During operation, the support legs 310 and wheels 312 can provide support to the chassis 302 so that the conveyor segment 320 does not need to support the entire weight of the robot system 300. As will be described in more detail below, the wheels 312 can also be motorized to move the chassis 302, for example, closer to or further away from a cargo carrier (e.g., a truck). The wheels 312 may be motorized wheels including one or more moving drive motors, brakes, sensors (e.g., position sensors, pressure sensors, etc.), hubs, and tires. The components and configurations of the wheels 312 can be selected based on operation and environment. In some embodiments, the wheels 312 are connected to the drive system of the chassis 302. The wheels 312 can also be locked (e.g., using brakes) to prevent accidental movement, for example, when unloading cargo from a cargo carrier and loading cargo onto a cargo carrier.

[0056] The ability of the support legs 310 and the wheels 312 attached thereto to lift and lower can be advantageous for several reasons. For example, the support legs 310 can rotate to a position indicated by a dotted line (e.g., to a distance D2) to lift and / or rotate the chassis 302, further extending the range of the gripper 306. The support legs 310 can also rotate to a position indicated to lower and / or rotate the chassis 302 (e.g., to a distance D1). In another example, the floor 372 may have steps so that the conveyor segment 320 and the wheels 312 contact the floor 372 at different heights. Thus, the robot system 300 can adapt to variability in a warehouse environment without requiring additional support mechanisms. In another example, the wheels 312 can be lifted (e.g., while the wheels 312 are locked) to move the conveyor segment 320 (e.g., extend it horizontally). Once the conveyor segment 320 has moved or extended to the desired position, the wheels 312 can be lowered. In yet another example, the robotic system 300 can be moved at least partially into the cargo carrier (e.g., the rear of the truck) to reach deeper into the cargo or space within the cargo carrier. If the floor of the cargo carrier is higher or lower than the warehouse floor 372, the support legs 310 can be raised or lowered accordingly.

[0057] In other embodiments, the components described above may be arranged differently from those shown in the embodiments. For example, the actuator 314 may be fixedly coupled to the chassis 302. In another example, the actuator 314 may be positioned behind or near the support leg 310 so that the support leg 310 is pushed up and pulled down.

[0058] Figure 5 is a side view of a robotic system 300 showing the vertical operation of a first segment 304 according to an embodiment of the present technology. In the embodiment shown, the robotic system 300 is positioned and operated to reach a target area 334. The target area 334 may include cargo (e.g., stacks of objects such as boxes and containers) or other items to be loaded and unloaded. The first segment 304 is shown tilted in the lowered position. A gripper 306, which can rotate around a pivot point near the actuator 308 (e.g., via the actuator 308), is generally shown oriented horizontally. The shown position of the first segment 304 may correspond to a dotted line 350a extending from a pivot point near the actuator 336. During operation, the first segment 304 can be rotated around the pivot point (e.g., by the actuator 336) to a horizontal position corresponding to dotted line 350b, to an elevated position corresponding to dotted line 350c, and to any position in between. In some embodiments, the dotted lines 350a and 350c represent the lowest and highest positions from which the first segment 304 can be rotated.

[0059] As the first segment 304 rotates around a pivot point near the actuator 336, the reach of the gripper 306's suction cup 340 extends along the dotted curve 352. In the embodiment shown, the dotted curve 352 may touch the target area 334 so that the suction cup 340 can reach the target area 334 when the first segment 304 is in a horizontal position (dotted 350b), but cannot reach it when the first segment 304 is in a lowered position (dotted 350a) or raised position (dotted 350c). To enable the suction cup 340 to reach the entire target area 334 (for example, positioning the suction cup 340 along a generally vertical, planar target area 334), the robotic system 300 can be moved along a linear direction L1 (for example, via an extension of the motorized wheel and / or conveyor segment 320 (Figure 4)). As the robot system 300 moves, the first segment 304 is translated to a new lowered position corresponding to the dotted line 354a and a new raised position corresponding to the dotted line 354c. As shown by the dotted lines, the suction cup 340 can reach the farthest edge of the target area 334 when the first segment 304 is in either the new lowered (dotted line 354a) position or the new raised position (dotted line 354c). As will be described in more detail below, the actuator 308 can be operated to rotate the gripper 306 perpendicular to the first segment 304 at any time to reach the object as needed. In addition, the upper vision sensor 324 and lower vision sensor 325 on the sensor arm 330 can be used to determine the position and / or orientation of the first segment 304, the gripper 306, and / or the area of ​​the target area 334 and relay the information to the controller 338 for real-time control.

[0060] Figure 6 is a top view of a robot system 300 showing the horizontal operation of the first segment 304 according to an embodiment of the present technology. In the embodiment shown, the first segment 304 is shown tilted in a leftward-tilted position. A gripper 306, which can rotate around a pivot point near the actuator 308 (e.g., via the actuator 308), is shown generally parallel to the chassis 302 (e.g., facing the target area 334). The shown position of the first segment 304 may correspond to a dotted line 360a extending from a pivot point near the actuator 336. During operation, the first segment 304 can be rotated around the pivot point (e.g., by the actuator 336) to a horizontal position corresponding to dotted line 360b, to a rightward-tilted position corresponding to dotted line 360c, and to any position in between. In some embodiments, dotted lines 360a and 360c represent the leftward and rightward tilted positions to which the first segment 304 can rotate.

[0061] As the first segment 304 rotates around a pivot point near the actuator 336, the reach of the gripper 306's suction cup 340 extends along the dotted curve 362. In the shown embodiment, the dotted curve 362 touches the target area 334 such that the suction cup 340 can reach the target area 334 when the first segment 304 is in a horizontal position (dotted 360b), but cannot reach it when the first segment 304 is tilted to the left (dotted 360a) or to the right (dotted 360c). To enable the suction cup 340 to reach the entire target area 334, the robot system 300 can be moved along a linear direction L1 (e.g., via an extension of the motorized wheel and / or conveyor segment 320 (Figure 4)). As the robot system 300 moves, the first segment 304 is translated to a new left-tilted position corresponding to dotted line 364a and a new right-tilted position corresponding to dotted line 364c. As shown by the dotted lines, the suction cup 340 can reach the farthest edge of the target area 334 when the first segment 304 is in either the new left-tilted position (dotted line 364a) or the new right-tilted position (dotted line 364c). As will be described in more detail below, the actuator 308 can be operated to rotate the gripper 306 horizontally relative to the first segment 304 at any time to reach the object as needed. In addition, the upper vision sensor 324 and lower vision sensor 325 on the sensor arm 330 can be used to determine the position and / or orientation of the first segment 304 and the gripper 306 and relay the information to the controller 338 for real-time control.

[0062] In some embodiments, the vertical movement of the first segment 304 and gripper 306 shown in Figure 5 can be coupled with the horizontal movement of the first segment 304 and gripper 306 shown in Figure 6. For example, the target area 334 may include a rectangular volume (e.g., corresponding to the interior of a truck), and the first segment 304 can be controlled to pivot horizontally, vertically, diagonally, move laterally, and / or move in other directions to reach any desired position within the rectangular target area 334. The entire length or a large portion of the robot system 300 can be moved distally into a trailer (e.g., a semi-trailer in Figures 1 and 7B) to access objects at the front of the trailer so that the robot system 300 can unload the entire trailer without contacting the trailer's side walls or ceiling. The robot system 300 can use a maximum cover for the environment, a limited cover for accessing objects, and an operating or working cover for performing tasks. The robot system 300 can determine the robotic work cover for emptying a trailer, for example, using a robotic work cover specific to the trailer, or a user-selected robotic work cover. The trailer-specific robotic work cover can be determined based on an inspection of the trailer's interior and can be modified any number of times during use. The user-selected robotic work cover can be entered by the user based on the trailer's configuration (e.g., dimensions, trailer type, etc.). The robotic work cover may include the area the robot system 300 is allowed to move or reach, the range of motion, etc. The robot system 300 can run one or more simulations to evaluate the set of robotic work covers and predicted outcomes, including the number of unloading operations, potential adverse events (e.g., slippage of objects, possibility of falling objects, possibility of damage to fragile objects, etc.), and an acceptable conveyor belt speed based on the orientation of the conveyor belt. Based on the simulations and predicted outcomes, the robot system 300 can select a robotic work cover from the simulated set of robotic work covers.

[0063] Figures 7A and 7B are side views of a robotic system 300 showing the vertical operation of the first segment 304 relative to a cargo carrier according to an embodiment of the present technology. Referring to both Figures 7A and 7B, the conveyor segment 320 is on the floor 372, and the chassis 302 is positioned above the conveyor segment 320, with the wheels 312 in contact with the floor 372. The cargo carrier 332 (e.g., a loading truck) is positioned so that the rear end of the cargo carrier 332 faces the warehouse bay opening 374. In particular, the conveyor segment 320 can be positioned such that the distal end of the conveyor segment 320 is at a distance D4 from the rear end cargo carrier 332, and the proximal end of the conveyor segment 320 is at a distance D5 from the rear end of the cargo carrier 332. The conveyor segment 320 can have a height of D3 so that the chassis 302 rises from the floor 372 at a height of D3. In some embodiments, the distance D4 may be at least 3 meters (m), 4m, 5m, 6m, 7m, or within the range of 3 to 7m (e.g., 4.7m). In some embodiments, the distance D5 may be at least 8 meters (m), 10m, 12m, 14m, 16m, or within the range of 8 to 16m (e.g., 12.2m). In some embodiments, the height D3 may be at least 0.7 meters (m), 0.8m, 0.9m, 1.0m, 1.1m, or within the range of 0.7 to 1.1m. These dimensions can be used, for example, to generate trailer-specific robotic work covers. The cargo items 334 can be positioned somewhere within the cargo carrier 332 (e.g., in the rear section as shown) for unloading and / or loading by the robotic system 300. Trailer-specific robotic work covers can be used to access any of those cargo items 334, which can be updated or modified when the cargo items 334 are removed.

[0064] Referring first to Figure 7A, the first segment 304 is in a raised position such that it forms an angle θ1 with the horizontal position. The angle θ1 represents the maximum angle by which the first segment 304 can be raised and may be at least 16°, 18°, 20°, 22°, 24°, or in the range of 16–24°. In the embodiment shown, the length of the first segment 304 and the angle θ1 are such that the gripper 306 reaches the top of the rear end of the cargo carrier 332. To reach further into the cargo carrier 332, it can be advanced distally toward the cargo carrier 332, and as a result the robotic system 300 and / or conveyor segment 320 can enable the gripper 306 to reach further into the cargo carrier 332.

[0065] Referring next to Figure 7B, the first segment 304 is in a lowered position such that it forms an angle θ2 with the horizontal position. The angle θ2 represents the maximum angle to which the first segment 304 can be lowered and may be at least 16°, 18°, 20°, 22°, 24°, or in the range of 16–24°. In the embodiment shown, the length of the first segment 304 and angle θ1 is such that the gripper 306 reaches the bottom of the rear end of the cargo carrier 332. The angles θ1 and θ2 can be used to determine a robotic work cover designed to access an object.

[0066] As described above, the first segment 304 and gripper 306 can move (e.g., pivot) in multiple directions (e.g., vertically, horizontally, diagonally) between various angles, and the robotic system 300 can move distally to reach any desired cargo 334 or space within the cargo carrier 332. For example, the conveyor segment 320 can be extended distally so that the wheel 312 enters the cargo carrier 332, and / or the wheel 312 can be operated to move the chassis 302 distally. In the shown embodiments, since the floor 372 of the warehouse 370 and the floor of the cargo carrier 332 are horizontal, the wheel 312 can remain at the shown height while entering the cargo carrier 332. In some embodiments, the wheel 312 can be lifted to avoid any gap between the floor 372 of the warehouse 370 and the floor of the cargo carrier 332. In some embodiments, the floor of the cargo carrier 332 is higher or lower than the warehouse floor 372, in which case the robotic system 300 can raise or lower the wheels 312 accordingly, as described above with respect to Figure 4.

[0067] How to operate a robot system Figure 8 is a schematic side view of a robot system 800 according to one or more embodiments. In the embodiment of Figure 8, the robot system includes a chassis 802. The chassis 802 supports a segment 804. As described herein, the segment 804 is configured to rotate relative to the chassis 802 with two rotational degrees of freedom. The robot system further includes a gripper 806 operably coupled to the segment 804 by a joint 808. The joint 808 may provide the gripper 806 with multiple degrees of freedom relative to the segment 804. The rotational degrees of freedom of the gripper 806 may be the same as those of the segment 804. In this way, the orientation of the gripper 806 may be maintained relative to an environmental reference frame or a local reference frame, while the position of the gripper 806 is changed by a change in the orientation of the segment 804.

[0068] As shown in Figure 8, the robot system 800 includes legs 810 that support the chassis 802. The legs 810 include wheels 812 at the lower ends of the legs. The wheels 812 are configured to rotate to allow the chassis 802 to move in a first translational degree of freedom (e.g., a horizontal degree of freedom). For example, the chassis 802 can move in a direction generally parallel to the longitudinal direction, the central plane, etc. When the chassis 802 is positioned in the horizontal plane, the chassis 802 can move linearly in the horizontal direction. In the embodiment shown, the legs are coupled to the chassis at the upper ends of the leg joints 816. In the example of Figure 8, the legs 810 are configured to rotate around the leg joints 816 to move the wheels 812 vertically, thereby moving the chassis 802 correspondingly in a second translational degree of freedom (e.g., a vertical degree of freedom) perpendicular to the first translational degree of freedom. For example, the chassis 802 can move linearly in a direction generally parallel to its cross-section. When the chassis 802 is positioned in a horizontal plane, the chassis 802 can move linearly in the vertical direction. In some embodiments shown in Figure 8, the robot system 800 includes a leg actuator 814 configured to move the leg in the vertical direction. The leg actuator 814 is configured to rotate the leg 810 around the leg joint 816 in the example of Figure 8.

[0069] The robot system 800 is configured to move an object 834 (e.g., a box) located within a cargo carrier 832 in a proximal direction to unload an object from the cargo carrier. In the example in Figure 8, the robot system 800 is configured to move an object to a warehouse conveyor 818 located within a warehouse or other object processing center. The warehouse conveyor 818 includes a telescopic segment 820 configured to extend and retract. The chassis 802 may be coupled to the distal end of the warehouse conveyor 818. As shown in Figure 8, the robot system may include a proximal conveyor 822 located above the warehouse conveyor and configured to move an object from segment 804 to the warehouse conveyor 818. Segment 804 includes a segment conveyor configured to move an object 834 to the proximal conveyor 822.

[0070] In the example shown in Figure 8, the cargo carrier 832 is a truck trailer and includes multiple objects 834. As shown in Figure 8, the multiple objects may be arranged in a vertical plane (for example, generally parallel to the crown and / or front of the cargo carrier, such as the yz plane shown in Figures 17A–17F). In some cases, the objects may be arranged in a vertical stack that approximates a vertical plane rather than in a perfectly vertical plane. The robot system 800 includes one or more upper vision sensors 824 and one or more lower vision sensors 825 configured to acquire images of the cargo carrier 832 and the multiple objects 834. Specifically, the vision sensors are configured to capture images of the vertical plane of the objects 834. The image information may be used by a local controller to control the operation of the robot system, examples of which are further described with reference to Figures 12A–12F and Figures 15–18. As shown in the example in Figure 8, the upper vision sensor 824 may have a first field of view 826a, and the lower vision sensor 825 may have a second field of view 826b. In some cases, it may be desirable to use multiple vision sensors to ensure complete coverage of the cargo carrier 832 and the objects 834 placed within it. In other embodiments, a single vision sensor or any number of vision sensors may be used. The upper and lower vision sensors may, in some embodiments, be cameras (e.g., two-dimensional (2D) or image sensors and / or three-dimensional (3D) depth sensors such as LiDAR, corresponding to the imaging device 222 in Figure 2). As shown in Figure 8, the upper vision sensor 824 and the lower vision sensor 825 are supported by an arm 830 coupled to the chassis 802. Placing the vision sensors on the arm 830 can reduce interference caused by parts of the robot system itself. Furthermore, placing the vision sensors on the arm 830 allows the vision sensors to enter the cargo carrier 832. The vision sensors are configured to image the objects 834 at an imaging distance 828 that is less than the combined length of the segment 804 and the gripper 806.Furthermore, the arm 830 can be directly mounted to the chassis 302, or otherwise separate from segments 304, 804, etc., so as to provide a view / image that is not referenced by the chassis 302 and is not affected by the attitude / movement of segment 304.

[0071] Figure 9 is a schematic top view of the robot system 800 of Figure 8 in the first state. In the state shown in Figure 9, the robot system 800 has reached into the cargo carrier 832. Specifically, the distal end of the robot system 800 is located inside the cargo carrier 832, while the proximal end of the robot system remains outside the cargo carrier. Thus, the gripper 806 of the robot system 800 can access the object 834 located inside the cargo carrier 832. As shown in Figure 9, segment 820 of the warehouse conveyor 818 extends to accommodate the movement of the robot system 800 into the cargo carrier 832. Figure 9 also shows the local controller 838 of the robot system 800 located on the chassis 802. The controller 838 can control various components of the robot system, as will be further described herein with reference to exemplary methods.

[0072] Figure 9 shows that the robot system 800 may be generally symmetrical around the longitudinal axis. For example, the robot system 800 may include a first leg 810a and a first wheel 812a. On the opposite side, the robot system 800 includes a second leg 810a and a second wheel 812a. The first leg 810a is movable vertically by a first leg actuator 814a, and the second leg 810b is movable by a second leg actuator 814b. In the example in Figure 9, the robot system 800 includes a first upper vision sensor 824a and a second upper vision sensor 824b positioned on either side of the longitudinal axis of the robot system. The upper vision sensors, each pointed on a symmetrical arm 830, are mirrored across the longitudinal axis. The first upper visual sensor 824a has a first field of view 826a, and the second upper visual sensor 824b has a second field of view 826c. The fields of view ensure complete coverage of multiple objects 834 placed within the cargo carrier 832. Furthermore, placing visual sensors on two sides of segment 804 (and above and below segment 804 as shown in Figure 8) ensures that a complete image of the objects 834 can be captured without obstruction by segment 804 and gripper 806. In some embodiments, placing a visual sensor below segment 804 (or below the zero-pitch position of the segment) has particular advantages when the unloading process starts at the top of a vertical stack. In such cases, the segment may be positioned at a high pitch angle to start, allowing the visual sensor positioned below the segment to obtain an unobstructed field of view of multiple objects.

[0073] As shown in Figure 9, segment 804 is coupled to chassis 802 and proximal conveyor 822 by joint 836. Joint 836 is configured to provide segment 804 with multiple rotational degrees of freedom relative to chassis 802 and proximal conveyor 822. In some embodiments, proximal conveyor 822 may be fixed to chassis 802. Joint 836 provides segment 804 with two rotational degrees of freedom. In the embodiment shown in Figure 9, joint 836 provides a yaw degree of freedom (e.g., rotation around a vertical axis, generally parallel to the longitudinal plane P1 of chassis 802 shown in Figure 9) and a pitch degree of freedom (e.g., rotation perpendicular to the page and generally parallel to the cross-section P2 of chassis 802). In some embodiments shown in Figure 9, joint 836 includes a plurality of rollers 837 configured to move an object 834 proximal to the proximal conveyor 822 from segment 804.

[0074] The gripper 806 is connected to the segment 804 by a joint 808. The joint 808 is configured to provide the gripper 806 with multiple rotational degrees of freedom relative to the segment 804. The joint 808 provides the gripper 806 with two rotational degrees of freedom. In the embodiment of Figure 9, the joint 808 provides a yaw degree of freedom (e.g., rotation around a vertical axis such as the first axis A1 in Figure 3) and a pitch degree of freedom (e.g., rotation around a transverse horizontal axis such as the second axis A2 in Figure 3). As shown in Figure 9, in some embodiments, the joint 808 includes a plurality of rollers 809 configured to move the object 834 proximal to the segment 804 from the gripper 806.

[0075] In the example shown in Figure 9, the robot system 800 is configured to grasp object 834 of a plurality of objects with a gripper 806 and move the object proximal along a series of conveyors. As shown in Figure 9, the gripper 806 includes a plurality of suction cups 840 (and / or any other suitable gripper elements) and a plurality of distal conveyors 842. The suction cups 840 are positioned in contact with object 834 and are configured to grasp the object when a vacuum force (or other suitable driving force) is applied to the suction cups 840. The distal conveyors 842 are belt conveyors in the example of Figure 9 and are configured to move the object proximal once the object is grasped by the suction cups 840. Examples of grasping objects are further described herein with reference to gripper and end-of-arm tool (EOAT) configurations. Object 834 moves proximal to joint 808 and into contact with roller 809. Roller 809 may be driven and may further move object 834 on a segment conveyor 805 positioned on segment 804. Segment conveyor 805 may be a belt conveyor and may be configured to move objects to joint 836 and roller 837. Roller 837 may move objects to proximal conveyor 822. Proximal conveyor may also be a belt conveyor. Proximal conveyor may move objects to warehouse conveyor 818. In some embodiments, the device conveyor may be a belt conveyor or a roller conveyor.

[0076] Figure 10 is a top schematic of the robot system 800 of Figure 9 in a second state, illustrating the yaw range of motion provided by joints 808 and 836. Compared to the state shown in Figure 9, segment 804 rotates around joint 836 in the yaw direction (e.g., clockwise around the axis inward, such as the first axis A1 in Figure 3). Correspondingly, gripper 806 rotates around joint 808 in the opposite direction (e.g., counterclockwise around the axis inward). The rotation of segment 804 changes the position of gripper 806 relative to the cargo carrier 832 and object 834. However, the orientation of gripper 806 relative to the cargo carrier 832 and object 834 remains the same. Such a configuration may be advantageous in allowing gripper 806 to reach the edges of rectangular or triangular prism-shaped cargo carriers (such as box trucks, shipping containers, or semi-truck trailers). For example, as shown in Figure 10, the gripper 806 may be able to align with the side wall of the cargo container even if its position changes due to the rotation of segment 804. Furthermore, the gripper's suction cup 840 may remain neatly aligned with the object 834 to ensure that the suction cup can securely grip the object. In the example in Figure 10, the distal conveyor 842 and the proximal conveyor 822 may remain parallel to each other through the change in the gripper's position. The angle of the segment conveyor 805 may change as the gripper moves through its range of motion. The roller 809 of joint 808 and the roller 837 of joint 836 may accommodate this change in angle, allowing the object to move proximal from the distal conveyor 842 to the segment conveyor 805 and the proximal conveyor 822.

[0077] Figure 11 is a schematic diagram showing a robotic system 800 arranged inside a cargo carrier 832 according to one or more embodiments. Figure 11 is an enlarged view of the configuration shown in Figure 8. Segment 804 is arranged inside the cargo carrier 832 and includes a segment conveyor 805. On the first side of segment 804 is a first vision sensor 824a located on arm 830a. On the opposite, second side of segment 804 is a second vision sensor 824b located on second arm 830b. The total width between the first vision sensor 824a and the second vision sensor 824b may be less than the total width of the cargo carrier 832. A tolerance clearance distance 844 is provided between the wall of the cargo carrier (not shown) and the vision sensors. In some embodiments shown in Figure 11, wheels 812a, 812b of the robotic system 800 may enter the cargo carrier 832.

[0078] Figures 12A to 12F illustrate a robotic system through a process of unloading a cargo carrier 1232 adjacent to a warehouse or other structure 1213, according to one or more embodiments. Specifically, Figures 12A to 12F show how various components of the robotic system interact and are controlled (e.g., by a local controller 1236) to access and unload multiple objects 1234 that may be stacked vertically within the cargo carrier 1232.

[0079] As shown in Figure 12A, the robot system 1200 includes a chassis 1202, a proximal conveyor 1204, a segment 1206, and a gripper 1208. The segment is operably coupled to the proximal conveyor 1204 at its proximal end via a first joint that provides two rotational degrees of freedom. Thus, the distal end of the segment 1206 may have a hemispherical range of motion. The gripper 1208 is operably coupled to the segment 1206 via a second joint 1212 that provides two rotational degrees of freedom. Thus, the distal end of the gripper 1208 may have a hemispherical range of motion. The gripper 1208 includes a plurality of suction cups 1210 (or other suitable gripper elements) configured to grip an object 1234. The chassis 1202 is supported by legs 1220, each including a wheel 1222. Wheel 1222 contacts the environment 1214 in which the robot system is located, which may be a warehouse in Figures 12A–12F. The environment 1214 includes a warehouse bay opening 1215 through which a cargo carrier 1232 is accessed. Legs 1220 are vertically movable by corresponding leg actuators 1224. In the examples of Figures 12A–12F, the robot system includes two legs, two wheels, and two leg actuators. In the examples of Figures 12A–12F, the robot system 800 is coordinated with a warehouse conveyor 1216, which in some cases may have been present in the environment 1214 from the beginning. The warehouse conveyor 1216 includes multiple telescopic segments 1218, which allow the warehouse conveyor to extend and retract. The warehouse conveyor 1216 in Figures 12A–12F includes a belt 1217. The chassis 1202 may be connected to the distal end of the warehouse conveyor, so that the distal end of the warehouse conveyor and the chassis move together in translational degrees of freedom.

[0080] According to embodiments of Figures 12A to 12F, the robot system 1200 includes a controller 1236 (e.g., including a processor 202 in Figure 2, a memory device 204 in Figure 2, etc.) configured to control various components of the robot system using one or more actuators. The controller 1236 is also configured to receive information from one or more sensors (e.g., sensor 216 in Figure 2), including an upper vision sensor 1226 and a lower vision sensor 1228 attached to the arm 1230. Control algorithms that may be implemented by the controller 1236 are further described with reference to Figures 15 to 18. For example, the controller 1236 can implement a control algorithm by using the processor 202 to execute instructions or the software 210 in Figure 2.

[0081] The state in Figure 12A may represent the starting state, where the robot system 1200 is positioned entirely on one side of the warehouse bay opening 1215. The warehouse conveyor segment 1218 is fully retracted. Segment 1206 may be positioned such that the gripper 1208 is in its uppermost position. For example, with respect to pitch, segment 1206 may be at the uppermost end of its range of motion. As shown in Figure 12A, the gripper 1208 may rotate around the second joint 1212 to ensure that the gripper remains horizontal (e.g., aligned with the horizontal plane, such as the cross-section of the chassis shown in Figure 9). In the state in Figure 12B, the chassis 1202 of the robot system is moving distally (e.g., to the right relative to the page). Correspondingly, the telescopic segment 1218 is extended distally. The gripper 1208 and its suction cup 1210 pass through the warehouse bay opening 1215 and approach a plurality of objects 1234 within the cargo carrier 1232, which are arranged in vertical rows approximating a vertical plane (e.g., a plane generally parallel to the crown and / or front surface of the cargo carrier, such as the yz plane shown in Figures 17A–17F). Distal movement of the chassis 1202 may be provided by driving the wheels 1222 with one or more wheel motors. In some embodiments, the unloading process of the cargo carrier 1232 may begin with unloading the objects 1234 located on top of the cargo carrier.

[0082] Figures 12C and 12D show the robotic system 1200 of Figure 12A in a second state of the process of unloading a cargo carrier, according to one or more embodiments. As shown in Figure 12C, the upper vision sensor 1226 may have a first field of view 1238, and the lower vision sensor 1228 may have a second field of view 1240. Figure 12D shows a perspective view of how the gripper 1208 and segment 1206 are positioned to allow the gripper 1208 to reach the top of a plurality of objects 1234. Figure 12D further shows the chassis 1202 and a first joint 1250 that provides the segment 1206 with rotational degrees of freedom relative to the proximal conveyor 1204. The gripper 1208 includes a distal conveyor 1242 configured to move the objects 1234 continuously in the proximal direction toward the segment 1206. Segment 1206 includes a segment conveyor 1246 that moves objects 1234 continuously in the proximal direction toward the proximal conveyor 1204. The proximal conveyor 1204 is located above the warehouse conveyor 1216 and is configured to move objects 1234 continuously in the proximal direction toward the warehouse conveyor. In some embodiments shown in Figure 12D, the robotic system 1200 may include guides for objects to ensure that the objects remain on the series of conveyors. The first joint 1212 includes a gripper guide 1244 that serves as a boundary for objects moving in the proximal direction. Segment 1206 also includes a segment guide 1248 that serves as a boundary for moving objects along the segment conveyor 1246.

[0083] Figure 12E shows the robotic system 1200 of Figure 12A in a third state of the process of unloading a cargo carrier, according to one or more embodiments. Figure 12E specifically illustrates how an object 1234 moves along a series of conveyors in the robotic system so that the object can be delivered to a warehouse conveyor 1216. In the robotic systems of Figures 12A to 12F, the object 1234 moves continuously (e.g., one at a time) along a series of conveyors. The object 1234 is first grasped by the suction cup 1210 of the gripper 1208. The suction cup 1210 places the object 1234 onto the distal conveyor 1242, which moves the object 1234 proximately to the second joint 1212. The object 1234 then continues to the segment conveyor 1246, which continues to move the object proximately to the first joint 1250. The object 1234 then continues to the proximal conveyor 1204, which continues to move the object proximal to the warehouse conveyor 1216. In some embodiments, the joints between the various components include joint conveyors (e.g., rollers) that assist in transporting objects between the components. In some embodiments shown in Figure 12E, the proximal conveyor 1204 is inclined downward relative to the warehouse conveyor 1216. The segment conveyor 1246 may be inclined upward or downward proximal depending on the orientation of the segments around the first joint 1250.

[0084] The robot system 1200 in Figures 12A to 12F may repeatedly grasp and move objects proximally until the entire vertical stack of objects is removed. The chassis 1202 may then move distally to advance the gripper 1208 to the next stack of objects 1234. The grasping and moving process may then be repeated for the next stack of objects. The chassis 1202 may then move distally again to advance the gripper 1208 to the next stack of objects 1234. This pattern may be repeated until all objects 1234 have been unloaded from the cargo carrier 1232. The chassis 1202 may move at any time throughout this process.

[0085] As described above with reference to Figures 5-6, when segment 1206 rotates, a change in the position of gripper 1208 is achieved. However, when gripper 1208 is coupled to the distal end of segment 1206, gripper 1208 moves in an arc as segment 1206 rotates. Due to two rotational degrees of freedom (e.g., pitch and yaw), segment 1206 moves gripper 1208 within a hemispherical range of motion. Therefore, if the chassis 1202 remains stationary with respect to the vertical plane of object 1234, gripper 1208 moves toward or away from the object depending on the angle of segment 1206 in pitch and yaw. The maximum reach of gripper 1208 is where segment 1206 corresponds to zero pitch and zero yaw. Correspondingly, the minimum reach of gripper 1208 is where segment 1206 corresponds to maximum pitch or maximum yaw. At the minimum reach, the suction cup 1210 may not be able to reach object 1234 positioned in the vertical plane. Therefore, in some embodiments, the chassis 1202 may move distally or proximal to compensate for changes in the position of the gripper 1208 relative to the object 1234 caused by the rotation of the segment 1206. For example, as the pitch angle of the segment increases, the wheel 1222 may be driven (e.g., by a wheel motor) to move the chassis 1202 distally to maintain the gripper 1208 at a desired distance from the plane of the object 1234. Continuing this example, as the pitch angle of the segment decreases (e.g., returns to zero), the wheel 1222 may be driven to move the chassis 1202 proximal to maintain the gripper 1208 at a desired distance from the plane of the object 1234. A similar technique may be used to further change the yaw angle of the segment 1206. In this way, the position of the gripper 1208 can be changed relative to the object 1234 without moving the gripper 1208 outside the object's range.

[0086] Figure 12F shows the robotic system 1200 of Figure 12A in a fourth state of the process of unloading a cargo carrier, according to one or more embodiments. The state in Figure 12F shows, in particular, how the robotic system 1200 moves into the cargo carrier 1232 to continue unloading objects 1234 in an additional vertical stack. In some embodiments shown in Figure 12F, the wheels 1222 of the robotic system 1200 may move into the cargo carrier 1232 and rest on the floor 1233 of the cargo carrier. Legs 1220 may move vertically to ensure that components of the robotic system 1200 clear the internal vertical dimensions of the cargo carrier. As shown in Figure 12F, the telescopic segment 1218 may extend into the cargo carrier 1232 beyond the warehouse bay opening 1215 in some embodiments. In other embodiments, the warehouse conveyor may remain entirely within the warehouse, as this disclosure is not limited in that way. Furthermore, in some embodiments, the proximal conveyor 1204 can extend and retract in place of or in addition to the warehouse conveyor 1216.

[0087] Figure 13 is a flowchart illustrating the process of operating a robotic system according to one or more embodiments. For example, one or more of the robotic systems described above can execute software via one or more processors, thereby controlling one or more actuators / motors, interacting with sensors, and implementing the process.

[0088] In block 1302, the process includes rotating a first wheel and / or a second wheel to adjust the position of the robot system's chassis in a first translational degree of freedom (e.g., distal / proximal degrees of freedom). In some embodiments, rotating the first wheel and / or the second wheel may include driving a first wheel motor coupled to the first wheel and a second wheel coupled to the second wheel.

[0089] The robot system can control the first and / or second wheels to position the chassis and / or actuators of the proximal conveyor 1204 (e.g., the exit location), so that the chassis and / or end of the proximal conveyor 1204 overlaps with the warehouse conveyor 1216 (e.g., the receiving structure location) as the transported object moves past the rear segment. For example, the robot system can control the position of the chassis as the warehouse conveyor 1216 moves, so that the exit location remains at the target receiving position on the warehouse conveyor 1216.

[0090] In block 1304, the process further includes moving the first and / or second legs perpendicular to the chassis to adjust the position of the chassis in a second translational degree of freedom perpendicular to the first translational degree of freedom. Thus, the robot system can maintain the chassis above the warehouse conveyor 1216. In some embodiments, moving the first and / or second legs includes rotating the first and / or second legs relative to the chassis. In some embodiments, the first and second legs may be moved vertically independently of each other. Moving the first and / or second legs may include commanding one or more leg actuators to move the first and / or second legs relative to the chassis. Furthermore, the robot system can maintain the end of the proximal conveyor 1204 within a threshold height range from the top surface of the warehouse conveyor 1216 by controlling the actuators of the proximal conveyor 1204 to adjust its angle / orientation.

[0091] In block 1306, the process further includes rotating the first segment with respect to a proximal conveyor with respect to a first joint with respect to a first rotational degree of freedom. In some embodiments, the first degree of freedom is a pitch degree of freedom. In some embodiments, the process may further include rotating the first segment with respect to a roll degree of freedom. Rotating the first segment may include commanding one or more actuators to move the first segment with respect to a first joint. In some embodiments, one or more actuators may be positioned at the first joint.

[0092] In block 1308, the process further includes rotating the gripper around a second joint with respect to a first segment with a second rotational degree of freedom. In some embodiments, the second rotational degree of freedom is a pitch degree of freedom. In some embodiments, the process may further include rotating the gripper with a roll degree of freedom. Rotating the gripper may include instructing one or more actuators to move the gripper around the second joint. In some embodiments, one or more actuators may be located at the second joint.

[0093] In block 1310, the process includes moving an object proximal to a first segment along a distal conveyor positioned on a gripper. In block 1312, the process further includes moving an object proximal to a proximal conveyor along a first segment conveyor positioned on the first segment.

[0094] In block 1314, the process further includes moving an object in the proximal direction along a proximal conveyor. In some embodiments, the object may be moved from the proximal conveyor onto a warehouse conveyor. The robotic system can control the speed of the proximal conveyor according to the attitude and / or height of the exit point above the warehouse conveyor.

[0095] In some embodiments, the process may include detecting an object with one or more visual sensors; that is, image information including the object is acquired and processed to identify the object. The actions of 1306 and 1308 may be based in part on the image information and the identified object. The object may be gripped (for example, by one or more gripping elements of a gripper) and placed on a distal conveyor of the gripper.

[0096] Figure 14 is a flowchart illustrating the process of operating a robotic system according to one or more embodiments. For example, one or more of the robotic systems described above can execute software via one or more processors, thereby controlling one or more actuators / motors, interacting with sensors, and implementing the process.

[0097] In block 1402, the process includes rotating the first segment with a first rotational degree of freedom around the first joint relative to the proximal conveyor in order to adjust the pitch angle of the first segment. In some embodiments, rotating the first segment may include operating one or more actuators to move the first segment. In some embodiments, one or more actuators may be located at the first joint.

[0098] In block 1404, the process includes moving a gripper positioned at the distal end of a first segment in a vertical arc. Since the gripper may be attached to the distal end of the first segment and the first segment may rotate around its proximal end at a first joint, the movement in a vertical arc may be based on the rotation of the first segment. Thus, the gripper moves in a vertical arc due to a change in the pitch of the first segment.

[0099] In block 1406, the process includes rotating the first segment with a second rotational degree of freedom around the first joint relative to the proximal conveyor in order to adjust the yaw angle of the first segment. In some embodiments, rotating the first segment may include operating one or more actuators to move the first segment.

[0100] In block 1408, the process includes moving a gripper positioned at the distal end of a first segment in a horizontal arc. Since the gripper may be attached to the distal end of the first segment and the first segment may rotate around its proximal end at a first joint, the horizontal arc movement may be based on the rotation of the first segment. Thus, a change in the pitch of the first segment causes the gripper to move in a horizontal arc. In some embodiments, the gripper moves within a hemispherical range of motion through horizontal and vertical arc movements.

[0101] In block 1410, the process includes rotating the first wheel of the first leg and / or the second wheel of the second leg to adjust the position of the first segment by a first translational degree of freedom. The translational degree of freedom may be distal / proximal, aligned with the longitudinal axis of the robot system. As the first wheel and / or the second wheel rotate, the gripper may also move by the translational degree of freedom. In this way, the distance between the gripper and the vertical plane can be maintained despite the arcuate movement of the gripper. In some embodiments, the first translational degree of freedom is perpendicular to the vertical plane. The vertical plane may represent a stack of objects within the cargo carrier (e.g., a plane generally parallel to the crown and / or front surface of the cargo carrier, such as the yz plane shown in Figures 17A–17F).

[0102] In block 1412, the process includes gripping an object with a gripper. In some embodiments, gripping an object with a gripper includes applying a vacuum force (and / or another suitable driving force to another suitable gripping element) to one or more suction cups in contact with the object.

[0103] In block 1414, the process includes moving an object proximal to a proximal conveyor along a first segment conveyor positioned on a first segment. In some embodiments, a gripper may place an object on the first segment conveyor. In some embodiments, one or more suction cups may place an object on one or more distal conveyors of the gripper, thereby moving the object proximal to the first segment conveyor.

[0104] In block 1416, the process includes moving an object in the proximal direction along a proximal conveyor. In some embodiments, the process may include moving an object to a warehouse conveyor. In some embodiments, the first segment conveyor may include a belt, and the proximal conveyor may include a belt.

[0105] In some embodiments, the process further includes moving the gripper both linearly and arc-shaped to a target gripping position for gripping an object (e.g., immediately in front of the object or otherwise adjacent to the object). In some embodiments, the process includes selecting an object and translating a first segment relative to the object, while the gripper moves along a first arc and / or a second arc to move the gripper toward a target gripping position for gripping the object. In some embodiments, the process further includes determining a pickup path (e.g., including linear and / or arc-shaped path portions) for moving the gripper toward a target gripping position for gripping an object, and reconfiguring the robotic system to move the gripper along the pickup path while the gripper moves along the first arc and / or the second arc. In some embodiments, the pickup path is determined at least in part on one or more joint parameters of a first joint and / or a second joint. In some embodiments, one or more joint parameters include at least one of range of motion, joint velocity, joint strength (e.g., high torque), or joint precision.

[0106] In some embodiments, the process further includes controlling the robotic system to move the gripper along a pickup path toward a target gripping position for grasping an object, the pickup path being a linear or non-linear path. In some embodiments, the process further includes moving the robotic system along a support surface while a first joint and / or a second joint moves the gripper. In some embodiments, the process further includes controlling the robotic system to move the gripper toward an object to position the gripper toward a gripping position for grasping the object, compensating for movement along at least one of the first or second arcs.

[0107] Exemplary EOAT for Terminal Interface Systems Figure 15 is a schematic side view of a gripper assembly 1500 for a robotic system according to one or more embodiments. According to the embodiment of Figure 15, the gripper assembly includes a gripper frame 1502. The gripper frame has a proximal end 1504 and a distal end 1506. The gripper includes a plurality of suction cups 1508 (and / or any other suitable gripping elements). As further described herein, the suction cups 1508 may move relative to the gripper frame 1502 to lift an object and pull / carry it onto a distal conveyor 1510. In some embodiments shown in Figure 15, the distal conveyor 1510 may extend to the distal end 1506 of the gripper assembly. In some embodiments, the distal conveyor may include belts, each configured to move an object toward the proximal end. As shown in Figure 15, the gripper frame 1502 may include an inclined portion 1512. The inclined portion can help the gripper frame 1502 engage with the cargo carrier and reach objects positioned near the inner wall of the cargo carrier. Furthermore, such a configuration can help move objects on the conveyor and avoid objects sticking. The distal conveyor 1510 may be inclined together with the inclined portion 1512. The gripper assembly also includes a gripper guide 1516 configured to guide objects across a joint 1514. The joint 1514 connects the gripper frame 1502 to a segment 1524. The segment 1524 includes a segment guide 1526 that holds objects on the segment.

[0108] As shown in Figure 15, the gripper assembly 1500 includes a distance sensor 1518. The distance sensor 1518 may be a distance sensor configured to collect multiple distance measurements 1520 in the vertical direction 1522. As will be further described below, such distance measurements may supplement image information and may be used to identify objects and remove objects from a vertical stack of objects. In some embodiments shown in Figure 15, the distance sensor 1518 may acquire distance measurements 1520 below the distal gripper frame 1502.

[0109] Figure 16 is a schematic top view of the gripper assembly 1500 of Figure 15. The diagram in Figure 16 better illustrates the conveyor arrangement and joint 1514. As shown in Figure 16, the gripper assembly 1500 includes a plurality of suction cups 1508 (and / or other suitable gripping elements) and a distal conveyor 1510. In the example shown, the distal conveyor 1510 and the suction cups 1508 are staggered. For example, each suction cup is positioned between two conveyors, and the conveyor is positioned between two suction cups. Each distal conveyor 1510 includes the example belt in Figure 16, configured to support an object and move the object proximal toward a segment 1524. The segment 1524 includes a segment conveyor 1600, configured to receive an object and continue moving the object proximal toward a segment.

[0110] The joint 1514 shown in Figure 16 provides rotational freedom to the gripper frame 1502, which will be described with reference to other embodiments herein. In the example shown, the joint 1514 includes a first joint portion 1604 (e.g., a socket portion) and a second joint portion 1606 (e.g., a ball portion) configured to rotate within the first joint portion 1604. The joint 1514 also includes a plurality of rollers 1602 that can be driven to move an object across the first joint from a distal conveyor 1510 to a segment conveyor 1600. In some embodiments, at least some of the rollers can be driven to rotate. In some embodiments, at least some of the rollers may be passive and freely rotating. As shown in Figure 16, the rollers on the first joint portion 1604 and the rollers on the second joint portion 1606 overlap each other so that even as the joint moves, an object can move from the first joint portion to the second joint portion on the rollers.

[0111] As shown in Figure 16, the gripper assembly 1500 further includes a distance sensor 1608. The distance sensor 1608 may be a distance sensor configured to collect multiple distance measurements 1610 in the horizontal direction 1612. As will be further described below, such distance measurements may supplement image information and may be used to identify objects and remove objects from vertical stacks of objects. In some embodiments shown in Figure 15, the distance sensor 1608 may acquire its distance measurements 1610 in the distal direction. In some embodiments, the distance measurements 1610 may be taken below the gripper frame 1502. In some embodiments, the distance sensor 1608 may include a distance sensor 1518. For example, a single distance sensor may be configured to acquire horizontal / lateral distance measurements. In some embodiments, two distance sensors 1608 are shown in Figure 16, but in other embodiments, a single distance sensor or any number of distance sensors may be used, as this disclosure is not limited in that way.

[0112] Figures 17A to 17F are schematic diagrams illustrating the process of operating a robotic system according to one or more embodiments. The schematic diagrams shown in Figures 17A to 17F represent an image 1700 (e.g., a visual 2D representation, 3D representation, or a combination thereof, such as a color image or a grayscale image) and illustrate how the image is used to control a gripper (e.g., gripper 306 in Figure 3, 806 in Figure 8, 1500 in Figure 15, etc.) to remove an object from a vertical stack in a reliable and efficient manner. In the example of Figure 17A, the image could represent a first object 1702A, a second object 1702B, a third object 1702C, and a fourth object 1702D. The objects in Figures 17A to 17F may represent, for example, mixed-size boxes (e.g., mixed stockkeeping units (SKUs)) placed on a cargo carrier. Image 1700 may be imaged horizontally perpendicular to the vertical plane on which objects 1702A to 1702D are positioned (e.g., a plane generally parallel to the coronal and / or frontal plane of the cargo carrier, such as the yz plane). As shown in the examples in Figures 17A to 17F, each object has four boundaries: two side boundaries (e.g., side boundary 1707A), a top boundary, and a bottom boundary (e.g., bottom boundary 1705A).

[0113] Figure 17A is a schematic diagram showing the first state of the process. The image in Figure 17A may be acquired from one or more vision sensors (e.g., the upper and / or lower vision sensors mentioned above, such as in Figures 3 and 8). The vision sensors may be mounted on parts of the robot system. Based on image 1700, the minimum viable area (MVR) 1704 is calculated and applied to image 1700. The MVR 1704 can represent a portion of the image that has sufficient probability (e.g., according to a given confidence threshold / requirement) to correspond to one object or one continuous plane. Thus, the robot system can calculate a unique instance of the MVR 1704 for one or more of the objects 1702A to 1702D.

[0114] When calculating MVR1704, the robot system can process 2D and / or 3D features in image 1700 to identify reference or starting points, such as exposed 3D angles and corresponding edges. Using the reference, the robot system can calculate MVR1704 by determining, overlaying, or calculating a rectangular region (e.g., an axially aligned bounding box (AABB)) aligned with the reference angle / edge. The rectangular region (e.g., an edge that complements / opposes the reference edge and intersects the reference edge) can be calculated using various features, such as the minimum gripping area / shape of the gripper and / or the dimensions of a known / expected minimum object / SKU. Alternatively, the robot system can calculate the rectangular region based on features such as edges and related attributes depicted in the image. Some examples of edge attributes may include whether the detected edge is a 2D or 3D edge, the confidence value associated with the edge, whether the edge intersects with another edge, the angle between intersecting edges, the length of the edges between the intersections, the thickness or width of the edges, a measure of edge clarity, and / or the separation between an edge and its corresponding / parallel edge.

[0115] Furthermore, the robotic system can be configured to calculate MVR1704 as the region that overlaps with and / or is contained within the actual exposed surface of the corresponding object. In other words, the robotic system can be configured to include MVR1704 within the corresponding object. Otherwise, when object / edge detection provides sufficiently accurate results, the robotic system can determine that MVR1704 coincides with the exposed surface of the object, so that the edges of MVR1704 coincide with the actual edges of the corresponding object. In this way, MVR represents a safe place for holding the object, spaced apart from the boundaries of adjacent objects, such as a bottom boundary 1705A and a side boundary 1707A. As a result, MVR1704 may have a vertical delta 1706 to the bottom boundary 1705A and a horizontal delta 1708 to the side boundary 1707A. In other embodiments, MVR1704 may be assigned by one or more computer vision algorithms with error limits.

[0116] When using 3D angles (e.g., the intersection of two bisecting edges detected in 3D image data), the robotic system can calculate an initial MVR and iterate through the MVR as objects are removed from the stack to expose new 3D angles. In some embodiments shown in Figure 17A, the initial MVR may correspond to the uppermost and leftmost object depicted in the image (e.g., an object whose left vertical edge is exposed or is in contact with a container wall). In other embodiments, the identified initial MVR may correspond to the uppermost and rightmost object depicted in the image (e.g., an object whose right vertical edge is exposed or is in contact with a container wall). In some embodiments, the identified initial MVR may correspond to any uppermost object.

[0117] Figure 17B is a schematic diagram showing the second state of the process. After the MVR 1704 is calculated using image 1700, grippers 1710 (e.g., one or more of the aforementioned grippers such as those in Figures 3, 8, and 15) can be used to grasp the first object 1702A corresponding to the MVR 1704. Specifically, the robotic system can calculate the maximum number of suction cups 1712 that can fit within the MVR 1704 and their corresponding locations. The robotic system can operate actuators to position the grippers 1710 so that the target suction cup(s) 1712 align with the calculated location(s). The robotic system can then grasp the first object within the MVR 1704 using one or more suction cups 1712 (and / or other suitable gripping elements). If the gripper 1710 includes other suction cups 1712 that terminate outside the MVR 1704, the robotic system may utilize or operate only the suction cups located inside the MVR 1704 to grip the first object 1702A. In some embodiments, as described with reference to other embodiments herein, the gripper 1710 may be positioned adjacent to the first object 1702A by rotating segments and moving the chassis in translational degrees of freedom. In some embodiments, the gripper includes a vacuum generator connected to the suction cups, configured to generate a vacuum force and supply it to the suction cups. In such embodiments, gripping the first object 1702A may include positioning the suction cups 1712 in contact with the first object and generating a vacuum force in the suction cups using the vacuum generator.

[0118] Figure 17C is a schematic diagram showing a third state of the process. As shown in Figure 17C, the gripper 1710 can initially displace (e.g., lift) the first object 1702A after it has been gripped within the MVR 1704. For example, the robotic system can perform an initial lift by retracting the suction cup 1508 in Figure 15 to a higher position from a fully extended position (e.g., as shown in Figure 16 and / or having its bottom coplanar with or below the distal conveyor 1510). Lifting the first object 1702A creates a gap 1716 between the bottom boundary 1705A of the first object 1702A and the object below it (e.g., the fourth object 1702D). As shown in Figure 17C, the gripper 1710 may include a distance sensor 1714 (e.g., distance sensors 1518 and 1608 described above with respect to Figures 15 and 16) configured to acquire multiple distance measurements in the vertical and / or lateral directions. In the example of Figure 17C, the distance sensor 1714 is configured to acquire a series of distance measurements in the vertical direction (e.g., z-direction) across the gap 1716 and the bottom boundary 1705A of the first object 1702A. In some embodiments, the distance sensor may be a laser rangefinder that measures distance by, for example, the time-of-flight or phase shift of a laser. Multiple distance measurements may be used to detect the position of the bottom boundary 1705A of the first object 1702A. For example, there may be a stepwise change between the measurement of the gap 1716 and the measurement of the bottom boundary 1705A. In some cases, such a stepwise change may indicate the presence of the bottom boundary 1705A. In some such embodiments, a change in distance measurement may be compared to a predetermined non-zero threshold, where exceeding the threshold indicates a bottom boundary 1705A. In other embodiments, the disclosure is not so limited, and other criteria may be used, such as a profile of distance measurement that matches a predetermined profile.

[0119] Figure 17D is a schematic diagram showing a fourth state of the process. In some cases shown in Figure 17D, the first object 1702A may be released by the gripper, and as a result, the gap 1716 in Figure 17C is eliminated. For example, the engaged suction cup 1508 may be returned to its fully extended position and then stopped moving to release the object at or around its initial position. Based on distance measurements taken in the vertical direction, the position of the bottom boundary 1705A may be identified, and the MVR 1704 may be updated to eliminate the vertical delta 1706 shown in Figures 17A-17C. For example, the robotic system may re-establish the bottom edge of the MVR 1704 and / or verify the bottom edge according to the bottom boundary 1705A observed during the initial lift. Thus, at the point of the state in Figure 17D, the MVR may have vertical dimensions that match the vertical dimensions of the first object 1702A. In some cases, the step of releasing the first object shown in Figure 17D may be optional.

[0120] Figure 17E is a schematic diagram showing the fifth state of the process. As shown in Figure 17E, the first object 1702A is gripped within the updated MVR 1704 using one or more suction cups 1712. For example, the robotic system can position the suction cups 1712 closer to or aligned with the verified bottom boundary 1705A. Furthermore, based on one or more lateral distance measurements from the first initial lift, the robotic system can verify that the previous minimum gap width exceeds the lateral dimension of the MVR 1704. Thus, the robotic system can determine that the lateral dimension of the MVR 1704 can be expanded accordingly and additional suction cups can be used to grip the object. Based on the re-established or updated grip, the gripper 1710 further lifts the first object 1702A to create a gap 1716 again. In some embodiments, the suction cup 1712 can lift the first object 1702A relative to the gripper conveyor, for example, in a vertical direction.

[0121] As shown in Figure 17E, the gripper 1710 may include a second distance sensor 1718 configured to acquire a plurality of lateral distance measurements. In the example in Figure 17E, the second distance sensor 1718 is configured to acquire a series of distance measurements in the horizontal direction (e.g., the y-direction) across the gap 1716 and the lateral boundary 1707A of the first object 1702A. In some embodiments, the second distance sensor may be a laser rangefinder that measures distance by, for example, the time-of-flight or phase shift of a laser.

[0122] Multiple distance measurements may be used to detect the location of the side boundary 1707A of the first object 1702A. For example, there may be a stepwise change between the measurement of the gap 1716 and the measurement of the second object 1702B adjacent to the first object 1702A. In some cases, such a stepwise change may indicate the presence of the side boundary 1707A, inferred from the boundary shared with the second object 1702B. In some such embodiments, the change in distance measurements may be compared to a predetermined non-zero threshold, and exceeding the threshold indicates the side boundary 1707A. In other embodiments, the disclosure is not so limited, and other criteria may be used, such as a profile of distance measurements that matches a predetermined profile. In some embodiments, the distance sensor 1714 and the second distance sensor 1718 may be a single distance sensor.

[0123] For illustrative purposes, the robotic system is described as performing two initial lifts using corresponding directional measurements to detect / confirm the actual edge. However, it is understood that the robotic system can perform the measurements and confirm the edge with a single initial lift. For example, one or more distance sensors (e.g., LiDAR sensors) may be used to acquire distance measurements in the vertical and horizontal directions. Thus, the measurements shown in Figure 17E and Figure 17C may be taken following or simultaneously during a single initial lift.

[0124] Figure 17F is a schematic diagram showing the sixth state of the process. In some cases, as shown in Figure 17F, the first object 1702A can be released again by the gripper, and as a result, the gap 1716 in Figure 17E is eliminated. Based on distance measurements taken in the horizontal direction, the position of the side boundary 1707A can be identified, and the MVR 1704 can be updated or enlarged to eliminate the horizontal delta 1708 shown in Figures 17A to 17E. Thus, at the point of the state in Figure 17F, the MVR shares the horizontal dimension with the first object 1702A.

[0125] When the MVR 1704 is updated vertically and horizontally, the gripper 1710 re-gripping the first object 1702A across the entire MVR, for example, using multiple / additional suction cups 1712. In some embodiments, the multiple suction cups 1712 may be arranged in a straight line (e.g., horizontally). For example, the first suction cup, the second suction cup, and the third suction cup may be arranged in a straight line (e.g., in the y-direction), as shown in Figure 17F. In some embodiments, once the MVR 1704 is fully updated and thus verified, the gripper 1710 may grip the first object close to the bottom boundary 1705A.

[0126] For illustrative purposes, the robotic system is described as releasing or repositioning the object after the initial lift and then re-gripping it according to the validated MVR. However, it is understood that the robotic system may update / validate the MVR, identify additional suction cups, and operate additional suction cups without releasing or repositioning the object. In other words, the robotic system may determine and apply additional gripping while the object is in the initially lifted position.

[0127] The process shown in Figures 17A to 17F can be repeated for other objects in the vertical stack within image 1700 (e.g., objects with a detection confidence value below a predetermined threshold). If the first object is removed after updating the MVR 1704, the MVR can be subtracted from image 1700. For example, the robotic system can adjust or update the initial image captured by the upper / lower image sensors by overlaying the verified MVR of the removed object onto the initial image and treating the overlaid MVR as a gap or empty space. Thus, the next MVR can be assigned based on the remaining image, including other objects. For example, the robotic system can identify object 1702B as the next target object and consider its upper left corner and left edge (which previously contacted object 1702A) to be exposed based on the update to image 1700. Furthermore, based on the update to image 1700, the robotic system can similarly identify object 1702D as having its upper left corner and upper edge (which was previously in contact with object 1702A) exposed.

[0128] In this way, the robotic system can also remove multiple objects or entire stacks (e.g., exposed layers of objects) using a single image provided by the upper / lower image sensors. In some embodiments, the process described with reference to Figures 17A–17F may be repeated for multiple objects or each object in a vertical stack. In some embodiments, the robotic system may prioritize the removal of objects at the highest height in the cargo container (e.g., the leftmost or rightmost object located at the highest height in the corresponding layer / stack) from left to right.

[0129] Furthermore, the robotic system can prioritize sufficiently detected objects. When detecting an object, the robotic system can determine that the depicted portion of image 1700 in Figure 17A (e.g., an area bounded by sufficiently detected edges) matches the corresponding attributes of the registered object in the master data 252 in Figure 2 (e.g., dimensions and / or surface image / texture). When the confidence level of the match exceeds a predetermined detection threshold, the robotic system can consider / recognize the corresponding portion of the image as representing the registered object. Based on image 1700 and the registered attributes, the robotic system can identify and / or confirm the location of the object's edges by using the matched portion and extrapolating the surface using the registered dimensions. Based on the detection, the robotic system can prioritize the removal of the detected object in the upper column / layer as shown in the image. Furthermore, or alternatively, the robotic system can consider portions of the image adjacent to the detected object as either empty or belonging to an unrecognized object (e.g., by comparing them with the 3D portion of the image). Similar to the use of the verified MVR of a removed object, the robotic system can consider the edges / corners of an unrecognized object in contact with the detected object to be exposed. Therefore, the robotic system can leverage the detected object when calculating the MVR of the unrecognized object, as described above.

[0130] Figure 18 is a flowchart illustrating the process of operating a robotic system according to one or more embodiments. In some embodiments, the flowchart in Figure 18 may correspond to the processes shown in Figures 17A to 17F. In block 1802, the process includes identifying the MVR corresponding to a first object among a plurality of objects based on images acquired from one or more visual sensors (e.g., upper image sensor / lower image sensor, such as sensors located between the chassis and the gripper). In block 1804, the process further includes commanding the gripper to grasp and lift the first object within the MVR. In block 1806, the process includes acquiring a plurality of vertical distance measurements using one or more distance sensors. In block 1808, the process includes detecting the bottom boundary of the first object based on the plurality of vertical distance measurements. In block 1810, the process includes updating the vertical dimensions of the MVR based on the detected bottom boundary of the first object. In block 1812, the process includes obtaining multiple horizontal distance measurements below the detected bottom boundary using one or more distance sensors. In block 1814, the process includes detecting the side boundary of the first object based on the multiple horizontal distance measurements. In block 1816, the process includes updating the horizontal dimension of the MVR based on the detected side boundary of the first object.

[0131] As described above, the process may further include grasping an object based on the updated / verified MVR and transporting the grasped object. For example, the robotic system can transport a grasped object from a container onto a conveyor segment in a warehouse (e.g., via a gripper and conveyor on the section) as described above. Based on the verified MVR and / or the removal of the corresponding object, the robotic system can update the image. The robotic system can use the updated image when processing the next target object. Thus, the robotic system can iteratively implement the process and remove multiple unrecognized and / or detected objects using a single image. The robotic system can acquire a new image based on reaching predetermined conditions such as the removal of a predetermined number of objects, the removal of all exposed / accessible areas depicted in the image, and / or other similar operating conditions. An example chassis of a robotic system

[0132] Figure 19 is a perspective view of a robot system 1900 according to one or more embodiments of the present technology. The robot system 1900 may be an example of the robot system 100 described above, with reference to Figure 1. The robot system 1900 is positioned on top of a conveyor segment 1920, which may already be present in a warehouse or other work site. In the shown embodiment, the robot system 1900 includes a chassis 1902, a first segment 1904 coupled to the chassis 1902 and extending toward the distal portion 1901a of the robot system 1900, a second segment 1921 coupled to the chassis 1902 and extending toward the proximal portion 1901b of the robot system 1900, and a gripper 1906 coupled to the first segment 1904 at the distal portion 1901a. The robot system 1900 may also include support legs 1910 coupled to the chassis 1902, one or more controllers (individually labeled 1938a, 1938b and collectively referred to as "controllers 1938"), one or more counterweights supported by the chassis 1902 (individually labeled 1939a, 1939b and collectively referred to as "counterweights 1939"), a first joint roller 1909 coupled between the first segment 1904 and the gripper 1906, and a second joint roller 1937 coupled between the first segment 1904 and the second segment 1921. The chassis 1902, the first segment 1904, the second segment 1921, the support legs 1910, and / or other components of the robot system 1900 may be made from metal (e.g., aluminum, stainless steel), plastic, and / or other suitable materials.

[0133] The chassis 1902 may include a frame structure supporting a first segment 1904, a second segment 1921, a controller 1938, a counterweight 1939, and / or a sensor mount 1930 coupled to the chassis 1902. In the shown embodiments, the sensor mount 1930 extends vertically on both sides of the first segment 1904 and horizontally on the first segment 1904. One or more sensors 1924 (e.g., vision sensors) are coupled to the sensor mount 1930 and positioned to generally face the distal portion 1901a. In some embodiments, the sensor mount 1930 does not extend horizontally on the first segment 1904 so that cargo 1934 can move along the position of the first segment 1904 without height restrictions imposed by the sensor mount 1930.

[0134] The first segment 1904 is coupled to extend cantilever-like from the chassis 1902 toward the distal portion 1901a. The first segment 1904 supports a first conveyor 1905 (e.g., a conveyor belt) extending along and / or around the first segment 1904. The second segment 1921 is coupled to extend from the chassis 1902 toward the proximal portion 1901b of the robot system 1900. The second segment 1921 supports a second conveyor 1922 (e.g., a conveyor belt) extending along and / or around the second segment 1921. In some embodiments, one or more actuators 1936 (e.g., motors) configured to move the first and second conveyors 1905, 1922 are coupled to the chassis 1902. In some embodiments, the actuator is located elsewhere (e.g., housed in or coupled to the first and / or second segments 1904, 1921). Actuator 1936 (or other actuators) can be operated to rotate the first segment 1904 around a fifth axis A5 and / or a sixth axis A6. In some embodiments, actuator 1936 can also pivot the second joint roller 1937 around the first and second axes A5, A6 or different axes. In some embodiments, as shown in Figure 19, the fifth axis A5 may be generally orthogonal to the cross-section of the chassis 1902 (e.g., the second plane P2 shown in Figure 8), while the sixth axis A6 may be generally parallel to the cross-section of the chassis 1902. As a result, the movement and rotation of the first segment 1904 relative to the chassis 1902 may be generally similar to the movement and / or rotation of the first segment 304, as described above in more detail with respect to Figures 5 to 7B.

[0135] As described above, the gripper 1906 can be coupled with a first joint roller 1909 positioned between the first segment 1904 and the distal portion 1901a, extending from the first segment 1904 toward the distal portion 1901a. In some embodiments, the gripper 1906 includes a suction cup 1940, any other suitable gripping element, and / or a distal conveyor 1942. In some embodiments, one or more actuators 1908 (e.g., motors) are configured to rotate the gripper 1906 and / or the first joint roller 1909 relative to the first segment 1904 around a seventh axis A7 and / or an eighth axis A8. As shown in Figure 19, the seventh axis A7 may be generally parallel to the longitudinal plane of the gripper 1906 (e.g., the third plane P3 shown in Figure 43A), while the eighth axis A8 may be generally perpendicular to the longitudinal plane of the gripper 1906. Furthermore, or alternatively, the seventh axis A7 is the cross-section of the gripper 1906 (for example, the fourth plane P shown in Figure 42A). 4) The eighth axis A8 may be generally orthogonal to the cross-section of the gripper 1906, while the eighth axis A8 may be generally parallel to the cross-section of the gripper 1906. In some embodiments, as will be described in more detail below, the robot system 1900 can maintain a cross-section of the gripper 1906 that is generally parallel to the cross-section of the chassis 1902, (for example, a rotation around the sixth axis A6 is filled by a counter-rotation around the eighth axis A8). As a result, in some embodiments, for example, the seventh axis A7 may be generally orthogonal to the cross-section of the chassis 1902, and / or the eighth axis A8 may be generally parallel to the cross-section of the chassis 1902.

[0136] In some embodiments, the actuator 1908 (or other actuator) is configured to operate the suction cup 1940 and / or the distal conveyor 1942. In some embodiments, the actuator 1908 is coupled to the first segment 1904, the first joint roller 1909, and / or the gripper 1906. The movement and / or rotation of the gripper 1906 relative to the components of the first segment 1904 and the gripper 1906 are described in further detail herein.

[0137] In the embodiment shown, two front support legs 1910a are rotatably coupled to the chassis 1902 around their respective front pivots 1916a (see Figure 20), which are located on either side of the chassis 1902. Front wheels 1912a are mounted on the distal portion of each front support leg 1910a. Similarly, two rear support legs 1910b are rotatably coupled to the chassis 1902 around their respective rear pivots 1916b, which are located on either side of the chassis 1902. Rear wheels 1912b are mounted on the distal portion of each rear support leg 1910b. The chassis 1902 also supports two front actuators 1914a (e.g., linear actuators, motors) (see Figure 20) operably coupled to the front support legs 1910a, and two rear actuators 1914b operably coupled to the rear support legs 1910b. In some embodiments, the robot system 1900 includes fewer or more support legs 1910, and / or support legs 1910 configured in different positions and / or orientations. In some embodiments, the wheels 1912 can be motorized to move the chassis 1902, and thus the rest of the robot system 1900, along the linear direction L2. The operation of the actuators 1914 will be described in further detail below with reference to Figures 22 and 23.

[0138] The controller 1938 (for example, the processor(s) 202 in Figure 2) can be operablely coupled (e.g., wired or wirelessly) to control the actuators 1908, 1936, 1914 and / or other actuators (e.g., corresponding to the actuation device 212 in Figure 2). The counterweight 1939 can be positioned (e.g., toward the proximal portion 1901b) to counteract any moment applied to the chassis 1902 by the cargo 1934 carried by the gripper 1906 and / or the first segment 1904.

[0139] Figure 20 is an enlarged side view of a robot system 1900 according to an embodiment of the present technology. As shown, the first segment 1904 is rotatable around axes A5, A6, but axes A5, A6 may not intersect and instead be separated by a distance D9. The distance D9 may be approximately 200 mm, 300 mm, 400 mm, 500 mm, 600 mm, any distance between them, or other distances. When the chassis 1902 is positioned above the conveyor segment 1920 as shown, the sixth axis A6 may be positioned at a distance D10 from the floor on which the conveyor segment 1920 and the wheel 1912 are located. The distance D10 may be approximately 1100 mm, 1200 mm, 1300 mm, 1400 mm, 1500 mm, any distance between them, or other distances. However, as will be described in more detail herein, the wheel 1912 may be moved vertically to change the distance D10.

[0140] The sixth axis and the eighth axis A 6、 A8 may be separated horizontally by only a distance D11 (for example, along the first segment 1904). Distance D11 may be approximately 3000mm, 3500mm, 4000mm, 4500mm, 5000mm, any distance between them, or any other distance.

[0141] The gripper 1906 is rotatable around axes A7 and A8, but axes A7 and A8 may not intersect and instead be separated by a distance D12. The distance D12 may be approximately 220 mm, 250 mm, 280 mm, 310 mm, 340 mm, any distance between them, or other distances. When the chassis 1902 is positioned above the conveyor segment 1920 as shown, and the first segment 1904 remains horizontal, the eighth axis A8 can be positioned at a distance D13 from the floor on which the conveyor segment 1920 and wheel 1912 are located. The distance D13 may be approximately 1200 mm, 1300 mm, 1400 mm, 1500 mm, 1600 mm, any distance between them, or other distances. However, as will be described in more detail herein, the first segment 1904 may be rotated around the sixth axis A6 to change the distance D13.

[0142] Figure 21 is a perspective view of a robotic system 1900 on the floor of a warehouse 1972 according to an embodiment of the present technology. As described above, the robotic system 1900 may include two front wheels 1912a, each positioned on either side of the chassis 1902 and near the first segment 1904, and two rear wheels 1912b, each positioned on either side of the chassis 1902 and near the second segment 1921 (one of which is not visible in the figure). Each front wheel 1912a is coupled to a front support leg 1910a rotatably mounted to the chassis 1902 around a front pivot 1916a, and each rear wheel 1912b is coupled to a rear support leg 1910b rotatably mounted to the chassis 1902 around a rear pivot 1916b. The front actuator 1914a and the rear actuator 1914b are connected between the chassis 1902 and the front support leg 1910a and the rear support leg 1910b, respectively.

[0143] In the embodiment shown, each support leg 1910 has a triangular shape with a first vertex connected to a pivot 1916, a second vertex connected to a wheel 1912, and a third vertex connected to an actuator 1914. Furthermore, the actuators 1914 (e.g., electric linear actuators) can be coupled to the chassis 1902 between the front pivot and the rear pivot 1916 so that during operation, the first actuator 1914a can push the front support leg 1910a forward and pull the front support leg 1910a backward, and the rear actuator 1914b can push the rear support leg 1910b backward and pull the front support leg 1910a forward. When the actuator 1914 pushes the support leg 1910, the corresponding wheel 1912 is lifted vertically from the floor 1972. Conversely, when the actuator 1914 pulls the support leg 1910, the corresponding wheel 1912 descends vertically toward the floor 1972. As described above with respect to Figure 4, descending the wheel 1912 may be advantageous when moving the robot system 1900 to a lower floor. Furthermore, in some embodiments, the vertical distances by which the wheel 1912 can be raised and / or lowered may generally be similar to the distances D1 and D2 described above with respect to Figure 4.

[0144] Figures 22 and 23 are enlarged side views of a robot system 1900 showing the operation of a support leg according to an embodiment of the present technology. In some embodiments, the four actuators 1914 (e.g., two front actuators 1914a and two rear actuators 1914b) can be operated independently of each other. For example, comparing Figure 23 with Figure 22, the two front actuators 1914a can be operated to lift the first segment 1904, thereby rotating the chassis 1902 in one direction around the pitch axis, while the two rear actuators 1914b remain stationary. More specifically, the front actuators 1914a can be operated to pull the front support leg 1910a so that the front wheel 1912a remains in contact with the ground and the front pivot 1916a rises accordingly.

[0145] In another example, two rear actuators 1914b can be operated to lift a second segment 1921 while the two front actuators 1914a remain stationary, thereby rotating the chassis 1902 in opposite directions around the pitch axis. In yet another example, the right front and rear actuators 1914 (e.g., shown in Figures 22 and 23) can be operated to lift the right side of the chassis 1902 while the left front and rear actuators 1914 (e.g., difficult to see in the figure) remain stationary, so that the chassis 1902 rotates around the roll axis. In yet another example, four actuators 1914 can be operated to move by different amounts to also achieve rotation of the chassis 1902 around the pitch axis and / or roll axis. Other combinations of controlling the four actuators 1914 are within the scope of this art.

[0146] Raising, lowering, and / or rotating the chassis 1902 around the pitch axis and / or roll axis may be advantageous in extending the range of the gripper 1906, operating the robot system 1900 through a constrained space, and shifting the weight distribution on the robot system 1900 and the mechanical stresses acting on it. In some embodiments, the robot system 1900 also includes sensors (e.g., distance sensors) coupled to the chassis 1902 to measure and detect, for example, the degree of rotation of each support leg 1910 and / or the height of the wheels 1912 relative to the chassis 1902.

[0147] Figure 24 is an enlarged perspective view of a front wheel 1912a according to an embodiment of the present technology. In the embodiments shown, a motor 2410 is operably coupled to each front wheel 1912a. During operation, the motor 2410 can be used to drive the front wheels 1912a and move the robot system 1900 in a desired direction (e.g., forward, backward). In some embodiments, the motor 2410 is coupled to a reducer (e.g., a gearbox) and / or braking component so that the speed and acceleration of the front wheels 1912a can be controlled for deceleration and / or braking.

[0148] In some embodiments, the front wheel 1912a is motorized as shown, while the rear wheel 1912b is not. In some embodiments, the rear wheel 1912b is motorized instead or even further. In some embodiments, the front wheel 1912a is made of a material with relatively high traction (e.g., rubber), while the rear wheel 1912b is made of a material with relatively normal traction (e.g., polyurethane). Different materials can help improve consistency between the extension and contraction direction of the conveyor segment 1920 and the direction of movement of the robot system 1900.

[0149] Figure 25 is an enlarged perspective view of the rear support leg 1910b and the corresponding rear wheel 1912b according to an embodiment of the present technology. As shown, the robot system 1900 includes a stopper 2510 positioned above the rear wheel 1912b. The stopper 2510 can be coupled to the chassis 1902. The stopper 2510 can be configured to define the maximum rotational degree of the rear support leg 1910b by physically preventing the rear support leg 1910b and / or the rear wheel 1912b from moving beyond the stopper 2510. The stopper 2510 can be made from silicone, rubber, or other suitable material to avoid damaging the rear support leg 1910b and / or the rear wheel 1912b. In some embodiments, the stopper 2510 is relied upon only under emergency conditions, such as when the rear actuator 1914b fails and / or detaches from the chassis 1902. In some embodiments, the robot system 1900 may instead or further include other stoppers configured to define the maximum degree of rotation relative to the front support leg 1910a.

[0150] In some embodiments, a method for operating a robot system (e.g., robot system 1900) involves obtaining an image from one or more sensors (e.g., sensor 1924) of at least one object (e.g., cargo 1934) that is engaged by a gripper (e.g., gripper 1906) and transported along the chassis conveyor belt of the chassis (e.g., chassis 1902) and the arm conveyor belt of the arm (e.g., first segment 1904), and determining (1) the first position of the chassis or the second position of the chassis based on the image. The method includes determining (1) an angular position, (2) a second position of the gripper, and (3) a second angular position of the arm; acting (e.g., via actuator 1914) one or more support legs (e.g., support legs 1910) coupled to the chassis so that the chassis is at least in the first position or the first angular position; and acting one or more joints (e.g., around axes A5-A8) of the robot system so that the gripper is in the second position and the arm is in the second angular position.

[0151] In some embodiments, the combination of first and second angular positions is configured to prevent, or at least reduce, the slippage of an object along the chassis conveyor belt and / or arm conveyor belt. In some embodiments, the method further includes detecting slippage of an object along the arm conveyor belt. Upon detecting such slippage, the method may further include raising or lowering the first position of the chassis while maintaining the gripper in a second position, thereby acting one or more support legs to lower the second angular position of the arm. Alternatively, the method may further include raising or lowering the second position of the gripper while maintaining the chassis in a first position, thereby acting one or more joints to lower the second angular position of the arm. Alternatively, the method may further include acting one or more support legs to raise or lower the first position of the chassis and acting one or more joints to raise or lower the second position of the gripper, thereby lowering the second angular position of the arm.

[0152] In some embodiments, the method further includes detecting the slippage of an object along a chassis conveyor belt via one or more sensors and acting one or more support legs to lower a first angular position of the chassis. In some embodiments, the method further includes detecting the tilt of the robot system caused by a stepped surface on which the robot system is placed and acting one or more support legs to compensate for the tilt of the robot system caused by the stepped surface. For example, the surface may be stepped such that the chassis is tilted at an angle (e.g., laterally and away from the longitudinal axis extending along the chassis conveyor belt). Support legs on both sides of the chassis can be actuated independently (e.g., by different angles) to tilt the chassis in opposite directions to compensate for the stepped surface.

[0153] In some embodiments, the method further includes driving one or more wheels (e.g., wheel 1912) attached to corresponding legs of one or more support legs to move the chassis forward or backward relative to at least one object such that the gripper maintains a second position relative to at least one object. For example, by rotating the support legs around a pivot (e.g., pivot 1916) on the chassis, the wheels maintain contact with the surface so that the chassis can move forward or backward.

[0154] In some embodiments, the robot system is positioned on the warehouse conveyor belt such that the chassis conveyor belt and the warehouse conveyor belt form a continuous movement path for at least one object, and one or more support legs are actuated to maintain the continuous movement path while the chassis is actuated to at least one of a first position or a first angular position.

[0155] In some embodiments, determining a first position or at least one of a first angular position includes determining a first range of acceptable positions or a first range of acceptable angular positions. In some embodiments, determining a second position includes determining a second range of acceptable positions. In some embodiments, determining a second angular position includes determining a second range of acceptable angular positions. In some embodiments, the first and second positions are determined with respect to a support surface on which the robot system is positioned. In some embodiments, the first and second positions are determined with respect to at least one object.

[0156] Figure 26 is a perspective view of a chassis joint 2600 for a robotic system according to one or more embodiments. As described above with reference to Figures 22 to 25, the chassis of a robotic system may have multiple degrees of freedom. For example, the independent movement of the four legs of a robotic system may (1) move the chassis with a translational degree of freedom (e.g., vertically), (2) rotate the chassis with a chassis roll degree of freedom, and (3) rotate the chassis with a chassis pitch degree of freedom. Such movement may be desirable to allow the robotic system to adapt to various environments and cargo containers, particularly in modified environments. However, a conveyor fixed to a local environment (e.g., a warehouse conveyor) is usually limited to a single degree of freedom, namely extension and contraction. As will be further described below, the chassis joint 2600 gives the chassis these degrees of freedom, while allowing the warehouse conveyor or other proximal conveyor to which the chassis is operably coupled to remain fixed to a single degree of freedom or otherwise constrained. Furthermore, controlling the robot system to maintain its relative position with respect to the distal end of an extending conveyor is difficult if the conveyor and robot system have separate controllers. As will be further described below, the chassis joint 2600 allows the robot system to automatically follow the warehouse conveyor to which the chassis is operably coupled as the warehouse conveyor extends or retracts. Conversely, in some embodiments, the chassis joint 2600 may allow the conveyor to extend or retract in accordance with the movement of the robot system chassis in the distal or proximal direction.

[0157] The chassis joint 2600 includes a conveyor mount 2602 and a chassis mount 2604. The conveyor mount 2602 is configured to be coupled to a portion of a conveyor (e.g., a warehouse conveyor or other proximal conveyor). The chassis mount 2604 is configured to be coupled to the chassis of a robot system. In some embodiments shown in Figure 26, the conveyor mount 2602 includes a conveyor mounting plate 2606 having a plurality of holes 2608 for receiving fasteners (e.g., bolts, screws, rivets, etc.). The chassis mount similarly includes a chassis mounting plate 2622 having holes 2624 configured for receiving fasteners.

[0158] According to the embodiment shown in Figure 26, the chassis mount 2604 is configured to move relative to the conveyor mount 2602 with a first translational degree of freedom 2636, such as horizontally along the proximal / distal axis. The conveyor mount 2602 includes two horizontal shafts 2610. The chassis mount includes two horizontal couplers 2612 configured to slide on the horizontal shafts. Thus, the chassis mount 2604 can slide relative to the conveyor mount 2602 in the example of Figure 26, and can accommodate relative movement between the extension of the conveyor and the movement of the robot system's chassis. In some embodiments shown in Figure 26, the chassis joint 2600 includes a spring 2614 configured to deflect the chassis mount 2604 and the conveyor mount 2602 to a predetermined position. In some embodiments, the predetermined position may be a neutral position in which the chassis mount and the conveyor mount can slide relative to each other in either direction. In some embodiments, the spring 2614 may be a compression spring.

[0159] The chassis joint 2600 includes a position sensor 2616 configured to provide information indicating the relative positions of the chassis mount 2604 and the connector or mount 2602. In some embodiments, the position sensor may be a linear potentiometer. In other embodiments, other sensors may be used, as this disclosure is not limited in that way. In some embodiments, the output of the position sensor may be received by a local controller and used to command the rotation of the wheels of the robot system. For example, a change in relative position measured by the position sensor 2616 may trigger the controller to drive the wheels of the robot system. In this way, the robot system may be automatically controlled to follow the conveyor (as indicated by the movement of the conveyor mount 2602). In other embodiments, the output of the position sensor 2616 may be received by a conveyor controller. In such embodiments, a change in relative position measured by the position sensor 2616 may trigger the conveyor controller to extend or retract the conveyor. In this way, the conveyor may be automatically controlled to follow the robot system (as indicated by the movement of the chassis mount 2604).

[0160] The chassis joint 2600 is further configured to accommodate relative vertical movement between the robot system chassis and the conveyor (for example, vertically) with a second translational degree of freedom 2638. In the example in Figure 26, the chassis mount 2604 includes two vertical shafts 2618 and two vertical couplers 2620 configured to slide on the vertical shafts 2618. The vertical shafts 2618 are attached to the chassis mounting plate 2622. Thus, the remainder of the chassis joint 2600, including the conveyor mount 2602, is configured to slide vertically along the vertical shafts 2618.

[0161] The chassis joint 2600 is further configured to accommodate relative pitch rotation between the robot system chassis and the conveyor (for example, from the movement of the chassis in the chassis pitch rotation degrees of freedom). In some embodiments, the vertical coupler 2620 may be further configured to rotate around a pitch axis perpendicular to the plane of the vertical axis of the vertical shaft 2618. According to such a configuration, the chassis mounting plate 2622 and the vertical shaft 2618 may rotate as the pitch angle of the chassis changes. The vertical coupler 2620 may pivot around its respective axis to accommodate this change in pitch angle without movement of the conveyor mount 2602.

[0162] The chassis joint 2600 is further configured to accommodate relative roll rotation between the robot system chassis and the conveyor (e.g., from the movement of the chassis in the chassis roll rotational degrees of freedom). The vertical couplers 2620 are coupled together to the axle 2626. The axle 2626 is coupled to the conveyor mount 2602 via a swivel joint 2628. The swivel joint is configured to allow the axle to rotate around the roll axis (e.g., parallel to the longitudinal axis or the distal / proximal axis plane). In some embodiments shown in Figure 26, the chassis mount includes a pair of support brackets 2630 that support the axle 2626 and allow the axle to rotate in the roll direction. The axle includes two bushings 2634 that slide within channels 2632 of each support bracket. Thus, the relative heights of the first and second vertical couplers may differ. For example, when the axle rotates within the swivel joint 2628, the first vertical coupler may move upward and the second vertical coupler may move downward. The rotation of the axle 2626 within the swivel joint 2628 may occur while the conveyor mount 2602 remains stationary.

[0163] According to the embodiment of Figure 26, a single position sensor 2616 for a first translational degree of freedom (e.g., horizontal) is included in the chassis joint 2600. In other embodiments, additional sensors may be included to monitor the relative positions of the chassis mount 2604 and the conveyor mount 2602 in other degrees of freedom. The outputs of such sensors may be received by a local controller and used, for example, to control various actuators of the robot system to avoid reaching the end of a movement. In some embodiments, the chassis joint may include a vertical position sensor configured to acquire position information of a vertical coupler 2620 on a vertical shaft 2618. In some embodiments, the chassis joint may include a pitch position sensor configured to acquire orientation information of the vertical coupler 2620 relative to the vertical axis. In some embodiments, the chassis joint may include a roll position sensor configured to acquire orientation information of an axle 2626 relative to the longitudinal axis. Any single sensor, subcombination, or combination of these sensors may be used. Sensors may include, but are not limited to, potentiometers or encoders. In some embodiments, such sensors may be located on the robot system and / or conveyor, but may not be included as part of the chassis joint.

[0164] In the embodiment shown in Figure 26, the chassis joint provides relative movement between the chassis and the conveyor with four degrees of freedom (e.g., horizontal, vertical, pitch, and roll), but in other embodiments, the chassis joint may provide fewer or more degrees of freedom. For example, in some embodiments, the chassis joint may provide only relative horizontal movement between the chassis and the conveyor. Any single relative degree of freedom, subcombination, or combination of relative degrees of freedom may be provided by the chassis joint in some embodiments.

[0165] Figure 27 is a flowchart showing a process for operating a robot system according to one or more embodiments. In block 2702, the process includes extending a telescopic conveyor in the distal direction. Extending the conveyor in the distal direction may, in some embodiments, include moving the distal end of the conveyor in the distal direction. In block 2704, the process includes sliding a conveyor mount attached to the telescopic conveyor in the distal direction relative to a chassis mount. The chassis mount may be attached to a chassis that remains stationary relative to the telescopic conveyor. In block 2706, the process includes obtaining position information indicating the relative position between the conveyor mount and the chassis mount. In some embodiments, the position information may be obtained from one or more distance sensors. In some embodiments, one or more distance sensors may include potentiometers. In block 2708, the process includes comparing the distance information against one or more criteria. In some embodiments, the criteria may be numerical thresholds. For example, the magnitude of the position change indicated by the position information may be compared against a predetermined non-zero threshold.

[0166] In block 2710, the process includes driving wheels operably coupled to the chassis and commanding a wheel motor to move the chassis distally based on a comparison with a reference. For example, if the magnitude of the change in position indicated by position information exceeds a predetermined non-zero threshold, the wheel motor may be commanded to rotate the wheel and move the chassis distally. In some embodiments, the speed of the wheel motor may be controlled based on the position information. For example, the wheel motor may be controlled to move the chassis to maintain a neutral position relative to the telescopic conveyor. For example, if the change in relative position is large, the wheel speed may be accelerated to allow a delta from the neutral position to decrease. Correspondingly, as the delta decreases and the telescopic conveyor and chassis approach their neutral positions relative to each other, the wheel speed may be decelerated to match the speed of the distal end of the conveyor. Thus, the method may include driving the wheel motor to ensure that the chassis follows the telescopic conveyor. In other embodiments, the process may be reversed so that the conveyor is controlled to follow the chassis. In the optional Act 2712, the process includes spring-biasing the conveyor mount and chassis mount to a neutral position. The springs may reduce impact loads and may help the chassis return to a neutral position relative to the telescopic conveyor.

[0167] Figure 28 is a front view of a robot system 2800 and chassis joint in a first state according to one or more embodiments, and Figure 29 is a front view of the robot system in a second state. The views in Figures 28 to 29 are taken proximal along the longitudinal axis of the robot system 2800. As shown in Figure 28, the robot system 2800 includes a chassis 2802, a first leg 2804A, and a second leg 2804B. The first leg 2804A is located on the first side of the chassis 2802, and the second leg 2804B is located on the second side of the chassis 2802. The first leg 2804A includes a first wheel 2806A, and the second leg includes a second wheel 2806B. The first and second wheels are configured to rotate, allowing the chassis to move with translational degrees of freedom corresponding to movement along the longitudinal axis of the robot system (e.g., moving the chassis in the proximal or distal direction). In some embodiments shown in Figure 28, the first wheel 2806A is coupled to a first wheel motor 2808A, and the second wheel 2806B is coupled to a second wheel motor 2808B. The first wheel may be driven directly by the first wheel motor, and the second wheel may be driven directly by the second wheel motor. Furthermore, in some embodiments, the wheels may be driven independently. The first wheel 2086A and the second wheel 2806B may be front wheels formed from a rubber material. In some embodiments, this rubber material may be a different material from the rear wheel material, and in some embodiments may be polyurethane.

[0168] As shown in Figure 28, the robot system includes a chassis mount 2810. In some embodiments, the chassis mount may be similar to that shown and described with reference to Figure 26. In the example shown, the chassis mount 2810 includes an axle 2812. The axle 2812 is connected at both ends to a vertical coupler 2814, one of which is shown through a transparent film. The vertical coupler 2814 is configured to slide along a vertical shaft 2816 mounted on the chassis 2802. The axle 2812 may be configured to rotate in a roll direction 2900, for example, inwards around the axle (for example, parallel to the longitudinal axis of the robot system). As the axle 2812 rotates in the roll direction 2900, one end of the axle may move upward and the other downward. Thus, one vertical coupler 2814 may move upward and the other may move downward. In some embodiments shown in Figure 29, the axle 2812 slides within the support bracket 2818. Such rotation may allow the chassis mount 2810 to accommodate the rotation of the chassis 2802 in the chassis's roll degrees of freedom. As shown in Figure 29, as the chassis rolls, the axle 2812 may also roll, while allowing the associated conveyor to maintain a fixed orientation. Rolling of the chassis 2802 may be caused by irregularities in the floor 2824 of the operating environment, such as bumps, holes, or uneven surfaces. Rolling of the chassis 2802 may also be caused by differences in height between the first leg 2804A and the second leg 2804B.

[0169] In some embodiments shown in Figures 28-29, the robot system 2800 may include a vision sensor 2820 positioned below segment 2822 of the robot system, as described with reference to other embodiments herein. Such a configuration may enable the vision sensor 2820 to more easily acquire images of multiple objects within the cargo carrier with less interference from segment 2822 and other components of the robot system.

[0170] Figure 30 is a flowchart illustrating a process for operating a robot system according to one or more embodiments. In block 3002, the process includes moving a first leg coupled to the chassis vertically, independently of a second leg coupled to the chassis. In block 3004, the process includes rotating the chassis within a chassis roll rotational degree of freedom. In block 3006, the process includes rotating the axle around an axle roll rotational degree of freedom in response to the rotation of the chassis. In block 3008, the process includes moving a first vertical coupler in a first direction on a first vertical shaft. In block 3010, the method includes moving a second vertical coupler in a second direction opposite to the first direction on a second vertical shaft. A robotic system (for example, robotic system 1900 in Figure 19) can use one or more controllers, such as the controller 1938 in Figure 19 and the circuits within it, to operate various actuators (for example, actuators 1908 and 1936 in Figure 19, which correspond to actuator device 212 in Figure 2) to perform one or more of the above-described operations.

[0171] Exemplary EOAT for robotic systems Figure 31 is a partial schematic isometric view of a robotic system 3100 configured according to several embodiments of the present technology. In the embodiments shown, the robotic system 3100 includes a movable arm 3110, an end effector 3120, and a distal joint 3130 operably coupled between the movable arm 3110 and the end effector 3120. The movable arm 3110 may be generally similar to any of the movable arms described above with reference to Figures 3 to 12F, and positions the end effector 3120 (sometimes referred to herein as the “end-of-arm tool”) adjacent to one or more target objects (e.g., boxes in cargo carriers such as shipping containers and / or trucks). Furthermore or alternatively, the movable arm 3110 can establish a transport path between the target objects and unloading units (e.g., warehouse conveyor systems, warehouse carts, warehouse trucks, etc.). As will be described in more detail below, the end effector 3120 and the distal joint 3130 may include various features that help pick up and / or otherwise grasp target objects from various locations. For example, the end effector 3120 may include features that allow the robot system 3100 to lift individual objects onto the conveyor system at least partially, picking up individual objects without (or with minimal) interference with surrounding objects. In another example, the distal joint 3130 may include various features that help improve the range of motion of the robot system 3100 and / or the end effector 3120 within it.

[0172] Figures 32A and 32B are a partial schematic upper and lower side view and a schematic lower side view of an end effector 3200 configured according to several embodiments of the present technology. The end effector 3200 may be generally similar to the end effectors (sometimes called “end-of-arm tools,” “grippers,” etc.) described above with reference to Figures 3 to 12F, 15, 19 to 21, etc. As shown in Figure 32A, the end effector 3200 includes a frame 3210, a plurality of joint conveyors 3220, a plurality of frame conveyors 3230, and a gripping component 3240. The frame 3210 has a proximal end region 3212 that can be coupled to a robotic system (for example, via a distal joint 3130 in Figure 31) and a distal end region 3214 opposite the proximal end region 3212. The plurality of joint conveyors 3220 are coupled to the proximal end region 3212 of the frame 3210. Each of the multiple frame conveyors 3230 extends from the distal end region 3214 to the proximal end region 3212.

[0173] In the embodiment shown in Figure 32A, the multiple joint conveyors 3220 are rollers extending laterally along the transverse axis of the end effector 3200, while the multiple frame conveyors 3230 include multiple individual conveyor belts 3232 extending along (or generally parallel to) the longitudinal axis of the end effector 3200. Each individual conveyor belt 3232 is operably coupled to a common drive component 3234 (e.g., a common drive pulley, drive shaft, etc.) to operate each of the multiple frame conveyors 3230 at the same (or generally the same) speed.

[0174] As further shown in Figure 32A, each of the individual conveyor belts 3232 is spaced apart from adjacent conveyor belts to define channels 3236 between the individual conveyor belts 3232. The gripping components 3240 (sometimes referred to herein as “gripping components”) include a drive component 3242 supported by the frame 3210, and a plurality of gripping assemblies 3250, each including an extendable component 3252 and a gripping element 3254 (sometimes referred to herein as “gripper element,” “engaging element,” etc.) supported by the extendable component 3252. Each gripping assembly (sometimes referred to herein as “gripper assembly”) is coupled to the drive component 3242 and positioned in one of the channels 3236. Therefore, the drive component 3242 can move each of the multiple gripping assemblies 3250 along a first motion path 3262 between the distal end region 3214 and the proximal end region 3212 (for example, generally along the longitudinal axis of the end effector 3200) within and / or above one of the corresponding channels 3236. Furthermore, each of the extendable components 3252 can move one of the corresponding gripping elements 3254 along a second motion path 3264.

[0175] As will be described in more detail below, during a gripping operation using the end effector 3200, the gripping component 3240 can pick up (and / or otherwise grip) the object over the distal end region 3214 of the frame 3210, place (and / or otherwise release) the object on top of the frame conveyor 3230, and move between various positions to clear a path for the object to move proximally along the frame conveyor 3230. Furthermore, when the object is placed on multiple frame conveyors 3230, the multiple frame conveyors 3230 and the multiple joint conveyors 3220 can move the object proximally (for example, toward the movable base component to unload a cargo carrier). Furthermore, or alternatively, multiple joint conveyors 3220 and multiple frame conveyors 3230 move the object distally, and then the gripping components 3240 can pick up the object and place it distally in the distal end region 3214 of the frame 3210 (for example, to load a cargo carrier, which may be referred to herein as a “shipping unit”).

[0176] As best illustrated in Figure 32B, the end effector 3200 may also include one or more sensors 3270 (three are shown in Figure 32B). Sensors 3270 may include proximity sensors, image sensors, motion sensors, and / or any other suitable sensors to help identify one or more object objects, to help identify the location of one or more object objects, etc. In certain non-limiting examples, sensor 3270 may include an image sensor that images a shipping unit to enable a suitable component (e.g., processor 202 in Figure 2 and / or another suitable component) to identify one or more object objects in a shipping container and / or an action plan for unpacking the shipping unit. While unpacking the shipping unit, sensor 3270 may then monitor the environment around the shipping unit and / or the end effector 3200 to prompt changes to the action plan and / or detect changes in the environment. For example, if one or more target objects shift (fall, tilt, rotate, and / or otherwise move) during the unpacking process, the motion plan can be updated. In another example, the sensor 3270 can detect and avoid hazards in the environment around the end effector 3200 (e.g., humans or other living creatures, other robotic units, movement of shipping units, etc.).

[0177] Figures 33A to 33F are partial schematic side views of the end effector 3300 at various stages of the process for picking up an object according to some embodiments of the present technology. As shown in Figure 33A, the end effector 3300 may be generally similar to (or identical to) the end effector 3200 described above with reference to Figures 32A and 32B. For example, the end effector 3300 (sometimes referred to herein as an end-of-arm tool) includes a frame 3310, a plurality of joint conveyors 3320, a plurality of frame conveyors 3330, and a gripper component 3340.

[0178] Figure 33A shows the end effector 3300 after the target object 3302 has been identified (for example, using the sensor 3270 and / or any other suitable sensor described above with reference to Figure 32B) and the end effector 3300 has been positioned adjacent to the target object 3302. At this position, the target object 3302 is distal to the distal end region 3314 of the frame 3310.

[0179] Figure 33B shows the end effector 3300 while acting the gripper component 3340 distally toward the distal end region 3314. In various embodiments, the end effector 3300 can actuate the gripper component 3340 by extending (or retracting) expandable components (e.g., pistons, gripping mechanisms, etc.), driving one or more carts along a guide track, driving pulleys for moving belts and / or gear tracks coupled to the gripper component 3340, and / or any other suitable mechanism. As will be described in more detail below, the end effector 3300 can actuate the gripper component 3340 by moving a common drive component 3342 to move multiple gripping assemblies 3350 in a column (e.g., in parallel, generally simultaneously, etc.). Next, simultaneous movement of the gripping assembly 3350 may help ensure that the gripping assembly 3350 is aligned to its distal point, and may help ensure that the gripping elements 3354 within the gripping assembly 3350 can engage an object (e.g., target object 3302) simultaneously (or nearly simultaneously).

[0180] Figure 33C shows the end effector 3300 after one or more gripping elements 3354 (sometimes referred to herein as “gripper elements,” “engaging elements,” etc.) in the gripping assembly 3350 are distally positioned in the distal end region 3314 and operated to engage the object 3302 (sometimes referred to herein as “first position,” “pickup position,” and “engaging position,” etc.). In various embodiments, the gripping element 3354 may include vacuum components (sometimes referred to herein as suction components), magnetic components, mechanical gripper components, etc., for engaging the object 3302 (e.g., gripping, picking up, and / or otherwise binding). In the embodiment shown, the gripping element 3354 includes a vacuum component that uses vacuum (or suction) force to engage the object 3302. Once engaged, the gripping assembly 3350 can lift and / or move the object 3302 at least partially. In some embodiments, the robot system can position the gripping assembly 3350 at the center of mass (CoM), midpoint, and / or lower half of the target object 3302. For example, the robot system can align the bottom of the suction cup with the bottom edge of the target object 3302, or within a threshold distance from the bottom edge when gripping the target object 3302.

[0181] Figure 33D shows the end effector 3300 after the gripping element 3354 and the extendable component 3352 in the gripping assembly 3350 have been actuated to move the object 3302 engaged by the gripping element 3354 and at least partially above the upper surface 3331 of the multiple frame conveyors 3330. In the embodiment shown, the extendable component 3352 (sometimes referred to herein as the “vertical actuating component”) includes a gripping mechanism coupled between the gripping element 3354 and the common drive component 3342. In various embodiments, the extendable component 3352 may include a shape memory device, a piston, a telescopic component, a gripping mechanism, a coupling mechanism, and / or any other suitable extension component that is movable between an extended configuration (e.g., as shown in Figure 33D) and a folded configuration (e.g., as shown in Figure 33C).

[0182] In some embodiments, the gripping assembly 3350 may include a hinge that allows the gripping element 3354 to rotate, thereby lifting the front / gripping surface upward and allowing the gripped object to tilt, such as when the top of the front / gripping surface rotates away from the end effector 3300. Thus, the contact surface between the gripped object and the lower support object may decrease, for example, away from the gripping surface, to the bottom / edge of the gripped object.

[0183] When the object 3302 is lifted, at least partially, onto the upper surfaces 3331 of the multiple frame conveyors 3330, the end effector 3300 can actuate the gripper component 3340 in the vicinity, as shown in Figure 33E. As a result, the gripper component 3340 moves the object 3302 onto one or more upper surfaces 3331 of the multiple frame conveyors 3330 (which may be referred to herein as the “second position,” “object transfer position,” “release position,” etc.). Once the object 3302 is movably supported by one or more upper surfaces 3331 of the multiple frame conveyors 3330, the end effector 3300 can operate the gripping element 3354 to release the object 3302, and then actuate the gripper component 3340 to clear a path for the multiple frame conveyors 3330 to move the object 3302 in the vicinity. In some embodiments, detachment from the object may include providing a burst of fluid (e.g., air, argon gas, and / or another suitable fluid) to the gripping element 3354 to counteract the vacuum and / or suction forces within it, thereby releasing the object 3302. Once detached, activating the gripper component 3340 may include moving the common drive component 3342 proximately while folding the extendable component 3352. The drive component 3342 can move proximate more quickly than the multiple frame conveyors 3330 move the object 3302. As a result, the gripper component 3340 can move proximate more quickly than the object 3302, while positioning all components of the gripper component 3340 below the upper surface 3331 of the multiple frame conveyors 3330, thereby creating some separation between the gripper component 3340 and the object 3302.

[0184] Figure 33F shows the end effector 3300 after the gripper component 3340 has been fully positioned beneath the upper surface 3331 of the multiple frame conveyors 3330 to clear a path for the object 3302 (which may be referred to herein as the “third position,” “lowered position,” “standby position,” etc.). As shown in Figure 33F, the multiple frame conveyors 3330 can then move the object 3302 proximately onto the joint conveyor 3320. The joint conveyor can then continue to move the object 3302 proximately (towards a movable base component that carries the end effector 3300, such as the chassis 302 in Figure 3).

[0185] In the end effector embodiments shown in Figures 33A to 33F, the frame has a generally consistent thickness between the proximal and distal end regions. Consistent thickness may help improve the stability of the frame (and / or its end effector). However, as shown in Figures 33A to 33F, consistent thickness may require that the gripper component 3340 be able to fully lift any object targeted by the end effector to place it on the top surface of the frame conveyor, which may limit the number of objects that can be unloaded, for example, from a shipping unit (e.g., from a truck, shipping container, etc.) with the type of end effector shown in Figures 33A to 33F. However, in various other embodiments, the frame may have a different shape, which may help expand the usability of the end effector.

[0186] Figure 34 is a partial schematic top side view of an end effector 3400 configured according to several embodiments of the present technology. As shown in Figure 34, the end effector 3400 is generally similar to the end effector 3200 described above with reference to Figures 32A and 32B. For example, the end effector 3400 (sometimes referred to herein as an end-of-arm tool) includes a frame 3410, a plurality of joint conveyors 3420, a plurality of frame conveyors 3430, and a gripper component 3440. Furthermore, the frame 3410 extends from a proximal end portion 3412 to a distal end portion 3414, the plurality of joint conveyors 3420 are supported by the proximal end portion 3412, and the plurality of frame conveyors 3430 extend from the distal end portion 3414 to the proximal end portion 3412. Furthermore, the gripper component 3440 includes a drive component 3442 and one or more gripping assemblies 3450 (eight are shown in Figure 34) coupled to the drive component 3442. Similar to the components described above, the drive component 3442 can move the gripping assembly 3450 along the longitudinal axis of the end effector 3400. In addition, the gripping assembly 3450 can be actuated to move the gripping element 3454 within the gripping assembly 3450 upward.

[0187] However, in the embodiment shown, the frame 3410 has a wedge-shaped structure having a smaller vertical thickness at the distal end portion 3414 than at the proximal end portion 3412. As shown with reference to and described in more detail with reference to Figures 36A to 36E, the wedge-shaped structure allows the gripping assembly 3450 to place and / or otherwise position an object on at least a portion of the upper surface 3431 of the multiple frame conveyors 3430 without requiring the object to be lifted completely. As a result, the end effector 3400 can be used to unpack a variety of objects from a shipping unit, including objects that cannot be lifted completely by the gripper components 3440 (for example, due to their weight).

[0188] As further shown in Figure 34, the end effector 3400 may include one or more guide components 3470 (two are shown in Figure 34) coupled to the frame 3410. The guide components 3470 may be positioned to help orient objects on the upper surface 3431 of the multiple frame conveyors 3430 towards the center of the upper surface 3431 as the multiple frame conveyors 3430 move objects in the proximal direction. In other words, the guide components 3470 may function as side rails to help prevent objects placed on the multiple frame conveyors 3430 from falling off the sides of the end effector 3400 as the end effector 3400 moves proximal.

[0189] Figure 35 is a partial schematic side view of a gripper component 3500 of the type shown in Figure 34 according to some embodiments of the present technology. That is, the gripper component 3500 shown in Figure 35 may be generally similar to (or identical to) one of the gripper components 3440 in Figure 34. In the embodiments shown, the gripper component 3500 (which may also be referred to herein as the “gripping component”) includes a drive component 3510 and a gripping assembly 3520 operably coupled to the drive component 3510. Although only a single gripping assembly 3520 is shown in Figure 35, it is understood that in some embodiments the drive component 3510 may be operably coupled to multiple similar gripping assemblies to control the position of the gripping assemblies along a vertical row (or generally a vertical row) of end effectors. In the embodiments shown, the gripping assembly 3520 includes a pivotable link 3530, a connecting housing 3540, and a gripping element 3550.

[0190] In the embodiments shown, the pivotable link 3530 (sometimes referred to herein as the “coupling mechanism”) includes a proximal end 3532 pivotally coupled to the drive component 3510 and a distal end 3534 pivotally coupled to the connecting housing 3540. As a result, the pivotable link 3530 can actuate the gripping assembly 3520 between a first position 3522 (shown by a solid line) and a second position 3524 (shown by a dashed line).

[0191] As will be described and shown in more detail below, the transition between the first and second positions allows the gripping assembly 3520 to engage with the object to be lifted, at least partially, over the top surface of the end effector (e.g., over the top surface 3331 of the multiple frame conveyors 3330 in Figure 33D, over the top surface 3431 of the frame conveyor 3430 in Figure 34, etc.). For example, in the first position 3522, the gripping assembly 3520 may protrude beyond the distal end of the end effector frame to engage with the object to be lifted. In the first position 3522, the gripping assembly can extend along a direction parallel to the bottom of the frame 3410 (e.g., the bottom of the wedge). For example, the bottom / bottom surface of the gripping assembly may be coplanar with the bottom surface of the frame 3410. The first position 3522 may further have a horizontally oriented bottom / bottom surface of the frame 3410.

[0192] Once the suction cup is engaged and an object is gripped, the gripping assembly 3520 can transition to a second position 3524 while lifting the object at least partially (e.g., fully lifting, lifting one side of the object). For example, the transition may cause the front / gripping surface of the object to rise with its top rotating away from the frame. The portion of the bottom surface that is on the gripped object and away from the gripping surface may remain in contact with the bottom / support surface. Thus, the transition can reduce the contact area of ​​the contacting surfaces of the gripped object and the support object by tilting / rotating the object, thereby reducing the possibility of contact between surface / contour features (e.g., surface irregularities that form vertical protrusions or recesses). Furthermore, since tilting the object involves a partial lift of the gripped object (e.g., the front), the weight of the gripped object may be reduced so that it is experienced / supported by the object below. The reduction in weight can provide a decrease in frictional force between the gripped object and the support object, thus reducing the likelihood of disturbing and moving the bottom support object during the transport of the gripped object.

[0193] The drive component 3510 can then move proximally to lift the object onto the slanted upper surface. In other words, the pivotable link 3530 has a transport configuration and a standby configuration (e.g., a first position 3522). In the transport configuration, the pivotable link 3530 positions the gripping element 3550 so that it can hold an object separated from one or more conveyors while the connecting assembly rotates relative to the frame of the end effector. The rotation allows the gripping element 3550 to move the object above the multiple conveyors (e.g., to a second position 3524 above the upper surface 3331 of the multiple frame conveyors 3330 in Figure 33D). In the standby configuration, the pivotable link 3530 positions the gripper element within the end effector (e.g., directly below the upper surface 3331 of the multiple frame conveyors 3330 in Figure 33D).

[0194] As the drive component 3510 moves and pulls the gripped object toward the frame 3410, the bottom surface of the gripped object can come into contact with the front / distal portion of the frame 3410 (e.g., the front / corner of the wedge). Thus, the frame 3410 and the conveyor provide lifting support, thereby reducing the load on the gripping assembly 3520. Furthermore, by rotating the gripped object, the rear corner of the object is supported by its bottom surface. Thus, the load experienced by the gripping assembly 3520 and / or on the gripping assembly 3520 can be reduced to less than the weight of the gripped object due to the support from the support object and / or the distal portion of the frame 3410. Furthermore, the configuration and operation described can shorten or eliminate the period during which the gripping assembly 3520 supports the full weight of the device being gripped. As a result, the configuration and operation of the gripping assembly 3520 can increase the maximum weight of the object being gripped and transported.

[0195] In addition to the additional support, the distal end of the frame 3410 can interact with the oblique / inclined direction of the conveyor (e.g., the upper surface of the wedge), allowing the gripped object to be lifted from the support surface. The combination of the shape and orientation of the frame 3410, as well as the direction of movement of the conveyor and gripping assembly, allows the gripped object to be lifted immediately or within a threshold period after the bottom surface of the gripped object contacts the distal portion / distal end of the frame 3410. Thus, in addition to reducing the contact area with the support surface and the corresponding friction with the support surface, various configurations and operations can reduce the distance the gripped object travels while it is in contact with the support surface. In other words, the above-described features of the gripper assembly can reduce the distance and duration during which the gripped object experiences frictional force with the support surface. As a result, the gripper assembly can reduce the shift in the object directly beneath the gripped / transported object that previously supported the gripped / transported object.

[0196] In some embodiments, movement between the first position 3522 and the second position 3524 is driven by a belt and pulley system operably coupled to a pivotable link 3530 and / or a connecting housing 3540. For example, returning to the description of Figure 34, the gripper component 3440 may include a plurality of belts 3446 coupled to a connecting housing 3448. When the plurality of belts 3446 are pulled backward (e.g., by a drive shaft and / or one or more pulleys), the belts pull on the corresponding connecting housing 3448, thereby acting the gripper component 3440 into an elevated position (e.g., rotating, pivoting, and / or otherwise moving), such as the second position shown in Figure 35. Returning to Figure 35, in some embodiments, movement between the first position 3522 and the second position 3524 is driven by a rotor and / or other electrically driven mechanism operably coupled to the pivotable link 3530. In some embodiments, the pivotable link 3530 is operably coupled to a common actuation mechanism among multiple gripping assemblies to generally simultaneously control movement between a first position 3522 and a second position 3524.

[0197] As further illustrated in Figure 35, the pivotable link may include an anchor 3536 positioned between the proximal end 3532 and the distal end 3534. The anchor 3536 may help manage various connections 3560 (e.g., wires, vacuum tubes, vacuum lines, fluid lines, etc.) extending between the drive component 3510 and the connecting housing 3540. That is, the anchor 3536 provides a fixed point for the connections 3560 as the gripping assembly 3520 transitions between a first position 3522 and a second position 3524. As a result, for example, the anchor 3536 may help reduce the possibility of the connections 3560 getting caught on the gripper component 3500, the end effector, and / or other parts of the surrounding environment. Next, control may help to increase the speed and precision of the gripper component 3500 (for example, the gripping assembly 3520 can transition more quickly between the first position 3522 and the second position 3524 if the likelihood of snagging is reduced).

[0198] The connecting housing 3540 can then route the connecting portion 3560 to a suitable final position. For example, in some embodiments, the gripping element 3550 (which may also be referred to herein as the “gripper element,” “engaging element,” etc.) includes a vacuum component. In such embodiments, the connecting housing 3540 can route a vacuum tube to the input for the vacuum component to provide vacuum pressure (and / or positive pressure) to engage (and disengage) the object to be engaged. In another example, the gripping element 3550 includes a magnetic component. In this example, the connecting housing 3540 can route an electrical connection to the magnetic component to generate (and stop generating) a magnetic force to engage (and disengage) the object to be engaged. In yet another example, the gripping element 3550 includes a mechanical gripper component (e.g., a clamp). In this example, the connecting housing 3540 can route an electrical connection to the clamp to actuate the mechanical gripper component to engage (and disengage) the object to be engaged.

[0199] Figures 36A to 36E are partial schematic side views of the end effector 3600 at various stages of the process for picking up an object according to some embodiments of the present technology. The end effector 3600 may be generally similar to (or identical to) the end effector 3400 described above with reference to Figure 34. For example, as shown in Figure 36A, the end effector 3600 (sometimes referred to herein as an end-of-arm tool) includes a frame 3610, a plurality of frame conveyors 3630, and a gripper component 3640. Furthermore, the frame 3610 extends from a proximal end portion 3612 to a distal end portion 3614, and the plurality of frame conveyors 3630 are arranged to move objects on them between the distal end portion 3614 and the proximal end portion 3612.

[0200] As further shown in Figure 36A, the gripper component 3640 may be generally similar to (or identical to) the gripper component 3500 described with reference to Figure 35. For example, the gripper component 3640 may include a drive component 3642 and one or more gripping assemblies 3650 (six shown in Figure 36A) operably coupled to the drive component 3642. Each gripping assembly 3650 includes a pivotable link 3652, a connecting housing 3654, and a gripping element 3656. Similar to the components described above, the drive component 3642 can be actuated to move the gripping assembly 3650 along the longitudinal axis of the end effector 3600. For example, as shown in Figure 36A, the gripper component 3640 (or another suitable controller) can move the drive component 3642 to position the gripping assembly 3650 at the distal end of the frame 3610 to a distal position (which may be referred to herein as the “first position,” “pickup position,” and “engagement position,” etc.). At this position, one or more of the gripping assemblies 3650 can be operated to engage with the target object 3602 (three are shown in the illustrated embodiment).

[0201] In the embodiments shown, engagement can be achieved by delivering a driving force to the gripping element 3656 via a connector 3660, which is individually coupled between the driving component 3642 and each of the gripping elements 3656. In various embodiments, the driving force may be a vacuum force (sometimes referred to herein as an attractive force delivered, for example, by a vacuum tube), an electrically driven force (e.g., supplied to a magnetic component, a mechanical gripper component, etc.), an aerodynamic force (e.g., delivered to a mechanical gripper component), and / or any other suitable force. The driving force enables each of the gripping elements 3656 to releasably engage (e.g., grip, pick up, and / or otherwise bind) the object 3602. The end effector 3600 may be in the first position described above with the gripping elements extending distally toward the object 3602. The frame of the end effector 3600 can be oriented so that its top surface (e.g., a plurality of frame conveyors 3630) is angled / tilted.

[0202] As shown in Figure 36B, after one or more of the gripping elements 3656 engage with the object 3602, the gripper component 3640 (or any other suitable controller) can actuate the pivotable link 3652 to raise the connecting housing 3654 and the gripping elements 3656, thereby lifting the object 3602 at least partially. In the shown embodiment, the gripper component 3640 thereby tilts the object 3602 over its trailing edge, with its leading edge raised above the upper surface 3631 of the multiple frame conveyors 3630. In other words, the end effector 3600 can transition from a first position to a second position. In some embodiments, the overall orientation of the end effector 3600 or its frame may remain constant in space, or it may move by a predetermined amount along the distal and / or vertical directions to deviate or complement the transition.

[0203] Tilting the object 3602 may offer several advantages to the end effector 3600. For example, tilting the object 3602 does not require the gripping assembly to lift the object 3602 completely. This may be relatively difficult for heavier objects and / or otherwise difficult for objects to engage with the gripping element 3656. As a result, for example, the end effector 3600 can be used to unload a wide variety of objects from a shipping unit. Furthermore, or alternatively, tilting the object 3602 can reduce the surface area of ​​the object in contact with the surface below, thereby reducing friction with the surface below. Reduced friction can then reduce the force required to pull the object 3602 proximal to the upper surface 3631 of the multiple frame conveyors 3630, and / or reduce the likelihood of damaging the object below by pulling the object 3602 (e.g., tipping over a stack of boxes below which are then targeted).

[0204] As shown in Figure 36C, after the leading edge of the object 3602 rises above the upper surface 3631 of the multiple frame conveyors 3630, the gripper component 3640 (or another suitable controller) can move the drive component 3642 to move the gripping assembly 3650 to a proximal position. As a result, the gripping assembly 3650 can lift the object 3602 above the upper surface 3631 of the multiple frame conveyors 3630 (which may be referred to herein as the “second position,” “object transfer position,” “release position,” etc.).

[0205] As shown in Figure 36D, as the drive component 3642 continues to move proximal, the gripper component 3640 (or another fluid moment controller) can actuate the gripping element 3656 to release the object. In some embodiments, release is achieved by disconnecting the drive force from the gripping element 3656. In some embodiments, release involves delivering a release force to the gripping element 3656. For example, in embodiments where a vacuum force is used to engage the object 3602, the vacuum pressure can continue to exist between the gripping element 3656 and the object 3602 even after the vacuum force has been disconnected. In such embodiments, the gripper component 3640 can release the gripping element 3656 by delivering positive pressure (e.g., a burst of air, argon gas, and / or another suitable fluid) to the gripping element via the connector 3660.

[0206] In some embodiments, the gripper component 3640 (or another suitable controller) causes the gripping element 3656 to release the object 3602 at a predetermined position between the distal end portion 3614 and the proximal end portion 3612 of the frame 3610. The predetermined distance can be configured so that multiple frame conveyors 3630 can move the object 3602 proximal without the assistance of the gripper component 3640. In some embodiments, the end effector 3600 may include one or more sensors (see Figures 32A and 32B) to detect when the gripping element 3656 and / or the object 3602 reach the predetermined position. In some embodiments, the position of the gripper component 3640 and / or the gripping element 3656 can be measured by monitoring a drive mechanism coupled to the drive component 3642 (for example, by measuring the rotation of a rotor coupled to the drive component 3642 to determine the position of the gripper component 3640).

[0207] When the gripping element 3656 releases the object 3602, the gripper component 3640 (or another suitable controller) can operate the drive component 3642 to move the gripping element 3656 of the gripper component 3640 closer than the multiple frame conveyors 3630 can move the object 3602. As a result, the drive component 3642 can cause some separation between the gripping element 3656 and the object 3602, allowing the gripping element 3656 to be positioned beneath the multiple frame conveyors 3630.

[0208] For example, as shown in Figure 36E, after the gripping element 3656 has been separated from the object 3602, the gripper component 3640 (or another suitable controller) can actuate the pivotable link 3652 to lower the connecting housing 3654 and the gripping element 3656 below the upper surface 3631 of the multiple frame conveyors 3630. As a result, the gripper component 3640 is positioned completely outside the proximal movement path for the object 3602 along the multiple frame conveyors 3630 (which may be referred to herein as the “third position,” “lowered position,” “standby position,” etc.). The multiple frame conveyors 3630 can then move the object 3602 proximal (for example, toward the movable base component) while (or before) the end effector 3600 is moved adjacent to the next object.

[0209] Figure 37 is a flowchart of a process for picking up an object according to some embodiments of the present technology. The process can be carried out to unload an object from a shipping unit (e.g., a shipping container, a truck, etc.) using the end effectors of the types described above, their components, and / or other components of a robotic system, with reference to Figures 3 to 31. Furthermore, the process can be carried out at least partially using the end effectors of the types described above, with reference to Figures 32A to 36E.

[0210] The process begins in block 3702 by identifying the object to be engaged. The identification process in block 3702 may be generally similar to (or identical to) one or more parts of the processes described above, with reference to Figure 18. For example, the identification process may include detecting one or more target objects using sensors mounted on the end effector and / or any other suitable sensors in the robotic system. Alternatively, the identification process may include selecting one or more target objects that have been previously detected and / or are otherwise known to the process (e.g., loaded from a map of target objects) using sensors.

[0211] In block 3704, the process includes positioning an end effector adjacent to the identified object. In various embodiments, positioning the end effector may include moving and / or acting the chassis, first segment, and / or distal joint of the robot system. When the end effector is positioned adjacent to the identified object (for example, as shown in Figure 33A), the identified object is distal to the farthest end of the end effector. When positioning the gripper, the robot system may have the frame conveyor 3630 tilted to pull and lift the gripped object during the initial part of the transfer.

[0212] In block 3706, the process includes acting distally the gripping assembly within the end effector to position one or more gripping elements within the gripping assembly to contact an identified object (as shown in Figure 33B). As described above, acting the gripping assembly may include acting the drive components of the gripping assembly using a belt and pulley system, a gear and track system, driving one or more carts along a track, or operating one or more expandable components (e.g., pistons, telescopic elements, etc.).

[0213] In block 3708, the process involves manipulating one or more elements to engage an identified object (for example, as shown in Figures 33C and 36A). In various embodiments, as described above, the gripping elements may include vacuum components (sometimes referred to herein as suction components), magnetic components, mechanical gripper components, etc., which are operated by delivering driving forces and / or driving signals (e.g., vacuum force, power, command signals, etc.) to the gripping elements through connections within the gripping assembly.

[0214] In block 3710, the process includes lifting the identified object at least partially (as shown in Figures 33D and 36C). Lifting can be achieved, for example, via the extendable component 3252 in Figure 32A, the pivotable link 3530 in Figure 35, and the like. Furthermore, as described above, the lifting process can fully pick up the identified object to reduce friction between the identified object and the object below, and / or to avoid (or reduce) interference with the object below while the identified object is being removed. In some embodiments, the process does not need to lift the identified object (for example, when pulling an object proximately from a shelf). In such embodiments, block 3710 can be omitted, and instead, one or more components in the gripping assembly (for example, the extendable component 3252 in Figure 32A, the pivotable link 3530 in Figure 35, and the like) can be actuated in block 3706 to raise the gripping element before engaging the identified object.

[0215] When contacting and grasping an object, the robot system can extend one or more gripping elements toward the object. Using the extended gripping elements, the robot system can contact and grasp the object by activating suction cups at the ends of the extended one or more gripping elements. Once the gripper engages with the object, the robot system can rotatably retract one or more pivotable links to lift one or more gripping elements and the grasped object. When rotatably retracting one or more pivotable links, the robot system can effectively tilt the grasped object so that the upper part of the gripping surface of the object rotates away from the EOAT and the vertical axis.

[0216] In block 3712, the process includes acting the gripping assembly proximal to position the gripping element above at least a first portion of the conveyor (e.g., a frame conveyor) within the end effector (as shown, for example, in Figures 33E and 36D). In some embodiments, the process can be carried out generally simultaneously in blocks 3710 and 3712 to lift the identified object at least partially while acting the gripping assembly proximal. As the gripping assembly moves proximal, it pulls the identified object onto the upper surface of the end effector, and one or more conveyors can then move the identified object proximal toward the movable base of the robot system. The robot system can effectively move the bottom surface of the gripped object to contact the distal end portion of the EOAT, which can support the gripped object while it is fully moved onto the EOAT. The robotic system can lift the object onto an EOAT (e.g., a conveyor above it) while maintaining the tilted orientation of the gripped object, in order to reduce surface friction between the gripped object and the support object below it that is in contact with the gripped object.

[0217] In block 3714, the process involves manipulating the gripping element to release the identified object. As described above, in various embodiments, releasing the identified object may include interrupting the driving force (e.g., stopping the delivery of vacuum force, stopping the delivery of power and / or other electric drive signals), and / or delivering various other control signals. In some embodiments, releasing the identified object may include delivering a release force (e.g., a burst of air, argon gas, and / or another suitable fluid to overcome the vacuum pressure between the gripping element and the identified object). Once released, the identified object is placed completely on the end effector conveyor. Furthermore, as described above, releasing the identified object may include moving the gripping assembly closer than the end effector conveyor can move the identified object. This movement may help create a separation between the gripping assembly and the identified object, for example, which can provide space for the gripping element to actuate in the lowered position.

[0218] In block 3716, the process includes acting on the gripping assembly to position the gripping element below at least the second portion of the conveyor (as shown in Figures 33F and 36E). Similar to the lifting described above, the actuation can be achieved via, for example, the extendable component 3252 in Figure 32A, the pivotable link 3530 in Figure 35, and so on. Furthermore, in some embodiments, the actuation includes causing some separation between the gripping assembly and the identified object by moving the gripping assembly proximally faster than the conveyor moves the identified object (for example, when separation did not occur in block 3714 and / or to expand the separation). Once below the second portion of the conveyor, the gripping assembly is positioned off the proximal movement path along the conveyor. Thereafter, the process may include operating the conveyor to move the identified object proximally toward the movable base of the robot system.

[0219] Figures 38A and 38B are partial schematic upper side views showing additional features in the end region 3802 of an end effector 3800 configured according to some embodiments of the present technology. As best shown in Figure 38A, the end effector 3800 may be generally similar to (or identical to) the types of end effectors described above with reference to any of Figures 32A to 36E. For example, in the embodiments shown, the end effector 3800 includes a frame 3810, a plurality of frame conveyors 3830, and a gripper component 3840 including a plurality of gripping assemblies 3850. Furthermore, the end effector 3800 may include one or more sensors 3880 positioned to detect when the gripper component 3840 and / or an object engaged by the gripper component 3840 passes a predetermined position on the frame 3810 during a gripping operation. As described above, a predetermined position can be selected so that the plurality of frame conveyors 3830 can carry and / or move the object in front of it beyond that position. In some embodiments, a predetermined position takes into account the distance the gripper component 3840 (and the object it engages) travels before the gripper component 3840 can release the object in response to a signal from the sensor 3880. In the embodiment shown in Figure 38A, the sensor 3880 is supported by the furthest portion 3815 of the frame 3810. As a result, the end effector 3800 may rely on a delay and / or propulsion force for the release of the object to ensure that the object is placed on multiple frame conveyors 3830.

[0220] Figure 38B is a close-up view of the furthest portion 3815 of frame 3810 (e.g., an enlarged view of area A circled in the diagram). As shown in Figure 38B, the sensor 3880 may include a proximity sensor that detects when an object of interest passes over the sensor 3880, thereby being positioned above at least a portion of the multiple frame conveyors 3830. However, proximity sensors (and other sensors that can be used) may be sensitive to dust, dirt, and / or other contaminants. To help reduce interference with the sensor 3880, the end effector 3800 may include one or more outlet nozzles 3882 directed across the sensor 3880. The outlet nozzles 3882 can periodically direct air (and / or any other suitable fluid) across the sensor 3880 to help keep the proximity sensor away from dust, dirt, and / or other contaminants. In some embodiments, the outlet nozzle 3882 may be fluid-coupled to a connection in the gripping assembly (e.g., the connection 3560 described above with reference to Figure 35). In some embodiments, the air (and / or any other suitable fluid) no-burst used to release the gripping element from the object can be partially directed to the outlet nozzle 3882. As a result, the outlet nozzle 3882 can direct a portion of the burst across the sensor 3880 after each cycle through the gripping operation.

[0221] Figures 39A and 39B are a partial schematic top view and a schematic upper side view, respectively, of an end effector 3900 configured according to several embodiments of the present technology. In the embodiments shown, the end effector 3900 may be generally similar to (or identical to) any of the end effectors described above with reference to Figures 32A to 36E, 38A, and 38B. For example, as shown in Figure 39A, the end effector 3900 may include a frame 3910, a plurality of frame conveyors 3930, and a gripper component 3940. Furthermore, similar to the end effector 3400 described above with reference to Figure 34, the end effector 3900 may include one or more guide components 3970 positioned on the side of the frame 3910.

[0222] As best shown in Figure 39A, the guide component 3970 includes an angled portion 3972 and a straight portion 3974. The angled portion 3972 is inclined inward toward the central longitudinal axis of the end effector 3900. As a result, as the multiple frame conveyors 3930 move the object 3902 proximally, the angled portion 3972 can push (or otherwise propel) the object 3902 placed on the side of the end effector 3900 toward the central longitudinal axis of the end effector 3900. The straight portion 3974 extends parallel to the longitudinal axis of the end effector 3900. As a result, the straight portion 3974 can function as a rail along the rest of the end effector 3900.

[0223] In some embodiments, as best shown in Figure 39B, the straight section 3974 can be movably coupled to a track 3976 (or another suitable component such as a piston or telescopic component). The track 3976 allows the guide component 3970 to move distally and proximal along the longitudinal direction of the end effector 3900. As a result, for example, the guide component 3970 can adjust its position to maximize the object guidance advantage of the guide component 3970 and / or to improve clearance around the end effector 3900. In certain non-limiting examples, the guide component 3970 may be in a retracted (proximal) position, while the end effector 3900 is positioned adjacent to one or more target objects to reduce the possibility of the guide component 3970 getting caught in the surrounding environment during motion. Once the end effector 3900 is in place, the guide component 3970 can be moved to an extended (distal) position to help push the object toward the central longitudinal axis of the end effector 3900 and / or prevent it from falling to the side.

[0224] Although not explicitly stated above with reference to Figures 31 to 39B, it will be understood that in some embodiments the end effector may include a controller operably coupled to any of the components described herein. The controller may be communicatively coupled to another controller (e.g., processor 202 in Figure 2 and / or any other suitable component) to help control the operation of any of the components of the end effector described above. Furthermore or alternatively, the controller may include a processor and memory that stores instructions, when executed by the processor, causing the controller to perform any of the operations of the end effector described above.

[0225] Exemplary distal joint for robotic systems Figure 40 is a partial schematic top side view of a distal joint 4010 for a robotic system 4000 configured according to several embodiments of the present technology. As shown in Figure 40, the robotic system 4000 includes a first segment (which may be referred to herein, for example, as the “movable arm”), a distal joint 4010 (which may be referred herein, as the “wrist joint”, “second joint”, “end effector joint”, etc.) operably coupled to the first segment 4002, and an end effector 4004 operably coupled to the distal joint 4010. It is understood that the first segment 4002 may be generally similar to (or identical to) any of the first segments described above with reference to Figures 3 to 12F. Similarly, the end effector 4004 may be generally similar to (or identical to) any of the end effectors described above with reference to Figures 32A to 39.

[0226] As shown in Figure 40, and similar to the above description with reference to Figures 15 and 16, the distal joint 4010 allows the end effector 4004 to rotate relative to the first segment 4002 along both the third axis A3 and the fourth axis A4. In other words, the distal joint 4010 provides the end effector 4004 with two degrees of freedom relative to the first segment 4002. These degrees of freedom then allow the end effector 4004 (and more broadly the robot system 4000) to be positioned in various suitable configurations. As a result, the robot system 4000 can unload various shipping units without external assistance (e.g., human assistance or robotic assistance).

[0227] In the illustrated embodiment, the distal joint 4010 includes a first drive system 4020 that rotatably couples the distal joint 4010 to the first segment 4002. As will be described in more detail below, the first drive system 4020 can include various components that can rotate the distal joint 4010 (and the end effector 4004 coupled thereto) about the fourth axis A4 relative to the first segment 4002. For example, in the embodiment shown in FIG. 40, the first drive system 4020 (which may also be referred to herein as the "first drive mechanism") includes a pivotable link 4022 that helps support the weight of the distal joint 4010 and / or the end effector 4004 at various angles relative to the first segment 4002. In some embodiments, as will be described in more detail below, the first drive system 4020 can be operably coupled to the pivotable link 4022 to assist in driving the rotation of the distal joint 4010 about the fourth axis A4. In the illustrated embodiment, the robotic system 4000 also includes a second drive system 4030 (shown schematically) that rotatably couples the distal joint 4010 to the end effector 4004. As will be described in more detail below, the second drive system 4030 can include a mechanism for rotating the end effector 4004 about the third axis A3 relative to the distal joint 4010. In the particular non-limiting example described, the second drive system 4030 can include a rotary motion joint (which may also be referred to herein as a rotary union) having a central passage for a connection.

[0228] As further illustrated in Figure 40, the distal joint 4010 may include a plurality of joint conveyors 4012 (e.g., rollers) arranged to receive the object 4006 from the end effector 4004 and move the object 4006 in the proximal direction (e.g., toward and over the first segment 4002). The distal joint 4010 may also include one or more fixed support plates 4014 (one shown in Figure 40) that help support the object 4006 along the motion path, allow a drive mechanism (e.g., a belt, servo motor, gears, etc.) to be coupled to the joint conveyors 4012, and / or help adapt the distal joint 4010 to one or more conveyors on the end effector 4004 (e.g., the plurality of joint conveyors 3220 in Figure 32A, the plurality of joint conveyors 3420 in Figure 34, etc.). Furthermore, the distal joint 4010 may include one or more retractable elements 4044 (one is shown in Figure 40) operably coupled to the retractable system 4042. The retractable elements 4044 may include additional conveyors, passive rollers, support plates (and / or other low-friction elements), etc. As will be described in more detail below with reference to Figures 42A to 43C, the retractable system 4042 may raise (and lower) the retractable elements 4044 to fill the gap (and open space) between the distal joint 4010 and the end effector 4004 as the end effector 4004 rotates around the third axis A3. For example, in various embodiments, the retractable system 4042 may include various telescopic components, pneumatic actuators, pistons, shape memory devices, scissor components, etc. In a specific non-limiting example shown in Figure 40, the contraction system 4042 includes a stepped track that rotates with the end effector 4004 to automatically raise (and lower) the contractible element 4044 as the end effector 4004 rotates.

[0229] FIG. 41 is a partial schematic bottom side view of a distal joint 4100 for a robotic system configured according to some embodiments of the present technology. The distal joint 4100 may generally be similar (or identical) to the distal joint 4010 described above with reference to FIG. 40. For example, the distal joint 4100 may be operably coupled between the first segment 4002 and the end effector 4004. However, FIG. 41 shows additional details regarding a first drive mechanism 4110 within the distal joint 4100 for controlling rotation of the distal joint 4100 with respect to the first segment 4002 (e.g., along the fourth axis A4 shown in FIG. 40). In the embodiment shown, the first drive mechanism 4110 includes a connection pulley 4112, connection belts 4114 and a drive shaft 4116 each operably coupled to the connection pulley 4112, and a reducer system 4120 operably coupled to the drive shaft 4116. As the first segment 4002 rotates with respect to the proximal joint (e.g., rotates about the second axis A2 of FIG. 3), the connection belt 4114 extends from the connection pulley 4112 to a pulley at the proximal joint (e.g., to the actuator 336 described above with reference to FIG. 3) so that the connection belt 4114 converts motion to the connection pulley 4112. Next, the connection pulley 4112 can convert motion to the drive shaft 4116, and the drive shaft 4116 converts motion through the reducer system 4120.

[0230] The reducer system 4120 may include a pulley reducer and / or other breaking mechanisms (e.g., a resistance breaking mechanism) and / or an acceleration mechanism (e.g., a gear increase). As a result, the reducer system 4120 can help smooth the motion so that the rotation of the proximal joint (e.g., around the second axis A2 in Figure 3) matches the rotation of the distal joint (e.g., around the fourth axis A4 in Figures 3 and 40), and / or convert the motion from the connecting belt 4114 to the rotation of the distal joint 4100. The general agreement of motion then helps maintain the end effector 4004 in a generally horizontal configuration, so that the object to be engaged therewith can be moved by one or more conveyors within the end effector 4004 (for example, to maintain the generally flat top surface 3331 of the multiple frame conveyors 3330 in Figure 33D and / or to generally maintain a predetermined inclination of the top surface 3431 of the multiple frame conveyors 3430 in Figure 34).

[0231] In some embodiments, the reducer system 4120 includes one or more servo motors to help smooth the motion from the connecting belt 4114 and / or to help convert the motion to various other components within the first drive mechanism 4110. In certain non-limiting examples described in more detail below, the reducer system 4120 can convert the motion from the connecting belt 4114 to the pivotable links of the type described above with reference to Figure 40.

[0232] In the embodiment shown in Figure 41, the distal joint 4100 also includes a floating joint 4130 operably coupled between the first drive mechanism 4110 and the first segment 4002. The floating joint 4130 includes a compression component 4132, a proximal reference 4134 coupled between the compression component 4132 and the first segment 4002, and a distal reference 4136 coupled between the compression component 4132 and the first drive mechanism 4110. The compression component 4132 can be compressed and / or expanded in response to the rotation of the distal joint 4100 relative to the first segment 4002 (for example, along the fourth axis A4 in Figure 40). As a result, the floating joint 4130 can help maintain a predetermined distance between the distal joint 4100 and the first segment 4002. As a result, the floating joint 4130 can help avoid interference between the conveyor in the distal joint 4100 and the conveyor in the first segment 4002, and / or help avoid the formation of too large a gap between the distal joint 4100 and the first segment 4002.

[0233] Figures 42A and 42B are partial schematic side views of a distal joint 4210 for a robotic system 4200 configured according to a further embodiment of the present art. More specifically, Figures 42A and 42B show additional details relating to a first drive system 4220 within the distal joint 4210 according to several embodiments of the present art. In the embodiments shown, the distal joint 4210 is generally similar (or identical) to the distal joints 4010, 4100 described above with reference to Figures 40 and 41. For example, the distal joint 4210 can be operably coupled between a first segment 4202 and an end effector 4204.

[0234] Furthermore, the first drive system 4220 is coupled between the distal joint 4210 and the first segment 4202. As shown in Figures 42A and 42B, the first drive system 4220 may include a reducer system 4222 supported by the distal joint 4210, and a pivotable link 4224 and an expandable component 4226 coupled between the distal joint 4210 and the first segment 4202, respectively. As described above, the reducer system 4222 can help convert rotation at the proximal joint of the robot system into an opposite rotation at the distal joint 4210. More specifically, the reducer system 4222 drives rotation at the pivotable link 4224, thereby causing the distal joint 4210 to rotate around the fourth axis A4 relative to the first segment 4202. For example, Figure 42A shows the robot system 4200 in a lowering configuration, while Figure 42B shows the robot system 4200 in an upward configuration. To move between the lowering and upward configurations, the reducer system 4222 can drive a pivotable link 4224 clockwise around a fourth axis A4, thereby also rotating the distal joint 4210 relative to the first segment 4202. As further illustrated in Figure 42A, the fourth axis A4 may be generally orthogonal to the longitudinal plane of the end effector 4204 (e.g., the third plane P3). Alternatively, the fourth axis A4 may be generally orthogonal to the cross-section of the end effector 4204 (e.g., the fourth plane P4 shown in Figure 42A).

[0235] In some embodiments, the expandable component 4226 can help drive the rotation of the pivotable link 4224 and / or the distal joint 4210. For example, the expandable component 4226 can expand and / or contract in response to a signal from the controller, thereby coupling to the controller to cause the distal joint 4210 (and pivotable link 4224) to rotate around the fourth axis A4. Furthermore or alternatively, the expandable component 4226 can help stabilize the rotation of the distal joint 4210 and / or help support the distal joint 4210 and / or the end effector 4204 during operation. For example, since the expandable component 4226 is coupled between the distal joint 4210 and the first segment 4202, the expandable component 4226 provides an additional anchor between them. Additional support may be useful, for example, to help reduce noise on the end effector 4204 while objects of varying weights are engaged and loaded onto the end effector 4204. One consequence is that, for example, fewer objects may be dropped by the end effector 4204 and / or distal joint 4210 as a result of noise during operation and / or movement between components.

[0236] Figures 43A to 43C are partial schematic top views of distal joints 4310 for a robot system 4300 configured according to several embodiments of the present technology. As shown in Figure 43A, distal joints 4310 may be generally similar to (or identical to) distal joints 4310 described above with reference to Figures 40 to 42B. For example, distal joints 4310 can be operably coupled between a first segment 4302 and an end effector 4304 of the robot system 4300. Furthermore, distal joints 4310 include a second drive system 4330 that helps control the rotation of the end effector 4304 around a third axis A3 relative to the distal joint 4310. As illustrated in Figure 43A, the third axis A3 may be generally orthogonal to a cross-section of the end effector 4204 (e.g., a fourth plane P4).

[0237] As shown in Figure 43A, the distal joint 4310 may include features that help fill the gap between the distal joint 4310 and the end effector 4304 as the end effector 4304 rotates. For example, Figure 43A shows a robotic system 4300 having a distal joint 4310 and an end effector 4304 in an aligned (e.g., non-rotating) configuration. In this configuration, there is no significant gap between one or more first conveyors 4312 (e.g., rollers, etc.) in the distal joint 4310 and one or more second conveyors 4305 (e.g., conveyor belts, one or more rollers, etc., frame conveyors and / or joint conveyors as described above with reference to Figures 32A and 34). As a result, the second conveyors 4305 can transfer objects to the first conveyors 4312 without additional support. Therefore, the first retractable system 4313 and the second retractable system 4316 may be in a retracted position and / or a lowered position (which may also be referred to herein as the "lowered position," "standby position," "retracted position," etc.) below one or more first conveyors 4312.

[0238] As shown in Figure 43B, as the end effector 4304 rotates counterclockwise along the third axis A3 relative to the distal joint 4310, the first conveyor 4312 moves away from the second conveyor 4305, thereby forming a gap that may be too large for the object to be traversed without additional support. Thus, as the end effector 4304 rotates counterclockwise, the first retractable system 4313 can transition (e.g., rise) to an extended position and / or an elevated position (which may also be referred to herein as the “elevated position”, “carrying position”, “active position”, etc.) to provide additional support. In the embodiments shown, the first retractable system 4313 includes a first retractable conveyor 4314 and a first retractable support surface 4315. The first retractable conveyor 4314 may be rollers (passive or driven) and / or any other suitable conveyor. The first retractable support surface 4315 may be any surface that allows the object to continue moving (e.g., sliding) in the proximal direction, such as a clamping friction plastic and / or metal surface.

[0239] Similarly, as shown in Figure 43C, as the end effector 4304 rotates clockwise along the third axis A3 relative to the distal joint 4310, the first conveyor 4312 moves away from the second conveyor 4305, thereby forming a gap on the opposite transverse side of the distal joint 4310. Thus, as the end effector 4304 rotates clockwise, the second retractable system 4316 can transition (e.g., up) to an extended position and / or an elevated position to provide additional support. Similar to the first retractable system 4313, the second retractable system 4316 may include a second retractable conveyor 4317 and a second retractable support surface 4318. The second retractable conveyor 4317 may be rollers (passive or driven) and / or any other suitable conveyor. The second retractable support surface 4318 may be any surface that allows the object to continue moving (e.g., sliding) in the proximal direction, such as a clamping friction plastic and / or metal surface.

[0240] As further shown in Figures 43A to 43C, when the end effector 4304 rotates around the third axis A3 (relative to the distal joint 4310), the angles of the first and second conveyors 4312 and 4305 relative to each other change. For example, in Figure 43A, the first and second conveyors 4312 and 4305 are positioned to transport (e.g., move) objects in the same direction. However, in Figures 43B and 43C, the first conveyor 4312 is positioned to transport objects in a first direction, while the second conveyor 4305 is positioned to transport objects in a second direction that forms an angle with respect to the first direction. In other words, the conveyors of the distal joint 4310 are configured to modify the direction of transport to account for the rotation of the end effector 4304 around the third axis A3.

[0241] Figure 43D is a partial schematic bottom view of the distal joint 4310 of Figures 43A-43C according to some embodiments of the present technology. More specifically, Figure 43D provides additional details relating to a second drive system 4330 within the distal joint 4310. For example, in the embodiments shown, the second drive system 4330 includes a rotary joint 4332 (sometimes referred to herein as a rotary union) comprising a shaft 4334, one or more bearings 4336 (Schematically shown in Figure 43D), a housing 4338, and a retaining component 4340. The shaft 4334 is coupled to the frame 4311 of the distal joint 4310, while the housing 4338 is coupled to the end effector 4304. The bearing 4336 is coupled between the shaft 4334 and the housing 4338, thereby allowing the housing 4338 (and the end effector 4304) to rotate relative to the frame 4311 (and the distal joint 4310). The retaining component 4340 is coupled to the distal end of the frame 4311 to help hold the second drive system 4330 together. In the embodiment shown, the rotational joint 4332 also includes a central opening 4342. As will be described in more detail below with reference to Figures 45 and 46, the central opening 4342 can allow one or more connectors to pass from the distal joint 4310 to the end effector 4304 without risk of being pinched, caught, and / or otherwise struck during rotation.

[0242] In some embodiments, the bearing 4336 is an electronic bearing capable of controlling the rotation of the housing 4338 (and end effector 4304) relative to the frame 4311 (and distal joint 4310). In some embodiments, the bearing 4336 is passive, and a second drive system 4330 includes one or more expandable components (e.g., pistons, telescopic components, etc.) coupled to the transverse sides of the end effector 4304 and distal joint 4310 to control the rotation around the bearing 4336. Furthermore or alternatively, the housing 4338 may be coupled to a belt (or other suitable component such as a gear track) supported by the distal joint 4310 to drive rotation around the bearing 4336. Furthermore or alternatively, the housing 4338 may include a cart and / or other drive mechanism to drive rotation relative to the shaft 4334.

[0243] As further shown in Figure 43D and as introduced above, the end effector 4304 may include a drive mechanism 4306 operably coupled to each of the second conveyors 4305 (Figure 43A). For example, in the embodiment shown, the drive mechanism 4306 includes a servo motor 4307 coupled to each of the second conveyors 4305 (Figure 43A) via a series of common shafts and belts. Since the drive mechanism 4306 drives each of the second conveyors 4305 (Figure 43A) simultaneously, the robot system 4300 can produce generally uniform motion in the second conveyors 4305 (Figure 43A) without synchronizing multiple drive components (e.g., multiple servo motors). As a result of generally uniform motion, as described above, the end effector 4304 can transport objects without rotating them and / or driving them toward the transverse sides of the end effector 4304.

[0244] Figures 44A to 44C are partial schematic side views of a distal joint 4410 of the type shown in Figures 43A to 43D, configured according to some embodiments of the present technology. For example, as shown in Figure 44A, the distal joint 4410 is operably coupled between a first segment 4402 and an end effector 4404 of a robot system 4400. Furthermore, the distal joint 4410 includes a plurality of first conveyors 4412 (e.g., rollers) and a retractable system 4414. As described above, the retractable system 4414 is movable between an elevated position (e.g., as shown in Figure 44A) and a lowered position (e.g., as shown in Figure 44C). In the elevated position, the retractable system 4414 can help fill the gap between the distal joint 4410 and the end effector 4404 to help support an object moving in the proximal direction. In the lowered position, the retractable system 414 is positioned below the first conveyor 4412, allowing the first conveyor 4412 to be positioned adjacent to the end effector 4404.

[0245] In the embodiments shown in Figures 44A to 44C, the retractable system 4414 can automatically move between an elevated position and a lowered position as the end effector 4404 rotates around a third axis A3. For example, as shown in Figures 44A to 44C, the retractable system 4414 may include a first retractable component 4416 supported by a first arm 4418 and a first guide component 4420 supported by the end effector 4404. The first guide component 4420 includes a first track 4422 having inclined steps. The first arm 4418 is slidably coupled to the first track 4422. The first guide component 4420 is coupled to the end effector 4404 such that the first guide component 4420 rotates when the end effector 4404 rotates. In contrast, the first arm 4418 is coupled to the distal joint 4410 so that the first arm 4418 does not rotate. Instead, the first arm 4418 slides along the first track 4422. As a result, the first arm 4418 can slide downward (or upward) on the steps within the first track 4422 as the end effector 4404 rotates, thereby moving the first retractable component 4416 from an elevated position (Figure 44A) to a lowered position (Figure 44C) and / or vice versa.

[0246] As further shown in Figures 44A to 44C, the retractable system 4414 may also include a second retractable component 4424 supported by a second arm 4426 and a second guide component 4428 supported by an end effector 4404. As described above, the second arm 4426 is coupled to the distal joint 4410, while the second guide component 4428 is supported by the end effector 4404. Furthermore, the second guide component 4428 includes a second track 4430 having inclined steps, and the second arm 4426 is slidably coupled to the second track 4430. As a result, similar to the above description, the second arm 4426 can slide downward (or upward) on the step in the second track 4430 as the end effector 4404 rotates, thereby moving the second retractable component 4424 from the raised position (Figure 44A) to the lowered position (Figure 44C) or vice versa.

[0247] As best shown in Figure 44B, the retractable system 4414 can raise and / or lower the first and second retractable components 4416, 4424 at separate times. For example, in the embodiment shown, the second guide component 4428 is rotated around the third axis A3 relative to the first guide component 4420 so that a step in the second track 4430 is displaced around the third axis A3 from the first track 4422. As a result, when the end effector 4404 rotates, the second arm 4426 reaches a step in the second track 4430 so that the first arm 4418 reaches a step in the first track 4422. Thus, as shown in Figure 44B, the second retractable component 4424 is lowered in front of the first retractable component 4416.

[0248] In the embodiments shown in Figures 44A to 44C, the first retractable component 4416 includes rollers (e.g., an active conveyor and / or passive rollers), and the second retractable component 4424 includes a low-friction support surface. However, in various other embodiments, the retractable system 4414 may include various other elements. Simply as an example, both the first and second retractable components 4416 and 4424 may include rollers. In another example, both the first and second retractable components 4416 and 4424 may include a low-friction support surface. In yet another example, either the first or second retractable component 4416 or 4424 may include any other suitable component (e.g., another conveyor). Furthermore, in various other embodiments, the retractable system 4414 may include any other suitable number of retractable components (e.g., one, three, four, five, and / or any other suitable number of retractable components) to help fill the gap between the end effector 4404 and the distal joint 4410 as the end effector 4404 rotates.

[0249] Furthermore, it is understood that the retractable system 4414 may include other suitable systems for raising and / or lowering the retractable component. Simply as an example, the retractable system 4414 may include one or more driveable pistons, telescopic elements, scissor elements, etc., that are actuated to raise and / or lower the retractable component. In some such embodiments, the retractable system 4414 is controllable independently of the end effector 4404 and requires acting the retractable system 4414 in addition to rotating the end effector 4404 to help fill the gap.

[0250] Figures 45 and 46 are, respectively, a partial schematic top view and a partial schematic cross-sectional view of a distal joint 4500 of the type shown in FIGS. 40-43C, according to some embodiments of the present technology. As best shown in FIG. 45, the distal joint 4500 includes a drive system 4510 that can control the rotation of an end effector about a third axis A3. The drive system 4510 may generally be similar to the second drive system 4330 described above with reference to FIG. 43D. For example, in the illustrated embodiment, the drive system 4510 includes a rotary motion joint 4512 that allows one or more connectors 4520 to pass through the distal joint 4500 to the end effector without having to rotate and / or with minimal risk of snagging as the end effector rotates.

[0251] For example, as best shown in FIG. 46, the rotary motion joint 4512 having an opening 4516 extending from an upper end 4415a to a lower end 4415b of a shaft 4514 includes the shaft 4514. The opening 4516 allows the connector 4520 to be routed through a central portion of the drive system 4510. Since the end effector rotates about the distal joint 4500 via the drive system 4510, the connector 4520 is routed through the center of the rotary motion. As a result, the connector does not require slack to accommodate rotary motion that might otherwise snag during movement of the end effector and / or without a more complex system for routing the connector 4520 through the distal joint 4500.

[0252] Example of additional drive components within the end effector Figure 47 is a partial schematic isometric view of a drive component 4700 configured according to several embodiments of the present technology. The drive component 4700 shown in Figure 47 can be integrated with any of the gripping components in the end effector described above with reference to Figures 31 to 39B to help control the position of one or more gripping elements. In the embodiments shown, the drive component 4700 includes a frame 4702, an input / output ("I / O") board 4710 coupled to the frame 4702, and one or more gripping generation units 4720 (eight are shown in Figure 47) coupled to the I / O board 4710. The I / O board 4710 includes a plurality of input nodes 4712 (one of which is labeled in Figure 47), a plurality of output nodes 4714 (one of which is labeled in Figure 47), and a redistribution network 4716 inside the I / O board 4710. The I / O board 4710 (which may also be referred to herein as the “branching component” or “branching board”) can send inputs (e.g., electrical signals, pneumatic pressure, vacuum pressure, etc.) from another component within the robot system (e.g., the robot system 300 in Figure 3) to the gripping generation unit 4720. The gripping generation unit 4720 can then use the inputs to provide driving forces (e.g., vacuum force, magnetic force, actuation force, etc.) to each of the gripping elements within the gripping component.

[0253] In the embodiment shown in Figure 47, for example, the redistribution network 4716 is an electronic redistribution network that can send input signals from the input node 4712 to one or more grip generation units 4720 through the output node 4714. The electronic equipment 4724 in the grip generation unit 4720 that receives the input signals can then generate a driving force and provide that driving force to the individual and / or corresponding grip elements within the gripping component. In this example, the driving force (e.g., vacuum pressure, magnetic force, tea force, etc.) is generated locally within the driving component 4700 and therefore entirely within the end effector. As a result, for example, the connections reaching the input node 4712 may be only electrical connections rather than, for example, vacuum tubes. Since electrical connections are not sensitive to bending, twisting, sagging, entanglement, etc., the connections may be relatively easy to manage.

[0254] Furthermore, or alternatively, the magnitude of the driving force to be transmitted over any communication line can be reduced by local generation of the driving force within the electronic equipment 4724 (for example, at the scale of individual gripping elements). For example, if a vacuum force is generated near the end effector, the connection to the I / O board 4710 must transmit a vacuum force that is large enough to be divided among each of the gripping elements engaging the object. Moreover, that force must be transmitted through a distal joint with multiple rotational degrees of freedom. In contrast, local generation within the electronic equipment 4724 allows the vacuum force to be a portion of the magnitude, thus avoiding long path lines.

[0255] As further shown in Figure 47, each electronic component 4724 of the gripping generation unit 4720 can be at least partially housed within a housing 4722. The housing 4722 can help limit the amount of dust and other contaminants that reach the electronic components 4724. Furthermore, or alternatively, the housing 4722 can help protect the electronic components 4724 from impacts (e.g., from the target object, the environment around the operating end effector, other objects, etc.).

[0256] Figure 47 also provides additional details on how the drive component 4700 helps actuate the gripping assemblies within the gripping component (see, for example, Figures 32A and 34). For example, in the embodiment shown, the drive component 4700 includes a plurality of belts 4730 operably coupled to a single shared drive shaft 4732. Each of the belts 4730 can be coupled to the appropriate mechanism within the gripping assembly to control its movement between an elevated and a lowered position (for example, to rotate the pivotable link 3530 in Figure 35). Because each of the belts 4730 is coupled to the drive shaft 4732, the drive component 4700 can control the movement of each gripping assembly simultaneously, thereby keeping the gripping assemblies synchronized when they lift an object. Furthermore, each of the gripping assemblies can be coupled to the frame 4702 of the drive component 4700 to simultaneously control the respective longitudinal positions of the gripping assemblies.

[0257] Figure 48 is a partial schematic isometric view of a branching component 4800 of a drive component configured according to some embodiments of the present technology. As shown in Figure 48, the branching component 4800 may be generally similar to the I / O board 4710 described above with reference to Figure 47. For example, in the embodiment shown, the branching component 4800 includes a housing 4810, a redistribution network 4812, a plurality of first input nodes 4814, and a plurality of output nodes 4816. Each of the plurality of first input nodes 4814 can accept one or more connections and be coupled to the redistribution network 4812. For example, each of the plurality of first input nodes 4814 can be coupled to an electrical line (e.g., a power line, a signal routing line, etc.) to the redistribution network 4812. The redistribution network 4812 can then send inputs (e.g., power, control signals, driving force, etc.) to any (and / or all) of the plurality of output nodes 4816. Next, the multiple output nodes 4816 can be connected to one or more connection lines within the drive component to connect, for example, the redistribution network 4812 to a gripping generation unit, gripping assembly, etc.

[0258] In some embodiments, the redistribution network 4812 can send inputs received by a plurality of first input nodes 4814 to a subset of a plurality of output nodes 4816. For example, a first control signal received by a plurality of first input nodes 4814 can be sent to a first subset of a plurality of output nodes 4816, while a second control signal received by a plurality of first input nodes 4814 can be sent to a second subset of a plurality of output nodes 4816. The subset of a plurality of output nodes 4816 can then send the first control signal to a first subset, such as a gripping generation unit or gripping assembly, in order to grip a first target object. Similarly, the second subset of a plurality of output nodes 4816 can then send a second control signal to a second subset, such as a gripping generation unit or gripping assembly, in order to grip a second target object. As a result, for example, different subsets of gripping units and / or gripping assemblies can be operated to grip different target objects (e.g., to grip target objects aligned with different subsets of various sizes and / or end effectors).

[0259] As further shown in Figure 48, the branching component 4800 may also include one or more second input nodes 4818 (one is shown in Figure 48). Similar to the multiple first input nodes 4814, the second input node(s) 4818 may connect to route one or more connections into the redistribution network 4812. However, as further shown in Figure 48, the second input node(s) 4818 may have a different size and / or shape than the multiple first input nodes 4814. As a result, the connections received by the second input node(s) 4818 may differ from those received by the multiple first input nodes 4814. In a particular non-limiting example, the multiple first input nodes 4814 may receive connections related to the control and / or operation of various components within the gripping component, while the second input node(s) 4818 may receive connections that provide power to components within the gripping component. In another specific, non-limiting example, a plurality of first input nodes 4814 may receive a plurality of gripping assemblies coupled to a control unit and / or drive component, while a second input node(s) 4818 may receive connections related to the control and / or operation of the drive component.

[0260] Figure 49 is a partial schematic isometric view showing additional details regarding various components for the gripping component 4900 according to some embodiments of the present technology. In the embodiments shown, the gripping component 4900 includes a drive component 4910, an assembly actuation component 4950 coupled to the drive component 4910, and a plurality of gripping assemblies 4960 coupled to the assembly actuation component 4950.

[0261] The drive component 4910 may be generally similar to (or identical to) the drive component 4700 described above with reference to Figure 47. For example, as shown in Figure 49, the drive component 4910 may include a frame 4912, a branch component 4920 coupled to the frame 4912, and one or more grip generation units 4940 (five are shown in Figure 49) coupled to the branch component 4920. As further shown in Figure 49, the branch component 4920 may be generally similar to (or identical to) the branch component 4800 described above with reference to Figure 48. For example, the branch component 4920 may include a redistribution component 4922, a plurality of first input nodes 4924, a plurality of output nodes 4926 (one is labeled in Figure 47), and one or more second input nodes 4928.

[0262] As described above, multiple first input nodes 4924 can connect multiple first connectors 4932 to the redistribution component 4922. The redistribution component 4992 can then send inputs (e.g., power inputs, control inputs, force inputs, etc.) from the first connectors 4932 to one or more of the multiple output nodes 4926. The multiple output nodes 4926 connect the redistribution component 4922 to multiple third connectors 4936 that extend from the branching component 4920 to the grip generation unit 4940. More specifically, each of the multiple third connectors 4936 extends from one of the multiple output nodes 4926 to the grip generation unit 4940. As a result, the redistribution component 4922 can send inputs (e.g., power inputs, control inputs, force inputs, etc.) to the appropriate destination during a gripping operation using the gripping component 4900. Each of the gripping generation units 4940 can then generate (or send) a driving force (e.g., an attractive force, a magnetic force, and / or any other suitable force) to a corresponding one of the multiple gripping assemblies 4960.

[0263] Furthermore, a second input node 4928 on the branching component 4920 can connect one or more second connectors 4934 to the redistribution component 4922. As described above, inputs received via the second connectors 4934 may differ from inputs received from the multiple first connectors 4932. For example, inputs received via the multiple first connectors 4932 may relate to controlling the grip generation unit 4940 and / or providing power to the grip generation unit 4940, while inputs 4934 received via the second connectors 4934 may relate to controlling other components of the gripping component 4900 (e.g., the assembly operating component 4950 and / or multiple gripping assemblies 4960) and / or providing power to the gripping component 4900.

[0264] As further shown in Figure 49, the assembly actuation component 4950 may include one or more rotary drive mechanisms 4952 (e.g., a servo motor, a pulley and drive belt, a gear and track, and / or any other suitable mechanism) and a drive shaft 4954 coupled to the rotary drive mechanism 4952. Furthermore, each of the multiple gripping assemblies 4960 may be operably coupled to the drive shaft 4954 (which may also be referred to herein as a “common drive shaft,” “shared drive shaft,” etc.). As a result, the drive shaft 4954 can help actuate each of the multiple gripping assemblies 4960 simultaneously (or generally simultaneously) to help synchronize the motion of the gripping components 4900 during a gripping operation. In certain non-limiting examples, the proximal end of a pivotable link of the type described above with reference to Figure 35 may be coupled to the drive shaft 4954 to rotate between a first lowered position and a second raised position during a gripping operation. In another specific, non-limiting example, with reference to Figure 32A, the type of expandable component described above can be operably coupled to the drive shaft 4954 to move up and down in response to the rotation of the drive shaft 4954.

[0265] Exemplary Visual Processing for Object Placement Figure 50 shows various images illustrating the visual processing of object placement according to one or more embodiments. The processing shown in Figure 50 concerns deriving a gripping position for grasping and transporting an object. In some embodiments, the object may be an unrecognized object having an unknown size in one or more objects and placement in one or more objects. In embodiments of unrecognized objects, deriving the gripping position involves deriving an initial gripping position based on the object's MVR (e.g., corresponding to MVR1704 in Figure 17A). Based on the initial gripping position, as illustrated in the process shown in Figures 17A to 17F, the object can be lifted and its dimensions can be derived. Thus, previously unrecognized objects can be recognized, verified, registered, and / or transported as a result.

[0266] In some embodiments, the robot system can detect (e.g., identify and verify) objects as registered objects without deriving an MVR. For example, one or more objects in a placement can be compared with and matched to registered information in master data. Based on the match, the robot system can derive a gripping position for the removal of the matched object. The robot system can derive a gripping position for transport according to physical attributes in the matching registered data, such as known dimensions, known CoM position, and / or a predetermined gripping position.

[0267] Furthermore, or alternatively, the robotic system can further or partially identify objects that may not match the registered object without utilizing an initial lift. In the case of a portion of image 5000 that does not match the registered object or its known features, the robotic system can calculate with high accuracy that the depicted portion corresponds to a single object without an initial lift. For such a determination, the robotic system can analyze the depicted features according to predetermined rules that reflect various logical reasoning bases. For example, the robotic system can evaluate the height of a depicted portion relative to the floor of a container. When the evaluated height of the area is less than or equal to the known maximum height or a corresponding threshold, the robotic system can determine that the area corresponds to one row of objects (e.g., no other objects are stacked below or above the row). Alternatively, for example, the robotic system can determine the last or most peripheral box in a row when the corresponding edge has an edge confidence level higher than a predetermined threshold.

[0268] The images shown in Figure 50 represent an image 5000 (e.g., a 2D visual representation, a 3D representation, or a combination thereof, such as a color image or a grayscale image) and show how the image is processed to control a gripper (e.g., gripper 306 in Figure 3, 806 in Figure 8, 1500 in Figure 15, etc.) to remove objects from a stack or arrangement of objects in a reliable and efficient manner. The image 5000 (e.g., a 2D and / or 3D depiction of a stack of objects or a portion thereof) may be imaged along a horizontal direction perpendicular to the vertical plane in which objects A-E are arranged (e.g., a plane generally parallel to the crown and / or front of the cargo carrier, such as the xy plane). The image 5000 may be acquired from one or more visual sensors (e.g., the upper and / or lower visual sensors described above, such as in Figures 3 and 8). In Section I of Figure 50, the image 5000 may show a portion of an object arrangement 5002, which includes multiple objects stacked on top of each other. The object arrangement 5002 includes at least objects A, B, C, D, and E. Objects A, B, C, D, and E may include, for example, boxes of mixed sizes (e.g., mixed stockkeeping units (SKUs)) placed within a cargo carrier.

[0269] For illustrative purposes, Section I of Figure 50 shows a stack of mixed SKUs. However, it is understood that the robotic system can apply the described operations, processes, methods, etc., to other arrangements or conditions. For example, objects may correspond to common or uniform sizes and shapes, such as a single or integrated SKU. Furthermore, the robotic system can handle object arrangements that consider one or more objects, such as a single object, multiple objects arranged in a row, or structures on the floor or of another type that are not removable or applicable.

[0270] Section II of Figure 50 shows a detection region 5003 identified from image 5000. The detection region 5003 may be a portion of image 5000 that is identified or targeted by a robotic system for an object detection process, such as to identify a target object.

[0271] In some embodiments, the robotic system can process the image 5000 based on identifying / separating portions within the image 5000 and then further detecting objects within the image 5000. For example, the robotic system can identify a detected region 5003 based on identifying an enclosed region defined by a continuous set / connected set of detected edges (e.g., via the processor described above). When processing the image 5000 to initially detect objects depicted within the image 5000, such as to identify the type of object shown and / or its corresponding real-world location, the robotic system can detect 2D and / or 3D edges from the image 5000 using, for example, a Sobel filter. The robotic system can further detect 2D and / or 3D corners or joints where the edges intersect. Using the detected corners, joints, and edges, the robotic system can identify distinct surfaces or bounded segments, each representing one or more vertical faces or portions thereof within the image 5000. The robotic system can trace the edges along the entire connection to identify the containment boundaries and then define each containment boundary as a detection region 5003.

[0272] The robotic system can produce a verified detection by comparing the vertical plane and / or vertical portion of the detection area 5003 with the known size and shape and / or texture (e.g., visual characteristics on the rendered surface(s)) of the registered object. The robotic system can calculate a match score or measurement, or calculate the overlap between the detection area 5003 and the registered information. When the calculated score / measurement of the corresponding portion of the detection area 5003 exceeds a detection threshold, the robotic system can detect that the corresponding portion of the detection area 5003 indicates a matching object. Thus, the robotic system can identify the rendered object and verify the position / boundary of the rendered object based on the detection. The robotic system can identify one object, a set of matching objects, or multiple different objects within a given detection area.

[0273] In some embodiments, the detection region 5003 may include an unrecognized region 5004. The unrecognized region 5004 may be a portion of the detection region 5003 that the robot system does not detect or identify as an object matching or corresponding to registered information. The unrecognized region 5004 may represent, for example, a portion of the image 5000 that has an unknown vertical surface (e.g., a surface facing one or more vision sensors) that cannot be matched with a registered object. The robot system can determine each contiguous region (e.g., a region enclosed by a set of contiguous / connected edges) that does not match a registered object with at least a confidence value of a threshold amount as an unrecognized region 5004. In other words, the robot system can perform the initial detection as described above and then identify the rest of the image 5000 or detection region 5003 as an unrecognized region 5004. The robot system thereby identifies, based on the initial object detection process, the possibility that the corresponding region may contain one or more objects, or initially unknown objects, that cannot be distinguished from the image 5000 based on the initial object detection process.

[0274] The unrecognized region 5004 may correspond to multiple objects having vertical surfaces aligned within a threshold depth from each other (for example, none of the objects are positioned in front of another). For example, the vertical surfaces may be aligned within the threshold sensitivity of one or more sensors (e.g., 0.01 cm, 2 cm, 5 cm, or 10 cm from each other). Therefore, the robotic system may not be able to distinguish individual surfaces with the required confidence level and classify the corresponding region as the unrecognized region 5004.

[0275] In some embodiments, the robotic system can process the detection region 5003 by determining or estimating that multiple objects, rather than a single object, are depicted within the detection region. The robotic system can determine possible depictions of multiple objects based on one or more features associated with the detection region 5003, such as the number of corners, the relative angles of the corners (e.g., recessed or concave corners compared to protruding corners), the overall shape, and the length of the boundary edges. In the example shown in Section II of Figure 50, the robotic system can determine a number of likely objects because (1) the overall shape of the region is different from a rectangle, (2) the region contains more than four right angles, (3) the region contains at least one concave corner, and (4) the bottom edge 5006 exceeds the longest edge length among the registered objects or combinations thereof. In the example shown, the unrecognized regions 5004 may correspond to depictions of objects A-E or parts thereof that are adjacent to each other and within a threshold distance from each other. In some embodiments, the robotic system can process the detection region 5003 to identify registered information of registered objects and corresponding objects. In some embodiments, the rendered surfaces of objects may have negligible differences in depth and gaps between them (e.g., below the edge detection threshold / capability). Even if the assumption of multiple objects is inaccurate, the region may contain a single object with a size and / or shape that does not match any of the registered objects. In any case, the robotic system may determine that further processing is required before an object can be picked up from the region.

[0276] The robotic system can further process the image 5000 by identifying edges within the detection area 5003. In some embodiments, the robotic system can identify one or more verified edges 5013, which may be 2D and / or 3D edges with sufficient edge detection confidence to generate a verified detection. For example, the robotic system 100 can determine whether a detected edge, a verified edge 5013, or a combination thereof corresponds to an edge in the registered information of a registered object.

[0277] As shown in Section II, the robotic system 100 can process the detection region 5003 to determine that the region bounded by the vertical edge 5008, the top edge 5012, the bottom edge 5006, and the confirmed edge 5013 corresponds to the registration information of object A. In some situations, the robotic system may not be able to identify the confirmed edge 2013 in the image 5000, but it can identify candidate 2D and / or 3D edges from an initial detection process that did not have sufficient edge detection confidence and / or could not intersect with other edges. Among such candidate edges, the robotic system can identify edges located within the unrecognized region 5004, as shown in Section III of Figure 50. In some embodiments, the initial detection can be performed based on 3D data (e.g., a depth map) of the image 5000, and subsequent edge identification within the unrecognized region 5004 can be performed by detecting edges in the corresponding 2D portion or visual data of the image 5000.

[0278] In situations where the robotic system does not fully generate the verified detections from image 5000 (e.g., portions of depth corresponding to unknown surfaces that remain undetected), the corresponding unrecognized region 5004 in section II of Figure 50 may include regions defined by continuous boundaries formed by intersecting detected edges. For example, the unrecognized region 5004 may have an upper edge 5012 and a lower edge 5006. The unrecognized region 5004 may be located between vertical edges 5008 and 5010. The vertical edges 5008 and 5010 are positioned opposite each other and intersect with the upper edge 5012, thereby forming a 3D angle with the upper edge 5012. Thus, the vertical edges 5008 and 5010 can be determined as the outermost edges of the unrecognized region 5004.

[0279] In some embodiments, the upper edge 5012 can be identified from the image 5000 as the topmost 3D edge or known edge of the arrangement 5002. The bottom edge 5006 can be identified as one or more detected outer edges directly below the upper edge 5012 (e.g., with no other outer edges in between). In some examples, the bottom edge 5006 can be identified as being within a threshold distance range from the upper edge 5012. The threshold distance range may correspond to the maximum dimension (e.g., height) within the registered object.

[0280] The robotic system can (1) use the upper edge 5012 and the lower edge 5006 (the highest and lowest edges within the unrecognized region 5004) as reference outer edges, and (2) use the edges 5008 and 5010 (e.g., the outermost vertical edges) as reference vertical edges. The robotic system can use the reference edges to estimate the potential position of an object within the unrecognized region 5004.

[0281] Estimating the potential location of an object may involve calculating a hypothesis for the location of a vertically extending edge within the unrecognized region 5004. In other words, for estimation purposes, the robotic system may assume that a reference outer edge represents the top and bottom edges of one or more objects depicted in the unrecognized region 5004, and a reference vertical edge may represent one peripheral / vertical edge of a corresponding object depicted in the unrecognized region 5004. The vertical edge hypothesis may represent the location of a potential vertical edge along the horizontal axis (e.g., x-axis) and between the vertical reference edges. The vertical hypothesis can be calculated by deriving the potential vertical edges from the 2D and 3D image data of Figure 5000, which are parallel (parallel within threshold confidence) to the reference vertical edges 5008 and 5010. The vertical hypothesis may include potential edges with an edge detection confidence value lower than the threshold, and / or edges with at least one edge that is separated from the outer edge (e.g., not intersecting). Furthermore or alternatively, the vertical hypothesis may include 2D features. Potential vertical edges may extend at least partially between the bottom edge 5006 and the top edge 5012. The robotic system can assume that one or more potential vertical edges may represent gaps between each object in the object arrangement 5002. Potential vertical edges may also represent other vertical features identified from the image 5000, such as deformations on the object surface or visual features on the object surface (e.g., printed designs).

[0282] In the example shown in Figure 50, Section III may represent a vertical hypothesis 5016 derived from the potential vertical edge 5014 of Section II. The robotic system can calculate a vertical hypothesis 5016 that overlaps with the potential vertical edge 5014 and extends to intersect the upper edge 5012 and the lower edge 5006. Similarly, the robotic system can be configured to calculate a lateral hypothesis for an unrecognized region 5004 based on lateral reference edges (e.g., the upper edge 5012 and the lower edge 5006), in addition to or instead of the vertical hypothesis 5016.

[0283] In some embodiments, the process further includes identifying potential 3D angles of an object within an object arrangement 5002 based on reference outer edges (e.g., top edge 5012 and bottom edge 5006) and reference vertical edges (e.g., edges 5008 and 5010). In the example shown in Figure 50, the robotic system can calculate potential 3D angle 1 as the intersection of edge 5008 and top edge 5012, and potential angle 2 as the intersection of edge 5010 and top edge 5012. When anticipating a rectangular / cubic box, the robotic system can infer that angle 1 represents a portion of an unrecognized area 5004 belonging to a single object (e.g., object A), and angle 2 represents a portion logically belonging to a single object (e.g., object E). In other words, given an expected object, the robotic system can assume that each 3D angle corresponds to a surface that is reasonably likely to belong to one object. Thus, the robotic system can use the 3D angles as a basis for estimating and assuming the size, location, boundaries, etc., of an object. In some embodiments, the robotic system can use the intersections of key 3D edges, such as the upper outer edge and the outermost vertical edge of the unrecognized region 5004, as reference 3D angles to later assume the corresponding object and determine its position.

[0284] When estimating the position of an object depicted in an unrecognized region 5004, the robotic system can calculate one or more MVRs within the unrecognized region 5004 using a reference 3D angle and a vertical hypothesis 5016. An MVR refers to a portion of the surface of an object that is estimated to belong to a single object, or that is logically likely to belong to a single object. In some embodiments, the robotic system can calculate each MVR as an axially aligned bounding box (AABB) aligned with the corresponding top reference angle and extended to the bottom edge and the nearest vertical hypothesis. When the corresponding MVR has dimensions that are (1) less than the minimum dimension of the registered object, or (2) greater than the maximum dimension of the registered object, the robotic system may ignore or disregard the vertical hypothesis 5016. Furthermore, or alternatively, the robotic system may compare the candidate MVRs to a shape template of the registered object required for verification.

[0285] The robotic system can use the MVR to identify the gripping position of the corresponding estimated object. In the example shown in Section III of Figure 50, the robotic system can calculate the MVR 5018 of the object containing corner 1 (e.g., object A) that logically corresponds to corner 1 (e.g., object A). As described above, the robotic system can calculate the MVR 5018 based on the information derived for the unrecognized region 5004, which includes corner 1, the top edge 5012, the bottom edge 5006, the edge 5008, and the vertical hypothesis 5016. In the example shown, the robotic system can ignore the first vertical hypothesis because the corresponding MVR will have a width smaller than the minimum dimension of the registered object. Thus, the robotic system can extend the MVR to the next hypothesis / second hypothesis. Based on the MVR5018, the robotic system can derive the gripping position of object A (indicated by a star in Section III of Figure 50) based on predetermined rules, such as positioning the gripper / suction cup at or within a threshold distance of the bottom edge of the MVR5018.

[0286] After deriving the initial gripping position, the robot system can perform the processes described above with respect to Figures 17A to 18. The robot system can generate and implement an initial lift command to operate the gripper (e.g., gripper 306 in Figure 3) to contact object A in the initial gripping position, grip it, lift the gripped object A, and thereby separate the lifted object A from the support object(s) in the arrangement 5002.

[0287] Figure 51 shows various images illustrating the visual processing of an unrecognized object after the removal of an object (e.g., an object that was not previously recognized) according to one or more embodiments. Section I of Figure 51 shows the unrecognized region 5004 after object A has been removed. Referring again to the previous example, the robotic system can perform an initial lift operation on object A using the initial gripping position in the MVR 5018. The initial lift allows the robotic system to verify the actual dimensions of object A and then perform the transfer of the object using the actual / verified dimensions.

[0288] Following the removal of object A, the robotic system can identify the portion of unrecognized region 5004 corresponding to the removed object, for example, by using a mask to overlay the portion of unrecognized region 5004 that previously represented the removed object A. The robotic system can reclassify the hidden region as empty region 5102, as shown in Section II of Figure 51. In some embodiments, the robotic system can reclassify the edges of the empty region 5102 that abut the remaining unrecognized region 5004 as detected 3D edges. Furthermore, or alternatively, the robotic system can adjust the 3D depth measurement (for example, by increasing the depth measurement by a predetermined value) and / or update the 2D visual image so that the empty region 5102 represents a surface with a different texture and / or is away from the sensor.

[0289] Therefore, the robot system can update the unrecognized region 5004 to exclude portions corresponding to empty region 5102 or portions corresponding to transported objects (e.g., object A). As a result, the robot system can generate the adjusted unrecognized region 5104 without recapturing the image and / or redetecting objects in the image. Using the empty region 5102, the robot system can generate the edge 5108 of the adjusted unrecognized region 5104 adjacent to the empty region 5102. The robot system can set the edge 5108 as a reference vertical edge and process the adjusted unrecognized region 5104 as described above with respect to Figure 50. For example, the robot system can (1) identify the MVR corresponding to the next upper peripheral object (e.g., MVR 5106 corresponding to object B) that aligns with the 3D angle 3 corresponding to the edge 5108 and the upper edge 5012 and extends to one of the (e.g., nearest) vertical hypotheses 5016. In some embodiments, the robotic system can perform re-detection in the adjusted unrecognized area 5104 to potentially identify one or more of the confirmed detections. For example, in some situations, the removal of an object may increase the confidence of a candidate detection that might not meet the detection threshold. Thus, the robotic system can leverage the aftermath of the removed object to detect an object that was previously unrecognized. Furthermore or alternatively, as described in detail below, the robotic system can (1) register the removed object in the acquired sensor measurements or in the portion of the image corresponding to the empty area 5102, and (2) use the registration information to detect a matching object within the adjusted unrecognized area 5104.

[0290] In some embodiments, the robot system can repeat the process of calculating the MVR of an object, verifying the dimensions of the object after an initial lift, removing the object from the stack, and updating the unrecognized area 5004 according to the removed object. Thus, the robot system can iteratively remove objects (e.g., objects B, C, D, and E) that are depicted in the unrecognized area 5004 from the stack. As described above, the robot system can process and transport images depicted in the unrecognized area 5004 using one initial image (e.g., without reacquiring the image) and / or without redetecting objects depicted in the initial image. It should be noted that calculating the MVR of subsequent objects can be done using the initially acquired image (e.g., image 5000), and there is no need to collect further images (e.g., by the upper and / or lower vision sensors described above, such as in Figures 3 and 8). Furthermore, the robot system can transport unrecognized objects without redetecting objects depicted in the initially provided image.

[0291] If the system derives that the adjusted unrecognized area is less than the threshold area for identifying a subsequent MVR, the system can acquire additional sensor data and / or disqualify the hypothesis (e.g., by extending the MVR to the next vertical hypothesis). The system can then repeat the process described with respect to Figures 50 and 51 to identify additional unrecognized areas that have dimensions exceeding the minimum dimensions of an object expected to qualify as a subsequent MVR.

[0292] Figure 52 shows various images illustrating a visual processing for verifying an unrecognized object according to one or more embodiments. Figure 52 particularly shows an example in which an object has dimensions different from its virtual dimensions. Section I of Figure 52 shows an unrecognized region 5202 defined by a boundary including an upper edge 5204, a bottom edge 5206, and a side edge 5210. The unrecognized region 5202 can be calculated using the process described above with respect to Figures 50 and 51. The unrecognized region 5202 can represent the region of a stack of objects including objects F and G.

[0293] The robotic system can derive (1) an MVR5214 substantially corresponding to object F, and (2) an initial gripping position within the MVR5214 (indicated by a star). However, as shown in the example in Section I of Figure 52, object F has an actual bottom edge 5208 that differs from the estimated / hypothetical bottom edge 5206 derived from an unrecognized region 5202.

[0294] Section II of Figure 52 shows an embodiment of the initial lift and the corresponding measurements taken by the distance sensor 1714. The process for performing such measurements is described above with respect to Figures 17A to 17F. As shown, the robotic system can use the vertical (e.g., along the y-axis) distance measurements to verify that the lifted object (e.g., object F) has a greater height 5216 between the upper edge 5204 and the verified bottom edge 5208 than the estimated edge (e.g., lower than the estimated edge). Thus, the system derives that the verified bottom (i.e., the actual bottom) 5208 is lower than the bottom edge 5206 estimated based on the unrecognized area 5202.

[0295] The robotic system can further adjust the gripping position based on the verified bottom edge. For example, the robotic system can adjust the gripping position according to the same rules / parameters as the initial lift gripping position, so that the adjusted gripping position is either in contact with the verified bottom edge of an object or within a threshold gripping distance from the verified bottom edge.

[0296] Selection of examples of unrecognized objects Figure 53 shows various images illustrating target selection for an unrecognized object according to one or more embodiments. The process described with respect to Figure 53 concerns selecting a suspected object to be lifted from among multiple potential objects in a stack of objects. In other words, a robotic system can use the process illustrated and described with respect to Figure 53 to determine which part of an unrecognized area (e.g., a corner and / or corresponding MVR) should be examined first.

[0297] Section I of Figure 53 shows Image 5300. Similarly, as described with respect to Image 5000 in Figure 50, Image 5300 is a visual 2D representation, 3D representation, or a combination thereof, such as a color or grayscale image, showing a stack of objects 5301 including real-world objects H, I, J, K, and J. Objects H, I, J, K, and J may represent, for example, boxes of mixed sizes (e.g., mixed stockkeeping units (SKUs)) placed on a cargo carrier.

[0298] Section II of Figure 53 includes an unrecognized region 5302 derived from image 5300 based on the process described with respect to Figure 50. For example, the unrecognized region 5302 is defined by edges 5304, 5308, 5312, 5310, and 5314. Furthermore, the unrecognized region 5302 includes an edge 5306 representing the gap between objects I and J. For example, the system identifies the gap between objects I and J from the 3D representation in image 5300 with a sufficiently high probability and classifies the gap as the central edge 5306 of the stack 5301. Based on the identified edges, the system can identify multiple 3D angles (e.g., angles 1, 2, 3, and 4) that are predicted to correspond to multiple objects within the unrecognized region 5302. These multiple angles could be used to calculate the MVR and initial lift position for different objects.

[0299] The robotic system can be configured to calculate the order in which initial lifts should be performed and to effectively determine which object should be lifted first. To reduce confusion in the stack 5301 (for example, to prevent damage or displacement of adjacent objects), the system may prioritize the 3D angles of the outermost objects in the unrecognized region 5202 over the 3D angles of objects located in the central part of the unrecognized region 5302. For example, the robotic system may select an angle / MVR to lift (1) the leftmost object (e.g., object H) based on angle 1, or (2) the rightmost object (e.g., object L) based on angle 2, compared to selecting an angle corresponding to an inner object I or K. Furthermore, or alternatively, the robotic system may consider an inner angle if the lateral separation between the target surface / MVR and the adjacent surface exceeds a separation threshold.

[0300] In some embodiments, the robot system can derive a lifting priority for candidate objects (e.g., the outermost or surfaces with sufficient separation) based on the relative position of the gripper (e.g., gripper 306 in Figure 3) to each of the candidate objects, for example, to reduce the time required for the gripper to move during the lift. For example, in an example where the gripper is positioned closer to object H on the left side of the stack 5301 than object L, the robot system can select the MVR corresponding to object L first for the initial lift. Furthermore, after object L is removed, the robot system can decide to lift object I next because the gripper is positioned closer to object I than object L.

[0301] The system can lift an object at the top before lifting an object at a lower position within the stack 5301. For example, the robotic system can lift object K based on angle 3 before lifting objects H, I, J, or L. In particular, the robotic system can calculate multiple vertical hypotheses 5316 and horizontal hypotheses 5318. Based on the positions of angle 3 and hypotheses 5316 and 5318, the robotic system can calculate the MVR of object K adjacent to angle 3. The robotic system may have a predetermined hierarchy or sequence for processing multiple selection rules. For example, the system may prioritize the highest MVR over the outermost MVR.

[0302] Example of grasping calculation for an unrecognized object Figures 54A to 54B show images illustrating gripping calculations for rotating objects according to one or more embodiments. The gripping calculations shown can be applied to unrecognized objects and / or detected objects. In some examples, the robotic system derives that a set of edges (e.g., an unrecognized region 5004 as described with respect to Figure 50) corresponds to a rotated orientation of the object (e.g., rotated around the z-axis). As shown in Figures 54A to 54B, an object M with a potential bottom edge 5412 is in a rotated (or tilted) orientation, so that the lower left corner (angle 1) of object M is positioned higher (e.g., along the y-axis) than the lower right corner (angle 2). The robotic system can detect such a rotated orientation by detecting a set of intersecting edges in 2D and / or 3D image data that are offset from the vertical / horizontal axis by a complementary angle.

[0303] Based on the inclined or angled edge, the robotic system can calculate one or more MVRs for an object that is also in a rotated position. For example, MVR5402 associated with angle 1 and MVR5406 associated with angle 2 are in a rotated position according to the rotated position of object M.

[0304] In Figure 54A, based on detecting the rotated orientation, the robot system can calculate the initial gripping position according to the corresponding rule. In some embodiments, the robot system can deviate from the lowest gripping position to derive a gripping position for a rotating object. For example, to account for a downward-facing verification sensor, the robot system can identify the second lowest angle (e.g., angle 1, which is positioned higher than angle 2 along the y-axis) as the reference for the rotated gripping position. As a result, the corresponding initial lift can indicate a change around the edge extending between the two lowest angles of the MVR, and a possible separation from the support object. Alternatively, for example, the robot system can select the angle with the widest dimension for the hypothesis. As shown in Figure 54A, the two suction cups of the gripper 306 (e.g., suction cups 340A and 340B) can grip the lower portion of the MVR 5402 to lift the object M.

[0305] During lifting, distance measurements can be performed using one or two distance sensors (e.g., distance sensors 1714 and 1418, as described with respect to Figures 17A to 17F) to verify the dimensions and / or position of the bottom edge 5412 of the object M. In particular, once the object is lifted in the initial gripping position by the MVR 5402, the width of the object M can be verified using the lateral distance measurement from distance sensor 1718, and the height and / or position of the inclined bottom edge 5412 of the object M can be verified using the vertical distance measurement from distance sensor 1714.

[0306] After verifying the dimensions and / or bottom edge 5412 of object M, the robot system can generate a transfer gripping position 5420 that is within the MVR ...

Claims

1. A method for operating a robotic system, To acquire sensor data representing the object at the starting position, This includes operating one or more segments and an end-of-arm tool (EOAT) to generate one or more commands for transporting the object from the starting position toward a target position along a set of frame conveyors over the robotic arms of the EOAT and the one or more segments. Generating one or more commands includes positioning the EOAT to grip the object using one or more gripping elements, The EOAT, together with the frame conveyor set of the EOAT, is positioned at an angle to pull and lift the gripped object during the initial portion of the transfer.

2. The one or more commands are for operating one or more pivotable links of the EOAT, The one or more pivotable links are operably connected to the one or more gripping elements. Positioning one or more gripping elements at the first position toward the object, Gripping the object using the extended gripping element, To lift the one or more gripping elements and the gripped object, the one or more pivotable links are rotatably retracted to a second position. The method according to claim 1, configured to perform the following:

3. The one or more commands described above are for operating the one or more pivotable links to move the bottom of the grasped object to contact the distal end portion of the EOAT, The method according to claim 2, wherein the distal end portion supports the grasped object while the grasped object is moved onto the EOAT.

4. The method according to claim 2, wherein the one or more commands are for positioning the EOAT and operating the pivotable link to tilt the gripped object such that the upper part of the gripping surface of the object rotates away from the EOAT.

5. The method according to claim 4, wherein the one or more commands are to operate the EOAT to lift the object onto the local conveyor, while maintaining an inclined position of the gripped object to reduce surface friction between the gripped object and a support object that contacts the gripped object below the gripped object.

6. The acquired sensor data represents one or more depictions of the object from the side-facing sensor. The aforementioned objects are (1) located in a cargo storage room or container, and (2) each stacked on top of and / or adjacent to one or more objects having surfaces exposed to each other and to the lateral sensor within a threshold distance. The method according to claim 1, wherein the one or more generated commands are for operating the one or more segments and the EOAT to remove the object from the cargo storage room or the container along a continuous path on the EOAT and the one or more segments.

7. The further includes determining a receiving structure position for arranging a structure configured to receive the transported object, The one or more commands generated above are: (1) Arrange one or more segments around the chassis and arrange the EOAT, (2) In order to grasp the object, (3) In order to transport the object along the EOAT and one or more segments, (4) The chassis (a) above the receiving structure and / or on top of the receiving structure, (b) The method according to claim 1, for the purpose of maintaining within a threshold distance from the receiving structure.

8. The further includes determining a receiving structure position for arranging a structure configured to receive the transported object, The one or more commands generated above are: To operate one or more segments, including at least (1) a front segment connecting the chassis to the EOAT, and (2) a rear segment attached to the chassis on the opposite side of the front segment, (1) To position the chassis so as to transport the object along a path along the EOAT, the front segment, the chassis, and the rear segment, The method according to claim 1, for positioning the chassis and / or the rear segment such that, when the transported object moves past the rear segment, the rear segment has an exit portion that overlaps with the receiving structure position.

9. A method for operating a robotic system, Acquiring first sensor data from a first sensor, wherein the first sensor data includes two-dimensional (2D) visual representations and / or three-dimensional (3D) representations of a plurality of objects at a starting position. Identifying unrecognized regions within the first sensor data, wherein the unrecognized regions represent one or more vertically oriented object surfaces that are adjacent to each other and located within each other's threshold depths, The calculation of the minimum viable region (MVR) within the unrecognized region, wherein the MVR estimates at least a portion of a continuous plane belonging to one object located within the unrecognized region. To contact and grasp the aforementioned object, the target grasping position in the MVR for the end-of-arm tool (EOAT) of the robot system is derived, The EOAT is operated to (1) grasp the one object at the target gripping position, and (2) perform initial displacement to generate one or more initial displacement commands for separating the one object from the bottom support object and / or laterally adjacent objects. Acquiring second sensor data from a second sensor position different from the acquisition position of the first sensor data, wherein the second sensor data includes at least a representation of the bottom edge of the one object separated from the bottom support object by the initial displacement, To generate a verified detection of the one object based on the second sensor data, wherein the verified detection includes a verified bottom edge and / or verified side edge of the one object, Based on the verified detection, the robot system is operated to generate one or more transport commands for transporting the one object from the starting position over the EOAT and one or more subsequent segments, Methods that include...

10. One or more vertical hypotheses about the potential object position of the aforementioned object, From the first sensor data, the reference vertical edge and / or reference outer edge are identified, Deriving one or more potential vertical edges and / or one or more potential outer edges within the unrecognized region from the first sensor data, wherein the one or more potential vertical edges are parallel to and / or opposite to the reference vertical edge, and the one or more potential outer edges are parallel to and / or opposite to the reference outer edge, Identifying a reference 3D angle based on one or more identified potential vertical edges and / or one or more potential outer edges, wherein the reference 3D angle represents a portion corresponding to one of the objects, and the MVR is calculated based on the one or more vertical hypotheses. Calculate based on, The method according to claim 9, further comprising:

11. The process further includes adjusting the unrecognized region based on reclassifying the portion of the unrecognized region corresponding to one of the aforementioned objects, The aforementioned unrecognized region is adjusted after generating one or more transport commands. The method according to claim 9, wherein the adjusted unrecognized region is used to (1) identify a subsequent MVR corresponding to a subsequent object depicted in the adjusted unrecognized region, and (2) generate an instruction to transport the subsequent object.

12. The further includes determining that at least a portion of the unrecognized region corresponds to a rotated rectangular orientation, The MVR is calculated to have the rotated posture, The target gripping position for the initial displacement is based on a higher angle corresponding to the hypothetical bottom edge, The method according to claim 12, wherein the one or more verified transport commands are for transporting the one object based on gripping against a lower corner corresponding to a verified bottom edge.

13. Identifying the aforementioned unrecognized region means Detecting the 3D edge based on the 3D representation of the first sensor data, Identifying a 3D angle based on the intersection of the aforementioned 3D edges, Identifying a boundary region based on detecting the set of 3D edges and the set of 3D corners that form a continuous encapsulation boundary, The boundary region is identified as the unrecognized region when (1) it contains four or more 3D angles, (2) it contains dimensions exceeding the maximum dimension among the expected objects registered in the master data, (3) it contains dimensions less than the minimum dimension among the expected objects, or (4) it has a shape different from a rectangle, or a combination thereof. The method according to claim 9, including the method described in claim 9.

14. Identifying the 3D angles in the previously unrecognized region, wherein each of the 3D angles represents a portion that uniquely corresponds to one associated object. The method further includes identifying the current location of the EOAT, The method according to claim 9, wherein deriving the target gripping position includes selecting the MVR corresponding to one of the 3D angles closest to the current position.

15. Calculating the height based on the aforementioned vertical base edge, Registering the aforementioned object by updating the master data to include the aforementioned height of the aforementioned object, Identifying a newly detected object by comparing the remaining portion of the unrecognized area with the updated master data and identifying the boundary region of the remaining portion having the calculated height, The method according to claim 9, further comprising:

16. To estimate the centroid (CoM) position relative to the continuous plane represented by the vertical detection, The calculation of the zero moment point (ZMP) range for gripping and transporting the object, The ZMP is calculated based on at least one dimension of the vertical plane and the acceleration associated with the transfer of the object. The ZMP range is centered on the CoM position and represents one or more support positions on the vertical plane or the object rendering region where the reaction force to the object is balanced during the transfer. A gripping posture is derived based on arranging at least one gripping element of the EOAT that partially or completely overlaps the ZMP range, wherein the gripping posture is such that the EOAT is positioned to grip the vertical surface of the object when transporting the object from the container. Methods that include...

17. Based on the verified detection, derive an action plan for the operation of the robot system to transport the one object, wherein one or more transport commands are generated according to the action plan. Monitoring in real time the workload measurement values ​​representing the work capacity of the EOAT, the segment, and / or the conveyor set, Controlling the execution of the operation plan according to the monitored workload measurements, The method according to claim 9, further comprising:

18. It is a robotic system, Chassis and, A segment rotatably connected to the chassis and configured to move relative to the chassis via a segment actuator, An end-of-arm tool (EOAT) rotatably connected to the segment and configured to move relative to the segment via an EOAT actuator, the end-of-arm tool (EOAT) includes a movable and operable gripper interface configured to grip the vertical surface of an object, A first sensor is located between the EOAT and the chassis and is configured to acquire a three-dimensional (3D) and / or two-dimensional (2D) depiction of the space beyond the EOAT. A second sensor is positioned on the EOAT and configured to acquire at least a depiction of the detection space below and / or beyond the EOAT, The system comprises the segment actuator, the EOAT actuator, the first sensor, the second sensor, and a processor communicatively coupled to the EOAT, The processor is configured to (1) receive outputs from the first and second sensors and (2) generate commands for the segment actuator, the EOAT actuator, and the EOAT in a robotic system.

19. The processor further comprises a memory that is communicably coupled to the aforementioned processor, When the memory is executed by the processor, the processor will The first sensor data is obtained from the first sensor, wherein the first sensor data represents a plurality of objects at the starting position. Identifying unrecognized regions within the first sensor data, wherein the unrecognized regions represent one or more vertical object surfaces and adjacent object surfaces that are within a threshold distance of each other. The calculation of the minimum viable region (MVR) within the unrecognized region, wherein the MVR estimates at least a portion of a continuous plane belonging to one object located within the unrecognized region. The EOAT is operated to derive a target gripping position within the MVR in order to contact and grip the one object, The EOAT is operated to (1) grasp the one object at the target gripping position, and (2) perform initial displacement to generate one or more initial displacement commands for separating the one object from the bottom support object and / or laterally adjacent objects. Acquiring second sensor data from the second sensor, wherein the second sensor data also includes a representation of at least 3D of the bottom edge of the one object separated from the bottom support object by the initial displacement, To generate a verified detection of the one object based on the second sensor data, wherein the verified detection includes a verified bottom edge and / or verified side edge of the one object, Based on the verified detection, the EOAT, the segment, and the chassis are operated to generate one or more transport commands for transporting the one object over and across the EOAT, the segment, and the chassis toward a downstream robot or location that interfaces with it. The robot system according to claim 18, including an instruction to perform the following:

20. The aforementioned EOAT has a wedge-shaped side profile and includes a local conveyor on its upper part. The method according to claim 18, wherein the one or more generated commands are to position the EOAT together with its local conveyor at an inclination to pull and lift the grasped object during the initial portion of the transfer.

Images