Robot multi-gripper assembly and method for grasping and holding objects

Abstract

To provide improved robotic systems and techniques for controlling and managing various aspects of the robots.SOLUTION: A method for operating a transport robot includes receiving image data representative of a group of objects, and identifying target objects in the group. Addressable vacuum regions are selected based on the identified target objects. The transport robot gives a command to cause the selected addressable vacuum regions to hold and transport the identified target objects. The transport robot includes a multi-gripper assembly having an array of addressable vacuum regions each configured to independently provide a vacuum. A vision sensor device can capture the image data, which is representative of the target objects adjacent to the multi-gripper assembly.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 62 / 889,562, filed on August 21, 2019, which is hereby incorporated by reference in its entirety.

[0002] This technology generally relates to robotic systems, and more particularly to robotic multi - gripper assemblies configured to selectively grasp and hold objects.

Background Art

[0003] Robots (e.g., machines configured to automatically or autonomously perform physical actions) are currently widely used in many fields. For example, robots can be used to perform various tasks (e.g., manipulate or transfer objects) in manufacturing, packaging, transportation, and / or shipping. When performing tasks, robots can reproduce human actions, thus replacing or reducing the human intervention that would otherwise be required to perform dangerous or repetitive tasks. However, robots often lack the advanced capabilities necessary to reproduce the human sensitivity and / or adaptability required to perform more complex tasks. For example, robots often have difficulty selectively grasping objects (s) from a group of objects, including nearby objects and objects of irregular shape / size. Therefore, improved robotic systems and technologies for controlling and managing various aspects of robots are still needed.

Brief Description of the Drawings

[0004] [Figure 1] FIG. is a diagram of an exemplary environment in which a robotic system transfers an object according to one or more embodiments of the present technology. [Figure 2] FIG. is a block diagram showing a robotic system according to one or more embodiments of the present technology. [Figure 3] This document shows a multi-component transfer assembly according to one or more embodiments of the present technology. [Figure 4] This is a front view of an end effector coupled to the robot arm of a transport robot, according to one or more embodiments of the present technology. [Figure 5] Figure 4 is a bottom view of the end effector. [Figure 6] This is a functional block diagram of a robot transfer assembly according to one or more embodiments of the present technology. [Figure 7] These are front and top isometric views of an end effector having a multi-gripper assembly according to one or more embodiments of the present technology. [Figure 8] Figure 7 shows isometric front and bottom views of the end effector. [Figure 9] This is an exploded front isometric view of the components of a vacuum gripper assembly according to one or more embodiments of the present technology. [Figure 10] This is an isometric view of a vacuum gripper assembly according to one or more embodiments of the present technology. [Figure 11] Figure 10 is a plan view of the assembly. [Figure 12] This is an isometric view of a vacuum gripper assembly according to one or more embodiments of the present technology. [Figure 13] This is an isometric view of a multi-gripper assembly according to another embodiment of the present technology. [Figure 14] Figure 13 is an exploded isometric view of the multi-gripper assembly. [Figure 15] This is a partial cross-sectional view of a multi-gripper assembly according to one or more embodiments of the present technology. [Figure 16] This is a flowchart illustrating how to operate a robotic system according to several embodiments of this technology. [Figure 17] This is another flowchart for operating a robotic system according to one or more embodiments of the present technology. [Figure 18]This shows the steps of grasping and transporting an object with a robot according to one or more embodiments of this technology. [Figure 19] This shows the steps of grasping and transporting an object with a robot according to one or more embodiments of this technology. [Figure 20] This shows the steps of grasping and transporting an object with a robot according to one or more embodiments of this technology. [Figure 21] This shows the steps of grasping and transporting an object with a robot according to one or more embodiments of this technology. [Modes for carrying out the invention]

[0005] This specification describes systems and methods for gripping selected objects. The system may include a transport robot having a multi-gripper assembly configured to operate independently or in conjunction to grip / release a single or multiple objects. For example, the system may pick up multiple objects simultaneously or sequentially. The system may select the objects to be transported based, for example, the transport capacity of the multi-gripper assembly, the transport plan, or a combination thereof. The multi-gripper assembly can reliably grip objects from groups of objects, irregular objects, objects of different shapes / sizes, etc. For example, the multi-gripper assembly may include addressable vacuum regions or banks, each configured to draw in air so that only selectable objects are held via vacuum gripping. The multi-gripper assembly can be moved by the robot to transport the held objects to a desired location and then release the objects. The system may also release the gripped objects simultaneously or sequentially. This process can be repeated to transport any number of objects between different locations.

[0006] At least some embodiments relate to methods for operating a transport robot having a multi-gripper assembly with an addressable pick-up area. The pick-up area can be configured to independently provide vacuum gripping. Target objects are identified based on captured image data. The pick-up area can draw in air to grip the identified target object(s). In some embodiments, the transport robot robotically moves the multi-gripper assembly to carry the identified target object.

[0007] In some embodiments, the robotic transport system includes a robotic device, an object detector, and a vacuum gripper device. The vacuum gripper device includes a plurality of addressable regions and a manifold assembly. The manifold assembly can be fluidly coupled to each addressable region and at least one vacuum line so that each addressable region can independently provide negative pressure through an array of suction elements. The negative pressure may be sufficient to hold at least one object against the vacuum gripper device while the robotic device moves the vacuum gripper device between different locations.

[0008] A method for operating a transport robot includes receiving image data representing a group of objects (e.g., a stack or pile of objects). Based on the received image data, one or more target objects within the group are identified. An addressable vacuum region is selected based on the identified one or more target objects. The transport robot issues a command to the selected vacuum region to hold and transport the identified one or more target objects. The transport robot includes a multi-gripper assembly having an array of vacuum regions, each configured to independently provide vacuum gripping. A visual sensor device may capture image data representing target objects adjacent to or held by the vacuum gripper device.

[0009] The following details are numerous in order to provide a complete understanding of the currently disclosed technology. In other embodiments, the technology described herein can be implemented without these specific details. In other embodiments, well-known features, such as certain functions or routines, are not described in detail to avoid unnecessarily obscuring the disclosure. References to “embodiments,” “one embodiment,” etc., in this specification mean that the particular features, structures, materials, or properties described are included in at least one embodiment of this disclosure. Thus, the appearance of such phrases in this specification does not necessarily refer to the same embodiment. On the other hand, such references are not necessarily mutually exclusive. Furthermore, particular features, structures, materials, or properties can be combined in any suitable way in one or more embodiments. It should be understood that the various embodiments shown in the figures are merely illustrative and are not necessarily depicted to scale.

[0010] Some details describing structures or processes, which are well known and often associated with robotic systems and subsystems, but which could unnecessarily obscure some important aspects of the disclosed technology, are omitted from the following description for clarity. Furthermore, while the following disclosure shows several embodiments of different aspects of the technology, some other embodiments may have different configurations or components than those described in this section. Thus, the disclosed technology may have other embodiments that have additional elements or do not have some of the elements described below.

[0011] Many embodiments or aspects of the disclosure described below can take the form of computer-executable instructions or controller-executable instructions, including routines executed by a programmable computer or controller. Those skilled in the art will understand that the disclosed technology can be implemented on computer or controller systems other than those shown and described below. The technology described herein can be embodied in a dedicated computer or data processor that is specifically programmed, configured, or constructed to execute one or more of the computer-executable instructions described below. Thus, the terms “computer” and “controller” as used herein refer to any data processor and may include Internet devices and handheld devices (including palmtop computers, wearable computers, mobile phones or mobile phones, multiprocessor systems, processor-based or programmable home appliances, network computers, minicomputers, etc.). Information processed by these computers and controllers can be displayed on any suitable display medium, including liquid crystal displays (LCDs). Instructions for executing computer-executable tasks or controller-executable tasks can be stored in any suitable computer-readable medium, including hardware, firmware, or a combination of hardware and firmware. The instructions can be contained in any suitable memory device, including, for example, flash drives, USB devices, and / or other suitable media, including tangible, non-temporary computer-readable media.

[0012] The terms "coupled" and "connected," along with their derivatives, can be used herein to describe the structural relationship between components. It should be understood that these terms are not intended to be synonyms for each other. Rather, in certain embodiments, "connected" can be used to indicate that two or more elements are in direct contact with each other. Unless otherwise apparent from the context, the term "coupled" can be used to indicate that two or more elements are in contact with each other, either directly or indirectly (with other intervening elements therebetween), that two or more elements are in cooperation or interaction with each other (such as in a causal relationship like signal transmission / reception or function calls), or both.

[0013] appropriate environment FIG. 1 is a diagram of an exemplary environment in which a robot system 100 transports an object. The robot system 100 can include a unloading unit 102, a transfer unit or assembly 104 ("transfer assembly 104"), a conveying unit 106, a loading unit 108, or a combination thereof, in a warehouse or a distribution / shipping hub. Each unit of the robot system 100 can be configured to perform one or more tasks. The tasks can be combined in sequence to perform actions that achieve goals, such as unloading an object from a truck or van for storage in a warehouse or unloading an object from a storage location and loading it onto a truck or van for shipping. In another example, the tasks can include moving an object from one container to another. Each unit can be configured to perform a series of actions to perform a task (e.g., operating one or more of its components).

[0014] In some embodiments, the task may include manipulating (e.g., moving and / or reorienting) an object or package 112 (e.g., a box, case, cage, pallet, etc.) from a starting location 114 to a task location 116. For example, an unloading unit 102 (e.g., a devanning robot) may be configured to transfer the target package 112 from a location in a transport vehicle (e.g., a truck) to a location on a conveyor belt. A transfer assembly 104 (e.g., a palletizing robot assembly) may be configured to load the package 112 onto a transport unit 106 or onto a conveyor 120. In another embodiment, the transfer assembly 104 may be configured to transfer one or more target packages 112 from one container to another. The transfer assembly 104 may include robot end effectors 140 ("end effector 140") having vacuum grippers (or vacuum areas) that operate individually to pick up and carry each object(s) 112. When the end effector 140 is positioned adjacent to an object, air can enter the gripper(s) adjacent to the target package 112, thereby creating a pressure difference sufficient to hold the object. The object can be picked up and transported without damaging or scratching its surface. The number of packages 112 transported at one time can be selected based on the stacking arrangement of objects at the picking location, the available space at the dropping location, the transport path between the picking and dropping locations, optimization routines (e.g., routines for optimizing unit usage, robot usage, etc.), or a combination thereof. The end effector 140 may have one or more sensors configured to output measurements indicating information about the held objects (e.g., the number and configuration of the held objects), the relative positions between any held objects, etc.

[0015] Imaging system 160 can provide image data used to monitor the operation of components, identify target objects, track objects, or perform tasks in other ways. The image data can be analyzed, for example, to evaluate the arrangement of stacked packages (such as stacked packages like cardboard boxes, packing containers, etc.), the position information of objects, available transport routes (such as the transport route between the pick-up section and the drop-off section), the position information regarding the gripping assembly, or combinations thereof. Controller 109 can communicate with imaging system 160 and other components of robot system 100. Controller 109 can generate a transport plan including the order for picking up and dropping objects (such as shown as stable containers), positioning information, command information for picking up objects, command information for dropping objects, stacking plans (such as plans for stacking objects in the drop-off section), re-stacking plans (such as plans for re-stacking at least a part of the containers in the pick-up section), or combinations thereof. The information and instructions provided by the transport plan can be selected based on the arrangement of the containers, the content of the containers, or combinations thereof. In some embodiments, controller 109 can include electronic / electrical devices such as one or more processing units, processors, storage devices (such as external or internal storage devices, memories, etc.), communication devices (such as communication devices for wireless or wired connections), and input / output devices (such as screens, touch screen displays, keyboards, keypads, etc.). Exemplary electronic / electrical devices and controller components are described in relation to FIGS. 2 and 6.

[0016] The transport unit 106 can transport the target package 112 (or multiple target packages 112) from a range associated with the transport assembly 104 to a range associated with the loading unit 108, and the loading unit 108 can transport the target package 112 to a storage location (for example, by moving a pallet on which the target package 112 is placed). In some embodiments, the controller 109 can coordinate the operation of the transport assembly 104 and the transport unit 106 to efficiently load the objects onto storage shelves.

[0017] The robot system 100 may include other units not shown in Figure 1, such as manipulators, service robots, and modular robots. For example, in some embodiments, the robot system 100 may include a depalletizing unit for transferring objects from a cage cart or pallet to a conveyor or other pallet, a container switching unit for transferring objects from one container to another, a packaging unit for wrapping objects, a sorting unit for grouping objects according to one or more characteristics of the objects, a piece picking unit for manipulating objects in different ways according to one or more characteristics of the objects (e.g., for sorting, grouping, and / or transferring), or a combination thereof. The components and subsystems of system 100 may include different types of end effectors. For example, the unloading unit 102, the transport unit 106, the loading unit 108, and other components of the robot system 100 may also include a robot multi-gripper assembly. The configuration of the robot gripper assembly can be selected based on the desired carrying capacity. For illustrative purposes, the robotic system 100 is described in the context of a distribution center, but it should be understood that the robotic system 100 can be configured to perform tasks in other environments, such as manufacturing, assembly, packaging, healthcare, and / or other types of automation. Further details regarding the tasks and related actions are described below.

[0018] robot systems Figure 2 is a block diagram showing the components of a robot system 100 according to one or more embodiments of the present technology. In some embodiments, for example, the robot system 100 (e.g., one or more of the units or assemblies and / or robots described above) may include electronic / electrical devices such as one or more processors 202, one or more storage devices 204, one or more communication devices 206, one or more input / output devices 208, one or more actuator devices 212, one or more transport motors 214, one or more sensors 216, or a combination thereof. Various devices can be coupled to each other via wired and / or wireless connections. For example, the robot system 100 may include buses such as a system bus, Peripheral Component Interconnect (PCI) bus or PCI-Express bus, Hypertransport or Industry Standard Architecture (ISA) bus, Small Computer System Interface (SCSI) bus, Universal Serial Bus (USB), IIC (I2C) bus, or IEEE (Institute of Electrical and Electronics Engineers) Standard 1394 bus (also known as "FireWire"). Furthermore, for example, the robot system 100 may include bridges, adapters, controllers, or other signal-related devices to provide wired connections between devices. Wireless connections may be based on, for example, cellular communication protocols (e.g., 3G, 4G, LTE, 5G, etc.), wireless local area network (LAN) protocols (e.g., Wireless Fidelity (Wi-Fi)), peer-to-peer or device-to-device communication protocols (e.g., Bluetooth®, Near Field Communication (NFC), etc.), Internet of Things (IoT) protocols (e.g., NB-IoT, Zigbee, Z-wave, LTE-M, etc.), and / or other wireless communication protocols.

[0019] The processor 202 may include a data processor (e.g., a central processing unit (CPU), a dedicated computer, and / or an onboard server) configured to execute instructions (e.g., software instructions) stored in a memory device 204 (e.g., computer memory). The processor 202 may execute program instructions for controlling / interfacing with other devices, thereby causing the robot system 100 to perform actions, tasks, and / or movements.

[0020] The storage device 204 may include a non-temporary computer-readable medium on which program instructions (e.g., software) are stored. Some embodiments of the storage device 204 may include volatile memory (e.g., cache and / or random access memory (RAM) and / or non-volatile memory (e.g., flash memory and / or magnetic disk drives)). Other embodiments of the storage device 204 may include a portable memory drive and / or a cloud storage device.

[0021] In some embodiments, a storage device 204 can be used to further store and provide access to master data, processing results, and / or predetermined data / thresholds. For example, the storage device 204 may store master data including descriptions of objects (e.g., boxes, cases, containers, and / or products) that can be operated by the robot system 100. In one or more embodiments, the master data may include dimensions, shape (e.g., templates for potential poses and / or computer-generated models for recognizing objects in different poses), mass / weight information, color scheme, image, identification information (e.g., barcodes, quick response (QR) codes®, logos, etc., and / or their expected locations), expected mass or weight, or a combination thereof, of the object expected to be operated by the robot system 100. In some embodiments, the master data may include operation-related information about the object, such as the location of the center of gravity of each object, expected sensor measurements corresponding to one or more actions / operations (e.g., force, torque, pressure, and / or contact measurements), or a combination thereof. The robot system can retrieve pressure levels (e.g., vacuum levels, suction levels, etc.), gripping / lifting ranges (e.g., range or bank of vacuum grippers to be operated), and other stored master data for controlling the transport robot. The storage device 204 can also store object tracking data. In some embodiments, the object tracking data may include a log of scanned or manipulated objects. In some embodiments, the object tracking data may include image data (e.g., photographs, point clouds, live video feeds, etc.) of objects at one or more locations (e.g., designated lifting or lowering locations and / or conveyor belts). In some embodiments, the object tracking data may include the location and / or orientation of objects at one or more locations.

[0022] The communication device 206 may include circuitry configured to communicate with external or remote devices over a network. For example, the communication device 206 may include a receiver, transmitter, modulator / demodulator (modem), signal detector, signal encoder / decoder, connector port, network card, etc. The communication device 206 may be configured to transmit, receive, and / or process electrical signals according to one or more communication protocols (e.g., Internet Protocol (IP), wireless communication protocol, etc.). In some embodiments, the robot system 100 may use the communication device 206 to exchange information between units of the robot system 100 and / or with systems or devices outside the robot system 100 (e.g., for reporting, data collection, analysis, and / or troubleshooting purposes).

[0023] The input / output device 208 may include a user interface device configured to communicate information with and / or receive information from a human operator. For example, the input / output device 208 may include a display 210 and / or other output devices (e.g., a speaker, haptic circuit, or haptic feedback device) for conveying information to a human operator. The input / output device 208 may also include control or receiving devices such as a keyboard, mouse, touchscreen, microphone, user interface (UI) sensors (e.g., a camera for receiving motion commands), or wearable input devices. In some embodiments, the robot system 100 may use the input / output device 208 to interact with a human operator when performing actions, tasks, movements, or a combination thereof.

[0024] In some embodiments, the controller (e.g., controller 109 in Figure 1) may include a processor 202, a storage device 204, a communication device 206, and / or an input / output device 208. The controller may be a standalone component or part of a unit / assembly. For example, each unloading unit, transport assembly, conveying unit, and loading unit of system 100 may include one or more controllers. In some embodiments, a single controller may control multiple units or standalone components.

[0025] The robot system 100 may include physical or structural members (e.g., robot manipulator arms) connected by joints for motion (e.g., rotation and / or translational displacement). The structural members and joints may form a kinetic chain configured to operate end effectors (e.g., grippers) configured to perform one or more tasks (e.g., grasping, spinning, welding, etc.) according to the use / operation of the robot system 100. The robot system 100 may include actuation devices 212 (e.g., motors, actuators, wires, artificial muscles, electroactive polymers, etc.) configured to drive or operate (e.g., displace and / or reorient) the structural members around or at corresponding joints. In some embodiments, the robot system 100 may include transport motors 214 configured to transport corresponding units / chassis to various locations. For example, the actuation devices 212 and transport motors may be connected to or part of a robot arm, linear slide, or other robotic component.

[0026] Sensor 216 can be configured to acquire information used for performing tasks such as manipulating structural members and / or transporting robot units. Sensor 216 may include devices configured to detect or measure one or more physical properties of the robot system 100 (e.g., the condition, state, and / or location of one or more structural members / their joints) and / or one or more physical properties of the surrounding environment. Some embodiments of sensor 216 may include contact sensors, proximity sensors, accelerometers, gyroscopes, force sensors, strain gauges, torque sensors, position encoders, pressure sensors, vacuum sensors, and the like.

[0027] In some embodiments, for example, the sensor 216 may include one or more imaging devices 222 (e.g., two-dimensional and / or three-dimensional imaging devices) configured to detect the surrounding environment. The imaging devices may include cameras (including visual cameras and / or infrared cameras), LiDAR devices, radar devices, and / or other distance measuring or detection devices. The imaging devices 222 may generate representations of the detected environment, such as digital images and / or point clouds, which are used to implement machine / computer vision (e.g., for automated inspection, robot guidance, or other robotic applications).

[0028] Referring here to Figures 1 and 2, the robot system 100 can process image data and / or point clouds (e.g., via the processor 202) to identify the target package 112 in Figure 1, the starting location 114 in Figure 1, the task location 116 in Figure 1, the orientation of the target package 112 in Figure 1, or a combination thereof. The robot system 100 can use the image data to determine how to access and pick up the object. By analyzing images of the object, it can determine a pick-up plan for positioning the vacuum gripper assembly to grasp the target object, even if adjacent objects are close to the gripper assembly. Image output from onboard sensors 216 (e.g., LiDAR devices) and image data from remote devices (e.g., imaging system 160 in Figure 1) can be used individually or in combination. The robot system 100 can acquire and analyze images of a specified range (e.g., a pick-up location for an object in a truck, a container, or on a conveyor belt) (e.g., via various units) to identify the target package 112 and its starting location 114. Similarly, the robot system 100 can capture and analyze images of another designated area (for example, a descent location for placing an object on a conveyor belt, a location for placing an object inside a container, or a location on a pallet for stacking purposes) to identify the task location 116.

[0029] Furthermore, for example, the sensor 216 in Figure 2 may include a position sensor 224 (e.g., a position encoder, a potentiometer, etc.) configured to detect the position of a structural member (e.g., a robot arm and / or end effector) and / or a corresponding joint of the robot system 100. The robot system 100 can use the position sensor 224 to track the location and / or orientation of the structural member and / or joint during task execution. Unloading units, transfer units, transport units / assemblies, and loading units disclosed herein may include the sensor 216.

[0030] In some embodiments, the sensor 216 may include a contact sensor 226 (e.g., a force sensor, strain gauge, piezoresistive / piezoelectric sensor, capacitance sensor, piezoresistive and strain sensor, and / or other tactile sensor) configured to measure properties related to direct contact between multiple physical structures or surfaces. The contact sensor 226 may measure properties corresponding to the gripping of an end effector (e.g., a gripper) on the target package 112. Thus, the contact sensor 226 may output a contact measurement value representing a quantified measurement value (e.g., measured force, torque, position, etc.) corresponding to physical contact, the degree of contact or adhesion between the gripper and the target package 112, or other contact properties. For example, the contact measurement value may include one or more force, pressure, or torque measurements related to the force related to the gripping of the target package 112 by the end effector. In some embodiments, the contact measurement value may include both (1) pressure measurements related to vacuum gripping and (2) force measurements (e.g., moment measurements) related to the transported object(s). Further details regarding the contact measurement value are described below.

[0031] As will be described in more detail below, the robot system 100 can perform different actions to accomplish a task based on contact measurements, image data, or combinations thereof, etc., (e.g., via the processor 202). For example, the robot system 100 can re-grasp the target package 112 if the initial contact measurement falls below a threshold, such as when the vacuum grip is reduced (e.g., the suction level falls below the vacuum threshold), or a combination thereof. The robot system 100 can also intentionally lower the target package 112, adjust the task location 116, adjust the speed or acceleration of the movement, or adjust a combination thereof, based on one or more transport rules (e.g., when the contact measurement or suction level falls below a threshold during task execution), and contact measurements, image data, and / or other measurements or data.

[0032] Robot transfer assembly Figure 3 shows a transfer assembly 104 according to one or more embodiments of the present technology. The transfer assembly 104 may include an imaging system 160 and a robotic arm system 132. The imaging system 160 can provide image data acquired from the target environment using a depalletizing platform 110. The robotic arm system 132 may include a robotic arm assembly 139, as well as an end effector 140 including a vision sensor device 143 and a multi-gripper assembly 141 ("Gripper Assembly 141"). The robotic arm assembly 139 can position the end effector 140 over a group of objects in a stack 165 placed in a pick-up environment 163. The vision sensor device 143 can detect nearby objects without touching, moving, or removing objects in the stack 165.

[0033] The target object can be fixed to the bottom of the end effector 140. In some embodiments, the gripper assembly 141 may have addressable regions, each capable of selectively drawing in air to provide a vacuum grip. In some operating modes, only the addressable region adjacent to the target object(s) draws in air, providing a direct pressure difference between the vacuum gripper device and the target object(s). This allows only the selected package (i.e., the target package) to be pulled or otherwise fixed to the gripper assembly 141, even if other gripping portions of the gripper assembly 141 are adjacent to or in contact with other packages.

[0034] Figure 3 shows a gripper assembly 141 that carries a single object or package 112 ("package 112") positioned on a conveyor belt 120. The gripper assembly 141 can release the package 112 onto the conveyor belt 120, and the robotic arm system 132 can then acquire packages 112a and 112b by positioning the unloaded gripper assembly 141 directly above both packages 112a and 112b. The gripper assembly 141 can then hold both packages 112a and 112b via vacuum gripping, and the robotic arm system 132 can carry the held packages 112a and 112b to a position directly above the conveyor belt 120. The gripper assembly 141 can then release packages 112a and 112b onto the conveyor belt 120 (e.g., simultaneously or sequentially). This process can be repeated any number of times to transport objects from the stack 165 to the conveyor belt 120.

[0035] The visual sensor device 143 may include one or more optical sensors configured to detect packages held directly beneath the gripper assembly 141. The visual sensor device 143 may be positioned to the side of the gripper assembly 141 to avoid interference with package pick-up / lowering. In some embodiments, the visual sensor device 143 is movably coupled to an end effector 140 or robotic arm 139 so that it can move to different sides of the gripper assembly 141 to avoid collision with objects, while detecting the presence of one or more objects held by the gripper assembly 141 if objects are present. The position, number, and configuration of the visual sensor device 143 can be selected based on the configuration of the gripper assembly 141.

[0036] Referring again to Figure 3, the depalletizing platform 110 may include any platform, surface, and / or structure on which multiple objects or packages 112 (a single “package 112”) can be stacked and / or staged and prepared for transport. The imaging system 160 may include one or more imaging devices 161 configured to capture image data of the packages 112 on the depalletizing platform 110. The imaging devices 161 may capture distance data, position data, video, still images, Lida data, radar data, and / or motion in the pick-up environment or area 163. Note that while the terms “object” and “package” are used herein, these terms also include, but are not limited to, other items that can be grasped, lifted, transported, and delivered, such as “case,” “box,” “carton,” or any combination thereof. Furthermore, while polygonal boxes (e.g., rectangular boxes) are shown in the drawings disclosed herein, the shape of the boxes is not limited to such shapes and includes regular or irregular shapes that can be grasped, lifted, transported, and delivered, as will be discussed in detail below.

[0037] Similar to the depalletizing platform 110, the receiving conveyor 120 may include any platform, surface, and / or structure designated for receiving packages 112 for further tasks / operations. In some embodiments, the receiving conveyor 120 may include a conveyor system for transporting packages 112 from one location (e.g., a release point) to another location for further operations (e.g., sorting and / or storage).

[0038] Figure 4 is a front view of an end effector 140 coupled to a robot arm 139 according to some embodiments of the present technology. Figure 5 is a bottom view of the end effector 140 of Figure 4. The vision sensor device 143 may include one or more sensors 145 configured to detect a package and a calibration board 147 used to calibrate the position of the gripper assembly 141 relative to the vision sensor device 143, for example. In some embodiments, the calibration board 147 may be a placard having a pattern or design used to calibrate or define the position of the end effector 140 or the gripper assembly 141, the position of the robot arm 139, or a combination thereof in the operating environment. The gripper assembly 141 may include addressable vacuum sections or regions 117a, 117b, 117c (collectively, “vacuum regions 117”) that define the gripping section 125. Unless otherwise specified, the description of one vacuum region 117 applies to the other vacuum region 117. In some embodiments, each vacuum region 117 may be a suction channel bank containing components connected to a vacuum source outside the end effector 140. The vacuum region 117 may include a gripping interface 121 (illustrated in Figure 4) capable of holding an object.

[0039] Referring here to Figure 4, the vacuum region 117a can draw in air to hold the package 112, and can reduce or stop drawing in air to release the package 112. The vacuum regions 117b, 117c (shown not to hold the package) can draw in air independently (shown by arrows) and hold the package at their corresponding positions 113a, 113b (shown by dashed lines in Figure 4). Referring here to Figure 5, the vacuum region 117 can include a group or bank of suction elements 151 (shown in Figure 5) from which air is drawn in. The suction elements 151 can be arranged evenly / uniformly or unevenly spaced from one another and can be arranged in a desired pattern (e.g., irregular or regular pattern). The vacuum region 117 can have the same or different number, configuration, and / or pattern of suction elements 151. Air can be drawn in through each suction element 151 of the vacuum region 117 to carry a package that conforms to the shape of the vacuum region 117. To transport smaller packages, air can be drawn in through a subset of suction elements 151 that match the geometric shape of the package (e.g., suction elements 151 positioned within the package boundary or periphery). For example, air can be drawn in through a subset of suction elements in the vacuum region 117, such as a single suction element 151 that is directly adjacent to or covers the surface to be gripped. As shown in Figure 5, for example, a suction element 151 within the boundary 119 (shown by the dashed line) can be used to grip the corresponding circular surface of the package.

[0040] When all vacuum regions 117 are operating, the end effector 140 can provide a generally uniform gripping force along each of the gripping contact surfaces 121 or along the entire bottom surface 223. In some embodiments, the bottom surface 223 is a generally continuous and substantially uninterrupted surface, and the distance or pitch between the suction elements 151 of adjacent vacuum regions 117 can be smaller, equal to, or larger (e.g., twice, three times, four times, etc.) than the pitch between the suction elements 151 of the same vacuum region 117. The end effector 140 can be configured to hold or stick an object(s) via attractive force, achieved by forming and maintaining a vacuum between the vacuum regions 117 and the object. For example, the end effector 140 may include one or more vacuum regions 117 that are in contact with the surface of the object in question and configured to form / maintain a vacuum in the space between the vacuum regions 117 and the surface. The end effector 140 is lowered via the robot arm 139, thereby creating a vacuum when the vacuum region 117 is pressed against the surface of the object, pushing out or otherwise removing the gas between the opposing surfaces. When the robot arm 139 lifts the end effector 140, the pressure difference between the space inside the vacuum region 117 and the surrounding environment can keep the object attached to the vacuum region 117. In some embodiments, the airflow through the vacuum region 117 of the end effector 140 can be dynamically adjusted or adjusted based on the contact range between the object and the contact surface or gripping surface of the vacuum region 117 to achieve a grip sufficient to securely grasp the object. Similarly, the airflow through the vacuum region 117 can be dynamically adjusted to correspond to the weight of the object, such as increasing the airflow for heavier objects to achieve a grip sufficient to securely grasp the object. Embodiments of the suction element are described with reference to Figure 15.

[0041] Figure 6 is a functional block diagram of a transport assembly 104 according to one or more embodiments of the present technology. A processing unit 150 (PU) can control the movement and / or other actions of the robot arm system 132. The PU 150 can receive image data from sensors (e.g., sensor 161 of the imaging system 160 in Figure 3), sensor 145 of the vision sensor device 143, or other sensors or detectors that can collect image data including video, still images, LiDAR data, radar data, or a combination thereof. In some embodiments, the image data may show or represent a surface image (SI) of the package 112.

[0042] PU150 may include memory 152, a digital memory storage device, or any electronic data processing unit that executes software or computer instruction code that can be stored permanently or temporarily in a non-temporary computer-readable medium, including but not limited to random access memory (RAM), disk drives, magnetic memory, read-only memory (ROM), compact disks (CDs), solid-state memory, secure digital cards, and / or compact flash cards. PU150 may be driven by the execution of software or computer instruction code, including algorithms developed for specific functions embodied herein. In some embodiments, PU150 may be an application-specific integrated circuit (ASIC) customized for the embodiments disclosed herein. In some embodiments, PU150 may include one or more microprocessors, digital signal processors (DSPs), programmable logic devices (PLDs), programmable gate arrays (PGAs), and signal generators; however, in the embodiments herein, the term “processor” is not limited to such exemplary processing units, and its meaning is not intended to be narrowly interpreted. For example, PU150 may also include multiple electronic data processing units. In some embodiments, PU150 may be a processor(s) used by or in conjunction with any other system of the robotic system 100, including, but not limited to, the robotic arm system 130, the end effector 140, and / or the imaging system 160. PU150 in Figure 6 and processor 202 in Figure 2 may be the same or different components.

[0043] The PU150 may be electronically coupled to a system and / or source (e.g., via wired, bus, and / or wireless connections) to facilitate the reception of input data. In some embodiments, an operablely coupled PU150 may be considered interchangeable with an electronically coupled PU150. Direct connection is not required; instead, the reception of such input data and the provision of output data can be provided as signals received and / or transmitted by the PU150 via a bus, a wireless network, or a physical or virtual computer port. The PU150 may be programmed or configured to perform the methods discussed herein. In some embodiments, the PU150 may be programmed or configured to receive data from various systems and / or units, including, but not limited to, an imaging system 160 and an end effector 140. In some embodiments, the PU150 may be programmed or configured to provide output data to various systems and / or units.

[0044] The imaging system 160 may include one or more sensors 161 configured to capture image data representing a package (e.g., a package 112 placed on the depalletizing platform 110 in Figure 3). In some embodiments, the image data may represent a visual design and / or pattern appearing on one or more surfaces that can determine the registered state of the package. In some embodiments, the sensor 161 is a camera configured to operate within the electromagnetic spectral bandwidth of the subject (e.g., visible and / or infrared) and used to detect light / energy in the corresponding spectrum. In some camera embodiments, the image data is a set of data points forming a point cloud, depth map, or a combination thereof, captured from one or more three-dimensional (3D) cameras and / or one or more two-dimensional (2D) cameras. From these cameras, the distance or depth between the imaging system 160 and one or more exposed surfaces of the packaging 112 (e.g., relative to the field of view of the imaging system 160) can be determined. In some embodiments, the distance or depth can be determined by using image recognition algorithms, such as contextual image classification algorithms and / or edge detection algorithms. Once determined, the distance / depth value can be used to manipulate the package via a robotic arm system. For example, the PU150 and / or robotic arm system can use the distance / depth value to calculate the position in which the package can be lifted and / or grasped. It should be noted that the data described herein, such as image data, may include either discrete or continuous analog or digital signals that contain or can indicate information.

[0045] The imaging system 160 may include at least one display unit 164 configured to display operational information (e.g., status information, settings, etc.), images of package(s) 112 captured by sensor 162, or other information / output that can be viewed by one or more operators of the robot system 100, as will be discussed in detail below. Furthermore, the display unit 164 may be configured to display other information, including, but not limited to, target packages, non-target packages, registered packages, and / or symbols representing unregistered instances of packages.

[0046] The visual sensor device 143 can communicate with the PU 150 via a wired and / or wireless connection. The visual sensor 145 can be a video sensor, CCD sensor, lidar sensor, radar sensor, distance measuring device or detection device, etc. The output from the visual sensor device 143 can be used to generate a representation of a package(s) such as a digital image and / or point cloud, used to implement machine / computer vision (e.g., for automated inspection, robot guidance, or other robotic applications). The field of view (e.g., horizontal and / or vertical FOV of 30, 90, 120, 150, 180, 210, 270 degrees) and the range capability of the visual sensor device 143 can be selected based on the configuration of the gripper assembly 141. (Figure 4 shows an exemplary horizontal FOV of about 90 degrees.) In some embodiments, the visual sensor 145 is a lidar sensor having one or more light sources (e.g., lasers, infrared lasers, etc.) and an optical detector. The photodetector can detect light emitted from the light source and reflected off the surface of the package. The presence of a package and / or the distance to the package can be determined based on the detected light. In some embodiments, the sensor 145 can scan a range that is substantially the entire vacuum gripping section (e.g., vacuum gripping section 125 in Figure 4). For example, the sensor 154 may include one or more deflectors that move to deflect the emitted light across the detection section. In some embodiments, the sensor 154 is a scanning laser-based lidar sensor that can scan vertically and / or horizontally, such as 10° lidar scan, 30° lidar scan, 50° lidar scan, etc. The configuration, FOV, sensitivity, and output of the sensor 145 can be selected based on the desired detection capability. In some embodiments, the sensor 145 may include both a presence detector / distance detector (e.g., radar sensor, lidar sensor, etc.) and one or more cameras, such as a three-dimensional camera or a two-dimensional camera. The distance or depth between the sensor and one or more surfaces of the package can be determined, for example, using one or more image recognition algorithms.The display device 147 can be used to display image data, sensor status, execution of calibration routines, display logs and / or reports, or other information or data, such as symbols representing targeted, untargeted, registered, and / or unregistered instances of package 112, but not limited to these.

[0047] To control the robot system 100, the PU 150 can use the outputs from one or both of the sensors 145 and 161. In some embodiments, the image output from sensor 161 is used to determine an overall transport plan, including commands for transporting objects. The image outputs from sensor 145 and sensor 205 (e.g., a force detector assembly) can be used to position the multi-gripper assembly relative to an object, verify object pick-up, and monitor the transport steps.

[0048] Continuing to refer to Figure 6, the RDS170 may include any database and / or memory storage device (e.g., non-temporary computer-readable medium) configured to store registration records 172 for multiple packages 112 and data 173 for the vacuum gripper. For example, the RDS170 may include read-only memory (ROM), compact disk (CD), solid-state memory, secure digital card, compact flash card, and / or data storage server or remote storage device.

[0049] In some embodiments, each registration record 172 may include physical properties or attributes of the corresponding package 112. For example, each registration record 172 may include, but is not limited to, one or more template SIs, visual data (e.g., reference radar data, reference lidar data, etc.), 2D or 3D size measurements, weight, and / or center of gravity (CoM) information. The template SI may represent known or previously determined visible properties of the package, including the package design, pattern, appearance, shape / contour, or a combination thereof. The two- or three-dimensional size measurements may include the length, width, height, or a combination thereof of a known / expected package.

[0050] In some embodiments, the RDS170 can be configured to receive new instances of registration records 172 created according to embodiments disclosed below (e.g., for previously unknown packages and / or for previously unknown package configurations). Thus, the robot system 100 can automate the process of registering packages 112 by expanding the number of registration records 172 stored in the RDS170, thereby reducing the number of unregistered instances of package 112 and enabling more efficient depalletizing operations. By dynamically updating the registration records 172 in the RDS170 using live / operational data (e.g., during operation / deployment), the robot system 100 can efficiently implement a computer learning process that can take into account previously unknown or unexpected conditions (e.g., lighting conditions, unknown orientation, and / or stacking misalignment) and / or newly encountered packages. Thus, the robot system 100 can reduce failures resulting from “unknown” conditions / packages, associated human operator intervention, and / or failures of associated tasks (e.g., package loss and / or collision).

[0051] RDS170 may include vacuum gripper data 173 that includes characteristics or attributes, including, but not limited to, the number of addressable vacuum areas, the transport capacity of a vacuum gripper device (e.g., a multi-gripper assembly), a vacuum protocol (e.g., vacuum level, airflow rate, etc.), or other data used to control the robot arm system 130 and / or end effector 140. An operator can input information about a vacuum gripper installed in the robot arm system 130. RDS170 then identifies the vacuum gripper data 173 corresponding to the vacuum gripper device for operation. In some embodiments, a vacuum gripper device (e.g., a gripper assembly 141 in Figure 3) is automatically detected by the robot arm 139, and RDS170 is used to identify information about the detected vacuum gripper device. The identified information can be used to determine the configuration of the vacuum gripper device. Thus, different vacuum gripper devices or multi-gripper assemblies are installed and used with the robot arm system 130.

[0052] End effector Figure 7 is an isometric front and top view of a portion of the end effector 140 according to one or more embodiments of the present technology. Figure 8 is an isometric front and bottom view of the end effector 140 of Figure 7. Referring here to Figure 7, the end effector 140 may include a mounting interface or bracket 209 ("mounting bracket 209") and a force sensor assembly 205 coupled to the bracket 209 and the gripper assembly 141. A fluid line 207 may be fluidically coupled to a pressurizing device such as a vacuum source 221 (not shown in Figure 8) and the gripper assembly 141.

[0053] The field of view (FOV) (variable or fixed) of the vision sensor device 143 is generally directed downwards from the gripper assembly 141 to provide detection of any object being carried beneath the gripper assembly 141. The vision sensor device 143 can be positioned along the perimeter of the end effector 140 such that the vision sensor device 143 is substantially horizontal to one or more vacuum regions 117 (as shown), more specifically, below the gripping surface of the gripping interface 121 (as shown). The term "substantially horizontal" generally refers to an angle within approximately ±1 degree of the horizontal, such as an angle within approximately ±2 degrees of the horizontal, e.g., an angle within approximately ±0.7 degrees of the horizontal. Generally, the end effector 140 includes multiple vacuum regions 117, which enable the robot system 100 to grip objects that would ordinarily not be grippable by a single instance of a vacuum region 117. However, compared to the end effector 140 having a single instance of the vacuum region 117, the larger size of the end effector 140 means that a larger range is obscured from the detection sensor. One advantage is that the visual sensor device 143 positioned below the horizontal plane of the gripping interface 121 can provide an FOV including the gripping interface 121 during the initiation of contact with an object that would normally be obscured in other instances of the visual sensor device 143 that are not mounted on the end effector 140 or positioned in different locations within the operating environment of the robot system 100. Thus, an unobscured FOV can provide the robot system with real-time image sensor information that can enable real-time or immediate adjustments to the position and movement of the end effector 140 during the gripping operation. As a further advantage, the proximity between the visual sensor device 143 positioned below the horizontal plane of the gripping interface 121 and the object (e.g., non-objects 112a, 112b in Figure 3) improves accuracy and precision during the gripping operation and protects or prevents damage to the target object 112 and non-objects adjacent to the target objects 112,a,112b from the end effector 140, such as from crushing of the object.

[0054] For illustrative purposes, the visual sensor device 143 may be positioned at a corner of the end effector 140 along the effector width, but it will be understood that the visual sensor device 143 can be positioned at other locations as well. For example, the visual sensor device 143 may be positioned at the center of the width or length of the end effector 140. In another embodiment, the visual sensor device 143 may be positioned at another corner or other location along the length of the effector.

[0055] The vacuum source 221 (Figure 7) may include, but is not limited to, one or more pressurizing devices, pumps, valves, or other types of devices that can provide negative pressure, draw a vacuum (including a partial vacuum), or generate a pressure difference. In some embodiments, the air pressure may be controlled by one or more regulators, such as a regulator between the vacuum source 221 and the gripper assembly 141, or a regulator within the gripper assembly 141. When the vacuum source 221 draws a vacuum, air can be drawn into the bottom 224 of the gripper assembly 141 (indicated by arrows in Figure 8). The pressure level may be selected based on the size and weight of the object to be carried. If the vacuum level is too low, the gripper assembly 141 may not be able to pick up the object(s). If the vacuum is too high, the outside of the package may be damaged (for example, a package with an outer plastic bag may tear due to a high vacuum). According to some embodiments, the vacuum source 221 can provide vacuum levels such as approximately 100 millibars, approximately 500 millibars, approximately 1,000 millibars, approximately 2,000 millibars, approximately 4,000 millibars, approximately 6,000 millibars, and approximately 8,000 millibars. In alternative embodiments, higher or lower vacuum levels are provided. In some embodiments, the vacuum level can be selected based on a desired gripping force. The vacuum gripping force in each region 117 can be approximately 50N, 100N, 150N, 200N, or 300N or more at the vacuum level (e.g., the maximum vacuum level of the vacuum source 221 at 25%, 50%, or 75%). These gripping forces can be achieved when picking up cardboard boxes, plastic bags, or other suitable transport packages. Different vacuum levels can be used, including when transporting the same or different objects. For example, a relatively high vacuum can be provided to initially grip an object. When a package is gripped, the gripping force (and therefore the vacuum level) required to maintain the object can be reduced, thus providing a lower vacuum level. The gripping vacuum can be increased to maintain a secure grip when performing certain tasks.

[0056] The force detector assembly 205 may include one or more sensors 203 (one shown) configured to detect forces indicating the load being carried by the end effector 140. Detected measurements may include linear force measurements, moment measurements, pressure measurements, or combinations thereof along the axes and / or axes of a coordinate system. In some embodiments, the sensor 203 may be an FT sensor including a component having a 6-axis force sensor configured to detect up to 3-axis forces (e.g., forces detected along the x, y, and z axes of a Cartesian coordinate system) and / or 3-axis moments (e.g., moments detected about the x, y, and z axes of a Cartesian coordinate system). In some embodiments, the sensor 203 may include an internal amplifier and microcomputer for signal processing, the ability to perform static and dynamic measurements, and / or the ability to detect instantaneous changes based on sampling intervals. In some embodiments referencing a Cartesian coordinate system, force measurements(s) along one or more axes (i.e., F(x-axis), F(y-axis), and / or F(z-axis)) and / or moment measurements(s) along one or more axes (i.e., M(x-axis), M(y-axis), and / or M(z-axis)) may be acquired via sensor 203. By applying a CoM calculation algorithm, the package weight, package position, and / or number of packages can be determined. For example, the package weight may be calculated as a function of force measurements(s), and the package CoM may be calculated as a function of force measurements(s) and moment measurements(s). In some embodiments, the package weight is calculated as a function of force measurements(s), package position information from the visual sensor device 143, and / or gripping information (e.g., where sealing with the package(s) is achieved). In some embodiments, the sensor 203 can be coupled to a processing unit (e.g., PU150 in Figure 6) so as to be able to communicate via wired and / or wireless communication.

[0057] In some embodiments, output measurements from both the force detector assembly 205 and the visual sensor device 143 can be used. For example, the relative position of an object can be determined based on the output from the visual sensor device 143. Then, the output from the force detector assembly 205 can be used to determine information about each object, such as the weight / mass of each object. The force detector assembly 205 may include contact sensors, pressure sensors, force sensors, strain gauges, piezoresistive / piezoelectric sensors, capacitance sensors, torque sensors, linear force sensors, or other tactile sensors configured to measure properties related to direct contact between multiple physical structures or surfaces. For example, the force detector assembly 205 may measure properties corresponding to the gripping of an end effector on an object or measure the weight of the object. Thus, the force detector assembly 205 can output a contact measurement representing a quantified measurement, such as a measured force or torque, corresponding to the degree of contact or adhesion between the gripper and the object. For example, the contact measurement may include one or more force or torque measurements related to the force applied to the object by the end effector. Outputs from the force sensor assembly 205 or other sensors are integrated with or attached to the end effector 140. For example, sensor information from contact sensors, such as the weight or weight distribution of an object based on force torque sensor information, can be used by a robotic system to determine the uniqueness of an object, such as by an automatic registration or automated object registration system, in combination with image sensor information such as the dimensions of the object.

[0058] Figure 9 is an exploded isometric view of a gripper assembly 141 according to one or more embodiments of the present technology. The gripper assembly 141 includes a housing 260 and an internal assembly 263. The housing 260 can enclose and protect the internal components and can define an opening 270 configured to receive at least a portion of the force sensor assembly 205. The internal assembly 263 may include a gripper bracket assembly 261 ("bracket assembly 261"), a manifold assembly 262, and a plurality of grippers 264a, 264b, 264c (collectively, "grippers 264"). The bracket assembly 261 can hold each of the vacuum grippers 264, which can be fluidically coupled in series or in parallel to a fluid line (e.g., fluid line 207 in Figure 7) via the manifold assembly 262, as discussed in relation to Figures 10 and 11. In some embodiments, the bracket assembly 261 includes an elongated support 269 and a bracket 267 (shown) connecting the grippers 264 to the elongated support 269. The gripper assembly 141 may include suction elements, sealing members (e.g., sealing panels), and other components discussed in relation to Figures 13-15.

[0059] Figures 10 and 11 are rear, top, and top views, respectively, of components of a gripper assembly according to one or more embodiments of the present technology. The manifold assembly 262 may include gripper manifolds 274a, 274b, and 274c (collectively, “manifold 274”) coupled to the respective grippers 264a, 264b, and 264c. For example, manifold 274a controls the airflow associated with gripper 264a. In some embodiments, manifold 274 may be connected in parallel or in series to a pressurizing source, such as the vacuum source 221 in Figure 7. In other embodiments, each manifold 274 may be fluidly coupled to an individual pressurizing device.

[0060] The manifold 274 can be operated to distribute vacuum to one, some, or all of the grippers 264. For example, manifold 274a can be left open to allow air to flow through the bottom of the grippers 264a. The air flows through manifold 274a and exits the vacuum gripper assembly through lines such as line 207 in Figure 7. The other manifolds 274b, 274c can be left closed to prevent suction in manifolds 274b, 274c. Each manifold 274a may include one or more lines connected to each suction element, but is not limited. In other embodiments, the suction elements of the grippers 264a are connected to an internal vacuum chamber. The gripper manifold 274 may include one or more lines or passages, valves (e.g., check valves, globe valves, three-way valves, etc.), pneumatic cylinders, regulators, orifices, sensors, and / or other components capable of controlling the flow of fluid, but is not limited. Each manifold 274 can be used to distribute suction evenly or unevenly to a suction element or group of suction elements to generate a uniform or non-uniform vacuum gripping force. Electronic lines can be communicatively coupled to a controller to power and control the components of the module and its components. In one embodiment, individual manifolds 274 may include a common interface and plug for use with a common interface and plug, which allows for the quick and easy addition and removal of manifolds 274 and components, thereby facilitating the reconfiguration, maintenance, and / or repair of the system.

[0061] The number, arrangement, and configuration of the grippers can be selected based on a desired number of addressable vacuum areas. Figure 12 is an isometric view of the internal components of a vacuum gripper assembly 300 (housing not shown) suitable for use in the environments of Figures 1-2 and the transfer assembly 141 of Figures 3-6, according to one or more embodiments of the present technology. The vacuum gripper assembly 300 may include six vacuum grippers 302 (as shown) in a substantially rectangular arrangement. In other embodiments, the grippers may be in a circular arrangement, a square arrangement, or other suitable arrangement and may have similar or different configurations. The grippers may have other shapes, including but not limited to elliptical, non-polygonal, etc. The grippers may include suction elements (e.g., suction tubes, suction cups, sealing members, etc.), sealing members, valve plates, gripper mechanisms, and other fluid components for providing gripping functionality.

[0062] One or more sensors, vision sensor devices, and other components discussed in relation to Figures 1 to 11 can be incorporated into or used with the vacuum gripper assembly 300. Suction elements, sealing members, and other components are discussed in relation to Figures 13 to 15.

[0063] Vacuum grippers can be arranged in series. For example, vacuum grippers can be arranged side by side in an l×3 configuration, providing two lateral gripping positions and one central gripping position. However, it will be understood that end effectors can contain different numbers of vacuum grippers, suction channel banks, or vacuum areas in different configurations. For example, an end effector can contain four vacuum grippers or suction channel banks arranged in a 2×2 configuration. Vacuum areas can have the same or similar width dimensions as their length dimensions, so as to have a symmetrical square shape. In another embodiment, an end effector can contain different numbers of vacuum areas, such as two or more than three vacuum areas having the same or different length and / or width dimensions as one another. In yet another embodiment, vacuum grippers can be arranged in various configurations, such as a 2×2 configuration with four vacuum areas, a 1:2:2 configuration including five vacuum grippers, or other geometric arrangements and / or configurations.

[0064] Figure 13 shows a multi-gripper assembly 400 ("Gripper Assembly 400") suitable for use in a robotic system (e.g., robotic system 100 in Figures 1-2) according to several embodiments of the present technology. Figure 14 is an exploded view of the gripper assembly 400 of Figure 13. The gripper assembly 400 can be any gripper or gripper assembly configured to grip a package from a stationary position (e.g., a stationary position on a depalletizing platform such as platform 110 in Figure 3). The gripper assembly device 400 may include a gripper mechanism 410 and a contact or sealing member 412 ("Sealing Member 412"). The gripper mechanism 410 includes a body 414 and a plurality of suction elements 416 (shown in Figure 14), each configured to pass through an opening 418 (shown in Figure 14) of the member 412. When assembled, each of the suction elements 416 can extend partially or completely through the corresponding opening 418. For example, the suction element 416 can extend through the first side surface 419 toward the second side surface 421 of the sealing member 412.

[0065] Figure 15 is a partial cross-sectional view of the sealing member 412 and the suction element 416. The suction element 416 can be in fluid communication with a line (e.g., line 422 in Figure 14) via a vacuum chamber and / or internal conduit 430. A valve 437 (e.g., a check valve, safety valve, etc.) can be positioned along the air passage 436. A sensor 434 can be positioned to detect the vacuum level and can communicate with a controller (e.g., controller 109 in Figure 1) or a processing unit (e.g., processing unit 150 in Figure 6) via a wired or wireless connection. The lower end 440 of the suction element 416 may include, but is not limited to, a suction cup or another suitable feature for forming a desired seal with the surface of an object (e.g., a general hermetic seal or another suitable seal). When the lower end 440 is close to or in contact with an object, air can be drawn into the port / inlet 432 ("inlet 432") of the suction element 416 (as indicated by the arrows) and pull the object away from the sealing member 412. Air flows upward along the flow path 426 and through the passage 433 of the suction element 416. Air can flow into the conduit 430 through the valve 437. In some embodiments, the conduit 430 can be connected to a vacuum chamber 439. For example, some or all of the suction element 416 can be connected to the vacuum chamber 439. In other embodiments, different groups of the suction element 416 can be in fluid communication with different vacuum chambers. The suction element 416 may have a corrugated or bellows configuration to allow axial compression without constricting the airflow passage 433 within it, as shown in the figure. The configuration, height, and dimensions of the suction element 416 can be selected based on the desired amount of compression.

[0066] The sealing member 412 can be made of a compressible material configured to deform to accommodate surfaces of different geometric shapes, including a surface that is entirely or partially made highly curved. The sealing member 412 can be made of a foam that is entirely or partially closed-cell foam (e.g., foamed rubber). The material of the sealing member 412 can be porous to allow small amounts of airflow (i.e., air leakage) so as to avoid applying high negative pressure that could damage packaging such as a plastic bag.

[0067] Flow of operations Figure 16 is a flowchart of a method 490 for operating a robotic system according to one or more embodiments of the present disclosure. Generally, a transport robot can receive image data representing at least a portion of the picking environment. The robotic system can identify a target object based on the received image data. The robotic system can securely hold the identified target object(s) using a vacuum gripper assembly. Different units, assemblies, and subassemblies of the robotic system 100 in Figure 1 can perform method 490. Details of method 490 are discussed in detail below.

[0068] In block 500, the robot system 100 can receive image data representing at least a portion of the environment. For example, the received image data may represent at least a portion of the stack 165 in the pick-up environment 163 in Figure 3. The image data may include, but is not limited to, video, still images, LiDAR data, radar data, barcode data, or a combination thereof. In some embodiments, for example, the sensor 161 in Figure 3 may capture video or still images that are transmitted (for example, via a wired or wireless connection) to a computer or controller such as the controller 109 in Figures 1 and 6. In block 502, the computer 109 (Figure 1) can analyze image data to identify target objects, such as within a group of objects or in a stack of objects. For example, the controller 109 can identify individual objects based on the received image data and surface image / data stored by the RDS 170 (Figure 6). In some embodiments, information from the drop location is used to select target objects. For example, target objects can be selected based on the amount of available space at the drop location, a preferred stack arrangement, etc. The user can input selection criteria to determine the order in which objects are picked up. In some embodiments, a mapping of the pick-up environment (e.g., pick-up environment 163 in Figure 3) can be generated based on the received image data. In some mapping protocols, an edge detection algorithm is used to identify edges of objects, surfaces, etc. The mapping can be analyzed to determine which objects in the pick-up area can be transported together. In some embodiments, a group of objects that can be lifted and transported simultaneously by a vacuum gripper is identified as the target object.

[0069] The robot system 100 in Figure 1 can select a target package or object 112 from a source object as the target of a task to be performed. For example, the robot system 100 can select a target object to pick up according to a predetermined order, a set of rules, a template of object contours, or a combination thereof. In a specific embodiment, the robot system 100 can select a target package as an instance of a source package accessible to the end effector 140, such as an instance of a source package 112 placed on top of a stack of source packages, according to a point cloud / depth map representing the distance and position relative to a known location on an image device. In another specific embodiment, the robot system 100 can select a target object as an instance of a source package 112 located at a corner or edge and having two or more surfaces exposed to or accessible to the end effector 140. In a further specific embodiment, the robot system 100 can select a target object according to a predetermined pattern, such as from left to right with respect to a reference location, or from the nearest location to the furthest location, without interfering with or minimizing the displacement of other instances of the source package.

[0070] In block 504, the controller 109 can select a vacuum gripper or vacuum region for gripping the object. For example, the controller 109 (Figure 1) can select vacuum region 117a (Figure 4) for gripping the package 112 shown in Figure 3, since substantially the entire package 112 (i.e., the object) is directly below the vacuum region 117a. The vacuum is drawn through substantially all of the suction elements 151 in the vacuum region 117a in Figure 4 (e.g., at least 90%, 95%, and 98% of the suction elements 151).

[0071] In block 506, the controller 109 generates one or more commands to control the robot system 100. In some operating modes, the commands allow the robot system to suck air in an identified or selected addressable vacuum area. For example, the controller 109 can generate one or more lifting commands to provide vacuum at a selected vacuum level to a vacuum source (e.g., vacuum source 221 in Figure 7). The vacuum level can be selected based on the weight or mass of the object(s), the task to be performed, etc. A command can be sent to the gripper assembly 141 to operate the manifold 262 to provide suction in the selected area or gripper. The lifting and transfer process can be monitored using feedback from the visual sensor device 143 (Figure 7).

[0072] In block 508, the position of the end effector 140 relative to an object can be determined using the visual sensor device 143, including the source object or target object, such as the package 112 in Figure 1. The visual sensor device 143 can be used to continuously or periodically monitor the relative position of the end effector 140 to an object before and during object pick-up, during object transport, and / or during and after object descent. The output from the visual sensor device 143 can also be used to count objects (e.g., count the number of target or source objects) or to analyze objects, including analyzing the stacking of objects. The visual sensor device 143 can also be used to acquire environmental information used to navigate the robot system 100.

[0073] In block 510, the controller 109 generates commands to move the gripper assembly 141 to the actuators (e.g., actuator 212), motors, servos, actuators, and other components of the robot arm 139. Transport commands are generated by the robot system and can be used to move the robotic transport arm to carry the gripper assembly 141 to transport an object between locations. Transport commands can be generated based on a transport plan that includes a transport path to transport the object to its drop location without causing collisions with other objects. Collisions can be avoided using the vision sensor device 143 (Figure 7).

[0074] Method 490 can be performed to grasp multiple target objects. The end effector 140 can be configured to grasp multiple instances of a target package or target object from a source package or source object. For example, the robot system 100 can generate commands for the end effector 140 to engage with multiple instances of the vacuum area 117 to perform a grasping operation to grasp multiple instances of a target object simultaneously. In a specific embodiment, the end effector 140 can be used to execute commands for a grasping operation to grasp multiple instances of a target object one after another, separately and sequentially. For example, the command may include performing a grasping operation using one of the suction channel banks 117 to grasp a first instance of a target object 112 in one posture or orientation, and then, if necessary, repositioning the end effector 140 to engage with a second or different instance of the vacuum area 117 to grasp a second instance of the target object. In another specific embodiment, the end effector 140 can be used to execute a gripping command that simultaneously grips separate instances of the target object. For example, the end effector 140 can be positioned to simultaneously contact two or more instances of the target object and engage with each of the corresponding instances of the vacuum region 117 to perform a gripping operation on each of the multiple instances of the target object. In the above embodiment, each of the suction channel banks 117 can be operated independently as needed to perform different gripping operations.

[0075] Figure 17 is a flowchart of a method 700 for operating the robot system 100 of Figure 1 according to a basic plan, according to one or more embodiments of the present technology. Method 700 includes steps that can be incorporated into method 490 of Figure 16, and steps that can be carried out on the basis of executing instructions stored in one or more memory devices 204 of Figure 2 using one or more processors 202 of Figure 2 or the controller 109 of Figure 6. Data captured by the vision sensor device and sensor output can be used in various steps of method 700, as detailed below.

[0076] In block 702, the robot system 100 can examine (e.g., scan) one or more designated ranges, such as a pick-up range and / or descent range (e.g., a source descent range, a destination descent range, and / or a transport descent range). In some embodiments, the robot system 100 can generate imaging results for one or more designated ranges using one or more of the imaging device 222 in Figure 2, sensors 161 and / or 145 in Figure 6, or other sensors (e.g., via commands / prompts sent by the processor 202 in Figure 2). The imaging results may include, but are not limited to, captured digital images and / or point clouds, object position data, etc.

[0077] In block 704, the robot system 100 can identify the target package 112 and associated locations in Figure 1 (e.g., the starting location 114 and / or the task location 116 in Figure 1). In some embodiments, for example, the robot system 100 (e.g., via the processor 202) can analyze the imaging results according to a pattern recognition mechanism and / or a set of rules to identify the contours of an object (e.g., surrounding edges or surfaces). The robot system 100 can further identify groups of object contours (e.g., according to predetermined rules and / or pose templates) as corresponding to each unique instance of the object. For example, the robot system 100 can identify groups of object contours corresponding to patterns (e.g., the same value or varying by a known percentage / pattern) in color, brightness, depth / location, or a combination thereof across the object contour lines. Alternatively, for example, the robot system 100 can identify groups of object contours according to predetermined shape / pose templates defined in master data.

[0078] From the objects recognized at the pick-up location, the robot system 100 can select one as the target package 112 (for example, according to a predetermined order or set of rules and / or a template of object contours). For example, the robot system 100 can select the target package 112 as the object(s) to be placed on top, such as according to a point cloud representing the distance / position relative to a known location of the sensor. Alternatively, for example, the robot system 100 can select the target package 112 as the object(s) to be placed at a corner / edge, which may have two or more surfaces exposed / displayed in the imaging result. The target package can also be selected using available vacuum grippers and / or areas. Furthermore, the robot system 100 can select the target package 112 according to a predetermined pattern (for example, from left to right relative to a reference location, or from the nearest location to the furthest location).

[0079] In some embodiments, the end effector 140 can be configured to grasp multiple instances of the target package 112 from the source package. For example, the robot system 100 can generate commands for the end effector 140 to engage with multiple instances of the gripper region 117 to perform a gripping operation to simultaneously grasp multiple instances of the target package 112. In a specific embodiment, the end effector 140 can be used to execute commands for a gripping operation to grasp multiple instances of the target package 112 one after another, separately and sequentially. For example, the instruction may include performing a gripping operation using one of the gripper regions 117 to grasp a first instance of the target package 112 in one posture or orientation, and then, if necessary, repositioning the end effector 140 to engage with a second or different instance of the gripper region 117 to grasp a second instance of the target package 112. In another specific embodiment, the end effector 140 can be used to execute commands for a gripping operation to simultaneously grasp distinct instances of the target package 112. For example, the end effector 140 can be positioned to simultaneously contact two or more instances of the target package 112 and engage with each of the corresponding instances of the gripper region 117 to perform a gripping operation on each of the multiple instances of the target package 112. In the above embodiment, each of the gripper regions 117 can be operated independently as needed to perform different gripping operations.

[0080] For the selected target package 112, the robot system 100 can further process the imaging results to determine the starting location 114 and / or initial posture. For example, the robot system 100 can determine the initial posture of the target package 112 by selecting a template from a plurality of predetermined posture templates (e.g., different potential arrangements of the object's contour depending on the object's corresponding orientation) that corresponds to the measurement of the lowest difference when compared with a group of the object's contours. The robot system 100 can also determine the starting location 114 by converting the location of the target package 112 in the imaging results (e.g., a predetermined reference point of the determined posture) to a location within a grid used by the robot system 100. The robot system 100 can convert the location according to a predetermined calibration map.

[0081] In some embodiments, the robot system 100 can process the imaging results of the descent range to determine open spaces between objects. The robot system 100 can determine open spaces based on mapping object contours according to a predetermined calibration map that transforms image locations into real-world locations and / or coordinates used by the system. The robot system 100 can determine open spaces as the space between object contours (and thus object surfaces) belonging to different groups / objects. In some embodiments, the robot system 100 can determine suitable open spaces for the target package 112 based on measuring one or more dimensions of the open spaces and comparing the measured dimensions with one or more dimensions of the target package 112 (stored, for example, in master data). The robot system 100 can select one of the suitable spaces / open spaces as the task location 116 according to a predetermined pattern (e.g., from left to right relative to a reference location, from the nearest location to the furthest location, or from bottom to top).

[0082] In some embodiments, the robot system 100 can determine the task location 116 without processing the imaging results, or after further processing the imaging results. For example, the robot system 100 can place objects within the placement range according to a predetermined series of movements and locations without imaging the range. Furthermore, a sensor attached to the vacuum gripper assembly 141 (e.g., a vision sensor device 143) can output image data used to periodically image the range. The imaging results can be updated based on the additional image data. Also, for example, the robot system 100 can process the imaging results to perform multiple tasks (e.g., transporting multiple objects, such as objects placed on a common layer / stage of a stack).

[0083] In block 706, the robot system 100 can calculate a basic motion plan for the target package 112. For example, the robot system 100 can calculate a basic motion plan based on calculating a series of commands or settings, or a combination thereof, for the actuator device 212 in Figure 2 that operates the robot system 132 and / or end effector (e.g., the end effector 140 in Figures 3-5). For some tasks, the robot system 100 can calculate sequence and setting values ​​to operate the robot system 132 and / or end effector 140 to transport the target package 112 from the starting location 114 to the task location 116. The robot system 100 can implement a motion planning mechanism (e.g., a process, function, equation, algorithm, computer-generated / readable model, or a combination thereof) configured to calculate a path in space according to one or more constraints, goals, and / or rules. For example, the robot system 100 can use a predetermined algorithm and / or other grid-based search to calculate a path through space to move the target package 112 from the starting location 114 to the task location 116. The motion planning mechanism can use further processes, functions, or equations and / or transformation tables to translate the path into a series of commands or settings, or a combination thereof, for the actuator device 212. Using the motion planning mechanism, the robot system 100 can calculate the sequence in which to operate the robot arm 206 (Figure 3) and / or the end effector 140 (Figure 3) and make the target package 112 follow the calculated path. The visual sensor device 143 can be used to identify obstacles and recalculate the path to refine the basic plan.

[0084] In block 708, the robot system 100 can begin executing a basic motion plan. The robot system 100 can begin executing a basic motion plan based on operating the actuator device 212 according to a series of commands or settings or a combination thereof. The robot system 100 can execute a first set of motions in the basic motion plan. For example, the robot system 100 can operate the actuator device 212 to position the end effector 140 at a calculated location and / or orientation around a starting location 114 for grasping the target package 112, as shown in block 752.

[0085] In block 754, the robot system 100 can analyze the position of an object using sensor information acquired before and / or during the gripping operation (e.g., information from the visual sensor device 143, sensor 216, and force detector assembly 205), such as the weight of the target package 112, the center of gravity of the target package 112, the relative position of the target package 112 to the vacuum area, or a combination thereof. The robot system 100 can operate the actuator 212 and the vacuum source 221 (Figure 7) to engage and grip the target package 112 with the end effector 140. The position and number of target packages 112 can be analyzed using image data from the visual sensor device 143 and / or data from the force sensor assembly 205. In block 755, the position of the end effector 140 relative to the target package 112 or other objects can be confirmed using the visual sensor device 143. In some embodiments, as shown in block 756, the robot system 100 can perform an initial lift by moving the end effector upward by a predetermined distance. In some embodiments, the robot system 100 can reset or initialize an iteration counter "i" used to track several grasping operations.

[0086] In block 710, the robot system 100 can measure established gripping. The robot system 100 can measure established gripping based on measurements from other sensors such as the force detector assembly 205 in Figure 7, the vision sensor device 143, or the pressure sensor 434 (Figure 15). For example, the robot system 100 can determine gripping characteristics by using one or more of the force detector assemblies 205 in Figure 3 to measure force, torque, pressure, or a combination thereof at one or more locations on the robot arm 139, one or more locations on the end effector 140, or a combination thereof. In some embodiments, such as the gripping established by assembly 141, the contact or force measurements may correspond to the amount, location, or combination thereof of suction elements (e.g., suction element 416 in Figure 14) that make contact with the surface of the target package 112 and maintain a vacuum state within it. Additionally or alternatively, gripping characteristics can be determined based on the output from the vision sensor device 143. For example, image data from the sensor detector 143 can be used to determine whether an object is moving relative to the end effector 140 during transport.

[0087] In the decision block 712, the robot system 100 can compare the measured gripping to a threshold (e.g., an initial gripping threshold). For example, the robot system 100 can compare a measurement of contact or force to a predetermined threshold. The robot system 100 can also compare the image data from the detector 143 to reference image data (e.g., image data acquired at the initial object pick-up) to determine whether the gripped object has moved, for example, relative to each other or to the gripper assembly 141. Thus, the robot system 100 can determine whether the contact / gripping is sufficient to continue manipulating the target package(s) 112 (e.g., lifting, transporting, and / or reorienting).

[0088] If the measured gripping does not meet the threshold, the robot system 100 can evaluate whether the iteration count for regrasping the target package(s) 112 has reached the iteration threshold, as shown in the decision block 714. While the iteration count is below the iteration threshold, the robot system 100 may deviate from the basic motion plan if the contact or force measurements do not meet the threshold (e.g., fall below the threshold). Therefore, in block 720, the robot system 100 can operate the robot arm 139 and / or the end effector 140 to perform a regrasping operation not included in the basic motion plan. For example, the regrasping operation may include a predetermined set of commands or settings for the actuator 212, or a combination thereof, causing the robot arm 139 to lower the end effector 140 (e.g., reverse the initial lift) and / or cause the end effector 140 to release the target package(s) 112, and then regrasp the target package(s) 112. In some embodiments, a predetermined sequence allows the robot arm 139 to further operate to adjust the gripper position after releasing the object and before re-gripping the object or changing the range over which the vacuum is drawn. When performing a re-gripping operation, the robot system 100 may pause the execution of the basic motion plan. After performing a re-gripping operation, the robot system 100 may increment the iteration count.

[0089] After regrasping the object, the robot system 100 can measure the established grip for block 710 as described above and evaluate the established grip for block 712 as described above. The robot system 100 can attempt to regrasp the target package 112 as described above until the iteration count reaches the iteration threshold. When the iteration count reaches the iteration threshold, the robot system 100 can stop executing the basic motion plan as shown in block 716. In some embodiments, the robot system 100 can solicit operator input as shown in block 718. For example, the robot system 100 can generate operator notifications (e.g., a predetermined message) via the communication device 206 and / or the input / output device 208 in Figure 2. In some embodiments, the robot system 100 can cancel or delete the basic motion plan, record a predetermined state (e.g., an error code) for the corresponding task, or perform a combination thereof. In some embodiments, the robot system 100 can resume the process by imaging the pick-up / task area (block 702) and / or identifying another item within the pick-up area as the target object (block 704), as described above.

[0090] As shown in block 722, when the measured grip (e.g., the grip measured for each held package) meets the threshold, the robot system 100 can continue executing the rest of the basic motion plan / movement. Similarly, after re-gripping the target package 112, when the contact measurement meets the threshold, the robot system 100 can resume execution of the paused basic motion plan. Thus, the robot system 100 can continue executing the sequence of movements in the basic motion plan (i.e., following gripping and / or initial lifting) by operating the actuator device 212 and / or transport motor 214 in Figure 2 according to the remaining series of commands and / or settings. For example, the robot system 100 can transport (e.g., vertically and / or horizontally) and / or reorient the target package 112 according to the basic motion plan.

[0091] While executing the basic motion plan, the robot system 100 can track the current location and / or orientation of the target package 112. The robot system 100 can position one or more parts of the robot arm and / or end effector by tracking the current location according to the output from the position sensor 224 in Figure 2. In some embodiments, the robot system 100 can track the current location by processing the output of the position sensor 224 in a computer-generated model, process, equation, position map, or a combination thereof. Thus, the robot system 100 can calculate and track the current location 424 by further mapping the position to a grid by combining the position or orientation of the joints and structural members. In some embodiments, the robot system 100 can include multiple beacon sources. The robot system 100 can measure beacon signals at one or more locations of the robot arm and / or end effector and use the measured values ​​(e.g., signal strength, timestamp, or propagation delay, and / or phase shift) to calculate the separation distance between the signal source and the measured location. The robot system 100 can map separation distances to known locations of the signal source and calculate the current location of the signal reception location as the location where the mapped separation distances overlap.

[0092] In decision block 724, the robot system 100 can determine whether the basic plan has been executed completely to completion. For example, the robot system 100 can determine whether all actions (e.g., commands and / or settings) in the basic motion plan 422 have been completed. The robot system 100 can also determine that the basic motion plan has been completed when its current location matches the task location 116. Once the robot system 100 has finished executing the basic plan, the robot system 100 can restart the process by imaging the pick-up / task range (block 702) and / or identifying another item within the pick-up range as the target object (block 704), as described above.

[0093] If not completed, in block 726, the robot system 100 can measure gripping (i.e., by determining contact / force measurements) during the transfer of the target package 112. In other words, the robot system 100 can determine contact / force measurements while executing the basic motion plan. In some embodiments, the robot system 100 can determine contact / force measurements according to a sampling frequency or at a predetermined time. In some embodiments, the robot system 100 can determine contact / force measurements before and / or after executing a predetermined number of commands or settings on the actuator device 212. For example, the robot system 100 can sample the contact sensor 226 after or during a particular category of operation, such as lifting or rotating. Alternatively, for example, the robot system 100 can sample the contact sensor 226 when the direction and / or magnitude of the accelerometer output matches or exceeds a predetermined threshold representing a sudden or rapid movement. The robot system 100 can determine contact / force measurements using one or more of the processes described above (for example, in block 710).

[0094] In some embodiments, the robot system 100 can determine the orientation of the gripper and / or target package 112 and adjust the contact measurement accordingly. The robot system 100 can adjust the contact measurement based on orientation to take into account the directional relationship between the sensing direction of the contact sensor and the gravity acting on the target object according to the orientation direction. For example, the robot system 100 can calculate an angle between the sensing direction and a reference direction (e.g., "downward" or the direction of gravity) according to the orientation. The robot system 100 can decrease or increase the contact / force measurement according to a coefficient and / or sign corresponding to the calculated angle.

[0095] In the decision block 728, the robot system 100 can compare the measured grip to a threshold (e.g., a transfer grip threshold). In some embodiments, the transfer grip threshold can be less than or equal to an initial grip threshold related to the initial (e.g., pre-transfer) grip evaluation of the target package 112. Thus, the robot system 100 can implement stricter rules for evaluating the grip before initiating the transfer of the target package 112. The grip threshold requirement can be higher initially, since sufficient contact to pick up the target package 112 is likely sufficient to transfer the target package 112.

[0096] If the measured grasp meets a threshold (e.g., above the threshold) and the correct package is grasped (e.g., determined based on image data from the visual sensor device 143), the robot system 100 can continue executing the basic plan as shown in block 722 and described above. If the measured grasp does not meet a threshold (e.g., below the threshold) or the correct package is not grasped, the robot system 100 can deviate from the basic motion plan and perform one or more response actions as shown in block 530. If the measured grasp is insufficient in light of the threshold, the robot system 100 can operate the robot arm 139, the end effector, or a combination thereof according to commands and / or settings not included in the basic motion plan. In some embodiments, the robot system 100 can execute different commands and / or settings based on its current location.

[0097] For illustrative purposes, the response action is described using a controlled descent. However, it will be understood that the robot system 100 can perform other actions, such as by stopping the execution of the basic motion plan as shown in block 716, and / or by soliciting operator input as shown in block 718.

[0098] A controlled descent includes one or more actions to position the target package 112 in one of the descent ranges (for example, instead of task location 116) in a controlled manner (i.e., based on lowering and / or releasing the target package 112, and not as a result of a complete grip failure). When performing a controlled descent, the robot system 100 can dynamically (i.e., in real time and / or while executing a basic motion plan) calculate different locations, operations or paths, and / or commands or settings for actuator devices according to the current location. In some embodiments, the end effector 140 can be configured for grip-release operations of multiple instances of the target package 112. For example, in some embodiments, the end effector 140 can be configured to perform grip-release operations simultaneously or sequentially by selectively releasing the vacuum area 117 as needed to release each instance of the target package 112 as appropriate. The robot system 100 can choose whether to release the objects simultaneously or sequentially, and the order or release based on the position of the held objects, the placement of the objects in the descent range, etc.

[0099] In block 762, the robot system 100 can calculate a adjusted descent location and / or associated orientation for positioning the target package 112. When calculating the adjusted descent location, the robot system 100 can identify the descent range (e.g., source descent range, destination descent range, or transport descent range) closest to and / or ahead of the current location (e.g., between the current location and the task location). Also, if the current location is between descent ranges (i.e., not within a descent range), the robot system 100 can calculate the distance to a descent range (e.g., the distance to a representative reference location within the descent range). Thus, the robot system 100 can identify the descent range closest to and / or ahead of the current location. Based on the identified descent range, the robot system 100 can calculate a location within it as the adjusted descent location. In some embodiments, the robot system 100 can calculate the adjusted descent location based on selecting a location according to a predetermined order (e.g., left to right, bottom to top, and / or front to back relative to a reference location).

[0100] In some embodiments, the robot system 100 can calculate the distance from its current location to an open space within its descent range (for example, as identified by block 704 and / or tracked according to the arrangement of the moving object). The robot system 100 can select an open space ahead of its current location and / or the open space closest to its current location 424 as the adjusted descent location.

[0101] In some embodiments, before selecting a descent range and / or open space, the robot system 100 can use a predetermined process and / or equation to convert contact / force measurements into maximum transport distances. For example, the predetermined process and / or equation can estimate the corresponding maximum transport distance and / or time to complete gripping failure based on various values ​​of the contact measurements. Thus, the robot system 100 can exclude available descent ranges and / or open spaces that are farther than the maximum transport distance from the current location. In some embodiments, when the robot system 100 cannot identify an available descent range and / or open space (e.g., when the accessible descent range is full), the robot system 100 can stop executing the basic motion plan as shown in block 716 and / or prompt for operator input as shown in block 718.

[0102] In block 766, the robot system 100 can calculate a modified motion plan for transporting the target package 112 from its current location to a modified descent location. The robot system 100 can calculate a modified motion plan in a similar manner to that described above for block 506.

[0103] In block 768, the robot system 100 can execute a modified motion plan in addition to and / or instead of a basic motion plan. For example, the robot system 100 can operate the actuator device 212 according to a series of commands or settings or a combination thereof, thereby manipulating the robot arm 139 and / or end effector to move the target package 112 along a path.

[0104] In some embodiments, the robot system 100 can pause the execution of the basic motion plan and the execution of the adjusted motion plan. Once the target package 112 is positioned at the adjusted descent location based on the execution of the adjusted motion plan (i.e., once the controlled descent is complete), in some embodiments, the robot system 100 can attempt to regrasp the target package 112 as described above for block 720, and then measure the established grip as described above for block 710. In some embodiments, the robot system 100 can attempt to regrasp the target package 112 up to the iteration limit, as described above. If the contact measurement meets the initial grip threshold, the robot system 100 can reverse the adjusted motion plan (e.g., return to the paused point / location) and continue the execution of the rest of the paused basic motion plan. In some embodiments, the robot system 100 can update and recalculate the adjusted motion plan from the current location 424 (after regrasp) to the task location 116 and execute the adjusted motion plan to complete the task.

[0105] In some embodiments, the robot system 100 can update the range log of the accessed descent range (e.g., a record of open space and / or placed objects) to reflect the placed target package 112. For example, the robot system 100 can regenerate the imaging results of the corresponding descent range. In some embodiments, the robot system 100 can perform a controlled descent and, after placing the target package 112 in the adjusted descent location, cancel the remaining actions of the basic motion plan. In one or more embodiments, the transport descent range may include a pallet or container placed on top of one of the transport units 106 in Figure 1. At a specified time (e.g., when the pallet / container is full and / or when an incoming pallet / container is delayed), the corresponding transport unit can move from the descent range to the pick-up range. Thus, the robot system 100 can re-implement method 500, thereby re-identifying the lowered items as target packages 112 and transporting them to the corresponding task location 116.

[0106] Once the target package 112 is positioned at the adjusted descent location, the robot system 100 can repeat method 700 for a new target object. For example, the robot system 100 can determine the next object within its pick-up range as the target package 112 and calculate a new basic motion plan to transport the new target object.

[0107] In some embodiments, the robot system 100 may include a feedback mechanism that updates the path planning mechanism based on the contact measurement 312. For example, since the robot system 100 performs an action to re-grip the target package 112 at an adjusted position (e.g., as described above for block 720), the robot system 100 can store the position of the end effector that produced a contact / force measurement that satisfies a threshold (e.g., as described above for block 712). The robot system 100 can store the position relative to the target package 112. When the number of gripping failures and / or successful re-grip actions reaches a threshold, the robot system 100 can analyze the stored positions to grip the target package 112 (e.g., using a running window for analyzing a recent series of actions). When a predetermined number of re-grip actions have occurred for a particular object, the robot system 100 can update the motion planning mechanism to position the gripper at a new position relative to the target package 112 (e.g., the position corresponding to the highest number of successes).

[0108] Based on the actions represented by block 710 and / or block 726, the robot system 100 can track the progress of the execution of the basic motion plan (e.g., via the processor 202). In some embodiments, the robot system 100 can track progress according to the horizontal transfer of the target package(s) 112. The robot system 100 can track progress based on established grip measurements before the start of the horizontal transfer (block 710) and based on grip measurements during the transfer after the start of the horizontal transfer (block 726). Thus, based on the progress as described above, the robot system 100 can selectively generate a new set of actuator commands, actuator settings, or combinations thereof (i.e., different from the basic motion plan).

[0109] In other embodiments, for example, the robot system 100 can track progress based on tracking commands, settings, or combinations thereof communicated and / or executed by the actuator device 212. Based on the progress, the robot system 100 can selectively generate new sets of actuator commands, actuator settings, or combinations thereof to perform re-grasp response actions and / or controlled descent response actions. For example, when the progress is before any horizontal transfer of the target package 112, the robot system 100 can select an initial grip threshold and perform the actions represented in block 712 and later (e.g., via a function call or jump instruction). Also, when the progress is after a horizontal transfer of the target package 112, the robot system 100 can select a transfer grip threshold and perform the actions represented in block 728 and later (e.g., via a function call or jump instruction).

[0110] By performing fine-grained control / operation of the target package 112 via imaging data from the visual sensor device 143 (i.e., selecting whether to execute or deviate from a basic motion plan) according to contact / force measurements and visual monitoring, improved efficiency, speed, and accuracy for transporting objects are provided. For example, if the target package 112 is re-gripped when the contact measurement falls below the initial gripping threshold or when the package 112 is improperly positioned, the likelihood of gripping failure during transport is reduced, and the number of objects lost or unintentionally dropped during transport is decreased. The vacuum area and vacuum level can be adjusted to maintain the desired grip, further improving the handling of the package 112. In addition, each lost object requires human interaction to correct the result (e.g., moving the lost object away from the motion path of a subsequent task, inspecting the lost object for damage, and / or completing the task for the lost object). Thus, reducing the number of lost objects reduces the human effort required to perform the task and / or the overall operation.

[0111] Figures 18–21 illustrate the steps of a robot grasping and transporting an object according to one or more embodiments of the present disclosure, in accordance with method 490 of Figure 16 or method 700 of Figure 17. Figure 18 shows a gripper assembly 141 positioned on top of a stack of objects. A robotic arm 139 can position the gripper assembly 141 directly above the object to be transported. As discussed in block 704 of Figure 17, the controller can analyze image data from a visual sensor device 143 to identify, for example, the object to be transported 812a, 812b. Based on the collected image data, a plan (e.g., a pick-up or basic plan) can be generated. The plan can be generated based on (a) the carrying capacity of the gripper assembly 141 and / or (b) the configuration of the object to be transported.

[0112] Figure 19 shows the underside of the gripper assembly 141 covering the target objects 812a, 812b and a larger non-target object 818. The output from the visual sensor device 143 can be analyzed to determine the position of the gripper assembly 141 relative to the target objects. Based on the positions of objects 812a, 812b, the vacuum areas 117a, 117b are identified for drawing a vacuum. In some embodiments, measurements from the force sensor 203 are used to confirm that the gripper assembly 141 made contact with the top surface of the stack 814 before and / or after gripping the target objects 812a, 812b.

[0113] Figure 20 shows air drawn into vacuum regions 117a and 117b to hold the object 812a and 812b against the gripper assembly 141 without creating a vacuum (or effective vacuum) in other vacuum regions 117c, as indicated by the arrows. The vacuum level can be increased or decreased to increase or decrease the compression of the compliant panel(s) 412(s) (as shown). Vacuum gripping can be evaluated as discussed in relation to block 710 in Figure 17.

[0114] Figure 21 shows a raised gripper assembly 141 that firmly holds the target objects 812a and 812b. The position of the target objects 812a and 812b can be monitored using a visual sensor device 143. Additionally or alternatively, information about the load, such as the position and weight of the target objects 812a and 812b, can be determined using a force detector assembly 205. Vacuum areas 117a and 117b can continue to draw in air to firmly hold the target objects 812a and 812b. Vacuum gripping during transport can be monitored as discussed in block 726 of Figure 17. The applied vacuum can be stopped or reduced to release the objects 812a and 812b. This process can be repeated to transport each object in the stack.

[0115] conclusion The above detailed description of embodiments of the disclosed technology is not intended to be exhaustive or to limit the disclosed technology to the exact forms disclosed above. While specific embodiments of the disclosed technology are described above for illustrative purposes, various equivalent modifications are possible within the scope of the disclosed technology, as will be recognized by those skilled in the art. For example, while processes or blocks are presented in a given order, alternative embodiments may perform routines having steps, or use systems having blocks in a different order, and some processes or blocks may be deleted, moved, added, subdivided, combined, and / or modified to provide alternative or partial combinations. Each of these processes or blocks may be carried out in various ways. Also, while processes or blocks are sometimes shown to be executed in series, these processes or blocks may instead be executed or carried out in parallel, or at different times. Furthermore, any particular numbers described herein are merely examples, and different values ​​or ranges may be used in alternative embodiments.

[0116] These and other modifications can be made to the disclosed technology in light of the detailed description above. While the detailed description describes specific embodiments and possible best modes of the disclosed technology, the disclosed technology can be implemented in many ways, regardless of how detailed the above description may be in the text. System details may differ considerably in specific embodiments, while they are encompassed by the technology disclosed herein. As stated above, any specific terms used when describing specific features or aspects of the disclosed technology should not be construed as meaning that the terms are redefined herein to be limited to specific characteristics, features, or aspects of the disclosed technology to which they relate. Thus, the present invention is not limited except by the appended claims. In general, terms used in the following claims should not be construed as limiting the disclosed technology to specific embodiments disclosed herein unless such terms are explicitly defined in the detailed description section above.

[0117] Certain aspects of the present invention are presented below in the form of specific claims, but the applicant may consider various aspects of the present invention in any number of claim forms. Accordingly, the applicant reserves the right to pursue additional claims after filing the present application in order to pursue such additional claim forms in either the present application or a continuation application.

Claims

1. Selecting at least one of a plurality of addressable vacuum regions of a gripper assembly of a transport robot based on one or more target objects within a group of objects, wherein the gripper assembly is a multi-gripper assembly having the plurality of addressable vacuum regions, each independently controllable. Generate commands and / or settings for the transport robot, Bringing at least one of the selected plurality of addressable vacuum regions into contact with the one or more target objects, To draw air through at least one of the selected plurality of addressable vacuum regions, With respect to the grip established between the one or more target objects and at least one of the selected plurality of addressable vacuum regions, the characteristics including one or more forces, pressures, or torques related to the grip of the target objects by the vacuum region are measured. If the measured characteristics do not meet the threshold, release the one or more target objects and adjust the position of the gripper assembly to bring at least one of the selected plurality of addressable vacuum regions into contact with the one or more target objects. The robot moves the multi-gripper assembly to separate one or more gripped objects from other objects in the group and transport them, A method that includes this.

2. The multi-gripper assembly is A gripper mechanism comprising at least one of a plurality of addressable vacuum regions, wherein each of the addressable vacuum regions of the gripper mechanism comprises a plurality of suction elements, A contact member including a plurality of openings corresponding to the plurality of suction elements, The method according to claim 1, wherein generating the command and / or setting to bring the selected plurality of addressable vacuum regions into contact with the one or more target objects includes operating the transport robot to directly press at least a portion of the contact member onto the one or more target objects that overlap with at least one of the addressable vacuum regions.

3. Each of the plurality of suction elements extends through the corresponding opening on the upper surface of the contact member and without extending beyond the bottom surface of the contact member. The method of claim 2, wherein generating the command and / or setting to bring at least one of the selected plurality of addressable vacuum regions into contact with the one or more target objects includes operating the transport robot to compress the contact member so that the suction element in at least one of the addressable vacuum regions engages with the one or more target objects.

4. One or more of the aforementioned suction elements are in fluid communication with a vacuum line via a vacuum chamber and / or internal conduit. The method according to claim 2, wherein generating the command and / or setting to suction air includes operating the transport robot to suction air through one or more of the suction elements via the vacuum line.

5. The method according to claim 2, wherein generating the command and / or setting to draw in air corresponds to operating the transport robot to pull the target object against the contact member.

6. The contact member consists of one or more compressible materials configured to deform to correspond to the surfaces of objects with different geometric shapes. The method according to claim 2, wherein generating the command and / or setting to bring at least one of the selected plurality of addressable vacuum regions into contact with the one or more target objects includes operating the transport robot to compress the contact member against the one or more target objects.

7. The multi-gripper assembly further includes at least one manifold, each manifold including one or more lines connected to the suction element of the associated gripper mechanism. The method according to claim 2, wherein generating the command and / or setting to suction air includes operating the transport robot to suction air through one or more lines.

8. The method according to claim 7, wherein each of the at least one manifold distributes suction uniformly or non-uniformly through the suction element of the associated gripper mechanism to generate a uniform or non-uniform vacuum gripping force.

Images