Robotic picking apparatus and methods for using same

Abstract

A picking robot includes an end effector comprising a plurality of fingers and one or more squeeze pads that together combine a pinching mechanism and a soft-gripping mechanism to facilitate object picking. Each squeeze pad includes a membrane fluidically coupled to a pump and filled with granular media. An object is picked by actuating the fingers to bring the squeeze pads into physical contact with the object, thus generating a holding force via pinching. When the pressure within the squeeze pad is similar to ambient pressure, the squeeze pad is in a fluid-like state. Thus, the squeeze pad is readily able to conform in shape to the surface of an object. After physical contact with the object is made, the pressure within the squeeze pad is decreased below ambient pressure, thus causing the squeeze pad to transition to a solid-like state and provide a holding force to carry the object.

Description

ROBOTIC PICKING APPARATUS AND METHODS FOR USING SAMECROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims the priority benefit, under 35 U.S.C. 119(e), of U.S. Application No. 63 / 631,848, filed April 9, 2024 and entitled, “ROBOTIC PICKING APPARATUS AND METHODS FOR USING SAME,” which is incorporated herein by reference in its entirety.BACKGROUND

[0002] A robotic picking apparatus (also referred to as a “picking robot”) is a machine configured to grasp and manipulate objects. Picking robots are commonly used in industry especially in “pick-and-place” applications where objects are moved from one location (e.g., a container) to another location (e.g., another container). For example, picking robots are frequently used in automated order fulfillment centers to transfer an item from a stock container containing multiples of that item to an order container containing items for a customer.

[0003] Robotic picking is typically accomplished in one of three ways: 1) a finger pinching mechanism; 2) a suction mechanism; or 3) a soft-gripping mechanism.

[0004] The finger pinching mechanism typically includes an end effector with two or more fingers configured to clamp an object. Each finger may include multiple segments that are each coupled together via a rotary joint to provide greater range of motion to grasp objects of varying size and shape. The segments are often formed from a hard, rigid material.

[0005] The suction mechanism typically includes an end effector with a suction cup coupled to a vacuum pump. During operation, the suction cup physically contacts an object and forms a sealed cavity with the object. The vacuum pump thereafter provides vacuum suction to generate a holding force between the suction cup and the object sufficient to support the object even when solely carried by the suction cup.

[0006] The soft-gripping mechanism (also sometimes referred to as a “universal gripper”) uses granular jamming (also sometimes referred to as “particle jamming”) to grasp an object. This mechanism typically includes an end effector with a flexible membrane filled with fine granular particles (also collectively referred to herein as “granular media”) and a pump connected to the membrane to move air in and out of the membrane. When the membrane is filled with air (e.g., at ambient pressure), the granular particles can move around freely within the membrane and thus effectively behave as a fluid. This allows the membrane to conform in shape to the surface of an object when pressed against the object. When air is removed from the membrane by thepump, the resulting vacuum condition causes the granular particles to press up against one another (i.e., jam together) such that they interlock and / or are prevented from moving relative to each other due to friction. As a result, the granular particles effectively behave as a hard, rigid solid material. When this transition occurs while the membrane is pressed against a surface of an object, the membrane may retain the shape of that surface and thus provide a holding force sufficient to carry the object.SUMMARY

[0007] The Inventors have recognized and appreciated that picking robots can provide a way to automate picking operations on a wide range of objects. However, the Inventors have also recognized a picking robot capable of picking all or nearly all objects has remained elusive. This is because conventional picking mechanisms, such as a finger pinching mechanism, a suction mechanism, or a soft-gripping mechanism, are generally not well suited at picking certain objects. Herein, the objects under consideration may vary by size, shape, weight, texture, and / or compliance.

[0008] A finger pinching mechanism is typically assembled from hard, rigid components, which may limit the contact area between the segments of the fingers and the object, especially if the object has an irregular shape. Thus, an end effector with a finger pinching mechanism may apply a relatively larger force to the object to ensure a sufficient holding force is provided to support the object. However, the larger force may damage the object being picked. Fragile objects, such as eggs, certain fruits and vegetables susceptible to bruising, and the like, are especially susceptible to damage by a finger pinching mechanism.

[0009] A suction mechanism generally relies on a sealed cavity being formed between a suction cup and an object to achieve a sufficient holding force to carry the object. Thus, the ability of an end effector with a suction cup to carry an object depends, in part, on the permeability of that object to a gas (e.g., air). As a result, the suction mechanism is not suitable at picking objects that are more permeable to air, such as textiles (e.g., a woven fabric), or highly porous objects (e.g., a sponge).

[0010] A soft-gripping mechanism typically includes an end effector with a single membrane shaped as a bladder and filled with granular media. The membrane is dimensioned to be relatively large compared to the objects being picked to provide sufficient volume for the granular media to redistribute as the membrane is deformed to conform to the object. This approach typically works well when picking a single, isolated object. However, in manysettings, objects are seldom isolated. Instead, objects are typically grouped together in containers or bins. Thus, it is often challenging for an end effector with a soft-gripping mechanism to precisely pick one object from a group of objects piled together using a relatively large-sized membrane. Moreover, conventional soft-gripping mechanisms provide the greatest holding force when the membrane is able to deform around and interlock with a portion of an object. As a result, conventional soft-gripping mechanisms are often not suitable at picking objects with large flat surfaces (e.g., a piece of drywall) or objects that require extensive deformation of the membrane to interlock with the object (e.g., a bottle).10011 J The foregoing limitations illustrate that no conventional picking mechanism has been developed that can rival the human hand in its ability to safely and reliability grasp and manipulate virtually any object within given a size and weight limit. Thus, certain applications may require multiple picking robots equipped with different picking mechanisms to ensure at least one picking robot is able to pick a particular object. In some cases, a human operator may even be required to pick an object when a picking robot is unable to do so. This limitation is especially apparent in “pick-and-place” applications that involve handling objects with appreciably different physical characteristics, such as in automated order fulfillment applications involving perishable goods.

[0012] In view of the foregoing limitations of conventional picking mechanisms, the present disclosure is directed to various inventive implementations of a robotic picking apparatus (also referred to herein as a “picking robot”) with an end effector that combines together a finger pinching mechanism and a soft-gripping mechanism. The end effector may include, for example, at least two fingers where each finger includes one or more segments joined successively together by rotary joints. The end effector may further include one or more squeeze pads that are each disposed on one or multiple segments. Each squeeze pad (also referred to herein as a “pad”) may include a membrane shaped as a bladder and filled with granular media. The membrane may be further coupled to a pump. Thus, each squeeze pad provides a soft-gripping mechanism.

[0013] In one aspect, the combination of a finger pinching mechanism and a soft-gripping mechanism in a single end effector may compensate for the limitations associated with each picking mechanism when used in isolation. For example, the amount of force applied to an object by the segments may be appreciably reduced compared to conventional finger pinching mechanisms. Additionally, object slippage from the end effector may be appreciably mitigated by the soft-gripping mechanism providing a larger contact area with an object. This isfacilitated, in part, by the squeeze pads readily conforming to the surface of an object when not subjected to vacuum conditions. In another example, the inclusion of multiple squeeze pads disposed at different locations along each finger may allow the membrane in each squeeze pad to undergo less deformation when grasping an object compared to conventional soft-gripping mechanisms that have a single membrane. In some implementations, it may not be necessary for each squeeze pad to deform and interlock with a portion of the object. Rather, the combination of multiple squeeze pads in contact with different portions of the object may effectively provide the same holding force as a single membrane that is extensively deformed and interlocked with an object.

[0014] In another aspect, the squeeze pads may be comparable to or smaller than the size of the object being picked and, thus, smaller than conventional soft-gripping mechanisms with single membranes. The relatively smaller-sized squeeze pads may appreciably increase the precision with which the end effector is able to pick an object. For example, the squeeze pads near the tip of each finger may be sufficiently narrow in width so that the end effector may readily grasp a single object from a pile of objects. In some implementations, the squeeze pads may not extend to the tip of each finger. That way, the segment forming the tip of the finger may be used to manipulate objects in the same manner as a fingernail on a human finger.

[0015] In one example implementation, a picking robot configured to pick an object includes: (a) an end effector, including (a-1) a first picking mechanism including a plurality of fingers configured to apply a first holding force to the object via pinching and at least one rotary joint coupled to the plurality of segments where the plurality of fingers includes a first finger and each finger of the plurality of fingers includes a plurality of segments, and (a-2) a second picking mechanism including a plurality of pads configured to apply a second holding force to the object where the plurality of pads includes at least two pads directly coupled to the first finger; (b) a pump fluidically coupled to each pad of the plurality of pads; and (c) a controller communicatively coupled to the end effector and the pump, the controller being configured to controllably adjust the first holding force by actuating the plurality of fingers and controllably adjust the second holding force by moving air into or out of the plurality of pads using the pump, wherein when air is moved out of each pad of the plurality of pads, a pressure within the pad decreases below an ambient pressure of an ambient environment surrounding the picking robot thereby increasing the second holding force and the end effector is configured to use the first holding force and the second holding force to pick the object.

[0016] Each finger of the plurality of fingers may be identical to every other finger of the plurality of fingers. For at least one finger of the plurality of fingers, the plurality of segments may include three or more segments. For each finger of the plurality of fingers, the plurality of segments may include three or more segments. For each finger of the plurality of fingers, the plurality of segments may consist of two segments. For each finger of the plurality of fingers, the plurality of pads may include at least one pad disposed on each segment of the plurality of segments. For at least one finger of the plurality of fingers, the plurality of pads may include at least one pad spanning two or more segments of the plurality of segments. The plurality of fingers may include a second finger and the plurality of pads may include at least one pad disposed on the second finger such that the at least one pad does not extend to a tip of the second finger. For each finger of the plurality of fingers, the finger may include a first side and a second side opposite the first side, when the picking robot is picking the object, the first side may be oriented oriented to face towards the object and the second side may be oriented to face away from the object, and each pad of the plurality of pads disposed on the finger may be disposed on the first side. The end effector may further include at least one sensor disposed on each pad of the plurality of pads configured to detect physical contact between the pad and the object. The controller may be a closed loop controller.

[0017] Each pad of the plurality of pads may be configured to transition between a fluid-like state and a solid-like state via granular jamming and the controller may be configured to controllably transition each pad of the plurality of pads between the fluid-like state and the solid-like state by using the pump to move air into or out of the pad. Each pad of the plurality of pads may include a membrane, fluidically coupled to the pump, defining a cavity and granular media disposed in the cavity. The granular media may include elastically deformable particles having a Young's modulus ranging from about 106Pa to about 109Pa. The granular media may include at least one of expanded polystyrene beads, Styrofoam, rubber, or polyurethane. The granular media may include at least one of ground coffee, steel beads, glass beads, rice, salt, or sugar.

[0018] In another example implementation, a picking robot configured to pick an object includes: (a) an end effector, including (a-1) a plurality of fingers including a first finger where each finger of the plurality of fingers includes a plurality of segments and at least one joint coupled to the plurality of segments, and (a-2) a plurality of pads directly coupled to the plurality of fingers where the plurality of pads includes at least two pads directly coupled to the first finger of the plurality of fingers; and (b) a pump fluidically coupled to the plurality ofpads, wherein for each pad of the plurality of pads, the pump is configured to withdraw air from a portion of the pad so as to generate a pressure difference between the portion of the pad and an ambient environment surrounding the picking robot, the pressure difference generating a holding force to pick the object.[0019| Each finger of the plurality of fingers may be identical to every other finger of the plurality of fingers. For at least one finger of the plurality of fingers, the plurality of segments may include three or more segments. For each finger of the plurality of fingers, the plurality of pads may include at least one pad disposed on each segment of the plurality of segments. For at least one finger of the plurality of fingers, the plurality of pads includes at least one pad spanning two or more segments of the plurality of segments. The plurality of fingers may include a second finger and the plurality of pads includes at least one pad disposed on the second finger such that the at least one pad does not extend to a tip of the second finger.

[0020] In one example implementation, a picking robot configured to pick an object includes: (a) an end effector, including (a-1) a first picking mechanism including a plurality of fingers configured to apply a first holding force to the object via pinching and at least one rotary joint coupled to the plurality of segments where the plurality of fingers includes a first finger and each finger of the plurality of fingers includes a plurality of segments, and (a-2) a second picking mechanism including a plurality of pads configured to apply a second holding force to the object where the plurality of pads includes at least one pad directly coupled to the first finger such that the at least one pad does not extend to a tip of the first finger; and (c) a controller communicatively coupled to the end effector and the pump, the controller being configured to controllably adjust the first holding force by actuating the plurality of fingers and controllably adjust the second holding force by moving air into or out of the plurality of pads using the pump, wherein when air is moved out of each pad of the plurality of pads, a pressure within the pad decreases below an ambient pressure of an ambient environment surrounding the picking robot thereby increasing the second holding force and the end effector is configured to use the first holding force and the second holding force to pick the object.

[0021] For at least one finger of the plurality of fingers, the plurality of segments may include three or more segments. For each finger of the plurality of fingers, the plurality of pads may include at least one pad disposed on each segment of the plurality of segments. For at least one finger of the plurality of fingers, the plurality of pads may include at least one pad spanning two or more segments of the plurality of segments.|0022| In another example implementation, a picking robot includes: (a) an end effector, including a plurality of fingers where each finger of the plurality of fingers includes one or more segments, at least one rotary joint coupled to the one or more segments, and a plurality of squeeze pads coupled to at least one of the one or more segments or the at least one rotary joint where the plurality of squeeze pads are configured to transition between a fluid-like state and a solid-like state via granular jamming; (b) a pump coupled to the end effector; and (c) a controller, communicatively coupled to the end effector and the pump, to move the plurality of fingers by actuating the at least one rotary joint and to controllably transition the plurality of squeeze pads between the fluid-like state and the solid-like state by using the pump to move air into or out of the plurality of squeeze pads.

[0023] Each squeeze pad of the plurality of squeeze pads may include a membrane, fluidically coupled to the pump, defining a cavity and granular media disposed in the cavity. The granular media may include elastically deformable particles having a Young's modulus ranging from about 106Pa to about 109Pa. The granular media may include at least one of expanded polystyrene beads, Styrofoam, rubber, or polyurethane. The granular media may include at least one of ground coffee, steel beads, glass beads, rice, salt, or sugar.

[0024] It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein. It should also be appreciated that terminology explicitly employed herein that also may appear in any disclosure incorporated by reference should be accorded a meaning most consistent with the particular concepts disclosed herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The skilled artisan will understand that the drawings primarily are for illustrative purposes and are not intended to limit the scope of the inventive subject matter described herein. The drawings are not necessarily to scale; in some instances, various aspects of the inventive subject matter disclosed herein may be shown exaggerated or enlarged in the drawings to facilitate an understanding of different features. In the drawings, like reference characters generally refer to like features (e.g., functionally similar and / or structurally similar elements).

[0026] FIG. 1 A shows an example robotic arm with an end effector that combines a pinching finger mechanism and a soft gripping mechanism configured for granular jamming.

[0027] FIG. IB shows an inset view of the end effector of FIG. 1 A. The end effector is shown proximate to, but not in physical contact with a bottle.

[0028] FIG. 1C shows an inset view of the squeeze pad of FIG. IB. The squeeze pad is shown undeformed.

[0029] FIG. 2A shows an inset view of the end effector of FIG. IB actuated such that the squeeze pads are in physical contact with the bottle.

[0030] FIG. 2B shows an inset view of the squeeze pad of FIG. 2A in physical contact with a portion of the bottle.

[0031] FIG. 3 A shows an inset view of the end effector of FIG. 2A with a vacuum applied to each squeeze pad to induce granular jamming.

[0032] FIG. 3B shows an inset view of the squeeze pad of FIG. 3A where the vacuum is sufficient to cause the deformation in the granular media.

[0033] FIG. 4A shows another example robotic arm with an end effector that combines a pinching finger and a squeeze pad configured for granular jamming.

[0034] FIG. 4B shows an inset view of the end effector of FIG. 4A.

[0035] FIG. 4C shows an inset view of the squeeze pad of FIG. 4B. The squeeze pad is shown undeformed.

[0036] FIG. 5A shows an inset view of the end effector of FIG. 4B actuated such that the squeeze pads are in physical contact with a block.

[0037] FIG. 5B shows an inset view of the squeeze pad of FIG. 5 A in physical contact with a portion of the block.

[0038] FIG. 6A shows an inset view of the end effector of FIG. 5 A with a vacuum applied to each squeeze pad to induce granular jamming.

[0039] FIG. 6B shows an inset view of the squeeze pad of FIG. 6A where the vacuum is sufficient to cause the deformation in the granular media.DETAILED DESCRIPTION

[0040] Following below are more detailed descriptions of various concepts related to, and implementations of, a picking robot with one or more end effectors that combine together a finger pinching mechanism and a soft gripping mechanism as well as methods for using the picking robot. It should be appreciated that various concepts introduced above and discussed in greater detail below may be implemented in multiple ways. Examples of specific implementations and applications are provided primarily for illustrative purposes so as to enable those skilled in the art to practice the implementations and alternatives apparent to those skilled in the art.[00411 The figures and example implementations described below are not meant to limit the scope of the present implementations to a single embodiment. Other implementations are possible by way of interchange of some or all of the described or illustrated elements. Moreover, where certain elements of the disclosed example implementations may be partially or fully implemented using known components, in some instances only those portions of such known components that are necessary for an understanding of the present implementations are described, and detailed descriptions of other portions of such known components are omitted so as not to obscure the present implementations.

[0042] In the discussion below, various examples of a picking robot with an end effector are provided, wherein a given example or set of examples showcases one or more features of a robotic arm, a pump, a closed loop controller, a joint, a finger of an end effector, and a squeeze pad of an end effector. It should be appreciated that one or more features discussed in connection with a given example of a picking robot may be employed in other examples of picking robots according to the present disclosure, such that the various features disclosed herein may be readily combined in a given picking robot according to the present disclosure (provided that respective features are not mutually inconsistent).

[0043] Certain dimensions and features of the picking robot and its components and / or subsystems are described herein using the terms “approximately,” “about,” “substantially,” and / or “similar.” As used herein, the terms “approximately,” “about,” “substantially,” and / or “similar” indicates that each of the described dimensions or features is not a strict boundary or parameter and does not exclude functionally similar variations therefrom. Unless context or the description indicates otherwise, the use of the terms “approximately,” “about,” “substantially,” and / or “similar” in connection with a numerical parameter indicates that the numericalparameter includes variations that, using mathematical and industrial principles accepted in the art (e.g., rounding, measurement or other systematic errors, manufacturing tolerances, etc.), would not vary the least significant digit.

[0044] 1. A First Example of a Picking Robot

[0045] FIGS. 1A-1C show an example picking robot 100a picking an object 90 disposed in a bin 91. As shown, the picking robot 100a includes a robotic arm 101 with an end effector 110a. The robotic arm 101 may generally include a plurality of arms where each arm is connected to a joint (e.g., a rotary joint). In this example, the robotic arm 102 includes arms 102a and 102b connected together via joints 103a, 103b, and 103c. The arms 102a and 102b are referred to herein individually as the arm 102. The joints 103a-103c are referred to herein individually as the joint 103. Each joint 103 may be a motorized joint (e.g., the joint 103 includes an electric motor) or a cable driven joint (e.g., the joint 103 is actuated by winding or releasing a cable). The arms 102 and the joints 103 may provide multiple degrees of freedom to position and / or orient the end effector 110a. In some implementations, the robotic arm 101 may provide six degrees of freedom (e.g., three translational degrees of freedom and three rotational degrees of freedom). However, it should be appreciated that, in some implementations, the robotic arm 101 may provide fewer degrees of freedom or more degrees of freedom (e.g., one degree of freedom to provide coarse movement along one axis and another degree of freedom to provide fine movement along that same axis). The robotic arm 101 may further provide an air line 108 to connect the end effector 110a to a pump 105 and an electrical cable 109 to electrically couple the end effector 110a to a controller 106. The pump 105 may move air into or out of the squeeze pads 140, as described below. The electrical cable 109 may supply electrical power as well as communication to the end effector 110a.

[0046] In the example shown in FIG. IB, the end effector 110a may include two fingers 120a and 120b to provide a finger pinching mechanism, and each finger 120 may include two segments 122a and 122b connected to each other via a joint 124b (e.g., a rotary joint). The segment 122a may be connected to the base 112 via a joint 124a (e.g., a rotary joint). The fingers 120a and 120b are referred to herein individually as the finger 120. The segments 122a and 122b are referred to herein individually as the segment 122. The joints 124a and 124b are referred to herein individually as the joint 124. It should be appreciated that, in some implementations, the end effector 110a may include three fingers, four fingers, five fingers, and so on. Furthermore, each finger 120 may include a single segment, two segments, three segments (see, for example, the end effector 110b in FIG. 4B), and so on. More generally, theend effectors disclosed herein may include one or more fingers 120 connected to the base 112. Each finger 120 may include one or more segments 122 connected to one another or the base 112 via respective joints 124. Each joint 124 may be a motorized joint or a cable driven joint.

[0047] The end effector 110a may further include squeeze pads 140a and 140b mounted to each finger 120 to provide a soft-gripping mechanism. The squeeze pads 140a and 140b are referred to herein individually as the squeeze pad 140. In some implementations, the squeeze pad 140 may be securely coupled to the segment 122 using an adhesive (e.g., an epoxy). In this manner, the end effector 110a may provide a combined picking mechanism that combines both a finger pinching mechanism and a soft-gripping mechanism.

[0048] Generally, the end effectors disclosed herein may include one or more fingers 120 that each include one or more squeeze pads 140 mounted thereto. The end effector 110a may include a plurality of squeeze pads 140 placed onto one or more fingers 120 so that the squeeze pads 140 may contact different sides of an object 90. In some implementations, each segment 122 of each finger 120 may support a squeeze pad 140 as shown in FIG. IB. In some implementations, only a subset of the segments 122 in each finger 120 may support a squeeze pad 140. For example, a squeeze pad 140 may only be mounted to one segment 122 of each finger 120. In some implementations, each finger 120 may support two or more squeeze pads 140. In some implementations, the squeeze pad 140 may span multiple segments 122 (see, for example, the squeeze pad 140b in FIG. 4C). In some implementations, the finger 120 may include at least one squeeze pad 140 where the squeeze pad 140 does not extend to the tip of the finger 120. Said another way, the squeeze pad 140 may be offset from the tip of the finger 120. For example, the squeeze pad 140 may be offset from the end of the segment 122 furthest from the base 112. That way, the segment 122 having the tip may be used to manipulate objects to facilitate picking in a manner as a fingernail. For instance, the tip may rotate an object 90 or isolate the object 90 from other objects in the environment (e.g., by moving the other objects away) before picking.

[0049] The squeeze pads 140 may generally have various shapes and / or dimensions. For example, when the squeeze pad 140 is undeformed, the squeeze pad 140 may have a cross- sectional shape that includes, but is not limited to, a circle, an oval, a polygon (e.g., as square, a pentagon), and any combinations of the foregoing. In some implementations, the squeeze pad 140 may span only a portion of the length of the segment 122 to which it is mounted to. In some implementations, the squeeze pad 140 may span the entire length of the segment 122. In some implementations, the squeeze pad 140 may span multiple segments 122. In someimplementations, the squeeze pad 140 may be dimensioned to be comparable to or smaller than the object 90 being picked. By limiting the size of the squeeze pads 140 in this manner, the end effector 110a may be more readily able to pick a single object from a pile or cluster of objects closely grouped together.

[0050] As shown in FIG. IB, the squeeze pads 140a and 140b may be connected to the pump 105 via respective air lines 126a and 126b. The air lines 126a and 126b, in turn, may be fluidically coupled to the air line 108 disposed along the robotic arm 101. The air lines 126a and 126b are referred to herein individually as the air line 126. During operation, the pump 105 may move air into the squeeze pad 140 (e.g., to transition to a fluid-like state) or move air out of the squeeze pad 140 (e.g., to transition to a solid-like state). In some implementations, each squeeze pad 140 may be independently controllable. For example, if the end effector 110a relies on a subset of the squeeze pads 140 to grasp an object 90 during operation, only those squeeze pads 140 may be subjected to vacuum conditions to hold the object 90 while the remaining squeeze pads 140 may remain undeformed. In another example, different vacuum levels may be applied to different squeeze pads 140 to adjust the holding force applied by each squeeze pad 140 to the object 90.[005.1] Additionally, each squeeze pad 140 may include one or more sensors 150. In one example, a contact sensor 150 may be disposed on at least a portion of the squeeze pad 140 (e.g., the surface of the squeeze pad 140) to detect physical contact between an object 90 and the squeeze pad 140 and / or the contact force applied to that object 90 when the end effector 110a is grasping the object 90. In some implementations, the controller 106 may operate as a closed loop controller 106 that utilizes sensory data from the sensors 150 as feedback. For example, the controller 106 may automatically withdraw air from the squeeze pad 140 when the contact force with an object 90 exceeds a predetermined threshold to facilitate picking of the object 90. This may occur, for example, when the finger 120 supporting the squeeze pad 140 is actuated to move the squeeze pad 140 into physical contact with the object 90. In another example, the controller 106 may automatically move air into the squeeze pad 140 to facilitate release of the object 90 when the finger 120 is actuated in manner to release the object 90. In some implementations, the picking robot 100a may be communicatively coupled with a vision system, described below, that provides additional sensory feedback (e.g., imagery of end effector 110a with respect to an object 90) to facilitate closed loop operation of the fingers 120 and the squeeze pad 140 when picking and releasing the object 90.[0052| The mechanical properties of each squeeze pad 140 may be altered using a granular jamming mechanism. For example, FIG. 1C shows each squeeze pad 140 may include a membrane 142 shaped as a bladder defining a cavity 143. As shown, the cavity 143 contains granular media 144. When the cavity 143 is at ambient pressure, the volume of the cavity 143 may be sufficiently large such that the granular media 144 only occupies a portion of the volume. The granular particles of the granular media 144 may readily move around and redistribute themselves in the cavity 143, which allows the squeeze pad 140 to behave like a fluid (i.e., the squeeze pad 140 is in a fluid-like state). Moreover, the membrane 142 may be formed of a deformable, elastic material. FIGS. 2A and 2B show that the fingers 120a and 120b may be actuated such that the squeeze pads 140 physically contact an object 90 (e.g., a bottle). When this occurs, the membrane 142 may readily conform to the shape of the object 90 with the granular media 144 redistributing itself within the cavity 143 of the membrane 142.

[0053] When the pressure within the cavity 143 is reduced to a pressure lower than ambient pressure, i.e., atmospheric pressure, (e.g., by drawing air out of the cavity 143 via the air line 126 using the vacuum pump 105), the membrane 142 may physically contract due to the pressure difference between the cavity 143 and the ambient environment surrounding the membrane 142. This, in turn, forces the granular particles to press up against one another and jam due, for example, the static frictional forces that arise between adjacent granular particles. Under these conditions, the granular particles may no longer be free to move within the cavity of the membrane 142 or relative to each other. As a result, the granular media 144 may behave as a hard, rigid solid material that retains the shape it had before vacuum conditions were applied (i.e., the squeeze pad 140 is in a solid-like state). For example, FIGS. 3A and 3B show that the squeeze pad 140 may retain its deformed shape after contact with the object 90. The granular particles are jammed, the end effector 110a may thereafter pick and manipulate the object 90. As shown in FIG. 3B, the air line 126 connected to the squeeze pad 140 may include a filter 146 to prevent the granular media 144 from being drawn out of the cavity 143 when vacuum conditions are applied to the squeeze pad 140.

[0054] The squeeze pads 140 may thus provide a way to appreciably increase the contact area between the end effector 110a and the object 90 and / or to facilitate interlocking with the object 90. This, in turn, allows the fingers 120 to apply an appreciably smaller pinching force when grasping the object 90, thus reducing the risk of the object 90 being damaged by the fingers 120 while maintaining a sufficient holding force to carry the object 90. Additionally, the squeeze pads 140 may be appreciably reduced in size compared to conventional soft grippingmechanisms. Moreover, it may not be necessary for the squeeze pads 140 to undergo excessive deformation to grasp an object 90 since multiple squeeze pads 140 may be disposed on opposing sides of the object 90.

[0055] The membrane 142 may generally be formed from an elastic material. Various elastic materials may be used including, but not limited to, latex and silicone. The granular media 144 may be formed from various particulate materials. In some implementations, the granular media 144 may include relatively hard, rigid granular particles including, but not limited to, ground coffee, steel beads, glass beads, rice, salt, sugar, and the like. For example, the granular media 144 may have a Young’s modulus ranging from about 109Pa to about 1012Pa, including all sub-ranges and values in between. In some implementations, the granular media 144 may include relatively soft, elastically deformable granular particles including, but not limited to, expanded polystyrene beads, Styrofoam, rubber, polyurethane, and the like. For example, the granular media 144 may have a Young’s modulus ranging from about 106Pa to about 109Pa, including all sub-ranges and values in between. More generally, the granular media 144 may be formed from granular particles having a Young’s modulus ranging from about 106Pa to about 1012Pa, including all sub-ranges and values in between.

[0056] The rigidity of the granular particles may affect the variability in stiffness provided by the squeeze pad 140 as well as the time required to transition between the fluid-like and solidlike states as the vacuum level is varied. For example, relatively hard, rigid granular particles may provide a faster, more abrupt transition between the fluid-like and solid-like states of the squeeze pad 140. In another example, relatively soft, elastically deformable granular particles may provide a slower, more gradual transition between the fluid-like and solid-like states of the squeeze pad 140. Depending on the objects being picked, the end effector 110a may utilize only relatively hard, rigid granular particles, only soft, elastically deformable granular particles, or a combination of both.

[0057] In some implementations, the picking robot 100a may include a vision system (not shown) to identify objects for the end effector 110a to pick. The vision system may include, for example, one or more cameras (e.g., a stereoscopic camera, a LiDAR camera) to acquire data (e.g., imagery and / or depth information) on one or more objects. A controller may process the data and determine A) which object to pick and B) the best approach or trajectory for the end effector 110a to take to pick that object (e.g., to avoid picking or touching other objects, to reduce the time to pick an object). In some implementations, the vision system may be based, in part, on previous demonstrations of vision systems as disclosed in, for example, Breyer etal., “Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Clutter,” Proceedings of the 2020 Conference on Robot Learning, PMLR 155: 1602-1611, 2021, which is incorporated herein in its entirety.10058] 2. A Second Example of a Picking Robot

[0059] FIGS. 4A-4C show another example picking robot 100b with an end effector 110b. The picking robot 100b and the end effector 110b may include one or more of the same features as the picking robot 100a and the end effector 110a, respectively, as described in Section 1. For brevity, repeated discussion of these features is not provided below.

[0060] In this example, the end effector 110b may include a pair of fingers 120a and 120b that each have three segments (e.g., segments 122a, 122b, and 122c) joined together and to the base 112 via joints 124a, 124b, and 124c. Additionally, FIG. 4B shows that for each finger 120, a squeeze pad 140a may be disposed entirely on one segment (e.g., the segment 122a) and a squeeze pad 140b may be disposed on the remaining two segments 122b and 122c by covering a joint 124c. For some objects 90, it may be more natural for two segments 122 to bend around a corner of the object 90. Thus, the squeeze pad 140b may be better situated to deform and wrap around the corner of the object 90 compared to squeeze pads 140 disposed only on a single segment 122.[006.1] FIGS. 5A and 5B show the end effector 110b grasping an object 90 (e.g., a square block) while the squeeze pads 140 are in the fluid-like state. FIGS. 6A and 6B show the squeeze pads 140 of the end effector 110b in the solid-like state after vacuum conditions are applied.

[0062] 3. Example Applications

[0063] The picking robots disclosed herein may generally incorporate end effectors that combine a finger pinching mechanism and a soft gripping mechanism. In some implementations, the combination of these two mechanisms may allow the end effector to pick nearly all objects in a reliable and repeatable manner. Accordingly, the picking robots disclosed herein may be well suited for a variety of pick-and-place applications. Following below are several non-limiting example applications in which the picking robots disclosed herein may be used.

[0064] In one example, the picking robot may be used in automated order fulfillment systems to pick non-perishable goods and / or perishable goods (e.g., fruits, vegetables) typically found in a supermarket as part of a customer order.

[0065] In another example, the picking robot may be used in retail stores to pick and place items (e.g., packaged goods, clothing) for replenishment on shelves or in inventory.10066] In yet another example, the picking robot may be used for crop harvesting. For instance, the picking robots may be deployed in a farm, an orchard, a vineyard, a greenhouse, a forest to pick nuts or fruits, such as grapes, apples, oranges, strawberries, and the like.

[0067] In yet another example, the picking robot may be used for food processing. For instance, the picking robot may pick fruits or vegetables for processing and packaging at a food processing plant.

[0068] In yet another example, the picking robot may be used in horticulture and floriculture to pick flowers, plants, and other ornamental crops, which may be especially fragile if handled roughly.

[0069] In yet another example, the picking robot may be used to harvest fish or shellfish in aquaculture farms or fisheries by picking and manipulating netting, seafood, and so on.

[0070] In yet another example, the picking robot may be used to pick and sort medical supplies, surgical tools, medications, or lab samples at hospitals or pharmaceutical facilities.

[0071] In yet another example, the picking robot may be used to pick and place building materials used in construction, such as bricks, blocks, or prefabricated components.

[0072] In yet another example, the picking robot may be used to pick and sort components used during assembly or manufacture of a product and / or the finished product.

[0073] In yet another example, the picking robot may be used in mining operations to pick valuable minerals or metals from excavated materials.

[0074] In yet another example, the picking robot may be used in textile manufacturing facilities to pick and handle fabric or textile materials.

[0075] In yet another example, the picking robot may be used in waste management facilities to pick and sort different kinds of waste (e.g., separation of plastic and paper waste for recycling).

[0076] In yet another example, the picking robot may be used in outdoor settings, such as parks, streets, or campuses, to pick up debris, litter, or fallen leaves.

[0077] In yet another example, the picking robot may be used in hotels or restaurants to pick and place utensils, dishes, cups, and / or other items for guests.(0O78| 4. Conclusion

[0079] All parameters, dimensions, materials, and configurations described herein are meant to be example and the actual parameters, dimensions, materials, and / or configurations will depend upon the specific application or applications for which the inventive teachings is / are used. It is to be understood that the foregoing embodiments are presented primarily by way of example and that, within the scope of the appended claims and equivalents thereto, inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein.

[0080] In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent, is included within the inventive scope of the present disclosure. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions and arrangement of respective elements of the example implementations without departing from the scope of the present disclosure. The use of a numerical range does not preclude equivalents that fall outside the range that fulfill the same function, in the same way, to produce the same result.

[0081] The above-described embodiments can be implemented in multiple ways. For example, embodiments may be implemented using hardware, software or a combination thereof. When implemented in software, the software code can be executed on a suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.

[0082] Further, it should be appreciated that a computer may be embodied in any of a number of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer. Additionally, a computer may be embedded in a device not generally regarded as a computer but with suitable processing capabilities, including a Personal Digital Assistant (PDA), a smart phone or any other suitable portable or fixed electronic device.

[0083] Also, a computer may have one or more input and output devices. These devices can be used, among other things, to present a user interface. Examples of output devices that can be used to provide a user interface include printers or display screens for visual presentation of output and speakers or other sound generating devices for audible presentation of output. Examples of input devices that can be used for a user interface include keyboards, and pointingdevices, such as mice, touch pads, and digitizing tablets. As another example, a computer may receive input information through speech recognition or in other audible format.

[0084] Such computers may be interconnected by one or more networks in a suitable form, including a local area network or a wide area network, such as an enterprise network, an intelligent network (IN) or the Internet. Such networks may be based on a suitable technology, may operate according to a suitable protocol, and may include wireless networks, wired networks or fiber optic networks.

[0085] The various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and / or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine. Some implementations may specifically employ one or more of a particular operating system or platform and a particular programming language and / or scripting tool to facilitate execution.

[0086] Also, various inventive concepts may be embodied as one or more methods, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.

[0087] All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety.

[0088] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0089] The indefinite articles “a” and “an,” as used herein in the specification and in the claims, unless clearly indicated to the contrary, should be understood to mean “at least one.”

[0090] The phrase “and / or,” as used herein in the specification and in the claims, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and / or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements may optionally be present other than the elements specificallyidentified by the “and / or” clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to “A and / or B”, when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.[009.11 As used herein in the specification and in the claims, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list, “or” or “and / or” shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as “only one of’ or “exactly one of,” or, when used in the claims, “consisting of,” will refer to the inclusion of exactly one element of a number or list of elements. In general, the term “or” as used herein shall only be interpreted as indicating exclusive alternatives (i.e., “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.” “Consisting essentially of,” when used in the claims, shall have its ordinary meaning as used in the field of patent law.

[0092] As used herein in the specification and in the claims, the phrase “at least one,” in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, “at least one of A and B” (or, equivalently, “at least one of A or B,” or, equivalently “at least one of A and / or B”) can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.[00931 In the claims, as well as in the specification above, all transitional phrases such as “comprising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “composed of,” and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases “consisting of’ and “consisting essentially of’ shall be closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures, Section 2111.03.

Claims

CLAIMS1. A picking robot configured to pick an object, the picking robot comprising: an end effector, comprising: a first picking mechanism, comprising: a plurality of fingers configured to apply a first holding force to the object via pinching, the plurality of fingers including a first finger, each finger of the plurality of fingers comprising: a plurality of segments; and at least one rotary joint coupled to the plurality of segments; and a second picking mechanism, comprising: a plurality of pads configured to apply a second holding force to the object, the plurality of pads comprising at least two pads directly coupled to the first finger; a pump fluidically coupled to each pad of the plurality of pads; and a controller communicatively coupled to the end effector and the pump, the controller being configured to: controllably adjust the first holding force by actuating the plurality of fingers; and controllably adjust the second holding force by moving air into or out of the plurality of pads using the pump, wherein: for each pad of the plurality of pads, when air is moved out from the pad, a pressure associated with the pad decreases below an ambient pressure of an ambient environment surrounding the picking robot thereby generating the second holding force; and the end effector is configured to use the first holding force and the second holding force to pick the object.

2. The picking robot of claim 1, wherein each finger of the plurality of fingers is identical to every other finger of the plurality of fingers.

3. The picking robot of claim 1, wherein for at least one finger of the plurality of fingers: the plurality of segments comprises three or more segments.

4. The picking robot of claim 1, wherein for each finger of the plurality of fingers: the plurality of segments comprises three or more segments.

5. The picking robot of claim 1, wherein for each finger of the plurality of fingers: the plurality of segments consists of two segments.

6. The picking robot of claim 1, wherein for each finger of the plurality of fingers: the plurality of pads comprises at least one pad disposed on each segment of the plurality of segments.

7. The picking robot of claim 1, wherein for at least one finger of the plurality of fingers: the plurality of pads comprises at least one pad spanning two or more segments of the plurality of segments.

8. The picking robot of claim 1, wherein: the plurality of fingers comprises a second finger; and the plurality of pads comprises at least one pad disposed on the second finger such that the at least one pad does not extend to a tip of the second finger.

9. The picking robot of claim 1, wherein for each finger of the plurality of fingers: the finger comprises a first side and a second side opposite the first side; when the picking robot is picking the object, the first side is oriented to face towards the object and the second side is oriented to face away from the object; and each pad of the plurality of pads disposed on the finger is disposed on the first side.

10. The picking robot of claim 1, wherein the end effector further comprises: at least one sensor disposed on each pad of the plurality of pads configured to detect physical contact between the pad and the object.

11. The picking robot of claim 10, wherein the controller is a closed loop controller.

12. The picking robot of claim 1, wherein: each pad of the plurality of pads is configured to transition between a fluid-like state and a solid-like state via granular jamming; andthe controller is configured to controllably transition each pad of the plurality of pads between the fluid-like state and the solid-like state by using the pump to move air into or out of the pad.

13. The picking robot of claim 12, wherein each pad of the plurality of pads comprises: a membrane, fluidically coupled to the pump, defining a cavity; and granular media disposed in the cavity.

14. The picking robot of claim 13, wherein the granular media comprises elastically deformable particles having a Young’s modulus ranging from about 106Pa to about 109Pa.

15. The picking robot of claim 13, wherein the granular media comprises at least one of expanded polystyrene beads, Styrofoam, rubber, or polyurethane.

16. The picking robot of claim 13, wherein the granular media comprises at least one of ground coffee, steel beads, glass beads, rice, salt, or sugar.

17. A picking robot configured to pick an object, the picking robot comprising: an end effector, comprising: a plurality of fingers including a first finger, each finger of the plurality of fingers comprising: a plurality of segments; and at least one joint coupled to the plurality of segments; and a plurality of pads directly coupled to the plurality of fingers, the plurality of pads comprising at least two pads directly coupled to the first finger of the plurality of fingers; and a pump fluidically coupled to the plurality of pads, wherein for each pad of the plurality of pads, the pump is configured to withdraw air from a portion of the pad so as to generate a pressure difference between the portion of the pad and an ambient environment surrounding the picking robot, the pressure difference generating a holding force to pick the object.

18. The picking robot of claim 17, wherein each finger of the plurality of fingers is identical to every other finger of the plurality of fingers.

19. The picking robot of claim 17, wherein for at least one finger of the plurality of fingers: the plurality of segments comprises three or more segments.

20. The picking robot of claim 17, wherein for each finger of the plurality of fingers: the plurality of pads comprises at least one pad disposed on each segment of the plurality of segments.

21. The picking robot of claim 17, wherein for at least one finger of the plurality of fingers: the plurality of pads comprises at least one pad spanning two or more segments of the plurality of segments.

22. The picking robot of claim 17, wherein: the plurality of fingers comprises a second finger; and the plurality of pads comprises at least one pad disposed on the second finger such that the at least one pad does not extend to a tip of the second finger.

23. A picking robot configured to pick an object, the picking robot comprising: an end effector, comprising: a first picking mechanism, comprising: a plurality of fingers configured to apply a first holding force to the object via pinching, the plurality of fingers including a first finger, each finger of the plurality of fingers comprising: a plurality of segments; and at least one rotary joint coupled to the plurality of segments; and a second picking mechanism, comprising: a plurality of pads configured to apply a second holding force to the object, the plurality of pads comprising at least one pad directly coupled to the first finger such that the at least one pad does not extend to a tip of the first finger; a pump fluidically coupled to each pad of the plurality of pads; anda controller communicatively coupled to the end effector and the pump, the controller being configured to: controllably adjust the first holding force by actuating the plurality of fingers; and controllably adjust the second holding force by moving air into or out of the plurality of pads using the pump, wherein: when air is moved out of each pad of the plurality of pads, a pressure within the pad decreases lower than an ambient pressure of an ambient environment surrounding the picking robot thereby increasing the second holding force; and the end effector is configured to use the first holding force and the second holding force to pick the object.

24. The picking robot of claim 23, wherein for at least one finger of the plurality of fingers: the plurality of segments comprises three or more segments.

25. The picking robot of claim 23, wherein for each finger of the plurality of fingers: the plurality of pads comprises at least one pad disposed on each segment of the plurality of segments.

26. The picking robot of claim 23, wherein for at least one finger of the plurality of fingers: the plurality of pads comprises at least one pad spanning two or more segments of the plurality of segments.

27. A picking robot, comprising: an end effector, comprising: a plurality of fingers, each finger of the plurality of fingers comprising: one or more segments; at least one rotary joint coupled to the one or more segments; and a plurality of squeeze pads coupled to at least one of the one or more segments or the at least one rotary joint, the plurality of squeeze pads beingconfigured to transition between a fluid-like state and a solid-like state via granular j amming; a pump coupled to the end effector; and a controller, communicatively coupled to the end effector and the pump, to move the plurality of fingers by actuating the at least one rotary joint and to controllably transition the plurality of squeeze pads between the fluid-like state and the solid-like state by using the pump to move air into or out of the plurality of squeeze pads.

28. The picking robot of claim 27, wherein each squeeze pad of the plurality of squeeze pads comprises: a membrane, fluidically coupled to the pump, defining a cavity; and granular media disposed in the cavity.

29. The picking robot of claim 28, wherein the granular media includes elastically deformable particles having a Young’s modulus ranging from about 106Pa to about 109Pa.

30. The picking robot of claim 28, wherein the granular media comprises at least one of expanded polystyrene beads, Styrofoam, rubber, or polyurethane.

31. The picking robot of claim 28, wherein the granular media comprises at least one of ground coffee, steel beads, glass beads, rice, salt, or sugar.

Images