Mitigation for unintended multi-picks using a multi-element robotic end effector

Abstract

A robot includes an end effector comprising a plurality of gripping elements, at least one sensor configured to measure a sensor signal indicating whether any of the plurality of gripping elements is not gripping any object, and a controller. The controller is configured to, in each respective pick cycle, in a gripping phase, actuating all of the plurality of gripping elements, subsequent to the gripping phase, perform an element cycling by deactuating and actuating each of the plurality of gripping elements according to a sequence of cycling stages, and determine, during the element cycling and / or upon completion of the element cycling, whether a multi-pick has occurred based on the sensor signal measured by the at least one sensor.

Description

MITIGATION FOR UNINTENDED MULTI-PICKS USING A MULTI ELEMENT ROBOTIC END EFFECTORFIELD

[0001] Embodiments of the present invention relate to robots and methods of operating robots.BACKGROUND

[0002] Robots can be used for picking objects and transporting the objects to destinations. For example, in a warehouse, a robot can be used to pick merchandise items from a pick tote (e.g., a storage box) and placing them onto a conveyor or into a place tote (e.g., a shipping box). Such a robot can include an end effector configured to pick an object from the pick tote, hold the object while the end effector moves over the place tote, and then drop the object into the place tote. The end effector can include one or more gripping elements for gripping objects. Multi-element end effectors can have several advantages over single-element end effectors. For example, multi-element end effectors can afford increased lift capacity, provide stabilization, and allow larger gripping areas. But multi-element end effectors are more likely to unintentionally pick several items simultaneously. One solution for detecting multi-picks is to include a force transducer at the end effector. For example, if the weight is higher than expected at the end effector, it may be determined that an unintended multi-pick has likely occurred, and the robot can take appropriate action.However, weight scales can incur additional cost and complexity. Force transducers are also delicate, and can have long settling times before they can accurately detect weight.SUMMARY

[0003] Embodiments of the present invention provide a robot. The robot includes an end effector comprising a plurality of gripping elements. The robot is configured to, in each pick cycle, pick an object from a pick area via the plurality of gripping elements in a gripping phase, lift the object in a lifting phase, and transport the object to a destination area in a transportation phase. The robot further includes at least one sensor configured to measure a sensor signal indicating whether any of the plurality of gripping elements is not gripping any object, and a controller. The controller is configured to, in each respective pick cycle, in the gripping phase, actuating all of the plurality of gripping elements, subsequent to the gripping phase, perform an element cycling by deactuating and actuating each of the plurality ofgripping elements according to a sequence of cycling stages, and determine, during the element cycling and / or upon completion of the element cycling, whether a multi-pick has occurred based on the sensor signal measured by the at least one sensor.

[0004] Embodiments of the present invention also provide a method of operating a robot. The robot includes an end effector including a plurality of gripping elements. The robot is configured to, in each pick cycle, pick an object from a pick area via the plurality of gripping elements and transport the object to a destination area. The method includes, in each respective pick cycle, in a gripping phase, actuating all of the plurality of gripping elements, subsequent to the gripping phase, performing an element cycling by deactuating and actuating each of the plurality of gripping elements according to a sequence of cycling stages, during the element cycling and / or upon completion of the element cycling, detecting, using at least one sensor, a sensor signal indicating whether any of the plurality of gripping elements is not gripping any object, and determining whether a multi -pick has occurred based on the sensor signal detected by the at least one sensor.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] FIG. 1 shows an exemplary robot with an end effector that includes two gripping elements.

[0006] FIG. 2A shows an exemplary four-cup end effector picking up a package of apparel.

[0007] FIG. 2B shows an exemplary four-cup end effector having four vacuum cups.

[0008] FIG. 3 A shows an example for a two-element end effector in which items are arranged so that double-pick is unlikely.

[0009] FIG. 3B shows an example for a two-element end effector in which two items are arranged adjacent to each other, where double-pick is more likely.

[0010] FIGS. 4A and 4B illustrate examples of relocating items from a pick tote to a place tote using a robot.

[0011] FIG. 5A illustrates how a single pick would be unaffected by element cycling according to some embodiments.

[0012] FIG. 5B illustrates how a double pick can be detected by element cycling in a two-cup end effector according to some embodiments.

[0013] FIG. 6 illustrates how a double pick can be detected by element cycling in a three-cup end effector according to some embodiments.

[0014] FIG. 7 illustrates how a double pick can be detected by element cycling in a four- cup end effector according to some embodiments.

[0015] FIGS. 8A - 8C illustrate some examples of edge cases by a three-cup end effector according to some embodiments.

[0016] FIG. 9 illustrates an example in which two items with irregular shapes are positioned adjacent next to each other.

[0017] FIGS. 10A and 10B illustrate timing considerations for a vacuum-cup end effector according to some embodiments.

[0018] FIGS. 11A and 1 IB illustrate the effects of the dynamics of vacuum cups according to some embodiments.

[0019] FIG. 12 is a flow diagram of method steps for operating a robot to mitigate unintended multi -picks according to some embodiments.DETAILED DESCRIPTION

[0020] Embodiments of the present invention provide robot controllers and methods to mitigate unintended multi-picks using a multi-element end effector (also referred herein as a multi-element gripper). Each pick cycle can include a gripping phase, a lifting phase, and a transportation phase. In the gripping phase, the end effector grips an item in a pick area. In the lifting phase, the end effector lifts the item predominantly in a vertically upward direction. In the transportation phase, the end effector moves laterally over to a destination area, where the item is dropped.

[0021] During the gripping phase, all of the gripping elements are actuated simultaneously. Preferably, a single item is gripped by all of the gripping elements that are initially actuated for each pick cycle, so that the item is securely gripped by the end effector. In cases of double-picks or multi-picks, it is likely that one of the items is gripped by only one gripping element or is partially gripped by one or more gripping elements.

[0022] According to embodiments of the present invention, after the initial gripping phase, the gripping elements are individually deactuated and actuated in a predetermined sequence. For example, for an end effector that includes a plurality of vacuum cups and operates by suction of the vacuum cups, each vacuum cup can be actuated by opening a valve allowing air to flow from the vacuum cup to a vacuum system, and can be deactuated by closing the valve. For a jawed gripper, each jaw can be actuated by closing the jaw, and can be deactuated by opening the jaw. If there is a multi-pick, the item that is gripped by only onegripping element or is partially gripped by one or more gripping elements would be dropped during the element cycling. Thus, the likelihood that more items being delivered to their destination than are intended can be reduced. It can be advantageous to perform the element cycling during the lifting phase of the pick cycle when the end effector is still over the pick area, so that the item would fall back into the pick area instead of an uncontrolled area, such as the floor. Preferably, the element cycling is completed before or not long after the start of the transportation phase.

[0023] According to some embodiments, the robot includes a sensor configured to measure a sensor signal indicating whether any of the gripping elements is not gripping any item during the element cycling. For example, if the gripping elements are vacuum cups, a pressure sensor can measure the pressure of the vacuum system associated with the vacuum cups. If an item is dropped during the element cycling, the pressure would increase as compared to when all vacuum cups are gripping onto an item. Thus, the robot can determine that a multi -pick has occurred upon detecting the pressure increase. According to some embodiments, upon determining that a multi-pick has occurred, the robot can abort the current pick cycle and start a new pick cycle. In this case, the current pick cycle resulted in a miss-pick (i.e., the end effector fails to pick any item). It is assumed that a miss-pick is preferable to a double-pick or multi-pick, because the accidental relocation of items is a more sever error than dropping items back into the pick area. According to some embodiments, upon determining that a multi-pick has occurred and that an item is dropped during the element cycling, the robot can proceed to complete the current pick cycle.

[0024] As described above and below, embodiments of the present invention provide a robot, a method, and a computer-readable medium for mitigation of unintended multi-picks using a multi-element robotic end effector.

[0025] In a first aspect, embodiments of the present invention provide a robot. The robot includes an end effector that has a plurality of gripping elements. The robot is configured to, in each pick cycle, pick an object from a pick area via the plurality of gripping elements in a gripping phase, lift the object in a lifting phase, and transport the object to a destination area in a transportation phase. The robot further includes at least one sensor configured to measure a sensor signal indicating whether any of the plurality of gripping elements is not gripping any object, and a controller. The controller is configured to, in each respective pick cycle, in the gripping phase, actuate all of the plurality of gripping elements, subsequent to the gripping phase, perform an element cycling by deactuating and actuating each of the plurality of gripping elements according to a sequence of cycling stages, and determine, during theelement cycling and / or upon completion of the element cycling, whether a multi-pick has occurred based on the sensor signal measured by the at least one sensor.

[0026] In a second aspect, the present invention provides the robot according to the first aspect, wherein the controller is further configured to, upon determining that the multi-pick has occurred, abort the respective pick cycle and starting a new pick cycle.

[0027] In a third aspect, the present invention provides the robot according to the first aspect, wherein the controller is further configured to, upon determining that the multi-pick has occurred and that an object is dropped during the element cycling, proceed to complete the respective pick cycle.

[0028] In a fourth aspect, the present invention provides the robot according to the first aspect, wherein the element cycling is performed during the lifting phase of the respective pick cycle.

[0029] In a fifth aspect, the present invention provides the robot according to the first aspect, wherein the plurality of gripping elements includes a plurality of vacuum cups, and the at least one sensor includes a pressure sensor configured to measure a pressure of a vacuum system associated with the plurality of vacuum cups.

[0030] In a sixth aspect, the present invention provides the robot according to the first aspect, wherein the plurality of gripping elements comprises a plurality of vacuum cups, and wherein the at least one sensor comprises a plurality of pressure sensors, each respective pressure sensor configured to measure a respective pressure associated with a respective vacuum cup.

[0031] In a seventh aspect, the present invention provides the robot according to the first aspect, wherein each of the plurality of gripping elements includes one of a jawed gripping element, a soft gripping element, or a magnetic gripping element, and the at least one sensor includes one of a proximity sensor, an impedance sensor, or a magnetic sensor.

[0032] In an eighth aspect, the present invention provides the robot according to the first aspect, wherein the plurality of gripping elements includes two gripping elements, and performing the element cycling includes alternately deactuating one of the two gripping elements in the sequence of cycling stages.

[0033] In a ninth aspect, the present invention provides the robot according to the first aspect, wherein the plurality of gripping elements includes three gripping elements, and performing the element cycling includes sequentially deactuating one of the three gripping elements in the sequence of cycling stages.

[0034] In a tenth aspect, the present invention provides the robot according to the first aspect, wherein the plurality of gripping elements includes four gripping elements, and performing the element cycling includes, at each cycling stage, deactuating two of the four gripping elements.

[0035] In an eleventh aspect, embodiments of the present invention provide a method of operating a robot. The robot includes an end effector that has a plurality of gripping elements. The robot is configured to, in each pick cycle, pick an object from a pick area via the plurality of gripping elements and transport the object to a destination area. The method includes, in each respective pick cycle, in a gripping phase, actuating all of the plurality of gripping elements, subsequent to the gripping phase, performing an element cycling by deactuating and actuating each of the plurality of gripping elements according to a sequence of cycling stages, during the element cycling and / or upon completion of the element cycling, detecting, using at least one sensor, a sensor signal indicating whether any of the plurality of gripping elements is not gripping any object, and determining whether a multi-pick has occurred based on the sensor signal detected by the at least one sensor.

[0036] In a twelfth aspect, the present invention provides the method according to the eleventh aspect, wherein the method further includes, upon determining that the multi-pick has occurred, aborting the respective pick cycle and starting a new pick cycle.

[0037] In a thirteenth aspect, the present invention provides the method according to the eleventh aspect, wherein the method further includes, upon determining that the multi-pick has occurred and that an object is dropped during the element cycling, proceeding to complete the respective pick cycle.

[0038] In a fourteenth aspect, the present invention provides the method according to the eleventh aspect, wherein the element cycling is performed during a lifting phase of the respective pick cycle.

[0039] In a fifteenth aspect, the present invention provides the method according to the eleventh aspect, wherein the plurality of gripping elements includes a plurality of vacuum cups, and the at least one sensor includes a pressure sensor configured to measure a pressure of a vacuum system associated with the plurality of vacuum cups.

[0040] In a sixteenth aspect, the present invention provides the method according to the fifteenth aspect, wherein the method further includes, upon completion of the element cycling, actuating all of the plurality of vacuum cups. The detection of the sensor signal is performed after actuating all of the plurality of vacuum cups. Determining whether a multipick has occurred includes comparing the pressure of the vacuum system to a referencepressure, and upon determining that the pressure is greater than the reference pressure, determining that the multi-pick has occurred.

[0041] In a seventeenth aspect, the present invention provides the method according to the eleventh aspect, wherein the plurality of gripping elements includes a plurality of vacuum cups, and the at least one sensor includes a plurality of pressure sensors. Each respective pressure sensor is configured to measure a respective pressure associated with a respective vacuum cup. Determining whether a multi-pick has occurred includes, after a respective vacuum cup is actuated again subsequent to being deactuated, comparing the respective pressure associated with the respective vacuum cup to a reference pressure, and upon determining that the respective pressure is greater than the reference pressure, determining that the multi-pick has occurred.

[0042] In an eighteenth aspect, the present invention provides the method according to the eleventh aspect, wherein the plurality of gripping elements includes two or three gripping elements, and performing the element cycling includes deactuating one gripping element in each cycling stage of the sequence of cycling stages.

[0043] In a nineteenth aspect, the present invention provides the method according to the eleventh aspect, wherein the plurality of gripping elements includes four gripping elements, and performing the element cycling includes deactuating two of the four gripping elements at each cycling stage of the sequence of cycling stages.

[0044] In a twentieth aspect, embodiments of the present invention provide a non- transitory computer-readable medium having program code stored thereon. The program code, when executed by a computer processor, causes performance of the method according to the eleventh aspect.

[0045] In the following, embodiments of the present invention will be described in terms of a multi-cup gripper using vacuum. But embodiments of the present invention can be applied to other types of multi-element end effectors, such as jaw grippers, soft robotics grippers, magnetic grippers, and the like. The sensors can include proximity sensors, impedance sensors, or magnetic sensors.

[0046] FIG. 1 shows an exemplary robot with an end effector 110 that includes two gripping elements 110 and 120. In this example, the two gripping elements 110 and 120 operate by vacuum suction. Each gripping element 110 or 120 includes a tube with a suction cup 112 or 122 at the end. The tubes can be connected to a vacuum pump. When the suction cups 112 and 122 are placed on a surface of an object, the pressure inside the tubes can drop and cause the object to be gripped by the suction cups 112 and 122. FIG. 2 A shows anexemplary four-cup end effector 210 picking up a package 212 of apparel. FIG. 2B shows an exemplary four-cup end effector 220 having four vacuum cups 222.

[0047] FIG. 3A shows an example in which items are arranged so that double-pick is unlikely. In this top view, three items 310, 320 and 330 (e.g., three boxes) are well separated from each other. A two-element end effector with two gripping elements 302 and 304 is more likely to pick up a single item (e.g., the item 310). FIG. 3B shows an example in which two items 340 and 350 are arranged adjacent to each other. If the two items 340 and 350 are about the same height, it is possible that the two-element end effector picks up both items 340 and 350, with each gripping element 302 or 304 gripping each item 340 or 350. Double-pick can also occur when two thin items (e.g., two thin envelopes) partially overlap with each other.

[0048] FIG. 4A illustrates an example of relocating items from a pick area 410 (e.g., a pick tote) to a destination area 420 (e.g., a place tote) using a robot. In each pick cycle, an end effector of the robot first picks up an item from the pick area 410. This phase of the pick cycle may be referred to as the gripping phase. As the robot relocates the picked item from the pick area 410 to the destination area 420, the item may follow a trajectory described by two phases. During the lifting phase 430, the item is lifted vertically upward. This is followed by the transportation phase 440, in which the item is moved laterally to the destination area 420. The item is then dropped off at the destination area 420. FIG. 4B illustrates an exemplary trajectory of the item that is perhaps more likely in actual cases. Here, the transition from the lifting phase 430 to the transportation phase 440 is less clearly defined. The trajectory is predominantly vertical in the lifting phase 430, then transitions smoothly to predominantly lateral in the transportation phase 440.

[0049] The transportation phase 440 is riskier because, if an item is dropped during the transportation phase 440, the item may fall to some uncontrolled location (e.g., the floor). In contrast, if an item is dropped during the lifting phase 430, the item would fall back into the pick area 410. Thus, according to embodiments of the present invention, it would be advantageous to perform the element cycling during the lifting phase 430. Preferably, the element cycling is completed before or shortly after the start of the transportation phase 440. In this way, multi-picks are detected and aborted while the items are still above the pick area 410, to ensure that they fall back into the pick area 410 instead of to an uncontrolled area.

[0050] FIG. 5A illustrates how a good pick would be unaffected by element cycling according to some embodiments. A good pick refers to a single pick in which all gripping elements of an end effector are gripping the same item. FIG. 5A shows a top view of an item 502 (e.g., a box) being gripped by an end effector that has two vacuum cups 510 and 520(represented by the two dots 510 and 520). Each vacuum cup 510 or 520 can be individually actuated or deactuated (e.g., by opening or closing a valve). A solid dot represents when the vacuum cup is actuated, and an open dot represents when the vacuum cup is deactuated. In the initial gripping phase, both vacuum cups 510 and 520 are actuated. In a good pick, both vacuum cups 510 and 520 are gripping the same item 502. Next, the robot starts the sequence of cup cycling. First, the first vacuum cup 510 is deactuated, while the second vacuum cup 520 remains being actuated. The item 502 remains held by the end effector via the second vacuum cup 520. Next, the first vacuum cup 510 is actuated again, while the second vacuum cup 520 is deactuated. Because the second vacuum cup 520 is holding the item 502 in close proximity to the first vacuum cup 510 while the first vacuum cup 510 was deactuated, when the first vacuum cup 510 is actuated again, it reestablishes a grip onto the item 502, thus holding the item 502 in close proximity to the second vacuum cup 520 while the second vacuum cup 520 is deactuated. After the completion of the cup cycling, both vacuum cups 510 and 520 are actuated again and retain a grip on the item 502.

[0051] According to some embodiments, the end effector is equipped with at least one pressure sensor. The pressure sensor is configured to measure the pressure of the vacuum system associated with the vacuum cups 510 and 520. When both vacuum cups 510 and 520 are actuated again after the completion of the cup cycling, if the pressure is as low as expected (e.g., as compared to a reference pressure in a single pick), the robot can determine that it is a single pick (no item is dropped during the cup cycling). The robot can then proceed to transport the picked item 502 to the destination area. As discussed above, it is preferable that the cup cycling is performed during the lifting phase of the pick cycle, and completed before or shortly after the start of transportation phase.

[0052] FIG. 5B illustrates how a double pick can be detected by element cycling in a two- cup end effector according to some embodiments. In this example, two items 502 and 504 are positioned adjacent to each other in the pick area. If the two items 502 and 504 are of about the same height, and are close enough together (in this example, the two boxes abut against each other), it is possible that the two-cup end effector picks up both of them in the gripping phase, with each vacuum cup 510 or 520 gripping each respective item 502 or 504. Subsequent to the gripping phase, the robot starts the cup cycling. First, the second vacuum cup 520 is deactuated while the first vacuum cup 510 remains being actuated. The second item 504 that was gripped by the second vacuum cup 520 may be dropped. The first item 502 that gripped by the first vacuum cup 510 remains held by the end effector via the first vacuum cup 510. Next, the second vacuum cup 520 is actuated again, while the first vacuumcup 510 is deactuated. The first item 502 that was gripped by the first vacuum cup 510 may be dropped. Thus, the cup cycling resulted in a miss pick, and a double pick is averted.

[0053] After the completion of the cup cycling, both vacuum cups 510 and 520 are actuated again. Because neither of the two vacuum cups 510 and 520 is gripping any item, the pressure sensor would detect a pressure that is higher than expected (e.g., as compared to the reference pressure in a single pick). The robot can determine that a double pick has occurred. According to some embodiments, upon determining that a double pick has occurred, the robot aborts the current pick cycle, and starts a new pick cycle. As discussed above, if the cup cycling is performed during the lifting phase of the pick cycle, the two items 510 and 520 would drop back to the pick area. In some embodiments, instead of a pressure sensor, the robot can be equipped with a camera and image processing software, configured to detect whether any item is dropped during the cup cycling. In some embodiments, the end effector can include a weight scale. The robot can detect that an item is dropped during the cup cycling if the weight has decreased. In some embodiments, the pick tote is equipped with a weight scale. The robot can detect an item is dropped during the cup cycling if the weight has increased. In some embodiments, after the first stage of the cup cycling, the robot actuates the second vacuum cup 504 momentarily and detects that an item has been dropped. According to some embodiments, upon determining that an item has been dropped during the cup cycling (e.g., after the first stage of the cup cycling when the second item 520 is dropped), the robot can proceed to complete the pick cycle without going through the rest of the stages of cup cycling (e.g., by transporting the first item 510 to the destination).

[0054] FIG. 6 illustrates how a double pick can be detected by element cycling in a three- cup end effector according to some embodiments. Similar to the example illustrated in FIG. 5B, two items 602 and 604 with similar heights are positioned adjacent to each other in the pick area. In the gripping phase, all three vacuum cups 610, 620 and 630 are actuated. The end effector picks up both items 602 and 604, with the first vacuum cup 610 and the second vacuum cup 620 gripping the first item 602 and the third vacuum cup 630 gripping the second item 604.

[0055] Subsequent to the gripping phase, the robot starts the cup cycling. In a first stage, the first vacuum cup 610 is deactuated, while the second vacuum cup 620 and the third vacuum cup 630 remain being actuated. Both items 602 and 604 remain held by the end effector, as the first item 602 is still gripped by the second vacuum cup 620 and the second item 604 is still gripped by the third vacuum cup 630. In a second stage, the second vacuum cup 620 is deactuated, while the first vacuum cup 610 and the third vacuum cup 630 remainbeing actuated. Again, both items 602 and 604 remain held by the end effector, as the first item 602 is still gripped by the first vacuum cup 610 and the second item 604 is still gripped by the third vacuum cup 630. In a third stage, the third vacuum cup 630 is deactuated while the first vacuum cup 610 and the second vacuum cup 620 remain being actuated. The second item 604, which was gripped by the third vacuum cup 630 only, is dropped when the third vacuum cup 630 is deactuated.

[0056] After the completion of the cup cycling, all three vacuum cups 610, 620 and 630 are actuated again. Because the third vacuum cup 630 is not gripping any item, the pressure sensor would detect a pressure that is higher than expected (e.g., as compared to the reference pressure in a single pick). The robot can determine that a double pick has occurred.According to some embodiments, upon determining that a double pick has occurred, the robot is programmed to abort the current pick cycle and start a new pick cycle.

[0057] According to some embodiments, the end effector includes a pressure sensor for each of the vacuum cups 610, 620 and 630. Each respective pressure sensor is configured to measure the pressure associated with a respective vacuum cup 610, 620 or 630. Upon determining that a double pick has occurred and that the remaining item is still gripped by two vacuum cups, the robot is programmed to proceed to transport the remaining item to the destination area.

[0058] FIG. 7 illustrates how a double pick can be detected by element cycling in a four- cup end effector according to some embodiments. Similar to the example illustrated in FIG.6, two items 702 and 704 with similar heights are positioned adjacent to each other in the pick area. In the gripping phase, all four vacuum cups 710, 720, 730 and 740 are actuated. The end effector picks up both items 602 and 604, with the first vacuum cup 710 and the second vacuum cup 720 gripping the first item 602, and the third vacuum cup 730 and the fourth vacuum cup 740 gripping the second item 604.

[0059] Subsequent to the gripping phase, the robot starts the cup cycling. In a first stage, the first vacuum cup 710 and the third vacuum cup 730 are deactuated, while the second vacuum cup 720 and the fourth vacuum cup 740 remain being actuated. Both items 702 and 704 remain held by the end effector, as the first item 702 is still gripped by the second vacuum cup 720 and the second item 704 is still gripped by the fourth vacuum cup 740. In a second stage, the second vacuum cup 720 and the fourth vacuum cup 740 are deactuated, while the first vacuum cup 710 and the third vacuum cup 730 remain being actuated. Again, both items 702 and 704 remain held by the end effector, as the first item 702 is still gripped by the first vacuum cup 710 and the second item 704 is still gripped by the third vacuum cup730. In a third stage, the first vacuum cup 710 and the fourth vacuum cup 740 are deactuated, while the second vacuum cup 720 and the third vacuum cup 730 remain being actuated. Again, both items 702 and 704 remain held by the gripper, as the first item 702 is still gripped by the second vacuum cup 720 and the second item 704 is still gripped by the third vacuum cup 730. In a fourth stage, the second vacuum cup 720 and the third vacuum cup 730 are deactuated, while the first vacuum cup 710 and the fourth vacuum cup 740 remain being actuated. Again, both items 702 and 704 remain held by the gripper, as the first item 702 is still gripped by the first vacuum cup 710 and the second item 704 is still gripped by the fourth vacuum cup 740. In a fifth stage, the third vacuum cup 730 and the fourth vacuum cup 740 are deactuated, while the first vacuum cup 710 and the second vacuum cup 720 remain being actuated. This time, the second item 704 is dropped. In a sixth stage, the first vacuum cup 710 and the second vacuum cup 720 are deactuated, while the third vacuum cup 730 and the fourth vacuum cup 740 remain being actuated. This time, the first item 702 is dropped.

[0060] Afterthe completion of the cup cycling, all four vacuum cups 710, 720, 730 and 740 are actuated again. Because none of the vacuum cups 710, 720, 730 and 740 is gripping any item, the pressure sensor would detect a pressure that is higher than expected (e.g., as compared to the reference pressure in a single pick). The robot can determine that a double pick has occurred, and can abort the current pick cycle and start a new pick cycle. In this example, the robot may be able to detect a double pick as early as the sixth stage of the cup cycling, when the third vacuum cup 730 and the fourth vacuum cup 740 are actuated but neither of them is gripping any item.

[0061] In this example of the four-cup end effector, the double pick is not detected until the sixth stage of the cup cycling. Depending on the sequence of the cup cycling, the double pick could have been detected as early as the second stage of the cup cycling. The chosen sequence of the cup cycling can affect the end state of a pick cycle. In general, the number of stages needed to complete a cup cycling increases with the number of vacuum cups in the end effector.

[0062] FIGS. 8A - 8C illustrate some examples of edge cases by a three-cup end effector. Similar to the example illustrated in FIG. 6, the end effector has three vacuum cups 610, 620, and 630. Two items 602 and 604 of about the same height are abut against each other in the pick area. In the example illustrated in FIG. 8 A, the third vacuum cup 630 is positioned at the edge between the two items 602 and 604. Similar to the example illustrated in FIG. 6, the second item 604 may be dropped at the third stage of the cup cycling. After the completion of the cup cycling, all three vacuum cups 610, 620 and 630 are actuated again. The pressuresensor can detect a pressure increase as only a part of the third vacuum cup 630 is gripping the first item 602. Thus, the robot can determine that a double pick has occurred. Similar to the example illustrated in FIG. 6, the robot can either be programmed to abort the current pick cycle and start a new pick cycle, or be programmed to proceed to transport the remaining item 602 to the destination area (if the end effector includes a pressure sensor for each vacuum cup).

[0063] In the example illustrated in FIG. 8B, both the first vacuum cup 610 and the second vacuum cup 620 are positioned at the edge between the two items 602 and 604. The first item 602 may be dropped as early as the first stage of the cup cycling, as partial contact of the second vacuum cup 620 with the first item 602 may not be able to hold the first item 602. At the third stage of the cup cycling, the second item 604 is dropped. If the end effector is equipped with a pressure sensor for each vacuum cup, the robot may be able to determine that a double pick has occurred as early as the first stage of the cup cycling, as the second vacuum cup 620 is only partially contacting the second item 604 while it is actuated. Thus, the robot can abort the current pick cycle right then and start a new pick cycle.

[0064] In the example illustrated in FIG. 8C, the third vacuum cup 630 is positioned at the edge between the two items 602 and 604. The first item 602 may be dropped at the first stage of the cup cycling. If the end effector is equipped with a pressure sensor for each vacuum cup, the robot may be able to determine that a double pick has occurred as early as the first stage of the cup cycling, as the third vacuum cup 630 is only partially contacting the second item 604 while it is actuated. The robot can abort the current pick cycle right then and start a new pick cycle, if the robot is programmed to abort in cases of double picks. But if the robot is programmed to proceed to transport the remaining item 604 to the destination area, the robot may complete the cup cycling and then proceed to the transportation phase.

[0065] In certain situations, the element-cycling as described above may fail to avert a multi-pick. The effectiveness of the method may depend on the shapes of the items to be picked up. For example, if items are entangled with each other in the pick area, the additional item that is picked up may not fall off from the end effector during the element-cycling. FIG. 9 illustrates an example, in which two items 902 and 904 with irregular shapes are positioned adjacent next to each other. In this example, the irregular shapes of the two items 902 and 904 fit with each other almost like a jigsaw puzzle. Each of the three vacuum cups 910, 920 and 930 of the end effector is in contact with both items 902 and 904. As illustrated, at each stage of the cup cycling, both items 902 and 904 remain held by the end effector. Thus, the element-cycling can mitigate multi-picks, but does not prevent multi-picks in all situations.According to some embodiments, the element-cycling can be combined with other multi-pick mitigation methods.

[0066] As discussed above in relation to FIG. 4, it can be advantageous to perform the element cycling during the lifting phase of a pick cycle, and complete the element cycling before or not long after the start of the transportation phase. This way, the additional items that are dropped during the element cycling would fall back into the pick area 410 instead of an uncontrolled area such as the floor. The time duration required to complete the cup cycling may depend on the number of gripping elements and transient times for actuating and deactuating each gripping element.

[0067] FIGS. 10A and 10B illustrate timing considerations for a vacuum-cup end effector. In FIG. 10A, the vacuum cup 1020 is actuated and is gripping an item 1010. In FIG. 10B, the vacuum cup 1020 has been deactuated, and the item 1010 is dropped. Assume that the vacuum cup 1020 should not be actuated again until the item 1010 is separated from the vacuum cup 1020 by a standoff distance D that is no less than 3 mm (otherwise the item 1010 may be gripped by the vacuum cup 1020 again). Using the equation t(D) = where g =9.8 m / s2is the gravity of the earth, it can be calculated that it may require t (0.003) = 0.025 seconds for the object to drop 3 mm. Thus, the vacuum cup 1020 should remain deactuated for at least 0.025 seconds before being actuated again. This minimum time may vary depending on other factors, such as air resistance or end effector accelerations.

[0068] Assume that the robot is programmed to lift the item vertically by 30 cm during the lifting phase, before moving it laterally to transport it to the destination. Assume also that the robot moves the end effector at a speed of 1000 mm / s. This would allow about 0.3 seconds to complete the cup cycling during the lifting phase. Assume that the vacuum transient time for actuating a vacuum cup is 0.010 seconds, and the vacuum transient time for deactuating a vacuum cup is 0.012 seconds. (The vacuum transient times are also referred herein as the time constants.) The total time to cycle one vacuum cup would be equal to 0.010 + 0.012 + 0.025 = 0.047 seconds. Based on these assumptions, 0.3 seconds would allow about 6.4 cycling stages. For the exemplary cup-cycling sequence of the four-cup end effector illustrated in FIG. 7, the six-stage cup cycling can be completed during the lifting phase of the pick cycle. The time duration of 0.3 seconds would also allow sufficient time to complete the cup cycling of the two-cup end effector or the three-cup end effector as illustrated in FIGS. 5A, 5B, and 6, which require less cycling stages.

[0069] The effectiveness of the mitigation method can be evaluated by the rate at which single picks (good picks) are carried out successfully (the picked item is not dropped during the cup cycling and is transported to the destination area), and the rate at which double picks are aborted (the picked items are dropped during the cup cycling). The mitigation method can be considered effective if most of the single picks are carried out successfully and most of the double picks are aborted. Conversely, the mitigation method can be considered ineffective if most of the single picks are also aborted (the picked item is dropped during the cup cycling), or if the robot fails to abort most of the double picks. The effectiveness of the mitigation method can be affected by cup size, cup shape, gas flowrate through the cups, item weight, and item packaging. For example, tests show that the mitigation method is effective for envelopes and boxes. Tests also show that voluminous cups or pneumatic hoses have larger time constants than smaller cups and shorter hoses, because evacuating air in long tubes with large cups takes longer. If the cups are not cycled fast enough, single picks may be aborted. Reducing the time between the cycling stages can prevent aborting single picks, but it may also prevent aborting double picks because the item does not fully separate from the cup before the cup is actuated again.

[0070] FIGS. 11A and 1 IB illustrate the effects of the dynamics of the vacuum cups. The horizontal axis is time, and the vertical axis is the pressure of the vacuum system. The dashed curve 1110 represents an ideal case in which the pressure responds instantly to the switching between actuated and deactuated (a time constant of zero). The solid curves 1120 and 1130 represent the responses in actual cases. In actual cases, it takes some time for the pressure to drop after the cup has been actuated, and takes some time for the pressure to rise after the cup has been deactuated. These considerations for a vacuum cup apply for other types of gripping elements which are also dynamical systems. In the example shown in FIG. 11A, the system is driven too fast. The pressure has not been fully dropped to the target negative pressure before the cup is deactuated again. Likewise, the pressure has not fully risen to the target ambient pressure yet before the cup is actuated again. In the example shown in FIG. 1 IB, the system is driven slow enough, so that the pressure has dropped to the negative target pressure before the cup is deactuated again, and has risen to the ambient target pressure before the cup is actuated again. The cup dynamics represented by the curve 1130 in FIG. 1 IB is preferred over the cup dynamics represented by the curve 1120 in FIG. 11A.

[0071] FIG. 12 is a flow diagram of method steps for operating a robot to mitigate unintended multi-picks according to some embodiments. The robot includes an end effector that has a plurality of gripping elements. The robot is configured to, in each pick cycle, pickan object from a pick area via the plurality of gripping elements and transport the object to a destination area. The method includes, in each respective pick cycle, at 1210, in a gripping phase, actuating the plurality of gripping elements. The method further includes, at 1220, subsequent to the gripping phase, performing an element cycling by deactuating and actuating each of the plurality of gripping elements according to a predetermined sequence of cycling stages. The method further includes, at 1230, during the element cycling and / or upon completion of the element cycling, detecting, using at least one sensor, a sensor signal indicating whether any of the plurality of gripping elements is not gripping any object. The method further includes, at 1240, determining whether a multi-pick has occurred based on the sensor signal detected by the at least one sensor.

[0072] As described above, according to embodiments of the present invention, unintended multi-picks by a multi-element end effector can be detected and averted by performing a sequence of element cycling after the gripping phase. It can be advantageous to perform and complete the element cycling during the lifting phase of a pick cycle, so that any item that is dropped during the element cycling would fall back into the pick area instead of an uncontrolled area.

[0073] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

[0074] The use of the terms “a” and “an” and “the” and “at least one” and similar referents in the context of describing the invention (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The use of the term “at least one” followed by a list of one or more items (for example, “at least one of A and B”) is to be construed to mean one item selected from the listed items (A or B) or any combination of two or more of the listed items (A and B), unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein.

[0075] All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.

[0076] Preferred embodiments of this invention are described herein. Variations of those preferred embodiments may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the invention to be practiced otherwise than as specifically described herein. Accordingly, this invention includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the invention unless otherwise indicated herein or otherwise clearly contradicted by context.

[0077] It should be understood that the original applicant herein determines which technologies to use and / or productize based on their usefulness and relevance in a constantly evolving field, and what is best for it and its players and users. Accordingly, it may be the case that the systems and methods described herein have not yet been and / or will not later be used and / or productized by the original applicant. It should also be understood that implementation and use, if any, by the original applicant, of the systems and methods described herein are performed in accordance with its privacy policies. These policies are intended to respect and prioritize player privacy, and are believed to meet or exceed government and legal requirements of respective jurisdictions. To the extent that such an implementation or use of these systems and methods enables or requires processing of user personal information, such processing is performed (i) as outlined in the privacy policies; (ii) pursuant to a valid legal mechanism, including but not limited to providing adequate notice or where required, obtaining the consent of the respective user; and (iii) in accordance with the player or user’s privacy settings or preferences. It should also be understood that the original applicant intends that the systems and methods described herein, if implemented or used by other entities, be in compliance with privacy policies and practices that are consistent with its objective to respect players and user privacy.

Claims

CLAIMSWHAT IS CLAIMED IS:

1. A robot comprising: an end effector comprising a plurality of gripping elements, the robot configured to, in each pick cycle, pick an object from a pick area via the plurality of gripping elements in a gripping phase, lift the object in a lifting phase, and transport the object to a destination area in a transportation phase; at least one sensor configured to measure a sensor signal indicating whether any of the plurality of gripping elements is not gripping any object; and a controller configured to, in each respective pick cycle: in the gripping phase, actuate all of the plurality of gripping elements; subsequent to the gripping phase, perform an element cycling by deactuating and actuating each of the plurality of gripping elements according to a sequence of cycling stages; and determine, during the element cycling and / or upon completion of the element cycling, whether a multi-pick has occurred based on the sensor signal measured by the at least one sensor.

2. The robot of claim 1, wherein the controller is further configured to, upon determining that the multi-pick has occurred, abort the respective pick cycle and starting a new pick cycle.

3. The robot of claim 1, wherein the controller is further configured to, upon determining that the multi -pick has occurred and that an object is dropped during the element cycling, proceed to complete the respective pick cycle.

4. The robot of claim 1, wherein the element cycling is performed during the lifting phase of the respective pick cycle.

5. The robot of claim 1, wherein the plurality of gripping elements comprises a plurality of vacuum cups, and the at least one sensor comprises a pressure sensor configured to measure a pressure of a vacuum system associated with the plurality of vacuum cups.

6. The robot of claim 1, wherein the plurality of gripping elements comprises a plurality of vacuum cups, and wherein the at least one sensor comprises a plurality of pressure sensors, each respective pressure sensor configured to measure a respective pressure associated with a respective vacuum cup.

7. The robot of claim 1, wherein each of the plurality of gripping elements comprises one of a jawed gripping element, a soft gripping element, or a magnetic gripping element, and the at least one sensor comprises one of a proximity sensor, an impedance sensor, or a magnetic sensor.

8. The robot of claim 1, wherein the plurality of gripping elements comprises two gripping elements, and performing the element cycling comprises alternately deactuating one of the two gripping elements in the sequence of cycling stages.

9. The robot of claim 1, wherein the plurality of gripping elements comprises three gripping elements, and performing the element cycling comprises sequentially deactuating one of the three gripping elements in the sequence of cycling stages.

10. The robot of claim 1, wherein the plurality of gripping elements comprises four gripping elements, and performing the element cycling comprises, at each cycling stage, deactuating two of the four gripping elements.

11. A method of operating a robot, the robot comprising an end effector including a plurality of gripping elements, the robot configured to, in each pick cycle, pick an object from a pick area via the plurality of gripping elements and transport the object to a destination area, the method comprising, in each respective pick cycle: in a gripping phase, actuating all of the plurality of gripping elements; subsequent to the gripping phase, performing an element cycling by deactuating and actuating each of the plurality of gripping elements according to a sequence of cycling stages; during the element cycling and / or upon completion of the element cycling, detecting, using at least one sensor, a sensor signal indicating whether any of the plurality of gripping elements is not gripping any object; and determining whether a multi-pick has occurred based on the sensor signal detected by the at least one sensor.

12. The method of claim 11, further comprising, upon determining that the multipick has occurred, aborting the respective pick cycle and starting a new pick cycle.

13. The method of claim 11, further comprising, upon determining that the multipick has occurred and that an object is dropped during the element cycling, proceeding to complete the respective pick cycle.

14. The method of claim 11, wherein the element cycling is performed during a lifting phase of the respective pick cycle.

15. The method of claim 11, wherein the plurality of gripping elements comprises a plurality of vacuum cups, and the at least one sensor comprises a pressure sensor configured to measure a pressure of a vacuum system associated with the plurality of vacuum cups.

16. The method of claim 15, further comprising, upon completion of the element cycling, actuating all of the plurality of vacuum cups, wherein the detection of the sensor signal is performed after actuating all of the plurality of vacuum cups, and wherein determining whether a multi-pick has occurred comprises comparing the pressure of the vacuum system to a reference pressure, and upon determining that the pressure is greater than the reference pressure, determining that the multi-pick has occurred.

17. The method of claim 11, wherein the plurality of gripping elements comprises a plurality of vacuum cups, wherein the at least one sensor comprises a plurality of pressure sensors, each respective pressure sensor configured to measure a respective pressure associated with a respective vacuum cup, and wherein determining whether a multi-pick has occurred comprises: after a respective vacuum cup is actuated again subsequent to being deactuated, comparing the respective pressure associated with the respective vacuum cup to a reference pressure; and upon determining that the respective pressure is greater than the reference pressure, determining that the multi-pick has occurred.

18. The method of claim 11, wherein the plurality of gripping elements comprises two or three gripping elements, and performing the element cycling comprises deactuating one gripping element in each cycling stage of the sequence of cycling stages.

19. The method of claim 11, wherein the plurality of gripping elements comprises four gripping elements, and performing the element cycling comprises deactuating two of the four gripping elements at each cycling stage of the sequence of cycling stages.

20. A non-transitory computer-readable medium having program code stored thereon, the program code, when executed by a computer processor, causing performance of the method of claim 11.

Images