Robotic system and method for identifying and handling various objects

By using articulated arms and multiple perception systems in the robot system, picking and placing objects to ensure the barcode is visible, the problems of object recognition orientation and position limitation in the prior art are solved, and the efficiency of automatic identification and classification is achieved.

CN113635319BActive Publication Date: 2024-11-15BERKSHIRE GREY OPERATING CO INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110913420.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2015-12-18
Filing Date
2016-09-09
Publication Date
2024-11-15
Estimated Expiration
2036-09-09

AI Technical Summary

Technical Problem

The prior art has directional, position and skew restrictions when identifying and processing objects, making it difficult to automatically identify and process various objects in a dynamic environment, especially when the object barcode is not visible or invisible.

Method used

Using articulated arms and multiple additional perception systems, objects are picked up and placed under multiple views by robotic arms, ensuring barcodes are visible with multiple perception units and central control systems, including primary perception systems and secondary perception systems for maximum and minimize recognition success rate.

Benefits of technology

It realizes automatic and accurate identification and classification of objects in heterogeneous object flow, reduces manual intervention and improves sorting efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113635319B_ABST
    Figure CN113635319B_ABST
Patent Text Reader

Abstract

A robotic system is disclosed, comprising an articulated arm and a first perception system for inspecting an object and a plurality of additional perception systems, each of the additional perception systems being arranged to face a common area where the object can be positioned by the robotic arm such that a plurality of views within the common area can be obtained by the plurality of additional perception systems.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the Chinese patent application (PCT application number PCT / US2016 / 050949) with application date of September 9, 2016, application number 201680065881.2, and name “Robotic system and method for identifying and processing various objects”.

[0002] priority

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 217,200 filed on September 11, 2015 and U.S. Provisional Patent Application No. 62 / 269,640 filed on December 18, 2015, the disclosures of which are hereby incorporated by reference in their entireties. Technical Field

[0004] The present invention relates generally to robotic and other sorting systems, and more particularly to robotic and other sorting systems intended for use in dynamic environments requiring the system to adapt to handling a variety of objects. Background Art

[0005] Many order fulfillment operations achieve high efficiencies by employing a process known as wave picking. In wave picking, orders are picked from warehouse shelves and placed into bins that contain multiple downstream pick orders. During the picking phase, individual items are identified and multi-item orders are consolidated into a single bin or shelf location so that they can be packaged and shipped to customers. The process of picking these items is done manually. A human picker picks the item from the incoming bin, locates the barcode on the item, scans the barcode with a handheld barcode scanner, determines the appropriate bin or shelf location for the item from the scanned barcode, and then places the item into the determined bin or shelf location where all items for that order are placed.

[0006] Manually operated barcode scanners are typically either fixed or handheld systems. With a fixed system (such as those used at point-of-sale systems), the operator holds the item and positions it in front of the scanner so that the barcode faces the scanning device's sensor, and the scanner scans continuously to decode any barcodes it can detect. If the item is not detected immediately, the person holding the item typically needs to change the position or rotation of the item in front of the fixed scanner to make the barcode more visible to the scanner. With a handheld system, the person operating the scanner looks for the barcode on the item, then holds the scanner so that the item's barcode is visible to the scanner, then presses a button on the handheld scanner to begin scanning the barcode.

[0007] Other ways of identifying items by barcode scanning require that the position of the barcode be controlled or constrained so that a fixed or robotically held barcode scanner can reliably see the barcode. Automatic barcode scanners also involve fixed or handheld systems and the same principles apply. In the case of barcode scanners commonly used in industrial applications, the possible positions of the barcode must be strictly controlled so that the barcode is visible to one or more scanners. For example, one or more barcode scanners can be placed in a fixed position relative to a conveyor or a series of moving pallets so that the scanner can scan an object, usually a box, as the object passes through the scanner. In these devices, the placement range of the barcode is relatively limited because they must be affixed to one of the four sides of the box or on a label on the top, which also needs to be presented in an orientation suitable for scanning. The detected barcode is associated with the direct part of the conveyor or with a specific moving pallet where the object has been placed before scanning.

[0008] In all of these cases, the system uses sensors, either cameras or laser reflectivity sensors, and software to detect the barcode and decode it. These methods have inherent limitations, including the range of distances, relative to the orientation of the detection system, over which they can reliably scan the barcode. First, the barcode must be oriented toward the scanner; second, the range of the barcode must be such that the individual elements can be reliably distinguished; and third, the tilt and skew of the barcode must be such that the individual elements can be reliably distinguished. The type of sensor employed and the robustness of the software detection and decoding schemes determine these performance parameters. Therefore, there remains a need for an object recognition system for a robotic system that can accommodate the automatic recognition and processing of a variety of objects in a variety of orientations. Summary of the invention

[0009] According to one embodiment, the present invention provides a robotic system comprising an articulated arm and a first perception system for inspecting an object and a plurality of additional perception systems, each additional perception system being arranged to face a common area in which the object can be positioned by the robotic arm so that multiple views within the common area can be obtained by the plurality of additional perception systems.

[0010] According to another embodiment, the present invention provides a method of identifying an object in a robotic system including an articulated arm. The method includes the steps of inspecting the object using a first perception system and providing an inspection response signal indicating whether the object has been identified, moving the object to a plurality of additional perception systems in response to the inspection response signal, and inspecting the object using the plurality of additional perception systems, wherein each additional perception system is arranged to be directed toward the object in a common area from a different view of a plurality of views of the object in a common area that can be obtained by the plurality of additional perception systems.

[0011] According to a further embodiment, the present invention provides a perception system for facilitating identification of an object. The perception system includes a plurality of perception units, each perception unit being positioned toward a plurality of locations along an object path that an object may take as it travels through the perception system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The following description may be further understood with reference to the accompanying drawings, in which:

[0013] Figure 1 An illustrative diagrammatic view of a system according to one embodiment of the present invention is shown;

[0014] Figure 2 A plurality of sensing units are shown. Figure 1 an illustrative diagrammatic view of a portion of a system;

[0015] Figure 3 shows an illustrative diagrammatic view of perceived image data of a group of objects to be sorted presented at an infeed station in a system in accordance with an embodiment of the present invention;

[0016] Figure 4 An illustrative diagrammatic view showing a single simulated hold of an object to be identified according to an embodiment of the present invention;

[0017] Figure 5 Shows Figure 4 illustrative diagrammatic views of multiple overlapping simulations of objects maintained;

[0018] Figure 6 An illustrative schematic diagram showing simulated overlapping positions of a number of barcodes;

[0019] Figure 7 Shown based on Figure 4-6 illustrative diagrammatic views of the configuration of the perceptual units of the model;

[0020] Figure 8 shows an illustrative diagrammatic view of a system according to another embodiment of the present invention;

[0021] Fig. 9 Shown according to Figure 8 an illustrative diagrammatic isometric view of a drop sensing unit used by the system; and

[0022] Fig.10 Shows Fig. 9 An illustrative diagram top isometric view of a fall sensing unit.

[0023] The drawings are shown for illustration purposes only. DETAILED DESCRIPTION

[0024] According to one embodiment, the present invention provides an object sensing system for automatically sorting individual objects in a group. In applications such as order fulfillment, objects are collected into heterogeneous groups and need to be sorted. Individual objects need to be identified and then routed to locations specific to the objects. The described system reliably and automatically identifies these objects by employing an automatic barcode scanner and a robotic arm. According to a specific embodiment, for example, the system uses a robotic arm to hold an item in front of one or more barcode scanners so that the object can be scanned. According to various embodiments, the sensing unit (e.g., a camera or scanner) can look for various codes, such as markings (e.g., barcodes, radio frequency tags, stock keeping units (SKUs), universal product codes (UPCs), low wavelength infrared (LWIR), and invisible barcodes and digital watermarks such as Digimarc DWCode).

[0025] While fixed industrial scanners require that the barcode of an object be located where its barcode is visible to the scanner, the robotic arm of the present invention can pick up an object from a heterogeneous collection of objects whose barcodes are not visible and scan the item. The result is an automated barcode scanning system for any item in a heterogeneous stream of objects that can be used to accurately and reliably identify the item.

[0026] Sorting for order fulfillment is one application of automatically identifying objects from a heterogeneous stream of objects. Barcode scanners have a wide range of uses, including identifying stock keeping units of items or tracking packages. The described system can have many uses in the automatic identification and sorting of objects.

[0027] Operating in conjunction with a robotic pick and place system, such a system of an embodiment of the present invention automates part of the sorting process, particularly the step of identifying the object to be picked. Instead of a person picking an object from a bin, a robotic arm can pick the object from the bin, place the object in front of a barcode scanner, and then obtain the object's identification code and place the object in the appropriate bin or shelf location. Since barcode scanners use cameras or lasers to scan 1D or 2D symbols printed on labels attached to items, the barcode must be visible to the scanner's sensor for successful scanning in order to automatically identify an item in a heterogeneous stream of arbitrary items (such as in a jumbled group of items found in a bin).

[0028] Thus, according to various embodiments, the present invention provides a method for determining the identity of an object from a collection of objects, and a method for scanning a barcode of an object using one or more barcode scanners and a robotic arm having an end effector holding the object. According to certain embodiments, the present invention also provides a method for determining the placement of a fixed barcode scanner to maximize the probability of successfully scanning an object held by a robotic end effector, and a method for determining a sequence of placements of a robotic end effector to minimize the time it takes to configure one or more barcode scanners to successfully scan an object, and a method for scanning a barcode of an object by using a barcode scanner as an end effector on a robotic arm.

[0029] An important aspect is the ability to identify objects by their barcodes or other visual markings by using a robotic arm to pick up individual objects and place or drop them in front of one or more scanners. An automated scanning system will not be able to see a barcode on an object that is presented in such a way that the barcode is not exposed or visible. Since the system uses a robot to hold the object, it can manipulate the object to make the barcode visible, or use multiple scanners to view the item from multiple viewpoints to obtain the barcode regardless of how the object is held.

[0030] Figure 1 FIG. 4 shows an automatic object recognition system according to an embodiment of the present invention. Figure 1 A robotic system 10 is shown which includes an articulated arm 12 including an end effector 14 and articulated portions 16, 18 and 20. The articulated arm 12 selects items from a conveyor 22, which are located in bins on the conveyor 22 or on the conveyor itself. A support 24 includes an attached first detection unit 26 which faces the conveyor from above. The perception system may be a perception unit 26, such as a camera or scanner (such as a laser reflectivity scanner or other type of barcode reader) or a radio frequency ID scanner. A plurality of additional perception units may be provided on the perception system 28 (as will be described below with reference to Figure 2 discussed in more detail).

[0031] The robotic system 10 may further include a robotic environment, including a target station 30 of a plurality of bins 32 in which objects may be placed after identification. A central computing and control system 34 may communicate with the perception unit 26 and the perception system 28 and the articulated arm 12 via wireless communications, or in some embodiments, the central computing and control system 34 may be disposed in the base portion 20 of the articulated arm.

[0032] Figure 2A perception system 28 is shown for use with certain embodiments of the present invention including a plurality of perception units 40, 42, 44, 46, and 48 and a plurality of illumination sources 50, 52, 54, and 56. Each of the perception units 40, 42, 44, 46, and 48 may be, for example, a camera (e.g., 2D or 3D), or a scanner such as a laser reflectivity scanner or other type of barcode reader (e.g., 1D or 2D barcode scanner) or a radio frequency ID scanner and associated software to process the perception data.

[0033] In general, the system provides, in certain embodiments, an automatic object identification system that includes a robotic pick and place system capable of picking up objects, moving objects in space, and placing them. The system also includes a set of objects to be identified, the manner in which the inbound objects are organized, typically in a heterogeneous pile in a bin or a column on a conveyor, the manner in which the outbound objects are organized, typically in an array of outbound bins or shelf compartments, the manner in which the objects are tagged with bar codes or radio frequency identification tags, a fixed main scanner that operates above the incoming stream of objects, a bar code scanning station in which one or more bar code scanners and illuminators are activated when an object is held at the station, and a central computing and control system that determines the appropriate location to place the object, depending on the decoded bar code of the object.

[0034] As described above, a robotic pick and place system is typically a robot arm equipped with sensors and a computer, which when combined is assumed to have the following capabilities: (a) it is able to pick objects from a specified class of objects and separate them from a heterogeneous stream of objects, whether they are mixed in a box or separated on a motorized or gravity conveyor system, (b) it is able to move objects to arbitrary locations within its workspace, (c) it is able to place objects in outbound bin or shelf locations in its workspace; and (d) it is able to generate a map of the objects it is able to pick up, represented in the workcell as a set of candidate grip points and a list of polyhedrons containing the objects in the space.

[0035] The permissible objects are determined by the capabilities of the robotic pick and place system. Their size, weight, and geometry are assumed to enable the robotic pick and place system to pick up, move, and place them. These can be any kind of ordered goods, packages, parcels, or other items that benefit from automated sorting. In some embodiments, each object is associated with a stock keeping unit (SKU) that identifies the item.

[0036] For example, the inbound object can arrive in one of two configurations: (a) Figure 3 As shown in (a), inbound objects are stacked in a bin of heterogeneous objects; or (b) inbound objects are delivered by a moving conveyor. Figure 3As shown, the collection of objects includes some objects with exposed bar codes as shown at 60, 62, 64, 66, 68, 70 and 72 and other objects without exposed bar codes. It is assumed that the robotic pick and place system is capable of picking items from a bin or conveyor. The inbound object stream is the sequence of objects unloaded from a bin or conveyor.

[0037] Outbound objects are organized in such a way that items are placed in bins, shelf locations or rooms or other destination locations where all objects corresponding to a given order are consolidated. These outbound destinations may be arranged in a vertical array, horizontal array, grid or some other regular or irregular manner, but the arrangement is known to the system. It is assumed that the robotic pick and place system is able to place objects to all outbound destinations and determine the correct outbound destination from the object's SKU.

[0038] Objects are assumed to be marked with a visually unique marking, such as a barcode or radio frequency identification (RFID) tag or other unique identifier, at one or more locations on their exterior so that they can be identified by a scanner. The type of marking depends on the type of scanning system used, but may include 1D or 2D barcode symbologies. A variety of symbology or labeling methods may be used. The type of scanner used is assumed to be compatible with the labeling method. A symbol string, typically a string of letters and numbers, is encoded by the barcode, RFID tag or other means of marking. The symbol string uniquely associates the object with a SKU.

[0039] The main perception unit 26 is a device mounted above the area where inbound objects arrive, which scans all inbound objects to identify features. When an inbound object arrives at the box, the main perception unit is mounted above the box. The main perception unit consists of one or more devices that are capable of recognizing visually distinct markings, which may include a barcode or other identifying features, or an RFID tag on an object. The unit components may include a camera, an RFID scanner, an illuminator, and a decoding microprocessor. The main perception unit first identifies the items in the box. The main perception unit localizes any code it finds, and the robotic pick and place system is assumed to be able to use that location to pick up an item with a given code.

[0040] The perception system 28 (e.g., a secondary scanner) is a component that scans any objects that the primary perception unit cannot scan between inbound objects, or scans objects to confirm the results of the primary perception unit. In another embodiment, the secondary scanner can be used to detect any additional markings that can confirm the identification or can detect that more than one object has been grasped, in which case both are returned to the input inbound area. Inbound objects in boxes or on conveyors may have labels or markings that are partially or completely obscured by other objects, or the labels or tags may not be facing the scanner. The secondary perception system is installed in the work area of ​​the robotic pick and place system so that unidentified items can be presented to the secondary scanner. Like the primary perception unit, the secondary perception system consists of one or more perception devices, which may include cameras, RFID scanners, illuminators, and decoding microprocessors.

[0041] The operation of the above system is coordinated by a central control system 34. The system determines the SKU associated with the object and the outbound destination of the object based on the sensed data (e.g., symbol string). The central control system consists of one or more workstations or central processing units (CPUs). The correspondence between SKUs and outbound destinations is maintained by the central control system in a database, called a manifest. The central control system maintains the manifest by communicating with a warehouse management system (WMS).

[0042] During operation, the broad workflow is typically as follows. First, the system is equipped with a manifest that provides an outbound destination for each inbound object. Next, the system waits for the inbound objects to arrive at a bin or conveyor. When the robotic pick and place system identifies the presence of one or more inbound objects, the central control system instructs the master perception unit to scan the inbound objects. The master perception unit creates a list of detected tags, which includes their symbol strings and their locations in the workspace. The master perception unit sends this list to the central control system.

[0043] The central control system receives sensing information including detected markers from the main sensing unit and receives a list of unrecognized but pickable objects from the robotic pick and place system. The position coordinates in the two lists are registered with each other using calibration information about the main scanner and the robotic pick and place system. The central control system uses the mapping generated by the robotic pick and place system to determine the objects surrounding each detected marker by simple geometric methods. Therefore, the central control system associates for each marker the object it corresponds to. This step is based on data association of the markers. The central control system sorts each candidate item to be picked up according to a heuristic (such as selecting the topmost object in the pile) to generate a candidate pick list.

[0044] again, Figure 3An example of a marker-based data association between an object identified by a robotic pick and place system and a marker detected by a perception unit is shown. In this example, a barcode symbol (bounding quadrilateral) is associated with an unidentified object (shaded portion) where the barcode symbol is located.

[0045] If there is at least one candidate pick associated with a tag in the candidate pick list, the system picks the highest ranked pick. The expectation is that the object picked will correspond to a tag previously associated by the central control system and detected by the primary perception unit. The central control system performs a check assuming that there can be errors in this association. After the object has been removed and separated from the inbound object set, the system instructs the primary perception unit to scan the inbound items again. If the correct object was removed, the tag associated with it should no longer be in the list of tags detected by the primary perception unit. However, if the tag associated with the picked object is still there, then it must have picked the wrong object. If it picked the wrong object, then it puts the object back and repeats the process of generating pick candidates from the robotic pick and place system and the primary scanner.

[0046] If there is no candidate selection associated with a tag, the object associated with the highest ranked selection is selected. Since there is no tag associated with the object, it is an unrecognized object. After the robotic pick and place system picks up an item from the inbound object collection, the central control system instructs the robotic pick and place system to move the object to the secondary perception system for scanning. The central control system instructs the secondary perception system to scan for unrecognized objects.

[0047] If the secondary perception system successfully identifies the marking on the object, the object is then identified and the central control system commands the robotic pick and place system to deliver the item to the outbound destination determined from the SKU, which itself was determined from the detected marking.

[0048] If the secondary perception system cannot recognize the tag on the object, then, depending on the configuration of the scanner, the central control system may command the robotic pick and place system to rotate the object in an attempt to make the tag visible to the scanner and scan the object again. This may occur a number of times to ensure that if the tag is on the object, it will be detected. The sequence of positions and orientations of the objects is selected to minimize the average or maximum amount of time required for the secondary scan. If the object cannot be identified, or the secondary perception system detects a mismatched product identifier (possibly indicating a double pick), the object may be diverted to a special outbound destination for unidentified objects, or may be returned to the inbound stream.

[0049] The whole process runs in a loop until all objects in the inbound collection are exhausted. Objects in the inbound stream are automatically identified, sorted, and routed to the outbound destination.

[0050] Thus, according to one embodiment, the present invention provides a system for sorting objects arriving from an inbound bin and needing to be placed in an outbound bin, wherein the sorting will be based on barcode symbols. In this embodiment, the primary and secondary sensing systems are capable of detecting and decoding barcode symbols.

[0051] The key feature of this embodiment is the specific design of the primary and secondary perception systems to maximize the probability of successful scanning while minimizing the average scanning time. The probability of successful scanning and the average scanning time constitute the key performance characteristics. These key performance characteristics depend on the configuration and properties of the primary and secondary perception systems, as well as the set of objects and how they are labeled.

[0052] Two key performance characteristics can be optimized for a given set of items and a method of barcode labeling. The parameters optimized for the barcode system include the number of barcode scanners, including where and in what orientation they are placed, and the sensor resolution and field of view used by the scanners. In some embodiments, the optimization can be done by simulation using a model of the object.

[0053] The optimization through simulation uses a barcode scanner performance model. The performance model of a barcode scanner is the range of positions, orientations, and barcode element sizes over which a barcode symbol can be detected and decoded by a barcode scanner, where the barcode element size is the size of the smallest feature on the barcode. These are typically rated as minimum and maximum ranges, maximum skew angle, maximum pitch angle, and minimum and maximum tilt angles.

[0054] If the barcode scanner and symbol are positioned upright, and the barcode symbol faces the scanner so that the symbol is parallel to the sensor side of the scanner, the barcode symbol lies in what is called a front-parallel plane. The angle between the front-parallel plane and a plane rotated about a vertical axis is the skew angle. The angle between the front-parallel plane and a plane rotated about a horizontal axis is the pitch axis. The angle made by features on the front-parallel plane when rotated about an axis perpendicular to the front-parallel plane is the tilt axis.

[0055] The typical performance of camera-based barcode scanners is that they are able to detect barcode symbols at a range of distances as long as the pitch and yaw of the plane of the symbol are within plus or minus 45 degrees, while the tilt of the symbol can be arbitrary (between 0 and 360 degrees). The barcode scanner performance model predicts whether a given barcode symbol in a given position and orientation will be detected.

[0056] The barcode scanner performance model is combined with a model of the expected position and orientation of the barcode. The barcode symbol pose model is the range of all positions and orientations, in other words the poses in which the barcode symbol will be expected to be found. For the secondary scanner, the barcode symbol pose model is itself a combination of the item grasping model (which predicts how the robotic pick and place system will hold the object) and the barcode item appearance model (which describes the possible positions of the barcode symbol on the object). For the primary scanner, the barcode symbol pose model is itself a combination of the barcode item appearance model and the inbound object pose model, which models the distribution of poses that the inbound items present to the primary scanner. These models can be built empirically, using analytical modeling, or approximate models can be used, using a sphere model as the object and a uniform distribution on the sphere as the barcode item appearance model.

[0057] For example, in one embodiment, two objects, a bottle and a toothpaste container, represent a collection of objects, and barcode symbols are placed at fixed and known locations on all instances of these objects. Using the 3D models of these two objects, known functions of a robotic pick-and-place system are used to generate a random sample of object poses. These poses are associated with the end effector (usually a gripper) of the robotic pick-and-place system.

[0058] Figure 4 An example of a single simulated holding of a bottle 80 held by an end effector 82 of a vacuum gripper based robotic pick and place system is shown. Figure 5 In 84 it is shown Figure 4 Overlapping samples of bottle 80 are shown. Using these samples, and because the bar code symbols are located at fixed locations on both items, a set of poses for the bar code symbols are generated. Figure 6 An example of a quadrilateral representation of 100 sample bar code symbols is shown at 86. This sample represents the bar code symbol pose model for the secondary scanner. It is an approximation of the probability distribution of where a bar code is expected to be found when a robotic pick and place system holds an item.

[0059] With these models, the barcode symbology pose model and the barcode scanner performance model, all parameters of the system can be optimized.

[0060] Figure 7 FIG. 2 shows the final configuration determined by simulation of the above model, where the secondary scanner system is determined for an example where the items are bottles and toothpaste containers. Figure 7 As shown, the system includes scanners 90, 92, 94, 96 and 98, each scanner being generally oriented toward object area 88, with the position of each scanner 90-98 being selected to provide an optimal plane, angle, tilt and view for the object under consideration. Figure 7A mechanical model for implementing a secondary scanner is shown. In this example, the optimization criterion is the probability of a successful scan. In an example where only one scanner can be used, the optimization criterion may be the average scan time, in which case the optimization is over the sequence of poses for presenting the article to the secondary scanner such that the average scan time is reduced overall.

[0061] Thus, according to a further embodiment, the present invention may be used in conjunction with an object sorting system that produces a large (and very flexible) number of total collection bins, a very low per-bin transfer cost, a throughput as high as that of a manual system, and a much smaller need for manual operations.

[0062] For example, Figure 8 A system 200 is shown, which includes an articulated arm 202 with an end effector 204, an input area 206 in which objects are presented for sorting, a primary perception system (e.g., perception unit) 214 such as a camera for identifying objects to be sorted, and a receiving conveyor 208 for receiving objects to be sorted from either a human worker, another conveyor, or an input tray. The system also includes an unsortable output chute 210 that leads to an unsortable output bin 212 for providing objects that the system cannot identify or cannot sort for any other reason (e.g., cannot grasp or pick up).

[0063] In addition to the main perception unit 214, the system also includes a drop perception system 216, which includes an open top and an open bottom, and multiple perception units (e.g., cameras or sensors as discussed above with reference to the previous embodiments) are located within the perception system 216, which are aimed at the top, middle and lower center areas of the interior of the perception system 216. The multiple perception units (e.g., cameras) record perception data (e.g., images) of the objects as they are dropped by the end effector through the perception system 216. The drop perception system 216 can also include one or more sensors (e.g., laser sensors) located at the top of the system 216, which detect when an object falls into the drop perception system 216. The multiple perception units are designed to collect multiple images of each object from multiple views to help identify or confirm the identity of the dropped object.

[0064] The fallen objects then fall into a first bracket 218 disposed on a track 220 , and a conveyor 218 can automatically move between a first sorting station 222 and a second sorting station 224 on either side of the object falling area.

[0065] The first sorting station 222 includes a second bracket 226 that can receive objects from the first bracket 218 and travel along a track between two rows of collection boxes 228, and the objects can be dumped into the collection boxes along a guide wall 230. The second sorting station 224 includes a third bracket 232 that can receive objects from the first bracket 218 and travel along a track between two rows of collection boxes 234, and the objects can be dumped into the collection boxes along a guide wall 236.

[0066] Figure 8 The system shown has two shuttle sorting wings. When an object is picked up from the feed conveyor, it is dropped onto the first shuttle sorter 218. The shuttle sorter delivers the object to one of the two wings, places the object in a carrier in that wing, and then moves back home. Due to the limited travel, this reciprocating operation can be performed in the time it takes for an articulated arm to pick up another object (assuming the articulated arm picks up objects at a human rate that approximates throughput).

[0067] The fall sensing system 216 includes a plurality of detection units (e.g., cameras or scanners as described above) oriented toward the central path, such that when an object falls into the unit 216, multiple views of the object will be captured by the plurality of detection units. The fall sensing system may also include the above referenced Figure 2 The system discussed the lights.

[0068] Fig. 9 and 10 The detection unit 300 is shown at different angles toward the center path of the fall sensing system 216. As also shown, the laser source bar 302 can direct laser illumination toward the sensor bar 304 so that the fall sensing system 216 can accurately detect when an object enters the system 216. Fig.10 At 306 a graphical view of a plurality of detector angles and fields of view for each of a plurality of detectors is shown.

[0069] The plurality of additional perception systems may be positioned so that each surface of the object can be perceived regardless of the orientation of the object. In some embodiments, the first perception system may provide perception data regarding a unique object identifier, and the plurality of additional perception systems may provide additional perception data regarding any one of the confirmed object identifier or any additional object identifiers. In further embodiments, the first perception system may provide perception data regarding the shape of the object, and the plurality of additional perception systems may provide additional perception data regarding a unique object identifier that matches the shape of the object.

[0070] Those skilled in the art will appreciate that many modifications and variations can be made to the embodiments disclosed above without departing from the spirit and scope of the present invention.

Claims

1. An object processing system for identifying an object, the object processing system comprising: a first identity capture device for providing first identity data associated with a first view of an object at a first location; a plurality of additional identity capture devices arranged toward a perception area to provide further identity data relating to a plurality of views of the object within the perception area, the plurality of views providing views of all surfaces of the object; as well as an end effector for acquiring the object for movement from the first position to the sensing area if the identity of the object cannot be determined solely by using the first identity data; a routing system for receiving the object and for directing the object towards one of a plurality of destination locations based on the further identity data, The object is guided along a first path in a first linear direction and subsequently along one of a plurality of second paths selected based on the further identity data, each of the plurality of second paths extending in a linear direction non-parallel to the first path.

2. The object handling system of claim 1, wherein the plurality of additional identity capture devices comprises a plurality of cameras.

3. The object handling system of claim 2, wherein the plurality of cameras comprises five cameras, the five cameras being collectively arranged to simulate holding the object in at least four different poses.

4. An object handling system as claimed in claim 2, wherein the plurality of cameras are provided in a drop camera unit into which objects can be dropped and scanned by the plurality of cameras as the objects fall through the unit.

5. The object processing system of claim 1, wherein the plurality of additional identity capture devices comprises a plurality of scanners.

6. The object processing system of claim 5, wherein each of the plurality of scanners is a laser reflectivity scanner.

7. The object processing system of claim 5, wherein each of the plurality of scanners is a radio frequency ID scanner.

8. The object handling system of claim 5, wherein the plurality of scanners comprises five scanners, the five scanners being collectively arranged to simulate holding the object in at least four different poses.

9. The object processing system of claim 1, wherein one or more of the plurality of second paths are orthogonal to the first path.

10. The object processing system of claim 1, wherein the plurality of second paths are parallel to each other.

11. The object handling system of claim 1 , wherein the routing system comprises a reciprocating carriage.

12. The object handling system of claim 11, wherein the plurality of destination locations includes two rows of destination boxes, and wherein the shuttle carriage travels between the two rows of destination boxes.

13. The object handling system of claim 9, wherein the routing system comprises a plurality of reciprocating carriages.

14. The object handling system of claim 13, wherein one of the plurality of reciprocating carriages moves along the first path, and the other carriages of the plurality of reciprocating carriages each move along a corresponding one of the plurality of second paths.

15. A method for processing a plurality of different objects, the method comprising: presenting the plurality of different objects at a first position with each of the plurality of different objects contacting at least one other of the plurality of different objects; providing first sensing information by a first sensing unit, the first sensing unit scanning the plurality of different objects at the first position; Obtaining an identification mark from the first perception information; associating a first object included in the plurality of different objects with one of the identification marks from the first perception information; Grasping the first object included in the plurality of different objects using an end effector of a programmable motion device; separating the first object from the plurality of different objects using the programmable motion device; providing second perception information by the first perception unit scanning the plurality of different objects after the first object is separated from the plurality of different objects; determining whether an identification tag associated with the first object is present in the second sensory information; as well as The first object is processed based on the determination.

16. The method of claim 15, further comprising: determining from the first perception information that a second object does not have an identification mark included in the plurality of different objects; separating the second object from the plurality of different objects using the programmable motion device; The second object is presented using the programmable motion device to a second perception unit comprising a plurality of identity capture devices, wherein the plurality of identity capture devices are positioned in a generally bowl shape.

17. The method of claim 15, further comprising: determining from the first perception information that a second object does not have an identification mark included in the plurality of different objects; separating the second object from the plurality of different objects using the programmable motion device; dropping the second object from the end effector of the programmable motion device through a drop scanner, the drop scanner comprising an open top and an open bottom; scanning each surface of the second object for identification marks using the drop scanner as the second object drops through the drop scanner; as well as The second object is routed toward a destination location associated with the identification tag provided by the drop scanner.

18. The method of claim 17, wherein dropping the second object from the end effector of the programmable motion device through the drop scanner comprises: Illumination of the second object is provided by a plurality of illumination sources.

19. The method of claim 15, wherein the plurality of different objects at the first location includes both objects having exposed identification indicia and objects without exposed identification indicia.

20. The method of claim 15, wherein the method further comprises the steps of: At least two objects of the plurality of different objects are sorted with respect to a picking order.

21. The method of claim 15, wherein the step of grasping the first object comprises: The first object is grasped without identifying the first object.

22. The method of claim 15, wherein the method further comprises the steps of: The first object is rotated using the end effector of the programmable motion device.

23. The method of claim 16, wherein the method further comprises the steps of: A simulation of a pose of the first object is provided to optimize placement of the plurality of identity capture devices.

24. The method of claim 23, wherein the step of providing a simulation of the pose of the first object involves use of a bar code scanner performance model.

25. The method of claim 24, wherein the bar code scanner performance model incorporates pitch and skew of the plane of the bar code symbol.

26. A method of processing a plurality of different objects, the method comprising: presenting the plurality of different objects at a first position with each of the plurality of different objects contacting at least one other of the plurality of different objects; providing first sensing information by a first sensing unit, the first sensing unit scanning the plurality of different objects at the first position; determining from the first perception information that the first object does not have an identification mark included in the plurality of different objects; grasping the first object from the plurality of different objects using an end effector of a programmable motion device; separating the first object from the plurality of different objects using the programmable motion device; presenting the first object to a second perception unit including a plurality of identity capture devices using the programmable motion device; scanning each surface of the first object for identification marks using the plurality of identity capture devices of the second sensing unit; as well as The first object is routed toward a destination location associated with the identification provided by the second perception unit.

27. The method of claim 26, wherein the plurality of identity capture devices included in the second sensing unit are positioned in a generally bowl shape.

28. The method of claim 26, wherein presenting the first object to the plurality of identity capture devices comprises: The first object is dropped from the end effector into a drop scanner including an open top and an open bottom.

29. The method of claim 26, wherein presenting the first object to the plurality of identity capture devices comprises: Illumination of the first object is provided by a plurality of illumination sources.

30. The method of claim 26, wherein the plurality of different objects at the first location includes both objects having exposed identification indicia and objects without exposed identification indicia.

31. The method of claim 26, wherein the method further comprises the steps of: At least two objects of the plurality of different objects are sorted with respect to a picking order.

32. The method of claim 26, wherein the method further comprises the steps of: The first object is rotated using the end effector of the programmable motion device.

33. The method of claim 26, wherein the method further comprises the steps of: A simulation of a pose of the first object is provided to optimize placement of the plurality of identity capture devices.

34. The method of claim 33, wherein the step of providing a simulation of the pose of the first object involves use of a bar code scanner performance model.

35. The method of claim 34, wherein the bar code scanner performance model incorporates pitch and skew of the plane of the bar code symbol.

36. A method of processing a plurality of different objects, the method comprising: presenting the plurality of different objects at a first position with each of the plurality of different objects contacting at least one other of the plurality of different objects; providing first sensing information by a first sensing unit, the first sensing unit scanning the plurality of different objects at the first position; obtaining a plurality of unique markers and a position associated with each unique marker detected in the first sensory information; generating a map of the plurality of different objects; associating each unique marker detected in the first sensory information with an object located in the map based on the location of the unique marker; locating in the map a first object that is not associated with any unique marker detected in the first sensory information; Grasping the first object included in the plurality of different objects using an end effector of a programmable motion device; separating the first object from the plurality of different objects using the programmable motion device; dropping the first object from the end effector of the programmable motion device through a drop scanner, the drop scanner comprising a plurality of identity capture devices; scanning each surface of the first object for identifying indicia using the plurality of identity capture devices as the first object drops through the drop scanner; as well as The first object is routed toward a destination location associated with the identification mark provided by the drop scanner.

37. The method of claim 36, wherein the step of dropping the first object from the end effector of the programmable motion device through the drop scanner comprises: Illumination of the first object is provided by a plurality of illumination sources.

38. The method of claim 36, wherein the plurality of different objects at the first location includes both objects having exposed identification indicia and objects without exposed identification indicia.

39. The method of claim 36, wherein the method further comprises the steps of: associating a second object included in the plurality of different objects with one of the unique tags detected in the first perception information provided by the first perception unit; Grasping the second object included in the plurality of different objects using an end effector of the programmable mobile device; separating the second object from the plurality of different objects using the programmable mobile device; providing second perception information by the first perception unit scanning the plurality of different objects after the second object is separated from the plurality of different objects; In response to determining that the unique tag associated with the second object is absent in the second perception information of the plurality of different objects after the second object is separated from the plurality of different objects, the second object is routed toward a destination associated with the unique tag associated with the individual second object.

40. The method of claim 39, wherein the method further comprises the steps of: In response to the unique tag associated with the second object being present in the second perception information of the plurality of different objects after the second object is separated from the plurality of different objects, the second object is returned to the plurality of different objects.