Methods and systems for handling stock keeping units

Abstract

Disclosed herein is a shoebox gripper including: (A) a main body; (B) an adapter at a first distal end of said main body; (C) a pair of grippers at a second distal end of said main body, wherein said pair of grippers comprises a first suction mat and a second suction mat, wherein said first suction mat is substantially perpendicular to said second suction mat; and (D) one or more vacuum tubings gaseously coupled to said pair of grippers.

Description

65057 / 2024 / PMiMETHODS AND SYSTEMS FOR HANDLING STOCK KEEPING UNITSBACKGROUND

[0001] Order picking may be a process of selecting and gathering items from a warehouse or storage location. Items may be selected and gathered for various reasons. For example, items may be selected and gathered for fulfilling customer orders or for storing. Items may often be moved while in a stock keeping unit (SKU). SKUs may include any container for holding the item. One example of a SKU is a shoebox.

[0002] Selecting and gathering items may include retrieving the items from respective locations based on the order details, item type, quantity, item attributes, specific customer requirements, etc. While item picking has historically relied on manual labor, where workers navigated a warehouse to locate and retrieve items, certain automations have become popular in more recent times. For example, warehouse automation technologies have revolutionized the order picking process through technologies such as automated guided vehicles, robotic arms, conveyor belts, software algorithms, etc.SUMMARY

[0003] Systems, methods, computer-readable media, and techniques are disclosed herein for handling stock keeping units, including shoeboxes. The geometry of a shoebox, with opening lids (e.g., hinged lids or unhinged lids) presents challenges in handling shoeboxes without allowing the lid to open. The systems, the methods, the computer-readable media, and the techniques disclosed herein address the challenges in handling shoeboxes (or similar SKUs) via a unique gripper for transferring, sorting, order preparations, returns, etc. in the warehouse stock management process.

[0004] In one aspect, method for warehouse management, comprises: (A) identifying, in an infeed system comprising a plurality of stock keeping units that comprises one or more shoeboxes and one or more non-shoeboxes, a shoebox of the one or more shoeboxes; (B) at least in part in response to identifying the shoebox at (A), configuring a picker with a shoebox gripper; and (C) causing the picker to handle the shoebox using the shoebox gripper. In some embodiments, the infeed system comprises an unstructured source comprising the plurality of stock keeping units. In some embodiments, the unstructured source comprises one or more bins comprising the plurality of stock keeping units. In some embodiments, the one or more non-shoeboxes comprise one or more fashion stock keeping units. In some embodiments, the one or more fashion stock keeping units comprise one or more of: garment stock keeping units, handbag or backpack stock keeping units, or hat stockNomagic Inc. 59332-706.601 Application Specification.docx -1- WSGR Docket No. 59332-705.60165057 / 2024 / PMi keeping units. In some embodiments, identifying the shoebox at (A) comprises: (i) obtaining image data corresponding to the shoebox; and (ii) applying a machine learning model to the image data. In some embodiments, the code comprises one or more of: a one-dimensional code, a two-dimensional code, or a three-dimensional code. In some embodiments, causing the picker to handle the shoebox using the shoebox gripper at (C) comprises: causing the picker to move, using the shoebox gripper, the shoebox from the infeed system to an outfeed system. In some embodiments, the anomaly comprises damage to the shoebox. In some embodiments, the anomaly comprises dropping the shoebox while causing the picker to handle the shoebox using the shoebox gripper at (C). In some embodiments, the anomaly comprises the picker handling an additional stock keeping unit in addition to the shoebox while causing the picker to handle the shoebox using the shoebox gripper at (C). In some embodiments, the anomaly comprises a lid of the shoebox opening while causing the picker to handle the shoebox using the shoebox gripper at (C). In some embodiments, detecting the anomaly at (E) comprises: (i) obtaining image data corresponding to the shoebox; and (ii) applying a machine learning model to the image data. In some embodiments, the shoebox gripper comprises a first contact configured to contact a first surface of the shoebox and a second contact configured to contact a second surface of the shoebox, wherein the first surface of the shoebox is adjacent to the second surface of the shoebox. In some embodiments, the first surface of the shoebox is substantially perpendicular to the second surface of the shoebox. In some embodiments, the first contact comprises a first suction mat and the second contact comprises a second suction mat. In some embodiments, the first suction mat is gaseously coupled to a first vacuum tubing and the second suction mat is gaseously coupled to a second vacuum tubing. In some embodiments, the first surface of the shoebox comprises a lid of the shoebox. In some embodiments, the second surface of the shoebox comprises a body of the shoebox.

[0005] In another aspect, a shoebox gripper, comprises: (A) a main body; (B) an adapter at a first distal end of the main body; (C) a pair of grippers at a second distal end of the main body, wherein the pair of grippers comprises a first suction mat and a second suction mat, wherein the first suction mat is substantially perpendicular to the second suction mat; and (D) one or more vacuum tubings gaseously coupled to the pair of grippers. In some embodiments, the first suction mat is configured to contact a body of a shoebox or a lid of a shoebox. In some embodiments, the first suction mat and the second suction mat have a combined surface area of less than 0.5 square meters. In some embodiments, the one or more vacuum tubings are configured to provide a lifting force of less than about 3 kilograms. In some embodiments, the pair of grippers are configured to pick a shoebox out of an infeed system that comprises an unstructured source comprising a plurality of stock keepingNomagic Inc. 59332-706.601 Application Specification.docx -2- WSGR Docket No. 59332-705.60165057 / 2024 / PMi units that comprises the shoebox and one or more non-shoeboxes, and wherein the pair of grippers are configured to deposit the shoebox into an outfeed system. In some embodiments, the one or more non-shoeboxes comprise one or more fashion stock keeping units. In some embodiments, the one or more fashion stock keeping units comprise one or more of: garment stock keeping units, handbag or backpack stock keeping units, or hat stock keeping units. In some embodiments, the adapter is configured to detachably affix to a robotic arm. In some embodiments, the robotic arm comprises one or more optical sensors. In some embodiments, the robotic arm further comprises a control system configured to: (i) obtain image data from the one or more optical sensors; (ii) apply a machine learning model to the image data to identify a shoebox; and (iii) at least in part in response to identifying the shoebox, configure the robotic arm to detachably affix to the adapter. In some embodiments, the first suction mat is gaseously coupled to a first vacuum tubing of the one or more vacuum tubings and the second suction mat is gaseously coupled to a second vacuum tubing of the one or more vacuum tubings.

[0006] In another aspect, a system for warehouse management, comprises: (A) one or more optical sensors configured to collect image data of an infeed system comprising a plurality of stock keeping units that comprises one or more shoeboxes and one or more non-shoeboxes; (B) a computing device configured to identify, from the image data, a shoebox of the one or more shoeboxes; and (C) a control device configured to, at least in part in response to the computing device identifying the shoebox, send a control signal to cause (i) configuring a picker with a shoebox gripper, and (ii) the picker to handle the shoebox using the shoebox gripper. In some embodiments, the infeed system comprises an unstructured source comprising the plurality of stock keeping units. In some embodiments, the unstructured source comprises one or more bins comprising the plurality of stock keeping units In some embodiments, the one or more non-shoeboxes comprise one or more fashion stock keeping units. In some embodiments, the one or more fashion stock keeping units comprise one or more of: garment stock keeping units, handbag or backpack stock keeping units, or hat stock keeping units. In some embodiments, the computing device is configured to identify the shoebox by: (i) obtaining image data corresponding to the shoebox; and (ii) applying a machine learning model to the image data. In some embodiments, the control device is further configured to cause the picker to: (D) position the shoebox to have a code corresponding to the shoebox scanned, wherein the code comprises one or more of: a one-dimensional code, a two-dimensional code, or a three-dimensional code. In some embodiments, handling the shoebox using a shoebox gripper comprises moving the shoebox into an outfeed system. In some embodiments, the shoebox gripper comprises a first contact configured to contact a first surface of the shoebox and a second contact configured to contact aNomagic Inc. 59332-706.601 Application Specification.docx -3- WSGR Docket No. 59332-705.60165057 / 2024 / PMi second surface of the shoebox, wherein the first surface of the shoebox is adjacent to the second surface of the shoebox. In some embodiments, the first surface of the shoebox is substantially perpendicular to the second surface of the shoebox. In some embodiments, the first contact comprises a first suction mat and the second contact comprises a second suction mat. In some embodiments, the first suction mat is gaseously coupled to a first vacuum tubing and the second suction mat is gaseously coupled to a second vacuum tubing. In some embodiments, the first surface of the shoebox comprises a lid of the shoebox. In some embodiments, the second surface of the shoebox comprises a body of the shoebox. In some embodiments, the picker comprises a robotic arm.

[0007] Another aspect of the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto. The computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements any of the methods above or elsewhere herein.

[0008] Additional aspects and advantages of the present disclosure will become readily apparent to those skilled in this art from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As will be realized, the present disclosure is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.INCORPORATION BY REFERENCE

[0009] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent publications and patents or patent applications incorporated by reference contradict the disclosure contained in the specification, the specification is intended to supersede or take precedence over any such contradictory material.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings (also “Figure” and “FIG.” herein), of which:Nomagic Inc. 59332-706.601 Application Specification.docx -4- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0011] FIG. 1 shows an example of an operation flowchart for implementing stock keeping unit handling;

[0012] FIG. 2A shows a front view of an example of a shoebox gripper;

[0013] FIG. 2B shows a left view of the example of the shoebox gripper of FIG. 2A;

[0014] FIG. 3A shows a perspective view of an example of a shoebox gripper gripping a shoebox in a first orientation;

[0015] FIG. 3B shows a perspective view of an example of the shoebox gripper of FIG. 3A gripping a shoebox in a second orientation;

[0016] FIG. 3C shows a perspective view of an example of the shoebox gripper of FIG. 3A gripping a shoebox in a third orientation; and

[0017] FIG. 4 shows an example of a computer system that is programmed or otherwise configured to implement methods provided herein.DETAILED DESCRIPTION

[0018] While various embodiments of the invention have been shown and disclosed herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions may occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention disclosed herein may be employed.

[0019] As discussed in the Background Section, warehouse automation is a growing area of interest. There exist solutions for handling shoeboxes from structured sources including conveyor systems, robots with palletizer grippers or robots with slider grippers. A structured source is one in which the placement of shoeboxes, the position of shoeboxes, and the orientation of shoeboxes is predictable and at least roughly known upfront. Palletizer grippers typically include two moveable surfaces that can squeeze a shoebox for picking. Slider grippers typically include moveable suction cups that can pull a shoebox onto a slider and then lift it. However, current automated picking technologies suffer from various drawbacks. For example, none of those solutions allow for picking shoeboxes from unstructured sources with possibility to approach the shoeboxes from the top. Because there are currently no automated methods to pick shoeboxes from unstructured sources, this work is often done by human hands in logistic processes where source structurization is not possible.

[0020] Systems, methods, computer-readable media, and techniques are disclosed herein to provide a new functionality to logistic processes for handling shoebox stock keeping units (SKUs) from unstructured sources. Advantageously, the systems, the methods, the computer-readable media, andNomagic Inc. 59332-706.601 Application Specification.docx -5- WSGR Docket No. 59332-705.60165057 / 2024 / PMi the techniques disclosed herein for handling shoeboxes are designed to improve time efficiency, order accuracy, inventory management, scalability, flexibility, cost efficiency, etc. when used with robotics.

[0021] The systems, the methods, the computer-readable media, and the techniques disclosed herein provide robotics configured to handle fashion SKUs including shoeboxes. The robotics may include am infeed system, an outfeed system, a robotic arm with a gripper changer, one or more dedicated grippers including a shoebox gripper, an optical system, and a control system leveraging artificial intelligence enabled software. One example of the systems, the methods, the computer-readable media, and the techniques disclosed herein includes the process of: (A) a batch of fashion SKUs (including shoeboxes) arrives through an infeed system; (B) a control system, using optical sensors to collect image data and machine learning models to analyze the image data, determines the next SKU instance to be handled; (C) the control system selects an appropriate gripper; (D) a robotic arm is configured with the appropriate gripper (e.g., a shoebox gripper); (E) the robotic arm, with the appropriate gripper, performs picking, optional scanning and placing of the SKU into the outfeed system. This process may be repeated for each SKU instance in a batch (e.g., in a bin), or stopped after a defined number of cycles has been completed. During each cycle of the process, the robotics may be monitored using optical sensors, vision and machine learning models, to detect any anomalies that may lead to the cycle being unsuccessful, such as dropping part of or an entire SKU, picking multiple instances of a SKU at once, a lid of a SKU opening, etc.

[0022] Shoeboxes may house one individual pairs of footwear, a single item of footwear, or multiple pairs of footwear. The footwear may include shoes, boots, cleats, skates, slippers, loafers, heels, sandals, flip-flops, running shoes, sneakers, ballerinas, galoshes, water shoes, Oxfords, brogues, wedges, espadrilles, mules, slip-ons, clogs, moccasins, boat shoes, stilettos, pumps, combat boots, work boots, snow boots, cowboy boots, hiking shoes, or other types of footwear for men, women, children, or even animals.

[0023] Although the style and size of shoeboxes can vary widely, many shoeboxes include a cardboard box sufficiently large to hold two shoes, as well as a box cover or lid. Shoeboxes may be a substantially rectangular prism. However, in other cases, shoeboxes may be a different shape. Shoeboxes may include a lid. The lid may be a “clam shell,” having a lid hinged along one axis. In some cases, the hinged lid may be substantially planar in profile (e.g., with small, angled edges around the perimeter). However, in other cases, the hinged lid may include a facade that passes over the front face (“apron”) or, possibly in addition, the side faces, of the shoebox. Alternatively, the lid may have no hinges and be completely removable from the body of the shoebox. Shoeboxes may be Nomagic Inc. 59332-706.601 Application Specification.docx -6- WSGR Docket No. 59332-705.60165057 / 2024 / PMi formed from corrugated fiber board, cardboard or other similar materials. A pattern for a shoebox may be cut from a blank and folded into a shoebox. In some cases, once folded or assembled, a shoebox may be about 6 to 18 inches long, about 4 to 12 inches wide, and about 3 to 10 inches deep. In some cases, when shipping multiple pairs of shoes (from, e.g., a manufacturer or a distribution center to a retail store), filled shoeboxes may be placed in a larger box, often referred to as an MOC (“master outer carton”), also known as a secondary package or as a distribution package.

[0024] While the systems, the methods, the computer-readable media, and the techniques disclosed herein may be applied to shoeboxes for containing footwear, in some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may be applied to other similar SKUs, such as those having substantially rectangular prism designs, opening lids, etc. For example, other SKUs may be used for storage of items. For example, the SKUs may include one or more of: a storage bin (e.g., a plastic storage bin, a metal storage bin, a fabric storage bin, a wooden storage bin, a carboard storage bin, etc.), a storage box (e.g., a plastic storage box, a metal storage box, a fabric storage box, a wooden storage box, a carboard storage box, etc.), a case (e.g., a plastic storage case a metal storage case, a fabric storage case, a wooden storage case, a carboard storage case, etc.), a storage tote (e.g., a plastic storage tote, a metal storage tote, a fabric storage tote, a wooden storage tote, a carboard storage tote, etc.), a storage pallet (e.g., a plastic storage pallet, a metal storage pallet, a wooden storage pallet, etc.), or any other container suitable for storing items.

[0025] Some examples of other types of boxes (or SKUs) that may be compatible with the shoebox grippers or grippers employing similar design principles of the shoebox grippers (e.g., two substantially perpendicular suction mats) of the systems, the methods, the computer-readable media, and the techniques disclosed herein include gift boxes (often come with detachable lids, providing a neat, presentable package for various types of gifts), storage boxes (used for storing different items around the house, feature shoebox-style lids for easy access), jewelry boxes (typically smaller, many have shoebox-style lids for easy opening and closing), document boxes (used to store important papers and often have shoebox-like lids for easy access), memory boxes (typically used for keeping memorabilia, often come with shoebox-style lids), archive boxes (used for storing files and documents for long periods, often feature removable lids similar to shoeboxes for easy retrieval of documents), keepsake boxes (used for storing cherished items and often feature a shoebox-style lid), craft boxes (many craft or hobby boxes come with shoebox-style lids to keep supplies neatly stored and accessible), photo boxes (used for storing and organizing photographs, these boxes often come with shoebox-style lids), and game boxes (many board games are stored in boxes with lids that remove completely, just like shoeboxes), etc.Nomagic Inc. 59332-706.601 Application Specification.docx -7- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0026] In some cases, the SKUs (e.g., shoeboxes or other non-shoebox SKUs) are moved from picking locations (e.g., infeed system) to dropping locations (e.g., outfeed locations). For example, prior to or after moving the SKUs (e.g., shoeboxes or other non-shoebox SKUs) with a robotic arm, the SKUs may be contained in one or more of: a shipping bin (e.g., a plastic shipping bin, a metal shipping bin, a fabric shipping bin, a wooden shipping bin, a carboard shipping bin, etc.), a shipping box (e.g., a plastic shipping box, a metal shipping box, a fabric shipping box, a wooden shipping box, a carboard shipping box, etc.), a case (e.g., a plastic shipping case a metal shipping case, a fabric shipping case, a wooden shipping case, a carboard shipping case, etc.), a shipping tote (e.g., a plastic shipping tote, a metal shipping tote, a fabric shipping tote, a wooden shipping tote, a carboard shipping tote, etc.), a shipping pallet (e.g., a plastic shipping pallet, a metal shipping pallet, a wooden shipping pallet, etc.), or any other container suitable for containing SKUs.Example Processes of Handling Stock Keeping Units

[0027] FIG. 1 shows an example of an operation flowchart 100 for implementing stock keeping unit handling. The flowchart 100 may include operations for transferring fashion SKUs, including shoeboxes, from infeed to outfeed systems, while also scanning barcodes. Further, the flowchart 100 can be used (e.g., with some adaptation) for sorting, order preparation, returns, or other warehouse stock management processes. The flowchart 100 supports picking previously-unseen shoeboxes from unstructured sources, such as bins containing other SKUs, in random positions and orientations. Further, in some cases, the flowchart 100 may be used with grippers having functionalities to further prevent shoebox opening (e.g., mechanical fingers holding shoeboxes from the bottom during transfer). Still further, in some cases, the flowchart 100 may be used with functionalities to enable SKU handling in constrained spaces, such as rearranging shoeboxes layout to ease handling.

[0028] In some cases, the flowchart 100 may begin with identifying whether the next SKU to be picked is a shoebox at block 102. The next SKUs may be delivered via an infeed system. The infeed system may include, for example, a conveyor system, a chute system, a pusher system, etc. To identify whether the next SKU is a shoebox at the block 102, image data may be collected corresponding to the next SKU. This image data, collected by, for example, optical sensors, may be analyzed. In some cases, machine learning techniques (e.g., computer vision) may be used to determine whether the next SKU is a shoebox.

[0029] In some cases, if the next SKU is identified to be a shoebox, a shoebox gripper may be attached to a robotic arm at block 101. The shoebox gripper may be particularly configured to grip a shoebox. For example, the shoebox gripper may be particularly configured to grip a shoebox without Nomagic Inc. 59332-706.601 Application Specification.docx -8- WSGR Docket No. 59332-705.60165057 / 2024 / PMi the lid (e.g., a hinged lid or an unhinged lid) of the shoebox opening. Shoeboxes may be substantially rectangular prisms. The shoebox gripper may be the same as or similar to the shoebox gripper 200 of FIGs. 2A-3C. In some cases, the shoebox gripper may be configured to handle other SKUs similar to a shoebox.

[0030] In some cases, a controller may select a pair of adjacent faces to be grasped at block 104. If determined at block 106 that the adjacent faces of the shoebox can be grasped by the shoebox gripper, then the gripper may proceed with performing the grasp at block 107. If determined at the block 106 that the shoebox gripper cannot grasp the shoebox without a collision (e.g., a collision with other SKUs), then the shoebox may be repositioned. Repositioning may include sliding the shoebox away from a bin face at block 105. Once the shoebox is grasped at the block 107, if determined that both suction mats of the shoebox gripper are properly sealed at block 110, the shoebox is lifted at block 111. However, if one or both of the suction mats are not properly sealed against the surface of the shoebox at the block 110, the shoebox gripper may release the grasp and move away from the shoebox at block 109, which may restart some or all of the addressing and grasping process, as illustrated (e.g., returning to the block 104).

[0031] In some cases, once the shoebox is lifted by the shoebox gripper at the block 111, a check is performed to confirm the shoebox lid remained closed at block 113. If determined (e.g., by an optical system or by pressure sensors in the shoebox gripper) that the lid opened at the block 113, then robotic arm and shoebox gripper return to the grasping point at 112, which may restart some or all of the addressing and grasping process, as illustrated (e.g., returning to the block 104). Provided the lid of the shoebox did not open, the shoebox may be transferred to a scanning station block 114.

[0032] In some cases, once the shoebox is transferred to the scanning station at the block 114, a barcode (or other similar type of code, e.g., an identification number, quick response code, RFID tags, NFC tags, data matrix codes, universal product codes, European article numbers, alphanumeric stock keeping unit identifiers, global trade item numbers, etc.) of the shoebox, is scanned. At block 116, the scan of the barcode is checked for correctness. If correctly scanned, the shoebox may be inducted into an outfeed system at block 115. The outfeed system may include one or more of a conveyor system, a chute system, a pusher system, etc. If the barcode is not correctly scanned, the shoebox may be inducted to an exception system at block 117. The exception system may hold the shoebox for further review (e.g., from an optical system, from a human operator, etc.).

[0033] After inducting the shoebox into either the outfeed system at the block 115 or the exception system at the block 117, the controller may await a signal indicating a request to handle the next SKU at block 118, then returning to the block 102. If the next SKU is determined to be a non-Nomagic Inc. 59332-706.601 Application Specification.docx -9- WSGR Docket No. 59332-705.60165057 / 2024 / PMi shoebox at the block 102, a non-shoebox gripper (e.g., a universal gripper) may be attached to the robotic arm at block 103. Once the non-shoebox gripper is attached, the controller may instruct the robotic arm and the non-shoebox gripper to perform a standard picking cycle at block 108 and then transfer the non-shoebox to the scanning station at the block 114, which may continue the scanning process of the blocks 115-117, and eventually the waiting for the next request at the block 118.

[0034] In some cases, any number of operations of the one or more operations disclosed above with respect to the flowchart 100 may be added or removed. Further, the one or more operations of the flowchart 100 may be performed in any order. Further, at least one of the one or more operations of the flowchart 100 may be repeated, e.g., iteratively.Examples of Robotic Grippers

[0035] FIG. 2A shows a front view of an example of a shoebox gripper 200 and FIG. 2B shows a left view of the example of the shoebox gripper 200. The shoebox gripper 200 is capable of handling various types of shoeboxes with fixed lids, lids on a hinge, fully detachable lids, etc. Further, the shoebox gripper 200 is capable of picking shoeboxes from unstructured, constrained spaces, such as a bin containing other SKUs (e.g., fashion SKUs). Further, the shoebox gripper 200 enables scanning barcodes of shoeboxes and placing shoeboxes in an outfeed system.

[0036] Importantly, the shoebox gripper 200 achieves numerous advantages by grasping a shoebox by two adjacent faces. Compared to existing gripping solutions for shoeboxes, the shoebox gripper 200 provides a wider range of acceptable layouts (e.g., various orientations, occlusions, other items around, etc.) for which the shoebox gripper 200 can still successfully lift a shoebox.

[0037] At a high level, the shoebox gripper 200 comprises a main body 203 and two suction mats. The main body 203 may aid in storing the shoebox gripper 200 when the shoebox gripper 200 is not in use (e.g., when a different gripper is connected to the robotic arm). In some cases, the two suction mats may be substantially perpendicular (e.g., at about a 100° angle, at about a 95° angle, at about a 92° angle, at about a 90° angle, at about an 88° angle, at about an 85° angle, at about an 80° angle, etc.) to one another. For example, one suction mat may be in a substantially vertical orientation (illustrated as a vertical suction mat 205) and the other suction mat may be in a substantially horizontal orientation (illustrated as a horizontal suction mat 209). In some cases, working together, the horizontal suction mat 209 and the vertical suction mat 205 may lock shoebox movement in all 6 dimensions (XYZ position, XYZ orientation) while also reducing the likelihood of the lid of a shoebox from opening by pressing the lid to the body of the shoebox. In some cases, the horizontal suction mat 209 and the vertical suction mat 205 may comprise a malleable or impressionable surface to help the horizontal suction mat 209 and the vertical suction mat 205 conform aroundNomagic Inc. 59332-706.601 Application Specification.docx -10- WSGR Docket No. 59332-705.60165057 / 2024 / PMi imperfections, corners, edges, or other non-flush surfaces on the shoebox, maintaining an effective vacuum seal.

[0038] In some cases, the horizontal suction mat 209 may be mounted on a hinge 210 (or linear bearing). The hinge 210 may be actuated to help improve grasp precision of the horizontal suction mat 209. In some cases, the hinge 210 may be spring actuated. Spring actuation of the hinge 210 may help to passively adjust angle between the horizontal suction mat 209 and the vertical suction mat 205, compensating for shoebox deformations during lifting, thereby increasing grasp strength. In some cases, the hinge 210 may be in addition or in alternative to an actively actuated positioning of the horizontal mat 209, allowing for more complex grasp correction mechanisms or software.

[0039] In some cases, the vertical suction mat 205 may comprise a teeth-shape wedge 204 at the lower distal end of the vertical suction mat 205 to help the vertical suction mat 205 (and the shoebox gripper 200) squeeze and articulate in between shoeboxes. Advantageously, this wedge 204 may help to precisely pick up a shoebox from an unstructured source. In some cases, the wedge 204 may be an actuated wedge covering the entirety of the vertical suction mat 205 from bottom when extracted, allowing the vertical suction mat 205 to slide in between the face of a shoebox and another surface (e.g., another shoebox, another SKU, a bin wall, etc.) if the two are very close to each other. The wedge 204 may be then retracted to uncover the entirety of the vertical suction mat 205 before applying vacuum to the vertical suction mat 205 so that the full area of the vertical suction mat 205 can generate lifting force to a shoebox. In some cases, the wedge 204 may be used to slide or reposition a SKU or a shoebox to enable picking up the shoebox using the shoebox gripper 200.

[0040] In some cases, the horizontal suction mat 209 and the vertical suction mat 205 may connect with independent vacuum tubing 201. The vacuum tubing 201 may allow for independent control and monitoring of each of the horizontal suction mat 209 and the vertical suction mat 205. Further, the vacuum tubing 201 may enable implementing software grasp correction mechanisms. In other cases, the same vacuum tubing 201 may be shared between the vertical suction mat 205 and the horizontal suction mat 210. The vacuum tubing 201 may be connected to the horizontal suction mat 209 and the vertical suction mat 205 via a pneumatic connector 207. In some cases, the pneumatic connector 207 may be rigid, locking the vertical suction mat 205 in a substantially vertical position. In other cases, the pneumatic connector 207 may be flexible (e.g., hinged) enabling some movement of the vertical suction mat 205.

[0041] In some cases, the shoebox gripper 200 may comprise a tool changing adapter 212 and a tool holder 213. The tool changing adapter 212 and the tool holder 213 may be located towards the top of the shoebox gripper 200, in contact with an interface plate 211. The tool changing adapter 212 and Nomagic Inc. 59332-706.601 Application Specification.docx - 11 - WSGR Docket No. 59332-705.60165057 / 2024 / PMi the tool holder 213 may allow for tool changes (e.g., quickly or automatically). Accordingly, the tool changing adapter 212 and the tool holder 213 may help in allowing the shoebox gripper 200 to be quickly added to and removed from a robotic arm, enabling a single robotic arm to handle both shoeboxes and non-shoebox SKUs (e.g., using different grippers). When the shoebox gripper 200 is not in use and connected with the robotic arm, the shoebox gripper 200 may be stored on a tool

[0042] In some cases, the shoebox gripper 200 may further comprise a level compensator 202 that enables adjusting the distance between the connection point of the shoebox gripper 200 to the robotic arm and the horizontal suction mat 209 and the vertical suction mat 205. As the robotic arm may have a large field of motion, the precise control offered by the level compensator 202 enables carefully and precisely lowering or raising the horizontal suction mat 209 and the vertical suction mat 205 to best grip a shoebox.

[0043] In some cases, the vacuum tubing 201 may enable separate and independent suction control for the vertical mat 205 and the horizontal mat 209. In some cases, a vertical plate 206 may serve as a mount for the vertical suction mat 205. Further, in some cases, a vertical distribution plate 215 may be in contact with the vertical suction mat 205 and may aid in distributing suction power through the vertical suction mat 205. In some cases, a horizontal plate 214 may serve as a mount for the horizontal suction mat 209. Further, in some cases, a horizontal distribution plate 208 may be in contact with the horizontal suction mat 209 and may aid in distributing suction power through the horizontal suction mat 209. The vertical plate 206 may hold the vertical suction mat 205 in place. The vertical distribution plate 215, together with the vertical suction mat 205 forms the surface that contacts shoeboxes. The horizontal plate 214 is used just to position the contact surface of the horizontal suction mat 209. The horizontal distribution plate 208, together with the horizontal suction mat 209, contacts the shoebox. These various plate configurations may aid in enabling independent suction control for the two suction mats.

[0044] In some cases, the total surface area of the suction mats (e.g., the vertical suction mat 205 plus the horizontal suction mat 209) may be of similar magnitude to the total surface area of two sides of a shoebox. In some cases, the total surface area of the suction mats may be about 0.01 square meters to about 0.5 square meters. In some cases, the total surface area of the suction mats may be about 0.01 square meters to about 0.02 square meters, about 0.01 square meters to about 0.03 square meters, about 0.01 square meters to about 0.04 square meters, about 0.01 square meters to about 0.05 square meters, about 0.01 square meters to about 0.075 square meters, about 0.01 square meters to about 0.1 square meters, about 0.01 square meters to about 0.125 square meters, about 0.01 square meters to about 0.15 square meters, about 0.01 square meters to about 0.175 square meters, Nomagic Inc. 59332-706.601 Application Specification.docx - 12- WSGR Docket No. 59332-705.60165057 / 2024 / PMi about 0.01 square meters to about 0.2 square meters, about 0.01 square meters to about 0.5 square meters, about 0.02 square meters to about 0.03 square meters, about 0.02 square meters to about 0.04 square meters, about 0.02 square meters to about 0.05 square meters, about 0.02 square meters to about 0.075 square meters, about 0.02 square meters to about 0.1 square meters, about 0.02 square meters to about 0.125 square meters, about 0.02 square meters to about 0.15 square meters, about 0.02 square meters to about 0.175 square meters, about 0.02 square meters to about 0.2 square meters, about 0.02 square meters to about 0.5 square meters, about 0.03 square meters to about 0.04 square meters, about 0.03 square meters to about 0.05 square meters, about 0.03 square meters to about 0.075 square meters, about 0.03 square meters to about 0.1 square meters, about 0.03 square meters to about 0.125 square meters, about 0.03 square meters to about 0.15 square meters, about 0.03 square meters to about 0.175 square meters, about 0.03 square meters to about 0.2 square meters, about 0.03 square meters to about 0.5 square meters, about 0.04 square meters to about 0.05 square meters, about 0.04 square meters to about 0.075 square meters, about 0.04 square meters to about 0.1 square meters, about 0.04 square meters to about 0.125 square meters, about 0.04 square meters to about 0.15 square meters, about 0.04 square meters to about 0.175 square meters, about 0.04 square meters to about 0.2 square meters, about 0.04 square meters to about 0.5 square meters, about 0.05 square meters to about 0.075 square meters, about 0.05 square meters to about 0.1 square meters, about 0.05 square meters to about 0.125 square meters, about 0.05 square meters to about 0.15 square meters, about 0.05 square meters to about 0.175 square meters, about 0.05 square meters to about 0.2 square meters, about 0.05 square meters to about 0.5 square meters, about 0.075 square meters to about 0.1 square meters, about 0.075 square meters to about 0.125 square meters, about 0.075 square meters to about 0.15 square meters, about 0.075 square meters to about 0.175 square meters, about 0.075 square meters to about 0.2 square meters, about 0.075 square meters to about 0.5 square meters, about 0.1 square meters to about 0.125 square meters, about 0.1 square meters to about 0.15 square meters, about 0.1 square meters to about 0.175 square meters, about 0.1 square meters to about 0.2 square meters, about 0.1 square meters to about 0.5 square meters, about 0.125 square meters to about 0.15 square meters, about 0.125 square meters to about 0.175 square meters, about 0.125 square meters to about 0.2 square meters, about 0.125 square meters to about 0.5 square meters, about 0.15 square meters to about 0.175 square meters, about 0.15 square meters to about 0.2 square meters, about 0.15 square meters to about 0.5 square meters, about 0.175 square meters to about 0.2 square meters, about 0.175 square meters to about 0.5 square meters, or about 0.2 square meters to about 0.5 square meters. In some cases, the total surface area of the suction mats may be about 0.01 square meters, about 0.02 square meters, about 0.03 square meters, about 0.04 squareNomagic Inc. 59332-706.601 Application Specification.docx -13- WSGR Docket No. 59332-705.60165057 / 2024 / PMi meters, about 0.05 square meters, about 0.075 square meters, about 0.1 square meters, about 0.125 square meters, about 0.15 square meters, about 0.175 square meters, about 0.2 square meters, or about 0.5 square meters. In some cases, the total surface area of the suction mats may be at least about 0.01 square meters, about 0.02 square meters, about 0.03 square meters, about 0.04 square meters, about 0.05 square meters, about 0.075 square meters, about 0.1 square meters, about 0.125 square meters, about 0.15 square meters, about 0.175 square meters, or about 0.2 square meters. In some cases, the total surface area of the suction mats may be at most about 0.02 square meters, about 0.03 square meters, about 0.04 square meters, about 0.05 square meters, about 0.075 square meters, about 0.1 square meters, about 0.125 square meters, about 0.15 square meters, about 0.175 square meters, about 0.2 square meters, or about 0.5 square meters.

[0045] In some cases, the kilograms of lifting force generated by the shoebox gripper 200 may be of similar magnitude to the weight of a shoebox (e.g., filled with footwear or unfilled with footwear), including, for example, a safety factor. The lifting force generated by the shoebox gripper 200 may be controlled by the vacuum tubing 201. In some cases, the shoebox gripper 200 may be configured to generate about 0.5 kilograms of lifting force to about 25 kilograms of lifting force. In some cases, the shoebox gripper 200 may be configured to generate about 0.5 kilograms of lifting force to about 1 kilograms of lifting force, about 0.5 kilograms of lifting force to about 1.5 kilograms of lifting force, about 0.5 kilograms of lifting force to about 2 kilograms of lifting force, about 0.5 kilograms of lifting force to about 2.5 kilograms of lifting force, about 0.5 kilograms of lifting force to about 3 kilograms of lifting force, about 0.5 kilograms of lifting force to about 4 kilograms of lifting force, about 0.5 kilograms of lifting force to about 5 kilograms of lifting force, about 0.5 kilograms of lifting force to about 7 kilograms of lifting force, about 0.5 kilograms of lifting force to about 10 kilograms of lifting force, about 0.5 kilograms of lifting force to about 15 kilograms of lifting force, about 0.5 kilograms of lifting force to about 25 kilograms of lifting force, about 1 kilograms of lifting force to about 1.5 kilograms of lifting force, about 1 kilograms of lifting force to about 2 kilograms of lifting force, about 1 kilograms of lifting force to about 2.5 kilograms of lifting force, about 1 kilograms of lifting force to about 3 kilograms of lifting force, about 1 kilograms of lifting force to about 4 kilograms of lifting force, about 1 kilograms of lifting force to about 5 kilograms of lifting force, about 1 kilograms of lifting force to about 7 kilograms of lifting force, about 1 kilograms of lifting force to about 10 kilograms of lifting force, about 1 kilograms of lifting force to about 15 kilograms of lifting force, about 1 kilograms of lifting force to about 25 kilograms of lifting force, about 1.5 kilograms of lifting force to about 2 kilograms of lifting force, about 1.5 kilograms of lifting force to about 2.5 kilograms of lifting force, about 1.5 kilograms of lifting force to about 3 Nomagic Inc. 59332-706.601 Application Specification.docx - 14- WSGR Docket No. 59332-705.60165057 / 2024 / PMi kilograms of lifting force, about 1.5 kilograms of lifting force to about 4 kilograms of lifting force, about 1.5 kilograms of lifting force to about 5 kilograms of lifting force, about 1.5 kilograms of lifting force to about 7 kilograms of lifting force, about 1.5 kilograms of lifting force to about 10 kilograms of lifting force, about 1.5 kilograms of lifting force to about 15 kilograms of lifting force, about 1.5 kilograms of lifting force to about 25 kilograms of lifting force, about 2 kilograms of lifting force to about 2.5 kilograms of lifting force, about 2 kilograms of lifting force to about 3 kilograms of lifting force, about 2 kilograms of lifting force to about 4 kilograms of lifting force, about 2 kilograms of lifting force to about 5 kilograms of lifting force, about 2 kilograms of lifting force to about 7 kilograms of lifting force, about 2 kilograms of lifting force to about 10 kilograms of lifting force, about 2 kilograms of lifting force to about 15 kilograms of lifting force, about 2 kilograms of lifting force to about 25 kilograms of lifting force, about 2.5 kilograms of lifting force to about 3 kilograms of lifting force, about 2.5 kilograms of lifting force to about 4 kilograms of lifting force, about 2.5 kilograms of lifting force to about 5 kilograms of lifting force, about 2.5 kilograms of lifting force to about 7 kilograms of lifting force, about 2.5 kilograms of lifting force to about 10 kilograms of lifting force, about 2.5 kilograms of lifting force to about 15 kilograms of lifting force, about 2.5 kilograms of lifting force to about 25 kilograms of lifting force, about 3 kilograms of lifting force to about 4 kilograms of lifting force, about 3 kilograms of lifting force to about 5 kilograms of lifting force, about 3 kilograms of lifting force to about 7 kilograms of lifting force, about 3 kilograms of lifting force to about 10 kilograms of lifting force, about 3 kilograms of lifting force to about 15 kilograms of lifting force, about 3 kilograms of lifting force to about 25 kilograms of lifting force, about 4 kilograms of lifting force to about 5 kilograms of lifting force, about 4 kilograms of lifting force to about 7 kilograms of lifting force, about 4 kilograms of lifting force to about 10 kilograms of lifting force, about 4 kilograms of lifting force to about 15 kilograms of lifting force, about 4 kilograms of lifting force to about 25 kilograms of lifting force, about 5 kilograms of lifting force to about 7 kilograms of lifting force, about 5 kilograms of lifting force to about 10 kilograms of lifting force, about 5 kilograms of lifting force to about 15 kilograms of lifting force, about 5 kilograms of lifting force to about 25 kilograms of lifting force, about 7 kilograms of lifting force to about 10 kilograms of lifting force, about 7 kilograms of lifting force to about 15 kilograms of lifting force, about 7 kilograms of lifting force to about 25 kilograms of lifting force, about 10 kilograms of lifting force to about 15 kilograms of lifting force, about 10 kilograms of lifting force to about 25 kilograms of lifting force, or about 15 kilograms of lifting force to about 25 kilograms of lifting force. In some cases, the shoebox gripper 200 may be configured to generate about 0.5 kilograms of lifting force, about 1 kilograms of lifting force, about 1.5 kilograms of lifting Nomagic Inc. 59332-706.601 Application Specification.docx -15- WSGR Docket No. 59332-705.60165057 / 2024 / PMi force, about 2 kilograms of lifting force, about 2.5 kilograms of lifting force, about 3 kilograms of lifting force, about 4 kilograms of lifting force, about 5 kilograms of lifting force, about 7 kilograms of lifting force, about 10 kilograms of lifting force, about 15 kilograms of lifting force, or about 25 kilograms of lifting force. In some cases, the shoebox gripper 200 may be configured to generate at least about 0.5 kilograms of lifting force, about 1 kilograms of lifting force, about 1.5 kilograms of lifting force, about 2 kilograms of lifting force, about 2.5 kilograms of lifting force, about 3 kilograms of lifting force, about 4 kilograms of lifting force, about 5 kilograms of lifting force, about 7 kilograms of lifting force, about 10 kilograms of lifting force, or about 15 kilograms of lifting force. In some cases, the shoebox gripper 200 may be configured to generate at most about 1 kilograms of lifting force, about 1.5 kilograms of lifting force, about 2 kilograms of lifting force, about 2.5 kilograms of lifting force, about 3 kilograms of lifting force, about 4 kilograms of lifting force, about 5 kilograms of lifting force, about 7 kilograms of lifting force, about 10 kilograms of lifting force, about 15 kilograms of lifting force, or about 25 kilograms of lifting force.Examples of Control Algorithms

[0046] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may provide control algorithms for controlling one or more of the shoebox gripper, the robotic arm, the optical sensors, the infeed system, the outfeed system, etc. The control algorithms may be performed on a computing device, such as a controller. In some cases, the controller may be located in the shoebox gripper 200 or a robotic arm connect to the shoebox gripper. In some cases, the controller may be located external to the shoebox gripper 200 or the robotic arm. In some cases, the controller may be communicatively coupled to the shoebox gripper 200 or the robotic arm.

[0047] In some cases, the control algorithms may control the picking process. For example, the shoebox gripper 200 may be manipulated by a robotic arm in such a way that the horizontal mat 209 touches the top of a shoebox to find reference surface. Then the shoebox gripper 200 moves slightly up and then back down (e.g., via the level compensation 202) in combination with sideways movement, to establish contact for both vertical suction mat 205 and the horizontal suction mat 209. Next, in some cases, vacuum is applied to both the horizontal suction mat 209 and the vertical suction mat 205 via the vacuum tubing 201. In some cases, following application of the vacuum, a sealing check is performed for both the horizontal suction mat 209 and the vertical suction mat 205. If the sealing check is successful, the shoebox is lifted up by the shoebox gripper 200, such as from the infeed system.Nomagic Inc. 59332-706.601 Application Specification.docx -16- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0048] In some cases, the shoebox gripper 200 may include actively actuated positioning for the horizontal suction mat 209. In such cases, the shoebox gripper 200 may be manipulated by the robotic arm such that the horizontal suction mat 209 touches the top of a shoebox. Then, in some cases, the horizontal suction mat 209 may be actuated upwards (e.g., via the level compensation 202) and the robotic arm may move sideways until the vertical suction mat 205 touches the side of the shoebox. In some cases, vacuum may then be applied (e.g., via the vacuum tubing 201) to the vertical suction mat 205 and a first vacuum check may be performed. Next, in some cases, the horizontal suction mat 209 may be actuated downwards (e.g., via the level compensation 202) until the horizontal suction mat 209 touches the top of the shoebox and vacuum will be applied (e.g., via the vacuum tubing 201) to the horizontal suction mat 209 and a second vacuum check may be performed. If both the first vacuum check and the second vacuum check are successful, the shoebox may be lifted up by the shoebox gripper, such as from the infeed system.

[0049] In some cases, control algorithms may include a pick point selection mechanism. Due to the construction of certain shoeboxes, certain shoeboxes may be safely picked only by a subset of their adjacent face pairs. Alternatively, due to the construction of other shoeboxes, other shoeboxes may be safely picked by all of their adjacent face pairs. Accordingly, the systems, the methods, the computer-readable media, and the techniques disclosed herein may identify (e.g., via computer vision techniques) the type of shoebox to determine whether or not all adjacent face pairs may be used for picking up the shoebox. In cases in which the shoeboxes can only be picked by a subset of their adjacent face pairs, the systems, the methods, the computer-readable media, and the techniques disclosed herein may need to detect the orientation of the shoebox. Once the orientation of the shoebox is detected, the orientation may be matched with a database including indications of graspable and ungraspable pairs of adjacent faces by which the shoebox can be grasped. Based on these indications, the control algorithm may perform a 3D collision check to determine a safe grasping strategy that reduces the likelihood of the shoebox opening during handling and reduces the likelihood of collisions in the handling process. If only ungraspable pairs of adjacent faces of a shoebox can be reached by the robotic arm, in some cases, the wedge 204 mounted on the shoebox gripper 200 may be used to slide the shoebox (e.g., inside a container of SKUs) to enable reaching a graspable pair of adjacent faces of the shoebox.

[0050] In some cases, control algorithms may include quick detection of shoebox lid opening. Detecting if a shoebox lid opens may use computer vision techniques analyzing image data. The image data may be collected by optical sensors, such as the optical sensors disclosed herein. In some cases, the optical sensors may comprise cameras mounted on a robot arm, on the shoebox gripperNomagic Inc. 59332-706.601 Application Specification.docx -17- WSGR Docket No. 59332-705.60165057 / 2024 / PMi200, or in a robotic cell. The control algorithms may be configured to recognize opening of a shoebox lid in substantially real time. When the control algorithms detect opening of a shoebox lid, in some cases, the control algorithms may provide instruction to abort the picking action by the robotic arm and the shoebox gripper 200 to reduce the likelihood of shoes falling out of the shoebox. Further, in some cases, when the control algorithms detect opening of a shoebox lid, the control algorithms may provide instruction to activate a notification, such as an audio or visual alarm.

[0051] In some cases, based at least in part on the SKU the robotic arm is handling, the robotic arm may change tools using a tool changer. For example, the tool changer may provide the robotic arm with different tools depending on the weight of an item, the size of an item, the material of an item, the fragility of an item, the deformability of an item, etc. The tool changer may provide the robotic arm with the shoebox gripper 200 in response to a shoebox (or other similar SKU) being detected.

[0052] In some cases, the control algorithms may control the infeed systems, the outfeed systems, the robotic arm, or the shoebox gripper 200. The control algorithms may cause the robotic arm to pick up and move SKUs from the infeed systems according to an order that reduces the number of tool changes using the tool changer. For example, the infeed system may sequentially move a first subset of SKUs in the infeed system to a picking location (e.g., via using a buffer in the picking port) such that the first subset of SKUs may be moved (e.g., sequentially) by the robotic arm without the robotic arm changing tools using the tool changer. In another example, the robotic arm may reach into the infeed system to pick up (e.g., sequentially) a first subset of SKUs in the infeed system such that the first subset of SKUs may be moved by the robotic arm without the robotic arm changing tools using the tool changer. Then, once the first subset of SKUs in the infeed system are moved by the robotic arm, the robotic arm may perform a tool change using the tool changer to prepare to move a second subset of SKUs in the infeed system. Accordingly, the systems, the methods, the computer-readable media, and the techniques disclosed herein may improve efficiency and speed of SKU picking via reducing a number of tool changes performed while picking SKUs.Examples of Robotic Arms

[0053] As disclosed herein, shoebox grippers (e.g., the shoebox gripper 200) may be attached to a robotic arm. In some cases, the shoebox gripper may be removably attached to the robotic arm. Removable attachment may enable the shoebox gripper to be fitted to the robotic arm in preparation for picking up a shoebox (e.g., in response to detecting a shoebox in an infeed). In other cases, the shoebox gripper may be more permanently attached to the robotic arm. More permanent attachment may be useful when a robotic arm is at least primarily handling shoeboxes or other SKUs in which the shoebox gripper may be adapted to handle.Nomagic Inc. 59332-706.601 Application Specification.docx -18- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0054] At a high level, a robotic arm may comprise a base or pedestal, which provides support and allows for stationary or rotational movement. Connected to the base may be multiple joints, also known as axes, which permit movement and help the robotic arm to extend, retract, swing sideways, and perform other types of motion. These joints may be powered by actuators such as motors which may be electric, hydraulic, or pneumatic. For precision movements, the robotic arm may use gears or belts that provide controlled rotational force. Next, there may be links or segments, which are the structural elements that connect the various joints, similar to bones in a human arm. The end-effector is the part of the robotic arm that interacts with the environment and performs tasks. The endeffector could be a claw, gripper, vacuum pump, etc. Additionally, there may be sensors in the robotic arm that provide data about the robotic arm's position, speed, force, etc. These sensors may include encoders, potentiometers, accelerometers, optical systems, etc. One or more of these components of the robotic arm may be managed by a control system, which could be a computer or a programmable logic controller (PLC), executing pre-programmed instructions or responding to realtime sensory feedback.

[0055] In some cases, the robotic arm may be a mechanical arm that may be used in various applications including, for example, automotive, agriculture, scientific, manufacturing, construction, etc. Robotic arms may be programmable and may be able to perform similar functions to a human arm. While robotic arms may be reliable and accurate, often times they may be taught to only perform narrowly defined tasks such as picking a specific type of SKU from a specific location with a specific orientation. Accordingly, robotic arms are often times programmed to automate execution of repetitive tasks, such as applying paint to equipment, moving goods in warehouses, harvesting crops in a farm field, etc. Robotic arms may comprise links of manipulator that are connected by joints enabling either rotational motion (such as in an articulated robot) or translational (linear) displacement.

[0056] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may use shoebox grippers with robotic arms. In some cases, a robotic arm comprises one or more of robot joints connecting a robot base and an end effector receiver or end effector. A base joint may be configured to rotate the robot arm around a base axis. A shoulder joint may be configured to rotate the robot arm around a shoulder axis. An elbow joint may be configured to rotate the robot arm about an elbow axis. A wrist joint may be configured to rotate the robot arm around a wrist. A robot arm may be a six-axis robot arm with six degrees of freedom. In some cases, a robot arm may comprise less or more robot joints and may comprise less than six degrees of freedom.Nomagic Inc. 59332-706.601 Application Specification.docx -19- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0057] A robot arm may be operatively connected to a controller. The controller may comprise an interface device enabling connection and programming of the robot arm. The controller may comprise a computing device comprising a processor and software or a computer program installed thereon. The computing device may be provided as an external device. The computing device may be integrated into the robot arm. The robotic arm may be configurable. For example, the robotic arm may be communicatively coupled to the controller, where the controller may be configured to set specifications for controlling handling operations such as the speed of transport from infeed to outfeed, reversal of the transport direction, stop and start functions of the robotic arm, etc. The controller may be remote or co-located with the robotic arm. In some cases, the controller may correspond to a control panel that may also have a user interface, such as a screen or display configured to display the information (e.g., specifications) corresponding to the robotic arm or items held by the grippers (e.g., shoebox grippers) of the robotic arm. For example, information about the one or more items held by the robotic arm may include the weight of an item or an item container, a size of an item or an item container, a number of items in an item container, an identifier (e.g., barcode, identification number, quick response code, RFID tags, NFC tags, data matrix codes, universal product codes, European article numbers, alphanumeric stock keeping unit identifiers, global trade item numbers, etc.) of an item or an item container, a fragility of an item, a deformability of an item, etc. In some cases, the controller may be further communicatively coupled to infeed or outfeed systems corresponding to the robotic arm. By coupling the controller to the robotic arm and the infeed system or outfeed system, operations between the infeed system or outfeed system and the robotic arm may be coordinated (e.g., synchronized).

[0058] In some cases, the robotic arm can implement a wiggle movement. The robotic arm may wiggle a SKU to help segment the SKU from its surroundings (e.g., from other SKUs). In some cases, wherein a vacuum end effector is employed (e.g., with the shoebox gripper), the robotic arm may employ a wiggle motion in order to create a firm seal against the SKU. In some cases, a wiggle motion may be utilized if the system detects that more than one SKU has been unintendedly handled by the robotic arm. In some cases, the robotic arm may release and re-grasp a SKU at another location if the system detects that more than one SKU has been unintendedly handled by the robotic arm.

[0059] In some cases, in addition to the shoebox gripper, various end effectors of a robotic arm may comprise grippers, vacuum grippers, magnetic grippers, etc. In some cases, the robotic arm may be equipped with end effector, such as a suction gripper. In some cases, the gripper includes one or more suction valves that can be turned on or off either by remote sensing, single point distanceNomagic Inc. 59332-706.601 Application Specification.docx -20- WSGR Docket No. 59332-705.60165057 / 2024 / PMi measurement, or by detecting whether suction is achieved. In some cases, an end effector may include an articulated extension.

[0060] In some cases, the suction grippers, such as the shoebox grippers, of a robotic arm are configured to monitor a vacuum pressure to determine if a complete seal against a surface of a SKU is achieved. Upon determination of a complete seal, the vacuum mechanism may be automatically shut off as the robotic arm continues to handle the SKU. In some cases, sections of suction end effectors may comprise a plurality of folds along a flexible portion of the end effector (e.g., bellow or accordion style folds) such that sections of vacuum end effector can fold down to conform to the surface being gripped. In some cases, suction grippers comprise a soft or flexible pad to place against a surface of a SKU, such that the pad conforms to the surface.

[0061] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein comprises a plurality of end effectors to be received by the robotic arm. In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein comprise one or more end effector stages to provide a plurality of end effectors. Robotic arms may comprise one or more end effector receivers to allow the end effectors to removable attach to the robotic arm. End effectors may include single suction grippers, multiple suction grippers, area grippers, finger grippers, and other end effectors. These end effectors may be configured to grip shoeboxes, such as the shoebox gripper 200, or to grip other SKUs, such as other fashion SKUs.Examples of Optical Sensing

[0062] In some cases, an end effector (e.g., a shoebox gripper) is selected for the robotic arm to handle a SKU based on analyzation of one or more images captured by one or more image sensors, as disclosed herein. In some cases, the one or more image sensors are cameras. In some cases, an image sensor is placed before a robotic handler or arm. In some cases, the image sensor is in operative communication with a robotic handling system, which resides downstream from the image sensor. In some cases, the image sensor determines which product type is on the way or will arrive at the robotic handling system next. Based on the determination of the product, the robotic handling system may select and attach the appropriate end effector (e.g., shoebox gripper) to handle the specific product type, such as a specific SKU. Determination of a product type prior to the product reaching the handling station may improve efficiency of the system.

[0063] In some cases, an end effector is selected to handle a SKU based on information received by optical sensors scanning a machine-readable code located on the SKU. In some cases, an end effector is selected to handle a SKU based on information received from a product database, as disclosed herein.Nomagic Inc. 59332-706.601 Application Specification.docx - 1- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0064] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein use one or more optical sensors. In some cases, the optical sensors may comprise one or more of: CMOS Image Sensors, CCD (Charge-Coupled Device) Image Sensors, Photodiode Sensors, Phototransistor Sensors, Color Sensors, Infrared Sensors, Image Stabilization Sensors, LIDAR (Light Detection And Ranging) Sensors, Photometric Sensors, Ultrasonic Image Sensors, Laser Image Sensors, Ambient Light Sensors, Spectrometer Sensors, Light Field Sensors, Proximity Light Sensors, Thermal Imaging Sensors, Hyperspectral Imaging Sensors, Structured Light 3D Sensors, Depth Sensors, Optical Motion Sensors, or Multi-Spectral Imaging Sensors.

[0065] In some cases, the optical sensors may be operatively coupled to at least one processor (e.g., of the controller). In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may use data storage comprising instructions executable by the at least one processor to cause performance of various functions. For example, the functions may include causing the robotic arm to move at least one SKU through a designated space (e.g., from an infeed system to an outfeed system). The functions may further include causing one or more optical sensors to determine a location of a machine -readable code on the at least one SKU as the at least one SKU is moved to a target location (e.g., an outfeed system). Based on the determined location, at least one optical sensor may scan the machine -readable code as the SKU is moved so as to determine information associated with the SKU encoded in the machine -readable code.

[0066] In some cases, information obtained by a machine readable code is referenced to a product database. The product database may provide information corresponding to a SKU being handled by a robotic arm, as disclosed herein. The product database may provide information corresponding to a target location or position of the SKU and verify that the SKU is in a proper location.

[0067] In some cases, based on the information associated with the SKU obtained from the machine- readable code, the systems, the methods, the computer-readable media, and the techniques disclosed herein may determine a target location (e.g., a outfeed system) at which to cause a robotic arm to place a SKU. In some cases, based on the information associated with the SKU obtained from the machine-readable code, the systems, the methods, the computer-readable media, and the techniques may place a SKU at the target location.

[0068] In some cases, the information comprises proper orientation of a SKU. In some cases, proper orientation is referenced to the surface on which a machine -readable code is provided. Information comprising proper orientation of a SKU may determine the orientation at which the SKU is to be placed at the target location. Information comprising proper orientation of a SKU may be used toNomagic Inc. 59332-706.601 Application Specification.docx -22- WSGR Docket No. 59332-705.60165057 / 2024 / PMi determine a grasping or handling point at which a robotic arm grasps, grips, or otherwise handles the SKU.

[0069] In some cases, a robotic arm comprises the one or more optical sensors. The one or more optical sensors may be physically coupled to a robotic arm. In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may use multiple cameras oriented at various positions such that when one or more optical sensors are moved over a SKU, the optical sensors can view multiple surfaces of the SKU at various angles. Alternatively, the systems, the methods, the computer-readable media, and the techniques disclosed herein may use multiple mirrors, such that mirrors so that one or more optical sensors can view multiple surfaces of a SKU. In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein use one or more optical sensors located underneath a platform on which a SKU is placed or moved over during a scanning procedure. The platform may be transparent or semi-transparent so that the optical sensors located underneath can scan a bottom surface of the SKU.

[0070] In another example configuration, the robotic arm may bring a box through a reading station after or while orienting the box in a certain manner, such as in a manner in order to place the machine-readable code in a position in space where it can be easily viewed and scanned by one or more optical sensors.

[0071] In some cases, the one or more optical sensors comprise one or more images sensors. The one or more image sensors may capture one or more images of a SKU to be handled by a robotic arm. In some cases, the one or more images sensors comprise one or more cameras. In some cases, an image sensor is coupled to a robotic arm (or a gripper, e.g., the shoebox gripper 200). In some cases, an image sensor is placed near a workstation of a robotic arm to capture images of one or more SKUs to be handled by the robotic arm. In some cases, the image sensor captures images of a SKU being handled by a robotic arm.

[0072] In some cases, one or more image sensors comprise a depth camera. The depth camera may be a stereo camera, an RGBD (RGB Depth) camera, or the like. The camera may be a color or monochrome camera. In some cases, one or more image sensors comprise a RGBaD (RGB+active depth, e.g., an Intel RealSense D415 depth camera) color or monochrome camera registered to a depth sensing device that uses active vision techniques such as projecting a pattern into a scene to enable depth triangulation between the camera or cameras and the known offset pattern projector. In some cases, the camera is a passive depth camera. In some cases, cues, such as barcodes, texture coherence, color, 3D surface properties, or printed text on the surface may also be used to identify a SKU or find the pose (e.g., position, orientation, etc.) of the SKU in order to know where or how to Nomagic Inc. 59332-706.601 Application Specification.docx -23- WSGR Docket No. 59332-705.60165057 / 2024 / PMi place the SKU. In some cases, shadow or texture differences may be employed to segment SKUs as well. In some cases, an image sensor comprises a vision processor. In some cases, an image sensor comprises an inferred stereo sensor system. In some cases, an image sensor comprises a stereo camera system.

[0073] In some cases, a virtual environment including a model of the SKUs in 2D or 3D may be determined and used to develop a plan or strategy for picking up the SKU and verifying their properties are an approximate match to the expected properties. In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein use one or more sensors to scan an environment containing SKUs. In an embodiment, as a robotic arm moves, a sensor coupled to the robotic arm captures sensor data about a plurality of SKUs in order to determine shapes or positions of individual SKUs. A larger picture of a 3D environment may be stitched together by integrating information from individual (e.g., 3D) scans. In some cases, the image sensors are placed in fixed positions, on a robotic arm, or in other locations. In some cases, scans may be constructed and used in accordance with any or all of a number of different techniques.

[0074] In some cases, scans are conducted by moving a robotic arm upon which one or more image sensors are mounted. Data comprising a position of the robotic arm position may provide be correlated to determine a position at which a mounted sensor is located. Positional data may also be acquired by tracking key points in the environment. In some cases, scans may be from fixed-mount cameras that have fields of view (FOVs) covering a given area.

[0075] In some cases, a virtual environment built using a 3D volumetric or surface model to integrate or stitch information from more than one sensor. This may allow the systems, the methods, the computer-readable media, and the techniques disclosed herein to operate within a larger environment, where one sensor may be insufficient to cover a large environment. Integrating information from multiple sensors may yield finer detail than from a single scan alone. Integration of data from multiple sensors may reduce noise levels. This may yield better results for SKU detection or identification, surface picking, or other applications.

[0076] Information obtained from the image sensors may be used to select one or more grasping points of a SKU (e.g., a shoebox). In some cases, information obtained from the image sensors may be used to select an end effector (e.g., the shoebox gripper 200) for handling a SKU.

[0077] In some cases, an image sensor is attached to a robotic arm. In some cases, the image sensor is attached to the robotic arm at or adjacent to a wrist joint. In some cases, an image sensor attached to a robotic arm is directed to obtain images of one or more SKUs. In some cases, the image sensor scans a machine-readable code placed on a surface of a SKU.Nomagic Inc. 59332-706.601 Application Specification.docx -24- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0078] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may integrate edge detection software. One or more captured images may be analyzed to detect or locate the edges of a SKU. The SKU may be at an initial position prior to being handled by a robotic arm or may be in the process of being handled by a robotic arm when the images are captured. Edge detection processing may comprise processing one or more two- dimensional images captured by one or more image sensors. Edge detection algorithms utilized may include Canny method detection, first-order differential detection methods, second-order differential detection methods, thresholding, linking, edge thinning, phase congruency methods, phase stretch transformation (PST) methods, subpixel methods (including curve-fitting, moment-based, reconstructive, and partial area effect methods), and combinations thereof. Edge detection methods may utilize sharp contrasts in brightness to locate and detect edges of the captured images.Examples of Anomaly Detection

[0079] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may provide anomaly detection while handling shoeboxes or other SKUs. For example, information associated with a SKU obtained from at the machine-readable code may be used to determine one or more anomaly events. Anomaly events may include misplacement of the SKU within a warehouse or system, damage to the SKU, opening of the lid of a shoebox, failure to grip a SKU properly with a robotic arm, or other anomalies which would result in an error in placing a SKU in an appropriate position or otherwise causing an error in further processing to take place. In some cases, when an anomaly is detected, a warning, alert, or other indication will be provided (e.g., to a human operator).

[0080] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may determine that the SKU is at an improper location from the information associated with the SKU obtained from the machine-readable code. The systems, the methods, the computer-readable media, and the techniques disclosed herein may generate an alert that the SKU is located at an improper location, as disclosed herein. The systems, the methods, the computer- readable media, and the techniques disclosed herein may place the SKU into at an error or exception location. The exception location may be located within a container. In some cases, the exception location is designated for SKUs which have been determined to be at an improper location within the system or within a warehouse.

[0081] In some cases, information associated with a SKU obtained from at the machine -readable code may be used to determine one or more properties of the SKU. The information may include expected dimensions, shapes, or images to be captured. Properties of a SKU may include a SKU’s Nomagic Inc. 59332-706.601 Application Specification.docx -25- WSGR Docket No. 59332-705.60165057 / 2024 / PMi size, a SKU’s weight, flexibility of a SKU, and one or more expected forces to be generated as the SKU is handled by a robotic arm.

[0082] In some cases, via edge detection, the systems, the methods, the computer-readable media, and the techniques disclosed herein may record measured dimensional values of a SKU. The measured dimensional values may be compared to expected dimensional values of a SKU to determine if an anomaly event has occurred. Anomaly events based on dimensional comparison may indicate a misplaced SKU, unintentionally connected SKU, damage to a SKU, or combinations thereof. Determination of an anomaly occurrence may trigger an anomaly event, as discussed herein.

[0083] In some cases, one or more images captured of a SKU may be compared to one or more references images. A comparison may be conducted by an integrated computing device of the systems, the methods, the computer-readable media, and the techniques disclosed herein. In some cases, the one or more reference images are provided by a product database. Appropriate reference images may be correlated to a SKU by correspondence to a machine-readable code provided on the SKU.

[0084] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may compensate for variations in angles and distance at which the images are captured during the analysis. In some cases, an anomaly alert is generated if the difference between one or more captured images of a SKU and one or more reference images of the SKU exceeds a predetermined threshold. A difference one or more captured images and one or more reference images may be taken across one or more dimensions or may be a sum difference between the one or more images.

[0085] In some cases, reference images are sent to an operator during a verification process. The operator may view the one or more references images in relation to the one or more captured images to determine if generation of an anomaly event or alert was correct. The operator may view the reference images in a comparison module. The comparison module may present the reference images side-by-side with the captured images.Examples of Infeed Systems and Outfeed Systems

[0086] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may be used to transport SKUs, including shoeboxes, from infeed systems to outfeed systems.Nomagic Inc. 59332-706.601 Application Specification.docx -26- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0087] In some cases, the infeed systems and the outfeed systems comprise one or more locations that are configured to receive a SKU (e.g., a shoebox). For example, the one or more locations may include a picking location and a dropping location. The picking location may be the location at which a SKU is picked up by the robotic arm (e.g., using a shoebox gripper) from the infeed system. The dropping location may be the location at which a SKU is dropped (e.g., placed) by the robotic (e.g., using a shoebox gripper) into the outfeed system. In some cases, the one or more locations may be discrete with respect to one another (e.g., two separate conveyors or bins). In some cases, the one or more locations may be continuous with respect to one another (e.g., positions on a conveyor).

[0088] In some cases, the infeed systems and the outfeed systems may include one or more queues. The one or more queues may comprise one or more locations of the infeed systems and the outfeed systems. In some cases, the queues may transport the SKUs from the infeed system to the outfeed system. The queues may comprise one or more of a conveyor, a chute, a pusher, or any other device used to move (e.g., via gravitational force, mechanical force, electrical force, etc.) SKUs from one location to another.

[0089] In some cases, a robotic arm (e.g., via a shoebox gripper) may be configured to pick up a SKU from a picking location of the infeed system and place the SKU in a dropping location of the outfeed system. In some cases, the robotic arm may be built directly into the infeed systems or the outfeed systems or may be located in proximity to the infeed systems or the outfeed systems.

[0090] In some cases, the infeed system may comprise one or more bins. Each bin of the infeed system may comprise a plurality of SKUs. In some cases, at least some bins may comprise only shoeboxes. While in other cases, at least some bins may comprise a mix of shoeboxes and nonshoebox SKUs. Accordingly, the control algorithms disclosed herein may enable the optical systems to help identify shoeboxes, enable the robotic arm disclosed herein to receive the shoebox gripper disclosed herein, and enable the shoebox gripper to select and grasp a shoebox. In some cases, the one or more bins may be transported in the infeed system via a conveyor system. In some cases, the SKUs in the infeed system may be returned items, new items from a manufacturer, etc.

[0091] After the robotic arm, via the shoebox gripper, lifts the shoebox, prior to placing the shoebox into the outfeed system, the shoebox may be scanned. The outfeed system may comprise may number of warehouse mechanisms to transport the shoebox. For example, the outfeed system may comprise another bin. In another example, the outfeed system may comprise a pouch sorter. With a pouch sorter, every pouch may comprise a single shoebox (or other SKU).Example Stock Keeping Unit ManipulationNomagic Inc. 59332-706.601 Application Specification.docx -27- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0092] The systems, the methods, the computer-readable media, and the techniques disclosed herein for automated picking may cooperate with one or more other technologies in a warehouse or storage facility. As disclosed herein, these one or more other technologies may include robotic technologies such as robotic arms, conveyors, chutes, pushers, item tilters, etc.

[0093] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may sort, handle, pick, place, or otherwise manipulate one or more SKUs of a plurality of SKUs. The systems, the methods, the computer-readable media, and the techniques disclosed herein may replace tasks which may be performed manually or only in a semi-automated fashion. In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may be integrated with machine learning software, such that human involvement may be completely removed over time. In some cases, the systems, the methods, the computer- readable media, and the techniques disclosed herein are used in analyzing, filling, or packing one or more SKUs. In some cases, a surveillance system determines if human intervention is needed for one or more tasks.

[0094] Robotics, such as a robotic arm or other automated manipulators, may be used for applications involving picking up or moving SKUs. Picking up and moving SKUs may involve picking a SKU from a picking source location and placing it at a dropping location. A robotic device may be used to fill a container with SKUs, create a stack of SKUs, unload SKUs from a truck bed, move SKUs to various locations in a warehouse, and transport SKUs to one or more target locations. The SKUs may be of the same type. The SKUs may comprise a mix of different types of SKUs, varying in size, mass, material, etc. Robotics may direct a robotic arm to pick up SKUs based on predetermined knowledge of where SKUs are in the environment. In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may use a plurality of robotic arms, wherein each robotic arm transports SKUs to one or more dropping locations.

[0095] In some cases, a SKU manipulation may include using a device or apparatus for reorientation of SKUs. The device for re-orienting a SKU may be referred to herein as an item tilter. In some cases, an item tilter is used in conjunction with one or more robotic arms. The item tilter may reorient a SKU, such that it can be properly handled by a robotic arm. In some cases, an item tilter is provided to properly orient a SKU prior to placement within a container or box. The item tilter may facilitate proper packing of the container or box to maximize the number of SKUs the container may hold or minimize the additional packing / stuffing materials required for shipping of the SKUs within the container.Nomagic Inc. 59332-706.601 Application Specification.docx -28- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0096] In some cases, a database is provided containing information related to SKUs being handled by automated systems of a facility. In some cases, a database comprises information of how each SKU in an inventory should be handled or manipulated by the item til ter and or robotic arms. In some cases, a machine learning process dictates and improves upon the handling of a SKU. In some cases, the machine learning is trained by observation and repetition of a specific SKU being handled by a robot or automated handling system. In some cases, the machine learning is trained by observation of a human interaction with a SKU.

[0097] Additional details regarding example operations of item manipulation may be found in, for example, PCT / IB2021 / 000588, PCT / IB2023 / 000165, and PCT / US23 / 72313, all of which are incorporated by reference herein in their entireties.Shoebox Gripper

[0098] FIGs. 3A-3C show various perspective view of an example of a shoebox gripper 200 gripping a shoebox 300. The shoebox gripper 200 of FIGs. 3A-3C may be the same as or similar to the shoebox gripper 200 of FIGs. 2A and 2B. In some cases, the shoebox 300 of the FIGs. 3A-3C may have a hinged (“clam shell”) lid 305. In other cases, the shoebox 300 of FIGs. 3A-3C may have a completely removable lid 305. In practice, the orientations selected for gripping a shoebox may depend on: (A) where is space around the shoebox in the bin (e.g., some faces of the shoebox may be next to the walls of the bin or may be touching other SKUs), (B) orientation of the shoebox in the bin, (C) size or dimensions of the shoebox, etc. Selecting an orientation to grip the shoebox may aim to achieve approaching and handling the shoebox in a collision-free manner and the two suction mats of the gripper touching the lid and the body of the shoebox.

[0099] FIG. 3A shows a perspective view of an example of the shoebox gripper 200 gripping the shoebox 300 in a first orientation. In the first orientation, the horizontal suction mat of the shoebox gripper 200 (which may be the same as or similar to the horizontal suction mat 209 of FIGs. 2A and 2B) contacts a lid 305 of the shoebox 300. In the first orientation, the vertical suction mat of the shoebox gripper 200 (which may be the same as or similar to the vertical suction mat 205 of FIGs. 2A and 2B) contacts a body 310 of the shoebox 300. In some cases, in this first orientation, the vertical suction mat of the shoebox gripper 200 may further be in contact with the lid 305 of the shoebox 300. In other cases, in this first orientation, the vertical suction mat of the shoebox gripper 200 may be out of contact with the lid 305 of the shoebox 300. Advantageously, in this first orientation, the lid 305 is secured by the horizontal suction mat and the vertical suction mat working together. Further the two suction mats also provide lift and stability while moving the shoebox 300 through some or all of the 6 degrees of freedom.Nomagic Inc. 59332-706.601 Application Specification.docx -29- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0100] FIG. 3B shows a perspective view of an example of the shoebox gripper 200 gripping the shoebox 300 in a second orientation. In the second orientation, the horizontal suction mat of the shoebox gripper 200 (which may be the same as or similar to the horizontal suction mat 209 of FIGs. 2A and 2B) contacts both the lid 305 of the shoebox 300 and the body 310 of the shoebox. In the second orientation, the vertical suction mat of the shoebox gripper 200 (which may be the same as or similar to the vertical suction mat 205 of FIGs. 2A and 2B) also contacts both the lid 305 of the shoebox 300 and the body 310 of the shoebox. Advantageously, in this second orientation, the lid 305 is secured by the horizontal suction mat and the vertical suction mat working together. Further the two suction mats also provide lift and stability while moving the shoebox 300 through some or all of the 6 degrees of freedom. Compared to the first orientation, the two suction mats may have less surface area in contact with the shoebox 300. Therefore, this orientation may be less preferable if the shoebox 300 is particularly heavy. However, via increasing the vacuum force (e.g., through vacuum tubes that may be the same as or similar to the vacuum tubes 201 of FIGs. 2A and 2B) when in the second orientation, the lesser surface area may be compensated for and the shoebox gripper 200 may be able to achieve a similarly secure grip on the shoebox 300 as in the first orientation.

[0101] FIG. 3C shows a perspective view of an example of the shoebox gripper 200 gripping the shoebox 300 in a third orientation. In the third orientation, the horizontal suction mat of the shoebox gripper 200 (which may be the same as or similar to the horizontal suction mat 209 of FIGs. 2A and 2B) contacts the body 310 of the shoebox 300. In the third orientation, the vertical suction mat of the shoebox gripper 200 (which may be the same as or similar to the vertical suction mat 205 of FIGs. 2A and 2B) contacts the lid 305 of the shoebox 300. In some cases, in this third orientation, the horizontal suction mat of the shoebox gripper 200 may further be in contact with the lid 305 of the shoebox 300. In other cases, in this third orientation, the horizontal suction mat of the shoebox gripper 200 may be out of contact with the lid 305 of the shoebox 300. Advantageously, in this third orientation, the lid 305 is secured by the horizontal suction mat and the vertical suction mat working together. Further the two suction mats also provide lift and stability while moving the shoebox 300 through some or all of the 6 degrees of freedom. The third orientation may have a similar surface area over which the two suction mats of the shoebox 300 as in the first orientation.Examples of Conveyors

[0102] The systems, the methods, the computer-readable media, and the techniques disclosed herein may cooperate with a conveyor. A conveyor is a common piece of mechanical handling equipment that may move materials from one location to another. Many kinds of conveying systems are available and are used according to the various needs of different industries. For example, chainNomagic Inc. 59332-706.601 Application Specification.docx -30- WSGR Docket No. 59332-705.60165057 / 2024 / PMi conveyors (floor and overhead) may be types of conveying systems. Chain conveyors may include enclosed tracks, I-Beam, towline, power & free, and hand pushed trolleys. Conveyors may offer several advantages, including: increased efficiency, versatility, and cost-effectiveness. While conveyors are widely used and may offer numerous advantages, they also have certain limitations and shortcomings. For example, conveyors operate along a fixed path, which means they may not be suitable for applications that require flexible routing or changes in the material flow direction. Adding flexibility to the system may use additional complex mechanisms or multiple conveyor lines. Examples of Chutes

[0103] The systems, the methods, the computer-readable media, and the techniques disclosed herein may cooperate with a chute. A chute is a vertical or inclined plane, channel, or passage through which SKUs are moved by means of gravity. Chutes are commonly used in various industries for bulk material handling, allowing the controlled transfer of granular or bulky materials from higher to lower levels or between different processing stages. The design of chutes depends on the specific application and the characteristics of the materials being handled. The entry section of the chute is where the material is introduced into the chute from a higher elevation or conveyor. This section is designed to accommodate the flow of material smoothly and prevent any spillage or blockages. Chutes may include features like baffles or flow control gates to regulate the speed and flow of materials through the chute. These features can help prevent material surges and ensure a steady flow. The exit section of the chute is where the material discharges onto the lower level or conveyor. Chutes may be suited for free-flowing, granular, or bulk materials. Chutes may be less suited for handling cohesive materials, sticky substances, or materials with irregular shapes, as this can lead to blockages and flow issues. Depending on the drop height and material characteristics, the material flow in chutes can result in impact forces, potentially leading to material degradation or fines generation. For steeply inclined chutes, there may be limitations in controlling the material flow, leading to faster material acceleration and potentially causing material surges or damage to the chute.Examples of Pushers

[0104] The systems, the methods, the computer-readable media, and the techniques disclosed herein may cooperate with a pusher. A pusher, in the context of material handling and logistics, refers to a mechanical device or component used to move SKUs along a conveyor system or through a production line. The primary function of a pusher is to apply force to push or divert SKUs from one conveyor lane or processing stage to another. Pushers may be used in conveyor systems and automated manufacturing processes to perform tasks. Pushers may be used to divert products from Nomagic Inc. 59332-706.601 Application Specification.docx -31 - WSGR Docket No. 59332-705.60165057 / 2024 / PMi the main conveyor line to specific side lanes or different processing stages. This enables the sorting and distribution of SKUs based on certain criteria, such as destination, size, or product type. Pushers may be employed in sorting systems to direct SKUs to different designated destinations or shipping lanes based on predetermined criteria. Pushers may be used to stage SKUs for further processing or packaging. Pushers can transfer SKUs between conveyors or equipment in a production line, facilitating the smooth flow of materials. At very high speeds, pushers may not have enough time to properly engage with and push SKUs, leading to sorting or diverting errors. Achieving precise positioning and alignment of SKUs for proper pushing can be challenging, especially with varying sizes or misaligned SKUs. For applications involving complex sorting patterns or multiple destination lanes, the design and synchronization of multiple pushers can become intricate.Examples of Item Tilters

[0105] The systems, the methods, the computer-readable media, and the techniques disclosed herein may cooperate with an item tilter. An item tilter may be a mechanical device used to tilt or rotate SKUs, loads, or pallets to a specific angle. One use of an item tilter is to reorient materials or SKUs to facilitate easier handling, improve ergonomics, or aid in certain manufacturing or processing operations. The design of an item tilter may include a platform or surface on which the load or SKU is placed. The platform may be attached to a tilting mechanism that allows controlled tilting or rotation of a load. The tilting action can be achieved through hydraulic, pneumatic, or electric means, depending on the item tilter's design and intended application. Item tilters offer advantages in terms of improving efficiency, reducing manual handling strain, and enhancing the overall material handling process.

[0106] In some cases, automated systems, which may include robotic arms, handle unpacking SKUs from warehouse bins to cardboard boxes preparing them to be shipped to the final customer. In some cases, the order and position of incoming goods are random. Therefore, SKUs may be initially provided in positions which make it very difficult to place the SKU in the destination box in the position that optimizes the volume occupied inside the target box. In some cases, an item tilter is positioned at a good-to-robot station where SKUs are provided to a robot for picking and manipulation. The item tilter may facilitate proper packing in cases where the robot is unable to place SKUs on their flat side. In some cases, the item tilter will reorient SKUs in preparation for packing into a final container or box.

[0107] In some cases, the item tilter does not grasp, clamp, or grip the SKU being manipulated. In some cases, an item tilter which does not perform grasping, gripping, clamping, or similar actions prevents damage to the SKUs handled by the tilter. In some cases, the item tilter comprises aNomagic Inc. 59332-706.601 Application Specification.docx -32- WSGR Docket No. 59332-705.60165057 / 2024 / PMi substantially planar surface which the SKU is placed on. The surface may rotate in a specified direction to properly reorient the SKU in preparation for packing or manipulation by a robot. In some cases, the item tilter comprises two substantially planar surfaces, orthogonal to one another. In some cases, the SKU is placed against at both surfaces prior to rotation by the item tilter. In some cases, the SKU is placed only against one surface and gravity assists with abutting the SKU against the second surface. In some cases, the item tilter comprises two or more substantially planar surfaces, wherein connecting surfaces are orthogonal to one another. In some cases, the SKU is placed against at least one surface prior to rotation by the item tilter. In some cases, the item tilter is coupled to a product database, as disclosed herein. The product database may relay an appropriate speed of rotation to the item tilter based on characteristics of the SKU being handled, as to prevent damage, mishandling, or misplacement of the SKU.

[0108] In some cases, the item tilter comprises one or more surfaces which grasp, clamp, or otherwise hold the SKU during rotation. In some cases, the item tilter is capable of applying different pressures to hold the SKU. In some cases, the item tilter is coupled to an item database, as disclosed herein. The item database may relay an appropriate pressure based on characteristics of the SKU being handled, as to prevent damage, mishandling, or misplacement of the SKU. In some cases, a rotating surface of the item tilter comprises a suction effector to retain a SKU during rotation.

[0109] In some cases, the item tilter and robots of the system are operatively connected to a product database, programmable logic controller, computer system, or a combination thereof. In some cases, the device item tilter provides a ready for operation status. In some cases, the ready for operation comprises a digital output of ON and signifies when the new SKU can be placed in the device to be tilted. In some cases, the item tilter provides a final position digital output when a SKU is ready to be picked by an adjacent robot after being rotated into a desired orientation by the item tilter. In some cases, the item tilter receives a cycle start indication when rotation of the SKU is to begin. In some cases, automated systems disclosed herein may be able to make decisions not to put the SKU on the tilter based on the product database.

[0110] In some cases, an item tilter is provided at a product loading station. The product loading station may be a cell that includes a robot (e.g., possibly with a picking port), a tilter, a scanner, etc. The product loading station may comprise two or more item tilters. In some cases, the product loading station is provided in proximity to or adjacent to a loading apparatus or output thereof. In some cases, the loading apparatus provides the SKU to the item tilter. The loading apparatus may comprise a conveyor, a robotic handler, a chute, a pusher, or a combination thereof. In some cases, Nomagic Inc. 59332-706.601 Application Specification.docx -33- WSGR Docket No. 59332-705.60165057 / 2024 / PMi the loading apparatus provides two or more SKUs to the item tilter for simultaneous rotation of said two or more SKUs. In some cases, the product loading station comprises two or more item tilters. In some cases, an unloading apparatus is provided at the product loading station to move the SKU into proximity of or place the SKU inside the destination container. An unloading apparatus may include the item tilter, a conveyor, a robotic handler, a chute, a pusher, or a combination thereof.

[0111] In some cases, one or more sensors are provided at the product loading station. In some cases, a vision system comprising at least one optical sensor is provided at the product loading station. In some cases, the vision system identifies one or more characteristics of SKUs provided at the product loading station. In some cases, the vision system is in operative communication with the software module and a computer processor. In some cases, the software module instructs the unloading apparatus to move the SKU in proximity to or within a destination container. In some cases, the software module is operatively connected to a product database to determine one or more characteristics of a SKU, as disclosed herein. In some cases, the product database provides a desired orientation for a SKU provided at the product loading station. In some cases, automated systems may provide a decision based on the product database whether or not to put a SKU on the tilter. For example, a SKU may be deformable, and tilting will not help.

[0112] Operation of an item tilter may be understood as a cyclic process, wherein a cycle starts when a SKU is placed into the item tilter by a robot and a is complete when the SKU is provided in a final position and the item tilter is ready to receive a subsequent SKU. In some cases, when a SKU is placed into the item tilter by a robot, a signal is sent from a programmable logic controller (PLC) of the system to start the cycle. During the cycle, item tilter rotates the SKU, as disclosed herein. In some cases, as the item tilter is rotating the SKU, the robot will pick a second to be placed in the item tilter. In some cases, at the end of the cycle, the SKU is positioned in the final position. In some cases, the cycle ends when the device is ready for placing the next SKU. In some cases, the item tilter is ready for placing the next item even if the first one was not removed from the final position. In some cases, the robot picks the first SKU from the final position and places it in the destination container or box, as the second SKU is being rotated. In some cases, an additional robot is utilized, wherein one robot places SKUs into the item tilter and an additional robot places them into a destination container or box.

[0113] In some cases, the item tilter is ready to receive the next SKU for the operation while the previous SKU is positioned in the final position. In some cases, the item tilter is capable of tilting the next SKU even if the previous SKU is in the final position, as mentioned above. In some cases, the total cycle time as described above should take no more than 0.5, 1, 2, 3, 4, 5, 10, or 15 seconds.Nomagic Inc. 59332-706.601 Application Specification.docx -34- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0114] In some cases, the processes described above are carried out without prior determination of the shape of the SKU being handled. In some cases, the shape of the SKU being handled is provided by a product database or information gathered by sensors, as described herein.

[0115] In some cases, an item tilter is provided as a component of an automated warehouse. In some cases, an item tilter is adjacent to one or more conveyor belts. In some cases, an item tilter is adjacent to one or more components for automated movement of SKUs. The automated components may include a robotic arm, a conveyor belt or system, a chute system, a pushing apparatus, or a combination thereof.Examples of Humans in the Loop

[0116] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein communicate with an operator or other user. The systems, the methods, the computer-readable media, and the techniques disclosed herein may communicate with an operator using a computing device. The computing device may be an operator device. The computing device may be configured to receive input from an operator or user with a user interface. The operator device may be provided at a location remote from operations of the facility.

[0117] In some cases, an operator utilizes an operator device to verify one or more anomaly events or alerts. In some cases, the operator device receives captured images from one or more image sensors to verify that an anomaly has occurred in a SKU. An operator may provide verification that a SKU has been misplaced or that a SKU has been damaged based on the one or more images.

[0118] In some cases, captured images are provided in a module to be displayed on a screen of an operator device. In some cases, the module displays the one or more captured images adjacent to one or more reference images corresponding to said SKU. In some cases, one or more captured images are displayed on a page adjacent to a page displaying one or more reference images.

[0119] In some cases, an operator uses an interface of the operating device to verify that an anomaly event or alert was correctly generated. Verification provided by the operator may be used to train a machine learning algorithm, as disclosed herein. In some cases, verification that an alert was correctly generated adjusts a predetermined threshold which is used to generate an alert if a difference between one or more measured properties and one or more corresponding expected properties of a SKU exceeds said predetermined threshold. In some cases, verification that an alert was incorrectly generated adjusts a predetermined threshold which is used to generate an alert if a difference between one or more measured properties and one or more corresponding expected properties of a SKU exceeds said predetermined threshold.Nomagic Inc. 59332-706.601 Application Specification.docx -35- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0120] In some cases, verification of an alert instructs a robotic arm to handle a SKU in a particular manner. For example, if an anomaly alert corresponding to a SKU is verified as being correctly generated, the robotic arm may place the SKU at an exception location. In some cases, if an anomaly alert corresponding to a SKU is verified as being incorrectly generated, the robotic arm may place the SKU at a target location. In some cases, if an alert is generated and an operator verifies that two or more SKUs are unintentionally being handled simultaneously, then the robotic arm performs a wiggling motion in an attempt to separate the two or more SKUs.

[0121] In some cases, one or more images of a target container or target location wherein one or more SKUs are provided at are transmitted to an operator or user device. An operator or user may then verify that the one or more SKUs are correctly placed at the target location or with a target container. A user or operator may also provide feedback using an operator or user device to communicate errors if the one or more SKUs have been incorrectly placed at the target location or within the target container.

[0122] In some cases, it may be determined that human intervention is required for proper handling of a SKU type. In some cases, a specific SKU may require manual handling or packaging by human operators. As disclosed herein, a database may provide information as to which SKUs requires human intervention or handling. In some cases, a warehouse surveillance or monitoring system alerts human handlers to incoming SKUs which require human intervention. In some cases, upon detection of a SKU requiring human intervention, the systems, the methods, the computer-readable media, and the techniques disclosed herein route said SKU to a station designated for human intervention. The station may be separated from automated handling systems or robotic arms. Separation may be necessary for safety reasons or to provide an accessible area for a human to handle the SKUs. Example Surveillance System

[0123] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may operate using a surveillance system for monitoring operations or product flow in a facility. The surveillance system may operate with a picking port. For example, the surveillance system may monitor SKUs as the SKUs move through a picking port (e.g., from a picking location to a dropping location). In some cases, the surveillance system is integrated into an existing warehouse with automated handling systems. In some cases, the surveillance system comprises a database of information for each SKU to be handled in the warehouse. In some cases, the database is updated, as disclosed herein.

[0124] In some cases, the surveillance system comprises at least one image sensor. In some cases, the surveillance system allows for identification of a SKU type. In some cases, identification of aNomagic Inc. 59332-706.601 Application Specification.docx -36- WSGR Docket No. 59332-705.60165057 / 2024 / PMiSKU type at one or more points through a product flow in a facility allows for monitoring to determine if the facility is running efficiently or if an anomaly has occurred. In some cases, the surveillance system allows for determination of an appropriate SKU size for the one or more products to be placed and packaged within. In some cases, the surveillance system allows for automated quality control of products and packaging within a facility.

[0125] In some cases, an image sensor is provided prior to or upstream from an automated handling station. An image sensor provided prior to an automated handling system may allow for proper preparation by the handling system prior to arrival of a specific SKU type. In some cases, an image sensor provided prior to an automated handling system captures one or more images of a SKU to facilitate determination of an appropriate handler the SKU should be sent to.

[0126] In some cases, an image sensor provided prior to an automated handling system identifies if a SKU has been misplaced or will not be able to be handled by an automated system downstream from the image sensor.

[0127] In some cases, a surveillance system comprises one or more image sensors located after or downstream from an automated handling robot or system. In some cases, an image sensor provided downstream from a handling station captures one or more images of a SKU after being handled or placed to verify correct placement or handling. Verification may be done on SKUs handled on an automated system or by a human handler.

[0128] In some cases, the surveillance system includes further sensors, such as weight sensors, motion sensors, laser scanners, or other sensors useful for gathering information related to a SKU. Examples of Warehouse Automation

[0129] The systems, the methods, the computer-readable media, and the techniques disclosed herein may be implemented in existing warehouses to automate one or more processes within a warehouse. In some cases, software and robotic arms of the system are integrated with the existing warehouse systems to provide a smooth transition of manual operations being automated.

[0130] In some cases, a product database is provided in communication with the systems, the methods, the computer-readable media, and the techniques disclosed herein. The product database may comprise a library of SKU to be handled, e.g., in a picking port. The product database may include properties of each SKUs to be handled. In some cases, the properties of the SKUs provided by the product data base are expected properties of the SKUs. The expected properties of the SKUs may be compared to measured properties of the SKUs in order to determine if an anomaly has occurred.Nomagic Inc. 59332-706.601 Application Specification.docx -37- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0131] Expected properties may include expected dimensions, expected forces, expected weights, and expected machine -readable codes, as disclosed herein. Product databases may be updated according to the SKUs to be handled. Product databases may be generated input of information of the SKUs to be handled by handled.

[0132] In some cases, SKUs may be processed by the systems, the methods, the computer-readable media, and the techniques disclosed herein to generate a product database. For example, an undamaged SKU may be handled by one or more robotic arms to determine expected properties of the SKU. Expected properties of the SKU may include expected dimensions, expected forces, expected weights, and expected machine -readable codes, as disclosed herein. The expected properties may then be input into the product database.

[0133] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may process a plurality of SKUs of the same type to determine a standard deviation occurring within SKUs of that type. The determined standard deviations may be used to set a predetermined threshold, wherein a difference between expected properties and measured properties of a SKU may trigger an anomaly alert. In some cases, the predetermined threshold includes a standard deviation of different of one or more SKUs of the same type. In some cases, the standard deviation is multiplied by a constant factor to set a predetermined threshold.

[0134] In some cases, the product database comprises a set of filtering criterion. The filtering criterion may be used for routing SKUs to a proper handling station. Filtering criterion may be used for routing SKUs to a robotic handling station or a human handling station. Filtering criterion may be utilized for routing SKUs to an appropriate robotic handing station with an automated handler suited for handling a particular SKU.

[0135] In some cases, the database is continually updated. In some cases, the filtering criterion is continually updated. In some cases, the filtering criterion may be updated as new handling systems are integrated within a facility. In some cases, the filtering criterion is updated as new SKU types are handlined within a facility. In some cases, the filtering criterion is updated as new manipulation techniques or handling patterns are realized. In some cases, a machine learning program is utilized to update the database or filtering criterion.

[0136] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein track SKUs as they are handled. In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein integrate with existing tracking software of a warehouse which the system is implemented within. The systems, the methods, the computer-readable media, and the techniques disclosed herein may connect with existing software Nomagic Inc. 59332-706.601 Application Specification.docx -38- WSGR Docket No. 59332-705.60165057 / 2024 / PMi such that information which is normally received by manual input is now communicated electronically.

[0137] SKU tracking may include confirming a SKU has been received at a source locations or station. SKU tracking may include confirming a SKU has been placed at a target position. SKU tracking may include input that an anomaly has been detected. SKU tracking may include input that a SKU has been placed at an exception location. SKU tracking may include input that a SKU or target container has left a handling station or target position to be further processed at another location within a warehouse.Examples of Integrated Software

[0138] Many or all of the functions of a robotic device may be controlled by a control system. A control system may include at least one processor that executes instructions stored in a non- transitory computer readable medium, such as a memory. The control system may also comprise a plurality of computing devices that may serve to control individual components or subsystems of the robotic device.

[0139] In some cases, a memory comprises instructions (e.g., program logic) executable by the processor to execute various functions of robotic device disclosed herein. A memory may comprise additional instructions as well, including instructions to transmit data to, receive data from, interact with, or control one or more of a mechanical system, a sensor system, a product database, an operator system, or the control system.

[0140] In some cases, machine learning algorithms are implemented such that the systems, the methods, the computer-readable media, and the techniques disclosed herein become completely automated. In some cases, verification operations completed by a human operator are removed after training of machine learning algorithms are complete.

[0141] In some cases, the machine learning programs utilize incorporate a supervised learning approach. In some cases, the machine learning programs utilized incorporate a reinforcement learning approach. Information such as verification of alerts / anomaly events, measured properties of SKUs being handled, and expected properties of SKUs being handled by be received by a machine learning algorithm for training.

[0142] Other machine learning approaches such as unsupervised learning, feature learning, topical modeling, dimensionality reduction, and meta learning may be utilized by the system. Supervised learning may include active learning algorithms, classification algorithms, similarity learning algorithms, regressive learning algorithms, and combinations thereof.Nomagic Inc. 59332-706.601 Application Specification.docx -39- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0143] In some cases, machine learning may generally involve identifying and recognizing patterns in existing data in order to facilitate making predictions for subsequent data, machine learning may include a machine learning model (which may include, for example, a machine learning algorithm). Machine learning, whether analytical or statistical in nature, may provide deductive or abductive inference based on real or simulated data. The machine learning model may be a trained model, machine learning techniques may comprise one or more supervised, semi-supervised, selfsupervised, or unsupervised machine learning techniques. For example, a machine learning model may be a trained model that is trained through supervised learning (e.g., various parameters are determined as weights or scaling factors), machine learning may comprise one or more of regression analysis, regularization, classification, dimensionality reduction, ensemble learning, meta learning, association rule learning, cluster analysis, anomaly detection, deep learning, or ultra-deep learning, machine learning may comprise: k-means, k-means clustering, k-nearest neighbors, learning vector quantization, linear regression, non-linear regression, least squares regression, partial least squares regression, logistic regression, stepwise regression, multivariate adaptive regression splines, ridge regression, principal component regression, least absolute shrinkage and selection operation (LASSO), least angle regression, canonical correlation analysis, factor analysis, independent component analysis, linear discriminant analysis, multidimensional scaling, non-negative matrix factorization, principal components analysis, principal coordinates analysis, projection pursuit, Sammon mapping, t-distributed stochastic neighbor embedding, AdaBoosting, boosting, gradient boosting, bootstrap aggregation, ensemble averaging, decision trees, conditional decision trees, boosted decision trees, gradient boosted decision trees, random forests, stacked generalization, Bayesian networks, Bayesian belief networks, naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, hidden Markov models, hierarchical hidden Markov models, support vector machines, encoders, decoders, auto-encoders, stacked auto-encoders, perceptrons, multi-layer perceptrons, artificial neural networks, feedforward neural networks, convolutional neural networks, recurrent neural networks, long short-term memory, deep belief networks, deep Boltzmann machines, deep convolutional neural networks, deep recurrent neural networks, large language models, vision transformers, or generative adversarial networks.

[0144] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more computer vision techniques for tracking and monitoring SKUs, robotic arms, lid positioning of a shoebox, etc. Computer vision is a field of artificial intelligence that uses computers to interpret and understand the visual world at least in part by processing one or more ofNomagic Inc. 59332-706.601 Application Specification.docx -40- WSGR Docket No. 59332-705.60165057 / 2024 / PMi images, videos, or even in some cases, audio. In some instances, computer vision may use deep learning models (e.g., convolutional neural networks).

[0145] Object detection is a technique used in computer vision for both (1) identification and (2) localization of objects within an image or a video. Image identification aims to predict the class of an image or an object within an image into one of a category or class. Examples of categories or classes may include, e.g., human, animal, car, tree, bird, shirt, ocean, happy, outdoors, etc. Image localization is the process of identifying the correct location of one or multiple objects. Bounding boxes may be used in object detection techniques and image localization within computer vision. Bounding boxes may be annotation markers drawn around objects in an image (e.g., a frame of a video). Bounding boxes may be often, although not always, rectangularly shaped. In some cases, bounding boxes may be applied by humans to training data sets. However, bounding boxes may also be applied to images by a trained machine learning that is trained to detect one or more different objects (e.g., humans, hands, faces, cars, etc.). In addition or in alternative to bounding boxes, detection and tracking techniques may use any object detection annotation techniques, such as semantic segmentation, instance segmentation, polygon annotation, non-polygon annotation, landmarking, 3D cuboids, etc.

[0146] Machine learning algorithms may be applied to anomaly detection, as disclosed herein. In some cases, machine learning algorithms are applied to programed movement of one or more robotic arms. Machine learning algorithms applied to programmed movement of robotic arms may be used to optimize actions such as scanning a machine-readable code provided on a SKU. Machine learning algorithms applied to programmed movement of robotic arms may be used to optimize actions such performing a wiggling motion to separate unintentionally combined SKUs. Machine learning algorithms applied to programmed movement of robotic arms may be used to any actions of a robotic arm for handling one or more SKUs, as disclosed herein. In some cases, machine learning algorithms are applied to make decisions whether or not to put a SKU on the tilter.

[0147] In some cases, trajectories of SKUs handled by robotic arms are automatically optimized by the systems, the methods, the computer-readable media, and the techniques disclosed herein. In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein automatically adjust the movements of the robotic arms to achieve a minimum transportation time while preserving constraints on forces exerted on the SKU or package being transported.

[0148] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein monitor forces exerted on the SKU as they are transported from a source position to a target position, as disclosed herein. The systems, the methods, the computer-readable media, and Nomagic Inc. 59332-706.601 Application Specification.docx -41- WSGR Docket No. 59332-705.60165057 / 2024 / PMi the techniques disclosed herein may monitor acceleration, rate of acceleration (jerk), etc. of a SKU being transported by a robotic arm. The force experienced by the SKU as it is manipulated may be calculated using the known movement of the robotic arm (e.g., position, velocity, and acceleration values of the robotic arm as it transports the SKU) and force values obtained by the weight / torsion and force sensors provided on the robotic arm.

[0149] In some cases, optical sensors of the systems, the methods, the computer-readable media, and the techniques disclosed herein monitor the movement of SKUs being transported by the robotic arm. In some cases, the trajectory of SKUs is optimized to minimize transportation time including scanning of a digital code on the SKU. In some cases, the optical sensors recognize defects in the SKUs or packaging of SKUs as a result of mishandling (e.g., defects caused by forces applied to the SKU by the robotic arm). In some cases, the optical sensors monitor the flight or trajectory of SKUs being manipulated for cases which the SKUs are dropped. In some cases, detection of mishandling or drops will result in adjustments of the robotic arm (e.g., adjustment of trajectory or forces applied at the end effector). In some cases, the constraints and optimized trajectory information will be stored in the product database, as disclosed herein. In some cases, the constraints are derived from a history of attempts for the specific SKU or plurality of similar SKUs being transported. In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein are trained by increasing the speed at which a SKU is manipulated over a plurality of attempts until a drop or defect occurs due to mishandling by the robotic arm.

[0150] In some cases, a technician verifies that a defect or drop has occurred due to mishandling. Verification may include viewing a video recording of the SKU being handled and confirming that a drop or defect was likely due to mishandling by the robotic arm.Example Computer System

[0151] The present disclosure provides computer control systems that are programmed to implement the methods, the computer-readable media, and the techniques of the disclosure. FIG. 4 shows a computer system 401 that is programmed or otherwise configured to implement the methods, the computer-readable media, and the techniques disclosed herein, such as to control the systems or devices disclosed herein (e.g., a shoebox gripper, a robotic arm, a controller, an infeed system, an outfeed system, etc.). The computer system 401 can be an electronic device of a user or a computer system that is remotely located with respect to the electronic device. The electronic device can be a mobile electronic device.

[0152] The computer system 401 includes a central processing unit (CPU, also “processor” and “computer processor” herein) 405, which can be a single core or multi core processor, or a plurality Nomagic Inc. 59332-706.601 Application Specification.docx -42- WSGR Docket No. 59332-705.60165057 / 2024 / PMi of processors for parallel processing. The computer system 401 also includes memory or memory location 410 (e.g., random-access memory, read-only memory, flash memory), electronic storage unit 415 (e.g., hard disk), communication interface 420 (e.g., network adapter) for communicating with one or more other systems, and peripheral devices 425, such as cache, other memory, data storage or electronic display adapters. The memory 410, storage unit 415, interface 420 and peripheral devices 425 are in communication with the CPU 405 through a communication bus (solid lines), such as a motherboard. The storage unit 415 can be a data storage unit (or data repository) for storing data. The computer system 401 can be operatively coupled to a computer network (“network”) 430 with the aid of the communication interface 420. The network 430 can be the Internet, an isolated or substantially isolated internet or extranet, or an intranet or extranet that is in communication with the Internet. The network 430 in some cases is a telecommunication or data network. The network 430 can include one or more computer servers, which can enable distributed computing, such as cloud computing. The network 430, in some cases with the aid of the computer system 401 , can implement a peer-to-peer network, which may enable devices coupled to the computer system 401 to behave as a client or a server. The CPU 405 can execute a sequence of machine-readable instructions, which can be embodied in a program or software. The instructions may be stored in a memory location, such as the memory 410. The instructions can be directed to the CPU 605, which can subsequently program or otherwise configure the CPU 405 to implement methods of the present disclosure. Examples of operations performed by the CPU 405 can include fetch, decode, execute, and writeback. The CPU 405 can be part of a circuit, such as an integrated circuit. One or more other components of the system 401 can be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC).

[0153] The storage unit 415 can store files, such as drivers, libraries and saved programs. The storage unit 415 can store user data, e.g., user preferences and user programs. The computer system 401 in some cases can include one or more additional data storage units that are external to the computer system 401, such as located on a remote server that is in communication with the computer system 401 through an intranet or the Internet.

[0154] The computer system 401 can communicate with one or more remote computer systems through the network 430. For instance, the computer system 401 can communicate with a remote computer system of a user. Examples of remote computer systems include personal computers (e.g., portable PC), slate or tablet PC’s (e.g., Apple® iPad, Samsung® Galaxy Tab), telephones, Smart phones (e.g., Apple® iPhone, Android-enabled device, Blackberry®), or personal digital assistants. The user can access the computer system 401 via the network 430.Nomagic Inc. 59332-706.601 Application Specification.docx -43- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0155] Methods as disclosed herein can be implemented by way of machine (e.g., computer processor) executable code stored on an electronic storage location of the computer system 401, such as, for example, on the memory 410 or electronic storage unit 415. The machine executable or machine-readable code can be provided in the form of software. During use, the code can be executed by the processor 405. In some cases, the code can be retrieved from the storage unit 415 and stored on the memory 410 for ready access by the processor 405. In some situations, the electronic storage unit 415 can be precluded, and machine-executable instructions are stored on memory 410. The code can be pre-compiled and configured for use with a machine having a processor adapted to execute the code or can be compiled during runtime. The code can be supplied in a programming language that can be selected to enable the code to execute in a pre-compiled or as-compiled fashion.

[0156] Aspects of the systems and methods provided herein, such as the computer system 401, can be embodied in programming. Various aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of machine (or processor) executable code or associated data that is carried on or embodied in a type of machine readable medium. Machineexecutable code can be stored on an electronic storage unit, such as memory (e.g., read-only memory, random-access memory, flash memory) or a hard disk. “Storage” type media can include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server. Thus, another type of media that may bear the software elements includes optical, electrical, and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible “storage” media, terms such as computer or machine “readable medium” refer to any medium that participates in providing instructions to a processor for execution.

[0157] Hence, a machine readable medium, such as computer-executable code, may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical Nomagic Inc. 59332-706.601 Application Specification.docx -44- WSGR Docket No. 59332-705.60165057 / 2024 / PMi transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings. Volatile storage media include dynamic memory, such as main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system. Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution

[0158] Aspects of the systems and methods provided herein, such as the computer system 401, can be embodied in programming. Various aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of machine (or processor) executable code or associated data that is carried on or embodied in a type of machine readable medium. Machineexecutable code can be stored on an electronic storage unit, such as memory (e.g., read-only memory, random-access memory, flash memory) or a hard disk. “Storage” type media can include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server. Thus, another type of media that may bear the software elements includes optical, electrical, and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible “storage” media, terms such as computer or machine Nomagic Inc. 59332-706.601 Application Specification.docx -45- WSGR Docket No. 59332-705.60165057 / 2024 / PMi“readable medium” refer to any medium that participates in providing instructions to a processor for execution.

[0159] Hence, a machine readable medium, such as computer-executable code, may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings. Volatile storage media include dynamic memory, such as main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system. Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.

[0160] The computer system 401 can include or be in communication with an electronic display 435 that comprises a user interface (UI) 440 for providing, for example, images (e.g., micrographs) of the substrates or the plurality of beads, along with the analysis of the images (e.g., pitch, spacing, occupancy, intensity, nucleic acid sequence data, etc.). Examples of UTs include, without limitation, a graphical user interface (GUI) and web-based user interface.

[0161] Methods and systems of the present disclosure can be implemented by way of one or more algorithms. An algorithm can be implemented by way of software upon execution by the central processing unit 405. The algorithm can, for example, determine the occupancy, spacing, or other parameters (e.g., full-width half-maximum, mean fluorescence intensity) of an image (e.g., micrograph of a bead or plurality of beads on or adjacent to a substrate).Certain Definitions and Additional Considerations

[0162] Unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present subject matter belongs.Nomagic Inc. 59332-706.601 Application Specification.docx -46- WSGR Docket No. 59332-705.60165057 / 2024 / PMi

[0163] As used in this specification and the appended claims, “some embodiments,” “further embodiments,” or “a particular embodiment,” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in some embodiments,” or “in further embodiments,” or “in a particular embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0164] As used in this specification and the appended claims, when the term “at least,” “greater than,” or “greater than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “at least,” “greater than” or “greater than or equal to” applies to each of the numerical values in that series of numerical values. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.

[0165] As used in this specification and the appended claims, when the term “no more than,” “less than,” or “less than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “no more than,” “less than,” or “less than or equal to” applies to each of the numerical values in that series of numerical values. For example, less than or equal to 3, 2, or 1 is equivalent to less than or equal to 3, less than or equal to 2, or less than or equal to 1.

[0166] As used in this specification, “or” is intended to mean an “inclusive or” or what is also known as a “logical OR,” wherein when used as a logic statement, the expression “A or B” is true if either A or B is true, or if both A and B are true, and when used as a list of elements, the expression “A, B or C” is intended to include all combinations of the elements recited in the expression, for example, any of the elements selected from the group consisting of A, B, C, (A, B), (A, C), (B, C), and (A, B, C); and so on if additional elements are listed. As such, any reference to “or” herein is intended to encompass “and / or” unless otherwise stated.

[0167] As used in this specification and the appended claims, the indefinite articles “a” or “an,” and the corresponding associated definite articles “the” or “said,” are each intended to mean one or more unless otherwise stated, implied, or physically impossible. Yet further, it should be understood that the expressions “at least one of A and B, etc.,” “at least one of A or B, etc.,” “selected from A and B, etc.” and “selected from A or B, etc.” are each intended to mean either any recited element individually or any combination of two or more elements, for example, any of the elements from the group consisting of “A,” “B,” and “A AND B together,” etc.

[0168] As used in this specification and the appended claims “about” or “approximately” may mean within an acceptable error range for the value, which will depend in part on how the value isNomagic Inc. 59332-706.601 Application Specification.docx -47- WSGR Docket No. 59332-705.60165057 / 2024 / PMi measured or determined, e.g., the limitations of the measurement system. For example, “about” may mean within 1 or more than 1 standard deviation, per the practice in the art. Alternatively, “about” may mean a range of up to 20%, up to 10%, up to 5%, or up to 1% of a given value. Where values are described in the application and claims, unless otherwise stated the term “about” meaning within an acceptable error range for the particular value may be assumed.

[0169] While preferred embodiments of the present invention have been shown and disclosed herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. It is not intended that the invention be limited by the specific examples provided within the specification. While the invention has been described with reference to the aforementioned specification, the descriptions and illustrations of the embodiments herein are not meant to be construed in a limiting sense. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the invention. Furthermore, it shall be understood that all aspects of the invention are not limited to the specific depictions, configurations or relative proportions set forth herein which depend upon a variety of conditions and variables. It should be understood that various alternatives to the embodiments of the invention disclosed herein may be employed in practicing the invention. It is therefore contemplated that the invention shall also cover any such alternatives, modifications, variations, or equivalents. It is intended that the following claims define the scope of the invention and that methods and structures within the scope of these claims and their equivalents be covered thereby.

[0170] It should be noted that various illustrative or suggested ranges set forth herein are specific to their example embodiments and are not intended to limit the scope or range of disclosed technologies, but, again, merely provide example ranges for frequency, amplitudes, etc. associated with their respective embodiments or use cases. Where values are described as ranges, it will be understood that such disclosure includes the disclosure of all possible sub-ranges within such ranges, as well as specific numerical values that fall within such ranges irrespective of whether a specific numerical value or specific sub-range is expressly stated.

[0171] It should be understood that, unless a term is expressly defined in this patent, there is no intent to limit the meaning of that term, either expressly or by implication, beyond its plain or ordinary meaning, and such term should not be interpreted to be limited in scope based at least in part on any statement made in any section of this patent (other than the language of the claims). To the extent that any term recited in the claims at the end of this patent is referred to in this patent in a manner consistent with a single meaning, that is done for sake of clarity only so as to not confuse theNomagic Inc. 59332-706.601 Application Specification.docx -48- WSGR Docket No. 59332-705.60165057 / 2024 / PMi reader, and it is not intended that such claim term be limited, by implication or otherwise, to that single meaning.

[0172] Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0173] Additionally, certain embodiments are disclosed herein as including logic or a number of routines, subroutines, applications, or instructions. These may constitute either software (e.g., code embodied on a machine-readable medium) or hardware. In hardware, the routines, etc., are tangible units capable of performing certain operations and may be configured or arranged in a certain manner. In example embodiments, one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as disclosed herein.

[0174] In various embodiments, a hardware module may be implemented mechanically or electronically. For example, a hardware module may comprise dedicated circuitry or logic that is permanently configured (e.g., as a special-purpose processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC) to perform certain operations. A hardware module may also comprise programmable logic or circuitry (e.g., as encompassed within a general-purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.

[0175] Accordingly, hardware modules may encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations disclosed herein.Considering embodiments in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any oneNomagic Inc. 59332-706.601 Application Specification.docx -49- WSGR Docket No. 59332-705.60165057 / 2024 / PMi instance in time. For example, where the hardware modules comprise a general-purpose processor configured using software, the general-purpose processor may be configured as respective different hardware modules at different times. Software may accordingly configure processor, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time.

[0176] Hardware modules may provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple of such hardware modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) that connect the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and may operate on a resource (e.g., a collection of information). Elements that are described as being coupled and or connected may refer to two or more elements that may be (e.g., direct physical contact) or may not be (e.g., electrically connected, communicatively coupled, etc.) in direct contact with each other, but yet still cooperate or interact with each other.

[0177] The various operations of example methods disclosed herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions. The modules referred to herein may, in some example embodiments, comprise processor-implemented modules.

[0178] Similarly, the methods or routines disclosed herein may be at least partially processor- implemented. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented hardware modules. The performance of certain operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processor or processors may be located in a single location (e.g., within a home environment, an office Nomagic Inc. 59332-706.601 Application Specification.docx -50- WSGR Docket No. 59332-705.60165057 / 2024 / PMi environment or as a server farm), while in other embodiments the processors may be distributed across a number of locations.

[0179] The performance of certain operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the one or more processors or processor-implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the one or more processors or processor-implemented modules may be distributed across a number of geographic locations.

[0180] It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element may be termed a second element, and, similarly, a second element may be termed a first element, without departing from the scope of the present disclosure.Nomagic Inc. 59332-706.601 Application Specification.docx -51- WSGR Docket No. 59332-705.601

Claims

65057 / 2024 / PMiCLAIMSWHAT IS CLAIMED IS:

1. A method for warehouse management, comprising:(A) identifying, in an infeed system comprising a plurality of stock keeping units that comprises one or more shoeboxes and one or more non-shoeboxes, a shoebox of said one or more shoeboxes;(B) at least in part in response to identifying said shoebox at (A), configuring a picker with a shoebox gripper; and(C) causing said picker to handle said shoebox using said shoebox gripper.

2. The method of claim 1 , wherein said infeed system comprises an unstructured source comprising said plurality of stock keeping units.

3. The method of claim 2, wherein said unstructured source comprises one or more bins comprising said plurality of stock keeping units4. The method of any one of the preceding claims, wherein said one or more nonshoeboxes comprise one or more fashion stock keeping units.

5. The method of claim 4, wherein said one or more fashion stock keeping units comprise one or more of: garment stock keeping units, handbag or backpack stock keeping units, or hat stock keeping units.

5. The method of any one of the preceding claims, wherein identifying said shoebox at (A) comprises:(i) obtaining image data corresponding to said shoebox; and(ii) applying a machine learning model to said image data.

6. The method of any one of the preceding claims, further comprising:Nomagic Inc. 59332-706.601 Application Specification.docx -52- WSGR Docket No. 59332-705.60165057 / 2024 / PMi(D) scanning a code corresponding to said shoebox, wherein said code comprises one or more of: a one-dimensional code, a two-dimensional code, or a three-dimensional code.

7. The method of any one of the preceding claims, wherein causing said picker to handle said shoebox using said shoebox gripper at (C) comprises: causing said picker to move, using said shoebox gripper, said shoebox from said infeed system to an outfeed system.

8. The method of any one of the preceding claims, further comprising:(E) detecting an anomaly.

9. The method of claim 8, wherein said anomaly comprises damage to said shoebox.

10. The method of claim 8 or 9, wherein said anomaly comprises dropping said shoebox while causing said picker to handle said shoebox using said shoebox gripper at (C).

11. The method of any one of claims 8-10, wherein said anomaly comprises said picker handling an additional stock keeping unit in addition to said shoebox while causing said picker to handle said shoebox using said shoebox gripper at (C).

12. The method of any one of claims 8-11, wherein said anomaly comprises a lid of said shoebox opening while causing said picker to handle said shoebox using said shoebox gripper at (C).

13. The method of any one of claims 8-12, further comprising:(F) at least in part in response to detecting an anomaly at (E), indicating said anomaly via an alarm comprising one or both of audio or visual indications.

14. The method of any one of claims 8-13, wherein detecting said anomaly at (E) comprises:(i) obtaining image data corresponding to said shoebox; and(ii) applying a machine learning model to said image data.Nomagic Inc. 59332-706.601 Application Specification.docx -53- WSGR Docket No. 59332-705.60165057 / 2024 / PMi15. The method of any one of the preceding claims, wherein said shoebox gripper comprises a first contact configured to contact a first surface of said shoebox and a second contact configured to contact a second surface of said shoebox, wherein said first surface of said shoebox is adjacent to said second surface of said shoebox.

16. The method of claim 15, wherein said first surface of said shoebox is substantially perpendicular to said second surface of said shoebox.

17. The method of claim 15 or 16, wherein said first contact comprises a first suction mat and said second contact comprises a second suction mat.

18. The method of claim 17, wherein said first suction mat is gaseously coupled to a first vacuum tubing and said second suction mat is gaseously coupled to a second vacuum tubing.

19. The method of any one of claims 15-18, wherein said first surface of said shoebox comprises a lid of said shoebox.

20. The method of claim 19, wherein said second surface of said shoebox comprises a body of said shoebox.

21. A shoebox gripper, comprising:(A) a main body;(B) an adapter at a first distal end of said main body;(C) a pair of grippers at a second distal end of said main body, wherein said pair of grippers comprises a first suction mat and a second suction mat, wherein said first suction mat is substantially perpendicular to said second suction mat; and(D) one or more vacuum tubings gaseously coupled to said pair of grippers.

22. The shoebox gripper of claim 21 , wherein said first suction mat is configured to contact a body of a shoebox or a lid of a shoebox.

23. The shoebox gripper of claim 21 or 22, wherein said first suction mat and said second suction mat have a combined surface area of less than 0.5 square meters.Nomagic Inc. 59332-706.601 Application Specification.docx -54- WSGR Docket No. 59332-705.60165057 / 2024 / PMi24. The shoebox gripper of any one of claims 21-23, wherein said one or more vacuum tubings are configured to provide a lifting force of less than about 3 kilograms.

25. The shoebox gripper of any one of claims 21- 24, wherein said pair of grippers are configured to pick a shoebox out of an infeed system that comprises an unstructured source comprising a plurality of stock keeping units that comprises said shoebox and one or more nonshoeboxes, and wherein said pair of grippers are configured to deposit said shoebox into an outfeed system.

26. The shoebox gripper of claim 25, wherein said one or more non-shoeboxes comprise one or more fashion stock keeping units.

27. The shoebox gripper of claim 26, wherein said one or more fashion stock keeping units comprise one or more of: garment stock keeping units, handbag or backpack stock keeping units, or hat stock keeping units.

28. The shoebox gripper of any one of claims 21- 27, wherein said adapter is configured to detachably affix to a robotic arm.

29. The shoebox gripper of claim 28, wherein said robotic arm comprises one or more optical sensors.

30. The shoebox gripper of claim 29, wherein said robotic arm further comprises a control system configured to:(i) obtain image data from said one or more optical sensors;(ii) apply a machine learning model to said image data to identify a shoebox; and(iii) at least in part in response to identifying said shoebox, configure said robotic arm to detachably affix to said adapter.

31. The method of claim any one of claims 21-30, wherein said first suction mat is gaseously coupled to a first vacuum tubing of said one or more vacuum tubings and said second suction mat is gaseously coupled to a second vacuum tubing of said one or more vacuum tubings. Nomagic Inc. 59332-706.601 Application Specification.docx -55- WSGR Docket No. 59332-705.60165057 / 2024 / PMi32. A system for warehouse management, comprising:(A) one or more optical sensors configured to collect image data of an infeed system comprising a plurality of stock keeping units that comprises one or more shoeboxes and one or more non-shoeboxes;(B) a computing device configured to identify, from said image data, a shoebox of said one or more shoeboxes; and(C) a control device configured to, at least in part in response to said computing device identifying said shoebox, send a control signal to cause (i) configuring a picker with a shoebox gripper, and (ii) said picker to handle said shoebox using said shoebox gripper.

33. The system of claim 32, wherein said infeed system comprises an unstructured source comprising said plurality of stock keeping units.

34. The system of claim 44, wherein said unstructured source comprises one or more bins comprising said plurality of stock keeping units35. The system of any one of claims 32-34, wherein said one or more non-shoeboxes comprise one or more fashion stock keeping units.

36. The system of claim 35, wherein said one or more fashion stock keeping units comprise one or more of: garment stock keeping units, handbag or backpack stock keeping units, or hat stock keeping units.

37. The system of any one of claims 32-36, wherein said computing device is configured to identify said shoebox by:(i) obtaining image data corresponding to said shoebox; and(ii) applying a machine learning model to said image data.

38. The system of any one of claims 32-37, wherein said control device is further configured to cause said picker to:Nomagic Inc. 59332-706.601 Application Specification.docx -56- WSGR Docket No. 59332-705.60165057 / 2024 / PMi(D) position said shoebox to have a code corresponding to said shoebox scanned, wherein said code comprises one or more of: a one-dimensional code, a two-dimensional code, or a three-dimensional code.

39. The system of any one of claims 32-38, wherein handling said shoebox using a shoebox gripper comprises moving said shoebox into an outfeed system.

40. The system of any one of claims 32-39, wherein said shoebox gripper comprises a first contact configured to contact a first surface of said shoebox and a second contact configured to contact a second surface of said shoebox, wherein said first surface of said shoebox is adjacent to said second surface of said shoebox.

41. The system of claim 40, wherein said first surface of said shoebox is substantially perpendicular to said second surface of said shoebox.

42. The system of claim 40 or 41, wherein said first contact comprises a first suction mat and said second contact comprises a second suction mat.

43. The system of claim 42, wherein said first suction mat is gaseously coupled to a first vacuum tubing and said second suction mat is gaseously coupled to a second vacuum tubing.

44. The system of any one of claims 40-43, wherein said first surface of said shoebox comprises a lid of said shoebox.

45. The system of claim 44, wherein said second surface of said shoebox comprises a body of said shoebox.

46. The system of any one of claims 32-45, wherein said picker comprises a robotic arm.Nomagic Inc. 59332-706.601 Application Specification.docx -57- WSGR Docket No. 59332-705.601

Images