Storage and retrieval system with fault categorization

Abstract

An automated storage and retrieval system includes: a multi-level storage structure; a plurality of mobile robots configured to traverse the aisles and transport containers to and from storage locations in the multi-level storage structure and a control circuit communicatively coupled to the plurality of mobile robots. The control circuit being configured to: send tasks to the plurality of mobile robots to perform storage and retrieval tasks; receive mobile robot data from the plurality of mobile robots; store the mobile robot data and the tasks as system data in a system database; receive a fault report comprising an asset identifier and a fault timestamp associated with a fault event; retrieve system data associated with the fault event from the system database based on the asset identifier and the fault timestamp; and determine a fault type based on the system data associated with the fault event.

Description

STORAGE AND RETRIEVAL SYSTEM WITH FAULT CATEGORIZATIONCROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application is a non-provisional of and claims the benefit of United States provisional patent application number 63 / 682,264 filed on August 12, 2024, the disclosure of which is incorporated herein by reference in its entirety.BACKGROUND1. Field

[0002] The present disclosure generally relates to automated storage and retrieval systems, and more particularly to apparatuses, systems, and methods for fault categorization with mobile robots.2. Brief Description of Related Developments

[0003] Complex robotic systems such as an automated storage and retrieval (ASR) system can be susceptible to various fault conditions that prevent successful task execution. For example, issues may arise from faults in mechanical or electrical robotic components, robotic software execution, system software execution, environmental conditions, etc. Such faults can lead to downtime for individual mobile robots and / or the entire system. Conventionally, troubleshooting in an ASR system is mostly manual. When a fault report is made, a service ticket is created and placed in a queue for on-site or off-site personnel to manually determine the root cause of the fault and determine a remedial action. The lack of immediate actionable insight can prolong issue resolution and downtime. A delay in case resolution can also increase the risk of a lower-severity incident escalating to a higher severity incident.

[0004] Accordingly, the present disclosure addresses a number of those issues.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The foregoing aspects and other features of the present disclosure are explained in the following description, taken in connection with the accompanying drawings, wherein:

[0006] Fig. 1 is an illustration of a robotic storage and retrieval system in accordance with the present disclosure;

[0007] Fig. 2 is a block diagram of a fault categorization system in accordance with the present disclosure;

[0008] Fig. 3 is a flow diagram of a method fault categorization in accordance with the present disclosure;

[0009] Fig. 4 is a flow diagram of a process for fault handling in accordance with the present disclosure;

[0010] Fig. 5 is an illustration of a mobile robot in accordance with the present disclosure; and

[0011] Figs. 6A-C is an illustration of a decision tree in accordance with the present disclosure.

[0012] Elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions and / or relative positioning of some of the elements in the figures may be exaggerated relative to other elements to help improve understanding of various embodiments of the present invention. Also, common but well- understood elements that are useful or necessary in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various embodiments of the present invention. Certain actions and / or steps may be described or depicted in a particular orderof occurrence while those skilled in the art will understand that such specificity with respect to sequence is not actually required. The terms and expressions used herein have the ordinary technical meaning as is accorded to such terms and expressions by persons skilled in the technical field as set forth above except where different specific meanings have otherwise been set forth herein.DETAILED DESCRIPTION

[0013] The following detailed description is meant to assist the understanding of one skilled in the art, and is not intended in any way to unduly limit claims connected or related to the present disclosure.

[0014] The following detailed description references various figures, where like reference numbers refer to like components and features across various figures, whether specific figures are referenced, or not.

[0015] The word “each” as used herein refers to a single object (i.e., the object) in the case of a single object or each object in the case of multiple objects. The words “a,” “an,” and “the” as used herein are inclusive of “at least one” and “one or more” so as not to limit the object being referred to as being in its “singular” form.

[0016] Spatial terms such as “left,” “right,” “top,” “bottom,” “upper,” “lower,” “front,” “back,” “vertical,” and “horizontal” as may be used herein are by way of example and illustration only are not meant to limit the description and may be exchanged in position and orientation.

[0017] The terms “substantially” and “about” as may be used herein refer to a feature that may be varied within an acceptable manufacturing tolerance for a given application.

[0018] Generally speaking, the present disclosure provides for various aspects, systems, apparatuses, and methods for an automated storage and retrieval system with fault categorization. The description herein relates to an automated storage and retrieval system, including: a multi-level storage structure, a plurality of mobile robots, and a control circuit. The multi-level storage structure includes a plurality of racks separated by aisles, each rack including a set of horizontal container supports configured to store containers, which hold objects therein, at a plurality of storage levels within each aisle. The plurality of mobile robots are configured to traverse the aisles and transport containers to and from storage locations in the multi-level storage structure. The control circuit is communicatively coupled to the plurality of mobile robots. The control circuit is configured to: send tasks to the plurality of mobile robots to cause the plurality of mobile robots to perform storage and retrieval tasks within the multi-level storage structure; receive mobile robot data from the plurality of mobile robots traveling on the multi-level storage structure; store the mobile robot data and the tasks as system data in a system database; receive a fault report including an asset identifier and a fault timestamp, associated with a fault event; retrieve system data associated with the fault event from the system database based on the asset identifier and the fault timestamp; and determine a fault type based on the system data associated with the fault event.

[0019] It is understood that the present disclosure may be embodied in many different forms and should not be construed as being limited to the description set forth herein. Rather, the present disclosure is provided so as to be thorough and complete and will fully convey the invention to those skilled in the art. Indeed, the present disclosure is intended to cover alternatives, modifications and equivalents thereof, which are included within the scope and spirit of the invention as defined by the appended claims. Furthermore, in the following description, specific details are set forth in order to provide an understanding of the present disclosure.

[0020] In accordance with the present disclosure, systems and methods described herein provide an automatic categorization system for faults (i.e., a fault categorization system) in an ASR system. During the operation of the ASR system, a fault may be reported with an identification of a faulty asset (e.g., mobile robot, workstation) and a timestamp. The report may include any suitable fault location identifier including, but not limited to a store identifier, a site identifier, and a storage structure three-dimensional (3D) coordinate. The fault categorization system retrieves relevant data from a system database based on one or more of the asset, fault timestamp, and location. Thedata may be collected by a sensor and / or a controller of the faulty asset, other assets in the system, and / or include other system hardware or software metrics. The data may relate to the state of one or more assets, the state of one or more components of the faulty asset, the state of the storage structure, and / or software executed by the control system around the time of the fault. A fault categorization model, such as a decision tree, may be employed by the fault categorization system to determine a fault type based on the retrieved data. The fault can be categorized as a software fault or a hardware fault and identified based on a specific failure type (e.g., debris on sensor, sensor failure, charge toe failure, vertical collusion, break failure, etc.). The fault can be triaged for handling based on the fault type. A remedial action may be recommended and / or automatically executed based on the fault type. The fault categorization model may be updated and refined based on feedback on the fault identification such as from user verification and / or the success of associated remedial action. The fault categorization model may be updated / trained with machine learning (ML).

[0021] The fault categorization system may be configured to triage fault events, identify the cause, and suggest remedial tasks. In instances where the fault categorization system cannot determine the root cause and devise mitigation steps, the fault categorization system may append it findings (e.g., eliminated causes, list of likely causes) to a ticket for a human operator. This process significantly decreases the number of cases that a human operator manually triages. The fault categorization system may provide immediate insight and / or trigger immediate remedial action.

[0022] Referring now to Fig. 1, an example of a type of structure in a storage and retrieval system that may be verified by the systems and methods described herein is shown. The systems and methods described herein may verify differently configured storage structures and / or other types of structures. Fig. 1 shows a partial view of an exemplary order fulfillment facility 100 having a storage structure 102, where the storage structure includes a number of bays 104 of storage locations 106.

[0023] The storage structure 102 may be formed of a number of storage modules 110 that define the bays 104. The bays 104, defined by the storage modules 110, each include a y-z array of storage locations 106 in horizontal rows and level changing towers along the rows. The level changing towers may be vertical towers. Mobile robots 148 may travel between storage levels in the z- direction within the level changing towers. The storage modules 110 may form pairs of bays 104 that are arranged to face each other, separated by aisles 108. An aisle 108 may have a width such that a mobile robot 148 traveling within an aisle 108 may transfer containers to the bays 104 on either side of the aisle 108. The mobile robots 148 may also carry totes containing goods to and from portals for storage and retrieval of items from the storage structure 102.

[0024] The order fulfillment facility 100 may include decks 112 spaced apart at different horizontal levels of the storage structure 102. The decks 112 may extend between the aisles so that mobile robots 148 can maneuver in the x-y plane of each deck to travel between different aisles. At least one of the decks 112 may also extend into the respective aisles to allow technicians to walk into an aisle 108 to service components within the aisle.

[0025] Fig. 1 shows examples of integrated turning decks 115. The turning decks 115 may be positioned at different vertical locations between the storage modules 110. The turning decks 115 may have a turning deck input / output 115-I / O that may align with the aisle 108 or another turning deck input / output 115-I / O. The turning deck 115 may enable the mobile robot 148 to efficiently change a direction of travel through the storage structure 102.

[0026] The order fulfillment facility 100 includes a number of mobile robots 148 for transferring totes or other product or order containers to and from customer access portals and storage locations 106 in the bays 104. The mobile robots 148 may be self-guided and / or rail-guided so as to move horizontally and vertically within aisles 108 to transfer totes or other product containers between the mobile robots 148 and storage locations 106.

[0027] Further details of the storage structure 100 and mobile robots 148 that may be employed with the present disclosure are described, for example, in the following U.S. patents and patent applications: U.S. Pat. No. 9,139,363, entitled “AUTOMATED SYSTEM FOR TRANSPORTING PAYLOADS,” issued Sep. 22, 2015; U.S. Pat. No. 10,435,241, entitled, “STORAGE AND RETRIEVAL SYSTEM,” issued Oct. 8, 2019; U.S. Pat. No. 11,142,398, entitled, “ORDER FULFILLMENT SYSTEM,” issued Oct. 12, 2021; U.S. Pat. No. 10,984,375, entitled “PICKING WORKSTATION WITH MOBILE ROBOTS & MACHINE VISION VERIFICATION OF EACH TRANSFERS PERFORMED BY HUMAN OPERATORS,” issued Apr. 20, 2021; U.S. Pat. No. 10,952,533, entitled “MODULAR STRUCTURE FOR AN AUTOMATED STORAGE AND RETRIEVAL SYSTEM,” issued Mar. 23, 2021; U.S. Pat. No. 11,267,651, entitled, “SYSTEM HAVING WORKSTATION WITH TOTE RETENTION AND RELEASE MECHANISM,” issued Mar. 8, 2022; and U.S. Patent Application No. 63 / 127,762, entitled, “MICRO-FULFILLMENT CENTER WITH AUTOMATED DISPENSE AND RETURN USING MOBILE ROBOTS AND METHOD OF OPERATING SAME,” filed on Dec. 18, 2020. Each of these patents and applications are incorporated by reference herein in their entirety.

[0028] Referring also to Fig. 2, an exemplary block diagram of a system for fault categorization (i.e., fault categorization system) 200 in a storage and retrieval system 100 is shown. The fault categorization system 200 includes one or more mobile robots 210, a central computer system 220, and one or more user interface devices 240.

[0029] The central computer system 220 includes a control circuit 221, a memory 223, and a network interface device 227. The central computer system 220 may include one or more of a server, a central computing system, a cloud-based compute engine, a desktop computer system, a personal computer, a portable device, and the like. The control circuit 221 may include a processor, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), and the like and may be configured to execute non- transitory computer-readable instructions stored on a computer-readable storage memory 223. Thecomputer-readable storage memory 223 may include volatile and / or non-volatile memory and have stored upon it, a set of computer-readable instructions which, when executed by the control circuit 221, causes the central computer system 220 to send instructions for execution by the one or more mobile robots 210 and automatically perform fault categorization on received fault reports.

[0030] The computer-executable instructions may cause the control circuit 221 of the central computer system 220 and / or the one or more mobile robots 210 to perform one or more functions described herein, such as steps described with reference to Figs. 3 and 4. The memory 223 may store a fault categorization model FCM such as a decision tree and / or a machine learning model (described herein). For example, the memory 223 may store the decision tree shown in Figs. 6A- C herein. The fault categorization model FCM may take fault reports and system data as input, and output a fault categorization. Further examples and details of an exemplary fault categorization model FCM are described herein with reference to Figs. 3, 4, and 6A-C.

[0031] The network interface device 227 may include one or more of a data port, a wired network adapter, a wireless network adapter, and the like. The central computer system 220 may communicate with a plurality of mobile robots 210 and user interface devices 240 over one or more networks such as a local network, a private network, a cloud computing network, or the Internet. The central computer system 220 may communicate with other systems and devices such as an order database, an inventory database, operator user interface devices, customer user interface devices, etc.

[0032] The user interface device 240 may comprise a processor-based device configured to provide information to a user and receive user input. For example, the user interface device 240 may include one or more of a display screen, a touch input device, a keyboard, a motion sensor, a speaker, a microphone, etc. The user interface device 240 may be one or more of a desktop computer system, a personal computer, a portable device, and the like. The user interface device 240 may be configured to provide a task user interface based on data provided by the central computer system 220. For example, the central computer system 220 may comprise a plurality ofuser interface devices 240, each associated with a task team (e.g., robotic hardware team, software team, on-site servicing team, etc.). The central computer system 220 may selectively communicate task assignments to one or more of a plurality of user interface devices 240 based on the categorization of a fault event. The task user interface may display fault categorization and / or task instructions associated with the identified fault. The task user interface further allows a user to provide feedback on the fault categorization and / or task suggestion / assignment to update / retrain the ML model.

[0033] The mobile robot 210 is a motorized unit configured to travel on a structure, such as of the order fulfillment facility 100. The mobile robot 210 may be configured to autonomously travel on a multi-level storage structure to retrieve and / or store items and / or totes. The mobile robot 210 may be the mobile robot 148 described with reference to Fig. 1 and / or Fig. 5. The mobile robot 210 includes a control circuit 211, a memory 213, a network interface 217, a sensor system 215, and a drive system 219.

[0034] The control circuit 211 may include a processor, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), and the like and may be configured to execute computer-readable instructions stored on a computer-readable storage memory 213. The computer-readable storage memory 213 may include volatile and / or non-volatile memory and have stored upon it, a set of non-transitory computer- readable instructions which, when executed by the control circuit 211, causes the mobile robot 210 to autonomously or semi-autonomously travel on a structure (such as of the order fulfillment facility 100) via the drive system 219 and gather data via the sensor system 215.

[0035] The computer-executable instructions may cause the control circuit 211 to continuously or periodically provide mobile robot data to the central computer system 220 via the network interface 217 during normal operation. The computer-executable instructions may cause the control circuit 211 of the mobile robot 210 to send an error signal, including one or more fault codes, to the central computer system 220 when a fault occurs. Generally, mobile robot data mayinclude data collected by one or more sensors 215S of the sensor system 215 on a mobile robot 210 and / or fault codes generated in response to an unsuccessful execution of tasks assigned to the mobile robot 210.

[0036] The sensor system 215 may include one or more sensors 215S for monitoring states of components of the mobile robot 210 and / or the environment in which the mobile robot 210 travels. For example, the sensor 215 S may provide data for the mobile robot 210 to perform autonomous navigation and / or storage and retrieval functions. The sensors 215S comprise any suitable sensor(s) including, but not limited to, one or more of a speed sensor, a charge sensor, drive motor sensor, wheel position sensor, container sensor, storage structure feature sensor, an optical sensor, a radio frequency identification (RFID) sensor, a vibration sensor, a sound sensor, a tachometer, an odometer, a pressure sensor, a voltmeter, an accelerometer, a gyroscope, a magnetometer, and / or an inertial measurement unit (IMU). The sensor system 215 may detect the positions of one or more movable components of the mobile robot 210, force exerted by one or more actuators of the mobile robot 210, power supplied to one or more components of the mobile robot 210, and / or temperature, stress, or pressure experienced by one or more components of the mobile robot 210. The sensor system 215 may be configured to determine the location of the mobile robot 210 via detecting features on the structure (such as of the order fulfillment facility 100) such as optical markers, RFID markers, structural features, track connections, track gaps, fastening elements (e.g., bolts, clamps), support elements (e.g., poles, braces), and / or electrical elements (charging rails or pads). Sensor data may be provided directly to the central computer system 220 and / or be used to determine fault codes and / or location coordinates at the mobile robot 210. Fault codes may be generated based on failed executions of commands to one or more components of the mobile robot 210.

[0037] The drive system 219 may include a system for providing and controlling the movement of the mobile robot 210 on the structure (such as of the order fulfillment facility 100). The drive system 219 may include motor, wheels, gears, etc. The mobile robot 210 may include otherelectrical or mechanical components such as transport mechanism for loading and unloading totes to and from the mobile robot 210.

[0038] An example of mobile robot 210 is shown in Fig. 5. The mobile robot 210 includes an item / tote carrying portion 610, front drive wheels 630, vertical climbing gear 635, power storage 620, rare trailer wheelers 648, and charging contacts 640. In some embodiments, sensors of the sensor system 215 may be coupled to and / or monitor any portion of the mobile robot 210. For example, sensors may be placed to detect the positions of the vertical climbing gear 635 and the charging contacts 640. In some embodiments, sensors may determine the charge state of the power storage 620.

[0039] Next referring to Fig. 3, a method for fault categorization in an ARS system is shown. In some embodiments, one or more steps of Fig. 3 may be executed by a processor-based device executing computer-readable instructions stored on a memory device. In some embodiments, one or more steps of Fig. 3 may be performed by the central computer system 200 and / or the mobile robot 210 described with reference to Fig. 2 herein or other similar devices. In some embodiments, the multiple iterations of the process shown in Fig. 3 may simultaneously occur within an ASR system based on communications with a plurality of assets in the ASR system.

[0040] In step 310, a task is sent to mobile robots in an ASR system for execution. In some embodiments, the task may be defined by a central computer system. In some embodiments, the task causes the plurality of mobile robots to perform storage and retrieval tasks within the multilevel storage structure. In some embodiments, the task may be a storage and / or retrieval task, indicating one or more destination locations, tasks to be performed at one or more locations, and / or routes within the ARS storage structure. In some embodiments, the task may specify a plurality of destination locations in the structure in sequence. For example, a task instruction may specify a storage location for retrieving a tote and a workstation for delivery of the tote. In some embodiments, the mobile robot may be one of a plurality of mobile robots configured to perform storage and retrieval tasks within the same structure. In some embodiments, tasks assigned to eachof the mobile robots may be stored as system data in a system database 390. In some embodiments, the ASR system may include one or more sensors on the storage structure, not coupled to a mobile robot. Sensor data from the ASR structure may also be stored as system data in the system database 390.

[0041] In step 320, mobile robot data is received from the mobile robot in the ASR system. Generally, mobile robot data may refer to data generated by the mobile robot during a task execution. In some embodiments, the mobile robot data may include data normally reported during the operation of a mobile robot and / or may include a fault code generated when a fault occurs with the mobile robot. In some embodiments, mobile robot data may be generated based on software execution events, sensor signals, mechanical component signals, or electrical component signals on the mobile robot. In some embodiments, the mobile robot data further includes a timestamp and location data. The location data generally corresponds to the location of the mobile robot when data is recorded and / or when a fault event occurs. In some embodiments, a mobile robot’s location may be determined based on detecting features such as markers on the storage structure. In some embodiments, the mobile robot's location may be determined based on the distance traveled by a drive system of the mobile robot from a reference point (e.g., feature or marker) in the structure. In some embodiments, the mobile robot’s location is indicated as a 3D coordinate value indicating a location in a multi-level, multi-row storage structure. The mobile robot’s location may be used to determine the fault location identifier in a fault report.

[0042] In step 330, a fault report is received at a central computer system. In some embodiments, the fault report may be generated by a fault detection system and / or software module executed on the central computer system or on a different device based on signals from an asset of the ARS system. For example, a fault report may be generated when an asset transmits an error code or fails to complete an assigned task within a set period of time. In some embodiments, the fault report may be entered by a human operator. In some embodiments, the fault report may include an asset identifier and a fault timestamp associated with a fault event. In some embodiments, the fault report may further include a fault location identifier associated with the fault. The fault location identifiermay identify a location within the multi-level storage structure, such as the location of the mobile robot at the time of the fault event or the location of a stationary workstation. In some embodiments, the fault location identifier may be retrieved as part of system data based on the asset identifier. For example, a robot’s fault location may be determined based on the reported / tracked location of the mobile robot in the multi-level storage structure. In some embodiments, the fault location identifier may identify a store site location, the ASR system location, the fulfillment site location, etc.

[0043] In step 340, the system retrieves system data associated with the fault event identified in step 330 from the system database 390. In some embodiments, system data is retrieved based on the asset identifier and the fault timestamp. In some embodiments, the system data is further retrieved based on a fault location identifier. In some embodiments, the system data comprises data collected by sensors in the multi-level storage structure, mobile robot data received from a plurality of mobile robots in the multi-level storage structure, and / or data associated with software executions of the asset and / or the central computer system. For example, the system data may indicate the states of various components of a mobile robot at the time of the fault event, the states of various storage structure components near the location of the fault event, the location and state of one or more assets (e.g., mobile robots) near the faulty asset at the time and location of the fault event.

[0044] In step 350, the system determines a fault type based on the retrieved system data. In some embodiments, the fault type is identified based on a decision tree. In some embodiments, the fault type is determined based on a machine learning (ML) algorithm trained ML model 355 such as a fault categorization decision tree. In some embodiments, the ML model is trained and refined using a machine learning algorithm with a training data set, including prior system data associated with the multi-level storage or a similar storage structure. For example, the training data set can include a set of fault reports, associated system data, and operator-entered categorizations. In some embodiments, the machine learning algorithm may comprise a supervised, unsupervised, or semisupervised algorithm. In some embodiments, the machine learning algorithm may include artificialneural networks deep learning, decision tree learning, support vector machines, regression analysis, Bayesian networks, etc. In some embodiments, the fault type may identify the fault event as being associated with a hardware fault or a software fault. In some embodiments, the fault type may identify a hardware or software component associated with the fault event. For example, fault type may be one or more of a charging assembly failure, drive motor failure, wheel truck failure, spline shaft failure, container sensor failure, storage structure position feature missing, mag line missing, incorrect speed configuration, and software bug. In some embodiments, the system may determine one or more of a root cause summary, primary failure classification, secondary failure classification, task recommendation, and / or task assignment using the ML model 355 and according to the identified fault type in step 350. Further examples of system data and fault types are provided in Figs. 6A-C.

[0045] In step 360, the system assigns or triages the fault report received in step 330 based on the fault type identified in step 350. In some embodiments, the system triages the fault report into two or more task assignment categories based on the fault type. In some embodiments, the task assignment categories may each be associated with different work groups for taking remedial action. For example, faults may be handled by an on-site operator, a software team, or an off-site repair / service team, depending on the fault type. In some embodiments, the system may further be configured to output a remedial action based on the fault type. In some embodiments, the remedial action may be stored in a lookup table corresponding to each fault type. In some embodiments, the system is configured to automatically generate a task for remedial action in a workflow management user interface based on the fault type. In some embodiments, the system is further configured to assign the task for remedial action to one or more persons based on the fault type vis the workflow management user interface. In some embodiments, the system is configured to output for display on a user interface, a remedial action recommendation based on the fault type. In some embodiments, the system is configured to automatically determine a remedial action based on the fault type and cause an automatic execution of the remedial action by a mobile robot and / or a service system. For example, the system may include an automated robot retrieval robot or devicethat can be deployed to remove a faulty mobile robot from the storage structure or include a robotic arm configured to service or replace one or more parts of a faulty mobile robot (e g., battery, misaligned gear) in response to a fault report.

[0046] In step 365, the system receives feedback on the fault type identification. In some embodiments, the feedback may be an operator confirmation or correction of the fault type identification entered via the user interface. In some embodiments, the feedback may be an outcome (e.g., successful or unsuccessful) of performing a recommended remedial action. In step 370, the feedback is used to retrain the ML model 355.

[0047] With the process shown in Fig. 3, fault events within an ASR system may be categorized automatically and triaged to one or more teams or systems for redressing to reduce the downtown of faulty assets in the ASR.

[0048] Next referring to Fig. 4, a process for fault handling in an ASR system is shown. In some embodiments, one or more steps of Fig. 4 may be executed by a processor-based device executing computer-readable instructions stored on a memory device. In some embodiments, one or more steps of Fig. 4 may be performed by the central computer system 200 and / or the mobile robot 210 described with reference to Fig. 2 herein or other similar devices. In some embodiments, multiple iterations of the process shown in Fig. 4 may simultaneously occur within an ASR system based on communications with a plurality of assets in the ASR system.

[0049] In step 410, an asset is in regular operation. The regular operation may correspond to tasks and instructions communicated from a central computer system. In some embodiments, the system may record data emitted from assets and data from the central computer system- during step 410. In step 412, an asset encounters an event that prevents the continuing execution of its regular operation. In step 414, a fault report is created. The fault report may be automatically generated when a fault event is detected based on signals from sensors in the system or be manually inputted. In step 416, the system determines whether automated categorization / recommendations areavailable based on the data associated with the fault report. In some embodiments, the fault type determination may be performed based on the method described with reference to Fig. 3. In some embodiments, a fault type is identified when a confidence level associated with the ML model identification exceeds a predetermined threshold. If a fault type cannot be identified, in step 420, the system appends its automated findings (e.g., excluded fault type, selected relevant system data) to the fault report. In step 422, the appended fault report is sent to a human operator to triage and identify the root cause.

[0050] If fault type can be automatically identified in step 416, in step 430, the system determines whether the fault type is associated with a hardware failure. In some embodiments, each identifiable fault type may be associated with robot hardware failure, storage structure hardware issue, robot software issue, and / or central computer system software issue. If the fault event is not associated with a hardware failure, the fault report may be forwarded to a software development team in step 440 to apply a fix in step 436. If the fault event is associated with a hardware failure, in step 432, the fault report is assigned to a hardware team. In step 434, the fault report may be used to create a work order. In some embodiments, the work order may be generated based on system determined / recommended remedial task associated with the fault type. In step 436, fixed is applied according to the work order. In some embodiments, the fix may further mitigate the reoccurrence of fault events.

[0051] Next referring to Figs. 6A-C, a decision tree for fault type identification is shown. In some embodiments, the decision may be the ML model 355, and may be used in step 340 and trained via steps 360 and 370 as described with reference to Fig. 3. It should be understood however, that the decision tree in Figs. 6A-C is provided as an example only. A decision tree can be variously configured based on the ASR system and mobile robot configurations. For example, the parameters described in Figs. 6A-C are specific to one embodiment of a mobile robot. Other ASR systems or asset types may generate other parameters used for fault type categorization. For an ML decision tree, the model may further be constantly updated and modified based on a feedback loop. In some embodiments, each of the identified fault types in Figs. 6A-C may be associated with a particulartriage category, such as on-site, software, or off-site servicing / repair. Depending on the identified fault type, tasks may be automatically generated and / or assigned via user interfaces associated with each team / site. In some embodiments, one or more terminal nodes may be associated with an unidentifiable fault, and the fault report may be forwarded for manual identification. Several examples of fault type identification in Figs. 6A-C are described in further detail below.

[0052] Example parameters for identifying charging assembly failures 601 are shown in Fig. 6B. The mobile robot may have a charge assembly with a charge toe, which engages with a charge rail on the storage structure of an ASR system to transfer power to the robot. When the charge toe fails, the mobile robot would eventually lose communication with the system which is identifiable by “isCommTimeout.” The mobile robot may also have several failed charging attempts (“isBotFailedToCharr”). For example, the robot may be routed to a charging location successfully, engages the charge toe successfully (as identified by robot telemetry indicated a successfully extended toe), r robotbut does not gain charge. Therefore, when isCommTimeout=l, isHighvoltage=0, isBotdead=l, and isBotfailedToChar=l, charging assembly failure 601 is identified.

[0053] Example parameters for identifying drive motor failure are shown in Fig. 6A. The drive motor refers to a mobile robot component that enables the robot to perform horizontal and vertical movement. A common end state for a robot with drive motor issues is a vertical move failure, often accompanied by a related fault code (e.g., “05_0E_00”). In this particular case, the vertical lift of the robot is checked for a 200mm threshold (“is Actual X<200”). During motion, if the drive motor current saturates (e.g., above 60 amps for one of the motors) and the encoder position indicates a lag in one wheel despite the high current, the parameters may indicate a malfunctioning drive motor 602.

[0054] Example parameters for identifying a stuck wheel truck are shown in Fig. 6A. In some embodiments, a mobile robot in an ASR system includes a wheel truck configured to retract the wheel while the robot is traveling vertically and extend the wheels when the robot is traveling ona horizontal rail. When the wheel truck fails to retract the wheels, the robot can have a vertical move failure (“04_0C_00). In such case, the lift (z) of the robot could be between 150-250mm and the robot would show consecutive failures while trying to retract the wheel, leading to a warning / error code (“isMulitpleAattempts of 03_04_00 warning”). This parameter could indicate a wheel truck failure 603, which could correspond to a flag on the wheel not being tripped when the wheels are retracted or an actual problem with the trucks.

[0055] Example parameters for identifying pinion and spline shaft failures are shown in Fig. 6C. A mobile robot according to some embodiments includes pinions that extend and engage in the climbing channel in the structure facilitating the vertical motion of the robot. When a pinion fails, the robot will end in a retry disabled state. Two hardware failures can be associated with this state. If, based on the extension value, the pinions are extending / retracting beyond the given target value (e g., 703 mm) but the flags are not tripped, the pinion flags may not be set correctly 604. If, during extension, the pinions are unable to achieve the target position while the current is saturating over 75 amps, the pinions may be obstructed from extending and the spline shaft needs to be inspected 605. The algorithm may further verify whether there was slip involved in prior robot moves and the robot has aligned correctly with the feature before trying to extend the pinions.

[0056] Example parameters for identifying tote sensor issues are provided in Fig. 6A. A mobile robot may be equipped with tote sensors that facilitate the tote handling process. The robot may end in an all sensors triggered state. Whether the error occurred during a tote moving operation or other operations is checked. Flicker of the sensor may be validated for the flicker start time. If the flicker starts after performing a tote transition move, the sensor may have some debris 606. Otherwise, one of the three sensors may be misaligned where flicker was observed 607.

[0057] Parameters for identifying missing structure fine position features are shown in Fig. 6C. The fine position features in the structure help the robot identify its location in the system. When a structure feature is missing, the robot may not be able to identify its location correctly and would misalign. This typically leads to a failure in pinion extension issues while unloading / loading a tote.The last move of the robot is checked for a slip warning 0E_02_01. Accumulated slip from several previous moves is also calculated to make sure the robot has not slipped too much. If the checks pass, based on the move given to the robot by the central computer system, where the robot missed a fine position flag can be identified 608. Cases of slip in frozen / chilled storage areas of the ASR may be attributed to humidity in the chilled storage areas or frost buildup in frozen storage areas.

[0058] Example parameters for identifying missing magnetic lines on decks are shown in Fig. 6A. Magnetic lines are lines on a deck such as a turn deck that are used by the robots for navigation. When a magnetic line is missing, the robot might go down with a missed line error. Whether the robot was commanded to move from an aisle to a deck when the fault occurred is checked. The amount of slip involved is also determined. If the robot did not slip and still sees the same error, a part of the line may be missing (e.g., a break in the line) or lacking sufficient strength to be detected by the bot 609. The process may further verify if a robot had previously failed at the same location (e.g., in past 2 days, 7 days) to determine whether the fault is with the line or the bot sensor.

[0059] The system may also identify software issues. For example, the system may identify speed configuration issues of a mobile robot. Speed configurations are used to limit the robot velocity in certain areas of the system to reduce the risk of slip and allow for more consistent movement. One indicator of an incorrect speed configuration is the robot ending in a retry disabled state (failed the retry attempts for an action, in this case, pinion engagement attempts). The algorithm checks whether the robot failed when attempting to move up the ramp in a dynamic workstation. There may be a set sequence of start and end positions to check against the algorithm, which also checks whether the robot was posting warnings that the pinion engagement attempts were failing and the speed dictated for the move request. If the speed is above the prescribed limits of the system, the speed configuration value may be incorrectly set. Options for remedial action may include generating an alert that the speed configuration value is incorrect and needs to be updated or automatically correcting the speed config to the approved values. The root cause and containment steps of this action may be added to the failure report.

[0060] In another example, a bug in the code that causes a robot to run out of charge waiting on a space held by another robot may be identified. The algorithm may identify that robot A went down with a Low Voltage Comm Timeout while not at a charging location. If robot A is trying to move at the time of the fault, the presence of robot B at the location is checked. If robot B cannot be verified, the algorithm checks which robot C last held that location and if robot C has performed another move since. If robot C has not moved since, the algorithm checks the time of its last move. Using that timing if there are other messages within 1 minute for a passing traverse door response, a plan segment failed, and move failed for the robot C, a bug ticket may be generated.

[0061] The techniques described herein relate to an automated storage and retrieval system, including: a multi-level storage structure including a plurality of racks separated by aisles, each rack including a set of horizontal container supports configured to store containers, which hold objects therein, at a plurality of storage levels within each aisle; a plurality of mobile robots configured to traverse the aisles and transport containers to and from storage locations in the multilevel storage structure: and a control circuit communicatively coupled to the plurality of mobile robots, the control circuit being configured to: send tasks to the plurality of mobile robots to cause the plurality of mobile robots to perform storage and retrieval tasks within the multi-level storage structure; receive mobile robot data from the plurality of mobile robots traveling on the multi-level storage structure; store the mobile robot data and the tasks as system data in a system database; receive a fault report including an asset identifier and a fault timestamp associated with a fault event; retrieve system data associated with the fault event from the system database based on the asset identifier and the fault timestamp; determine a fault type based on the system data associated with the fault event.

[0062] The techniques described herein relate to a system for identifying and triaging faults in an automated storage and retrieval system, including a control circuit communicatively coupled to a plurality of mobile robots in traveling in a multi-level storage structure including a plurality of racks separated by aisles, each rack including a set of horizontal container supports configured to store containers, which hold objects therein, at a plurality of storage levels within each aisle, thecontrol circuit being configured to: send tasks to the plurality of mobile robots to cause the plurality of mobile robots to perform storage and retrieval tasks within the multi-level storage structure; receive mobile robot data from the plurality of mobile robots traveling on the multi-level storage structure; store the mobile robot data and the tasks as system data in a system database; receive a fault report including an asset identifier and a fault timestamp associated with a fault event; retrieve system data associated with the fault event from the system database based on the asset identifier and the fault timestamp; determine a fault type based on the system data associated with the fault event.

[0063] The techniques described herein relate to a computer-implemented method for managing an automated storage and retrieval system, the method including: sending, from a control circuit, tasks to a plurality of mobile robots to perform storage and retrieval tasks within a multi-level storage structure; receiving, at the control circuit, mobile robot data from the plurality of mobile robots traveling on the multi-level storage structure; storing the mobile robot data and the tasks as system data in a system database; receiving, at the control circuit, a fault report including an asset identifier and a fault timestamp associated with a fault event; retrieving system data associated with the fault event from the system database based on the asset identifier, the fault timestamp, and the location identifier; determining, with the control circuit, a fault type based on the system data associated with the fault event.

[0064] It should be understood that the foregoing description is only illustrative of the present disclosure. Various alternatives and modifications can be devised by those skilled in the art without departing from the present disclosure. Accordingly, the present disclosure is intended to embrace all such alternatives, modifications and variances that fall within the scope of any claims appended hereto. Further, the mere fact that different features are recited in mutually different dependent or independent claims does not indicate that a combination of these features cannot be advantageously used, such a combination remaining within the scope of the present disclosure.

[0065] What is claimed is:

Claims

CLAIMS1. An automated storage and retrieval system, comprising: a multi-level storage structure comprising a plurality of racks separated by aisles, each rack comprising a set of horizontal container supports configured to store containers, which hold objects therein, at a plurality of storage levels within each aisle; a plurality of mobile robots configured to traverse the aisles and transport containers to and from storage locations in the multi-level storage structure; and a control circuit communicatively coupled to the plurality of mobile robots, the control circuit being configured to: send tasks to the plurality of mobile robots to cause the plurality of mobile robots to perform storage and retrieval tasks within the multi-level storage structure; receive mobile robot data from the plurality of mobile robots traveling on the multi-level storage structure; store the mobile robot data and the tasks as system data in a system database; receive a fault report comprising an asset identifier and a fault timestamp associated with a fault event; retrieve system data associated with the fault event from the system database based on the asset identifier and the fault timestamp; and determine a fault type based on the system data associated with the fault event.

2. The system of claim 1, wherein the fault type is identified based on a decision tree.

3. The system of claim 2, wherein the decision tree is trained using a machine learning algorithm, wherein a training data set of the machine learning algorithm comprises prior system data associated with the multi-level storage or a similar storage structure.

4. The system of claim 2, wherein the control circuit is further configured to: receive feedback on the fault type; and retrain the decision tree based on the feedback.

5. The system of claim 4, wherein the feedback comprises an outcome of performing a recommended remedial action.

6. The system of claim 1, wherein the control circuit is further configured to: triage the fault report into two or more task assignment categories based on the fault type.

7. The system of claim 1, wherein determining the fault type comprises identifying whether the fault event corresponds to a hardware fault or a software fault.

8. The system of claim 1, wherein the control circuit is further configured to: automatically generate a task for remedial action in a workflow management user interface based on the fault type.

9. The system of claim 8, wherein the control circuit is further configured to assign the task for remedial action to one or more persons based on the fault type.

10. The system of claim 1 , wherein the control circuit is further configured to output for display on a user interface, a remedial action recommendation based on the fault type.

11. The system of claim 1, wherein the control circuit is further configured to determine a remedial action based on the fault type and cause an automatic execution of the remedial action by a mobile robot and / or a service system.

12. The system of claim 1, wherein the mobile robot data comprises data determined based on data collected by sensors on a mobile robot.

13. The system of claim 12, wherein the sensors comprise one or more of a speed sensor, a charge sensor, drive motor sensor, wheel position sensor, container sensor, or storage structure feature sensor.

14. The system of claim 1, wherein the mobile robot data comprises one or more fault codes generated by a mobile robot.

15. The system of claim 1, wherein the system data comprises data collected by sensors in the multi-level storage structure.

16. The system of claim 1, wherein the system data associated with the fault event comprises mobile robot data received from a plurality of mobile robots in the multi-level storage structure.

17. The system of claim 1, wherein the system data associated with the fault event comprises data associated with software executions of the mobile robot and / or the control circuit.

18. The system of claim 1, wherein the fault type comprises one or more of a charging assembly failure, drive motor failure, wheel truck failure, spline shaft failure, container sensor failure, storage structure position feature missing, mag line missing, incorrect speed configuration, and software bug.

19. The system of claim 1, wherein the fault report further comprises a location identifier and the system data is retrieved further based on the location identifier.

20. The system of claim 19, wherein the location identifier identifies a location within the multi-level storage structure.

21. A system for identifying and triaging faults in an automated storage and retrieval system, comprising a control circuit communicatively coupled to a plurality of mobile robots in traveling in a multi-level storage structure comprising a plurality of racks separated by aisles, each rack comprising a set of horizontal container supports configured to store containers, which hold objects therein, at a plurality of storage levels within each aisle, the control circuit being configured to: send tasks to the plurality of mobile robots to cause the plurality of mobile robots to perform storage and retrieval tasks within the multi-level storage structure; receive mobile robot data from the plurality of mobile robots traveling on the multi-level storage structure; store the mobile robot data and the tasks as system data in a system database; receive a fault report comprising an asset identifier and a fault timestamp associated with a fault event; retrieve system data associated with the fault event from the system database based on the asset identifier and the fault timestamp; and determine a fault type based on the system data associated with the fault event.

22. A computer-implemented method for managing an automated storage and retrieval system, the method comprising: sending, from a control circuit, tasks to a plurality of mobile robots to perform storage and retrieval tasks within a multi-level storage structure;receiving, at the control circuit, mobile robot data from the plurality of mobile robots traveling on the multi-level storage structure; storing the mobile robot data and the tasks as system data in a system database; receiving, at the control circuit, a fault report comprising an asset identifier and a fault timestamp associated with a fault event; retrieving system data associated with the fault event from the system database based on the asset identifier and the fault timestamp; and determining, with the control circuit, a fault type based on the system data associated with the fault event.

Images