Autonomous material handling system

Abstract

A system for autonomous material handling may include a material handler (402) having a movable support (424) and rigging (600) operatively coupled to the movable support configured for coupling with one or more target objects (300). The system may include one or more sensors (413) operatively coupled to the material handler, and an operator system (200) for receiving a target object based on an object identifier (301) of the target object to be retrieved by the material handler, and moving the movable support of the material handler based on the one or more sensors to the target object based at least on the one or more sensors.

Description

AUTONOMOUS MATERIAL HANDLINGFIELD

[0001] Implementations of the disclosure relate to a system for autonomous material handling, and more particularly a system for handling materials within an operating area.BACKGROUND

[0002] Material handling is often hindered by dynamic and unpredictable factors, such as the constant movement of objects and personnel navigating work areas. This variability forces operators of material handlers to frequently evaluate surroundings to avoid collisions, leading to workflow disruptions, pauses, and reduced efficiency. Additionally, there is minimal oversight to ensure that only qualified personnel operate these apparatuses, which increases the risk of improper rigging configurations that may not suit an object’s weight or dimensions. These issues can result in accidents, damage, or injury, compounding inefficiencies and safety concerns in material handling. As such, there is a need for autonomous material handling.BRIEF SUMMARY OF THE DISCLOSURE

[0003] The following presents a simplified summary of one or more implementations of the disclosure in order to provide a basic understanding of such implementations. This summary' is not an extensive overview of all implementations, and it is intended to neither identify key or critical elements of all implementations, nor delineate the scope of any or all implementations. The summary's sole purpose is to present some concepts of one or more implementations in a simplified form as a prelude to the more detailed description that is presented later.

[0004] To address challenges in material handling safety and efficiency, many facilities implement basic safety protocols, such as operator training programs, visual inspections, and manual checklists to verify operator qualifications and ensure proper rigging practices. Some facilities utilize signage and designated walkways to guide personnel and reduce the risk of collisions. In certain cases, supervisors oversee material handling to manually authorize usage and enforce compliance with safety standards. Additionally, equipment may be equipped with basic safety features, such as alarms and emergency stop mechanisms, to mitigate risks during material handling.Page 1 of 40018300-902175

[0005] While these solutions provide some level of risk mitigation, they may be inadequate due to their reliance on manual processes and human oversight. Training programs and checklists may be inconsistently applied or overlooked, and supervisors may not always be available to monitor material handling in real-time. Signage and designated walkways may be ineffective in highly dynamic operating areas where objects and personnel frequently change positions. Furthermore, basic safety features on equipment may not prevent unqualified operators from accessing and using apparatuses. These limitations may contribute to ongoing inefficiencies, safety hazards, and a lack of comprehensive control over material handling.

[0006] The present disclosure provides for systems and methods of autonomous material handling.

[0007] In one aspect, a system for autonomous material handling is presented, the system may include a material handler having a movable support and rigging operatively coupled to the movable support configured for coupling with one or more target objects, and one or more sensors operatively coupled to the material handler, an operator system operatively coupled to the material handler, wherein the operator system includes one or more memories having computer readable code stored thereon, and one or more processors operatively coupled to the one or more memories, wherein when executed the computer readable code may be configured to cause the one or more processors to receive a target object based on an object identifier of the target object to be retrieved by the material handler, and move the movable support of the material handler based on the one or more sensors to the target object based at least on the one or more sensors.

[0008] In some implementations, when executed the computer readable code may be further configured to cause the one or more processors to receive a user identifier of a user, determine authorization for the user for the material handler based on the user identifier, and authorize operation of the material handler when the user is authorized.

[0009] In some implementations, the user identifier for the user may be received from a user computer system sensing a user token of the user, identifying a user image, receiving wireless communication from a user beacon, receiving at least one biometric feature, receiving credentials, or receiving authentication via a user computer system.

[0010] In some implementations, when executed the computer readable code may be further configured to cause the one or more processors to receive a rigging identifier for the rigging, determine when the rigging is authorized for use for the target object, and authorize operation of the material handler when the rigging is authorized.Page 2 of 40018300-902175

[0011] In some implementations, the rigging identifier for the rigging may be received from a user computer system sensing a rigging token of the rigging, identifying a rigging image of the rigging, receiving wireless communication from a rigging beacon, receiving a selection from a rigging list, reading a marking on the rigging, or machine learning-based rigging recognition.

[0012] In some implementations, moving the movable support of the material handler may include receiving a location identifier of the movable support with the rigging, determining a path to move the movable support with the rigging from a first position to a second position based on determining location data of one or more material zones, structure configuration, or movable objects based on sensor information received from one or more sensors, moving the movable support with the rigging along the path, and modifying the movement of the movable support with the rigging based on determining the location data while moving the movable support with the rigging along the path based on the sensor information.

[0013] In some implementations, the one or more sensors comprise an imaging device, and wherein the location data may be inferred from rendering data received from at least one imaging device.

[0014] In some implementations, the one or more sensors are operatively coupled to the movable support, the rigging, or a structure with visibility to the material handler.

[0015] In some implementations, the one or more sensors includes an encoder, accelerometer, IR sensor, a camera, an NFC sensor, or RF sensor.

[0016] In some implementations, the one or more movable objects comprise one or more persons or one or more dynamic equipment, and wherein modifying the movement of the movable support may include determining the location data of the one or more persons or the one or more dynamic equipment and modifying the movement of the movable support with the rigging by stopping, speeding up, slowing down, or modifying the path of the movable support with the rigging.

[0017] In some implementations, the first position includes a current position of the movable support with the rigging, and wherein the second position includes a position adjacent the target object.

[0018] In some implementations, the second position may be determined based on a selection of a material zone of the one or more material zones or a listed material zone from the user computer system, a selection of the target object or a listed target object from thePage 3 of 40018300-902175user computer system, a sensing of a target object token, a verbal identification, an eyetracking position, or a digital path drawing.

[0019] In some implementations, determining the path to move the movable support with the rigging from the first position to the second position based on the location data may include retrieving size or weight parameters of the target object, determining the path between the first position to the second position based on the size or weight parameters of the target object and the location data of the one or more material zones, the structure configuration, or the movable objects.

[0020] In some implementations, determining the path implements a machine learning model or artificial intelligence.

[0021] In some implementations, modifying the movement of the movable support with the rigging occurs at a predetermined interval based on the location data determined from the location data while moving the movable support wi th the rigging along the path at the predetermined interval.

[0022] In some implementations, the predetermined interval may be shortened or lengthened when the movement of the movable support with the rigging speeds up, slows down, or stops.

[0023] In some implementations, moving the movable support of the material handler may further include generating an alert where, based on the one or more sensors determining the location data of the one or more material zones, the structure configuration, or the movable objects, the rigging or the material handler is within a predetermined distance from the one or more material zones, the structure configuration, or the movable objects.

[0024] In another aspect, a computer implemented method for autonomous material handling is presented. The method may include receiving a target object based on an object identifier of the target object to be retrieved by a material handler, wherein the material handler may include a movable support, and rigging operatively coupled to the movable support configured for coupling with one or more target objects, and one or more sensors operatively coupled to the material handler, and moving the movable support of the material handler based on the one or more sensors to the target object based at least on the one or more sensors.

[0025] In yet another aspect, a computer program product for autonomous material handling is presented. The computer program product may include at least one non-transitory computer-readable medium having computer-readable program code portions embodied therein, the computer-readable program code portions may include an executable portion Page 4 of 40018300-902175configured to receive a target object based on an object identifier of the target object to be retrieved by a material handler, wherein the material handler may include a movable support, and rigging operatively coupled to the movable support configured for coupling with one or more target objects, and one or more sensors operatively coupled to the material handler, and an executable portion configured to move the movable support of the material handler based on the one or more sensors to the target object based at least on the one or more sensors.

[0026] The functionality discussed above, as well as other functionality, of the systems, computer implemented methods and / or computer program products will be discussed in further detail throughout.

[0027] To the accomplishment of the foregoing and the related ends, the one or more implementations of the disclosure includes the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth certain illustrative features of the one or more implementations. These features are indicative, however, of but a few of the various ways in which the principles of various implementations may be employed, and this description is intended to include all such implementations and their equivalents.BRIEF DESCRIP ITION OF THE DRAWINGS

[0028] Having thus described implementations of the disclosure in general terms, reference will now be made to the accompanying drawings. The components illustrated in the Figures may or may not be present in certain implementations described herein. Some implementations may include fewer (or more) components than those shown in the Figures.

[0029] FIG. 1 illustrates a system environment for autonomous material handling, in accordance with implementations of the disclosure;

[0030] FIG. 2 illustrates a material handler, in accordance with implementations of the disclosure;

[0031] FIG. 3 illustrates a user interface menu in accordance with implementations of the disclosure;

[0032] FIG. 4 illustrates a user interface graphical representation of a floor layout of a material handling operating area, in accordance with implementations of the disclosure;

[0033] FIG. 5 illustrates a user interface rendering as output by a sensor, in accordance with implementations of the disclosure;

[0034] FIG. 6 illustrates a user interface rendering as output by a sensor, in accordance with implementations of the disclosure; andPage 5 of 40018300-902175

[0035] FIG. 7 illustrates a process flow for autonomous material handling, in accordance with implementations of the disclosure.DETAILED DESCRIPTION

[0036] Implementations of the present disclosure will now be described more fully hereinafter with reference to the accompany drawings, in which some, but not all, implementations of the disclosure are shown. Indeed, the disclosure may be implemented in many different forms and should not be constmed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will satisfy applicable legal requirements. Like number refer to like elements throughout. Where possible, any terms expressed in the singular form herein are meant to also include the plural form and vice versa, unless explicitly stated otherwise. Also, as used herein, the term “a” and / or "an" shall mean “one or more,” even though the phrase “one or more” is also used herein.

[0037] Implementations of the present disclosure provide advantages over existing material handling systems by improving flexibility, safety, and efficiency in the handling of objects. The disclosed system allows for multiple methods of identifying target objects and locations. For example, while target objects and locations may be selected graphically on a “layout rendering,” the present system also supports identification through scanning of object codes, detecting object beacons, or textual selection within a user interface. This increases adaptability for dynamic operating areas where objects or zones may change position or configuration.

[0038] Furthermore, the system improves safety through user authorization protocols. Unlike existing systems, the present system authenticates users through one or more of a variety of identifiers, such as user tokens, biometric data, or wireless beacons, or the like, thereby reducing the risk of unauthorized access or misuse.

[0039] Additionally, the present disclosure mitigates risks associated with mismatched rigging and objects to be handled. Traditional systems often lack real-time compatibility checks between rigging and objects and instead rely on user discretion. In contrast, the disclosed system may use object identifiers to retrieve data related to the object's weight, dimensions, or the like, and rigging identifiers to determine the rigging’s load capacity, constraints, or the like. The system may evaluate compatibility between the rigging and the object before initiating the handling process, thereby reducing the likelihood of equipment failure or accidents.Page 6 of 40018300-902175

[0040] Advantageously, the present system may also support path planning, which provides improvements over systems used in conventional material handling. The disclosed system may determine a path for objects based on real-time sensor data to consider obstacles such as other objects, personnel, restricted zones, or the like. The present system may also modify the path during movement by collecting real-time sensor data during the movement along the path to adjust for unexpected changes and avoid potential collisions.

[0041] Moreover, the system enhances operational efficiency by integrating real-time data from sensors such as accelerometers, imaging devices, RF sensors, and NFC sensors. This real-time data allows the system to dynamically adjust movement speed, stop, reverse, or otherwise alter the movement along a path.

[0042] The present disclosure also improves material handling processes through machine learning models. By collecting location and path data both before and during object movement, the system can provide continuous training data to refine its path-planning algorithms.

[0043] As may be appreciated by one of skill in the art, the present disclosure may be implemented as a system, method (computer implemented or otherwise), computer program product, or a combination of the foregoing. Accordingly, the present disclosure may take the form of an entirely software implementation (including firmware, resident software, microcode, and the like) or an implementation combining software and hardware aspects that generally may be referred to herein as a “system.” Furthermore, certain aspects of implementations of the present disclosure may take the form of a computer program product on a computer-readable medium having computer-usable program code implemented in the medium.

[0044] Any suitable computer-readable medium may be utilized. The computer-readable medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, or semiconductor system, apparatus, or device. More specific examples of the computer-readable medium include, but are not limited to, the following: a tangible storage medium such as a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory, a compact disc read-only memory (CD-ROM), or other optical or magnetic storage device, or other storage device.

[0045] Computer program code for earn ing out certain operations of implementations of the present disclosure may be written in an object oriented, scripted, or unscripted programming language such as Java, Perl, Smalltalk, C++, SAS or the like.Page 7 of 40018300-902175However, the computer program code for carrying out operations of implementations of the present disclosure may also be written in conventional procedural programming language, such as the "‘C” programming language or other similar programming language for carrying out certain operations of implementations of the present disclosure.

[0046] FIG. 1 depicts a schematic of the system 1, in accordance with implementations of the disclosure. As depicted in FIG. 1. the system may include a user computer system 100, an entity computer system 200. a material handler 402, and at least one object 300.

[0047] It shall be appreciated that some implementations of the disclosure may only contain a subset of the aforementioned components of the system 1. For example, in one implementation, the system 1 may include a user computer system 100 and at least one material handler 402, while not including an entity computer system 200. In such implementations, the material handling application may be executable on either the user computer system 100 and / or the material handler 402, and the one or more sensors (as will be described in detail herein) may be included entirely on the user computer system 100, entirely on the material handler 402. or split between the user computer system 100 and the matenal hander 402, such that certain functions and process steps are performed at different portions of the system 1.

[0048] Alternatively, in some implementations of the disclosure, the system 1 may not include a user computer system 100 or an entity computer system 200, and instead the system may include a material handler 402, where the material handling application is embedded in and executable from the material handler 402 such that the functions and process steps described herein are performed at the material handler 402. As such, the one or more sensors 413 may be included entirely on the material handler 402.

[0049] Alternatively, in some implementations of the disclosure, the system 1 may include an entity computer system 200 and a material handler 402, and not necessarily include a user computer system 100. In such implementations, the material handling application may be fully or partially embedded in and executable from the material handler 402 and / or fully or partially executable from the entity computer system 200. such that some or all of the functions and process steps described herein are performed at the material handler 402, while some or all of the functions and process steps described herein are performed by the entity' computer system 200. Accordingly, the one or more sensors may be included entirely on the entity computer system 200, entirely on the material handler 402, placed throughout the operating area, placed on persons, and / or moveable equipment, or split Page 8 of 40018300-902175between the entity computer system 200, the material hander 402, and / or within the operating area, on persons, and / or moveable equipment.

[0050] Alternatively, in some implementations of the disclosure, such as that which is depicted graphically in FIG. 1, the system 1 may include a user computer system 100, an entity computer system 200, and a material handler 402. In such implementations, the material handling application may be fully or partially embedded in and executable from the material handler 402, and / or fully or partially executable from the user computer system 100, and / or fully or partially executable from the entity’ computer system 200, such that some or all of the functions and process steps described herein are performed at the material handler 402, while some or all of the functions and process steps described herein are performed by the user computer system 100. and some or all of the functions and process steps described herein are performed by the entity computer system 200. Accordingly, the one or more sensors 113, 213, 413 may be included entirely on the user computer system 100, entirely on the material handler 402, entirely on the entity computer system 200, entirely placed throughout the operating area, placed on persons, and / or moveable equipment, or split between the user computer system 100, the material hander 402, the entity computer system 200, and / or throughout the operating area, on persons, and / or moveable equipment.

[0051] The user computer system 100 may be any computing device that may include communication device 110, memory 111, one or more processors 112, one or more sensors 113, and / or a display 114 devices, and computer readable instructions 120 stored in the memory 111 and configured to cause the one or more processors 112 to perform the process steps described herein, such as by executing a material handling application. In some implementations, the user computer system 100 may be a mobile device such as a mobile phone (e.g., smartphone, personal digital assistant, or the like), a tablet, laptop, or another type of mobile computing device. In other implementations, the user computer system 100 may be a desktop computer or other ty pe of computing device.

[0052] In some implementations of the disclosure, it should be understood that the user computer system 100 (and / or the entity computer system 200 or material handler 402), may be a wearable device (e.g., glasses, watches, helmets, devices attached to clothing, or the like), and as such, the wearable device may operate in the same or similar way as described herein with respect to the user computer systems (e.g., a user’s mobile device, such as a phone, or the like). In some implementations, the wearable devices may be considered a mobile device. In some implementations, a camera and / or lens associated with the wearablePage 9 of 40018300-902175device may operate as the one or more sensors 113 that are used to identify element(s) (via their corresponding identifier(s)) within an operating area.

[0053] As used herein, an ‘'element” may refer to one or more of the object(s), user(s), target area(s), material zone(s), rigging(s), walkway(s), or the like, which are located in an operating area and relevant to the material handling operations performed by the system. These elements may be used as inputs to path determination, safety checks, and object compatibility assessments, or other functions described herein. The system may track, identify’, and analyze these elements using various sensors, user inputs, or automated detection mechanisms to optimize performance and reduce operational risks.

[0054] As used herein, an “operating area” may refer to an environment of the material handler in which material handling operations occur. The operating area may include indoor environments such as warehouses, manufacturing facilities, distribution centers, or storage depots. Additionally, the operating area may include outdoor environments such as construction sites, shipyards, loading docks, or freight yards. In some implementations, the operating area may include a combination of indoor and outdoor spaces, such as facilities where materials are transferred between indoor storage and outdoor storage, and / or to transport vehicles. Furthermore, the operating area may extend to specialized environments, such as hazardous material zones, clean rooms, refrigerated storage units, or multi-level structures.

[0055] In some implementations, the user may utilize gesture controls to control the actions for the features described with respect to the material handling application through the user computer system 100 (and / or the entity computer system 200 or material handler 402). For example, the gestures may include using a touchscreen, making other hand movements that are captured without touching the user computer systems, making eye. head, mouth, arm, finger, or other body gestures in order to control actions within the material handling application. It should be understood that the gestures may be utilized to take any of the actions that can be taken with respect to the features of the material handling application described herein.

[0056] The one or more sensors 113 may be any type of device that captures information, such as a camera that captures an image (e g., image of a label, barcode, part number, object itself, biometric data of a user such as a facial image, pupil or iris image, fingerprint or handprint image, or the like), a scanner that scans a barcode or other identifier (e.g., a scanner that may communicate with the mobile device wirelessly or via a wired connection), an electronic capture device that can electronically communicate with an Page 10 of 40018300-902175identifier (e.g., via near-field communication, Bluetooth, RFID, IR, RF, or the like), or other similar sensors. In some implementations, (for example, implementations without the user computer system 100) one or more sensors 213, 413 may be included in addition to, or instead of, the one or more sensors 113, and may be structured in the same way and / or perform any or all of the same functions as those ascribed to the one or more sensors 113 herein.

[0057] The display 114 may be any display output device (e.g., screen, touchscreen, or other like display) capable of displaying images or other information (e g., identifier data, location data, or the like) to the user, such as from the material handling application 121 stored on the user computer system 100, and / or the material handling application 221 stored on the entity computer system 200.

[0058] In some implementations, the user computer system 100 may include a communication device 110 that is configured to facilitate communication over the network 2 between the user computer system 100 and an entity computer system 200, material handler 402, and / or other user computer system (e.g., other users) or entity computer systems (e.g.. third-party entities, or the like). In some implementations, the user computer system 100 may include a datastore 122 for storing data related to the user computer system 100, such as the material handling application 121. It should be understood that the system 1 illustrated in FIG. 1 may include the use of multiple user computer systems 100 used by one or more users.

[0059] The entity computing system 200 may be any computing infrastructure equipped with communication components 210, memory 211, and processing devices 212. The memory 211 contains computer-readable instructions 220 that, when executed by the processing devices 212, perform the functions of a material handling application 221.Additionally, the entity computing system 200 may incorporate a datastore 222 for storing relevant data. In some implementations, the entity computing system 200 operates as an offsite server or a network of servers handling these tasks in the background for support of the processes of the user computing system 100. The entity computing system 200 is designed to communicate over the network 2 with the user computing system 100 and / or other user or entity systems (e.g., users, equipment, sensors, or the like located within the operating area or remote from the operating area).

[0060] Also shown in FIG. 1 is an object 300. The object 300 may be any object which could be moved by the material handler 402. Examples of an object 300 include, but are not limited to, a coiled metal sheet, a stack of metal sheets, a beam, bar, rebar, pipe, a pipe bundle, truss joist, a shipping crate, a box, a piece of equipment, bagged or otherwise Page 11 of 40018300-902175packaged materials such as concrete, sand, vehicles, construction materials such as bricks, panels, or tiles, an appliance, a bulk container, a reel of cable, manufactured goods, or the like.

[0061] As illustrated in FIG. 1, in some implementations, an object 300 may include an object identifier 301. As depicted in FIG. 1, the object identifier 301 may be a barcode or QR code. However, as used herein, an ‘‘identifier” (e.g., an object identifier, rigging identifier, user identifier, location identifier, or any identifier corresponding to an element within or outside of the operating area) may refer to any form of representation that can be recognized, captured, or processed by one or more sensors 113. An identifier of an element (e.g., the object identifier 301) may be a phy sical or digital marker such as a barcode, QR code. RFID tag, NFC tag, or any alphanumeric code or label, SKU. UPC. part list, token number, electronic device, or other uniquely identifying piece of information. An identifier may be affixed to the outside of the element, located within the element, may be the element itself (e.g., shape, size, color, or the like), located adjacent the element, or the like. It should be further understood that in the cases of the identifier being an electronic device with its own communication device, it may be operatively coupled to the network 2 in the same or similar way as discussed with respect to the user computer system 100 and / or the entity computer system 200.

[0062] Additionally, or alternatively, an identifier may take the form of a graphical representation like an image or icon displayed on a screen that represents the corresponding element. In some implementations, an “identifier” could be a machine-learning inference identification based on one or more features or characteristics of the element itself (e.g., a specific shape, color, or marking) that sensors 113 can detect. Additionally, or alternatively, an “identifier” may appear as a graphical element on a display, such as a selectable item in a list presented on the user computer system 100 or the material handler 402 interface, or as a rendering or other representation of the corresponding element (e.g., within an operating area of the material handler 402, for example as depicted in a layout rendering). This may allow users to interact with identifiers digitally by selecting corresponding entries on a touchscreen or display interface.

[0063] Also shown in FIG. 1 is a user 500, which may be present in the system 1 in some implementations. The user 500 may be any user attempting to interact with the material handler 402. In some implementations, a user 500 may include a user identifier 502. The user identifier 502 may take the form of any of the physical or digital markers previously described (which may be attached to the clothing, headwear, badge, or other apparel in Page 12 of 40018300-902175possession of, or atached to, the user 500), graphical representation such as the icons or images previously described, a graphical element on a display selectable by list, rendering, or other representation of the user 500.

[0064] Additionally, or alternatively, the user identifier 502 may also include biometric data such as a fingerprint, facial image, iris scan, or handprint, which can be captured and processed by sensors 113, 213, 413 like cameras or biometric scanners. In some implementations, the user identifier could be a feature or characteristic of the user itself (e.g., a facial shape, eye color, or the like) that sensors 113, 213, 413 can detect.

[0065] It should be understood that in the cases of the user identifier 502 being an electronic device with its own communication device, it may be operatively coupled to the network 2 in the same or similar way as discussed with respect to the user computer system 100 and / or the entity computer system 200.

[0066] Also shown in FIG. 1 is a rigging 600, which may be present in the system 1 in some implementations. The rigging 600 may be any rigging available to use with the material handler 402 to handle the object 300. Rigging 600 may encompass a wide variety of equipment types used for lifting, securing, and moving objects. Examples of rigging include slings (e.g., wire rope slings, chain slings, synthetic web slings, and round slings), shackles (e.g., screw pin shackles and bolt-type shackles), hoists (e.g., chain hoists, lever hoists, and electric hoists), hooks (e.g., eye hooks, clevis hooks, and swivel hooks), lifting beams or spreader bars, turnbuckles, eyebolts, pulleys, winches, blocks, snatch blocks, lifting clamps, grabs, lifting plates, beams, pipes, or the like. In some implementations, a rigging 600 may include a rigging identifier 602. The rigging identifier 602 may take the form of any of the physical or digital markers previously described, graphical representation such as the icons or images previously described, a graphical element on a display selectable by list, rendering, or other representation of the rigging 600.

[0067] In some implementations, the rigging identifier could be a feature or characteristic of the rigging itself (e.g., a specific shape, color, or marking) that sensors 113, 213, 413 can detect.

[0068] The rigging identifier 602 may be affixed to the outside of the rigging 600, located within the rigging 600, may be the rigging itself (e g., shape, size, color, or the like), located adjacent the rigging 600, or the like. It should be further understood that in the cases of the rigging identifier 602 being an electronic device with its ow n communication device, it may be operatively coupled to the network 2 in the same or similar way as discussed with respect to the user computer system 100 and / or the entity computer system 200.Page 13 of 40018300-902175

[0069] Also show n in FIG. 1 is a location identifier 132, which may be present in the system 1 in some implementations. Although the location identifier 132 is illustrated as a graphical element on a display (e.g., a selectable target location of the layout rendering 131), it shall be appreciated that a target location may be selected through means similar to that of the object 300 itself. Stated differently, the location identifier may be a selectable location in a list presented on the user computer system 100 or the material handler 402 interface, or as a rendering or other representation of the corresponding target location. This may allow users to interact with the location identifiers digitally by selecting corresponding entries on a touchscreen or display interface.

[0070] The target location may be any target location available as a destination of the object 300 being handled by the material handler 402. A target location may include a pallet racks location, a shelving unit, bulk storage area, a floor stacking zone, a bin, a mezzanine, a cage or other secured enclosure, a refrigerated or cold storage zone, a climate-controlled area, a dock area, cantilever rack, a storage bay, or any other type of zone within an operating area.

[0071] The location identifier 132 may be a barcode or QR code attached to the target location. The location identifier 132 may be a physical or digital marker such as a barcode, QR code, RFID tag, NFC tag, or any alphanumeric code or label, SKU, UPC, part list, token number, electronic device, geo-fenced location, or other uniquely identifying piece of information.

[0072] In some implementations, the location identifier could be a feature or characteristic of the target location itself (e.g., a specific shape, color, or marking) that sensors 113, 213, 413 can detect.

[0073] The location identifier 132 may be affixed to a structural component, building component, shelving unit, zone, or the like outside of or within the target location, located adjacent the target location, or the like. It should be further understood that in the cases of the location identifier 132 being an electronic device with its own communication device, it may be operatively coupled to the network 2 in the same or similar way as discussed with respect to the target location computer system 100 and / or the entity computer system 200.

[0074] As will be described in greater detail herein, the entity computer system 200, the user computer system 100, and / or the material handler 402 may compare the received identifiers of elements (e.g., object identifier 301, the location identifier 132, the user identifier 502, the rigging identifier 602, or the like) to corresponding “identification data’' stored in the datastore 222. Additionally, or alternatively, “location data”, images, or other data received by the one or more sensors 113, 213, 413 may be transmitted between the item Page 14 of 40018300-902175datastores 122, 222, 422, of the entity computer system 200, the user computer system 100, and / or the material handler 402 to support the determining of a path for the movable support having the rigging 600, as will be discussed herein.

[0075] The datastore 222 of the entity computer system 200 (and / or, in some implementations, datastore 122 of the user computer system and / or datastore 422 of the material handler) stores identification data regarding objects and maintains correlation between the objects and their associated object identifier(s). For example, identification data of an object may include dimensions (e.g., length, width, and / or height), weight, a description of the object, part number, value or the object, current location (e.g., by coordinates or the like), data regarding any orders associated with the object (e.g., the supplier, the purchaser, transporter, or the like), quantity of objects of the same type, material composition, special storage requirements, rigging types required for handling the object, qualifications of a user (i.e., operator) required for handling the object, or the like.

[0076] Additionally, or alternatively, the datastore(s) 122, 222, 422 may store identification data regarding users, and maintain a correlation between the users and their associated user identifier(s) 502. For example, identification data of a user may include a user ID, name, employee number, site-specific authorization (e.g., whether a user performs tasks at a given site), material handler authorization (e.g., whether a user has been trained or has experience in using a particular material handler 402), trainings, qualifications, or certifications completed, entity’ hierarchical information such the name or identifier of supervisor(s) or subordinates, length of service, number of hours worked, or the like.

[0077] Additionally, or alternatively, the datastore(s) 122, 222, 422 may store identification data regarding rigging, and maintain a correlation between the rigging and their associated rigging identifier(s). For example, identification data of rigging may include weight limits of the object 300 to be attached thereto, number of cycles for which the rigging has been used, identifiers of material handlers for which the rigging is compatible, qualifications of a user (i.e., operator) required for using the rigging, or the like.

[0078] Additionally, or alternatively, the datastore(s) 122. 222, 422 may store identification data regarding target locations, and maintain a correlation between the target locations and their associated location identifier(s) 132. For example, identification data of a target location may include coordinates (or other location identification mechanisms) of the target location, boundary size and location of the boundaries that define the target location, weight limit of the target location, a total weight of the current occupants of the targetPage 15 of 40018300-902175location, environmental information such as indoor or outdoor, temperature ranges, humidity, or the like.

[0079] Additionally, or alternatively, the datastore(s) 122, 222, 422 may store database(s) containing correlation(s) between any one or more of the identification data regarding objects, users, rigging, and / or target locations, in order to have pre-authorized or pre- unauthorized combinations of the foregoing. For example, it may be known and predetermined that certain objects 300 shall not be handled with certain rigging 600.

[0080] It should be understood that the material handling application 121, 221, 421 may include portions that are stored on the user computer systems 100 (e.g., app, applet, downloaded software, or the like) and / or portions that are stored on the entity computer systems 200 (e.g., app. applet, downloaded software, base software, or the like). It should be further understood, generally, that any data described herein may be stored locally in a decentralized location on the user computer system 100 and / or may be stored centrally in a centralized location on the entity computer system 200. As such, data may be captured, stored and / or accessed from various systems at various locations throughout the system 1.

[0081] It should be understood that the system 1 may operate through a cloud computing resource. That is, the entity computer system 200 may be cloud based hosting sendee that may be operated by a third party for the entity or by the entity itself.Additionally, the entity may store some aspects of the data disclosed herein, while a third-party stores other aspects of the data. Moreover, it should be understood that the material handling application 121, 221, 421 may be a dedicated application that is downloaded on the user computer system 100 (and / or the entity computer system 200 or material handler 402), it may be a web-based solution that allows the user computer systems 100 (and / or the entity computer system 200 or material handler 402) to access the material handling application 121, 221, 421 located on the cloud computing resource, or both.

[0082] The user computing system 100 (or in some implementations, the entity computer system 200 and / or the material handler 402) may display information regarding the object, the rigging, the user, and / or the layout rendering 131 of an operating area on which the material handler 402 is capable of performing the handling of the object 300. The layout rendering 131 displayed on the display may be one or more views provided by one or more cameras with a view of at least a portion of the operating area, with overlays, such as boxes or other graphical elements, representing selectable elements. For example, a green box overlay may be a location identifier that indicates a selectable target location for directing the material handler 402 to move the object 300. In another example, the layout rendering may Page 16 of 40018300-902175depict a computer-generated model of the operating area, where selectable elements correspond to specific regions or components within the operating area. In some implementations, the rendering may represent a real-time view, wherein the movement of objects, users, rigging, or similar elements is dynamically represented by corresponding symbols, boxes, or other indicators to reflect real-time changes within the operating area. The rendering may be a 2D or 3D rendering of the operating area.

[0083] In some implementations of the disclosure, the layout rendering 131 may be displayed in the interface of the display of the user computer system 100 (and / or the entity computer system 200 or material handler 402) through the use of augmented reality, mixed reality or virtual reality. For example, a camera on the mobile device (e.g., phone, wearable device, or the like) may display the actual surroundings of the operating area as seen through the camera and overlay the layout rendering 131 of the operating area on the interface illustrated in the display that is providing the surroundings captured by the camera in realtime. As such, the layout rendering 131 may be rendered and positioned at full scale as a digital overlay upon the physical real world operating area that the user may be viewing through the user computer system (e.g., including potentially the through the wearable devices discussed herein, or the like). In other implementations, the virtual reality interface or mixed reality interface may be displayed as illustrating a rendering of the surroundings with the layout rendering 131 superimposed in the virtual reality or mixed reality environment.

[0084] In some implementations, users may share the interfaces on the user’s user computer system 100 (and / or the entity computer system 200 or material handler 402) with one or more other users in real-time. Additionally, the user may share the same information in the augmented reality, virtual reality, and / or mixed reality environments that was previously discussed herein as being shared in the traditional interfaces (e.g., on a touchscreen, or the like). That is, a user may allow another user (e.g., a guest user) to view what the user is viewing (e.g., in an augmented reality environment, or the like) and / or communicate with each other through a microphone, speakers or like.

[0085] Implementations of the disclosure will now be described with reference to FIG. 2, w hich illustrates a material handler, in accordance with implementations of the disclosure. As shown in FIG. 2, a material handler 402 may be implemented as a crane, where the crane includes a movable support 424, which is movable along one or more axes to a position proximate an object 300 to be handled.Page 17 of 40018300-902175

[0086] Attached to the movable support 424 may be the rigging 600, which may include connectors, straps, locks, or the like, to secure the object 300. during the handling (e.g., movement from a first position to a second position) of the object 300. A portion of the movable support 424 may be retractable, such as a spool or similar feature, which may raise or lower the rigging 600, or otherwise move the rigging closer to, or away from, the object 300 to facilitate the securing of the rigging 600 to the object 300 and / or to move the object 300 in a direction to avoid collisions with other objects, users, or the like during handling of the object 300.

[0087] While reference is made in FIG. 2 to a material handler 402 that resembles a crane, it shall be appreciated that various other implementations of material handler 402 are contemplated herein. For example, material handler 402 may take the form of a robotic arm, humanoid robot, autonomous forklift, a conveyor system, an overhead gantry, or the like. Moreover, while the movable support 424 depicted in FIG. 2 reflects that of a carriage movable in one or more horizontal directions, other implementations may include a movable support 424 capable of vertical movement, a pivoting arm, a telescoping mechanism, or the like.

[0088] The operating area that includes the material handler 402 may include one or more sensors, illustrated in FIG. 2 as a camera 336. As previously described, the one or more sensors are in operable communication with the user computer system 100, the entity computer system 200, or the material handler 402, may provide for location data of elements within the operating area. This location data may be in addition to, or instead of, any identification actions provided with the assistance of the one or more sensors, such as to identify a user in control of the material handler, identify the object 300, identify the rigging 600, identifying the target location, or the like.

[0089] The one or more sensors (show n as camera 336, or a plurality of cameras 336, or any other vision-type sensor) may be mounted to a mount 334 to monitor the operating area by capturing visual data of the operating area from one or more angles or vantage points. The mount 334 may be any structure capable of positioning the sensor(s), such as a wall, beam, tripod, or the like. The camera 336 may produce an image or video stream that may be fed into a processing system of the entity computer system 200 (or user computer system 100, or material handler 402) equipped with recognition capabilities. As previously described with respect to FIG. 1 , the system may analyze the captured images using techniques such as object detection and segmentation, to identify (i.e., capture identification data) elementsPage 18 of 40018300-902175within the field of view. Through feature extraction, the system may distinguish these elements based on shape, size, color, and other visual characteristics.

[0090] Additionally, or alternatively, identification data may be created through manual labeling of objects 300, users 500, target locations, rigging, and / or obstacles, walkways 338, or the like via the display of a computing system such as the user computing system 100, entity computing system 200, or material handler 402. Once labeled, the system may store the identification data alongside the corresponding spatial coordinates within a database.

[0091] To determine the locations (i.e., location data) of the elements within the operating area, the system may apply one or more spatial mapping algorithms to determine positions relative to reference points, such as walls, supports, joists, static equipment, shelving units, or floor markers. Triangulation techniques and / or stereo vision methods may be implemented in some implementations to increase positional accuracy by combining the perspectives of different cameras to produce a three-dimensional map of the operating area to result in a real-time representation of the operating area, with spatial coordinates / positions associated with each detected objects, supports, joists, static equipment, users, dynamic equipment, target locations, rigging, and / or obstacles, or the like.

[0092] In some implementations, both identification data receiving / determining and location data receiving / determining may be accomplished in parallel. For example, the system may merge location data with identification data to not only determine identification of the objects, users, target locations, rigging, and / or obstacles, or the like, but also their corresponding locations. As previously described with respect to FIG. 1, identification data can be acquired by one or more sensors 113, 213, 413 (which may be the same, in lieu of, or in addition to the camera(s) illustrated in FIG. 2) through the object identifier 301, user identifier 502, rigging identifier 602, and / or location identifier 132. The system may correlate this identification data with the spatial coordinates derived from the foregoing location data to associate each object’s identity with its real-time location.

[0093] FIG. 3 illustrates a sample user interface that may include a menu 11 and / or a layout rendering 131. While the user interface of FIG. 3 is illustrated as being a part of the user computer system 100 (e.g., the display 114), it shall be appreciated that the user interface may additionally, or alternatively, be included as part of the entity computer system 200 and / or the material handler 402. The menu 11 may provide the user with access to various aspects of the disclosure, such as those described with reference to FIGS. 1 and 2. For example, the menu 11 may include a listing of available target objects (e.g., as target object Page 19 of 40018300-902175identifiers) for handling with the material handler, a listing of available rigging (e.g., as rigging identifiers), a listing of available users (e.g.. as user identifiers), and / or a listing of available target locations (e.g., as location identifiers). In this way, a user may interact with these elements on the list(s) in order to facilitate the functions described herein. For example, a user may select a target object from the list, a rigging, and a target location in order to initiate the handling of the target object.

[0094] Additionally, or alternatively, the user interface may include one or more layout renderings 131, which will now be described with respect to FIG. 4. In some implementations, the one or more layout renderings 131 may be for monitoring the operating area, with the menu 11 being the portion interactive with a user to handle an object.Additionally, or alternatively, the one or more layout renderings 131 may be provided with interactive overlays over one or more objects 300 (e.g., shown as object identifier 510 in FIG.4), users 500 (e.g., shown as user identifier 508 in FIG. 4), target locations (e.g., shown as location identifier 504 in FIG. 4), storage zones (e.g., shown as zone 502 in FIG. 4), rigging, and / or obstacles (e.g., shown as obstacle 506 in FIG. 4), walkways 338, or the like, as identified using the methods described previously. In this way, a user may interact with any of the foregoing to facilitate the functions described herein. In such implementations, the menu 11 may be provided as a means of monitoring the identified objects, users, walkways, target locations, rigging, or the like, within view of the one or more sensors used to facilitate the rendering of the one or more layout renderings 131.

[0095] In some implementations, the one or more layout renderings 131 may include an illustrated path 512, which indicates a path through with the movable support with rigging (or just the movable support, or movable support with rigging in addition to the object) will travel when moving between a first position (e.g., the initial location) and a second position (e.g., the target location). In some implementations, a plurality of illustrated paths 512 may be provided, such that a user may choose the path 512 that best suits the circumstances. In such implementations, the material handler may be instructed to move the movable support with rigging (with or without the object) in accordance with the path selected.

[0096] While the one or more layout renderings 131 primarily provide a user with a visual aid and / or interactive elements to execute material handling, it shall be appreciated that the underlying data (e.g., location data, identification data, or the like) may be analyzed via underlying processes not visible to the user to determine characteristics of the movement between the first position and the second position, including, but not limited to, the path (e.g., the movement vectors of the movable support (with or without rigging and / or the object), Page 20 of 40018300-902175speed of the movable support along the path, acceleration / deceleration profiles, or the like, in order to automatically determine the safest path and movement characteristics between the first position and second position to avoid users, obstacles, objects, walkways, or the like, minimize the distance traveled between the first position and second position, restrict (e.g., remove, decrease, or the like) potential damage to objects and / or elements along the path by controlling speed / acceleration. or the like. Indeed, mechanisms for such analyses will be described in detail herein with respect to FIG. 7.

[0097] FIGS. 5 and 6 illustrate other implementations of user interfaces, with exemplary layout renderings therein. While the layout rendering 131 of FIG. 4 is shown as a top plan view- model, the layout rendering(s) may include a perspective view via video or image (e.g., as shown in FIG. 5), a perspective view of a model (e.g., as shown in FIG. 6), or any number of outputs from the one or more sensors.

[0098] As shown in FIGS. 5 and 6, some implementations of the user interface may include a search bar to allow for a user to enter a descriptor, part number, name or the like, thereby prompting the one or more layout renderings to show the corresponding object, user, target location in real time, such as by switching between views available from the one or more sensors.

[0099] As discussed in greater detail throughout the present disclosure, one or more sensors (e.g., camera, RFID, RF. Bluetooth, or the like) may capture an object identifier 301 associated with an object 300, and retrieve identification data related to the object (e.g., by capturing a barcode and accessing identification data for the barcode). Additionally, or alternatively, the sensor may capture a user identifier, a location identifier, and / or a rigging identifier, or the like. Thus, the user interface may include an interactive element (e.g., a button) to initiate the capturing of the aforementioned identifier(s). Additionally, or alternatively, the user interface may allow the user to interact with the layout rendering 131 in various ways. For example, the user interface may include a back button for navigating between previous displays of the layout rendering 131, a zoom tool that allows the user to use two-finger touch to zoom in on parts of the layout rendering 131, or a default zoom tool that zooms in on the layout rendering 131 by a preset value. Moreover, the user interface may allow' the user to configure the layout rendering 131 to only display certain objects, such as objects, rigging, target areas, walkways, users, other predetermined storage zones, or other selectable objects of the layout rendering 131.

[0100] FIG. 7 illustrates a process flow for autonomous material handling, in accordance with implementations of the disclosure. In some implementations, as illustrated Page 21 of 40018300-902175at block 702 of FIG. 7, the operation of the material handling device (e.g., a material handler) may first be authorized.

[0101] To do so, a user identifier of a user may be received. In some implementations, the system 1 may receive a selection of a user identifiers on the user interface. The selection may be made by selecting the user identifier such as a photo of a user, by selecting the user identifier from a list of users or user identifiers provided through the interface on the display, or the like. In some implementations, the user may manually enter the credentials, an employee number, name, or the like by typing into the user interface.

[0102] Additionally, or alternatively, the selection of one or more user identifiers may occur by capturing a user identifier 502 associated with a user with one or more sensors 113. A user identifier 502 from a user may be captured in several ways. In some implementations, the one or more sensors 113 may be an imaging device (e.g., a camera) that may be utilized to take an image of the user (e.g., face, hand, eye, finger, or the like) to perform biometric authentication of the user. Biometric authentication may occur by comparing the user identifier captured with that of one or more stored templates corresponding to various users within the entity.

[0103] Additionally, or alternatively, the one or more sensors 113 may be a scanner for scanning a barcode on a user’s badge, helmet, clothing, or the like. Similarly, capturing may occur by electronically communicating with the user identifier 502, such as through the use of RF scanning, near field communication, wireless communication, or the like wherein the user’s badge, helmet, clothing, or the like is equipped with an electronic identifier (e.g., a beacon or emitter that emits a radio frequency signal, other electronic signal, or other electronic communication) that can be detected by the user computer system 100 (and / or the entity computer system 200 or material handler 402).

[0104] Regardless of how the user identifier may be obtained, the user identifier may be used to authorize that the user (e.g., via the user identifier discussed previously) possesses the requisite qualifications for using the material handler. In some implementations, the user identifier of the user may be compared directly (e.g., via lookup in a database, or the like) to cross reference a predetermined list of user identifiers who are authorized to use the material handler. In other words, comparison of corresponding identifiers may be performed directly. Additionally, or alternatively, the system may retrieve user information (i.e., identification data) corresponding to the user identifier 301 for the user. To retrieve this identification data for the user, processing steps may include matching the user identifier for the user with identification data stored in the datastore 122, 222, 422. Relevant data of the material handler Page 22 of 40018300-902175may be retrieved from the identification data of the material handler (e.g., required qualifications or certifications, name or description of the material handler, or the like). Similarly, identification data for the user may be retrieved, whereinafter comparisons are performed (e.g., comparing the required certifications with certifications of the user retrieved from the identification data of the user, comparing the name or description of the material handler to names or descriptions of the material handler of which the user is permitted to use, and so forth).

[0105] In some implementations, if predetermined conditions are met (e.g., a match occurs between user and the material handler, or the like as described in the foregoing), the process may continue such that the user is considered authorized for using the material handler. Additionally, or alternatively, if predetermined conditions are not met, the process may be discontinued, such that the user is not able to proceed with any additional steps of the process or use the material handler. In other words, the application may be locked.Additionally, or alternatively, the application may prompt the user for an override, such as by requiring credentials of an authorized user, supervisor, or other user with predetermined permissions.

[0106] Continuing at block 704, the target object for the material handling may be selected. The target object may be received based on an object identifier of the target object to be handled by the material handler.

[0107] In some implementations, the system 1 may receive a selection of one or more target objects in the layout rendering 131. For example, a user may select one or more object identifiers from the rendered layout rendering 131 provided in the interface on the display. The selection may be made by selecting the object identifier within the layout rendering 131 illustrated on the interface of the display, by selecting the object identifier from a list of objects or object identifiers provided through the interface on the display, or the like. In some implementations, the user may manually enter the object identifier by typing the object identifier into the user interface, such as by ty ping in an object number, name, or other like identifier.

[0108] Additionally, or alternatively, the selection of one or more object identifiers by capturing an object identifier 301 associated with an object 300 with one or more sensors 113. An object identifier 301 from an object 300 may be captured in several ways. In some implementations, the one or more sensors 113 may be a camera and the camera may be utilized to take an image of the identifier (e.g., barcode, QR code, product number or name, object number or name, an image of the object or material itself, or the like).Page 23 of 40018300-902175

[0109] Additionally, or alternatively, the one or more sensors 113 may be a scanner, and a user may scan a barcode on an object. Similarly, capturing may occur by electronically communicating with the object identifier 301, such as through the use of RF scanning, near field communication, wireless communication, or the like wherein the object or product is equipped with an electronic identifier (e.g., a beacon or emitter that emits a radio frequency signal, other electronic signal, or other electronic communication) that can be detected by the user computer system 100 (and / or the entity computer system 200 or material handler 402.

[0110] In some implementations, an image or scan of the object may be provided by the one or more sensors 113 to an application for machine-learning or artificial intelligencebased object recognition, where the object (and corresponding identification data) may be inferred based on previously trained models, image classification algorithms, and / or feature extraction techniques. These models may analyze visual characteristics, such as shape, size, texture, or color patterns, to match the object to known categories within a dataset. The inferred identification data may then be cross-referenced with a database to verify or improve recognition accuracy.

[0111] Regardless of how the object identifier may be obtained, the object identifier may be used to authorize that the user (e.g., via the user identifier discussed previously) possesses the requisite qualifications for handing the target object. In some implementations, the object identifier of the target object may be compared directly (e.g., via lookup in a database) to cross reference a predetermined list of user identifiers who are authorized to handle the target object. In other words, comparison of corresponding identifiers may be performed directly. Additionally, or alternatively, the system may retrieve target object information (i.e., identification data) corresponding to the object identifier 301 for the object 300. To retrieve this identification data for the target object, processing steps may include matching the object identifier 301 for the object 300 with identification data stored in the datastore 122, 222, 422. Relevant data of the target object may be retrieved from the identification data (e.g., required qualifications or certifications, name or description of the target object, or the like). Similarly, identification data for the user may be retrieved, whereinafter comparisons are performed (e.g.. comparing the required certifications with certifications of the user retrieved from the identification data of the user, comparing the name or description of the target object to names or descriptions of the target object of which the user is permitted to handle, and so forth).

[0112] In some implementations, if predetermined conditions are met (e.g.. a match occurs between authorized users and the target object, or the like as described in the Page 24 of 40018300-902175foregoing), the process may continue such that the user is considered authorized for material handling with respect to the target object. Additionally, or alternatively, if predetermined conditions are not met, the process may be discontinued, such that the user is not able to proceed with any additional steps of the process or handling of the target obj ect. In other words, the application may be locked. Additionally, or alternatively, the application may prompt the user for an override, such as by requiring credentials of an authorized user, supervisor, or other user with predetermined permissions.

[0113] Next, at block 706, rigging may be selected, such as by receiving a rigging identifier. The rigging may be received based on rigging desired to be selected for use in coupling to the target obj ect.

[0114] In some implementations, the rigging may couple the target object to the movable support to facilitate material handling. This coupling may occur either after or before the rigging has been authorized by the system, and may occur autonomously (e.g., by automatically securing the target object), or manually. As previously described, the rigging authorization process may involve verifying the compatibility of the rigging with the target object based on parameters such as weight, dimensions, and load capacity, or the like, and / or verifying the user’s authorization to use the rigging. Upon successful authorization, the system may provide positive confirmation via the user interface, such as a visual indicator, audible signal, textual message, or the like.

[0115] In some implementations, the system 1 may receive a selection of one or more riggings in the layout rendering 131. For example, a user may select one or more rigging identifiers from the rendered layout rendering 131 provided in the interface on the display. The selection may be made by selecting the rigging identifier within the layout rendering 131 illustrated on the interface of the display, by selecting the rigging identifier from a list of riggings or rigging identifiers provided through the interface on the display, or the like. In some implementations, the user may manually enter the rigging identifier by typing the rigging identifier into the user interface, such as by ty ping in a rigging number, name, or other like identifier.

[0116] Additionally, or alternatively, the selection of one or more rigging identifiers by capturing a rigging identifier associated with a rigging wi th one or more sensors 113. A rigging identifier from a rigging may be captured in several ways. In some implementations, the one or more sensors 113 may be a camera and the camera may be utilized to take an image of the identifier (e.g., barcode, QR code, product number or name, rigging number or name, an image of the rigging or matenal itself, or the like).Page 25 of 40018300-902175

[0117] Additionally, or alternatively, the one or more sensors 113 may be a scanner, and a user may scan a barcode on a rigging. Similarly, capturing may occur by electronically communicating with the rigging identifier 602, such as through the use of RF scanning, near field communication, wireless communication, or the like wherein the rigging or product is equipped with an electronic identifier (e.g., a beacon or emitter that emits a radio frequency signal, other electronic signal, or other electronic communication) that can be detected by the user computer system 100 (and / or the entity computer system 200 or material handler 402).

[0118] In some implementations, an image or scan of the rigging may be provided by the one or more sensors 113 to an application for machine-learning or artificial intelligencebased rigging recognition, where the rigging (and corresponding identification data) may be inferred based on previously trained models, image classification algorithms, and / or feature extraction techniques. These models may analyze visual characteristics, such as shape, size, texture, or color patterns, to match the rigging to known categories within a dataset. The inferred identification data may then be cross-referenced with a database to verify or improve recognition accuracy.

[0119] Regardless of how the rigging identifier may be obtained, the rigging identifier may be used to authorize that (i) the user (e.g., via the user identifier discussed previously) possesses the requisite qualifications for using the rigging, and / or (ii) that the rigging is compatible with the target object.

[0120] In some implementations, the rigging identifier of the rigging may be compared directly (e.g., via lookup in a database) to cross reference a predetermined list of user identifiers who are authorized to use the rigging. Additionally, or alternatively, the system may retrieve rigging information (i.e., identification data) corresponding to the rigging identifier for the rigging. To retrieve this identification data for the rigging, processing steps may include matching the rigging identifier for the rigging with identification data stored in the datastore 122, 222, 422. Relevant data of the rigging may be retrieved from the identification data (e.g., required qualifications or certifications, name or description of the rigging, or the like). Similarly, identification data for the user may be retrieved, whereinafter comparisons are performed (e.g., comparing the required certifications with certifications of the user retrieved from the identification data of the user, comparing the name or description of the rigging to names or descriptions of the rigging of which the user is permitted to handle, and so forth).

[0121] In some implementations, the rigging identifier of the rigging may be compared directly (e.g., via lookup in a database) to cross reference a predetermined list of Page 26 of 40018300-902175object identifiers for target objects that are compatible with the rigging (i.e., are within weight limits, requisite securing redundancies, or the like). Identification data for the object may be retrieved, whereinafter comparisons are performed (e.g., comparing the weight of the target object with the weight limit of the rigging, and so forth).

[0122] In some implementations, if one or more predetermined conditions are met (e.g., a rigging weight limit is above the weight of the target object, a match occurs between users authorized to use the rigging and the user, or the like as described in the foregoing), the process may continue such that the user and / or target object is considered authorized with respect to the rigging. Additionally, or alternatively, if predetermined conditions are not met, the process may be discontinued, such that the user is not able to proceed with any additional steps of the process or handling of the target object. In other words, the application may be locked. Additionally, or alternatively, the application may prompt the user for an override, such as by requiring credentials of an authorized user, supervisor, or other user with predetermined permissions.

[0123] Once the rigging is coupled to the material handling system, to couple the rigging to the target object, one or more portions of the movable support may be moved into position proximate the target object, either manually by a user using one or more interactive elements of the user interface, or automatically by controlling the movable support with rigging until the location data of the movable support with rigging is determined to be within a predetermined distance of the location data of the target object.

[0124] In some implementations, a target location (e.g., a location where the movable support with rigging is to be moved, or where object is to be moved, or any “second position” relative a “first position”) may be selected based on a location identifier. One example of a target location may be a material zone for storing materials, which may include racks, pallets, boxes, containers, floor markings, zones, or the like. However, target locations may also be locations outside of material zones.

[0125] In some implementations, the system 1 may receive a selection of one or more target locations in the layout rendering 131. For example, a user may select one or more location identifiers from the rendered layout rendering 131 provided in the interface on the display. The selection may be made by selecting the location identifier within the layout rendering 131 illustrated on the interface of the display, by selecting the location identifier from a list of locations or location identifiers provided through the interface on the display, or the like. In some implementations, the user may manually enter the location identifier byPage 27 of 40018300-902175typing the location identifier into the user interface, such as by typing in a location number, name, or other like identifier.

[0126] Additionally, or alternatively, the selection of one or more location identifiers by capturing a location identifier associated with an operating area with one or more sensors 113. A location identifier from an operating area may be captured in several ways. In some implementations, the one or more sensors 113 may be a camera and the camera may be utilized to take an image of the identifier (e.g., barcode. QR code, product number or name, location number or name, an image of the location or material itself, or the like).

[0127] Additionally, or alternatively, the one or more sensors 113 may be a scanner, and a user may scan a barcode on a location. Similarly, capturing may occur by electronically communicating with the location identifier, such as through the use of RF scanning, near field communication, wireless communication, or the like wherein the location or product is equipped with an electronic identifier (e.g., a beacon or emitter that emits a radio frequency signal, other electronic signal, or other electronic communication) that can be detected by the user computer system 100 (and / or the entity computer system 200 or material handler 402).

[0128] Additionally, or alternatively, the target location may be determined based on a selection of a material zone of the one or more material zones or a listed material zone from a computer system or a selection of a target object identifier according to the methods described herein, for example, to move the movable support with rigging to the target object, or in implementations where the target location is a destination of the target object after moving, to initiate the stacking of the target object on top of or adjacent to another target object. In other implementations, the target location may be provided by verbal identification an eye-tracking position provided by the eyes of a user, a digital path drawing (e.g., the end of a path drawn by a user on top of the layout rendering 131 ), or the like.

[0129] As described with respect to the selection and / or authorization of the user, the target object, and / or the rigging, the operation of the material handling system may be contingent on the authorization of the material handling system, the rigging, and / or the user being allowed to move towards the target location. For example, the material handling system and / or a particular rigging may not be able to access and / or used within a particular target location. As such, the operation of the material handling system may only be used when the target location is authorized with the user and / or the rigging.

[0130] It should be understood that while the selection and / or authorization of the user, the selection and / or authorization of the target object (or target location), the selection Page 28 of 40018300-902175and / or authorization of the rigging, and / or the selection and / or authorization of the target location has been described in a particular order, it should be understood that the selection and / or authorization may or may not occur in the particular order described. As such, in some embodiment the selection and / or authorization of these may occur in any order.

[0131] At block 708, the system 1 may determine a path to move the movable support with the rigging of the material handling device. To do so, location data of elements (e.g., objects, users, target area, rigging, walkway, obstacles, or the like) may be retrieved (and / or determined) and plotted in a two-dimensional or three-dimensional computing environment.

[0132] In some implementations, the system may determine the locations (i.e., location data) of one or more elements by generating a geolocation stamp of the user computing system 100 (or material handler 402 or entity computer system 200) at the time of receiving the identifier (e.g., scanning a barcode, beacon, or the like, as previously described) and associating said geolocation stamp with the element (e.g., user, rigging, object, target location, or the like) that is being prompted for the user to receive. The geolocation may be determined based on GPS coordinates, triangulation techniques, or other positioning systems. Additionally, or alternatively, the geolocation data may be combined with inertial measurement unit (IMU) data, Wi-Fi positioning, Bluetooth signals, or RFID tags to provide more precise positioning.

[0133] Additionally, or alternatively, electronic identifiers may be used to triangulate location data of element(s) based on signals received from sensors in order to determine multiple reference points. The triangulation process may determine distances or angles between the reference points to compute an accurate position of one or more elements(s).

[0134] Additionally, or alternatively, the location data of the element(s), may be predetermined, such as by programming coordinates into a database, stored in a memory associated with the computing system, or retrieved from an external source, such as a cloud server, GPS-based geolocation service, scans of the operating area, or the like.

[0135] For implementations that include one or more sensors such as a camera or other vision system, location data of elements may be ascertained by analyzing captured image data using computer vision techniques. The vision system may include depth-sensing methods, such as stereoscopic imaging, structured light, or time-of-flight sensors, that may provide three-dimensional positional data. Techniques such as triangulation and / or structure-from-motion may be used to reconstruct the spatial coordinates by identifying corresponding features across images from different cameras. The system may further refine the locationPage 29 of 40018300-902175data by applying computer vision algorithms, including feature matching and / or keypoint detection, to determine location data of the element(s).

[0136] Once the location data has been determined, it shall be appreciated that the result may be one or more elements (objects, users, target area, rigging, walkway, obstacles, one or more material zones, structure configuration, or the like) located within a two-dimensional or three-dimensional computing environment, with identification data associated with the one or more elements within the two-dimensional or three dimensional computing environment.

[0137] In some implementations, a machine learning model may facilitate path determination between a first position and a second position while avoiding one or more elements along the path. The model, once trained on various spatial operating areas and element distributions, may generate a path based on input parameters, such as the coordinates of the first position, the second position, and / or the locations of elements.

[0138] In some implementations, the first position may be selected to correspond to location data previously determined. For example, the first position may be the current location data of the rigging, such as coordinates obtained from sensors affixed to the rigging or from sensors located in the operating area monitoring such rigging. Alternatively, the first position may be the current location data of one or more portions of the material handler, such as the movable support or from sensors located in the operating area monitoring such moveable support. Alternatively, the first position may be the current location data of the target object. Alternatively, the first position may be predetermined by selecting the first position on the layout rendering displayed via the user interface such that the user may specify the first position.

[0139] In some implementations, the second position may be the target location, with location data determined or received in a manner previously described in detail herein.

[0140] The machine learning model may produce a single optimal route, such as the shortest distance, by calculating the most efficient and / or safest trajectory' that bypasses elements that may be obstacles within the operating area (e.g., avoiding people, equipment, or the like). Alternatively’, the machine learning model may generate multiple viable paths to allow the user to select based on additional criteria such as traversal time and / or energy' efficiency, or other considerations.

[0141] The machine learning model may also integrate with a library of predetermined routes. The machine learning model may evaluate the available routes and select one or more suitable paths based on the initial conditions such as the first position, the Page 30 of 40018300-902175second position, and the locations of elements in the operating area. For example, if one of the predetermined routes intersects with an obstructing element, the model may disregard that path and consider alternative routes that avoid the element, thus mitigating collision risks.

[0142] In some implementations, path determination may occur without the use of a machine learning model. In such implementations, classical pathfmding algorithms may determine paths between the first and second positions. These algorithms may operate based on predefined heuristics, such as shortest distance, avoidance of walkways, equipment, or the like. In other embodiments, instead of or in addition to the machine learning and / or classical algorithms, neural networks, deep learning, artificial intelligence, generative artificial intelligence, or the like may be utilized in order to determine the path for the moveable support and / or rigging of the material handling apparatus, which may be individually or collectively described as automation.

[0143] In some implementations, the path determination process may include target object identification data such that the automation model considers parameters such as the object’s size, weight, or form factor, as recorded in the identification data for the object. For example, if the object is large or heavy, the model may avoid narrow corridors, low-clearance pathways, or fragile surfaces. Conversely, smaller or lighter objects may traverse more confined or less robust routes.

[0144] In some implementations, the location data of block 708 may be determined at a predetermined interval. In this way, movement of elements may be tracked in real-time (e.g., real or near-real time). In some implementations, elements that are in motion at a faster speed may have their corresponding location data determined at a higher frequency (i.e., shorter interval) than elements that are in motion at a slower speed.

[0145] Accordingly, the automation model may take into consideration updated location data, for example to determine the path the most up-to-date path at a time at which the material handler is instructed by the user to proceed with moving the movable support with the rigging (and in some implementations, the target object) from the first position to the second position.

[0146] At block 710. the movable support with the rigging may move from the first position to the second position according to the path determined at block 708. Such movement may be initiated by interaction with an element of the user interface, and / or activation of a mechanical device (e.g., a “dead man’s switch’’, foot pedal, toggle switch, or the like). As such, in some embodiments a user may be required to allow the movement toPage 31 of 40018300-902175continue by maintaining the selection (e.g., of a mechanical, digital, or the like button) for provide an additional safety feature for the material handling system.

[0147] In some implementations, the movement characteristics of the movable support with the rigging may be modified while following the path. For example, the user interface, mechanical device, or the like, may include a position indicator (e.g., a potentiometer, encoder, proximity sensor, or the like) such that v ary i ng positions of the user interface, mechanical device, or the like may cause the movable support with the rigging to speed up, slow down, stop, reverse, and so forth.

[0148] In some implementations, the movement of the movable support with the rigging may be controlled while moving along the path to maintain a predetermined distance between the target object and a user, calculated as location data differences between the two. In this way, a user may walk alongside of, in parallel to, and so forth with the target object while it is being moved from the first position to the second position.

[0149] At block 712, the path itself may be modified based on location data updated in real-time (e.g., while moving the movable support with the rigging along the path). This may occur in addition to, or instead of, any modifications to the speed, direction, or the like of the movable support with the rigging while it is traveling on the path.

[0150] To do so, the automation model may determine a path at a predetermined interval (the same or different than that of the predetermined interval for determining location data), such that a path is determined while the movable support with the rigging (and in some implementations, the target object) is moving from the first position to the second position.

[0151] In some implementations, the system may compare location data of elements across two or more distinct time intervals to calculate parameters such as velocity, acceleration, or direction. These parameters may be expressed as vectors by using one or both of magnitude and direction components. The system may further analyze these vectors to determine trajectories of elements in motion. The machine learning model may utilize this trajectory' data to predict future positions or paths of the elements. Accordingly, the automation model may avoid generating a path or determining a trajectory that intersects or conflicts with one or more predicted trajectories of the elements.

[0152] In some implementations, the system may generate an alert upon a condition where the location data of one or more of the elements is within a predetermined distance from one or more other elements, the movable support with the rigging, or the like. Such an alert may be displayed graphically on the user interface, trigger an audible alarm, buzzer, orPage 32 of 40018300-902175the like, and may automatically adjust (e.g., stop, change, or the like) the movement of the movable support with the rigging.

[0153] Once the moveable support with rigging has reached the second position, to decouple the rigging from the target object, one or more portions of the movable support may be moved to a position that facilitates the release of the target object, either manually by a user using one or more interactive elements of the user interface, or automatically by controlling the movable support with rigging. Once the moveable support with rigging reaches the appropriate position, the system may execute the decoupling procedure, such as releasing tension, disengaging mechanical locks, or retracting connectors, to separate the rigging from the target object.

[0154] It should be understood that the location of elements may be logged (e.g., recorded and / or stored) for future recall. The logging of these elements may be performed automatically and / or manually by a user. For example, while a material zone may be provided in the layout rendering 131, objects may not always end up in the material zone. Moreover, in some implementations a material zone may be relatively large and filled with hundreds of objects. As such, the ability to log the location of elements may allow users to quickly identify their location when needed to be recalled in the future. For example, a user may log a location of an object manually by standing near the object and selecting a log feature within the material handling application for a particular object and store the current location of the user computer system 100 through the use of a location determining device (e.g., GPS, or the like) in the user computer system 100. Alternatively, once an object has been moved along the path to the target location on the operating area, an electronic device associated with the object or within a user computer system 100 may automatically log the location of the object.

[0155] It should be understood that the location determining device (otherwise described as positioning determining device) may log the position of the user computer system 100 and / or the element within yards, feet, inches, or the like. As such, when a user would like to identify the location of an element, for example, the user may select the object in the layout rendering 131, or in a list, within the material handling application, and the material handling application may display the location of the element. The location of the element may be displayed in the layout rendering 131 and / or the material handling application may indicate their locations (e.g., provide the direction and distance) relative to the location of the user, such that the user is able to identify their relative locations. The system may also direct the user to the location of the element by illustrating in the layout Page 33 of 40018300-902175rendering 131 the movement of the user and / or the change in direction and / or distance as the user tries to find it based on the logged location. In other implementations, should the element have its own location determining device, the movements thereof may be displayed in the layout rendering 131 and / or logged by the material handling application as it is moved to different locations.

[0156] Moreover, in some implementations, the material handling application may create reports (e.g.. automatically and / or as requested by the user). The reports may be created and sent between the users. The reports may illustrate the statuses of the elements, such as but not limited to a list of the objects that were handled, moved from a first position to a second position, a list of the objects at a given target location, a list of users who participated in material handling and / or the objects such users handled, or the like.Moreover, in some implementations of the disclosure, the material handling application may allow- users to set targets (e.g., object handling completion percentage, quantities, or the like based on dates or time periods) for various activities. The reports may illustrate if an activity is behind the schedule and / or targets. The reports may be stored and / or compared over time and recalled as needed, and / or shared directly, through the cloud, or the like.

[0157] It shall be understood that, w ile some of the foregoing has been described with respect to moving a target object from a first position to a second position, the process may be implemented for use of any movement of the moveable support with rigging between a first position and a second position, with or without any rigging or target object. Indeed, the disclosure provides for an improvement to current methods of tracking elements within an operating area and provides for methods for determining paths of any movement of a material handler therein. Similarly, while the determining of identification data, and subsequent authorization of the various combinations of users, rigging, target objects, or the like has been described herein, some implementations of the disclosure may not include such processes and may instead embrace element tracking and / or path determination only. Moreover, other implementations of the disclosure may embrace the authorization of various combinations of the users, rigging, target objects, or the like, and implement other methods of moving a material handler within the operating environment.

[0158] It will be understood that, where possible, any of the advantages, features, functions, devices, and / or operational aspects of any of the implementations of the present disclosure described and / or contemplated herein may be included in any of the other implementations of the present disclosure described and / or contemplated herein, and / or vice versa. In addition, where possible, any terms expressed in the singular form herein are meant Page 34 of 40018300-902175to also include the plural form and / or vice versa, unless explicitly state otherwise.Accordingly, the terms 'a" and / or “an” shall mean “one or more.”

[0159] While certain exemplary implementations have been described and shown in the accompanying drawings, it is to be understood that such implementations are merely illustrative of and not restrictive on the broad disclosure, and that this disclosure not be limited to the specific constructions and arrangements shown and described, since various other changes, combinations, omissions, modifications and substitutions, in addition to those set forth in the above paragraphs are possible. Those skilled in the art will appreciate that various adaptations, modifications, and combinations of the just described implementations can be configured without departing from the scope and spirit of the disclosure. Therefore, it is to be understood that, within the scope of the appended claims, the disclosure may be practiced other than as specifically described herein.Page 35 of 40018300-902175

Claims

WHAT IS CLAIMED IS:

1. A system for autonomous material handling, the system comprising:a material handler comprising:a movable support; andrigging operatively coupled to the movable support configured for coupling with one or more target objects; andone or more sensors operatively coupled to the material handler;an operator system operatively coupled to the material handler, wherein the operator system comprises:one or more memories having computer readable code stored thereon; and one or more processors operatively coupled to the one or more memories, wherein when executed the computer readable code is configured to cause the one or more processors to:receive a target object based on an object identifier of the target object to be retrieved by the material handler; andmove the movable support of the material handler based on the one or more sensors to the target object based at least on the one or more sensors.

2. The system of claim 1, wherein when executed the computer readable code is further configured to cause the one or more processors to:receive a user identifier of a user;determine authorization for the user for the material handler based on the user identifier; andauthorize operation of the material handler when the user is authonzed.

3. The system of claim 2, wherein the user identifier for the user is received from a user computer system sensing a user token of the user, identify ing a user image, receiving wireless communication from a user beacon, receiving at least one biometric feature, receiving credentials, or receiving authentication via a computer system.

4. The system of claim 1, wherein when executed the computer readable code is further configured to cause the one or more processors to:receive a rigging identifier for the rigging;determine when the rigging is authorized for use for the target object; and authorize operation of the material handler when the rigging is authorized.

5. The system of claim 4, wherein the rigging identifier for the rigging is received from a user computer system sensing a rigging token of the rigging, identifying a rigging image of the rigging, receiving wireless communication from a rigging beacon, receiving a selection from a rigging list, reading a marking on the rigging, or machine learning-based rigging recognition.

6. The system of claim 1, wherein moving the movable support of the material handler comprises:receiving a location identifier of the movable support with the rigging; determining a path to move the movable support with the rigging from a first position to a second position based on determining location data of one or more material zones, structure configuration, or movable objects based on sensor information received from one or more sensors;moving the movable support with the rigging along the path; andmodifying movement of the movable support with the rigging based on determining the location data while moving the movable support with the rigging along the path based on the sensor information.

7. The system of claim 6, wherein the one or more sensors comprise an imaging device, and wherein the location data is inferred from rendering data received from at least one imaging device.

8. The system of claim 7, wherein the one or more sensors are operatively coupled to the movable support, the rigging, or a structure with visibility to the material handler.

9. The system of claim 6. wherein the one or more sensors comprises an encoder, accelerometer, IR sensor, a camera, a NFC sensor, or RF sensor.

10. The system of claim 6, wherein the one or more movable objects comprise one or more persons or one or more dynamic equipment, and wherein modifying the movement of the movable support comprises:Page 37 of 40018300-902175determining the location data of the one or more persons or the one or more dynamic equipment; andmodifying the movement of the movable support with the rigging by stopping, speeding up, slowing down, or modifying the path of the movable support with the rigging.

11. The system of claim 6, wherein the first position comprises a current position of the movable support with the rigging, and wherein the second position comprises a position adjacent the target object.

12. The system of claim 6, wherein the second position is determined based on a selection of a material zone of the one or more material zones or a listed material zone from a user computer system, a selection of the target object or a listed target object from the user computer system, a sensing of a target object token, a verbal identification, an eye-tracking position, or a digital path drawing.

13. The system of claim 6, wherein determining the path to move the movable support with the rigging from the first position to the second position based on the location data comprises:retrieving size or weight parameters of the target object;determining the path between the first position to the second position based on the size or weight parameters of the target object and the location data of the one or more material zones, the structure configuration, or the movable objects.

14. The system of claim 13, wherein determining the path implements a machine learning model or artificial intelligence.

15. The system of claim 6, wherein modifying the movement of the movable support with the rigging occurs at a predetermined interval based on the location data determined from the location data while moving the movable support with the rigging along the path at the predetermined interval.

16. The system of claim 15, wherein the predetermined interval is shortened or lengthened w hen the movement of the movable support with the rigging speeds up, slows down, or stops.Page 38 of 40018300-90217517. The system of claim 6, moving the movable support of the material handler further comprises:generating an alert where, based on the one or more sensors determining the location data of the one or more material zones, the structure configuration, or the movable objects, the rigging or the material handler is within a predetermined distance from the one or more material zones, the structure configuration, or the movable objects.

18. A computer implemented method for autonomous material handling, the method comprising:receiving a target object based on an object identifier of the target object to be retrieved by a material handler, wherein the material handler comprises:a movable support; andrigging operatively coupled to the movable support configured for coupling with one or more target objects; andone or more sensors operatively coupled to the material handler; andmoving the movable support of the material handler based on the one or more sensors to the target object based at least on the one or more sensors.

19. A computer program product for autonomous material handling, the computer program product comprising at least one non-transitory computer-readable medium having computer-readable program code portions embodied therein, the computer-readable program code portions comprising:an executable portion configured to receive a target object based on an object identifier of the target object to be retrieved by a material handler, wherein the material handler comprises:a movable support; andrigging operatively coupled to the movable support configured for coupling with one or more target objects; andone or more sensors operatively coupled to the material handler; andan executable portion configured to move the movable support of the material handler based on the one or more sensors to the target object based at least on the one or more sensors.Page 39 of 40018300-902175

Images