Automated material handling system

Abstract

The embodiment of the utility model provides an automatic material carrying system. The automatic material carrying system comprises a storage carrier and a shelf state monitoring device. The storage carrier is configured to perform a storage operation associated with the shelf unit, the storage operation including a placement operation including transferring the product unit from a load support assembly of the storage carrier to a storage location above the shelf unit, or a retrieval operation including transferring the product unit from the load support assembly to a storage location above the shelf unit. The retrieval operation includes transferring a product unit from a storage location above the shelf unit to the load support assembly storing the carrier. A shelf condition monitoring apparatus includes an image sensor coupled to a storage vehicle, the image sensor configured to capture an image of a view associated with a predetermined position of a shelf unit during a storage operation, and a computer configured to determine whether the shelf unit is associated with a potential displacement event based on the image. Shelf units are monitored during storage operations using a shelf condition monitoring device to automatically identify potentially damaged shelf units.

Description

Technical Field

[0001] Embodiments of the present utility model relate to a semiconductor system, and more particularly to an automated material handling system. Background Art

[0002] Semiconductor devices are formed on, in, and / or from semiconductor wafers and are used in a variety of electronic devices such as mobile phones, laptop computers, desktop computers, tablet computers, watches, gaming systems, and various other industrial, commercial, and consumer electronic components. One or more semiconductor manufacturing processes are performed to form semiconductor devices on, in, and / or from semiconductor wafers. During the time between semiconductor manufacturing processes, the semiconductor wafers are stored in a wafer storage device. Summary of the Utility Model

[0003] Embodiments of the present utility model provide an automated material handling system including a storage carrier and a shelf status monitoring device. The storage carrier is configured to perform a first storage operation associated with a first shelf unit, where the first storage operation includes a placement operation or a retrieval operation. The placement operation includes transferring a first product unit from a load support assembly of the storage carrier to a first storage position above the first shelf unit, and the retrieval operation includes transferring the first product unit from the first storage position above the first shelf unit to the load support assembly of the storage carrier. The shelf status monitoring device includes a first image sensor and a computer. The first image sensor is coupled to the storage carrier, where the first image sensor is configured to capture a first image of a view associated with a first predetermined position of the first shelf unit during the first storage operation, and the computer is configured to determine whether the first shelf unit is associated with a potential displacement event based on the first image.

[0004] Based on the above, the shelf status monitoring device is used to monitor the shelf unit during the storage operation to automatically identify potentially damaged shelf units. An inspection device is used to automatically inspect the shelf units marked by the shelf status monitoring device. In response to a determination that the determination that the shelf unit is associated with a potential displacement event is valid, the use of the shelf unit is blocked, such that the storage controller does not use the shelf unit to store product units based on the shelf unit being included in a list of blocked shelf units.

[0005] To make the above features and advantages of the embodiments of the present utility model more obvious and understandable, specific embodiments are hereinafter given and described in detail in conjunction with the accompanying drawings as follows. Description of the Drawings

[0006] Figure 1ASchematic diagram showing a first product unit transferred from a first position to a load support assembly of a storage vehicle.

[0007] Figure 1B Perspective view showing one or more components of an automated material handling system performing a storage operation according to some embodiments.

[0008] Figure 1C Perspective view showing one or more components of an automated material handling system performing a storage operation according to some embodiments.

[0009] Figure 1D Representation of an image captured during a storage operation according to some embodiments.

[0010] Figure 1E Perspective view showing one or more components of an automated material handling system performing a storage operation according to some embodiments.

[0011] Figure 1F Perspective view showing one or more components of an automated material handling system performing a storage operation according to some embodiments.

[0012] Figure 1G Representation of an image captured during a storage operation according to some embodiments.

[0013] Figure 2A Perspective view of a shelf unit according to some embodiments.

[0014] Figure 2B Stopper of a shelf unit according to some embodiments.

[0015] Figure 3A Representation of an image according to some embodiments.

[0016] Figure 3B Representation of an image according to some embodiments.

[0017] Figure 3C Representation of an image according to some embodiments.

[0018] Figure 3D Representation of an image according to some embodiments.

[0019] Figure 3E Representation of an image according to some embodiments.

[0020] Figure 3F Representation of an image according to some embodiments.

[0021] Figure 3G Representation of an image according to some embodiments.

[0022] Figure 4A A perspective view showing at least a part of a shelf status monitoring device according to some embodiments.

[0023] Figure 4B A perspective view showing at least a part of a shelf status monitoring device according to some embodiments.

[0024] Figure 5 A perspective view showing at least a part of an inspection device according to some embodiments.

[0025] Figure 6A Shows a representation of an image according to some embodiments.

[0026] Figure 6B Shows a representation of an image according to some embodiments.

[0027] Figure 6C Shows a representation of an image according to some embodiments.

[0028] Figure 7A Shows a representation of two images according to some embodiments.

[0029] Figure 7B Shows a representation of two images according to some embodiments.

[0030] Figure 7C Shows a representation of two images according to some embodiments.

[0031] Figure 8 Is a schematic block diagram showing a situation associated with an automated material handling system according to some embodiments.

[0032] Figure 9 Is a flowchart showing a method according to some embodiments.

[0033] Figure 10A Shows a side view of a shelf unit associated with a potential shift event according to some embodiments.

[0034] Figure 10B Shows a side view of a shelf unit associated with a potential shift event according to some embodiments.

[0035] Figure 11 Shows an exemplary computer-readable medium according to some embodiments, which may contain processor-executable instructions configured to implement one or more of the provisions stated herein.

[0036] Description of reference numerals

[0037] 100: Automated Material Handling System; 102: Loading Port; 104A: First Product Unit; 104B, 104D, 104E: Product Units; 104C: Second Product Unit / Product Unit; 106: Load Support Assembly; 108: Storage Carrier; 110: First Light Generation Device; 112: Second Light Generation Device; 114: First Image Sensor; 116: Transfer; 120: Calibration Assembly; 122: Wall; 124, 124B, 124C, 124D, 124E, 124G, 124H, 124I, 1002, 1004: Shelf Unit; 124A: First Shelf Unit; 124F: Second Shelf Unit / Shelf Unit; 126: Track; 128: First Storage Location; 129: Second Direction; 130: First Direction; 132: Second Storage Location; 133: First Reflection Label; 134: First Image; 136, 146: Rectangle; 138: First Reflection; 140A, 140B, 140C, 140D, 150A, 150B, 150C, 150D: Line; 142, 152: Region; 143: Second Reflection Label; 144: Second Image; 148: Second Reflection; 202, 204, 208, 210: Stopper; 206: Body; 212: Attachment Mechanism; 222: Reflection Label; 224: First Part; 226: Second Part; 228: First Circular Opening / Opening; 230: Cross-shaped Opening / Opening; 232: Second Circular Opening / Opening; 234: First Length; 236: Second Length; 302, 304, 306, 308, 310, 312, 314: Image; 400: First Part; 402: First Communication Module; 404: First DC (DC) / DC Converter; 406: Terminal Board; 408: First Light Controller; 410: Frame; 412, 414: Light Source; 415: Mounting Assembly; 416: Second Image Sensor; 418: Third Image Sensor; 430: Board; 450: Second Part; 500: Inspection Equipment; 502: Second DC / DC Converter; 504: Second Communication Module; 506: Second Light Controller; 508: Fourth Image Sensor; 510: First Distance Sensor; 512: Fifth Image Sensor; 514: Second Distance Sensor; 602: Third Image; 604: First Opening; 606: Second Opening; 608: Third Opening; 610, 710: Reference Line; 612, 712: Reference Cross; 702: Fourth Image; 704: Fourth Opening; 706: Fifth Opening; 708: Sixth Opening; 800: Situation; 802: Storage Controller; 804: Shelf Status Monitoring Equipment; 805: Computer; 806: Position Detection; 807: Movement; 808: Image Capture / Analysis; 812: Database; 814: Mark; 820: Trigger; 822: Scan; 824: Analysis; 826: Release; 828: Repair; 830: Block; 900: Method;902, 904, 906: Operations; 1000, 1050: Representations; 1100: Embodiment; 1102: Method; 1104: Processor-executable computer instructions; 1106: Computer-readable data; 1108: Computer-readable medium. Detailed implementation

[0038] The following disclosure provides numerous different embodiments or examples for implementing different features of the present disclosure. Specific examples of components and arrangements are set forth below to simplify the present disclosure. Of course, these are only examples and are not intended to limit the scope of the present disclosure. For example, in the following description, the formation of a first feature "on" or "above" a second feature may include embodiments in which the first feature and the second feature are formed in direct contact, and may also include embodiments in which additional features are formed between the first feature and the second feature such that the first feature and the second feature are not in direct contact. Additionally, the present disclosure may reuse component numbers and / or letters in various examples. Such reuse is for the purpose of simplifying and clearly describing the present disclosure and is not intended to limit the relationship between various embodiments and / or configurations.

[0039] Furthermore, for ease of description, spatially relative terms such as "beneath", "below", "lower", "above", "upper", etc. may be used herein to describe the relationship of one component or feature shown in the figures to another (other) component or feature. In addition to the orientation depicted in the figures, the spatially relative terms also encompass different orientations of the device during use or operation. The device may have other orientations (rotated 90 degrees or at other orientations), and the spatially relative descriptive terms used may be interpreted in the same manner.

[0040] According to some embodiments, an automated material handling system uses a shelving unit, such as a wafer storage device, to store product units. The automated material handling system includes a shelf status monitoring apparatus configured to monitor the shelving unit of the automated material handling system during a stocking operation. The shelf status monitoring apparatus includes an image sensor coupled to a stocking vehicle. The stocking vehicle performs a stocking operation to load product units onto or unload product units from the shelving unit. During the stocking operation, the image sensor captures an image associated with a predetermined position of the shelving unit. The image is used to determine whether the shelving unit is associated with a potential displacement event corresponding to a displacement of the shelving unit from the predetermined position to a different position. In some embodiments, the shelving unit is marked out of service in response to determining that the shelving unit is associated with a potential displacement event.

[0041] In some embodiments, in response to marking the shelving unit out of service, an inspection is triggered to determine the validity of the determination that the shelving unit is associated with a potential displacement event. The inspection is performed using an inspection device. The inspection includes: capturing a second image of the shelving unit; and determining based on the second image whether the shelving unit is damaged. In some embodiments, a first portion of the shelving unit is depicted in the first image captured using the shelf status monitoring apparatus and a second portion of the shelving unit is depicted in the second image captured using the inspection device. In some embodiments, one or more maintenance operations are performed to repair the shelving unit in response to determining via the inspection that the shelving unit is damaged. In some embodiments, an automated maintenance machine is used to repair the shelving unit in response to the inspection.

[0042] Thus, according to some embodiments, the shelf status monitoring apparatus is used to monitor the shelving unit during the stocking operation to automatically identify potentially damaged shelving units. According to some embodiments, an inspection device is used to automatically inspect the shelving units marked by the shelf status monitoring apparatus. In some embodiments, the shelving unit is repaired in response to the inspection device confirming that the shelving unit is damaged, such as by using an automated maintenance machine to automatically repair the shelving unit.

[0043] Figures 1A to 1GIllustrates aspects associated with an automated material handling system 100 for transporting and / or storing product units. In some embodiments, the automated material handling system 100 includes at least one of a storage vehicle 108, a shelf status monitoring device, or a storage system (the storage system includes a plurality of shelf units for storing product units). In some embodiments, at least a portion of the shelf status monitoring device is coupled to the storage vehicle 108. In some embodiments, the shelf status monitoring device is configured to monitor the shelf units of the storage system during a storage operation performed using the storage vehicle 108.

[0044] In Figures 1A to 1D Aspects of a first storage operation performed using the storage vehicle 108 are illustrated. The first storage operation includes: a first placement operation, in which a first product unit 104A is subjected to at least one of the following operations: (i) being transferred from a first position to the load support assembly 106 of the storage vehicle 108; or (ii) being transferred from the load support assembly 106 of the storage vehicle 108 to a first storage position 128 above a first shelf unit 124A (illustrated in Figures 1B to 1C and Figures 1E to 1F ).

[0045] Figure 1A A schematic diagram of a first product unit 104A being transferred 116 from a first position to the load support assembly 106 of the storage vehicle 108 is illustrated. In some embodiments, the first product unit 104A includes a wafer storage device. In some embodiments, the wafer storage device includes at least one of a cassette pod, a front opening unified pod (FOUP), a reticle pod, or other types of wafer storage devices. In some embodiments, the wafer storage device is used to store one or more wafers, such as a batch of wafers. In some embodiments, the one or more wafers are stacked in a vertical direction within the wafer storage device. In some embodiments, the one or more wafers are supported by a support frame of the wafer storage device, the support frame having at least one of a wafer shelf or a wafer slot. In some embodiments, the wafers stored in the wafer storage device include one or more layers, such as at least one of a semiconductor layer, a conductor layer, or an insulator layer.

[0046] In some embodiments, the first position corresponds to the position of the loading port 102 of the processing machine. In some embodiments, the processing machine includes at least one of the following equipment: (i) physical vapor deposition (PVD) equipment; (ii) chemical vapor deposition (CVD) equipment; (iii) plating equipment; (iv) etching equipment, such as at least one of plasma etching equipment, wet etching equipment, dry etching equipment, reactive-ion etching (RIE) equipment, atomic layer etching (ALE) equipment, buffered oxide etch equipment, or ion beam milling equipment; (v) lithography equipment; (vi) chemical mechanical planarization (CMP) equipment; or (vii) other equipment.

[0047] In some embodiments, a first storage operation is performed in response to completion of a first process (such as a semiconductor manufacturing process) performed on the one or more wafers stored in the first product unit 104A (such as a wafer storage device). In some embodiments, the first process includes at least one of the following processes: (i) a PVD process; (ii) a CVD process; (iii) a plating process; (iv) an etching process, such as at least one of a plasma etching process, a wet etching process, or a dry etching process; (v) a lithography process; (vi) a CMP process; or (vii) one or more other suitable processes.

[0048] In some embodiments, when the first product unit 104A is docked to the loading port 102, the wafers stored in the first product unit 104A are unloaded from the first product unit 104A and inserted into the processing machine. In some embodiments, after the wafers are inserted into the processing machine, a first process is performed on the wafers using the processing machine. In some embodiments, in response to completion of the first process on the wafers, the wafers are removed from the processing machine and loaded into the first product unit 104A (e.g., a wafer storage device). In some embodiments, multiple wafers are processed at a time using the processing machine. In some embodiments, a single wafer is processed at a time using the processing machine. In some embodiments, a first process is performed on each of one, some, and / or all of the wafers stored in the first product unit 104A using the processing machine to generate a set of one or more processed wafers, and the processed set of one or more wafers is loaded into the first product unit 104A. In some embodiments, after the processed set of one or more wafers is loaded into the first product unit 104A, the first product unit 104A is transferred 116 from the loading port 102 to the load support assembly 106 of the storage carrier 108. In some embodiments, a mechanical device (not shown) is used to transfer 116 the first product unit 104A from a first position (e.g., the loading port 102) to the load support assembly 106.

[0049] In some embodiments, the storage carrier 108 includes at least one of a forklift, a crane, a crane fork, an overhead transporter, a guided transporter traveling on a predetermined route or track, or other suitable vehicle. In some embodiments, the load support assembly 106 corresponds to a platform for supporting a load (e.g., the first product unit 104A) during movement of the storage carrier 108. In some embodiments, the load support assembly 106 includes the forks of the storage carrier 108 (e.g., a forklift or a crane fork). In some embodiments, the storage carrier 108 controls the vertical position of the load support assembly 106 (e.g., by raising or lowering the load support assembly 106) to access load ports and / or shelf units at different elevation levels.

[0050] In some embodiments, after the first product unit 104A is transferred to the load support assembly 106, the storage carrier 108 moves to a first position (e.g., the position of the storage carrier 108 shown in Figures 1B to 1C ) near the first shelf unit 124A to transfer the first product unit 104A from the load support assembly 106 to a first storage position 128 above the first shelf unit 124A (shown in Figures 1B to 1C ). Figures 1B to 1CA perspective view showing the transfer of the first product unit 104A from the load support assembly 106 to the first storage location 128 above the first shelf unit 124A. In some embodiments, the storage carrier 108 is moved to the first position using the track 126. In some embodiments, the track 126 guides the movement of the storage carrier 108. In some embodiments, at least one of the first shelf unit 124A or other shelf units 124B, 124C, 124D, 124E, 124F, 124G, 124H, and / or 124I is fixed to the wall 122, such as a storage partition wall of the storage system. Figure 1B Shows the first product unit 104A disposed on the load support assembly 106A before the transfer of the first product unit 104A from the load support assembly 106 to the first storage location 128 above the first shelf unit 124A. In some embodiments, the product units 104B, 104C, 104D, and 104E are respectively stored in storage locations above the shelf units 124C, 124F, 124G, and 124I. Figure 1C Shows the first product unit 104A stored in the first storage location 128 above the first shelf unit 124A after the transfer of the first product unit 104A to the first storage location 128. In some embodiments, a mechanical device (not shown) is used to transfer the first product unit 104A from the load support assembly 106 to the first storage location 128. In some embodiments, when the first product unit 104A is in the first storage location 128, the first product unit 104A is supported by the first shelf unit 124A. In some embodiments, when the first product unit 104A is in the first storage location 128, the first product unit 104A is set on and / or in direct contact with the first shelf unit 124A. In some embodiments, when the first product unit 104A is in the first storage location 128, the first shelf unit 124A is set on and / or in direct contact with at least one of the layers, objects, etc. above the first shelf unit 124A. In some embodiments, when the first product unit 104A is in the first storage location 128, at least one of the layers, objects, etc. separates the first shelf unit 124A from the first product unit 104A.

[0051] In some embodiments, the shelf status monitoring device includes a first image sensor 114, such as a camera, configured to capture a first image associated with the first shelf unit 124A. In some embodiments, the first image sensor 114 captures the first image during a first storage operation. In some embodiments, the first storage operation spans from a first time to a second time. In some embodiments, the first time corresponds to the time when (or before or after) the first product unit 104A is transferred 116 from a first position to the load support assembly 106 of the storage vehicle 108. In some embodiments, the second time corresponds to the time when (or before or after) the first product unit 104A is transferred from the load support assembly 106 of the storage vehicle 108 to a first storage position 128 above the first shelf unit 124A. In some embodiments, the view of the first image sensor 114 changes with the position of at least one of the load support assembly 106 or the storage vehicle 108. In some embodiments, the first image sensor 114 has a constant position relative to at least one of the load support assembly 106 or the storage vehicle 108 to have a changing view during the movement of at least one of the load support assembly 106 or the storage vehicle 108. In some embodiments, a mechanical device is used to adjust the position of the first image sensor 114 relative to at least one of the load support assembly 106 or the storage vehicle 108.

[0052] In some embodiments, the first image is mapped to an image of a first view associated with a first predetermined position of the first shelf unit 124A. In some embodiments, the first predetermined position corresponds to a proper position of the first shelf unit 124A. In some embodiments, during the first storage operation, the first image is captured when the first image sensor 114 has a first view of the first predetermined position of the first shelf unit 124A. In some embodiments, the shelf state monitoring device determines that the first image sensor 114 has a first view of the first predetermined position of the first shelf unit 124A based on a match between the current position of the first component and a first reference position corresponding to the first image sensor 114 having a first view of the first predetermined position. In some embodiments, the first component includes at least one of the first image sensor 114, the load support component 106, or other components coupled to the storage vehicle 108. In some embodiments, the position determination module of the shelf state monitoring device is used to determine the current position. In some embodiments, the position determination module is calibrated using at least one of one or more calibration components 120 or one or more predetermined positions. In some embodiments, a calibration component among the one or more calibration components 120 is coupled to the wall 122. In some embodiments, calibrating the position determination module includes at least one of the following operations: (i) capturing an image of a view of the calibration component 120 associated with the first predetermined position of the first component; or (ii) mapping the first predetermined position associated with the calibration component 120 to the position of the first component when capturing an image of a view of the calibration component 120. In some embodiments, when calibrating the position determination module of the shelf state monitoring device, the position determination module is capable of determining when the first component has the first reference position. In some embodiments, the first image is captured when the first component has the first reference position such that the first image has a first view of the first predetermined position associated with the first shelf unit 124A.

[0053] Figure 1DShows a representation of a first image (shown as reference numeral 134) according to some embodiments. In some embodiments, the first image 134 is used to determine whether the first shelf unit 124A is associated with a first potential displacement event (e.g., using one or more of the techniques provided below). In some embodiments, the shelf state monitoring device includes a computer configured to determine whether the first shelf unit 124A is associated with a first potential displacement event based on the first image 134. In some embodiments, the first potential displacement event corresponds to the event that the first shelf unit 124A is displaced from a first predetermined position to a different position, such as an improper position. In some embodiments, the first potential displacement event corresponds to the event that at least a portion of the first shelf unit 124A is bent, sagged, or damaged at least in part due to the impact and / or pressure of an object on the first shelf unit 124A. In some embodiments, the first potential displacement event corresponds to the event that the first shelf unit 124A is damaged during a storage operation, such as when loading a product unit into a first storage position 128 above the first shelf unit 124A or unloading a product unit from the first storage position 128 above the first shelf unit 124A, and at least one of the load support assembly 106, the storage vehicle 108, the product unit, etc. impacts the first shelf unit 124A and / or displaces the first shelf unit 124A.

[0054] Figures 1E to 1G Illustrates aspects of a second storage operation. The second storage operation includes a first acquisition operation in which at least one of the following operations is performed on the second product unit 104C: (i) transferring from a second storage position 132 above the second shelf unit 124F (shown in Figures 1B to 1C and Figures 1E to 1F shown) to the load support assembly 106 of the storage vehicle 108; or (ii) transferring from the load support assembly 106 of the storage vehicle 108 to a second position. In some embodiments, the second position includes a second loading port of a second processing machine.

[0055] In some embodiments, the second storage operation is performed after the first storage operation. In some embodiments, after the first storage operation, the storage vehicle 108 moves to a second position near the second shelf unit 124F (e.g., the position of the storage vehicle 108 shown in Figures 1E to 1F to transfer the second product unit 104C from the second storage position 132 to the load support assembly 106. In some embodiments, the storage vehicle 108 moves along the track 126 in a first direction 130 to move from a first position near the first shelf unit 124A (shown in Figures 1B to 1C shown) to a second position near the second shelf unit 124F (inFigures 1E to 1F (shown in). In some embodiments, the second shelf unit 124F is located at a different height level (e.g., a height level lower than the first shelf unit 124A) from the first shelf unit 124A. In some embodiments, the load support assembly 106 moves in the second direction 129 to move from a first height level associated with accessing the first storage location 128 above the first shelf unit 124A to a second height level associated with accessing the second storage location 132 above the second shelf unit 124F.

[0056] Figures 1E to 1F A perspective view showing the transfer of the second product unit 104C from the second storage location 132 above the second shelf unit 124F to the load support assembly 106. Figure 1E The second product unit 104C disposed in the second storage location 132 above the second shelf unit 124F is shown before being transferred from the second storage location 132 to the load support assembly 106. In some embodiments, when the second product unit 104C is located in the second storage location 132, the second product unit 104C is supported by the second shelf unit 124F. In some embodiments, when the second product unit 104C is located in the second storage location 132, the second product unit 104C is set on and / or in direct contact with the second shelf unit 124F. In some embodiments, when the second product unit 104C is located in the second storage location 132, the second shelf unit 124F is set on and / or in direct contact with at least one of the layers, objects, etc. above the second shelf unit 124F. In some embodiments, when the second product unit 104C is located in the second storage location 132, at least one of the layers, objects, etc. separates the second shelf unit 124F from the second product unit 104C.

[0057] Figure 1F The second product unit 104C disposed on the load support assembly 106 is shown after being transferred from the second storage location 132 to the load support assembly 106. In some embodiments, a mechanical device (not shown) is used to transfer the second product unit 104C from the second storage location 132 to the load support assembly 106.

[0058] In some embodiments, a first image sensor 114 is used to capture a second image associated with the second shelf unit 124F. In some embodiments, the first image sensor 114 captures the second image during a second storage operation. In some embodiments, the second storage operation lasts from a third time to a fourth time. In some embodiments, the third time corresponds to the time when (or before or after) the storage vehicle 108 starts moving from a first position near the first shelf unit 124A to a second position near the second shelf unit 124F. In some embodiments, the fourth time corresponds to the time when (or before or after) the second product unit 104C is transferred from a second storage position 132 above the second shelf unit 124F to the load support assembly 106 of the storage vehicle 108.

[0059] In some embodiments, the second image is mapped to an image of a second view associated with a second predetermined position of the second shelf unit 124F. In some embodiments, the second predetermined position corresponds to a proper position of the second shelf unit 124F. In some embodiments, during the second storage operation, the second image is captured once when the first image sensor 114 has a second view of the second predetermined position of the second shelf unit 124F. In some embodiments, the shelf state monitoring device determines that the first image sensor 114 has a second view of the second predetermined position of the second shelf unit 124F based on the matching of the current position of the first component with a second reference position corresponding to the first image sensor 114 having a second view of the second predetermined position. In some embodiments, the position determination module can determine when the first component has the second reference position. In some embodiments, the second image is captured when the first component has the second reference position, such that the second image has a second view of the second predetermined position associated with the second shelf unit 124F.

[0060] Figure 1GShows a representation of a second image (shown as reference numeral 144) according to some embodiments. In some embodiments, the second image 144 is used to determine whether the second shelf unit 124F is associated with a second potential shift event (e.g., using one or more of the techniques provided below). In some embodiments, the computer of the shelf state monitoring device is configured to determine whether the second shelf unit 124F is associated with a second potential shift event based on the second image 144. In some embodiments, the second potential shift event corresponds to the event that the second shelf unit 124F is shifted from a second predetermined position to a different position, such as an improper position. In some embodiments, the second potential shift event corresponds to the event that at least a portion of the second shelf unit 124F is bent, sagged, or damaged at least in part due to the impact and / or pressure of an object on the second shelf unit 124F. In some embodiments, the second potential shift event corresponds to the event that during a storage operation, the second shelf unit 124F is damaged, such as when a product unit is loaded into or unloaded from a second storage position 132 above the second shelf unit 124F, and at least one of the load support assembly 106, the storage carrier 108, the product unit, etc. impacts the second shelf unit 124F and / or displaces the second shelf unit 124F.

[0061] Figure 2A Shows a perspective view of a shelf unit 124 according to some embodiments. In some embodiments, the shelf unit 124 corresponds to at least one of the first shelf unit 124A, the second shelf unit 124F, or other shelf units of the storage system. In some embodiments, the shelf unit 124 is fixed to the wall 122 by one or more attachment mechanisms 212. In some embodiments, the one or more attachment mechanisms 212 include at least one of one or more shelf brackets, one or more bolts, one or more clips, or other suitable mechanisms for fixing the shelf unit 124 to the wall 122. In some embodiments, the shelf unit 124 includes a body 206.

[0062] In some embodiments, the shelf unit 124 includes one or more stoppers to perform at least one of the following operations: (i) support a product unit (not shown) disposed on the shelf unit 124; or (ii) prevent the product unit from falling off the shelf unit 124. In some embodiments, the stopper among the one or more stoppers protrudes upward from the body 206 of the shelf unit 124, so that the stopper provides resistance to the product unit disengaging from the shelf unit 124. In some embodiments, the one or more stoppers include at least one of stopper 208, stopper 210, stopper 202, or stopper 204.

[0063] Figure 2B FIG. 4 shows a stopper 208 according to some embodiments. In some embodiments, the stopper 208 includes at least one of a reflective label 222 in a first portion 224 of the stopper 208 or one or more openings in a second portion 226 of the stopper 208. In some embodiments, the reflective label 222 is fixed to the stopper 208, for example, by adhesively bonding the reflective label 222 to the stopper 208. In some embodiments, the reflective label 222 includes a sticker having an adhesive on its first surface. In some embodiments, the reflective label 222 includes a reflective material having at least a threshold reflectivity. In some embodiments, the reflective label 222 includes one or more markings (e.g., at least one of one or more lines, one or more shapes, etc.) printed on the reflective label 222. In some embodiments, the reflective label 222 has a rectangular shape, such as a square shape. In some embodiments, a first length 234 of a first side of the reflective label 222 is between about 1 millimeter and about 50 millimeters, such as about 3 millimeters. In some embodiments, a second length 236 of a second side of the reflective label 222 is between about 1 millimeter and about 50 millimeters, such as about 3 millimeters. In some embodiments, each side of the reflective label 222 has approximately the same length, such as a length of about 3 millimeters. In some embodiments, the one or more openings include at least one of opening 228, opening 230, or opening 232. Figure 2B In, the opening 228 is a first circular opening, the opening 230 is a cross-shaped opening, and the opening 232 is a second circular opening. Other shapes of the openings among the one or more openings are also within the scope of the present disclosure. In some embodiments, the stopper 210 has one feature, some features, and / or all features provided herein for the stopper 208.

[0064] Returning to Figure 1D, in some embodiments, when the first shelf unit 124A has a first predetermined position, the first view of the first image 134 corresponds to the first part of the first shelf unit 124A. For example, if the first shelf unit 124A has the first predetermined position and is not damaged and / or displaced during the first storage operation, the first part of the first shelf unit 124A is depicted in the first view. In some embodiments, when the first shelf unit 124A is damaged and / or displaced, at least a portion of the first part of the first shelf unit 124A is not depicted in the first image 134 (e.g., at least in part due to the displacement of the first part of the first shelf unit 124A relative to the predetermined position of the first part of the first shelf unit 124A). In some embodiments, when at least one of the situations where the first shelf unit 124A has the first predetermined position or the first shelf unit 124A is not damaged occurs, the predetermined position of the first part of the first shelf unit 124A corresponds to the position of the first part of the first shelf unit 124A. In some embodiments, the first part of the first shelf unit 124A includes at least a portion of the stopper of the first shelf unit 124A (e.g., Figure 2A the stoppers 208 and / or 210 shown in Figure 1D ). In some embodiments, the first part of the first shelf unit 124A includes the first reflective label 133 shown in the first image 134 shown in Figure 2B . In some embodiments, the first reflective label 133 corresponds to the reflective label 222 shown in

[0065] In some embodiments, it is determined whether the first shelf unit 124A is associated with a first potential shift event based on a comparison of the position of the first reflection 138 of the first light beam shown in the first image 134 with the position of the first landmark shown in the first image 134. The first reflection 138 of the first light beam corresponds to the reflection of the first light beam from the first reflection label 133 or from other areas of the first shelf unit 124A. In some embodiments, the first landmark corresponds to the first reflection label 133. In some embodiments, the first landmark includes one or more first markings of the first reflection label 133 shown in the first image 134. In some embodiments, the one or more first markings include at least one of line 140A, line 140B, line 140C, line 140D, or rectangle 136. In some embodiments, the one or more first markings have a color different from the background color of the first reflection label 133, such that the one or more first markings are visually obvious and / or visually distinguishable in the first image 134. In some embodiments, the one or more first markings form a grid that is used to determine, for example, whether the first shelf unit 124A is associated with a first potential shift event by determining the position of the first reflection 138 of the first light beam relative to the grid shown in the first image 134. In some embodiments, it is determined that the first shelf unit 124A is not associated with a first potential shift event based on the entirety of the first reflection 138 of the first light beam shown in the first image 134 being located within the rectangle 136. In some embodiments, it is determined that the first shelf unit 124A is not associated with a first potential shift event based on the entirety of the first reflection 138 of the first light beam shown in the first image 134 being located within the area 142 corresponding to the rectangle defined by lines 140A, 140B, 140C, and 140D. In some embodiments, the area 142 corresponds to the central area of the first reflection label 133. In some embodiments, it is determined that the first shelf unit 124A is not associated with a first potential shift event based on at least a threshold proportion of the first reflection 138 of the first light beam shown in the first image 134 being located within the rectangle 136. In some embodiments, it is determined that the first shelf unit 124A is not associated with a first potential shift event based on at least a threshold proportion of the first reflection 138 of the first light beam shown in the first image 134 being located within the area 142 defined by lines 140A, 140B, 140C, and 140D.

[0066] In some embodiments, the first light generating device 110 of the shelf state monitoring device is used (in Figures 1A to 1C and Figures 1E to 1FAs shown, a first light beam is generated. In some embodiments, the first light generating device 110 includes a laser generating device. In some embodiments, during the first storage operation, the first light generating device 110 emits a first light beam, such as a laser beam. In some embodiments, the position of the first light beam varies with the position of at least one of the load support assembly 106 or the storage carrier 108. In some embodiments, the first light generating device 110 has a constant position relative to at least one of the load support assembly 106 or the storage carrier 108. In some embodiments, a mechanical device is used to adjust the position of the first light generating device 110 relative to at least one of the load support assembly 106 or the storage carrier 108. In some embodiments, the first light generating device 110 emits a first light beam while the first image sensor 114 is used to capture the first image 134. In some embodiments, the first light generating device 110 is positioned such that when capturing the first image 134, it emits the first light beam towards a predetermined position of a first portion of the first shelf unit 124A (e.g., towards a region 142 (shown in Figure 1D as shown) defined by one or more first markings of the first reflection label 133 of the first reflection label 133) of a predetermined position. In some embodiments, the amplification value associated with generating the first light beam using an amplifier (such as an input amplifier) of the first light generating device 110 is at least approximately the first threshold amplification value. In some embodiments, the first threshold amplification value is between approximately 3,000 and approximately 5,000.

[0067] The following embodiments are contemplated: In the embodiments, one or more techniques other than the techniques provided herein are used to determine whether the first shelf unit 124A is associated with a first potential shift event, such as using one or more other types of sensors other than the first image sensor 114 (and / or also using one or more other types of sensors in addition to the first image sensor 114), using other types of landmarks and / or markings other than the landmarks and / or markings explicitly shown in the drawings (and / or also using other types of landmarks and / or markings in addition to the landmarks and / or markings explicitly shown in the drawings), etc.

[0068] Returning to Figure 1G, in some embodiments, when the second shelf unit 124F has a second predetermined position, the second view of the second image 144 corresponds to a first portion of the second shelf unit 124F. For example, if the second shelf unit 124F has the second predetermined position during the second storage operation and is not damaged and / or displaced, the first portion of the second shelf unit 124F is depicted in the second view. In some embodiments, when the second shelf unit 124F is damaged and / or displaced, at least a portion of the first portion of the second shelf unit 124F is not depicted in the second image 144 (e.g., at least in part due to the displacement of the first portion of the second shelf unit 124F relative to the predetermined position of the first portion of the second shelf unit 124F). In some embodiments, when at least one of the situations where the second shelf unit 124F has the second predetermined position or the second shelf unit 124F is not damaged occurs, the predetermined position of the first portion of the second shelf unit 124F corresponds to the position of the first portion of the second shelf unit 124F. In some embodiments, the first portion of the second shelf unit 124F includes at least a portion of a stopper of the second shelf unit 124F (e.g., Figure 2A the stoppers 208 and / or 210 shown in Figure 1G ). In some embodiments, the first portion of the second shelf unit 124F includes the second reflective label 143 shown in the second image 144 shown in Figure 2B . In some embodiments, the second reflective label 143 corresponds to the reflective label 222 shown in

[0069] In some embodiments, it is determined whether the second shelf unit 124F is associated with a second potential displacement event based on a comparison of a second reflection 148 of the second light beam shown in the second image 144 with a second landmark position shown in the second image 144. In some embodiments, the second landmark corresponds to the second reflection tag 143. In some embodiments, the second landmark includes one or more second markings of the second reflection tag 143 shown in the second image 144. In some embodiments, the one or more second markings include at least one of line 150A, line 150B, line 150C, line 150D, or rectangle 146. In some embodiments, the one or more second markings have a color different from the background color of the second reflection tag 143 such that the one or more second markings are visually obvious and / or visually distinguishable in the second image 144. In some embodiments, the one or more second markings form a grid that is used to determine, for example, whether the second shelf unit 124F is associated with a second potential displacement event by determining the position of the second reflection 148 of the second light beam relative to the grid shown in the second image 144. In some embodiments, it is determined that the second shelf unit 124F is associated with a second potential displacement event based on the entirety of the second reflection 148 of the second light beam shown in the second image 144 being outside of rectangle 146. In some embodiments, it is determined that the second shelf unit 124F is associated with a second potential displacement event based on the entirety of the second reflection 148 of the second light beam shown in the second image 144 being outside of region 152 corresponding to the rectangle defined by lines 150A, 150B, 150C, and 150D. In some embodiments, region 152 corresponds to the central region of the second reflection tag 143. In some embodiments, it is determined that the second shelf unit 124F is associated with a second potential displacement event based on at least a threshold proportion of the second reflection 148 of the second light beam shown in the second image 144 being outside of rectangle 146. In some embodiments, it is determined that the second shelf unit 124F is associated with a second potential displacement event based on at least a threshold proportion of the second reflection 148 of the second light beam shown in the second image 144 being outside of region 152 defined by lines 150A, 150B, 150C, and 150D.

[0070] In some embodiments, a first shelf health state associated with the second shelf unit 124F and the second potential shift event is determined based on the second image 144. In some embodiments, the first shelf health state is determined to be a first value, such as "warning", based on determining that the second image 144 meets one or more first conditions. In some embodiments, the first shelf health state is determined to be a second value, such as "damaged", based on determining that the second image 144 meets one or more second conditions. In some embodiments, the one or more second conditions are different from the one or more first conditions. In some embodiments, the first shelf health state is determined to be the first value based on at least a threshold proportion of a second reflection 148 of a second light beam shown in the second image 144 being outside a region 152 defined by lines 150A, 150B, 150C, and 150D. In some embodiments, the first shelf health state is determined to be the first value based on at least a threshold proportion of the second reflection 148 of the second light beam shown in the second image 144 being outside a rectangle 146.

[0071] Figures 3A to 3G Provide various exemplary scenarios for determining whether a shelf unit is associated with a potential shift event based on an image captured using the techniques provided herein. Figure 3A An exemplary representation of an image 302 captured using the techniques provided herein according to some embodiments is shown. In some embodiments, in a situation where one or more of the techniques provided herein are used, at least one of a storage controller or a computer of a shelf status monitoring device determines, based on the image 302, that the shelf unit associated with the image 302 is not associated with a potential shift event. Figure 3B An exemplary representation of an image 304 captured using the techniques provided herein according to some embodiments is shown. In some embodiments, in a situation where one or more of the techniques provided herein are used, at least one of a storage controller or a computer of a shelf status monitoring device determines, based on the image 304, at least one of the following: (i) the shelf unit associated with the image 304 is associated with a potential shift event; or (ii) the shelf health state associated with the shelf unit is a first value, such as "warning". Figure 3C An exemplary representation of an image 306 captured using the techniques provided herein according to some embodiments is shown. In some embodiments, in a situation where one or more of the techniques provided herein are used, at least one of a storage controller or a computer of a shelf status monitoring device determines, based on the image 306, at least one of the following: (i) the shelf unit associated with the image 306 is associated with a potential shift event; or (ii) the shelf health state associated with the shelf unit includes at least one of a second value or a third value indicating that the shelf unit is damaged by an impact from a storage vehicle, such as "crane fork shift".Figure 3D Shows an exemplary representation of an image 308 captured using the techniques provided herein according to some embodiments. In some embodiments, in the case of using one or more of the techniques provided herein, at least one of the storage controller or the computer of the shelf status monitoring device determines, based on the image 308, at least one of the following: (i) the shelf unit associated with the image 308 is associated with a potential shift event; or (ii) the shelf health status associated with the shelf unit includes at least one of a second value or a fourth value indicating at least one of the cases where the reflective label of the shelf unit is damaged or displaced relative to the proper position, such as "reflector shift". Figure 3E Shows an exemplary representation of an image 310 captured using the techniques provided herein according to some embodiments. In some embodiments, in the case of using one or more of the techniques provided herein, at least one of the storage controller or the computer of the shelf status monitoring device determines that the shelf unit associated with the image 310 is not associated with a potential shift event. Figure 3F Shows an exemplary representation of an image 312 captured using the techniques provided herein according to some embodiments. In some embodiments, in the case of using one or more of the techniques provided herein, at least one of the storage controller or the computer of the shelf status monitoring device determines, based on the image 312, at least one of the following: (i) the shelf unit associated with the image 312 is associated with a potential shift event; or (ii) the shelf health status associated with the shelf unit is a first value, such as "warning". Figure 3G Shows an exemplary representation of an image 314 captured using the techniques provided herein according to some embodiments. In some embodiments, in the case of using one or more of the techniques provided herein, at least one of the storage controller or the computer of the shelf status monitoring device determines, based on the image 314, at least one of the following: (i) the shelf unit associated with the image 314 is associated with a potential shift event; or (ii) the shelf health status associated with the shelf unit includes at least one of a second value or a fourth value indicating at least one of the cases where the reflective label of the shelf unit is damaged or displaced relative to the proper position, such as "reflector shift".

[0072] In some embodiments, the first light generating device 110 of the shelf status monitoring device is used (in Figures 1A to 1C and Figures 1E to 1Fas shown in) to generate a second light beam. In some embodiments, the first light generating device 110 emits the second light beam during the second storage operation, such as a laser beam. In some embodiments, the first light generating device 110 emits the second light beam while the first image sensor 114 is used to capture the second image 144. In some embodiments, the first light generating device 110 is positioned such that when capturing the second image 144, it emits the second light beam towards a predetermined position of the first portion of the second shelf unit 124F (e.g., towards the region 152 defined by the one or more second markings of the second reflection tag 143 of the second reflection tag 143 (as shown in Figure 1G as shown in) at a predetermined position).

[0073] In some embodiments, one or more components of the storage carrier 108 are positioned differently during a placement operation such as the first storage operation than during an acquisition operation such as the second storage operation. In some embodiments, due to the difference in the positions of the one or more components between the first storage operation and the second storage operation, the first light generating device 110 is not properly positioned to emit a light beam towards the predetermined position of the first portion of the second shelf unit 124F. In some embodiments, the light beam emitted by the first light generating device 110 can be emitted in a direction that is not towards the predetermined position of the first portion of the second shelf unit 124F. In some embodiments, a second light generating device 112 of the shelf state monitoring device (as shown in Figure 1A to FIGS. 1C and Figures 1E to 1F as shown in) is used to generate a second light beam. In some embodiments, the second light generating device 112 is positioned differently from the first light generating device 110. In some embodiments, the second light generating device 112 includes a laser generating device. In some embodiments, the second light generating device 112 emits the second light beam during the second storage operation, such as a laser beam. In some embodiments, the position of the second light beam varies with the position of at least one of the load support assembly 106 or the storage carrier 108. In some embodiments, the second light generating device 112 has a constant position relative to at least one of the load support assembly 106 or the storage carrier 108. In some embodiments, a mechanical device is used to adjust the position of the second light generating device 112 relative to at least one of the load support assembly 106 or the storage carrier 108. In some embodiments, the second light generating device 112 emits the second light beam while the first image sensor 114 is used to capture the second image 144. In some embodiments, the second light generating device 112 is positioned such that when capturing the second image 144, it emits the second light beam towards a predetermined position of the first portion of the second shelf unit 124F (e.g., towards the region 152 defined by the one or more second markings of the second reflection tag 143 of the second reflection tag 143 (as shown in Figure 1GIn some embodiments, the second light beam is emitted from a predetermined position (shown in FIG. 1 ) of the second light generating device 112. In some embodiments, the amplification value associated with generating the second light beam using an amplifier (e.g., an input amplifier) ​​of the second light generating device 112 is at least about a second threshold amplification value. In some embodiments, the second threshold amplification value is between about 3,000 and about 5,000.

[0074] In some embodiments, based on determining that the first shelf unit 124A is not associated with the first potential displacement event, the computer of the shelf status monitoring device transmits an indication to the storage controller indicating at least one of the following: (i) the first shelf unit 124A is not associated with the first potential displacement event; or (ii) the first shelf unit 124A is in an operating condition for storing a product unit in the first storage location 128 above the first shelf unit 124A. In some embodiments, based on determining that the first shelf unit 124A is not associated with the first potential displacement event, the storage controller uses the first shelf unit 124A to store the product unit in the first storage location 128. In some embodiments, based on determining that the first shelf unit 124A is not associated with the first potential displacement event, the first shelf unit 124A is included in a candidate shelf unit list. In some embodiments, the candidate shelf unit list indicates shelf units that are each determined to be in an operating condition to be used for storing a product unit. In some embodiments, the storage controller selects the first shelf unit 124A from the candidate shelf unit list for storing the product unit. In some embodiments, in response to selecting the first shelving unit 124A, the storage controller controls a storage carrier (eg, storage carrier 108) to transfer the product unit to the first storage location 128 on the first shelving unit 124A.

[0075] In some embodiments, in response to determining that the second shelf unit 124F is associated with a second potential shift event, at least one of the storage controller or a computer of the shelf status monitoring device marks the second shelf unit 124F as service aborted. In some embodiments, when the second shelf unit 124F is marked as service aborted, the storage controller does not store product units in the second storage location 132 above the second shelf unit 124F. In some embodiments, based on determining that the second shelf unit 124F is associated with a second potential shift event, the storage controller does not use the second shelf unit 124F to store product units in the second storage location 132. In some embodiments, marking the second shelf unit 124F as service aborted includes including the second shelf unit 124F in a marked shelf unit list. In some embodiments, the marked shelf unit list indicates shelf units marked as service aborted, for example, due to being associated with potential shift events. In some embodiments, based on the second shelf unit 124F being included in the marked shelf unit list, the storage controller does not use the second shelf unit 124F to store product units in the second storage location 132.

[0076] In some embodiments, in response to determining that the second shelf unit 124F is associated with a second potential shift event, the computer of the shelf status monitoring device transmits a shift event signal to the storage controller. In some embodiments, the shift event signal includes at least one of the following: (i) an indication of the second shelf unit 124F; (ii) an indication of the position of the second shelf unit 124F; (iii) an indication that the second shelf unit 124F is associated with a second potential shift event; (iv) a second image 144; (v) an indication of service abort of the second shelf unit; (vi) an indication of the soundness status of the first shelf; or (vii) other information associated with at least one of the second shelf unit 124F or the second potential shift event.

[0077] In some embodiments, in response to determining that the second shelf unit 124F is associated with a second potential shift event, at least one of the storage controller or a computer of the shelf status monitoring device transmits an instruction to repair the second shelf unit 124F to the maintenance event scheduling module. In some embodiments, the maintenance event scheduling module schedules a maintenance event for the second shelf unit 124F in response to at least one of the instruction to repair the second shelf unit 124F or determining that the second shelf unit 124F is associated with a second potential shift event. In some embodiments, one or more resources are allocated during the maintenance event to repair the second shelf unit 124F. In some embodiments, the second shelf unit 124F is removed from the marked shelf unit list in response to repairing the second shelf unit 124F.

[0078] Figures 4A to 4B Shows a shelf status monitoring device according to some embodiments. Figure 4A A perspective view of the first portion 400 of the shelf status monitoring device is shown. In some embodiments, the first portion 400 of the shelf status monitoring device includes at least one of the following devices: (i) a second image sensor 416; (ii) a third image sensor 418; (iii) a first light controller 408; (iv) a first communication module 402; (v) a first direct current (DC) / DC converter 404; (vi) a terminal block 406; (vii) a light source 412; (viii) a light source 414; (ix) a frame 410; or (x) other components. In some embodiments, the first image sensor 114 corresponds to at least one of the second image sensor 416 or the third image sensor 418. In some embodiments, the second image sensor 416 is used to capture an image of the shelf unit during a placement operation and the third image sensor 418 is used to capture an image of the shelf unit during a placement operation. In some embodiments, one or more components of the storage vehicle 108 are positioned differently during a placement operation such as a first storage operation than during an acquisition operation such as a second storage operation. In some embodiments, (i) the second image sensor 416 is properly positioned to capture a first view of a first predetermined position associated with the first shelf unit 124A during a first storage operation, (ii) the second image sensor 416 is not properly positioned to capture a second view of a second predetermined position associated with the second shelf unit 124F during a second storage operation, (iii) the third image sensor 418 is not properly positioned to capture a first view of a first predetermined position associated with the first shelf unit 124A during a first storage operation, or (iv) the third image sensor 418 is properly positioned to capture a second view of a second predetermined position associated with the second shelf unit 124F during a second storage operation. In some embodiments, the second image sensor 416 is used to capture a first image 134 and the third image sensor 418 is used to capture a second image 144. Embodiments are contemplated in which at least one of the following situations exists: the second image sensor 416 is used to capture the second image 144 or the third image sensor 418 is used to capture the first image 134.

[0079] In some embodiments, at least one of light source 412 or light source 414 generates light. In some embodiments, at least one of light source 412 or light source 414 generates light to illuminate the surroundings of the shelf status monitoring device. In some embodiments, at least one of light source 412 or light source 414 generates light to enable at least one of second image sensor 416 or third image sensor 418 to generate shelf monitoring images, such as at least one of first image 134, second image 144, etc., for example, with increased resolution and / or clarity. In some embodiments, first light controller 408 is configured to control one or more light generation parameters of at least one of first light generating device 110, second light generating device 112, light source 412 or light source 414. In some embodiments, the one or more light generation parameters include at least one of light wavelength, light intensity or other light generation parameters. The light wavelength of the light output by at least one of first light generating device 110, second light generating device 112, light source 412 or light source 414 is between approximately a first wavelength and approximately a second wavelength. In some embodiments, the first wavelength is between approximately 360 nanometers and approximately 400 nanometers. In some embodiments, the second wavelength is between approximately 760 nanometers and approximately 830 nanometers. In some embodiments, the light wavelength of at least one of first light generating device 110, second light generating device 112, light source 412 or light source 414 is a visible light wavelength. Other values of the light wavelength of at least one of first light generating device 110, second light generating device 112, light source 412 or light source 414 are also within the scope of this disclosure.

[0080] In some embodiments, first communication module 402 is used to communicate with at least one of a storage controller, a computer of the shelf status monitoring device, or one or more other devices. In some embodiments, the shelf status monitoring device uses first communication module 402 to transmit information such as at least one of the following: one or more captured shelf monitoring images; one or more determinations of whether a shelf unit is associated with a potential displacement event; or other information. In some embodiments, first communication module 402 transmits information wirelessly. In some embodiments, first communication module 402 includes at least one of an Ethernet module or other types of communication modules.

[0081] Figure 4B A perspective view showing a second part 450 of the shelf status monitoring device. In some embodiments, the second part 450 of the shelf status monitoring device includes at least one of the following devices: (i) first light generating device 110; (ii) second light generating device 112; (iii) plate 430; (iv) a platform (such as a fork) of load support assembly 106 of storage carrier 108; or (v) other components.

[0082] In some embodiments, at least one of the first portion 400 or the second portion 450 of the shelf status monitoring device is mounted to the storage carrier 108 by the frame 410 (shown in Figure 4A ), one or more mounting components 415 (shown in Figures 4A to 4B ), a fixture, or at least one of one or more other suitable components. In some embodiments, at least one of the first portion 400 or the second portion 450 of the shelf status monitoring device is mounted to the storage carrier 108 during a storage operation performed using the storage carrier 108. In some embodiments, during a storage operation performed using the storage carrier 108, at least one of the first portion 400 or the second portion 450 of the shelf status monitoring device is configured to perform at least one of the following operations: capturing a shelf monitoring image; or determining whether a shelf unit associated with the shelf monitoring image is associated with a potential displacement event.

[0083] In some embodiments, an inspection of the second shelf unit 124F is performed in response to determining that the second shelf unit 124F is associated with a second potential displacement event. In some embodiments, the inspection is performed using an inspection device. In some embodiments, in response to determining that the second shelf unit 124F is associated with a second potential displacement event, at least one of the storage controller or the computer of the shelf status monitoring device triggers the inspection, for example, by transmitting an instruction to inspect the second shelf unit 124F to the inspection device.

[0084] Figure 5Shows at least a part of an inspection device (shown as reference numeral 500) according to some embodiments. In some embodiments, the inspection device 500 includes at least one of the following devices: (i) a fourth image sensor 508, such as a camera; (ii) a fifth image sensor 512, such as a camera; (iii) a first distance sensor 510; (iv) a second distance sensor 514; (v) a second light controller 506; (vi) a second communication module 504; (vii) a second DC / DC converter 502; or (viii) other components. In some embodiments, the second light controller 506 is configured to control one or more light generation parameters of one or more light generation devices of at least one of the inspection device 500 or the second storage carrier on which the inspection device 500 is fixed. In some embodiments, the second storage carrier is the same as the storage carrier 108. In some embodiments, the second storage carrier is different from the storage carrier 108. The light wavelength of the light output by the one or more light generation devices is between about a third wavelength and about a fourth wavelength. In some embodiments, the third wavelength is between about 360 nanometers and about 400 nanometers. In some embodiments, the fourth wavelength is between about 760 nanometers and about 830 nanometers. In some embodiments, the light wavelength of the one or more light generation devices is a visible light wavelength. Other values of the light wavelength of the one or more light generation devices are also within the scope of this disclosure. In some embodiments, the inspection device 500 is mounted to the second storage carrier by at least one of a frame, one or more mounting components, a fixture, or one or more other suitable components.

[0085] In some embodiments, the inspection of the second shelf unit 124F includes using an inspection device 500 to capture a third image of a third view associated with a second predetermined position of the second shelf unit 124F. In some embodiments, the second storage carrier transports the inspection device 500 to a position close to the second shelf unit 124F. In some embodiments, at least one of a fourth image sensor 508 or a fifth image sensor 512 is used to capture the third image associated with the second shelf unit 124F. In some embodiments, the third image is captured when at least one of the fourth image sensor 508 or the fifth image sensor 512 has a third view of the second predetermined position of the second shelf unit 124F. In some embodiments, the inspection device 500 determines that at least one of the fourth image sensor 508 or the fifth image sensor 512 has a third view of the second predetermined position of the second shelf unit 124F based on a match between the current position of at least one of the components of the inspection device 500 or the components of the second storage carrier and a reference position corresponding to at least one of the fourth image sensor 508 or the fifth image sensor 512 having a third view of the second predetermined position. In some embodiments, the inspection device 500 uses at least one of a first distance sensor 510 or a second distance sensor 514 to determine that at least one of the fourth image sensor 508 or the fifth image sensor 512 has a third view of the second predetermined position of the second shelf unit 124F. In some embodiments, the second image is captured when the first component has a second reference position such that the second image has a third view of the second predetermined position associated with the second shelf unit 124F.

[0086] In some embodiments, when the second shelf unit 124F has the second predetermined position, the third view of the third image corresponds to a second portion of the second shelf unit 124F. For example, if the second shelf unit 124F has the second predetermined position and is not damaged and / or displaced, the second portion of the second shelf unit 124F is depicted in the third view. In some embodiments, when the second shelf unit 124F is damaged and / or displaced, at least a portion of the second portion of the second shelf unit 124F is not depicted in the third image, for example, at least in part due to the displacement of the second portion of the second shelf unit 124F relative to the predetermined position of the second portion of the second shelf unit 124F. In some embodiments, when at least one of the second shelf unit 124F has the second predetermined position or the second shelf unit 124F is not damaged occurs, the predetermined position of the second portion of the second shelf unit 124F corresponds to the position of the second portion of the second shelf unit 124F. In some embodiments, the second portion of the second shelf unit 124F includes a stopper of the second shelf unit 124F (e.g. Figure 2Aat least a portion of the stopper 208 and / or the stopper 210 shown in. In some embodiments, the second portion of the second shelf unit 124F includes Figure 2B a second portion 226 of the stopper 208 shown in.

[0087] In some embodiments, the inspection device 500 determines whether the determination associated with the second shelf unit 124F and the second potential shift event is valid based on the third image. According to some embodiments, in Figures 6A to 6CAn exemplary representation of a third image (shown as reference numeral 602) is shown. In some embodiments, the second portion of the second shelf unit 124F includes at least one of a first opening 604 shown in the third image 602, a second opening 606 shown in the third image 602, or a third opening 608 shown in the third image 602. In some embodiments, at least one of the first opening 604, the second opening 606, or the third opening 608 corresponds to at least one of a slot, an opening, etc. of a stopper passing through the second shelf unit 124F. In some embodiments, the inspection device 500 determines whether a determination associated with the second shelf unit 124F and a second potential shift event is valid based on at least one of the following: (i) the position of the first opening 604 shown in the third image 602; (ii) the position of the second opening 606 shown in the third image 602; or (iii) the position of the third opening 608 shown in the third image 602. In some embodiments, the inspection device 500 determines whether a determination associated with the second shelf unit 124F and a second potential shift event is valid based on at least one of the following: (i) the distance between the position of the first opening 604 and a predetermined position of the first opening 604 (e.g., a reference position of the first opening 604); (ii) the distance between the position of the second opening 606 and a predetermined position of the second opening 606 (e.g., a reference position of the second opening 606); or (iii) the distance between the position of the third opening 608 and a predetermined position of the third opening 608 (e.g., a reference position of the third opening 608). In some embodiments, the inspection device 500 determines that a determination associated with the second shelf unit 124F and a second potential shift event is valid based on at least one of the following: (i) the distance between the position of the first opening 604 and the predetermined position of the first opening 604 does not exceed a first threshold distance; (ii) the distance between the position of the second opening 606 and the predetermined position of the second opening 606 does not exceed a second threshold distance; or (iii) the distance between the position of the third opening 608 and the predetermined position of the third opening 608 does not exceed a third threshold distance. In some embodiments, the inspection device 500 determines that a determination associated with the second shelf unit 124F and a second potential shift event is invalid based on at least one of the following: (i) the distance between the position of the first opening 604 and the predetermined position of the first opening 604 exceeds the first threshold distance; (ii) the distance between the position of the second opening 606 and the predetermined position of the second opening 606 exceeds the second threshold distance; or (iii) the distance between the position of the third opening 608 and the predetermined position of the third opening 608 exceeds the third threshold distance.

[0088] In some embodiments, inspection device 500 compares one or more features of the third image 602 with one or more landmarks to determine whether the determination that the second shelf unit 124F is associated with the second potential displacement event is valid. In some embodiments, the one or more landmarks include a reference line 610 and a reference cross 612. The reference line 610 and the reference cross 612 are shown in Figures 6A to 6C using dashed lines. In some embodiments, inspection device 500 determines whether the determination that the second shelf unit 124F is associated with the second potential displacement event is valid based on at least one of the following: (i) whether the first opening 604 and the second opening 606 are aligned with the reference line 610; or (ii) whether the third opening 608 is aligned with the reference cross 612. In some embodiments, the first opening 604 and the second opening 606 are determined to be aligned with the reference line 610 based on determining at least one of the following: (i) the first opening 604 is within a fourth threshold distance of the reference line 610; or (ii) the second opening 606 is within a fourth threshold distance of the reference line 610. In some embodiments, the third opening 608 is determined to be aligned with the reference cross 612 based on determining that the center of the third opening 608 is within a fifth threshold distance of the center of the reference cross 612. In some embodiments, inspection device 500 determines that the determination that the second shelf unit 124F is associated with the second potential displacement event is valid based on at least one of the following: (i) determining that the first opening 604 and the second opening 606 are aligned with the reference line 610; or (ii) determining that the third opening 608 is aligned with the reference cross 612. In some embodiments, inspection device 500 determines that the determination that the second shelf unit 124F is associated with the second potential displacement event is invalid based on at least one of the following: (i) determining that at least one of the first opening 604 or the second opening 606 is not aligned with the reference line 610; or (ii) determining that the third opening 608 is not aligned with the reference cross 612.

[0089] The following embodiments are contemplated: in which embodiments, the inspection of the second shelf unit 124F is performed using one or more techniques other than the techniques provided herein, such as using one or more other types of sensors (and / or also using one or more other types of sensors in addition to the fourth image sensor 508, the fifth image sensor 512, the first distance sensor 510, and / or the second distance sensor 514) other than the fourth image sensor 508, the fifth image sensor 512, the first distance sensor 510, and / or the second distance sensor 514, using other types of landmarks and / or markers other than the landmarks and / or markers explicitly shown in the drawings (and / or also using other types of landmarks and / or markers in addition to the landmarks and / or markers explicitly shown in the drawings), etc.

[0090] ForFigure 6A An exemplary representation of the third image 602 provided in [description], in some embodiments, the inspection device 500 determines at least one of the following based on the third image 602: (i) the second shelf unit 124F is in an operating condition in the second storage position 132 intended for storing product units on top of the second shelf unit 124F; or (ii) the determination that the second shelf unit 124F is associated with a second potential displacement event is invalid.

[0091] For Figure 6B An exemplary representation of the third image 602 provided in [description], in some embodiments, the inspection device 500 determines at least one of the following based on the third image 602: (i) the second shelf unit 124F is associated with a second potential displacement event; or (ii) the determination that the second shelf unit 124F is associated with a second potential displacement event is valid.

[0092] For Figure 6C An exemplary representation of the third image 602 provided in [description], in some embodiments, the inspection device 500 determines at least one of the following based on the third image 602: (i) the second shelf unit 124F is associated with a second potential displacement event; or (ii) the determination that the second shelf unit 124F is associated with a second potential displacement event is valid.

[0093] In some embodiments, the inspection of the second shelf unit 124F includes using the inspection device 500 to capture a third image 602 of a third view and a fourth image of a fourth view associated with a second predetermined position of the second shelf unit 124F. In some embodiments, the third image 602 is captured using the fourth image sensor 508 and the fourth image is captured using the fifth image sensor 512. In some embodiments, the third image 602 and the fourth image are captured simultaneously. In some embodiments, the third image 602 and the fourth image are captured when at least one of the following occurs: (i) the fourth image sensor 508 has a third view of the second predetermined position of the second shelf unit 124F; or (ii) the fifth image sensor 512 has a fourth view of the second predetermined position of the second shelf unit 124F.

[0094] In some embodiments, when the second shelf unit 124F is in the second predetermined position, the fourth view of the fourth image corresponds to the third portion of the second shelf unit 124F. For example, if the second shelf unit 124F is in the second predetermined position and is not damaged and / or displaced, the third portion of the second shelf unit 124F is depicted in the third view. In some embodiments, when the second shelf unit 124F is damaged and / or displaced, at least a portion of the third portion of the second shelf unit 124F is not depicted in the fourth image, for example, at least in part because the third portion of the second shelf unit 124F is displaced relative to the predetermined position of the third portion of the second shelf unit 124F. In some embodiments, when at least one of the second shelf unit 124F being in the second predetermined position or the second shelf unit 124F not being damaged occurs, the predetermined position of the third portion of the second shelf unit 124F corresponds to the position of the third portion of the second shelf unit 124F. In some embodiments, the third portion of the second shelf unit 124F includes at least a portion of a stopper of the second shelf unit 124F (such as Figure 2A the stopper 208 and / or the stopper 210 shown in Figure 2A ). In some embodiments, the second portion of the second shelf unit 124F includes at least a portion of a first stopper of the second shelf unit 124F (such as Figure 2A the stopper 208 shown in

[0095] ), and the third portion of the second shelf unit 124F includes at least a portion of a second stopper of the second shelf unit 124F (such as Figures 7A to 7CAn exemplary representation of a third image 602 and a fourth image (shown as reference numeral 702) is shown. In some embodiments, the third portion of the second shelf unit 124F includes at least one of a fourth opening 704 shown in the fourth image 702, a fifth opening 706 shown in the fourth image 702, or a sixth opening 708 shown in the fourth image 702. In some embodiments, at least one of the fourth opening 704, the fifth opening 706, or the sixth opening 708 corresponds to at least one of a slot, an opening, etc. of a stopper passing through the second shelf unit 124F. In some embodiments, the inspection device 500 determines whether a determination associated with the second shelf unit 124F and the second potential shift event is valid based on at least one of the following: (i) the distance between the position of the first opening 604 shown in the third image 602 and a predetermined position of the first opening 604 (e.g., a reference position of the first opening 604); (ii) the distance between the position of the second opening 606 shown in the third image 602 and a predetermined position of the second opening 606 (e.g., a reference position of the second opening 606); (iii) the distance between the position of the third opening 608 shown in the third image 602 and a predetermined position of the third opening 608 (e.g., a reference position of the third opening 608); (iv) the distance between the position of the fourth opening 704 shown in the fourth image 702 and a predetermined position of the fourth opening 704 (e.g., a reference position of the fourth opening 704); (v) the distance between the position of the fifth opening 706 shown in the fourth image 702 and a predetermined position of the fifth opening 706 (e.g., a reference position of the fifth opening 706); or (vi) the distance between the position of the sixth opening 708 shown in the fourth image 702 and a predetermined position of the sixth opening 708 (e.g., a reference position of the sixth opening 708).

[0096] In some embodiments, the inspection device 500 compares one or more features of the fourth image 702 with one or more landmarks (e.g., reference line 710 and reference cross 712) to determine whether a determination associated with the second shelf unit 124F and the second potential shift event is valid. In some embodiments, the inspection device 500 determines whether a determination associated with the second shelf unit 124F and the second potential shift event is valid based on at least one of the following: (i) whether the first opening 604 and the second opening 606 are aligned with the reference line 610; (ii) whether the third opening 608 is aligned with the reference cross 612; (iii) whether the fourth opening 704 and the fifth opening 706 are aligned with the reference line 710; or (iv) whether the sixth opening 708 is aligned with the reference cross 712.

[0097] In some embodiments, the shelf status monitoring device 804 uses two image sensors to capture two images during a storage operation, for example, using one or more of the techniques provided herein for using the inspection device 500 to capture the third image 602 and the fourth image 702 using the fourth image sensor 508 and the fifth image sensor 512, respectively. In some embodiments, the two images are captured simultaneously. In some embodiments, the two images are captured using different image sensors (e.g., the second image sensor 416 and the third image sensor 418). The following embodiments are contemplated: in which a single image sensor is used to capture the two images and / or in which the two images map to sections of a larger image. Return reference Figure 2A , in some embodiments, the two images include: (i) an image of a view of a predetermined position of the first portion 224 of the stopper 208; and (ii) an image of a view of a predetermined position of a portion of the stopper 210 (e.g., the portion of the stopper 210 including the reflective label). In some embodiments, based on the two images, it is determined whether the shelf unit 124 is associated with a potential shift event. The following embodiments are contemplated: in which more than two images are captured or at least one of the operations is used to determine whether the shelf unit 124 is associated with a potential shift event.

[0098] In some embodiments, in response to determining that the determination that the second shelf unit 124F is associated with a second potential shift event is valid, at least one of the storage controller or the inspection device 500 marks the second shelf unit 124F as out of service.

[0099] In some embodiments, in response to determining that the determination that the second shelf unit 124F is associated with a second potential shift event is valid, the inspection device 500 transmits an indication that the second shelf unit 124F is damaged to the storage controller.

[0100] In some embodiments, in response to determining that the determination that the second shelf unit 124F is associated with a second potential shift event is valid, at least one of the storage controller or the inspection device 500 transmits a second instruction to repair the second shelf unit 124F to the maintenance event scheduling module. In some embodiments, the maintenance event scheduling module schedules a maintenance event for the second shelf unit 124F in response to at least one of the second instruction to repair the second shelf unit 124F or the determination that the determination that the second shelf unit 124F is associated with a second potential shift event is valid.

[0101] Figure 8A diagram showing a scenario 800 associated with an automated material handling system 100 according to some embodiments. In some embodiments, the automated material handling system 100 includes at least one of a shelf status monitoring device (shown as reference numeral 804), an inspection device 500, or a storage controller 802. In some embodiments, the storage controller 802 includes a material control system (MCS). In some embodiments, the storage controller 802 performs at least one of the following operations: communicate with one or more storage carriers of the automated material handling system 100 (such as storage carrier 108), control the one or more storage carriers, or provide instructions to the one or more storage carriers. In some embodiments, the shelf status monitoring device 804 uses a first communication module 402 (shown in Figure 4A as shown) to communicate with at least one of the storage controller 802 or the database 812. In some embodiments, the shelf status monitoring device 804 provides a shelf monitoring image of the shelf unit for storage on the database 812. In some embodiments, the database 812 is used to store information including at least one of the following: shelf soundness status, a list of candidate shelf units, a list of marked shelf units, or other information. In some embodiments, the inspection device 500 uses a second communication module 504 (shown in Figure 5 as shown) to communicate with the storage controller 802.

[0102] In some embodiments, at least one of the shelf status monitoring device 804 or the storage controller 802 performs position detection 806 to determine when the image sensor of the shelf status monitoring device 804 has a suitable view for capturing a shelf monitoring image of the shelf unit. In some embodiments, the shelf monitoring image is captured during a storage operation associated with the shelf unit. Embodiments are contemplated in which the shelf monitoring image is captured at a time other than during the storage operation associated with the shelf unit. In some embodiments, the shelf monitoring image is captured and / or analyzed 808 to determine whether the shelf unit is associated with a potential displacement event. In some embodiments, a computer 805 of the shelf status monitoring device 804 is used to determine whether the shelf unit is associated with a potential displacement event. In some embodiments, at least one of position detection 806 or image capture and / or analysis 808 is repeated after at least one of the shelf status monitoring device 804 or the storage controller 802 moves 807 to a subsequent shelf unit for a subsequent storage operation in response to determining that the shelf unit is not associated with a potential displacement event.

[0103] In some embodiments, in response to determining that a shelf unit is associated with a potential displacement event, at least one of the shelf state monitoring device 804 or the storage controller 802 marks 814 the shelf unit as out of service. In some embodiments, marking 814 the shelf unit as out of service triggers 820 the inspection device 500 to perform an inspection of the shelf unit. In some embodiments, at least one of the shelf state monitoring device 804 or the storage controller 802 transmits an instruction to perform an inspection of the shelf unit in response to at least one of the following: (i) determining that the shelf unit is associated with a potential displacement event; or (ii) marking the shelf unit as out of service.

[0104] In some embodiments, the inspection of the shelf unit includes using at least one of the fourth image sensor 508, the fifth image sensor 512, the first distance sensor 510, the second distance sensor 514, or other components to scan 822 the shelf unit to perform at least one of the following: (i) determining when at least one of the fourth image sensor 508 or the fifth image sensor 512 has a suitable view of a predetermined position of the shelf unit; or (ii) capturing an image of a suitable view of a predetermined position of the shelf unit. In some embodiments, the inspection of the shelf unit includes analyzing 824 the captured image to determine whether the determination that the shelf unit is associated with a potential displacement event is valid.

[0105] In some embodiments, the shelf unit is released 826 from the out-of-service state in response to determining that the determination that the shelf unit is associated with a potential displacement event is invalid. In some embodiments, the shelf unit is removed from the list of marked shelf units in response to determining that the determination that the shelf unit is associated with a potential displacement event is invalid.

[0106] In some embodiments, in response to determining that the determination that the shelf unit is associated with a potential displacement event is valid, use of the shelf unit is blocked 830 such that the shelf unit is not used for storing product units. In some embodiments, blocking 830 the shelf unit includes including the shelf unit in a list of blocked shelf units. In some embodiments, the list of blocked shelf units indicates the shelf units that are blocked from use. In some embodiments, the storage controller 802 does not use the shelf unit to store product units based on the shelf unit being included in the list of blocked shelf units.

[0107] In some embodiments, in response to determining that a determination that a shelf unit is associated with a potential shift event is valid, the shelf unit is repaired 828. In some embodiments, in response to determining that a determination that a shelf unit is associated with a potential shift event is valid, a maintenance device of the automated material handling system 100 transmits an instruction to repair the shelf unit to an automated maintenance machine (e.g., a robot). In some embodiments, in response to the instruction, the automated maintenance machine automatically repairs the shelf unit. In some embodiments, in response to repairing the shelf unit 828, a second inspection of the shelf unit is performed by scanning the shelf unit 822 to capture an image or analyzing the image 824 to determine whether the shelf unit is associated with a shift event.

[0108] According to some embodiments, in Figure 9 is shown a method 900 of determining whether a shelf unit is associated with a potential shift event. At operation 902, a first storage operation associated with a first shelf unit is performed using a storage carrier. In some embodiments, the first storage operation includes a placement operation that includes transferring a first product unit from a load support assembly of the storage carrier to a first storage location above the first shelf unit. In some embodiments, the first storage operation includes a retrieval operation that includes transferring a first product unit from the first storage location above the first shelf unit to the load support assembly of the storage carrier. At operation 904, a first image of a view associated with a first predetermined location of the first shelf unit is captured during the first storage operation. In some embodiments, the first image is captured using a first image sensor coupled to the storage carrier. At operation 906, based on the first image, it is determined whether the first shelf unit is associated with a potential shift event. In some embodiments, the first shelf unit is marked as out of service in response to determining that the first shelf unit is associated with a potential shift event.

[0109] Figure 10AShows a representation 1000 of a shelf unit 1002 determined to be associated with a potential displacement event according to some embodiments. In some embodiments, the shelf unit 1002 undergoes at least one of downward bending or tilting, for example, at least partially due to impacts and / or pressures from objects during a storage operation (such as impacts and / or pressures from at least one of the storage vehicle 108, the load support assembly 106 of the storage vehicle 108, or the product unit 104). In some embodiments, the storage vehicle 108 that places the product unit 104 onto the shelf unit 1002 may cause damage to at least one of the product unit 104, the shelf unit 1002, or other components, for example, at least partially due to the downward bending of the shelf unit 1002. In the case of using the techniques provided herein, in some embodiments, the shelf state monitoring device determines that the shelf unit 1002 is associated with a potential displacement event and performs at least one of the following operations: (i) marks the shelf unit 1002 as out of service; or (ii) facilitates the repair of the shelf unit, thereby reducing damage to at least one of the shelf unit 1002, the storage vehicle 108, or the product unit 104. Figure 10B Shows a representation 1050 of a shelf unit 1004 determined to be associated with a potential displacement event according to some embodiments. In some embodiments, the shelf unit 1004 undergoes at least one of downward bending or tilting, for example, at least partially due to impacts and / or pressures from objects during a storage operation (such as impacts and / or pressures from at least one of the storage vehicle 108, the load support assembly 106 of the storage vehicle 108, or the product unit 104).

[0110] In some embodiments, a system is provided. In some embodiments, the system includes an automated material handling system. In some embodiments, the system includes inspection equipment 500. In some embodiments, the system includes a shelf state monitoring device. In some embodiments, the system includes a shelf unit 124 (in Figures 2A to 2B(shown in). In some embodiments, the shelf unit 124 includes a stopper 208, and the stopper 208 includes a first portion 224 and a second portion 226. In some embodiments, the shelf unit 124 includes: (i) a reflective tag 222 in the first portion 224 of the stopper 208; (ii) a cross-shaped opening 230 in the second portion 226 of the stopper 208; (iii) a first circular opening 228 in the second portion 226 of the stopper 208; and (iv) a second circular opening 232 in the second portion 226 of the stopper 208. In some embodiments, at least one of the second image sensor 416 or the third image sensor 418 of the shelf status monitoring device captures an image having a view corresponding to the first portion 224 of the stopper 208. In some embodiments, at least one of the fourth image sensor 508 or the fifth image sensor 512 of the inspection device 500 captures a second image having a view corresponding to the second portion 226 of the stopper 208.

[0111] One or more embodiments relate to a computer-readable medium including processor-executable instructions configured to implement one or more of the techniques presented herein. Figure 11 An exemplary computer-readable medium is shown, where embodiment 1100 includes a computer-readable medium 1108 encoded with computer-readable data 1106 thereon (such as a compact disc-recorder (CD-R), a digital versatile disc-recorder (DVD-R), a flash drive, a magnetic disk of a hard disk drive, etc.). This computer-readable data 1106 in turn includes a set of processor-executable computer instructions 1104 configured to implement one or more of the principles stated herein when executed by a processor. In some embodiments 1100, the processor-executable computer instructions 1104 are configured to implement method 1102 when executed by a processor, such as at least some of the methods described above. In some embodiments, the processor-executable computer instructions 1104 are configured to implement a system when executed by a processor, such as at least some of the one or more systems described above. In some embodiments, the processor-executable computer instructions 1104 are configured to implement a device when executed by a processor, such as at least some of the one or more devices described above. Those of ordinary skill in the art can design many such computer-readable media configured to operate in accordance with the techniques presented herein.

[0112] In some embodiments, a method is provided. The method includes: performing a first storage operation associated with a first shelf unit using a storage vehicle. The first storage operation includes: a placement operation, including transferring a first product unit from a load support component of the storage vehicle to a first storage position above the first shelf unit; or a retrieval operation, including transferring a first product unit from a first storage position above the first shelf unit to a load support component of the storage vehicle. The method includes: during the first storage operation, using a first image sensor coupled to the storage vehicle to capture a first image of a view associated with a first predetermined position of the first shelf unit. The method includes: based on the first image, determining whether the first shelf unit is associated with a potential displacement event.

[0113] In some embodiments, the method further includes emitting a light beam using a light generating device coupled to the storage vehicle while capturing the first image, wherein determining whether the first shelf unit is associated with the potential shift event includes determining the reflection of the light beam shown in the first image relative to a landmark position shown in the first image. In some embodiments, the method further includes the landmark corresponding to a reflective label on a surface of the first shelf unit. In some embodiments, the method further includes marking the first shelf unit as out of service in response to determining that the first shelf unit is associated with the potential shift event. In some embodiments, the method further includes not transferring a second product unit to the first storage location above the first shelf unit based on the first shelf unit being marked as out of service. In some embodiments, the method further includes performing an inspection of the first shelf unit in response to determining that the first shelf unit is associated with the potential shift event. In some embodiments, performing the inspection includes capturing a second image of a second view associated with a first predetermined position of the first shelf unit and determining based on the second image whether the determination that the first shelf unit is associated with the potential shift event is valid. In some embodiments, the method further includes transmitting an instruction to repair the first shelf unit in response to determining that the determination that the first shelf unit is associated with the potential shift event is valid. In some embodiments, the method further includes marking the first shelf unit as out of service in response to determining that the determination that the first shelf unit is associated with the potential shift event is valid. In some embodiments, the method further includes performing at least one of the following operations in response to determining that the determination that the first shelf unit is associated with the potential shift event is invalid: removing the first shelf unit from a list of shelf units marked as out of service; or transferring a second product unit to the first storage location above the first shelf unit. In some embodiments, when the first shelf unit has the first predetermined position, the view of the first image corresponds to a first part of the first shelf unit and when the first shelf unit has the first predetermined position, the second view of the second image corresponds to a second part of the first shelf unit.In some embodiments, the method further includes using a light generating device coupled to the storage carrier to emit a light beam, wherein the first portion of the first shelf unit includes a reflective label located on a surface of the first shelf unit, determining whether the first shelf unit is associated with the potential displacement event includes determining a position of a reflection of the light beam shown in the first image relative to the reflective label shown in the first image, the second portion of the first shelf unit defines a first opening and a second opening, and determining whether the determination that the first shelf unit is associated with the potential displacement event is valid includes determining whether the determination that the first shelf unit is associated with the potential displacement event is valid based on a position of the first opening shown in the second image and a position of the second opening shown in the second image. In some embodiments, the potential displacement event corresponds to a displacement of the first shelf unit from the first predetermined position to a second position. In some embodiments, the first product unit includes a first wafer storage device.

[0114] In some embodiments, an automated material handling system is provided. The automated material handling system includes a storage carrier configured to perform a first storage operation associated with a first shelf unit. The first storage operation includes: a placement operation including transferring a first product unit from a load support assembly of the storage carrier to a first storage position above the first shelf unit; or a retrieval operation including transferring the first product unit from the first storage position above the first shelf unit to the load support assembly of the storage carrier. The automated material handling system includes a shelf status monitoring device including a first image sensor and a computer. The first image sensor is configured to capture a first image of a view associated with a first predetermined position of the first shelf unit during the first storage operation. The computer is configured to determine whether the first shelf unit is associated with a potential displacement event based on the first image.

[0115] In some embodiments, the shelf status monitoring device further includes a light generating device coupled to the storage carrier and configured to emit a light beam toward the first shelf unit while capturing the first image, wherein the computer is configured to compare the position of the reflection of the light beam shown in the first image from the first shelf unit with the position of a landmark shown in the first image to determine whether the first shelf unit is associated with the potential shift event. In some embodiments, the automated material handling system further includes an inspection device, wherein in response to determining that the first shelf unit is associated with the potential shift event, the computer is configured to trigger the inspection device to perform an inspection of the first shelf unit; in response to being triggered to perform the inspection of the first shelf unit, the inspection device is configured to capture a second image of a second view associated with the first predetermined position of the first shelf unit; and the computer is configured to determine whether the determination that the first shelf unit is associated with the potential shift event is valid based on the second image. In some embodiments, the first product unit includes a first wafer storage device.

[0116] In some embodiments, a method is provided. The method includes: performing a first storage operation associated with a first shelf unit using a storage carrier. The first storage operation includes: a placement operation including transferring a first product unit from a load support assembly of the storage carrier to a first storage position above the first shelf unit; or a retrieval operation including transferring the first product unit from the first storage position above the first shelf unit to the load support assembly of the storage carrier. The method includes: capturing a first image of a view associated with a first predetermined position of the first shelf unit during the first storage operation. The method includes: determining whether the first shelf unit is associated with a potential shift event based on the first image. The method includes: marking the first shelf unit as out of service in response to determining that the first shelf unit is associated with the potential shift event.

[0117] In some embodiments, the method further includes emitting a light beam while capturing the first image, wherein determining whether the first shelf unit is associated with the potential shift event includes comparing the position of the reflection of the light beam shown in the first image with the position of a landmark shown in the first image.

[0118] Although the subject matter has been described in language specific to structural features or method acts, it is to be understood that the claimed subject matter is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts are disclosed as example forms of implementing at least some of the claims.

[0119] Various operations of the embodiments are provided herein. The order in which some or all of the operations are recited should not be construed as implying that those operations must be performed in that order. Alternative orders will be appreciated that provide the beneficial effects of this description. In addition, it will be understood that not all operations need be present in every embodiment provided herein. Additionally, it should be understood that in some embodiments, not all operations are necessary.

[0120] It will be understood that in some embodiments, for example for purposes of clarity and ease of understanding, the layers, features, components, etc. illustrated herein are exemplified with respect to specific dimensions relative to each other (such as structural dimensions or orientations), and the actual dimensions of the layers, features, components, etc. are substantially different from those exemplified herein. Additionally, for example, there are various techniques such as at least one of the following to form the layers, regions, features, components, etc. mentioned herein: etching techniques, planarization techniques, implantation techniques, doping techniques, spin coating techniques, sputtering techniques, growth techniques, or deposition techniques (such as chemical vapor deposition (CVD)).

[0121] Furthermore, the term "exemplary" is used herein to mean serving as an instance, example, illustration, etc., and not necessarily being advantageous. The "or" used in this application is intended to mean an inclusive "or" rather than an exclusive "or". Additionally, unless otherwise specified or clearly indicated from the context to refer to the singular form, the "a (and an)" used in this application and the appended patent claims is generally regarded as meaning "one or more". Additionally, at least one of A and B and / or similar expressions generally mean A or B, or both A and B. Furthermore, with respect to the use of "include", "having", "with" or variations thereof, such terms are intended to mean inclusion in a manner similar to the term "comprising". Additionally, unless otherwise specified, "first", "second", etc. are not intended to imply a temporal, spatial, order, etc. Rather, such terms are merely used as identifiers, names, etc. of features, components, items, etc. For example, a first component and a second component generally correspond to component A and component B, or two different components, or two identical components, or the same component.

[0122] In addition, although the present disclosure has been shown and described with respect to one or more embodiments, equivalent changes and modifications will occur to those having ordinary skill in the art upon reading and understanding this specification and the drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the following claims. In particular, with respect to the various functions performed by the above-described components (e.g., components, resources, etc.), even if the components are not structurally equivalent to the disclosed structures, the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (e.g., functionally equivalent). Additionally, although a particular feature of the present disclosure may be disclosed with respect to only one of several embodiments, such feature may be combined with one or more other features of the other embodiments when it may be desirable and advantageous for any given or particular application.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them; although the embodiments of the present invention have been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automated material handling system, characterized in that: include: A storage carrier configured to perform a first storage operation associated with a first shelving unit, wherein the first storage operation comprises: a placing operation comprising transferring a first product unit from a load-supporting assembly of the storage carrier to a first storage location on the first shelving unit; or a retrieval operation, comprising transferring the first product unit from the first storage location on the first shelf unit to the load supporting assembly of the storage carrier; and a shelf status monitoring device, comprising: a first image sensor coupled to the storage carrier, wherein the first image sensor is configured to capture a first image of a view associated with a first predetermined position of the first shelving unit during the first storage operation; and A computer is configured to determine whether the first shelving unit is associated with a potential displacement event based on the first image.

2. The automated material handling system according to claim 1, characterized in that: The shelf status monitoring device further comprises: A light generating device, coupled to the storage carrier, is configured to emit a light beam toward the first shelf unit while capturing the first image, wherein the computer is configured to compare a position shown in the first image where the light beam is reflected from the first shelf unit with a position of a landmark shown in the first image to determine whether the first shelf unit is associated with the potential displacement event.

3. The automated material handling system according to claim 2, characterized in that: Wherein the landmark corresponds to a reflective label on a surface of the first shelving unit.

4. The automated material handling system according to claim 1, wherein: Also includes: Inspection equipment, including: In response to determining that the first shelving unit is associated with the potential displacement event, the computer is configured to trigger the inspection device to perform an inspection of the first shelving unit; In response to being triggered to perform the inspection of the first shelving unit, the inspection device is configured to capture a second image of a second view associated with the first predetermined position of the first shelving unit; and The computer is configured to determine whether the determination that the first shelving unit is associated with the potential displacement event is valid based on the second image.

5. The automated material handling system according to claim 4, characterized in that: Also includes: A maintenance machine is provided for repairing the first shelf unit in response to the inspection device confirming that the first shelf unit is damaged.

6. The automated material handling system according to claim 1, wherein: The first product unit includes a chip storage device.

7. The automated material handling system according to claim 1, wherein: The potential displacement event corresponds to displacement of the first shelf unit from the first predetermined position to a second position.

8. The automated material handling system according to claim 1, wherein: The storage carrier is further configured to perform a second storage operation associated with the second shelving unit, wherein the second storage operation includes a retrieval operation, the retrieval operation including: transferring a second product unit from a second storage location on the second shelving unit to the load supporting assembly of the storage vehicle; or The second product unit is transferred from the load supporting assembly of the storage carrier to a loading port.

9. The automated material handling system according to claim 8, characterized in that: The first image sensor captures a second image associated with the second shelving unit during the second storage operation.

10. The automated material handling system of claim 1, wherein: The first shelf unit includes a stopper protruding from the body of the first shelf unit to provide resistance for the first product unit to be separated from the first shelf unit.

Images