Intelligent inventory management cabinet

Abstract

Inventory management cabinets, systems, and related methods are described herein. An example inventory management cabinet includes a housing defining a storage area configured to receive a product, and a plurality of slots disposed within the housing, each of the slots configured to receive a respective unit of product. The cabinet also includes at least one imaging device configured to capture information about the product. The cabinet also includes a controller operably coupled to the at least imaging device, wherein the controller includes a processor and a memory having computer executable instructions stored thereon. The controller is configured to detect activity within the storage area of the housing and, in response to detecting activity within the storage area of the housing, control the at least one imaging device to initiate capturing one or more images of the product.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. non-provisional patent application No. 17 / 965,555, filed on October 13, 2022, and entitled “SMART INVENTORY MANAGEMENT CABINET,” and U.S. provisional patent application No. 63 / 334,421, filed on April 25, 2022, and entitled “SMART INVENTORY MANAGEMENT CABINET,” the disclosures of which are expressly incorporated herein by reference in their entireties. Background Art

[0003] The present disclosure relates to a smart inventory management cabinet for surgical implants. For example, the smart inventory management cabinet can be designed to store ophthalmic lenses, and more specifically to a system for dispensing ophthalmic objects, recording and tracking patient information, determining different lenses for patients, and for tracking and controlling the inventory of ophthalmic lenses in an eye care professional's office. Summary of the invention

[0004] Inventory management cabinets, systems, and related methods are described herein.

[0005] An example inventory management cabinet includes a housing defining a storage area configured to receive a product, and a plurality of slots disposed within the housing, each of the slots being configured to receive a respective unit of product. The cabinet also includes at least one imaging device configured to capture information about the product. The cabinet also includes a controller operably coupled to the at least imaging device, wherein the controller includes a processor and a memory having computer executable instructions stored thereon. The controller is configured to detect activity within the storage area of ​​the housing, and in response to detecting activity within the storage area of ​​the housing, control the at least one imaging device to initiate capturing one or more images of the product.

[0006] Additionally, at least one imaging device is optionally used to detect activity within the storage area of ​​the housing.

[0007] Alternatively or additionally, the cabinet comprises a motion sensor configured to detect the presence of an object within the storage area of ​​the housing. Optionally, the motion sensor is used to detect activity within the storage area of ​​the housing.

[0008] Alternatively, the cabinet comprises a camera configured to detect the presence of an object within a storage area of ​​the housing. Optionally, a camera is used to detect activity within the storage area.

[0009] Alternatively or additionally, the cabinet comprises a plurality of presence sensors configured to detect the presence of respective product units in respective slots of the plurality of slots. Optionally, the presence sensors are used to detect activity within the storage area of ​​the housing. Alternatively or additionally, each of the presence sensors comprises a light emitter and a photodetector.

[0010] Alternatively or in addition, the controller is further configured to extract information from one or more images of the product captured by at least one imaging device. The extracted information is used to count the product. For example, the controller is further configured to extract information from one or more images of the product captured by at least one imaging device by: receiving one or more images of the product from at least one imaging device, and analyzing the one or more images of the product to extract corresponding product identifiers associated with corresponding product units. The controller is further configured to count the product based at least in part on the extracted information, for example, by decoding the corresponding product identifiers associated with the corresponding product units, and using the corresponding product identifiers, associating the corresponding product units with the corresponding slots. Optionally, each of the corresponding product identifiers is a one-dimensional (1D) barcode, a two-dimensional (2D) barcode, a three-dimensional (3D) barcode, a universal product code (UPC), a stock keeping unit (SKU), text, or a graphic.

[0011] Alternatively or in addition, the cabinet includes a plurality of visual indicators configured to indicate the respective positions of the respective product units within the housing. Optionally, each of the visual indicators is at least one of a light emitter or a graphical display. In some implementations, the housing includes an outer frame, and at least one of the visual indicators is disposed on or adjacent to the outer frame. Alternatively or in addition, in some implementations, the respective visual indicators are disposed on, within, or adjacent to each of the respective slots.

[0012] Alternatively or additionally, the cabinet includes a plurality of imaging devices, each of the imaging devices being configured to capture information about products located in a corresponding area of ​​the storage area of ​​the housing.

[0013] Alternatively or additionally, the cabinet includes a human machine interface (HMI) configured to provide a communication interface between a user and the inventory management cabinet.

[0014] Alternatively or in addition, each of the respective product units is a product package. Optionally, the product package comprises a surgical implant, such as an intraocular lens or an orthopedic implant. Optionally, the product package comprises a surgical tool. The surgical tool may relate to an orthopedic implant, but devices in other medical fields are also contemplated. By way of non-limiting example, the following devices may be included: sutures; suturing tools; catheters; and capital equipment disposables.

[0015] Alternatively or additionally, for example, the controller is further configured to send the inventory of the product to a remote system, such as a remote database, via a network.

[0016] Alternatively or in addition, the controller is further configured to receive a request for a desired product unit, send the request for the desired product unit to a remote system via a network, and receive a response from the remote system. The response includes the slot where the desired product unit is located. Optionally, the remote system includes a database.

[0017] Another example inventory management cabinet includes a housing defining a storage area configured to receive a product, and a plurality of drawers arranged within the housing, each of the plurality of drawers including a plurality of slots, each of the slots being configured to receive a respective product unit. The cabinet also includes a plurality of imaging devices configured to capture information about the product. The cabinet also includes a controller operably coupled to at least the imaging device, wherein the controller includes a processor and a memory having computer executable instructions stored thereon. The controller is configured to detect activity within the storage area of ​​the housing, and in response to detecting activity within the storage area of ​​the housing, control the imaging device to initiate capturing one or more images of the product.

[0018] Also described herein is an example modular inventory management system. The system includes a first inventory management cabinet and a second inventory management cabinet. In addition, the system includes a human-machine interface (HMI) configured to provide a communication interface between a user and the first inventory management cabinet and the second inventory management cabinet. It is also contemplated that additional inventory management cabinets may be included in the system, so that the system may be composed of one or more cabinets that are physically attached or separated. The definition of a single specific location in the software or user interface may be composed of a leading cabinet and a following cabinet. By way of non-limiting example, further combinations of leading cabinets and following cabinets may constitute the location, thereby making the system expandable.

[0019] Also described herein is an example automated method for inventory management. The method includes detecting activity within a storage area of ​​a housing of an inventory management cabinet; in response to detecting activity within the storage area of ​​the housing, automatically initiating capture of one or more images of products within the storage area of ​​the housing; and counting the products based at least in part on using information extracted from the one or more images of the products. The method optionally also includes providing an inventory management cabinet. The present disclosure contemplates that the inventory management cabinet is a cabinet as described above.

[0020] Other systems, methods, features and / or advantages will be or may become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that the present description include all such additional systems, methods, features and / or advantages and be protected by the following claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The components in the drawings are not necessarily to scale relative to each other. Like reference numerals indicate corresponding parts throughout the several drawings.

[0022] Figure 1 is a perspective view of a sample inventory management cabinet.

[0023] Figure 2 yes Figure 1 Another perspective view of an example inventory management cabinet.

[0024] Figure 3 yes Figure 1 A cross-sectional view of an example inventory management cabinet showing multiple imaging devices.

[0025] Figure 4 yes Figure 3 Magnified view of the area shown in the dashed box.

[0026] Figure 5 yes Figure 1 A cross-sectional view of a portion of an example inventory management cabinet illustrating multiple imaging devices.

[0027] Figure 6 yes Figure 1 A cross-sectional view of a portion of an example inventory management cabinet showing a motion sensor.

[0028] Figure 7 yes Figure 1 Side view of a portion of an example inventory management cabinet showing multiple presence sensors.

[0029] Figure 8 yes Figure 7 An enlarged view of a portion of.

[0030] Fig. 9 yes Figure 1A perspective view of a portion of an example inventory management cabinet illustrating multiple slots in a storage area.

[0031] Fig.10 yes Figure 1 A cross-sectional view of a portion of an example inventory management cabinet showing multiple visual indicators.

[0032] Fig.11 yes Figure 1 A perspective view of a portion of an example inventory management cabinet showing an electronics compartment including a controller.

[0033] Fig.12 yes Figure 1 A perspective view of a portion of an example inventory management cabinet of FIG. 1 , showing a cut-away portion of a housing allowing a user to access the cabinet and an output device (e.g., a speaker).

[0034] Fig.13 is a system overview of the inventory management cabinet in the example described in this article.

[0035] Fig.14 is a schematic diagram of the system inputs for the inventory management cabinet in the examples described herein.

[0036] Fig.15 is a schematic diagram of a system output for an inventory management cabinet in the examples described herein.

[0037] Fig.16 is a diagram showing multiple slots (in rows and columns) of an inventory management cabinet used in the examples described herein.

[0038] Fig.17 is a flow chart illustrating example process operations for an inventory management cabinet in the examples described herein.

[0039] Fig.18 is an architectural diagram of the inventory management cabinet used in the examples described in this article.

[0040] Fig.19 is a perspective view of a modular system including a plurality of inventory management cabinets.

[0041] Fig. 20 is an example computing device.

[0042] Fig.21 is an example operating environment for implementations described herein.

[0043] FIG. 22A to FIG. 22B is a perspective view of another example inventory management cabinet. Fig.22A A four-drawer implementation with virtual storage space is illustrated. Fig. 22B A four-drawer implementation without virtual storage space is illustrated.

[0044] Fig.23 With open drawers Fig. 22B A perspective view of the cabinet shown in FIG.

[0045] FIG. 24A to FIG. 24C Illustrated FIG. 22A to FIG. 23 Parts inside the drawer of the cabinet shown in . Fig.24A A tray is illustrated. Fig. 24B A dispenser is illustrated which may be arranged in a tray. Fig.24C Multiple trays are illustrated arranged within a drawer with and without product units. DETAILED DESCRIPTION

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those of ordinary skill in the art. Methods and materials similar to or equivalent to those described herein may be used in the practice or testing of the present disclosure. As used in the specification and the appended claims, the singular forms "one", "a", "the" include plural objects unless the context clearly indicates otherwise. The term "include" and its variants as used herein are used synonymously with the term "include" and its variants, and are open non-restrictive terms. The term "optional" or "optionally" used herein means that the features, events or situations described subsequently may or may not occur, and the description includes instances where the features, events or situations occur and instances where they do not occur. The range may be expressed herein as "about" a specific value and / or "about" another specific value. When expressing such a range, one aspect includes from a specific value and / or to another specific value. Similarly, when a value is expressed as an approximation by using the preposition "about", it should be understood that the specific value forms another aspect. It should also be understood that the endpoints of each range are meaningful when related to and independent of another endpoint. As used herein, the term "about" or "approximately" when referring to a measurable value such as an amount, a percentage, etc., is meant to encompass variations from the measurable value of ±20%, ±10%, ±5%, or ±1%.

[0047] Example cabinets, systems, and related methods

[0048] Inventory management cabinets, systems, and methods are described herein. Such cabinets, systems, and methods can be used to track / inventory products, such as contact lenses. For example, the cabinets, systems, and methods described herein can: (i) track product units removed from storage, (ii) notify users of stocking needs, (iii) automatically place product orders, (iv) include storage space for all conventional prescription lenses, and / or (v) operate during power outages.

[0049] See now Figures 1 to 12, showing an inventory management cabinet 100. Inventory management operations are described in US 2020-0364648 and US 2020-0364650, the entire disclosures of which are incorporated herein by reference. Such operations may include counting products and facilitating product retrieval requests.

[0050] The cabinet 100 may include a housing 102 defining a storage area 102a and a plurality of slots arranged within the housing 102, the storage area 102a being configured to receive a product, each of the slots being configured to receive a corresponding product unit. Each of the corresponding product units is a product package. Optionally, the product package includes a surgical implant, such as an intraocular lens. It should be understood that the intraocular lens is provided only as an example. The present disclosure contemplates that the product package may include other articles or objects, including but not limited to orthopedic implants (e.g., patella, tibia, femur, spine, etc.) or surgical tools.

[0051] The cabinet 100 also includes at least one imaging device 104, which is configured to capture information about the product. Optionally, the cabinet includes a plurality of imaging devices, each of which is configured to capture information about the product in the corresponding area of ​​the storage area 102a of the housing 102. As described herein, the imaging device may be optionally arranged in the entire storage area 102a, and each imaging device is configured to capture images of different areas of the storage area 102a. The imaging device 104 is connected to the controller (discussed below) via one or more communication links. The present disclosure contemplates that the communication link is any suitable communication link. For example, the communication link may be implemented by any medium that facilitates data exchange, including but not limited to a wired link, a wireless link, and an optical link. This allows the imaging device 104 and the controller (discussed below) to exchange data (e.g., image data, control signals). Optionally, the imaging device 104 is a digital camera, such as a universal serial bus (USB) digital camera. The digital camera can capture compressed images (e.g., MJPEG format) or uncompressed images (e.g., UYVY format). An example of a suitable USB digital camera is the See3CAM_130-13MP 4K or more generally the e-con Systems See3Cam series. Another example of a large number of suitable cameras may be a USB 3.1 high resolution autofocus camera. Imaging devices are known in the art and are therefore not described in further detail herein.

[0052] As described above, the cabinet 100 may include a plurality of imaging devices 104, and the imaging devices 104 may be arranged throughout the storage area 102a. Figure 3In FIG. 1 , there are twelve imaging devices 104, of which six imaging devices 104 are arranged on each of the left and right vertical supports 102b of the housing 102. The vertical supports 102b are on the front side of the housing 102, and the imaging devices 104 are installed so that the products in the storage area 102a are within the field of view of the imaging devices 104. Figure 3 , the imaging devices 104 are arranged along the vertical support 102b at equal spacing from one another. Each of the imaging devices 104 is configured to capture images of different regions of the storage area 102a. Such regions may or may not overlap with regions captured by other imaging devices 104. It should be understood that Figure 3 The number and / or arrangement of imaging devices 104 shown in FIG. 1 are provided by way of example only. The present disclosure contemplates providing cabinets having different numbers and / or arrangements of imaging devices.

[0053] In one embodiment, the cabinet may include twelve digital cameras, such as USB cameras, and the digital cameras may be mounted in the cabinet, as described above with respect to Figure 3 Each of the USB cameras is connected to the controller via a peripheral component interconnect (PCI) card through a USB cable (e.g., a communication link). Each PCI includes multiple USB ports (e.g., up to five USB ports). Therefore, the USB camera can be connected to the controller via multiple PCI cards. The present disclosure contemplates connecting one, two, three, four, or five USB cameras to a single PCI card. In one aspect, the controller to which the USB camera is connected is a central processing unit (CPU). The CPU includes a memory and at least one or more processing units (see, e.g., Fig. 20 ). CPUs are well known in the art and are therefore not further described herein. In this aspect, the cabinet performs a full scan with all USB cameras in less than about 2 seconds, wherein the full scan includes a computer vision-based analysis of the presence / absence of one or more product units in the cabinet. In another aspect, the controller to which the USB cameras are connected is a graphics processing unit (GPU). A GPU is a dedicated device for processing images and graphics and includes a memory and at least one or more processing units (see, e.g., Fig. 20). GPUs are well known in the art and are therefore not further described herein. In this aspect, the cabinet performs a full scan with all USB cameras in less than about 1 second, wherein the full scan includes a computer vision-based analysis of the presence / absence of one or more product units in the cabinet. The GPU runs frames at a rate sufficient for applications applicable to the desired user experience of the system, and in this specific non-limiting example runs frames at about 50 frames per second. In either aspect, the USB cameras capture images simultaneously and the images are processed by the controller. It should be understood that the number of USB cameras provided above is merely an example. The present disclosure contemplates providing a cabinet having more than 12 USB cameras (e.g., 13, 14, 15, ..., 24 or more USB cameras) or less than 12 USB cameras (e.g., 11, 10, 9, ..., 1 USB camera).

[0054] The cabinet 100 also includes a controller 106 operatively coupled to at least the imaging device 104, wherein the controller 106 includes a processor and a memory having computer executable instructions stored thereon. The present disclosure contemplates that the HMI 114 includes a computing device (e.g., Fig. 20 The computing device 700 of the housing 102 can be implemented using a CPU or a GPU. In some implementations, the controller 106 can be implemented using a single board computer (SBC), one or more microcontrollers (MCUs), or a combination thereof. A suitable SBC is the AAEONMAX-C246 motherboard SBC from Aaeon Technology Inc. (New Taipei City, Taiwan, China). A suitable MCU is the SPC560P50L5 series 32-bit microcontroller from STMicroelectronics (Geneva, Switzerland). The controller 106 is configured to detect activity within the storage area 102a of the housing 102, and in response to detecting activity within the storage area 102a of the housing 102, control at least one imaging device 104 to start capturing one or more images of the product. As used herein, the detected activity can be an object in the storage area 102a (e.g., a user's hand and / or arm or any other part of the user's body or an extension thereof) and / or a state change within the storage area 102a (e.g., the addition or removal of a product).

[0055] In some implementations described herein, the cabinet 100 uses computer vision to detect activity within the storage area 102a of the housing 102. In these implementations, one or more imaging devices continuously capture images of products within the storage area 102b, and such images are processed with computer vision methods to detect changes (e.g., resupply or removal of products). Optionally, in some implementations, image capture and computer vision methods are triggered in response to activity detection by motion sensors and / or presence sensors described below.

[0056] Optionally, at least one imaging device 104 is used to detect activity within the storage area 102a of the housing 102. For example, the controller 106 can be configured to receive one or more images captured by the at least one imaging device 104, and analyze the received images to identify objects such as body parts of a user and / or product addition / removal. The present disclosure contemplates the use of any known technology to detect activity from images, including but not limited to computer vision. Optionally, activity detection is performed in real time.

[0057] Alternatively or in addition, the cabinet 100 optionally includes a motion sensor 108 configured to detect the presence of an object (e.g., a hand) within the storage area 102a of the housing 102. Optionally, the output of the motion sensor 108 can be used to trigger the computer vision method described below. The motion sensor 108 is connected to the controller 106 via one or more communication links. The present disclosure contemplates that the communication link is any suitable communication link. For example, the communication link can be implemented by any medium that facilitates data exchange, including but not limited to a wired link, a wireless link, and an optical link. This allows the controller 106 and the motion sensor 108 to exchange data (e.g., sensed signals, control signals, etc.). Optionally, the motion sensor 108 is used to detect activities within the storage area 102a of the housing 102. Example motion sensors include passive infrared (IR) or microwave sensors. Passive IR sensors sense temperature changes between background and objects (e.g., human body parts). Microwave sensors continuously emit radio waves and detect frequency shifts in reflected waves, which are the result of moving objects. Motion sensors are known in the art and are therefore not described in further detail below. The present disclosure contemplates that the motion sensor 108 may be disposed on, within, or adjacent to the outer frame of the housing 102. Optionally, the motion sensor 108 is disposed within the storage area 102a. It should be understood that the cabinet may include more than one motion sensor, i.e., a plurality of motion sensors. Optionally, activity detection is performed in real time.

[0058] Alternatively or in addition, the cabinet 100 optionally includes a plurality of presence sensors 110 configured to detect the presence of a corresponding product unit in a corresponding slot of the plurality of slots. Optionally, the output of the presence sensor 110 can be used to trigger the computer vision method described below. The presence sensor 110 is connected to the controller 106 via one or more communication links. The present disclosure contemplates that the communication link is any suitable communication link. For example, the communication link can be implemented by any medium that facilitates data exchange, including but not limited to a wired link, a wireless link, and an optical link. This allows the controller 106 and the presence sensor 110 to exchange data (e.g., sensed signals, control signals, etc.). Optionally, the presence sensor 110 is used to detect activities in the storage area 102a of the housing 102. The present disclosure contemplates that the presence sensor can be arranged on, in, or adjacent to the slot. For example, the presence sensor can be a light emitter and a photodetector. The example presence sensor 110 includes an IR sensor, which includes an IR emitter and an IR photodetector. In such sensors, the electrical characteristics of the IR photodetector can vary according to the intensity of the IR light. Thus, the presence or absence of a product unit can be detected by a change in the output voltage of the IR detector. For example, when a product unit is removed, the IR photodetector senses the IR beam, but when a product unit is present, the IR beam is interrupted, each state being associated with a different detector output voltage. Optionally, activity detection is performed in real time.

[0059] The controller 106 is further configured to extract information from one or more images of the product captured by at least one imaging device 104. The extracted information is used to count the products. These operations may be initiated in response to detecting activity within the storage area 102a, which may be accomplished using computer vision, motion sensors, presence sensors, or a combination thereof. For example, the controller 106 may be configured to receive an image of the product captured by the imaging device 104, analyze the image of the product to extract a corresponding product identifier associated with the corresponding product unit, and count the product based at least in part on the extracted information. The present disclosure contemplates using any known technology to analyze the image of the product, including but not limited to computer vision. Optionally, the image analysis is performed in real time. Optionally, the step of counting the product based at least in part on the extracted information may include decoding the corresponding product identifier associated with the corresponding product unit, and using the corresponding product identifier to associate the corresponding product unit with the corresponding slot. Each of the corresponding product identifiers is a one-dimensional (1D) barcode, a two-dimensional (2D) barcode, a three-dimensional (3D) barcode, a universal product code (UPC), a stock keeping unit (SKU), text, or a graphic.

[0060] Alternatively or in addition, the cabinet 100 optionally includes a plurality of visual indicators 112, which are configured to indicate the corresponding positions of the corresponding product units in the housing 102. The visual indicator 112 is connected to the controller 106 via one or more communication links. The present disclosure contemplates that the communication link is any suitable communication link. For example, the communication link can be implemented by any medium that facilitates data exchange, including but not limited to a wired link, a wireless link, and an optical link. This allows the controller 106 and the visual indicator 112 to exchange data (e.g., control signals, etc.). Each visual indicator in the visual indicator 112 can be a light emitter (e.g., LED) or a graphic display. In addition, each visual indicator in the visual indicator 112 can be arranged on each slot in the corresponding slot, in each slot in the corresponding slot, or adjacent to each slot in the corresponding slot. The visual indicator 112 can be used, for example, to identify a product by lighting in response to a user request. Such lighting can be changed by intensity, flashing on / off, etc. Additionally, visual indicator 112 may also be used to identify products by expiration date (eg, by flashing to identify the product that is closest to the expiration date).

[0061] Alternatively or in addition, the cabinet 100 includes a human-machine interface (HMI) 114 that is configured to provide a communication interface between a user and the inventory management cabinet. As described herein, the HMI 114 is connected to the controller 106 and / or the remote server via one or more communication links. The present disclosure contemplates that the communication link is any suitable communication link. For example, the communication link may be implemented by any medium that facilitates data exchange, including but not limited to wired links, wireless links, and optical links. This allows the HMI 114 and the controller 106 and / or the remote server to exchange data (e.g., sensed data, control signals, etc.). The present disclosure contemplates that the HMI 114 includes a computing device (e.g., Fig. 20 In some implementations, the HMI 114 is a tablet computer. Alternatively, in other implementations, the HMI 114 is a laptop computer, a desktop computer, or a mobile computing device.

[0062] Alternatively or in addition, the cabinet 100 includes an output device such as a speaker 116. The output device is connected to the controller 106 via one or more communication links. The present disclosure contemplates that the communication link is any suitable communication link. For example, the communication link can be implemented by any medium that facilitates data exchange, including but not limited to a wired link, a wireless link, and an optical link. This allows the output device and the controller 106 to exchange data (e.g., control signals, etc.). The controller 106 is further configured to generate an alarm using the output device. Such an alarm can be audible, visual, or a combination thereof.

[0063] Alternatively or in addition, the cabinet includes a power supply, which is optionally disposed in the housing 102. The present disclosure contemplates that the power supply can be used to provide power to one or more of the electronic components described herein. The power supply can optionally be a battery. For example, the cabinet can be configured to be connected to a grid power source (e.g., standard alternating current (A / C) power delivered to a residence / business) during normal operation. The power supply can deliver power to the cabinet in response to an interruption (e.g., a power outage).

[0064] A modular inventory management system is also described herein. The system includes a first inventory management cabinet and a second inventory management cabinet. The present disclosure contemplates that the first inventory management cabinet and the second inventory management cabinet are cabinets as described above. In addition, the system includes a human-machine interface (HMI) configured to provide a communication interface between a user and the first inventory management cabinet and the second inventory management cabinet. Fig.19 A modular system is shown in It is also contemplated that additional inventory management cabinets may be included in the system.

[0065] Also described herein is an automated method for inventory management. The method includes detecting activity within a storage area of ​​a housing of an inventory management cabinet; in response to detecting activity within the storage area of ​​the housing, automatically initiating capture of one or more images of products within the storage area of ​​the housing; and counting the products based at least in part on using information extracted from the one or more images of the products. The method optionally also includes providing an inventory management cabinet. The present disclosure contemplates that the inventory management cabinet is a cabinet as described above.

[0066] See now Fig.21 , shows an example operating environment for the specific implementations described herein. Fig.21 As shown in FIG. 1 , the inventory management cabinet 2100, the human-machine interface 2102, and the remote system 2104 may be operably connected by one or more networks 2150. The inventory management cabinet 2100 is described in detail below. For example, the inventory management cabinet 2100 may be the same as described above with respect to FIG. Figures 1 to 12 The cabinet 100 described herein or with respect to FIG. 22A to FIG. 24C In addition, the human-machine interface 2102 can be the cabinet 2200 described above. Figures 1 to 12 The remote system 2104 may be a computing device (e.g., Fig. 202104A. Alternatively or in addition thereto, the remote system 2104 may include or access an electronic medical record (EMR) 2104B.

[0067] As described above, the inventory management cabinet 2100, the human-machine interface 2102, and the remote system 2104 discussed above may be connected via one or more networks 2150. The present disclosure contemplates that the network 2150 is any suitable communication network. These networks may be similar to each other in one or more aspects. Alternatively or in addition, these networks may be different from each other in one or more aspects. The network 2150 may include a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a metropolitan area network (MAN), a virtual private network (VPN), etc., including a portion or combination of any of the above networks. The inventory management cabinet 2100, the human-machine interface 2102, and the remote system 2104 may be connected to the network 2150 via one or more communication links. The present disclosure contemplates that the communication link is any suitable communication link. For example, the communication link may be implemented by any medium that facilitates data exchange, including but not limited to a wired link, a wireless link, and an optical link. Example communication links include but are not limited to a LAN, a WAN, a MAN, Ethernet, the Internet, or any other wired or wireless link, such as WiFi, WiMax, 3G, 4G, or 5G.

[0068] The present disclosure contemplates that the inventory management cabinet 2100, the human-machine interface 2102, and the remote system 2104 can interact to perform the inventory and shipping / distribution functions as described in US2019-0311316, the disclosure of which is expressly incorporated herein by reference in its entirety. For example, as described below, the remote system 2104 can manage / maintain a database 2104A reflecting the inventory of products (e.g., contact lenses) stored in the inventory management cabinet 2100. By exchanging messages via the network 2150, the remote system 2104 can receive messages with product inventory updates from the inventory management cabinet 2100. The remote system 2104 can also query the database 2104A in response to requests from the inventory management cabinet 2100 and / or the human-machine interface 2102. The present disclosure contemplates that a user (e.g., a healthcare professional, such as an eye care professional (ECP)) can interact with the inventory management cabinet 2100 and / or the remote system 2104 using the human-machine interface 2102. For example, the human-machine interface 2102 may run an application and / or use a web browser to interact with the inventory management cabinet 2100 and / or the remote system 2104.

[0069] As described herein, a controller of an inventory management cabinet (e.g., Figures 1 to 12 The controller 106 of the embodiment may be configured to inventory the products based at least in part on the information about the products and cause a visual indicator (e.g., Figures 1 to 12 One or more visual indicators in the visual indicator 112 of the inventory management cabinet 2100 are actuated. An example process is now described. First, a user (e.g., an ECP) ​​inputs a request for a desired product unit using the human-machine interface 2102. The human-machine interface 2102 may send the request for the desired product unit to the inventory management cabinet 2100 via a network. The controller of the inventory management cabinet 2100 may be configured to receive the request for the desired product unit. In addition, the controller may be further configured to send the request for the desired product unit to the remote system 2104 via the network. As described herein, the remote system 2104 may include and / or access an inventory database such as database 2104A. The remote system may query the database to determine the location of the desired product unit within the inventory management cabinet 2100. The remote system may send a response to the controller via the network, and the controller may receive a response including one or more locations of one or more desired product units within the storage area. It should be understood that such locations may include specific slots located in the storage area of ​​the cabinet. As described herein, the controller may be configured to send a signal to actuate the visual indicator to help the user identify the location of the desired product unit. These visual indicators highlight the location of the desired product unit for the benefit of the user.

[0070] The user then removes the desired product unit. As described herein, activity within the storage area of ​​the cabinet can be detected by the inventory management cabinet 2100, for example using an imaging device, a motion sensor, and / or a presence sensor. This causes the controller to initiate computer vision processing. By initiating computer vision processing, the inventory management cabinet 2100 can read / decode machine-readable labels (e.g., barcodes, UPCs, SKUs, text, graphics) associated with the product units. The corresponding product units can then be associated with corresponding locations within the storage area. The corresponding location of each of the product units can then be sent by the controller to the remote system. In other words, the controller can be configured to send an updated inventory list of products to the remote system via a network, and the database can be updated accordingly.

[0071] Alternatively or in addition, the inventory management cabinet 2100 can be easily restocked. For example, a user (e.g., an ECP) ​​can restock products by placing product packages in any empty slot in the storage area. Unlike conventional storage systems, storage does not need to be organized in any way, such as by prescription, power, type, etc. Instead, product packages can be randomly placed in the storage area. When activity is detected within the storage area, the controller can initiate computer vision processing. By initiating computer vision processing, the inventory management cabinet 2100 can read / decode machine-readable labels (e.g., barcodes, UPCs, SKUs, text, graphics) associated with product units. The corresponding product units can then be associated with corresponding locations within the storage area. The corresponding location of each of the product units can then be sent by the controller to a remote system. In other words, the controller can be configured to send an updated inventory list of products to a remote system via a network, and the database can be updated accordingly.

[0072] As described herein, the imaging device of the inventory management cabinet 2100 may be a digital camera capable of capturing images of machine-readable product identifiers (such as 1D barcodes, 2D barcodes, 3D barcodes, UPCs, or SKUs). The imaging device may also be capable of capturing images of text and / or graphics, which may be used as machine-readable product identifiers. For example, the text and / or graphics may include, but are not limited to, brand names, product names, product descriptions, logos, etc. In these specific implementations, image processing techniques may be used to decode machine-readable product identifiers. Therefore, the step of counting products based at least in part on information about the products may include: receiving an image of the product captured by the imaging device; analyzing the image of the product to identify a corresponding product identifier associated with a corresponding product unit; decoding the corresponding product identifier associated with the corresponding product unit. After analyzing / decoding the corresponding product identifier, the corresponding product unit may be associated with a corresponding location within the storage area. The present disclosure contemplates performing such association with a controller and / or remote system 2104.

[0073] Optionally, in some embodiments, the step of counting the products based at least in part on information about the products further includes cropping a portion of the image of the product. By cropping the image, it is possible to focus on the portion of the image that is expected to contain the product identifier. Therefore, the cropped portion of the image is analyzed to identify the corresponding product identifier associated with the corresponding product unit. In addition, the controller may be configured to send the image of the product to the remote system 2104 via the network. In these embodiments, the image may be stored by the remote system for backup purposes, or the image processing (some or all) may be offloaded from the controller to the remote system. Alternatively or in addition, the controller may be configured to store the image of the product in a memory. In some embodiments, the image may be stored only temporarily (e.g., to allow image processing) and then rewritten to minimize storage requirements at the inventory management cabinet 2100.

[0074] Optionally, in some implementations, the step of counting the products based at least in part on the information about the products further includes analyzing the image of the product to identify one or more of the corresponding positions in the storage area associated with the missing, unrecognized or unreadable product identifier. Optionally, the controller can be configured to distinguish between missing product units and product units with unrecognized / unreadable product identifiers. It should be understood that the former can be replenished, while the latter can be repositioned (e.g., flipped, re-labeled) to correctly orient the product identifier for computer vision processing to read. For example, a machine learning algorithm can be used to determine whether one or more of the corresponding positions in the storage area associated with the missing, unrecognized or unreadable product identifiers accommodates a product unit. The present disclosure contemplates that in some implementations, the machine learning algorithm (e.g., pattern recognition) can be performed by the controller 109 using a conventional vision system, while in other implementations, the machine learning algorithm can be performed by a remote system (i.e., unloaded from the inventory management cabinet 2100). The machine learning algorithm can be trained using an existing data set to perform a specific task, such as identifying a missing, unrecognized or unreadable product identifier. Machine learning algorithms are known in the art and are therefore not described in further detail below. An example machine learning algorithm is TensorFlow, which is an open source machine learning algorithm known in the art. TensorFlow is only an example machine learning algorithm. The present disclosure contemplates the use of other machine learning algorithms, including but not limited to neural networks, support vector machines, nearest neighbor algorithms, supervised learning algorithms, and unsupervised learning algorithms.

[0075] Optionally, in some implementations, the step of counting the products based at least in part on information about the products further includes analyzing images of the products to determine the source of each product unit in the corresponding product units using a machine learning algorithm. This is particularly useful when, for example, the products originate from multiple suppliers or manufacturers. In other words, the inventory management cabinet 2100 can be used to store products from different sources (e.g., contact lenses from different manufacturers). As described above, a computer vision system including an imaging device (such as a camera) can be used to capture images of machine-readable codes (barcodes, UPCs, SKUs) and both text and graphics, and then the product identifier can be decoded using imaging processing techniques. The present disclosure contemplates that machine learning algorithms can be used to identify machine-readable codes associated with different suppliers or manufacturers. This allows the inventory management cabinet 2100 to select appropriate decoding rules. Alternatively or in addition, the machine learning algorithm can be used to identify the source of the product unit based on text and / or graphics (even in the absence of a machine-readable code). The present disclosure contemplates that in some implementations, the machine learning algorithm can be executed by a controller, while in other implementations, the machine learning algorithm can be executed by a remote system (i.e., unloaded from the inventory management cabinet 2100). Machine learning algorithms can be trained using existing data sets to perform specific tasks, such as identifying the source of a product unit. Machine learning algorithms are known in the art and are therefore not described in further detail below. Example machine learning algorithms are provided above.

[0076] See now FIG. 22A to FIG. 24C , describes an example inventory management cabinet 2200 according to another specific implementation. FIG. 22A to FIG. 24C The inventory management cabinet of the present invention uses computer vision to count the products contained in the cabinet described in this article.

[0077] Fig.22A An aspect is illustrated in which a cabinet 2200 includes a plurality of drawers 2202 (i.e., 4 drawers) and a virtual storage area 2204. The cabinet dimensions are 1721.50 mm (67.8 inches) in height, 600 mm (23.6 inches) in width, and 650 mm (25.6 inches) in depth. The dimensions within the drawer 2202 are 982 mm (38.66 inches) in width and 545 mm (21.46 inches) in depth. It should be understood that the configuration (e.g., number of drawers) and dimensions are provided as examples only. The present disclosure contemplates providing cabinets with different configurations and / or dimensions. In Fig.22A 2, two imaging devices 2206 are mounted at the top of the cabinet 2200. The present disclosure contemplates providing a cabinet with more or less than two imaging devices. The imaging devices 2206 are arranged to capture information about the products located within the drawer 2202. Optionally, the imaging devices 2206 are mounted flush with the top of the cabinet 2200. For example, each of the imaging devices 2206 is capable of capturing Fig.22AAn autofocus digital camera is shown with an image of the interior of any of the four drawers 2202. Fig.22A , the spacing between the auto-focus digital cameras is 491 mm (19.3 inches). It should be understood that this spacing is provided only as an example and may have another value. Optionally, the digital camera does not capture an image of the virtual storage area 2204.

[0078] Fig. 22B Another aspect is illustrated, in which a cabinet 2200 includes a plurality of drawers 2202 (i.e., 4 drawers) without virtual storage areas. The cabinet dimensions are 1413.50 mm (55.6 inches) in height, 600 mm (23.6 inches) in width, and 650 mm (25.6 inches) in depth. The dimensions within the drawer 2202 are 982 mm (38.66 inches) in width and 545 mm (21.46 inches) in depth. It should be understood that the configuration (e.g., the number of drawers) and dimensions are provided as examples only. The present disclosure contemplates providing cabinets with different configurations and / or dimensions. In Fig. 22B 2202. In the embodiment of the present invention, three imaging devices 2206 are mounted at the top of the cabinet 2200. The present disclosure contemplates providing a cabinet with more or less than three imaging devices. The imaging devices 2206 are arranged to capture information about the products located within the drawer 2202. Optionally, the imaging devices 2206 are mounted flush with the top of the cabinet 2200. For example, each of the imaging devices 2206 is capable of capturing Fig. 22B An autofocus digital camera is shown with an image of the interior of any of the four drawers 2202. Fig. 22B In FIG. 1 , the spacing between the autofocus digital cameras is 327 mm (12.9 inches). It should be understood that this spacing is provided only as an example and may have another value.

[0079] As described in this article, FIG. 22A to FIG. 22B The image captured by the imaging device 2206 shown in FIG. 2 is sent to a controller (eg, Figures 1 to 12 Optionally, in some implementations, the computer vision methods described herein are processed by a motion sensor (e.g., Figures 1 to 12 motion sensor 108) and / or product sensor (e.g., Figures 1 to 12 Alternatively or in addition, FIG. 22A to FIG. 22B The illustrated cabinet 2200 may include one or more visual indicators, such as LEDs (e.g., Figures 1 to 12 112) to illuminate the location of the desired product within the drawer 2202.

[0080] See now Figures 23 to 24C , describes Fig. 22B2200 and its contents with an open drawer 2202 as shown in the embodiment of the present invention. Each drawer 2202 may include one or more trays 2210. It should be understood that the number and / or arrangement of the trays 2210 in the drawers are provided as examples only. In addition, a dispenser 2212 may be arranged in the tray 2210. The dispenser 2212 forms a slot for holding a product unit. As described herein, the product may be a surgical implant, including but not limited to an IOL or an orthopedic implant. The dispenser 2212 may be formed in different sizes to accommodate product packages of different sizes. Using the tray 2210 and the dispenser 2212, reconfigurable storage may be provided.

[0081] Example computing device

[0082] It should be understood that the logical operations described herein with respect to the various figures may be implemented as (1) as a Fig. 20 The logical operations discussed herein are thus not limited to any particular combination of hardware and software. The specific implementation is a matter of choice depending on the performance and other requirements of the computing device. Thus, the logical operations described herein are variously referred to as operations, structural devices, actions, or modules. These operations, structural devices, actions, and modules may be implemented in software, in firmware, in dedicated digital logic, and in any combination thereof. It should also be understood that more or fewer operations than shown in the drawings and described herein may be performed. The operations may also be performed in a different order than those described herein.

[0083] See also Fig. 20 , illustrates an example computing device 700 on which the methods described herein may be implemented. It should be understood that the example computing device 700 is only an example of a suitable computing environment on which the methods described herein may be implemented. Optionally, the computing device 700 may be a well-known computing system, including but not limited to a personal computer, a server, a handheld or laptop device, a multiprocessor system, a microprocessor-based system, a network personal computer (PC), a minicomputer, a mainframe computer, an embedded system, and / or a distributed computing environment comprising any of a plurality of the above systems or devices. A distributed computing environment enables remote computing devices connected to a communication network or other data transmission medium to perform various tasks. In a distributed computing environment, program modules, application programs, and other data may be stored on local and / or remote computer storage media.

[0084] In the most basic configuration of computing device 700, the computing device typically includes at least one processing unit 706 and system memory 704. Depending on the exact configuration and type of computing device, system memory 704 may be volatile (such as random access memory (RAM)), non-volatile (such as read-only memory (ROM), flash memory, etc.), or some combination of the two. Fig. 20 700 is illustrated by dashed line 702. Processing unit 706 may be a standard programmable processor that performs the arithmetic and logic operations necessary for the operation of computing device 700. Computing device 700 may also include a bus or other communication mechanism for transferring information between the various components of computing device 700.

[0085] The computing device 700 may have additional features / functionality. For example, the computing device 700 may include additional storage devices, such as removable storage devices 708 and non-removable storage devices 710, including but not limited to magnetic or optical disks or tapes. The computing device 700 may also contain certain network connections 716 that allow the device to communicate with other devices. The computing device 700 may also have one or more input devices 714 such as a keyboard, a mouse, a touch screen, etc. One or more output devices 712 such as a display, a speaker, a printer, etc. may also be included. Additional devices may be connected to the bus to facilitate data communication between components of the computing device 700. All of these devices are well known in the art and need not be discussed in detail here.

[0086] Processing unit 706 may be configured to execute program code encoded in a tangible computer-readable medium. A tangible computer-readable medium refers to any medium capable of providing data that causes computing device 700 (i.e., machine) to operate in a particular manner. Various computer-readable media may be used to provide instructions to processing unit 706 for execution. Example tangible computer-readable media may include, but are not limited to, volatile media, non-volatile media, removable media, and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. System memory 704, removable storage device 708, and non-removable storage device 710 are all examples of tangible computer storage media. Example tangible computer-readable recording media include, but are not limited to, integrated circuits (e.g., field programmable gate arrays or application-specific ICs), hard disks, optical disks, magneto-optical disks, floppy disks, magnetic tapes, holographic storage media, solid-state devices, RAM, ROM, electrically erasable program read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage devices, cassettes, magnetic tapes, magnetic disk storage devices, or other magnetic storage devices.

[0087] In an example implementation, processing unit 706 may execute program code stored in system memory 704. For example, a bus may carry data to system memory 704, from which processing unit 706 receives and executes instructions. Data received by system memory 704 may optionally be stored on removable storage device 708 or non-removable storage device 710 before or after execution by processing unit 706.

[0088] It should be understood that the various techniques described herein can be implemented in combination with hardware or software, or in appropriate cases, their combination. Therefore, the method and apparatus of the subject disclosed in the present invention or some aspects or parts thereof may be embodied in a tangible medium such as a floppy disk, a CD-ROM, a hard disk drive, or any other machine-readable storage medium in the form of program code (i.e., instructions), wherein when the program code is loaded into a machine (such as a computing device) and executed by the machine, the machine becomes a device for practicing the subject disclosed in the present invention. In the case of executing program code on a programmable computer, the computing device generally includes a processor, a storage medium (including volatile and non-volatile memory and / or storage elements) readable by the processor, at least one input device, and at least one output device. One or more programs may be implemented or utilized in combination with the process described in the subject disclosed in the present invention, for example, by using an application programming interface (API), a reusable control, etc. Such programs may be implemented in a high-level procedural language or an object-oriented programming language to communicate with a computer system. However, one or more programs may be implemented in a component or machine language as required. In any case, the language may be a compiled or interpreted language and may be combined with a hardware specific implementation.

[0089] Example

[0090] The following examples are presented to provide a person of ordinary skill in the art with a complete disclosure and a description of how to prepare and evaluate the compounds, compositions, articles, devices and / or methods claimed herein, and these examples are intended to be exemplary only and are not intended to limit the present disclosure. Efforts have been made to ensure accuracy related to numbers (e.g., amounts, temperatures, etc.), but some errors and deviations should be taken into account. Unless otherwise indicated, parts are by weight, temperatures are in ° C or are ambient temperatures, and pressures are at or near atmospheric pressure.

[0091] Example 1

[0092] See now Fig.13, the example smart inventory management cabinet system consists of smart cabinets and IOL boxes deployed on customer sites, cloud backends, and customer applications (network and local interfaces). The system is used to identify and select the entrusted IOL box inventory, and the cabinet control system software reports it to IoT Edge, and it reports the inventory to the cloud in real time. Events (such as adding or removing IOL boxes by users) are tracked by sensors and coordinated with identification information (e.g., barcodes) for appropriate locations and passed to IoT Edge software to manage / mitigate low inventory situations. It uses a camera to read barcodes based on commands received from IoTEdge software through the IoT interface.

[0093] A smart cabinet is an IOL box storage box with smart connectivity. The smart cabinet contains a high-end single board computer (SBC) as the main processor (or any appropriately specified PC). Cameras and speakers are connected to the SBC. The system is connected to a local user interface through a graphical user interface (GUI). The example smart inventory management cabinet system is controlled and monitored through an IoT Edge device connected to the cloud. It also has a wireless connection through a Wi-Fi network. The barcode and the corresponding slot id will be saved in the local cabinet control system (CCS) software (SW).

[0094] ICD (Interface Control Command) commands are initiated from IoT Edge and transmitted to the cabinet control system through the IoT interface. CCS transmits corresponding responses to commands received from IoT Edge to IoT Edge. CCS transmits alarm messages to IoTEdge based on hardware change events.

[0095] For most of the commands received, CCS SW transmits a response / alert message. For any error / critical information, an alert message is sent to IoT Edge, and the ICD command received from IoT edge is transmitted to CCS SW through the IoT interface. However, for the alert message transmitted to IoT Edge, CCS will not receive any confirmation returned from IoT Edge. The example smart inventory management cabinet system has a human machine interface (HMI) as a user interface. The light emitting diode (LED) indicates the slot selection. The camera is connected through the USB port to capture the image and read the IOL box barcode, and the LED indication is used to locate the requested IOL box.

[0096] When CCS SW receives the command for full scan, the camera reads the barcode of the IOL box and identifies the box. Image processing techniques are used for correct identification. The slot id and the corresponding barcode details are retained in the CCS local file.

[0097] System Input

[0098] See now Fig.14The cabinet control system SW is responsible for managing the hardware connected to the cabinet. The CCS SW is connected to the following devices as input:

[0099] The motion sensor is connected to the microcontroller unit (MCU) expansion board through the general purpose input / output (GPIO). The output of the motion sensor communicates through the controller area network (CAN) bus.

[0100] The presence sensor is connected to the MCU expansion board through GPIO. The output of the presence sensor communicates through the CAN bus.

[0101] The camera connects via Universal Serial Bus (USB) via SBC.

[0102] A dual in-line package (DIP) switch is connected in the GPIO of the IO expander, and the MCU obtains the switch output through a serial peripheral interface (SPI) communication for identifying the MCU ID.

[0103] The ICD command received from IoT Edge is shared to CCS SW through a socket. The internal command scheduler is one of the modules in CCSSW, and it performs the following operations to execute commands: adds commands to a message queue (first-in-first-out (FIFO) logic); validates received commands; parses received commands; and transmits corresponding events to corresponding modules to execute commands.

[0104] The camera is connected via a USB 3.0 multi-port adapter. When a full scan query is requested from IoT Edge, CCS calls the image processing engine module. The image processing engine module performs the following operations to obtain the barcode result: trigger the camera to capture an image; capture images in parallel, where the number of cameras for parallel operation should be preset; process the captured image and read the barcode; and transmit the scanned barcode to IoT Edge.

[0105] The motion sensor is connected to the NMI pin of the MCU expansion board. The motion sensor observes the user's motion and stores the motion status in a queue. When motion is detected, the following actions are performed: trigger all MCU expansion boards from sleep mode to wake-up mode; turn on the power of all presence sensors (IR sensors) connected to the MCU expansion; the MCU expansion board transmits a motion detection message on the CAN bus.

[0106] The presence sensor is connected to the GPIO pins of the MCU expansion board. When the user inserts or removes the IOL box, the MCU expansion board identifies the slot ID where the IOL box is inserted or removed based on the presence sensor (IR sensor) input and transmits the corresponding message on the CAN bus. For a camera-only sensor system, the camera can continuously stream to detect changes in the field of view. The algorithm will detect, for example, the removal or placement of a package in the field of view. For a removal event, the system will ignore the product, update the database and transmit the information to the cloud. In the case of loading, the image covering the area of ​​interest from the camera is then passed to other parts of the algorithm to correct optical aberrations, crop, enhance image quality, and decode information about the cropped area. Once decoded, the information can be transmitted to the cloud for updating.

[0107] System Output

[0108] See now Fig.15 , CCS SW is connected to the following devices as output:

[0109] The LED tube lights are connected to the GPIO of the IO expander and are controlled by the MCU through SPI communication.

[0110] The main indicators are connected in the GPIO of the IO expander and controlled by the MCU through SPI communication.

[0111] The slot LEDs are directly connected in the GPIO of the MCU.

[0112] The speakers are connected via the SBC's audio line out.

[0113] The CCS SW communicates directly with the MCU expansion board via the CAN bus. The master indicator is connected to the MCU expansion board via SPI communication. The slot LED is an LED light that shines on the slot ID (or slot group) of the requested IOL box with the corresponding color to inform the user. To achieve this, the ICD command includes the information of the slot LED, color information and optional parameters (such as LED lighting time).

[0114] The CCS SW plays audio through the speaker to give an alert about false pickup of the IOL box. Connect the audio line output of the SBC to the speaker.

[0115] When the camera starts capturing the image of the barcode result for the IOL box, the LED tube light is used to get a better quality image. The light is turned on / off based on the scan query start / end event.

[0116] Process Model

[0117] See now Fig.17, shows the overall software block diagram with interfaces between each block.

[0118] IoT Edge SW is the main interface with users and the cloud. All hardware connected by IoT is managed by IoT Edge. IoT Edge connects to CCS software through IoT interface. IoT interface uses TCP / USB protocol to communicate with IoT Edge.

[0119] ICD (Interface Control Command) commands are initiated from IoT Edge and transmitted to the cabinet system. Responses and alerts are received in response to the transmitted commands and in response to hardware change events.

[0120] The overall system level context diagram is shown in the above figure. For most commands received, CCS SW transmits a response / alert message. For any error / critical information, an alert message will be transmitted to IoT Edge.

[0121] Visual Indicator

[0122] The CCS SW is indirectly connected to the LEDs via the CAN bus. Visual notification is used to locate the requested IOL box with a slot or slot group. To achieve this, the ICD command includes information about the slot LEDs, color information and optional parameters.

[0123] When CCS SW receives LED on / off command from IoT interface, CCS SW triggers the visual indicator to transmit LED on / off message on CAN bus to MCU expansion board to turn on the specified LED with given RGB value through router application.

[0124] Motion detector

[0125] The motion detector is one of the modules in CCS SW. When the motion sensor observes motion movement and verifies the motion change, an interrupt is triggered in the MCU expansion board. The MCU will transmit a message indicating the motion status (such as motion start and motion completion) on the CAN bus. On the SBC, the router application will receive the CAN message and decode the message and transmit the message to the motion detector. The motion detector then transmits the message to the back engine to transmit it to IoTEdge through the IoT interface.

[0126] Presence detector

[0127] The presence sensor is connected through GPIO in the MCU expansion board. The occupancy detector is one of the modules in CCS SW, which transmits the IR sensor current status query to the router application at system startup. The MCU firmware checks the status change of the presence sensor in a polling method. This status change will occur due to IOL box movement (insertion / removal change). The MCU transmits a message on the CAN bus to indicate the presence sensor status, such as box insertion / retrieval. On the SBC, the router application receives the CAN message, decodes it and transmits the message to the occupancy detector.

[0128] alarm

[0129] The Buzzer Alarm is responsible for turning on / off the speaker based on events received from the Command Scheduler. It plays audio through the speaker (unmutes before playing) to give an alert about false pickup of the IOL box. Connect the audio line output of the SBC to the speaker.

[0130] Example 2

[0131] See now Fig.18 and Fig.19 , illustrates an example modular smart inventory management cabinet system integrated with a vision system, cabinet control software, and IoT interface. The integrated system utilizes a vision system to convert physical IOL inventory into inventory data, which the cabinet control system then uploads to the cloud to obtain meaningful chunks. The information chunks can then be accessed via a web application.

[0132] exist Fig.18 In FIG. 1 , a plurality of cameras 1800 are shown. Fig.18There are 12 cameras 1800 in the cabinet. It should be understood that the present disclosure contemplates providing a smart inventory management cabinet system with more or less than 12 cameras 1800. At 1802, the images captured by the camera 1800 are time-stamped and queued for processing by a trained machine learning (ML) algorithm. At 1804, the system performs occupancy detection on all slots in the cabinet, for example, by analyzing the changes in the captured image from an "empty" to an "occupied" state, or vice versa. Occupancy detection is performed 2 times per second. It should be understood that the rate of 2 times per second is provided only as an example. The present disclosure contemplates performing occupancy detection at a higher or lower rate. The images of the slots with changes are then cropped and queued for barcode decoding and / or optical character recognition (OCR). At 1806, the system performs barcode decoding and / or OCR to determine the serial number, expiration date data, model / variant / diopter, etc. of the product package. At 1808, the information about occupancy status and product packages obtained at 1804 and 1806 is stored locally by the system, such as in a database. At 1810, the remote database is updated so that a web-based dashboard can be provided that shows the virtual cabinet status including real-time updates.

[0133] Users can then access this data and use the cabinet features to locate products via pick-to-light technology or identify expired products. Manual product loading or unloading triggers an automatic system response to update inventory data.

[0134] Occupancy Detector Model

[0135] The system is continuously scanned for boxes in the corresponding compartments every 100ms. If the compartment state changes from "empty" to "occupied", the event triggers the tag feature detector model.

[0136] Label feature detector

[0137] This model uses barcodes or OCR to decode label information. If the barcode cannot be decoded, the system will trigger OCR. If more than one meets the positioning criteria, the system will also indicate which box to pick first by flashing the corresponding LED of the identified lens.

[0138] Work Features

[0139] The user can directly locate the product in the cabinet and the cabinet will display its location in the web application and the location of the product in the cabinet via pick-to-light.

[0140] Expired product characteristics

[0141] Expired product location can be requested by the user. The system will identify all expired products in the cabinet by lighting the expired product box red via RGB LED.

[0142] Some of the above and additional aspects of the present invention are further described by the following examples:

[0143] Embodiment 1. An inventory management cabinet, the inventory management cabinet comprising: a shell, the shell defining a storage area, the storage area being configured to receive products; a plurality of slots, the plurality of slots being arranged in the shell, each of the slots being configured to receive a corresponding product unit; at least one imaging device, the at least one imaging device being configured to capture information about the product; and a controller, the controller being operably connected to the at least imaging device, the controller comprising a processor and a memory, the memory having computer executable instructions stored thereon, the computer executable instructions, when executed by the processor, causing the controller to: detect activity within the storage area of ​​the shell; and control the at least one imaging device to initiate capture of one or more images of the product in response to detecting activity within the storage area of ​​the shell.

[0144] Embodiment 2. An inventory management cabinet according to embodiment 1, wherein the at least one imaging device is used to detect activity within the storage area of ​​the housing.

[0145] Embodiment 3. The inventory management cabinet of Embodiment 1 or 2, further comprising a motion sensor configured to detect the presence of an object within the storage area of ​​the housing.

[0146] Embodiment 4. An inventory management cabinet according to embodiment 3, wherein the motion sensor or camera is used to detect activities within the storage area of ​​the housing to compare changes in the live stream of frame images.

[0147] Embodiment 5. According to any one of embodiments 1 to 4, the inventory management cabinet further comprises a plurality of presence sensors configured to detect the presence of the corresponding product unit in a corresponding slot of the plurality of slots.

[0148] Embodiment 6. The inventory management cabinet of Embodiment 5, wherein the presence sensor is used to detect activity within the storage area of ​​the housing.

[0149] Embodiment 7. An inventory management cabinet according to Embodiment 5 or 6, wherein each of the presence sensors comprises a light emitter and a photodetector.

[0150] Embodiment 8. An inventory management cabinet according to any one of Embodiments 1 to 7, wherein the memory has additional computer executable instructions stored thereon, which, when executed by the processor, cause the controller to extract information from the one or more images of the product captured by the at least one imaging device.

[0151] Embodiment 9. The inventory management cabinet of Embodiment 8, wherein extracting information from the one or more images of the product captured by the at least one imaging device comprises:

[0152] receiving the one or more images of the product from the at least one imaging device; and

[0153] The one or more images of the product are analyzed to extract respective product identifiers associated with the respective product units.

[0154] Embodiments The inventory management cabinet of Embodiment 9, wherein the memory has additional computer executable instructions stored thereon, which when executed by the processor cause the controller to inventory the products based at least in part on the extracted information.

[0155] Embodiment 11. An inventory management cabinet according to embodiment 10, wherein counting the products based at least in part on the extracted information includes: decoding the corresponding product identifier associated with the corresponding product unit; and using the corresponding product identifier, associating the corresponding product unit with the corresponding slot.

[0156] Embodiment 12. An inventory management cabinet according to any one of Embodiments 9 to 11, wherein each of the corresponding product identifiers is a one-dimensional (1D) barcode, a two-dimensional (2D) barcode, a three-dimensional (3D) barcode, a universal product code (UPC), a stock keeping unit (SKU), text, or a graphic.

[0157] Embodiment 13. According to any one of embodiments 1 to 12, the inventory management cabinet also includes a plurality of imaging devices, each of the imaging devices being configured to capture information about the products located in a corresponding area of ​​the storage area of ​​the housing.

[0158] Embodiment 14. According to any one of embodiments 1 to 13, the inventory management cabinet also includes a plurality of visual indicators, and the plurality of visual indicators are configured to indicate the corresponding positions of the corresponding product units within the housing.

[0159] Embodiment 15. The inventory management cabinet of Embodiment 14, wherein the housing includes an outer frame, and wherein at least one of the visual indicators is disposed on or adjacent to the outer frame.

[0160] Embodiment 16. An inventory management cabinet according to Embodiment 14, wherein a corresponding visual indicator is arranged on each of the corresponding slots, within each of the corresponding slots, or adjacent to each of the corresponding slots.

[0161] Embodiment 17. The inventory management cabinet of any one of Embodiments 14 to 16, wherein each of the visual indicators is at least one of a light emitter or a graphic display.

[0162] Embodiment 18. The inventory management cabinet according to any one of embodiments 1 to 17, further comprising an output device configured to provide a visual alarm or an audible alarm.

[0163] Embodiment 19. An inventory management cabinet according to Embodiment 18, wherein the memory has additional computer executable instructions stored thereon, which, when executed by the processor, cause the controller to generate an alarm signal and send the alarm signal to the output device.

[0164] Embodiment 20. According to any one of embodiments 1 to 19, the inventory management cabinet further includes a human-machine interface (HMI), and the human-machine interface is configured to provide a communication interface between a user and the inventory management cabinet.

[0165] Embodiment 21. An inventory management cabinet according to any one of Embodiments 1 to 20, wherein each of the corresponding product units is a product package.

[0166] Embodiment 22. The inventory management cabinet of Embodiment 21, wherein the product packages include surgical implants.

[0167] Embodiment 23. The inventory management cabinet of Embodiment 22, wherein the surgical implant is an intraocular lens.

[0168] Embodiment 24. The inventory management cabinet of Embodiment 22, wherein the surgical implant is an orthopedic implant.

[0169] Embodiment 25. The inventory management cabinet of Embodiment 21, wherein the product packaging comprises surgical instruments.

[0170] Embodiment 26. An inventory management cabinet according to any one of Embodiments 1 to 25, wherein the memory has additional computer executable instructions stored thereon, and the additional computer executable instructions, when executed by the processor, cause the controller to send the inventory of the product to a remote system via a network.

[0171] Embodiment 27. The inventory management cabinet of Embodiment 26, wherein the remote system comprises a database.

[0172] Embodiment 28. An inventory management cabinet according to any one of Embodiments 1 to 27, wherein the memory has additional computer executable instructions stored thereon, which, when executed by the processor, cause the controller to receive a request for the desired product unit.

[0173] Embodiment 29. An inventory management cabinet according to Embodiment 28, wherein the memory has additional computer executable instructions stored thereon, and the additional computer executable instructions, when executed by the processor, cause the controller to: send a request for a desired product unit to a remote system; and receive a response from the remote system, the response including the slot in which the desired product unit is located.

[0174] Embodiment 30. The inventory management cabinet of Embodiment 29, wherein the remote system comprises a database.

[0175] Embodiment 31. A modular inventory management system, comprising: a first inventory management cabinet according to any one of Embodiments 1 to 30; a second inventory management cabinet according to any one of Embodiments 1 to 30; and a human-machine interface (HMI), wherein the human-machine interface is configured to provide a communication interface between a user and the first inventory management cabinet and the second inventory management cabinet.

[0176] Embodiment 32. An automated method for inventory management, the automated method comprising:

[0177] Detecting activity within a storage area of ​​a housing of an inventory management cabinet; automatically initiating capture of one or more images of products within the storage area of ​​the housing in response to detecting activity within the storage area of ​​the housing; and counting the products based at least in part on using information extracted from the one or more images of the products.

[0178] Embodiment 33. A method according to Embodiment 32, wherein the inventory management cabinet is an inventory management cabinet according to any one of Embodiments 1 to 30.

[0179] Embodiment 34. The method according to Embodiment 32 or Embodiment 33, the method further comprising providing an inventory management cabinet according to any one of Embodiments 1 to 30.

[0180] Embodiment 35. An inventory management cabinet, comprising: a shell defining a storage area, the storage area being configured to receive products; a plurality of drawers, the plurality of drawers being arranged within the shell, each of the drawers being configured to receive the corresponding product unit; a plurality of imaging devices, the plurality of imaging devices being configured to capture information about the products; and a controller, the controller being operably connected to at least the imaging devices, the controller comprising a processor and a memory, the memory having computer executable instructions stored thereon, the computer executable instructions, when executed by the processor, causing the controller to: detect activity within the storage area of ​​the shell; and in response to detecting activity within the storage area of ​​the shell, control the imaging device to initiate capture of one or more images of the product.

[0181] Although the subject matter of the present invention is described in terms of structural features and / or method steps, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or steps described above. Rather, the specific features and steps described above are disclosed as example forms of implementing the claims.

Claims

1. An inventory management cabinet, comprising: a housing defining a storage area configured to receive a product; a plurality of slots disposed within the housing, each of the slots being configured to receive a respective product unit; at least one imaging device configured to capture information about the product; and a controller operably coupled to at least the imaging device, the controller comprising a processor and a memory having computer executable instructions stored thereon that, when executed by the processor, cause the controller to: detecting activity within the storage area of ​​the housing; as well as In response to detecting activity within the storage area of ​​the housing, the at least one imaging device is controlled to initiate capture of one or more images of the product.

2. The inventory management cabinet of claim 1, wherein the at least one imaging device is used to detect activity within the storage area of ​​the housing.

3. The inventory management cabinet according to claim 1 further comprises a motion sensor configured to detect the presence of an object within the storage area of ​​the housing, wherein the motion sensor is used to detect activity within the storage area of ​​the housing.

4. The inventory management cabinet of claim 1, further comprising a camera configured to detect the presence of an object within the storage area of ​​the housing, wherein the camera is used to detect activity within the storage area.

5. The inventory management cabinet of claim 1, further comprising a plurality of presence sensors configured to detect the presence of the corresponding product unit in a corresponding slot of the plurality of slots.

6. The inventory management cabinet of claim 5, wherein the presence sensor is used to detect activity within the storage area of ​​the housing.

7. The inventory management cabinet of claim 5, wherein each of the presence sensors comprises a light emitter and a photodetector.

8. An inventory management cabinet according to claim 1, wherein the memory has additional computer executable instructions stored thereon, which, when executed by the processor, cause the controller to extract information from the one or more images of the product captured by the at least one imaging device.

9. The inventory management cabinet of claim 8, wherein extracting information from the one or more images of the product captured by the at least one imaging device comprises: receiving the one or more images of the product from the at least one imaging device; as well as The one or more images of the product are analyzed to extract respective product identifiers associated with the respective product units.

10. The inventory management cabinet of claim 9, wherein the memory has additional computer executable instructions stored thereon that, when executed by the processor, cause the controller to inventory the products based at least in part on the extracted information.

11. The inventory management cabinet of claim 10, wherein counting the products based at least in part on the extracted information comprises: decoding the respective product identifier associated with the respective product unit; as well as The respective product unit is associated with the respective slot using the respective product identifier.

12. An inventory management cabinet according to claim 9, wherein each of the corresponding product identifiers is a one-dimensional (1D) barcode, a two-dimensional (2D) barcode, a three-dimensional (3D) barcode, a universal product code (UPC), a stock keeping unit (SKU), text, or a graphic.

13. The inventory management cabinet of claim 1, further comprising a plurality of imaging devices, each of the imaging devices being configured to capture information about the products located in a corresponding area of ​​the storage area of ​​the housing.

14. The inventory management cabinet of claim 1, further comprising a plurality of visual indicators configured to indicate respective positions of the respective product units within the housing.

15. The inventory management cabinet of claim 14, wherein the housing includes an outer frame, and wherein at least one of the visual indicators is disposed on or adjacent to the outer frame.

16. An inventory management cabinet according to claim 14, wherein a corresponding visual indicator is arranged on each of the corresponding slots, within each of the corresponding slots, or adjacent to each of the corresponding slots.

17. The inventory management cabinet of claim 14, wherein each of the visual indicators is at least one of a light emitter or a graphical display.

18. The inventory management cabinet of claim 1, further comprising an output device configured to provide a visual alert or an audible alert.

19. The inventory management cabinet of claim 18, wherein the memory has additional computer executable instructions stored thereon that, when executed by the processor, cause the controller to generate an alarm signal and send the alarm signal to the output device.

20. The inventory management cabinet of claim 1, further comprising a human-machine interface (HMI) configured to provide a communication interface between a user and the inventory management cabinet.

21. The inventory management cabinet of claim 1, wherein each of the respective product units is a product package.

22. The inventory management cabinet of claim 21, wherein the product packages include surgical implants.

23. The inventory management cabinet of claim 22, wherein the surgical implant is an intraocular lens.

24. The inventory management cabinet of claim 22, wherein the surgical implant is an orthopedic implant.

25. The inventory management cabinet of claim 21, wherein the product packages include surgical tools.

26. The inventory management cabinet of claim 1, wherein the memory has additional computer executable instructions stored thereon, which when executed by the processor cause the controller to send the inventory of the product to a remote system over a network.

27. The inventory management cabinet of claim 26, wherein the remote system comprises a database.

28. The inventory management cabinet of claim 1, wherein the memory has additional computer executable instructions stored thereon that, when executed by the processor, cause the controller to receive a request for the desired product unit.

29. The inventory management cabinet of claim 28, wherein the memory has additional computer executable instructions stored thereon that, when executed by the processor, cause the controller to: sending the request for the desired product unit to a remote system over a network; and A response is received from the remote system, the response including the slot in which the desired product unit is located.

30. The inventory management cabinet of claim 29, wherein the remote system comprises a database.

31. A modular inventory management system, the modular inventory management system comprising: The first inventory management cabinet according to claim 1; The second inventory management cabinet according to claim 1; and A human machine interface (HMI) configured to provide a communication interface between a user and the first inventory management cabinet and the second inventory management cabinet.

32. An automated method for inventory management, the automated method comprising: detecting activity within a storage area of ​​a housing of an inventory management cabinet; automatically initiating capture of one or more images of products within the storage area of ​​the housing in response to detecting activity within the storage area of ​​the housing; as well as The products are inventoried based at least in part on using information extracted from the one or more images of the products.

33. The method of claim 32, wherein the inventory management cabinet is an inventory management cabinet according to claim 1.

34. The method of claim 32, further comprising providing an inventory management cabinet of claim 1.

35. An inventory management cabinet, comprising: a housing defining a storage area configured to receive a product; a plurality of drawers disposed within the housing, each of the drawers being configured to receive a respective product unit; a plurality of imaging devices configured to capture information about the product; and a controller operably coupled to at least the imaging device, the controller comprising a processor and a memory having computer executable instructions stored thereon that, when executed by the processor, cause the controller to: detecting activity within the storage area of ​​the housing; as well as In response to detecting activity within the storage area of ​​the housing, the imaging device is controlled to initiate capture of one or more images of the product.

Images