Inventory management method and system having synchronized shelving and sensor activation

Abstract

The present invention relates to an inventory management method, comprising the steps of controlling shelf devices arranged on an equipment shelf to cause the shelf devices to execute a location process, controlling a sensor to acquire a sensor signal during the location process, and locating each of the shelf devices based on the acquired sensor signal, wherein the step of controlling the shelf devices comprises the step of sending a device command to each of the shelf devices, the device command including a location process execution time, causing each of the shelf devices to switch from a sleep mode to a wake-up mode during the location process execution time and to emit a location signal before switching back to sleep mode, and the step of controlling the sensor comprises the step of sending a sensor command to the sensor (CAM), the sensor command including the location process execution time, causing the sensor to switch from a sleep mode to a wake-up mode during the location process execution time and to capture the location signal emitted by each of the shelf devices before switching back to sleep mode.

Description

Technical Field

[0001] The field of the present invention is the field of inventory management systems and methods for warehouses or retail stores. More specifically, the present invention relates to localizing shelf devices using sensors placed in such warehouses or retail stores, for example, to initialize or update a planogram indicating the arrangement of goods within the spare parts.

Background Art

[0002] There is known an inventory management system based on computer vision technology that captures electronic shelf labels (hereinafter referred to as ESLs) placed in a sales area and creates a real-time planogram of the items currently placed on the shelves of the sales area. Automatic shelf monitoring can improve the availability on the shelves by directly recognizing and marking out-of-stock items in the visual representation of the shelves, thereby triggering reordering of the items.

[0003] One such system is described, for example, in international application WO2019 / 073063A1 in the name of the applicant. In this system, an ESL localization process is carried out, which consists of controlling the ESLs to emit optical signals, such as a specific blinking sequence captured by a camera. Then, image processing techniques are used to recognize the ESLs emitting optical signals within the images captured by the camera, and from this recognition, the position of the items associated with the ESLs within the sales area is derived.

[0004] This localization process is carried out for each of the ESLs placed in the sales area in order to initialize or update a planogram indicating the arrangement of the items within the sales area.

[0005] However, ESLs and cameras are powered by embedded batteries whose service life is defined as limited. Therefore, it is important to maintain this service life as long as possible in order to avoid unexpected replacement costs.

Summary of the Invention

[0006] Therefore, there is a need for an inventory management system that can perform the above-described location process with low power consumption. The present invention aims to satisfy this need, and for this effect, — A step of controlling a shelf device placed on a shelf of equipment and causing the shelf device to perform a location identification process, — A step of controlling the sensor to acquire a sensor signal during the positioning process, - A step of locating each of the shelf devices based on the acquired sensor signals, We propose an inventory management method that includes this.

[0007] The step of controlling the shelf device includes the step of sending a device command to each of the shelf devices. The device command includes a positioning process execution time, during which each of the shelf devices is switched from sleep mode to wake-up mode, and a positioning signal is issued before switching back to sleep mode.

[0008] The step of controlling the sensor includes the step of sending a sensor command to the sensor (CAM). The sensor command includes the positioning process execution time, during which the sensor is switched from sleep mode to wake-up mode, and before switching back to sleep mode, it is caused to capture the positioning signal emitted by each of the shelf devices.

[0009] A preferred but non-limiting embodiment of this method is as follows:

[0010] — The sensor command is transmitted from the synchronization server to the sensor, and the device command is transmitted from the shelf device management server to each of the shelf devices following the reception of the management command from the synchronization server by the shelf device management server.

[0011] — The sensor command and the management command are transmitted by the synchronization server at predetermined intervals before the execution time of the location determination process.

[0012] — Upon receiving the device command, each of the shelf devices schedules a wake-up event based on the time the device command arrived and the time the location identification process was executed.

[0013] — Upon receiving the aforementioned sensor command, the sensor schedules a wake-up event based on the time the sensor command arrived and the execution time of the location determination process.

[0014] — The sensor is a camera configured to capture an image of the fixture during the positioning process, the positioning signal emitted by each of the shelf devices during the positioning process is an optical signal, and positioning each of the shelf devices includes determining the position of each of the shelf devices in the captured image.

[0015] ― Determining the position of each of the shelf devices in the captured image comprises the camera detecting the optical signal emitted by each of the shelf devices at the corresponding position in the captured image.

[0016] —This further includes the camera associating each detected light signal with its corresponding location in a report file and sending the report file to the inventory management server before returning to sleep mode. [Brief explanation of the drawing]

[0017] Other aspects, purposes, advantages, and features of the present invention are provided as non-limiting examples and will become more apparent by reading the following detailed description of preferred embodiments of the invention, which are made with reference to the accompanying drawings.

[0018] [Figure 1]This shows an electronic shelf label equipped with a light indicator that can be used to locate it. [Figure 2] This shows a shelf management system for a sales area. [Figure 3] The steps taken in an inventory management method according to possible embodiments of the present invention are shown. [Modes for carrying out the invention]

[0019] In all the following, we describe inventory management methods and systems for locating shelving devices arranged on equipment shelves, for example, based on computer vision technology. Preferred but non-limiting applications of the present invention relate to shelving devices, e.g., ESLs, that can display item information for customers and / or employees in a sales area. However, the present invention described below can also be considered for inventory management in warehouses using shelving devices that may be placed in areas where specific items are expected to be retrieved, although this is not an essential part of a system for displaying information about products in a sales area, for example.

[0020] In a preferred embodiment, the shelving device is an ESL (Exhibition Storage Unit) placed in the sales area, and the location of the ESL may be used to update or initialize a realogram database that stores correspondences between “matching areas” visible in an acquired image of the equipment (e.g., a gondola) and the ESL, also visible in the image. The matching area of ​​the ESL is the area of ​​the gondola where the item associated with the ESL is expected to be displayed. A predetermined rule for automatic recognition of the matching area of ​​the gondola, starting from a given ESL location, may be used in practice. For example, a rule for identifying the matching area may be defined as follows: For a given detected ESL in the gondola, the zone located directly above the ESL and between the ESL and the adjacent ESL to its right is determined to be the matching area of ​​the ESL in the gondola. This problem can also be considered in reverse; that is, for a given area of ​​the gondola intended to display an item, the corresponding ESL is the nearest ESL located on the left, in the row immediately below the area of ​​the gondola.

[0021] An exemplary electronic shelf label 10 having a light-emitting function is shown in Figure 1. The ESL has a unique label identifier that unmistakably identifies the ESL. Advantageously, the label identifier is visible on the ESL, typically in the form of a scannable barcode. In the case of ESL 10, the label identifier is an eight-digit hexadecimal string, shown in numerical form as a barcode to the right of the ESL. Furthermore, the ESL 10 includes a screen display 11 on the front, which is typically a liquid crystal display or LED display, for displaying information about the goods. This information typically includes the price of the goods in accordance with regulatory requirements, along with other necessary information such as the price per kilogram. The ESL 10 further includes a casing 14 on the rear that houses the electronics necessary to operate the ESL. The electronics include an RF (radio frequency) chip and may further include short-range communication peripherals such as an NFC (near-field communication) or RFID (radio frequency identification) chip. The RF chip allows the ESL to receive RF signals and optionally emit RF signals. The casing 14 also includes a microcontroller for converting the received RF signal into an electronic signal for controlling the screen display 11.

[0022] Furthermore, ESL10 also includes a light-emitting means. The light-emitting means can include two or more light-emitting indicators 12A and 12B. The two light-emitting indicators can emit optical signals that are not necessarily within the visible range in response to commands transmitted to ESL10 via radio frequency. The two light-emitting indicators may typically be LED light sources, or the light-emitting indicators may be any other type of light source that can emit detectable optical signals for a short period of time. In the case of ESL10, the two light sources are arranged on the front side of the ESL. Alternatively, an ESL having only one light-emitting indicator can also be used in the method described below. The light-emitting indicators 12A and 12B of ESL10 are connected to a control circuit configured to receive electronic signals from the RF peripheral devices of the ESL. In this way, the light-emitting means, in this case the light-emitting indicators 12A and 12B, can be lit in response to commands transmitted to ESL10 via radio frequency. Advantageously, the light-emitting means (here the light-emitting indicators 12A and 12B) is configured to emit optical signals in one or more colors. For example, indicator 12A can emit light in blue and indicator 12B can emit light in red. Alternatively, one or more light-emitting indicators can emit light in a predetermined timing pattern (i.e., a series of emissions of optical signals of a predetermined duration) in order to facilitate the recognition of the ESL that emits the optical signal for recognition or to enable the distinction of several ESLs that transmit optical signals simultaneously.

[0023] A system for indicating article information in a gondola in a sales area is shown in FIG. 2. This system includes an electronic shelf labeling subsystem 1 arranged on one or more gondolas, an ESL management server 2 (hereinafter also referred to as a shelf device management server), and an inventory management server 4 that can be part of the system in the sales area or outside the sales area. Both servers 2 and 4 can communicate with each other via a wired connection, a wireless connection, or a combination thereof.

[0024] In the example shown in Figure 2, the electronic shelf labeling subsystem 1 comprises three ESLs 10 positioned on shelf rails 13 located on the edges of shelves for indicating items, situated in rows of gondolas. Preferably, shelf rails similar to shelf rail 13, on which the ESLs are placed, are distributed throughout the sales area and configured to ensure consistency between product information provided throughout the sales area and information stored in the ESL management server 2. Each ESL 10 corresponds to one item being sold, which can be identified by a specific item identifier, such as an EAN code. A mobile terminal 3, typically a smartphone, may be used to obtain the label identifier of the ESL and to communicate remotely with the ESL management server 2. Communication between the mobile terminal 3 and the ESL management server 2 can be performed via any communication network, such as Wi-Fi, 3G or 4G, or DECT. The mobile terminal 3 can obtain the label identifier of the ESL by image reading of a barcode located on the ESL, or alternatively, by short-range communication with the ESL, such as NFC short-range communication with the ESL's NFC chip. The ESL management server 2 can communicate wirelessly with each ESL 10. In fact, the ESL management server 2 can transmit an RF signal (radio frequency signal) containing a label identifier, and as a result, the information contained in the RF signal is specifically received by the ESL identified by the label identifier. RF communication between the ESL management server 2 and the ESL can be performed within any known radio frequency range. For example, the electronic price display management server 2 can communicate with the electronic price display 10 within the range of 700 to 1000 MHz, particularly at frequencies of 779 MHz, 868 MHz, or 915 MHz. Alternatively, the frequency range can correspond to very high frequencies such as 2.4 GHz (Wi-Fi frequency). In another alternative example, the frequency range can correspond to low frequencies below 40 kHz.

[0025] The ESL management server shown in FIG. 2 includes a central file DB that stores the association information between the ESLs existing in the sales area and the articles on sale, for example, in the form of a table association between unique label identifiers and the EAN numbers of the articles. The inventory management server 4 includes a real-time log database P that includes the association information between the shelf spaces and the label identifiers of the ESLs. Further, the real-time log database P can also include the association information between the shelf space field and article identifiers such as the EAN numbers of the articles.

[0026] Each shelf space can be identified by three numerical indexes. The first index i corresponds to the gondola in the sales area, and this gondola can include one or a plurality of rows stacked parallel to each other. The second index j corresponds to a specific row j of the gondola i. The third index k corresponds to a specific electronic shelf label within the row j. The combination of the indexes i, j, k in the triplet (i, j, k) enables the clear identification of a single area of the gondola that coincides with a single ESL. The unique label identifier that identifies one ESL in the sales area, associated with the shelf space (i, j, k) of the gondola i in the real-time log database P, can be denoted as Pijk. As described above, each shelf space (i, j, k) is preferably also associated with an article identifier that can be represented as P'ijk. In the latter case, the unique label identifier Pijk is thus associated with the article identifier P'ijk in the central file DB of the inventory management server 4.

[0027] As shown in Figure 2, the inventory management server 4 is separate from the ESL management server 2. Therefore, the communication function with the electronic shelf labels 10, the management (access and modification) function of the central file DB, and other functions of the management (access and modification) of the realogram database P are provided by the two separate servers. The ESL management server 2 is accessible to sales area personnel for the management of ESL 10 and the central file DB, but the inventory management server 4 is not necessarily accessible by sales area personnel, and depends on whether said personnel are also responsible for the management of the realogram database P. The realogram database P can only be used by the administrator of the inventory management server 4, in a state where the administrator of the ESL management server 2 cannot modify the realogram database P.

[0028] In any case, the management of the central file database and the realogram database P may, alternatively, be performed by the same server (ESL management server 2 or inventory management server 4).

[0029] The inventory management method according to the present invention will be described with reference to Figure 3. This method includes the following steps. - Controlling the shelf device AID (e.g., ESL) placed on the shelves of equipment (e.g., fixtures for sales areas such as gondolas) to cause the shelf device to perform a location identification process. — A step of controlling at least one sensor CAM (e.g., a camera configured to acquire an image or sequence of images of an instrument, typically a video sequence showing an instrument) to acquire sensor signals during the localization process. - A step of locating each of the shelf devices AID based on the acquired sensor signals. According to the present invention, controlling the shelves and devices involves transmitting device commands D to Cd to each of the shelves and devices AID. The device commands include a location process execution time, during which each of the shelves and devices is switched from sleep mode SLPd to wake-up mode WU-BLK, and while in wake-up mode, it emits a location signal before switching back to sleep mode. Furthermore, according to the present invention, controlling the (at least one) sensor includes transmitting a sensor command C-Cd to the sensor CAM. The sensor command includes a positioning process execution time, during which the sensor is switched from sleep mode SLPc to wake-up mode WU-CAP, and during the wake-up mode, it captures positioning signals emitted by each of the shelf devices before returning to sleep mode.

[0030] This control may include controlling all shelving units and sensors located in a specific area, such as all shelving units and sensors located in a sales area, for example, all shelving units and sensors located in a retail store.

[0031] As a result of the control of the shelf devices and sensors, all shelf devices and sensors are woken up almost simultaneously to perform the positioning process of all shelf devices at the same time, and then switched to sleep mode. This helps to minimize the startup time of the shelf devices and sensors, and therefore minimizes their battery consumption and improves their service life.

[0032] In the embodiment shown in Figure 3, sensor commands C-Cd are transmitted from the synchronization server CS to the sensor CAM (typically wirelessly, based on cellular technology such as 4G or 5G, or based on WiFi technology), and device commands D-Cd are transmitted from the shelf device management server AIDMS (e.g., ESL management server 2 shown in Figure 2) to each of the shelf device AIDs (e.g., wirelessly, based on a store / warehouse RF network) following the reception of management commands M-Cd from the synchronization server CS by the shelf device management server AIDMS. Note that the synchronization server CS may or may not be separate from the inventory management server 4 shown in Figure 2.

[0033] In another embodiment, sensor commands C-Cd are transmitted from the synchronization server CS to the sensor CAM, and device commands D-Cd are transmitted from the sensor CAM to the shelf device AID located near the sensor, following the sensor CAM receiving the management command M-Cd from the synchronization server CS. In this embodiment, the sensor and the shelf device are equipped with appropriate means, such as short-range radio, to enable communication between the sensor and the nearby shelf device.

[0034] In these two embodiments, the sensor command C-Cd may be sent individually to each sensor, for example, based on the sensor's wake-up cycle.

[0035] The synchronous server CS may be configured to allow the user to set a location process execution time (e.g., 11 p.m.) and select shelf devices (e.g., all ESLs located in a specific store section) on which the location process should be performed. The synchronous server CS may further be configured to allow the user to select at least one sensor that should be involved in the location process. The synchronous server CS may be further configured to initialize the location process by incorporating information indicating the location process execution time, along with information identifying the selected sensor if necessary, into the sensor command, and by incorporating information indicating the location process execution time, along with information identifying the selected shelf device if necessary, into the management command.

[0036] In one embodiment, sensor commands and management commands are transmitted by the synchronization server during a predetermined period prior to the execution time of the location tracking process (for example, 10 minutes before the execution time of the location tracking process).

[0037] Upon receiving the management command M-Cd, the shelf device management server AIDMS sends a device command to each shelf device AID that should perform the location process, for example, by relaying the management command (box RLY in Figure 3). The device command may be a broadcast command sent to all shelf devices involved in synchronization (e.g., a signal with a duration long enough to cover the wake-up period of each shelf device), or a selective multicast or unicast command based on, for example, the shelf device identifier or the shelf device wake-up cycle.

[0038] In one embodiment, the shelf device management server AIDMS then, in an acknowledgment (box ACK - Figure 3), sends a device command from each shelf device that should perform the location process. If no acknowledgment is received from a shelf device, typically after a predetermined period, the shelf device management server AIDMS resends the device command to the shelf device that failed to acknowledge receipt of the device command. The device command may be resent multiple times until it is confirmed that the shelf device has received the device command.

[0039] Upon receiving a device command, the shelf device AID is configured to switch from sleep mode to wake-up mode. The shelf device then sends an acknowledgment (box ACK in Figure 3) to the shelf device management server AIDMS and schedules a wake-up event (boxes DEF-WUd) based on the time the device command arrived and the time the location process is running before returning to sleep mode (box SLPd1). The shelf device AID calculates the time difference (e.g., in milliseconds) between the time the device command was received and the time the location process is running, and sets an internal timer based on the calculated difference so that it can be woken up when the location process is running. When the internal timer counts down to zero, the shelf device wakes up (boxes WU-BLK), runs the location process, and then returns to sleep (box SLPd2).

[0040] After sending sensor command C-Cd, the synchronization server CS waits for an acknowledgment from the sensor from which the sensor command was sent. If no acknowledgment is received from the sensor, typically after a predetermined period, the synchronization server resends the sensor command to the sensor that failed to acknowledge receipt of the sensor command. The sensor command may be resent multiple times until it is confirmed that the sensor has received the sensor command.

[0041] Upon receiving a device command, the sensor CAM is configured to switch from sleep mode to wake-up mode. The sensor then sends an acknowledgment (box ACK in Figure 3) to the synchronization server and schedules a wake-up event (boxes DEF-WUc) based on the sensor command arrival time and the localization process execution time before returning to sleep mode (box SLPc1). The sensor CAM calculates the time difference (e.g., in milliseconds) between the time the sensor command was received and the localization process execution time, and sets an internal timer based on the calculated difference so that it can be woken up when the localization process execution time is reached. When the internal timer counts down to zero, the sensor wakes up (box WU-CAP), performs the localization process (box DTC), and then returns to sleep (box SLPc2).

[0042] In one embodiment, the sensor is a camera configured to capture an image of a fixture during a localization process, the localization signal emitted by each of the shelf devices during the localization process is an optical signal, and localizing each of the shelf devices includes determining the position of each of the shelf devices in the captured image.

[0043] The optical signals emitted by the shelf device (for example, by the light emission indicators 12A and / or 12B of the ESL) may be emitted in the visible range or in the invisible range, such as the infrared range. In a preferred embodiment, the optical signals emitted by the shelf device may include a specific (i.e., shelf device-specific) temporal and / or spatial pattern of optical changes, such as changes in intensity or spectrum, e.g., a specific series of optical pulses.

[0044] Determining the position of each of the shelf devices in the captured image may include the camera detecting an optical signal emitted by each of the shelf devices at the corresponding position in the captured image (box DTC in Figure 3). In this embodiment, the optical signal detection is performed directly in the camera rather than in the synchronization server, which proves useful in avoiding bandwidth issues that arise when transmitting video streams from multiple cameras to the synchronization server.

[0045] The camera CAM may also be configured to associate each detected light signal with its corresponding location in a report file Rp and send the report file to the synchronization server CS (which may or may not be separate from the inventory management server, such as server 4 in Figure 2) before returning to sleep mode (box SLPc2 in Figure 3). Taking a blinking light signal as an example, the report file may include an image with the marked blinking area (i.e., the location corresponding to the detected light signal), along with an on / off pattern for each detected blinking spot.

[0046] The synchronization server can then check whether the detected optical signal is a valid optical signal that matches the signal emitted by the shelving device during the localization process. If yes, the synchronization server can associate the shelving device with its corresponding location in the captured image of the equipment and derive its localization within the sales area / warehouse based on the known location of the camera.

[0047] In one embodiment, the shelving device is an article matching device, e.g., an ESL, positioned on a shelf of fixtures, such that article location can be derived from the location of the associated article matching device. In this embodiment, the synchronization server CS can localize article areas based on the detected optical signal in a report file (typically when the detected optical signal is specific to the article matching device), by identifying the article matching device identifier stored in the database (box CHC in Figure 3), and determining the article identifier associated with the article matching device identifier. The localized article areas can then be used to fill in a planogram showing the arrangement of articles within the fixture.

[0048] For example, additional image processing steps may be performed based on the image captured by the camera to check for the presence of items within a local area (identification of empty areas may be performed by color recognition against a predetermined pattern of the fixture), to check whether the items within the local area are correct (for example, based on image comparison between the image and the product photograph), or to count the products.

[0049] The present invention is not limited to the method described above, but also extends to an inventory management server comprising a processing unit configured to perform the control steps of the method. The present invention further relates to a computer program product having instructions that cause a computer to perform the control steps of the method when the program is executed by the computer.

Claims

1. Inventory management method, The steps include controlling a shelving device placed on a shelf for equipment and causing the shelving device to perform a location identification process, The steps include controlling the sensor to acquire a sensor signal during the positioning process, The step includes locating each of the shelf devices based on the acquired sensor signals, The step of controlling the shelf device includes the step of sending a device command to each of the shelf devices. The device command includes a positioning process execution time, during which each of the shelf devices is switched from sleep mode to wake-up mode, and before switching back to sleep mode, it is caused to emit a positioning signal. The step of controlling the sensor includes the step of sending a sensor command to the sensor (CAM). The sensor command includes the positioning process execution time, during which the sensor is switched from sleep mode to wake-up mode, and before switching back to sleep mode, the sensor is caused to capture the positioning signal emitted by each of the shelf devices.

2. The method according to claim 1, wherein the sensor command is transmitted from the synchronization server to the sensor, and the device command is transmitted from the shelf device management server to each of the shelf devices following the reception of a management command from the synchronization server by the shelf device management server.

3. The method according to claim 2, wherein the sensor command and the management command are transmitted by the synchronization server at predetermined intervals before the execution time of the positioning process.

4. The method according to claim 1, wherein upon receiving the device command, each of the shelf devices schedules a wake-up event based on the device command arrival time and the location identification process execution time.

5. The method according to claim 1, wherein upon receiving the sensor command, the sensor schedules a wake-up event based on the time the sensor command arrived and the time the location determination process was executed.

6. The method according to claim 1, wherein the sensor is a camera configured to capture an image of the equipment during the positioning process, the positioning signal emitted by each of the shelf devices during the positioning process is an optical signal, and positioning each of the shelf devices includes determining the position of each of the shelf devices in the captured image.

7. The method according to claim 6, wherein determining the position of each of the shelf devices in the captured image is performed by the camera detecting the optical signal emitted by each of the shelf devices at the corresponding position in the captured image.

8. The method according to claim 7, further comprising the steps of: the camera associating each detected optical signal with its corresponding position in a report file; and transmitting the report file to the inventory management server before returning to sleep mode.

9. The method according to claim 8, wherein the shelf device is an item identification device placed on the shelf of the equipment, and the item location can be derived from the location of the associated item identification device, and the inventory management server identifies an item identification device identifier stored in the database based on the optical signal detected in the report file, and determines the item identifier associated with the item identification device identifier to locate the item area.

10. The method according to claim 9, wherein a planogram showing the arrangement of articles within the fixture is filled in using the located article regions.

11. A computer program that, when executed by a computer, includes instructions causing the computer to perform the control steps of the method according to claim 1.

12. An inventory management server comprising a processing unit configured to perform the control steps described in claim 1.

Images