Method for positioning imaging device for shelf monitoring and system for positioning imaging device for shelf monitoring

Abstract

An embodiment provides a method for positioning an imaging device for shelf monitoring, the method determining an installation condition of a shelf imaging device for monitoring a shelf and being executed by a server. The method for positioning an imaging device for shelf monitoring comprises the steps of: obtaining related information of at least one shelf disposed inside a store; and determining an installation condition of at least one imaging device for monitoring the at least one shelf, on the basis of the related information of the at least one shelf.

Description

Imaging device positioning method for shelf monitoring and imaging device positioning system for shelf monitoring

[0001] The present disclosure relates to an imaging device positioning method for shelf monitoring and an imaging device positioning system for shelf monitoring, and more specifically, to an imaging device positioning method for shelf monitoring and an imaging device positioning system for shelf monitoring that determines the installation conditions of at least one imaging device monitoring a shelf based on relevant information of a shelf and / or relevant information of an imaging device.

[0002] - Research Project Information

[0003] [Project ID] 1415188782

[0004] [Assignment No.] P0024369

[0005] [Ministry Name] Ministry of Trade, Industry and Energy

[0006] [Name of Project Management (Specialized) Agency] Korea Institute for Industrial Technology Promotion

[0007] [Research Project Name] World Class Plus Project Support

[0008] [Project Title] Development of a Cloud-based Future Retail Integrated Solution Applying Deep Learning Algorithms

[0009] [Name of Implementing Agency] SoluM Co., Ltd.

[0010] [Research Period] April 1, 2023 ~ December 31, 2026

[0011] The use of electronic labels is gradually expanding in applications such as displaying product information on items displayed in stores. Also known as electronic shelf labels or electronic tags, electronic labels connect to a server via a gateway to receive product information and display it on an electronic paper display. Their widespread adoption is attributed to the fact that they operate at low power, allowing for extended battery life, and enable changes to display information via a communication network, thereby reducing labor costs in store management.

[0012] When a new product is displayed on a shelf, a procedure is required to assign it so that the product's information is displayed on an electronic label. Generally, when a store manager reads the barcode attached to the electronic label with a terminal and then reads the product barcode and transmits it to a server, the server assigns the corresponding electronic label to the product, registers it in a database, and processes the product information by transmitting it to the electronic label so that the display is changed.

[0013] In this way, multiple newly displayed products and multiple electronic labels assigned information to those products can be paired and placed on the shelf. Each of the multiple electronic labels includes unique identification information, and the identification information may include location information of the electronic labels.

[0014] A real-gram containing the display status of multiple products arranged on a shelf can be formed based on the location information of the electronic label based on the identification information of the electronic label and the product information assigned to the electronic label.

[0015] Realogram is a real-time product display diagram that includes an image of products displayed on multiple shelves placed in a store, and related information of multiple products and multiple electronic labels included in the image, and can be utilized by a user to manage the store.

[0016] In order to generate such a real program, multiple shelves placed in the store must be photographed using an imaging device, and to do this, multiple imaging devices must be installed at appropriate locations within the store to photograph multiple products and electronic shelf labels displayed on multiple shelves.

[0017] For example, when initially installing multiple imaging devices inside a store, there are methods to determine how many devices to install and where to place them, or to adjust the installation points after the store employee personally checks images of the shelves taken with the installed devices.

[0018] However, according to this method, if store employees with insufficient store management experience install the imaging device or correct the installation location based on captured images, it may take a long time and be difficult to optimize the installation.

[0019] Accordingly, in order to increase the efficiency of store management, there is a need for research on a system that automatically determines the installation conditions of a shelf imaging device to efficiently photograph shelves placed inside the store.

[0020] According to various embodiments of the present disclosure, the present invention aims to provide an imaging device positioning method for shelf monitoring and an imaging device positioning system for shelf monitoring, which determine the installation conditions of at least one imaging device for efficiently monitoring a plurality of shelves arranged inside a store.

[0021] According to various embodiments of the present disclosure, the present invention aims to provide an imaging device positioning method for shelf monitoring and an imaging device positioning system for shelf monitoring, wherein the installation conditions of at least one imaging device are automatically changed based on the analysis results of images taken of a plurality of shelves by at least one imaging device installed inside a store according to the determined installation conditions.

[0022] However, the technical problems that the various embodiments of the present disclosure aim to solve are not limited to the technical problems described above, and other technical problems may exist.

[0023] One embodiment is,

[0024] A method for positioning an imaging device for shelf monitoring, which determines the installation conditions of an imaging device for monitoring a shelf executed by a server, comprises the steps of: acquiring relevant information of at least one shelf placed inside a store; and determining the installation conditions of at least one imaging device for monitoring the at least one shelf based on the relevant information of the at least one shelf.

[0025] In another aspect, the relevant information of the at least one shelf may include at least one of the position, width, height, depth, number of shelf plates, and distance from surrounding shelves of the at least one shelf.

[0026] In another aspect, in the step of determining the installation conditions of the at least one imaging device, the installation conditions of the at least one imaging device may be determined based on the relevant information of the at least one shelf through an imaging device installation condition output model that has been learned based on the relevant information data set of the shelf.

[0027] In another aspect, the imaging device positioning method for shelf monitoring further includes the step of acquiring relevant information of the at least one imaging device, and in the step of determining the installation conditions of the at least one imaging device, the installation conditions of the at least one imaging device may be determined based on the relevant information of the at least one shelf and the relevant information of the at least one imaging device.

[0028] In another aspect, the relevant information of the at least one imaging device may include at least one of the optical characteristics and tilting range of the at least one imaging device.

[0029] In another aspect, in the step of determining the installation conditions of the at least one imaging device, the installation conditions of the at least one imaging device may be determined based on the relevant information of the at least one shelf and the relevant information of the at least one imaging device through a pre-learned imaging device installation condition output model based on the relevant information data set of the shelf and the relevant information data set of the imaging device.

[0030] In another aspect, the imaging device positioning method for shelf monitoring may further include the step of changing the determined installation conditions based on the analysis result of at least one image of the at least one shelf captured by the at least one imaging device installed according to the determined installation conditions.

[0031] In another aspect, the step of changing the determined installation conditions may include comparing the at least one image with the planogram of the at least one shelf, and changing the installation conditions of the at least one imaging device based on the result of the comparison.

[0032] In another aspect, the step of changing the installation conditions of the at least one imaging device based on the above comparison result may include: a step of identifying at least one target product or at least one target electronic shelf label among a plurality of products or a plurality of electronic shelf labels included in the planogram that is not included in the at least one image; a step of obtaining location information of the at least one target product or the at least one target electronic shelf label; and a step of changing the installation conditions of the at least one imaging device so that the at least one target product and the at least one target electronic shelf label can be monitored by the at least one imaging device based on the location information of the at least one target product or the location information of the at least one target electronic shelf label.

[0033] In another aspect, the step of specifying at least one target product or at least one target electronic shelf label may include: detecting a pattern code displayed by at least one electronic shelf label included in at least one image; obtaining at least one identification information corresponding to the detected pattern code; obtaining a plurality of identification information of a plurality of electronic shelf labels included in the planogram; comparing the at least one identification information obtained based on the at least one image with the plurality of identification information obtained based on the planogram; determining at least one target identification information among the plurality of identification information obtained based on the planogram that is not included in the at least one identification information obtained based on the at least one image; and specifying at least one target electronic shelf label corresponding to the at least one target identification information.

[0034] In another aspect, the step of changing the installation conditions of the at least one imaging device may include the step of extracting an overlapping region between a plurality of images of the at least one shelf captured by a plurality of imaging devices installed according to the determined installation conditions, and the step of changing the installation conditions of the at least one imaging device so that a plurality of images of the at least one shelf in which the overlapping region does not overlap can be obtained.

[0035] In another aspect, the imaging device positioning method for shelf monitoring further includes the step of receiving a user input selecting some of a plurality of shelves arranged inside the store, and in the step of obtaining relevant information of at least one shelf, relevant information of at least one shelf selected according to the user input among the plurality of shelves arranged inside the store can be obtained.

[0036] One embodiment is,

[0037] An imaging device positioning system for shelf monitoring is provided, comprising a server including a memory storing instructions and a program for performing an imaging device positioning method and at least one processor performing operations for performing the imaging device positioning method according to the instructions and the program, and an electronic device providing information regarding the installation conditions of at least one imaging device determined by the server.

[0038] In another aspect, the at least one processor may acquire relevant information of at least one shelf and determine the installation conditions of the at least one imaging device for monitoring the at least one shelf based on the relevant information of the at least one shelf.

[0039] In another aspect, the electronic device may be configured to receive user input that inputs relevant information of the at least one shelf.

[0040] In another aspect, the electronic device is configured to receive user input selecting some of a plurality of shelves arranged inside a store, and the at least one processor can obtain relevant information of at least one shelf selected according to the user input among the plurality of shelves arranged inside the store.

[0041] In another aspect, the at least one processor may acquire relevant information of the at least one imaging device and, in determining the installation conditions of the at least one imaging device, determine the installation conditions of the at least one imaging device based on relevant information of the at least one shelf and relevant information of the at least one imaging device.

[0042] In another aspect, the at least one processor receives data of at least one image of the at least one shelf captured by the at least one imaging device installed according to the determined installation conditions, and can change the determined installation conditions based on the analysis results of the at least one image.

[0043] According to various embodiments of the present disclosure, a method for positioning an imaging device for shelf monitoring and a system for positioning an imaging device for shelf monitoring can be provided, which enable optimal monitoring of products and electronic shelf labels arranged on a plurality of shelves by determining the installation conditions of at least one imaging device based on relevant information of a plurality of shelves arranged inside a store and / or relevant information of at least one imaging device.

[0044] According to various embodiments of the present disclosure, a method for positioning an imaging device for shelf monitoring and a system for positioning an imaging device for shelf monitoring can be provided, which can increase the efficiency of shelf monitoring by automatically changing the installation conditions of at least one imaging device based on the analysis results of images of a plurality of shelves captured by at least one imaging device installed inside a store according to determined installation conditions, and systematically optimizing the installation conditions of at least one imaging device.

[0045] However, the effects obtainable through the various embodiments of the present disclosure are not limited to those mentioned above, and other unmentioned effects can be clearly understood from the description below.

[0046] FIG. 1 illustrates an exemplary configuration of an imaging device positioning system for shelf monitoring according to one embodiment.

[0047] FIG. 2 illustrates a plurality of shelves arranged inside a store according to one embodiment.

[0048] FIG. 3 illustrates an exemplary configuration of an electronic shelf label according to one embodiment.

[0049] FIG. 4 is a block diagram illustrating an exemplary configuration of an electronic shelf label according to one embodiment.

[0050] FIG. 5 is a block diagram illustrating an exemplary configuration of a server according to one embodiment.

[0051] FIG. 6 illustrates the configuration of a shelf according to one embodiment.

[0052] FIGS. 7 and 8 are intended to explain an imaging device installation condition output model utilized by a server installation condition determination unit according to one embodiment.

[0053] FIG. 9 is a block diagram illustrating an exemplary configuration of an electronic device according to one embodiment.

[0054] FIG. 10 is a flowchart of an imaging device positioning method for shelf monitoring according to one embodiment.

[0055] FIG. 11 is intended to illustrate the installation conditions of a plurality of imaging devices for a plurality of shelves arranged inside a store according to one embodiment.

[0056] FIG. 12 is a flowchart of an imaging device positioning method for shelf monitoring according to another embodiment.

[0057] FIG. 13 is a flowchart of an imaging device positioning method for shelf monitoring according to another embodiment.

[0058] FIGS. 14 and 15 are intended to illustrate a plurality of imaging devices installed according to determined installation conditions in one embodiment, capturing images of a shelf.

[0059] FIG. 16 is intended to illustrate the appearance of a modified installation condition of a plurality of imaging devices for a plurality of shelves arranged inside a store according to one embodiment.

[0060] FIG. 17 is a flowchart of exemplary steps that may be included in the step of changing the determined installation conditions included in the method of FIG. 13.

[0061] FIG. 18 is intended to explain a method for a plurality of imaging devices according to one embodiment to compare a plurality of images of at least one shelf with a planogram of at least one shelf.

[0062] FIG. 19 is a flowchart of exemplary steps that may be included in the step of specifying at least one target electronic shelf label of FIG. 17.

[0063] FIG. 20 is a flowchart of other exemplary steps that may be included in the step of changing the determined installation conditions included in the method of FIG. 13.

[0064] FIG. 21 is a flowchart of an imaging device positioning method for shelf monitoring according to another embodiment.

[0065] The present invention is capable of various modifications and may have various embodiments; therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various forms. In the following embodiments, terms such as "first," "second," etc., are used not in a limiting sense but for the purpose of distinguishing one component from another. Furthermore, singular expressions include plural expressions unless the context clearly indicates otherwise. Also, terms such as "include" or "have" mean that the features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added. Additionally, in the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily depicted for convenience of explanation, so the present invention is not necessarily limited to what is depicted.

[0066] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals, and redundant descriptions thereof will be omitted.

[0067] FIG. 1 illustrates an exemplary configuration of an imaging device positioning system (1000) for shelf monitoring according to one embodiment. FIG. 2 illustrates a configuration in which a plurality of shelves are arranged inside a store according to one embodiment. FIG. 3 illustrates an exemplary configuration of an electronic shelf label (100) according to one embodiment. FIG. 4 is a block diagram illustrating an exemplary configuration of an electronic shelf label (100) according to one embodiment. FIG. 5 is a block diagram illustrating an exemplary configuration of a server (200) according to one embodiment. FIG. 6 illustrates a configuration of a shelf (30) according to one embodiment. FIG. 7 and FIG. 8 are intended to explain imaging device installation condition output models (M1, M2) utilized by an installation condition determination unit (29) of a server (200) according to one embodiment. FIG. 9 is a block diagram illustrating an exemplary configuration of an electronic device (300) according to one embodiment. FIG. 10 is a flowchart of an imaging device positioning method (S100) for shelf monitoring according to one embodiment. FIG. 11 is intended to illustrate the display of installation conditions of a plurality of imaging devices (400) for a plurality of shelves placed inside a store according to one embodiment. FIG. 12 is a flowchart of an imaging device positioning method (S200) for shelf monitoring according to another embodiment. FIG. 13 is a flowchart of an imaging device positioning method (S300) for shelf monitoring according to yet another embodiment. FIG. 14 and FIG. 15 are intended to illustrate the view of a plurality of imaging devices (400) installed according to determined installation conditions according to one embodiment capturing a shelf. FIG. 16 is intended to illustrate the display of modified installation conditions of a plurality of imaging devices (400) for a plurality of shelves placed inside a store according to one embodiment. FIG. 17 is a flowchart of exemplary steps that may be included in the step (S305) of changing the determined installation conditions included in the method (S300) of FIG. 13.FIG. 18 is intended to explain a method in which a plurality of imaging devices (400) according to one embodiment compare a plurality of images of at least one shelf (30) with a planogram of at least one shelf. FIG. 19 is a flowchart of exemplary steps that may be included in the step (S3053) of specifying at least one target electronic shelf label of FIG. 17. FIG. 20 is a flowchart of other exemplary steps that may be included in the step (S305) of changing determined installation conditions included in the method (S300) of FIG. 13. FIG. 21 is a flowchart of an imaging device positioning method (S400) for shelf monitoring according to another embodiment.

[0068] -System(1000)

[0069] Referring to FIG. 1, an imaging device positioning system (1000) for shelf monitoring according to one embodiment may include an electronic shelf label (100) provided on a shelf (30), a server (200), an electronic device (300), and at least one imaging device (400).

[0070] For example, the electronic device (300) may include a portable user terminal. Additionally, for example, at least one imaging device (400) may include an RGB camera for photographing the shelf (30).

[0071] The system (1000) can update product-related information displayed on a plurality of electronic shelf labels (100) provided on a shelf (30).

[0072] For example, an electronic shelf label (100) included in the system (1000) can be linked with a server (200), and information related to a product displayed on the electronic shelf label (100) can be updated from the server (200). Data of the updated information related to the product is transmitted from the server (200) to the electronic shelf label (100), and the electronic shelf label (100) can display the updated information related to the product. In this case, the server (200) can update the information related to the product based on user input through an electronic device (300).

[0073] For example, a plurality of electronic shelf labels (100) and a plurality of products may be provided on a shelf (30). Each of the plurality of products may be assigned to each of the plurality of electronic shelf labels (100). A first product may be assigned to any first electronic shelf label among the plurality of electronic shelf labels (100), and a second product different from the first product may be assigned to any second electronic shelf label. A first electronic shelf label and a first product may be provided adjacent to each other, and a second electronic shelf label and a second product may be provided adjacent to each other.

[0074] The server (200) can update the relevant information of multiple products assigned to multiple electronic shelf labels (100). The server (200) can update the relevant information of multiple products in bulk and transmit the updated relevant information of multiple products to the multiple electronic shelf labels (100).

[0075] For example, the server (200) can update the relevant information of any first product among multiple products and transmit the updated relevant information of the first product to the first electronic shelf label to which the first product is assigned. Additionally, the server (200) can update the relevant information of any other second product among multiple products and transmit the updated relevant information of the second product to the second electronic shelf label to which the second product is assigned.

[0076] Additionally, the system (1000) can determine and provide to the user the installation conditions of at least one imaging device (400) for photographing multiple shelves (30) arranged inside the store in order to generate a real program for multiple products and multiple electronic shelf labels (100) arranged on multiple shelves (30).

[0077] For example, referring to FIG. 2, various types of shelves may be arranged in different areas within the store. For example, a first plurality of shelves (a1~a8), a second plurality of shelves (b1~b4), a third plurality of shelves (c1~c4), a fourth plurality of shelves (k1~k4), a fifth plurality of shelves (m1~m6), a sixth plurality of shelves (n1~n5), a seventh plurality of shelves (j1~j3), an eighth plurality of shelves (i1~i4), a ninth plurality of shelves (z1~z10), a tenth plurality of shelves (s1~s4), an eleventh plurality of shelves (p1~p4), a ttwelfth plurality of shelves (t1~t2), and a thirteenth plurality of shelves (q1~q2), each having different shapes and sizes, may be provided at various locations within the store.

[0078] At least one imaging device (400) may be configured to be provided in a portion of a plurality of shelves (30) arranged inside the store and to photograph the plurality of shelves (30).

[0079] In this way, at least one imaging device (400) may be installed inside the store according to the determined installation conditions, and the system (1000) may generate a real program for multiple products and multiple electronic shelf labels (100) arranged on multiple shelves (30) based on an image taken by at least one imaging device (400) of multiple shelves (30).

[0080] For example, the server (200) can perform operations to generate a real-time real-gram based on data of an image of a shelf (30) received from at least one imaging device (400).

[0081] The electronic shelf label (100), server (200), electronic device (300), and imaging device (400) can be connected to each other through a network (500). Here, the network (500) according to the embodiment may refer to a connection structure capable of exchanging information between each node, such as the electronic shelf label (100), server (200), electronic device (300), and imaging device (400).

[0082] For example, the network (500) may include, but is not limited to, a 3GPP (3rd Generation Partnership Project) network, an LTE (Long Term Evolution) network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, a DMB (Digital Multimedia Broadcasting) network, etc.

[0083] The electronic shelf label (100) can be wirelessly connected to the server (200) through a separate gateway (not shown). In this case, multiple gateways may be placed at regular intervals within the store, and multiple gateways may be connected to the server (200) via wired connections. The electronic shelf label (100) can wirelessly communicate with adjacent gateways, and the gateways can transmit data from the electronic shelf label (100) to the server (200).

[0084] However, it is not limited to this, and the electronic shelf label (100) can be connected to an electronic device (300) via wireless communication. For example, the electronic shelf label (100) can be paired with an electronic device (300) based on Bluetooth functionality.

[0085] Below, the configuration of the electronic shelf label (100) will be described with reference to FIGS. 3 and FIGS. 4.

[0086] - Electronic shelf label (100)

[0087] The electronic shelf label (100) may be an electronic device mounted on a shelf (30) to display information related to a product displayed on the shelf (30). For example, the electronic shelf label (100) may display information related to the product, such as a Korean product name, an English product name, the country of origin of the product, the price of the product, raw materials, weight / calories, and discount information.

[0088] Referring to FIG. 3, the electronic shelf label (100) provided on the shelf (30) may include a display module that displays information related to the product provided in one area of ​​the outer surface and an LED (15) provided in another area.

[0089] Here, LED (15) is an abbreviation for 'light emitting device' and may be a light-emitting device included in the electronic shelf label (100). A pattern code (PC) based on a signal from a server (200) or an electronic device (300) may be displayed on the display module of the electronic shelf label (100). Here, the pattern code (PC) may be a 2D code. For example, the pattern code (PC) may be a 4x4 2D code, and each of the multiple pixels included in the pattern code (PC) may be displayed in black or white. Accordingly, the pattern code (PC) may be displayed in 65,536 (=216) different forms. However, it is not limited thereto, and the size of the pattern code (PC) may be designed to have various sizes other than 4x4.

[0090] Additionally, the electronic shelf label (100) may include at least one button (1, 2). For example, the electronic shelf label (100) may include a first button (1) and a second button (2). The first button (1) and the second button (2) may be provided in an area of ​​the outer surface of the electronic shelf label (100).

[0091] The LED (15) can be controlled based on a signal from a server (200) or an electronic device (300). Additionally, the types of light-emitting colors of the LED (15) may vary. For example, the light-emitting colors of the LED (15) may include seven colors: blue, red, green, yellow, sky blue, gray, and white. However, it is not limited thereto, and the light-emitting colors of the LED (15) may include various other colors in addition to the seven listed above.

[0092] A pairing signal may be transmitted from the electronic shelf label (100) to the electronic device (300) according to user input through the first button (1) and / or the second button (2). Accordingly, the electronic shelf label (100) may be paired with the electronic device (300) based on a Bluetooth method. For example, when a user presses the first button (1) and / or the second button (2), a pairing signal is transmitted from the electronic shelf label (100) to the electronic device (300), and the electronic shelf label (100) may be paired with the electronic device (300) based on a Bluetooth function.

[0093] Referring to FIG. 4, the electronic shelf label (100) may include a control unit (10), a display module (11), a communication module (12), a battery (13), and a memory (14).

[0094] The control unit (10) is a device for controlling the operation of the electronic shelf label (100) and may include a system-on-chip (SOC) structure including a central processing unit (CPU) and / or a graphics processing unit (GPU), etc.

[0095] The control unit (10) can control the operation of the display module (11) so that product-related information displayed on the display module (11) is updated based on a control signal received from the outside.

[0096] The display module (11) may be a device that displays information related to a product. The display module (11) may include an Electronic Paper Display (EPD) that maintains the product information display state even when power is not supplied.

[0097] The electronic paper display is suitable for an electronic shelf label (100) that needs to reduce power consumption due to its bistability, which maintains the display state for a long time even when the power supply is interrupted.

[0098] Electronic paper displays are known to include twist ball types utilizing hemispherical twist balls charged with electrostatic charge, electrophoretic displays applying electrophoresis and microcapsules, and cholesterol liquid crystal displays utilizing cholesterol liquid crystals.

[0099] However, it is not limited to this, and the display module (11) may include any one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, and a 3D display.

[0100] The communication module (12) may include various types of communication devices that enable an external device and an electronic shelf label (100) to transmit and receive data. For example, an electronic device (300) and an electronic shelf label (100) can be paired through the communication module (12), and data from the electronic device (300) can be transmitted to the electronic shelf label (100) while paired. Additionally, data transmitted from the server (200) and delivered by the gateway can be received by the communication module (12).

[0101] The battery (13) may be a device that supplies power for driving components included in the electronic shelf label (100). For example, the battery (13) may include a lithium-ion battery. However, it is not limited thereto, and the battery (13) may include various types of batteries other than lithium-ion batteries.

[0102] Additionally, the battery (13) may include a coin-shaped power supply. However, it is not limited thereto, and the battery (13) may include various types of power supplies other than a coin-shaped one.

[0103] The memory (14) may be a storage device in which data, instructions, and various programs are stored to perform operations necessary to update product-related information displayed on the display module (11). For example, the memory (14) may be a various storage device such as ROM, RAM, EPROM, flash drive, hard drive, etc.

[0104] The memory (14) and the control unit (10) can be provided on a printed circuit board and electrically connected to each other.

[0105] Below, the configuration of the server (200) will be described with reference to FIG. 5.

[0106] -Server(200)

[0107] The server (200) can update the relevant information of a product assigned to an electronic shelf label (100) provided on a shelf (30) and transmit the updated relevant information of the product to the electronic shelf label (100). In this case, the server (200) can update the relevant information of the product based on user input received through the electronic device (300).

[0108] Specifically, in order to update the relevant information of a product assigned to an electronic shelf label (100), the server (200) can exchange necessary data with the electronic shelf label (100) and the electronic device (300). Accordingly, the server (200) can provide the environment necessary to update the relevant information of the product.

[0109] For example, the server (200) may provide an environment in which product-related information can be updated using an electronic device (300) (e.g., a portable user terminal type electronic device, a desktop type electronic device, etc.). The server (200) may include an application, data and / or commands, etc., for the product-related information update application to operate, and may transmit data based thereon to the electronic device (300, 400).

[0110] Additionally, the server (200) may perform a predetermined operation to determine the installation conditions of at least one imaging device (400) for photographing the shelf (30). For example, the server (200) may determine the installation conditions of at least one imaging device (400) by performing a predetermined operation based on relevant information of the shelf (30) and / or relevant information of at least one imaging device (400).

[0111] The server (200) can transmit data of the installation conditions of at least one determined imaging device (400) to the electronic device (300), and the user can obtain information regarding the installation conditions of at least one determined imaging device (400) through the electronic device (300).

[0112] Referring to FIG. 5, the server (200) may include at least one processor (21) for data processing, a memory (22) for storing applications, data and / or instructions, etc., at least one communication module (23) for exchanging data with an external device, a location information database (24) for storing location information of an electronic shelf label (100), a template database (25) for storing a template containing product-related information, a shelf information database (26) for storing shelf-related information, and a planogram database (27) for storing planogram data containing information regarding the display status of a plurality of products and a plurality of electronic shelf labels (100) arranged on a shelf (30).

[0113] Additionally, the server (200) may be implemented as a predetermined computing device comprising a product-related information update unit (28) for updating product-related information, an installation condition determination unit (29) for determining the installation conditions of at least one imaging device (400), an image analysis unit (81) for performing analysis on an image captured by at least one imaging device (400), and a pattern code identification unit (82) for identifying pattern codes displayed on a plurality of electronic shelf labels (100).

[0114] The processor (21) can control the overall operation of the components included in the server (200) to provide an environment in which a product-related information editing application and / or an imaging device positioning application can operate on the electronic device (300).

[0115] The processor (21) may be a system-on-chip (SOC) including a central processing unit (CPU) and / or a graphics processing unit (GPU), and may execute an operating system (OS) and / or application programs stored in memory (22).

[0116] The processor (21) can communicate internally with each component included in the server (200) via a system bus and may include one or more predetermined bus structures, including a local bus.

[0117] The processor (21) can be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and other electrical units for performing functions.

[0118] The memory (22) can store one or more of an operating system (OS), various applications, data, and commands to provide an environment for performing an imaging device positioning method for shelf monitoring and an environment for performing a product-related information editing method.

[0119] The memory (22) may include a program area and a data area. Here, the program area according to the embodiment may be linked between an operating system (OS) and functional elements that boot the server (200), and the data area may store data generated by the use of the server (200).

[0120] In one embodiment, the memory (22) may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and may be web storage that performs storage functions on the internet. Additionally, the memory (22) may be a recording medium that is detachable from the server (200).

[0121] The communication module (23) may include various types of communication devices that enable the server (200) to transmit and receive data with an external device.

[0122] The server (200) can transmit data based on an application, data and / or commands, etc., for operating a product-related information editing application and / or an imaging device positioning application to an electronic device (300) through a communication module (23).

[0123] The location information database (24) can store location information of the electronic shelf labels (100). A plurality of electronic shelf labels (100) may be provided on the shelf (30) to correspond to a plurality of products. Each of these plurality of electronic shelf labels (100) may be placed at a specific location on the shelf (30), and specific location information of the plurality of electronic shelf labels (100) on the shelf (30) may be stored in the location information database (24). In this case, the location information of each of the plurality of electronic shelf labels (100) may be matched with the unique identification information of each of the plurality of electronic shelf labels (100) and stored in the location information database (24).

[0124] For example, among a plurality of electronic shelf labels (100), a first electronic shelf label may be provided in the 2nd layer, 3rd column of a first shelf provided in area A among a plurality of areas. In this case, the location information of the first electronic shelf label, 'the 2nd layer, 3rd column of the first shelf in area A', may be stored in a location information database (24) as the location information of the first electronic shelf label, corresponding to the unique identification information of the first electronic shelf label.

[0125] Here, the unique identification information of the electronic shelf label (100) may include a unique identification number assigned to the electronic shelf label (100). For example, the identification information of the electronic shelf label (100) may be in the form of a string of characters, a sequence of numbers, and / or a combination thereof. However, it is not limited thereto, and the identification information of the electronic shelf label (100) may include a combination of various symbols other than characters and numbers.

[0126] However, it is not limited to this, and the unique identification information of the electronic shelf label (100) may include a unique pattern code displayed by the electronic shelf label (100).

[0127] The location information database (24) is included in the server (200), but is not limited thereto, and may be implemented as an independent server configured separately from the server (200).

[0128] The template database (25) may be a database that stores templates to which product-related information is applied. For example, the template database (25) may store data regarding multiple templates to which product-related information can be entered. The multiple templates may have different forms depending on user input.

[0129] For example, multiple templates can be generated based on user input from a middle manager of a store using an electronic device (300). The middle manager can create a template of the desired form through the electronic device (300).

[0130] The electronic device (300) can transmit data regarding a plurality of templates to the server (200), and the data regarding a plurality of templates can be stored in the template database (25). However, it is not limited thereto, and the template database (25) may store data regarding a plurality of previously created templates.

[0131] The template database (25) is included in the server (200), but is not limited thereto, and may be implemented as an independent server configured separately from the server (200).

[0132] The shelf information database (26) can store data of related information for multiple shelves (30) placed in the store.

[0133] For example, referring to FIG. 6, at least one shelf (30) placed in a store may have a predetermined width (w1), height (h1), and depth (d1).

[0134] Additionally, the shelf (30) may include a plurality of shelf plates (pt1, pt2, pt3, pt4) arranged in layers where a plurality of products can be provided. For example, the shelf (30) may include a first shelf plate (pt1), a second shelf plate (pt2), a third shelf plate (pt3), and a fourth shelf plate (4) arranged in order from top to bottom. In this case, the plurality of shelf plates (pt1, pt2, pt3, pt4) may be arranged at equal intervals.

[0135] Furthermore, as illustrated in FIG. 2, a plurality of shelves (30) may be provided at various locations within the store. In this case, the distance between each of the plurality of shelves (30) and surrounding shelves may vary.

[0136] For example, referring to FIG. 2, the spacing between the first plurality of shelves (a1 to a8) and the second plurality of shelves (b1 to b4) may be different from the spacing between the fifth plurality of shelves (m1 to m6) and the eleventh plurality of shelves (p1 to p4).

[0137] The shelf information database (26) may store data of information related to at least one shelf (30), including at least one of the location, width (w1), height (h1), depth (d1), number of shelf plates, and distance from surrounding shelves. However, it is not limited thereto, and the information related to at least one shelf (30) may further include various information about at least one shelf (30) other than those mentioned above.

[0138] The shelf information database (26) is included in the server (200), but is not limited thereto, and may be implemented as an independent server configured separately from the server (200).

[0139] The planogram database (27) can store a planogram showing the arrangement of products and electronic shelf labels (100) on shelves (30) placed inside the store. The planogram may be an image showing the arrangement of products and electronic shelf labels (100) on shelves (30). Additionally, the planogram may include further information regarding the location of the products and the location of the electronic shelf labels (100).

[0140] A plurality of products and a plurality of electronic shelf labels (100) may be provided on the shelf (30). For example, a plurality of electronic shelf labels (100) and a plurality of products assigned to each of the plurality of electronic shelf labels (100) may be displayed at specific locations on the shelf (30). A planogram may show a configuration in which a plurality of products and a plurality of electronic shelf labels (100) are displayed correspondingly on the shelf (30).

[0141] The planogram database (27) is included in the server (200), but is not limited thereto, and may be implemented as an independent server configured separately from the server (200).

[0142] The product-related information update unit (28) can update the product-related information displayed on the electronic shelf label (100). For example, the product-related information update unit (28) can control the electronic shelf label (100) so that the product-related information displayed on the electronic shelf label (100) is periodically updated according to the product-related information update rule stored in the memory (22). The product-related information update rule stored in the memory (22) is predetermined by the user and can be modified at any time.

[0143] However, it is not limited to this, and the product-related information update unit (28) may also control the electronic shelf label (100) so that the product-related information displayed on the electronic shelf label (100) is updated according to user input editing the product-related information.

[0144] For example, the product-related information update unit (28) can control the electronic shelf label (100) so that information related to the product is updated by inputting information from user input into a template provided from the template database (25) and the product-related information is displayed on the electronic shelf label (100).

[0145] The installation condition determining unit (29) can determine the installation conditions of at least one imaging device (400) for monitoring at least one shelf (30) based on relevant information of at least one shelf (30).

[0146] For example, the installation condition determining unit (29) can obtain relevant information of at least one shelf (30) stored in the shelf information database (26). In this case, the relevant information of at least one shelf (30) may include at least one of the position, width (w1), height (h1), depth (d1), number of shelf plates, and distance from surrounding shelves of at least one shelf (30).

[0147] However, it is not limited to this, and the installation condition determining unit (29) may also obtain relevant information of at least one shelf (30) input into the electronic device (300) by the user.

[0148] The installation condition determination unit (29) can determine the installation conditions of at least one imaging device (400) optimized for monitoring multiple products and multiple electronic shelf labels (100) provided on the shelf (30) through analysis of relevant information of at least one shelf (30) obtained. In this case, the installation conditions can be determined such that at least one imaging device (400) is installed on another shelf (30) provided around the shelf (30) to be monitored.

[0149] For example, referring to FIG. 7, the installation condition determination unit (29) can output the installation condition of at least one imaging device (400) by utilizing a first imaging device installation condition output model (M1) that has been learned based on a relevant information data set of the shelf (30). The first imaging device installation condition output model (M1) may be an artificial intelligence model learned to output the installation condition of at least one imaging device (400) optimized for photographing the shelf (30) based on a relevant information data set of the shelf (30).

[0150] An artificial intelligence model is characterized by being created through learning based on a specific data set. Here, being created through learning means that an artificial intelligence model is created by being trained using multiple learning data by a learning algorithm, thereby setting up an artificial intelligence model to perform a desired characteristic (or objective).

[0151] Such learning may be performed on the device itself where the artificial intelligence model according to the present invention is utilized, or through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples mentioned above.

[0152] An artificial intelligence model may be composed of multiple artificial neural network layers. Each of the multiple artificial neural network layers has multiple weight values ​​and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated so that the loss value or cost value obtained by the artificial intelligence model during the learning process is reduced or minimized.

[0153] Artificial neural networks may include deep neural networks (DNN), such as Convolutional Neural Networks (CNN), Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restructured Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory Network (LSTM), Classification Network, etc., such as GoogleNet, AlexNet, VGG Network, etc., but are not limited to the examples mentioned above.

[0154] For example, when relevant information of at least one shelf (30) is input into the first imaging device installation condition output model (M1), the installation condition of at least one imaging device (400) can be output.

[0155] In this way, the installation condition determination unit (29) can determine the installation conditions of at least one imaging device (400) optimized for photographing the shelf (30) according to the relevant information of the shelf (30) by utilizing the first imaging device installation condition output model (M1).

[0156] However, it is not limited to this, and the installation condition determination unit (29) may determine the installation condition of at least one imaging device (400) corresponding to the relevant information of the shelf (30) obtained based on data stored in an installation condition database (not shown) where the relevant information of the shelf (30) and the installation condition of at least one imaging device (400) are matched.

[0157] Additionally, the installation condition determination unit (29) may further utilize relevant information of at least one imaging device (400) when determining the installation condition of at least one imaging device (400).

[0158] For example, the installation condition determination unit (29) can obtain relevant information of at least one imaging device (400) through user input via the electronic device (400), or obtain relevant information of at least one imaging device (400) that has been stored in an imaging device database (not shown).

[0159] In this case, the relevant information of at least one imaging device (400) may include at least one of the optical characteristics and tilting range of at least one imaging device (400). The optical characteristics of at least one imaging device (400) may include, for example, the angle of view and focal length of at least one imaging device (400). However, it is not limited thereto, and the relevant information of at least one imaging device (400) may further include various information related to at least one imaging device (400) other than those mentioned above.

[0160] For example, referring to FIG. 8, the installation condition determination unit (29) can output installation conditions by utilizing a second imaging device installation condition output model (M2) that has been learned based on the relevant information data set of the shelf (30) and the relevant information data set of the imaging device (400). The second imaging device installation condition output model (M2) may be an artificial intelligence model learned to output installation conditions of the imaging device (400) optimized for photographing the shelf (30) based on the relevant information data set of the shelf (30) and the relevant information data set of the imaging device (400).

[0161] When the relevant information of at least one shelf (30) and the relevant information of at least one imaging device (400) are input into the second imaging device installation condition output model (M2), the installation condition of at least one imaging device (400) can be output.

[0162] In this way, the installation condition determination unit (29) can determine the installation conditions of at least one imaging device (400) optimized for photographing the shelf (30) based on the relevant information of the shelf (30) and the relevant information of the imaging device (400) by utilizing the second imaging device installation condition output model (M2).

[0163] However, it is not limited to this, and the installation condition determination unit (29) may determine the installation condition of at least one imaging device (400) corresponding to the relevant information of the shelf (30) and the relevant information of the imaging device (400) obtained based on data stored in an installation condition database (not shown) in which the relevant information of the shelf (30) and the relevant information of the imaging device (400) and the installation condition of at least one imaging device (400) are matched.

[0164] The image analysis unit (81) can detect an object by analyzing an image captured by at least one imaging device (400).

[0165] For example, at least one imaging device (400) can photograph an area on a shelf (30) where products and electronic shelf labels (100) are provided, and data of the image of the area where products and electronic shelf labels (100) are provided can be transmitted from at least one imaging device (400) to a server (200).

[0166] The image analysis unit (81) can detect the product and / or electronic shelf label (100) among various objects included in an image in which an area where the product and electronic shelf label (100) are provided is captured, based on an image extraction algorithm. Additionally, the image analysis unit (81) can detect the pattern code displayed by the electronic shelf label (100) in the captured image.

[0167] Here, image extraction algorithms may include artificial intelligence models such as R-CNN, YOLO, and ViT. However, the types of image extraction algorithms are not limited to these.

[0168] Additionally, the image analysis unit (81) can detect a desired specific product and / or electronic shelf label (100) from the planogram based on an image extraction algorithm.

[0169] Furthermore, the image analysis unit (81) can compare the image of the product and / or electronic shelf label (100) detected from the image of the area where the product and electronic shelf label (100) are provided with the specific product and / or electronic shelf label (100) detected from the planogram.

[0170] The pattern code identification unit (82) can detect the pattern code displayed by the electronic shelf label (100) detected in an image in which at least one imaging device (400) captures an area where the electronic shelf label (100) is provided.

[0171] The electronic shelf label (100) can display a unique pattern code, and this pattern code can correspond to unique identification information of the electronic shelf label (100).

[0172] The pattern code identification unit (82) can obtain unique identification information of the electronic shelf label (100) corresponding to the pattern code displayed by the identified electronic shelf label (100).

[0173] Here, the identification information of the electronic shelf label (100) may be in the form of a string, a sequence of numbers, and / or a combination thereof. However, it is not limited thereto, and the identification information of the electronic shelf label (100) may include a combination of various symbols other than characters and numbers.

[0174] For example, the pattern code identification unit (82) can detect the pattern code displayed on the electronic shelf label (100) from an image in which an area where the electronic shelf label (100) is provided is captured based on an image extraction algorithm.

[0175] The detected pattern code may correspond to the identification information of the electronic shelf label (100). The corresponding pattern code and the identification information of the electronic shelf label (100) may be matched with each other and stored in a pattern code database (not shown). The pattern code identification unit (82) may extract the identification information of the electronic shelf label (100) corresponding to the detected pattern code from the pattern code database.

[0176] In this way, the pattern code identification unit (82) can detect the pattern code displayed by the electronic shelf label (100) based on the data of the image of the electronic shelf label (100) and obtain identification information of the electronic shelf label (100) corresponding to the pattern code.

[0177] Meanwhile, the processor (21) of the server (200) can generate a real-time program including the display status of the products displayed on the shelf (30) and the electronic shelf label (100) based on the image data of the shelf (30) acquired by at least one imaging device (400).

[0178] For example, the processor (21) of the server (200) can generate a real-gram of the shelf (30) in real time by connecting multiple images of the shelf (30) acquired by at least one imaging device (400). The real-gram of the shelf (30) may include not only image data but also information regarding the location of the at least one imaging device (400) that took the image. Accordingly, the real-gram may include both an image showing the product display status of the product and the electronic shelf label (100) and information regarding the location where the image was taken.

[0179] In the above description, it has been explained that a server (200) according to one embodiment performs functional operations as described above; however, depending on the embodiment, at least a portion of the functional operations performed by the server (200) may be performed by an external device (e.g., an electronic device (300)), and at least a portion of the functional operations performed by the external device may be further performed by the server (200), and various other embodiments may be possible.

[0180] Below, the configuration of the electronic device (300) will be described with reference to FIG. 9.

[0181] -Electronic device (300)

[0182] The electronic device (300) may be a device installed with an imaging device positioning application and / or a product-related information editing application. The electronic device (400) may include a portable user terminal.

[0183] For example, the user of the electronic device (300) may be a store employee or a customer of the store.

[0184] The electronic device (300) can be implemented, for example, as a computer or portable terminal that can access the server (200) through a network (500).

[0185] Here, the computer may include, for example, a laptop, desktop, laptop, VR HMD (e.g., HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.) equipped with a web browser.

[0186] A portable terminal is a wireless communication device that ensures portability and mobility, and may include, for example, smartphones, tablet PCs, and wearable devices, as well as various devices equipped with communication modules such as Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, Ultrasonic, Infrared, WiFi, and LiFi.

[0187] Referring to FIG. 9, the electronic device (300) may include a processor (91), memory (92), a communication module (93), an input module (94), and a display module (95).

[0188] Various components included in the electronic device (300) can be designed to be contained within the housing of the electronic device (300).

[0189] In an embodiment, the processor (91) can control the overall operation of the components included in the electronic device (300) through the imaging device positioning application and / or product related information editing application of the memory (92) to provide an imaging device positioning environment and / or an environment for editing product related information.

[0190] For example, the processor (91) can control the operation of the communication module (93) and the input module (94) so ​​that a signal according to user input received through the input module (94) can be transmitted to the electronic shelf label (100) through the communication module (93).

[0191] The processor (91) may include a central processing unit (CPU) and / or a graphics processing unit (GPU). Additionally, the processor (91) may include at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and other electrical units for performing functions.

[0192] The memory (92) can store commands and data that can be used to create an imaging device positioning environment and / or an environment for editing related information of a product.

[0193] In memory (92), an imaging device positioning application and / or a product related information editing application may be stored.

[0194] An imaging device positioning application can determine the installation conditions of at least one imaging device (400) for monitoring a shelf (30) and provide a user interface capable of providing information regarding the determined installation conditions.

[0195] A product related information editing application can provide various types of user interfaces to provide an environment for editing product related information.

[0196] The memory (92) may include at least one non-transient computer-readable storage medium and a transient computer-readable storage medium. For example, the memory (92) may be various storage devices such as ROM, EPROM, flash drive, hard drive, etc. Additionally, the memory (92) may include web storage that performs data storage functions on the internet.

[0197] The communication module (93) may include various types of communication devices capable of transmitting and receiving data with an external device. For example, the communication module (93) may transmit and receive data with the electronic shelf label (100) and / or server (200) via a wireless network.

[0198] The communication module (93) may be configured to transmit and receive data with an external device via Bluetooth. For example, the communication module (93) may receive a pairing signal from the electronic shelf label (100) and be paired with the electronic shelf label (100) via Bluetooth.

[0199] The input module (94) may be configured to receive various forms of user input from a user using the electronic device (300). For example, the input module (94) may include a touch screen that receives touch input from the user.

[0200] When the input module (94) is implemented as a touch screen, the input module (94) may be formed by being integrally combined with the display module (95). However, it is not limited thereto, and the input module (95) may further include a keyboard capable of receiving user input in the form of characters.

[0201] The display module (95) can display content related to a shelf monitoring application and / or a product-related information update application included in the memory (92).

[0202] For example, the display module (95) may include a display device that displays various types of user interface images to provide an imaging device positioning environment or an environment for updating relevant information about a product.

[0203] For example, the display device may display information on the installation conditions of at least one imaging device (400) determined by the server (200) mapped onto a map inside the store.

[0204] The display module (95) may include any one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, and a 3D display.

[0205] - Imaging device (400)

[0206] At least one imaging device (400) may be installed around the shelf (30) and configured to photograph the shelf (30) on which the products and electronic shelf labels (100) are provided. At least one imaging device (400) may be configured to periodically photograph the products and electronic shelf labels (100) on the shelf (30).

[0207] For example, multiple products of different types may be displayed on a shelf (30), and multiple electronic shelf labels (100) may be provided on the shelf (30) such that each of the multiple products corresponds to each of the locations where the multiple products are provided. In this case, information for each of the multiple products of different types may be assigned to each of the multiple electronic shelf labels (100) provided at the location corresponding to each of the multiple products.

[0208] At least one imaging device (400) can obtain a captured image by photographing a plurality of products and a plurality of electronic shelf labels (100) arranged on a shelf (30). Accordingly, the captured image may include a plurality of products and a plurality of electronic shelf labels (100) arranged to correspond to each of the plurality of products.

[0209] At least one imaging device (400) may be configured to rotate in various directions to photograph an area where products and electronic shelf labels (100) are arranged through a field of view. For example, at least one imaging device (400) may include an RGB camera.

[0210] At least one imaging device (400) may include a wireless communication module capable of exchanging signals with an external device. For example, at least one imaging device (400) may include a Bluetooth low energy (BLE) device, an RFID (Radio frequency identification) tag, an NFC (Near field communication) tag, etc.

[0211] At least one imaging device (400) can transmit data of an image of a shelf (30) to a server (200) via a wireless communication module.

[0212] Furthermore, at least one imaging device (400) may include a wireless signal receiver that receives wireless signals from a plurality of beacons placed within the store.

[0213] Meanwhile, at least one imaging device (400) may be configured to transmit data related to at least one imaging device (400) to a server (200).

[0214] For example, at least one imaging device (400) can transmit data of a captured image to a server (200). Additionally, at least one imaging device (400) can transmit data of attitude information of the imaging device (400) sensed by its own IMU to a server (200).

[0215] At least one imaging device (400) may be provided on another shelf (30) placed around the shelf (30) on which the product and electronic shelf label (100) are provided. However, it is not limited thereto, and at least one imaging device (400) may be provided at various locations capable of photographing the product and electronic shelf label (100) provided on the shelf (30).

[0216] Hereinafter, an imaging device positioning method (S100, S200, S300, S400) for shelf monitoring is described with reference to FIGS. 10 to 21.

[0217]

[0218] - Imaging device positioning method (S100, S200, S300, S400)

[0219] An imaging device positioning method (S100, S200, S300, S400) for shelf monitoring can be performed by a processor (21) of a server (200) according to one embodiment.

[0220] However, it is not limited to this, and at least part of the method (S100, S200, S300, S400) may be performed by the processor (91) of the electronic device (300), and other part may be performed by the processor (21) of the server (200).

[0221] For example, at least one of the processor (21) of the server (200) and the processor (91) of the electronic device (300) can perform an imaging device positioning method (S100, S200, S300, S400) for shelf monitoring by executing at least one command stored in the memory (22) of the server (200) or the memory (92) of the electronic device (300).

[0222] In the following description, the processor (21) of the server (200) performs an imaging device positioning method (S100, S200, S300, S400) for shelf monitoring.

[0223] Referring to FIG. 10, an imaging device positioning method (S100) for shelf monitoring according to one embodiment may include the step (S101) of obtaining relevant information of at least one shelf (30) placed inside a store, and the step (S103) of determining the installation conditions of at least one imaging device (400) for monitoring at least one shelf (30) based on the relevant information of at least one shelf (30).

[0224] In step (S101), the processor (21) of the server (200) can obtain relevant information of at least one shelf (30). Here, the relevant information of at least one shelf (30) may include at least one of the location, width (w1), height (h1), depth (d1), number of shelf plates, and distance from surrounding shelves of at least one shelf (30).

[0225] For example, the processor (21) of the server (200) can obtain relevant information about at least one shelf (30) placed in a store, which is stored in the shelf information database (26).

[0226] However, it is not limited to this, and the processor (21) of the server (200) can obtain relevant information of at least one shelf (30) that the user has directly entered into the electronic device (300). For example, the user can enter relevant information of at least one shelf (30) for monitoring into the electronic device (300), and the data of the relevant information of at least one shelf (30) entered into the electronic device (300) can be transmitted to the server (200).

[0227] Furthermore, the processor (21) of the server (200) may obtain relevant information of at least one shelf (30) based on a marker (MK) provided on at least one shelf (30) that the user has captured with an electronic device (300).

[0228] For example, referring to FIG. 18, a marker (MK) corresponding to information related to at least one shelf (30) may be provided on one side of at least one shelf (30).

[0229] The processor (21) of the server (200) can identify the marker (MK) by analyzing the image of the marker (MK) based on an image extraction algorithm, and obtain shelf-related information corresponding to the marker (MK) from the shelf information database (26).

[0230] In step (S103), the processor (21) of the server (200) can determine the installation conditions of at least one imaging device (400) based on relevant information of at least one shelf (30).

[0231] For example, the installation condition determination unit (29) of the server (200) can determine the installation condition of at least one imaging device (400) optimized for monitoring multiple products and multiple electronic shelf labels (100) arranged on at least one shelf (30) through analysis of relevant information of at least one shelf (30).

[0232] For example, the installation condition determination unit (29) of the server (200) can obtain the installation condition of at least one imaging device (400) by utilizing a pre-learned imaging device installation condition output model based on a data set of relevant information of various types of shelves (30).

[0233] Here, the imaging device installation condition output model may be an artificial intelligence model trained to output the installation condition of at least one imaging device (400) by taking relevant information of at least one shelf (30) as input.

[0234] However, it is not limited to this, and the installation condition determination unit (29) of the server (200) may determine the installation condition of at least one imaging device (400) corresponding to the relevant information of at least one shelf (30) based on data stored in an installation condition database (not shown) in which the relevant information of at least one shelf (30) and the installation condition of at least one imaging device (400) are matched.

[0235] For example, referring to FIG. 11, there may be cases where the installation conditions of at least one imaging device (400) for monitoring the 10-2 shelf (s2), 10-3 shelf (s3), 11-1 shelf (p1), 11-2 shelf (p2), 11-3 shelf (p3), and 11-4 shelf (p4) provided in the first area (Ar1) inside the store among the 10-2 shelves (s1~s4) and 11-3 shelves (p1~p4) must be determined.

[0236] In this case, the installation condition determination unit (29) of the server (200) can determine a plurality of installation locations (po1, po2, po3, po4, po5, po6) of a plurality of imaging devices (400) as installation conditions for at least one imaging device (400) based on the relevant information of each of the 10-2 shelf (s2), 10-3 shelf (s3), 11-1 shelf (p1), 11-2 shelf (p2), 11-3 shelf (p3), and 11-4 shelf (p4).

[0237] Here, for example, each of the multiple installation positions (po1, po2, po3, po4, po5, po6) may correspond to the upper region of the center of each of the 10-2 shelf (s2), 10-3 shelf (s3), 11-1 shelf (p1), 11-2 shelf (p2), 11-3 shelf (p3), and 11-4 shelf (p4).

[0238] As illustrated in FIG. 11, each of the multiple installation locations (po1, po2, po3, po4, po5, po6) can be mapped onto a map inside a store where multiple shelves are provided and displayed through the display module (95) of the electronic device (300).

[0239] In addition, similarly, there may be cases where the installation conditions of at least one imaging device (400) for monitoring the 6-1 shelf (n1), 6-2 shelf (n2), 6-3 shelf (n3), 6-4 shelf (n4), 6-5 shelf (n5), 13-1 shelf (q1), and 13-2 shelf (q2) provided in the second area (Ar2) inside the store among the 6 multiple shelves (n1~n5) and 13 multiple shelves (q1~q2) must be determined.

[0240] In this case, the installation condition determination unit (29) of the server (200) can determine a plurality of installation locations (po7, po8, po9, po10, po11) of a plurality of imaging devices (400) as installation conditions for at least one imaging device (400) based on the relevant information of each of the 6-1 shelf (n1), 6-2 shelf (n2), 6-3 shelf (n3), 6-4 shelf (n4), 6-5 shelf (n5), 13-1 shelf (q1), and 13-2 shelf (q2).

[0241] Here, for example, each of the multiple installation positions (po7, po8, po9, po10, po11) may correspond to the upper region of the center of each of the 6-1 shelf (n1), 6-3 shelf (n3), 6-5 shelf (n5), 13-1 shelf (q1), and 13-2 shelf (q2).

[0242] As illustrated in FIG. 11, each of the multiple installation locations (po7, po8, po9, po10, po11) can be mapped onto a map inside a store where multiple shelves are provided and displayed through the display module (95) of the electronic device (300).

[0243] Furthermore, similarly, there may be cases where it is necessary to determine the installation conditions of at least one imaging device (400) for monitoring the 2-3 shelf (b3), 5-1 shelf (m1), 5-2 shelf (m2), 5-3 shelf (m3), 5-4 shelf (m4), 5-5 shelf (m5), 5-6 shelf (m6), 11-1 shelf (p1), and 11-2 shelf (p2) provided in the 3rd area (Ar3) inside the store among the 2-3 shelf (b1~b4), 5-6 shelf (m6), and 11-1 shelf (p1~p4).

[0244] In this case, the installation condition determination unit (29) of the server (200) can determine a plurality of installation locations (po12, po13, po14, po15, po16, po17) of a plurality of imaging devices (400) as installation conditions for at least one imaging device (400) based on the relevant information of each of the 2-3 shelf (b3), 5-1 shelf (m1), 5-2 shelf (m2), 5-3 shelf (m3), 5-4 shelf (m4), 5-5 shelf (m5), 5-6 shelf (m6), 11-1 shelf (p1), and 11-2 shelf (p2).

[0245] Here, for example, each of the multiple installation positions (po12, po13, po14, po15, po16, po17) may correspond to the upper area of ​​the center of the 5-1 shelf (m1), the upper area of ​​the center of the 5-3 shelf (m3), the upper area of ​​the center of the 5-6 shelf (m6), the upper area of ​​the right edge of the 2-3 shelf (b3), the upper area of ​​the right edge of the 11-2 shelf (p2), and the upper area of ​​the left edge of the 11-3 shelf (p2), respectively.

[0246] As illustrated in FIG. 11, each of the multiple installation locations (po12, po13, po14, po15, po16, po17) can be mapped onto a map inside a store where multiple shelves are provided and displayed through the display module (95) of the electronic device (300).

[0247] Thus, in step (S103), the processor (21) of the server (200) can determine installation conditions such as the number and location of at least one imaging device (400) optimized for monitoring at least one shelf (30) based on relevant information of at least one shelf (30) to be monitored.

[0248] Referring to FIG. 12, a method for positioning an imaging device for shelf monitoring (S200) according to another embodiment may include the step of obtaining relevant information of at least one shelf (30) placed inside a store (S201), the step of obtaining relevant information of at least one imaging device (400) (S203), and the step of determining the installation conditions of at least one imaging device (400) for monitoring at least one shelf (30) based on the relevant information of at least one shelf (30) and the relevant information of at least one imaging device (400) (S205).

[0249] In step (S201), the processor (21) of the server (200) can obtain relevant information of at least one shelf (30). Since step (S201) is substantially the same as step (S101), a description thereof is omitted here.

[0250] In step (S203), the processor (21) of the server (200) can obtain relevant information of at least one imaging device (400). Here, the relevant information of at least one imaging device (400) may include at least one of the optical characteristics and tilting range of at least one imaging device (400). The optical characteristics of at least one imaging device (400) may include, for example, the angle of view, focal length, etc. of at least one imaging device (400).

[0251] For example, the processor (21) of the server (200) can obtain relevant information of at least one imaging device (400) stored in an imaging device database (not shown).

[0252] However, it is not limited to this, and the processor (21) of the server (200) can obtain relevant information of at least one imaging device (400) from at least one imaging device (400). In this case, the at least one imaging device (400) can transmit data of relevant information of at least one imaging device (400) to the server (200) along with data of an image of at least one shelf (30).

[0253] Furthermore, the processor (21) of the server (200) can obtain relevant information of at least one imaging device (400) that the user has directly input into the electronic device (300). For example, the user can input relevant information of at least one imaging device (400) into the electronic device (300), and data of relevant information of at least one imaging device (400) input into the electronic device (300) can be transmitted from the electronic device (300) to the server (200).

[0254] In step (S205), the processor (21) of the server (200) can determine the installation conditions of at least one imaging device (400) based on the relevant information of at least one shelf (30) and the relevant information of at least one imaging device (400).

[0255] For example, the installation condition determination unit (29) of the server (200) can determine the installation condition of at least one imaging device (400) optimized for monitoring multiple products and multiple electronic shelf labels (100) arranged on at least one shelf (30) through analysis of relevant information of at least one shelf (30) and relevant information of at least one imaging device (400).

[0256] For example, the installation condition determination unit (29) of the server (200) can obtain the installation condition of at least one imaging device (400) by utilizing a pre-learned imaging device installation condition output model based on a relevant information data set of various types of shelves (30) and a relevant information data set of various types of imaging devices (400).

[0257] Here, the imaging device installation condition output model may be an artificial intelligence model trained to output the installation condition of at least one imaging device (400) by taking relevant information of at least one shelf (30) and relevant information of at least one imaging device (400) as input.

[0258] However, it is not limited to this, and the installation condition determination unit (29) of the server (200) may determine the installation condition of at least one imaging device (400) corresponding to the relevant information of at least one shelf (30) and the relevant information of at least one imaging device (400) based on data stored in an installation condition database (not shown) in which the relevant information of at least one shelf (30) and the relevant information of at least one imaging device (400) are matched.

[0259] Referring to FIG. 13, a method for positioning an imaging device for shelf monitoring (S300) according to another embodiment may include the steps of: obtaining relevant information of at least one shelf (30) placed inside a store (S301); determining the installation conditions of at least one imaging device (400) for monitoring at least one shelf (30) based on the relevant information of at least one shelf (30) (S303); and changing the determined installation conditions based on the analysis results of at least one image of at least one shelf (30) taken by the at least one imaging device (400) installed according to the determined installation conditions (S305).

[0260] Since steps (S301) and (S303) of the method (S300) are substantially identical to steps (S101) and (S103) of the method (S100), the description of steps (S301) and (S303) is omitted here.

[0261] Meanwhile, the step (S305) of changing the determined installation conditions described below may also be included in the method (S200) of FIG. 12.

[0262] In step (S305), the installation condition determination unit (29) of the server (200) can change the installation condition of at least one imaging device (400) that has been determined based on the analysis result of at least one image of at least one shelf (30) by the image analysis unit (81).

[0263] For example, according to the installation conditions determined in step (303), at least one image of at least one shelf (30) taken by at least one imaging device (400) installed inside the store may not include all of the multiple products and multiple electronic shelf labels (100) arranged on at least one shelf (30). In this case, it is necessary to change the installation conditions of at least one imaging device (400) so that all of the multiple products and multiple electronic shelf labels (100) arranged on the shelf (30) can be included in at least one image of at least one shelf (30).

[0264] Additionally, for example, according to the installation conditions determined in step (303), there may be an overlapping area between at least one image of at least one shelf (30) captured by at least one imaging device (400) installed inside the store. In this case, in order to efficiently position at least one imaging device (400), it is necessary to change the installation conditions of at least one imaging device (400) so that the overlapping area is minimized for at least one image of at least one shelf (30).

[0265] To this end, the installation condition determination unit (29) of the server (200) can change the installation condition of at least one imaging device (400) based on the result of the image analysis unit (81) analyzing at least one image of at least one shelf (30) taken by at least one imaging device (400) installed according to the predetermined installation condition.

[0266] For example, referring to FIG. 14, a plurality of images of the 11-2 shelf (p2), 10-3 shelf (s3), and 11-4 shelf (p4) can be obtained by a plurality of imaging devices (400) installed at a first position (po1), a second position (po2), and a third position (po3) according to determined installation conditions.

[0267] Additionally, for example, referring to FIG. 15, a plurality of images of the 11-1 shelf (p1), the 10-2 shelf (s2), and the 11-3 shelf (p3) can be obtained by a plurality of imaging devices (400) installed at the 4th position (po4), the 5th position (po5), and the 6th position (po6) according to the determined installation conditions.

[0268] The image analysis unit (81) can perform analysis on a plurality of images taken of the 10-2 shelf (s2), 10-3 shelf (s3), 11-1 shelf (p1), 11-2 shelf (p2), 11-3 shelf (p3), and 11-4 shelf (p4).

[0269] For example, the image analysis unit (81) may determine that at least some of the products and electronic shelf labels (100) provided on the 10-2 shelf (s2), 10-3 shelf (s3), 11-1 shelf (p1), 11-2 shelf (p2), 11-3 shelf (p3), and 11-4 shelf (p4) are not included in the plurality of images in which the 10-2 shelf (s2), 10-3 shelf (s3), 11-1 shelf (p1), 11-2 shelf (p2), 11-3 shelf (p3), and 11-4 shelf (p4) are captured.

[0270] Accordingly, the installation condition determination unit (29) can change the installation condition of at least one imaging device (400) that has been determined based on the analysis results of a plurality of images taken of the 10-2 shelf (s2), 10-3 shelf (s3), 11-1 shelf (p1), 11-2 shelf (p2), 11-3 shelf (p3), and 11-4 shelf (p4) of the image analysis unit (81).

[0271] For example, referring to FIG. 16, the installation condition determining unit (29) can determine a plurality of installation positions (po18, po19, po20, po21, po22, po23, p24, p25) that are increased in number from the first to sixth positions (po1 to po6) as installation conditions for at least one modified imaging device (400).

[0272] Here, for example, the modified multiple installation positions (po18, po19, po20, po21, po22, po23, p24, p25) of the multiple imaging devices (400) may correspond to the upper region of the center of each of the 11-1 shelf (p1), the 11-2 shelf (p2), the 11-3 shelf (p3), and the 11-4 shelf (p4), the upper region of two regions spaced apart by a predetermined distance from the center of the 10-2 shelf (s2), and the upper region of two regions spaced apart by a predetermined distance from the center of the 10-3 shelf (s3).

[0273] Thus, in step (S305), the installation condition determination unit (29) of the server (200) can change the installation condition of at least one imaging device (400) that has been determined based on the analysis result of at least one image of at least one shelf (30) by the image analysis unit (81) so that monitoring of multiple products and multiple electronic shelf labels (100) arranged on at least one shelf (30) can be performed smoothly.

[0274] Hereinafter, embodiments in which step (S305) is embodied are described with reference to FIGS. 17 to 20.

[0275] Referring to FIG. 17, the step (S305-1) of changing the installation conditions of at least one imaging device determined according to one embodiment that may be included in the step (S305) of FIG. 13 may include the step (S3051) of comparing at least one image of at least one shelf (30) taken by at least one imaging device (400) with a planogram of at least one shelf (30), and the step (S3053, S3055, S3057) of changing the installation conditions of at least one imaging device based on the comparison result of at least one image of at least one shelf (30) taken and the planogram of at least one shelf (30).

[0276] The step of changing the installation conditions of at least one imaging device based on the comparison result (S3053, S3055, S3057) may include the step of identifying at least one target product or at least one target electronic shelf label that is not included in at least one image among a plurality of products (PR) or a plurality of electronic shelf labels (100) included in the planogram (S3053), the step of obtaining location information of at least one target product or at least one target electronic shelf label (S3055), and the step of changing the installation conditions of at least one imaging device so that at least one target product and at least one target electronic shelf label can be monitored by at least one imaging device (400) based on the location information of at least one target product or at least one target electronic shelf label (S3057).

[0277] In step (S3051), the image analysis unit (81) of the server (200) can compare at least one image of at least one shelf (30) captured by at least one imaging device (400) with a planogram of at least one shelf (30).

[0278] For example, the image analysis unit (81) can detect a plurality of products (PR) and / or electronic shelf labels (100) from at least one image in which at least one imaging device (400) has captured at least one shelf (30).

[0279] For example, referring to FIG. 18, the image analysis unit (81) can acquire a plurality of images (CA1, CA2, CA3) in which at least one imaging device (400) captures the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3) provided in the first area (Ar1) inside the store.

[0280] In this case, the plurality of images (CA1, CA2, CA3) may include a first image (CA1) including a portion of the left edge of the 10-2 shelf (s2) and the 11-3 shelf (p3), a second image (CA2) including a portion of the center of the 10-2 shelf (s2), and a third image (CA3) including a portion of the right edge of the 10-2 shelf (s2) and the 11-1 shelf (p1).

[0281] The image analysis unit (81) can detect a plurality of products (PR) and / or electronic shelf labels (100) included in the first to third images (CA1, CA2, CA3).

[0282] Additionally, the image analysis unit (81) can detect multiple products (PR) and / or electronic shelf labels (100) in the planogram of at least one shelf (30) that is the subject of the image.

[0283] For example, information regarding the location of at least one imaging device (400) that has captured at least one shelf (30) may be stored in the shelf information database (26) of the server (200).

[0284] The image analysis unit (81) can detect the planogram of at least one shelf (30) that was photographed in the entire planogram inside the store based on information regarding the location of at least one imaging device (400) that photographed at least one shelf (30). In this case, the image analysis unit (81) can detect the planogram of at least one shelf (30) viewed from the location of at least one imaging device (400) that photographed at least one shelf (30) in the entire planogram inside the store.

[0285] For example, with reference to FIGS. 15 and 18, the image analysis unit (81) can detect the first planogram (PL1) viewed from the side of the 10-3 shelf (s3), 11-2 shelf (p2), and 11-4 shelf (p4) that are the subjects of the photograph in the entire planogram inside the store, including the 10-2 shelf (s2), 11-1 shelf (p1), and 11-3 shelf (p3).

[0286] Additionally, the image analysis unit (81) can detect a marker provided on at least one shelf (30) from a plurality of images (CA1, CA2, CA3) that have captured at least one shelf (30), and based on the detected marker (MK), can detect the planogram of at least one shelf (30) that was the subject of the capture in the entire planogram inside the store.

[0287] For example, referring to FIG. 18, at least one marker (MK) may be provided on one side of each of the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3). In this case, at least one marker (MK) may be provided on one side of the area corresponding to the area where the goods of each of the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3) are provided.

[0288] Here, the marker (MK) can be formed to have a specific pattern and can correspond to the relevant data of the shelf (30). For example, the marker (MK) may include a QR code, but is not limited thereto.

[0289] For example, a marker (MK) may be provided on one side corresponding to the area where the products of the first, second, and third layers of the 11-3 shelf (p3) are provided.

[0290] The image analysis unit (81) can detect a marker (MK) provided on at least one shelf (30) from a plurality of images (CA1, CA2, CA3) that have captured at least one shelf (30).

[0291] For example, the image analysis unit (81) can detect a plurality of markers (MK) provided to correspond to each layer of the 11-3 shelf (p3) from the first image (CA1). Additionally, the image analysis unit (81) can detect a plurality of markers (MK) provided to correspond to each layer of the 10-2 shelf (s2) from the second image (CA2), and detect a plurality of markers (MK) provided to correspond to each layer of the 11-3 shelf (p3) from the first image (CA1).

[0292] The image analysis unit (81) can detect the planogram of at least one shelf (30) that was the subject of the shot in the entire planogram inside the store based on at least one marker (MK) detected from the first to third images (CA1, CA2, CA3). For example, the image analysis unit (81) can detect the planogram of at least one shelf (30) that was the subject of the shot, which includes at least one marker (MK) detected in the entire planogram inside the store.

[0293] The image analysis unit (81) can detect a plurality of products (PR) and / or electronic shelf labels (100) in the planogram of at least one shelf (30) detected.

[0294] For example, the image analysis unit (81) can detect a plurality of products (PR) and / or electronic shelf labels (100) in the first planogram (PL1) of the 10-2 shelf (s2), 11-1 shelf (p1), and 11-3 shelf (p3) that are the subject of the image.

[0295] The image analysis unit (81) can compare a plurality of products (PR) and / or electronic shelf labels (100) detected from at least one image in which at least one imaging device (400) has captured at least one shelf (30) with a plurality of products (PR) and / or electronic shelf labels (100) detected from a planogram of at least one shelf (30) that was the subject of the image capture.

[0296] For example, the image analysis unit (81) can compare a plurality of products (PR) and / or electronic shelf labels (100) detected from the first to third images (CA1, CA2, CA3) with a plurality of products (PR) and / or electronic shelf labels (100) detected from the first planogram (PL1).

[0297] In step (S3053), the image analysis unit (81) of the server (200) can identify at least one target product or at least one target electronic shelf label that is not included in at least one image of at least one shelf (30) among a plurality of products (PR) and / or a plurality of electronic shelf labels (100) included in the planogram of at least one shelf (30) that was photographed.

[0298] For example, referring to FIG. 18, the image analysis unit (81) of the server (200) can identify target areas (TA1, TA2) that include a plurality of products (PR) and / or a plurality of electronic shelf labels (100) that are not included in the first to third images (CA1, CA2, CA3) in the first planogram (PL1), based on the comparison result of step (S3051).

[0299] In this case, the image analysis unit (81) of the server (200) can identify a first target area (TA1), which is the area between the first image (CA1) and the second image (CA2) in the first planogram (PL1), and a second target area (TA2), which is the area between the second image (CA2) and the third image (CA3).

[0300] The image analysis unit (81) of the server (200) can identify a plurality of products (PR) and a plurality of electronic shelf labels (100) included in the first target area (TA1) and the second target area (TA2) of the first planogram (PL1) as a plurality of target products and a plurality of target electronic shelf labels.

[0301] In step (S3055), the image analysis unit (81) of the server (200) can obtain location information of at least one target product or at least one target electronic shelf label.

[0302] In this case, the image analysis unit (81) of the server (200) can obtain location information of at least one target product or at least one target electronic shelf label based on the planogram of at least one shelf (30) that was taken.

[0303] For example, the image analysis unit (81) of the server (200) can obtain information regarding the locations of the 10-2 shelf (s2), 11-1 shelf (p1), and 11-3 shelf (p3) included in the 1st planogram (PL1) from the shelf information database (26).

[0304] The image analysis unit (81) of the server (200) can obtain information regarding the locations of a plurality of products (PR) and a plurality of electronic shelf labels (100) included in the first planogram (PL1) based on information regarding the locations of the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3). For example, the image analysis unit (81) can obtain information regarding the locations of a plurality of products (PR) and a plurality of electronic shelf labels (100) provided on the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3) from the planogram database (27).

[0305] Additionally, the image analysis unit (81) can select and obtain information regarding the locations of multiple target products and multiple target electronic shelf labels that are not included in the first to third images (CA1, CA2, CA3) that captured the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3), among the information regarding the locations of multiple products (PR) and multiple electronic shelf labels (100) arranged on the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3).

[0306] In step (S3057), the installation condition determination unit (29) of the server (200) can change the installation condition of at least one imaging device (400) based on location information of at least one target product or location information of at least one target electronic shelf label so that at least one target product and at least one target electronic shelf label can be monitored by at least one imaging device (400).

[0307] For example, at least one image of at least one shelf (30) may not include parts of multiple products (PR) and multiple electronic shelf labels (100) included in the planogram of at least one shelf (30). In this way, parts of multiple products (PR) and multiple electronic shelf labels (100) provided on at least one shelf (30) and on multiple electronic shelf labels (100) may not be captured by at least one imaging device (400).

[0308] In this case, at step (S3057), the installation condition determination unit (29) of the server (200) can change the installation condition of at least one imaging device (400) so that at least one imaging device (400) can photograph at least one target product and at least one target electronic shelf label that are not included in at least one image of at least one shelf (30).

[0309] For example, referring to FIG. 18, the installation condition determining unit (29) can change the installation condition of at least one imaging device (400) so that the plurality of target products and the plurality of target electronic shelf labels included in the first target area (TA1) and the second target area (TA2) specified in the first planogram (PL1) can be monitored by at least one imaging device (400). In this case, the installation condition of at least one imaging device (400) can be changed so that at least one imaging device (400) can be installed at a position facing the plurality of target products and the plurality of target electronic shelf labels.

[0310] Meanwhile, in step (S3053) included in the method (S305-1) of FIG. 17, a pattern code displayed on at least one electronic shelf label (100) may be utilized to specify at least one target electronic shelf label.

[0311] For example, referring to FIG. 19, step (S3053) may include: detecting a pattern code displayed by at least one electronic shelf label (100) included in at least one image (S3151); obtaining at least one identification information corresponding to the detected pattern code (S3153); obtaining a plurality of identification information of a plurality of electronic shelf labels included in a planogram (S3155); comparing at least one identification information obtained based on at least one image with a plurality of identification information obtained based on the planogram (S3157); specifying at least one target identification information that is not included in at least one identification information obtained based on at least one image among the plurality of identification information obtained based on the planogram (S3159); and specifying at least one target electronic shelf label corresponding to at least one target identification information (S3161).

[0312] In step (S3151), the image analysis unit (81) of the server (200) can detect at least one pattern code displayed by at least one electronic shelf label (100) in at least one image in which at least one imaging device (400) has captured at least one shelf (30).

[0313] In this case, the image analysis unit (81) can detect all of the multiple pattern codes displayed by all electronic shelf labels (100) included in at least one image. For example, referring to FIG. 18, the image analysis unit (81) can detect all of the multiple pattern codes displayed by all of the electronic shelf labels (100) included in the first to third images (CA1, CA2, CA3).

[0314] In step (S3153), the pattern code identification unit (82) of the server (200) can identify at least one pattern code detected in at least one image of at least one shelf (30) and obtain at least one identification information of at least one electronic shelf label (100) corresponding to at least one pattern code.

[0315] Here, at least one identifying information may be in the form of a string, a sequence of numbers, and / or a combination thereof. However, it is not limited thereto, and at least one identifying information may include a combination of various symbols other than characters and numbers.

[0316] In step (S3155), the processor (21) of the server (200) can obtain identification information of a plurality of electronic shelf labels (100) included in the planogram of at least one shelf (30) that is the subject of the photograph from the planogram database (27).

[0317] Specifically, the processor (21) of the server (200) can obtain location information of at least one electronic shelf label (100) included in the planogram of at least one shelf (30) that is the subject of the image. Subsequently, the processor (21) of the server (200) can obtain identification information of at least one electronic shelf label (100) that is matched to the location information of at least one electronic shelf label (100) and stored in the location information database (24).

[0318] For example, referring to FIG. 18, the processor (21) of the server (200) can obtain location information of all multiple electronic shelf labels (100) included in the first planogram (PL1) of the 10-2 shelf (s2), 11-1 shelf (p1), and 11-3 shelf (p3) that are the subject of the image. Additionally, the processor (21) of the server (200) can obtain identification information of all multiple electronic shelf labels (100) included in the first planogram (PL1).

[0319] In step (S3157), the processor (21) of the server (200) can compare at least one identification information obtained based on at least one image of at least one shelf (30) and a plurality of identification information obtained based on a planogram of at least one shelf (30) that was the subject of the image.

[0320] For example, referring to FIG. 18, the processor (21) of the server (200) can compare the identification information of all multiple electronic shelf labels (100) included in the first to third images (CA1, CA2, CA3) with the identification information of all multiple electronic shelf labels (100) included in the first planogram (PL1).

[0321] In step (S3159), the processor (21) of the server (200) can identify at least one target identification information that is not included in the plurality of identification information obtained based on at least one image of the at least one shelf (30) that was photographed, among the plurality of identification information obtained based on the planogram of at least one shelf (30) that was photographed.

[0322] For example, referring to FIG. 18, the processor (21) of the server (200) can identify at least one identification information that is not included in the identification information of all the electronic shelf labels (100) included in the first to third images (CA1, CA2, CA3) among the identification information of all the electronic shelf labels (100) included in the first planogram (PL1) as at least one target identification information.

[0323] In step (S3161), the processor (21) of the server (200) can identify at least one target electronic shelf label corresponding to at least one target identification information.

[0324] For example, the processor (21) of the server (200) can identify at least one electronic shelf label (100) displaying specific target identification information as at least one target electronic shelf label.

[0325] Referring to FIG. 20, the step (S305-2) of changing the installation conditions of at least one imaging device determined according to another embodiment that may be included in the step (S305) of FIG. 13 may include the step (S3251) of extracting overlapping regions between multiple images of at least one shelf (30) captured by a plurality of imaging devices (400) installed according to the determined installation conditions, and the step (S3253) of changing the installation conditions of at least one imaging device (400) so that multiple images of at least one shelf (30) in which the overlapping regions do not overlap can be obtained.

[0326] Step (S305-1) of FIG. 17 relates to changing the installation conditions of at least one imaging device (400) when multiple products (PR) and multiple electronic shelf labels (100) included in the planogram of at least one shelf (30) are not all included in the image of at least one shelf (30), whereas step (S305-2) relates to changing the installation conditions of at least one imaging device (400) when an overlapping area occurs between at least one image of at least one shelf (30).

[0327] In step (S3251), the image analysis unit (81) of the server (200) can extract overlapping regions between multiple images of at least one shelf (30) using a feature extraction algorithm.

[0328] For example, the image analysis unit (81) can extract overlapping areas between multiple images acquired by multiple imaging devices (400) at different locations for at least one shelf (30).

[0329] In step (S3253), the installation condition determination unit (29) of the server (200) may change the installation condition of at least one imaging device (400) so that when at least one imaging device (400) photographs at least one shelf (30), the overlapping area between multiple images of at least one shelf (30) does not overlap. In this case, for example, the installation condition determination unit (29) may reduce the number of at least one imaging device (400) from the previously determined installation condition or narrow the spacing between at least one imaging device (400).

[0330] Referring to FIG. 21, a method for positioning an imaging device for shelf monitoring (S400) according to another embodiment may include receiving a user input selecting some of a plurality of shelves (30) placed inside a store (S401), obtaining relevant information of at least one shelf (30) selected according to the user input (S403), and determining the installation conditions of at least one imaging device (400) for monitoring at least one shelf (30) based on the relevant information of at least one shelf (30) (S403).

[0331] In step (S401), a view of multiple shelves (30) arranged inside a store can be displayed on the display module (95) of the electronic device (300) and provided to the user. The user can select some of the multiple shelves (30) arranged inside the store through the electronic device (300). In this case, for example, the user can select some of the multiple shelves (30) through an input module (94) implemented as a touch screen.

[0332] For example, referring to FIG. 11, a first plurality of shelves (a1~a8), a second plurality of shelves (b1~b4), a third plurality of shelves (c1~c4), a fourth plurality of shelves (k1~k4), a fifth plurality of shelves (m1~m6), a sixth plurality of shelves (n1~n5), a seventh plurality of shelves (j1~j3), an eighth plurality of shelves (i1~i4), a ninth plurality of shelves (z1~z10), a tenth plurality of shelves (s1~s4), an eleventh plurality of shelves (p1~p4), a ttwelfth plurality of shelves (t1~t2), and a thirteenth plurality of shelves (q1~q2), each having different shapes and sizes, may be displayed on the display module (95) of the electronic device (300) in a manner that they are each arranged at various locations inside the store.

[0333] For example, the user can select the 10-2 shelf (s2), 10-3 shelf (s3), 11-1 shelf (p1), 11-2 shelf (p2), 11-3 shelf (p3), and 11-4 shelf (p4) provided in the 1st area (AR1) among a plurality of shelves provided inside the store.

[0334] In step (S403), the processor (21) of the server (200) can obtain relevant information of at least one shelf (30) selected according to user input. Since step (S403) is substantially the same as step (S101) of the method (S100), a description thereof is omitted here.

[0335] In step (S405), the processor (21) of the server (200) can determine the installation conditions of at least one imaging device (400) based on relevant information of at least one shelf (30). Since step (S405) is substantially the same as step (S103) of the method (S100), a description thereof is omitted here.

[0336] The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Hardware devices may be modified into one or more software modules to perform processing according to the present invention, and vice versa.

[0337] The specific embodiments described in this invention are examples and do not limit the scope of the invention in any way. For the sake of brevity of the specification, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted. Additionally, the connections of lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in actual devices. Furthermore, unless specifically stated as “essential,” “importantly,” etc., a component may not be strictly necessary for the application of the invention.

[0338] Furthermore, although the detailed description of the present invention has been explained with reference to preferred embodiments of the invention, those skilled in the art or those with ordinary knowledge in the relevant technical field will understand that various modifications and changes can be made to the invention without departing from the spirit and technical scope of the invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the claims.

[0339] The present invention has industrial applicability in that it can improve the efficiency of managing products and electronic shelf labels by enabling more efficient shelf monitoring through the systematic and automatic determination of installation conditions for an imaging device that monitors a plurality of shelves placed inside a store, based on relevant information of the plurality of shelves and / or relevant information of the imaging device.

Claims

1. An imaging device positioning method for shelf monitoring that determines the installation conditions of an imaging device monitoring a shelf executed by a server, A step of obtaining relevant information of at least one shelf placed inside a store; and A method for positioning an imaging device for shelf monitoring, comprising the step of determining the installation conditions of at least one imaging device for monitoring the at least one shelf based on relevant information of the at least one shelf.

2. In Paragraph 1, An imaging device positioning method for shelf monitoring, wherein the relevant information of the at least one shelf includes at least one of the position, width, height, depth, number of shelf plates, and distance from surrounding shelves of the at least one shelf.

3. In Paragraph 1, In the step of determining the installation conditions of the above-mentioned at least one imaging device, An imaging device positioning method for shelf monitoring, wherein the installation condition of at least one imaging device is determined based on the relevant information of at least one shelf through a pre-learned imaging device installation condition output model based on the relevant information dataset of the shelf.

4. In Paragraph 1, The step of acquiring relevant information of at least one imaging device; further comprising In the step of determining the installation conditions of the above-mentioned at least one imaging device, An imaging device positioning method for shelf monitoring, wherein the installation conditions of the at least one imaging device are determined based on relevant information of the at least one shelf and relevant information of the at least one imaging device.

5. In Paragraph 4, In the step of determining the installation conditions of the above-mentioned at least one imaging device, An imaging device positioning method for shelf monitoring, wherein the installation condition of the at least one imaging device is determined based on the relevant information of the at least one shelf and the relevant information of the at least one imaging device through a pre-learned imaging device installation condition output model based on the relevant information data set of the shelf and the relevant information data set of the imaging device.

6. In Paragraph 1, An imaging device positioning method for shelf monitoring, further comprising: a step of changing the determined installation conditions based on an analysis result of at least one image taken of the at least one shelf by the at least one imaging device installed according to the determined installation conditions.

7. In Paragraph 6, The step of changing the installation conditions determined above is, A step of comparing the at least one image with the planogram of the at least one shelf; and A method for positioning an imaging device for shelf monitoring, comprising the step of changing the installation conditions of at least one imaging device based on the above comparison results.

8. In Paragraph 7, The step of changing the installation conditions of the at least one imaging device based on the above comparison result is, A step of identifying at least one target product or at least one target electronic shelf label among a plurality of products or a plurality of electronic shelf labels included in the above planogram that is not included in the at least one image; A step of obtaining location information of at least one target product or at least one target electronic shelf label; and An imaging device positioning method for shelf monitoring, comprising: a step of changing the installation conditions of the at least one imaging device so that the at least one target product and the at least one target electronic shelf label can be monitored by the at least one imaging device based on the location information of the at least one target product or the location information of the at least one target electronic shelf label.

9. In Paragraph 8, The step of specifying at least one target product or at least one target electronic shelf label is, A step of detecting a pattern code displayed by at least one electronic shelf label included in the above at least one image; A step of obtaining at least one identification information corresponding to the detected pattern code; A step of obtaining multiple identification information of multiple electronic shelf labels included in the above planogram; A step of comparing at least one identification information obtained based on at least one image with a plurality of identification information obtained based on the planogram; A step of determining at least one target identification information not included in at least one identification information obtained based on at least one image among a plurality of identification information obtained based on the above planogram; and An imaging device positioning method for shelf monitoring, comprising the step of specifying at least one target electronic shelf label corresponding to at least one target identification information.

10. In Paragraph 6, The step of changing the installation conditions of the above-mentioned at least one imaging device is, A step of extracting an overlapping region between a plurality of images in which a plurality of imaging devices installed according to the above-determined installation conditions have captured the at least one shelf; and An imaging device positioning method for shelf monitoring, comprising: a step of changing the installation conditions of the at least one imaging device so that a plurality of images for the at least one shelf, wherein the overlapping areas do not overlap, can be obtained.

11. A server comprising a memory storing instructions and a program for performing an imaging device positioning method, and at least one processor that performs operations for performing the imaging device positioning method according to the instructions and the program; and An electronic device that provides information regarding the installation conditions of at least one imaging device determined by the server; comprising The above-mentioned at least one processor is, Obtain relevant information for at least one shelf, and An imaging device positioning system for shelf monitoring, which determines the installation conditions of the at least one imaging device for monitoring the at least one shelf based on relevant information of the at least one shelf.

12. In Paragraph 11, The above electronic device is, An imaging device positioning system for shelf monitoring configured to receive user input for inputting relevant information of at least one shelf.

13. In Paragraph 11, The above electronic device is, It is configured to receive user input selecting some of a plurality of shelves placed inside the store, and The above-mentioned at least one processor is an imaging device positioning system for shelf monitoring, which acquires relevant information of at least one shelf selected according to user input among the plurality of shelves arranged inside the store.

14. In Paragraph 11, The above-mentioned at least one processor is, Acquire relevant information of at least one imaging device, and An imaging device positioning system for shelf monitoring, wherein the installation conditions of the at least one imaging device are determined based on relevant information of the at least one shelf and relevant information of the at least one imaging device.

15. In Paragraph 11, The above-mentioned at least one processor is, The at least one imaging device installed according to the above-determined installation conditions receives data of at least one image of the at least one shelf, and An imaging device positioning system for shelf monitoring that changes the determined installation conditions based on the analysis results of at least one image.

Images