Electronic shelf label recognition system

Abstract

An electronic shelf label recognition system according to one embodiment comprises: a plurality of electronic shelf labels that are arranged on a shelf on which a plurality of products are displayed, and that correspond to each of the plurality of products; a plurality of cameras for capturing the plurality of electronic shelf labels; and an electronic shelf label management server, which manages the plurality of electronic shelf labels, recognizes the plurality of electronic shelf labels captured by each of the plurality of cameras, and manages location information corresponding to the plurality of recognized electronic shelf labels.

Description

Electronic Shelf Label Recognition System

[0001] The embodiments relate to an electronic price tag recognition system, and more particularly, to a system for recognizing an electronic price tag photographed by a camera.

[0002] This refers to an e-paper terminal that displays real-time information, such as price and country of origin, for each product via a wireless network on store shelves. It's also called an Electronic Shelf Label (ESL). Utilizing an ESL offers the advantage of displaying desired information at any time, eliminating the need for frequent replacement of paper price tags. The market has recently been growing, centered around various retail outlets such as hypermarkets, supermarkets, health and beauty (H&B), and fashion stores.

[0003] The electronic shelf label system consists of a gateway, which handles wired and wireless communication, and tag devices that display product information. Using Internet of Things (IoT) technology, product information is transmitted from the ESL central management server to the gateway, which then wirelessly transmits it to multiple ESL tags. The tags display text, barcodes, and images.

[0004] Electronic paper (EPD), used in electronic shelf labels, is a technology that can replace the functions of existing displays and paper, such as liquid crystal displays (LCDs) and light-emitting diodes (LEDs). It consumes less power and allows stored information to be viewed even when the power is off.

[0005] Electronic price tags can automatically and simultaneously reflect product-specific inventory status, price changes, and discount periods across all stores, improving store management efficiency. Stores can also reduce costs associated with printing price tags.

[0006] Meanwhile, searching for desired products in large retail spaces like hypermarkets and department stores is not easy. To solve this problem, electronic shelf labels placed near products can be recognized to identify their location, allowing store managers or consumers to check the product's location or related information. However, the barcode or QR code displayed on the electronic shelf labels must be scanned with a specific terminal, and even when captured with a camera, identification is difficult due to issues such as camera resolution and shooting distance.

[0007] Various embodiments of the present disclosure provide an electronic price tag recognition system that can easily manage location information and product-related information of a product, etc. by displaying a specific identification code on an electronic price tag, taking a picture of the displayed identification code using a camera, and then recognizing the electronic price tag.

[0008] The problems to be solved through the embodiments of the present disclosure are not limited to the problems described above, and problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the embodiments belong from this specification and the attached drawings.

[0009] According to one embodiment, an electronic price tag recognition system includes: a plurality of electronic price tags arranged on a shelf on which a plurality of products are displayed, each corresponding to a plurality of products; a plurality of cameras that photograph the plurality of electronic price tags; and an electronic price tag management server that manages the plurality of electronic price tags, recognizes the plurality of electronic price tags photographed by each of the plurality of cameras, and manages location information corresponding to the plurality of recognized electronic price tags, wherein the electronic price tag management server generates a pattern image corresponding to each of the plurality of electronic price tags, controls the generated pattern image to be displayed at a corresponding electronic price tag time, acquires the displayed pattern image through the plurality of cameras, and recognizes the electronic price tag based on the acquired pattern image.

[0010] The above electronic price tag recognition system further includes a shared gateway connected to the plurality of electronic price tags through a predetermined network, and the electronic price tag management server can manage the plurality of electronic price tags through the shared gateway.

[0011] The above electronic price tag management server can, when the plurality of electronic price tags are recognized through the shared gateway, generate a pattern image corresponding to the recognized electronic price tags and transmit the generated pattern image to the corresponding electronic price tags through the shared gateway.

[0012] The above electronic price tag management server can transmit the generated pattern image to the corresponding electronic price tags through the shared gateway when product information including price information is transmitted to the plurality of electronic price tags through the shared gateway.

[0013] The above electronic price tag management server can control the timing for controlling the display of the generated pattern image on the electronic price tag and the shooting timing of the plurality of cameras to be synchronized.

[0014] The above plurality of cameras may be placed on a shelf facing the shelf on which the plurality of products are displayed.

[0015] The above electronic price tag management server can store the electronic price tag index corresponding to the generated pattern image by matching it.

[0016] The above electronic price tag management server can check the stored electronic price tag index corresponding to each of the plurality of recognized electronic price tag periods.

[0017] The above electronic price tag management server can update a planogram that maps location information corresponding to the plurality of recognized electronic price tag periods, the locations of the plurality of products, and the locations of shelves on which the plurality of products are arranged.

[0018] The above pattern image may be an N×N (N is a natural number greater than or equal to 2) matrix pattern image.

[0019] An electronic price tag recognition system according to various embodiments of the present disclosure can easily manage location information and product-related information of a product, etc. by displaying a specific identification code on an electronic price tag, taking a picture of the displayed identification code using a camera, and then recognizing the electronic price tag.

[0020] The effects of the embodiments are not limited to the effects described above, and effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the embodiments belong from this specification and the attached drawings.

[0021] Figure 1 is a perspective view showing an electronic price indicator.

[0022] Figure 2 is a schematic diagram of an electronic price tag recognition system according to one embodiment.

[0023] Figure 3 is a detailed schematic diagram of the ESL management server (300) illustrated in Figure 2.

[0024] FIG. 4 is a flowchart for explaining an electronic price tag recognition method according to another embodiment.

[0025] Figure 5 is an example attempt using the result of recognizing an electronic price tag according to another embodiment.

[0026] The terms used in the embodiments have been selected from widely used general terms as much as possible while considering the functions of the present invention. However, these may vary depending on the intention of engineers working in the field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, there are terms arbitrarily selected by the applicant, in which case the meanings thereof will be described in detail in the description of the relevant invention. Therefore, the terms used in the present invention should be defined not simply based on the name of the term, but based on the meaning of the term and the overall content of the present invention. When a part throughout the specification is said to "include" a certain component, this does not mean that other components are excluded, but that other components may be included, unless specifically stated otherwise. In addition, terms such as "-unit" and "-module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

[0027] As used herein, when an expression such as "at least one" precedes an array of elements, it modifies the entire array of elements rather than each individual element. For example, the expression "at least one of a, b, and c" should be interpreted to include a, b, c, or a and b, a and c, b and c, or a and b and c.

[0028] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings so that those skilled in the art can easily practice them. The present disclosure may be implemented in a form that can be implemented in the aerosol generating devices of the various embodiments described above, or may be implemented and practiced in various different forms, and is not limited to the embodiments described herein.

[0029] In the embodiment, an "electronic shelf label" or "electronic shelf label" is a device that displays specific information, such as a product name, price, and barcode, on a plastic tag. Also known as an Electric Shelf Label (ESL), ESL can be used. Gateway-related software can be installed where an electronic shelf label is needed, and the content to be displayed on the electronic shelf label can be set and applied. Here, the gateway acts as a public communication network between computers when multiple electronic shelf labels are interconnected with a network. For example, it acts as an intermediary between an electronic shelf label, a store POS device, and a product information management server. Through this intermediary, product information or product information is exchanged on the display of the electronic shelf label, such as electronic paper or an electronic panel, to display product information to consumers. Here, electronic paper (e-paper) is a display device that has a texture similar to regular paper, is easy on the eyes when viewed for a long time, and allows for free writing and erasing on the screen. It consumes little electronic power, and the electronic ink (e-ink), the core of the e-paper, has a memory effect, allowing it to remain in place even when the power is cut off.

[0030] In the embodiment, the "user terminal" may be implemented as a computer or portable terminal that can access a server or other terminal via a network. Here, the computer may include, for example, a notebook, desktop, laptop equipped with a web browser, VR HMD (e.g., HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.), etc. Here, the VR HMD includes all of the following: for PC (e.g., HTC VIVE, Oculus Rift, FOVE, Deepon, etc.), for mobile (e.g., GearVR, DayDream, Storm Magic, Google Cardboard, etc.), for console (PSVR), and Stand Alone models (e.g., Deepon, PICO, etc.) that are implemented independently. A portable terminal is, for example, a wireless communication device that ensures portability and mobility, and may include not only a smart phone, a tablet PC, and a wearable device, but also various devices equipped with communication modules such as Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, ultrasonic, infrared, WiFi, and LiFi. In addition, "network" refers to a connection structure that enables information exchange between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), the Internet (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, and wired and wireless television communication networks.Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth, infrared, ultrasonic, visible light communication (VLC), and LiFi.

[0031] In an embodiment, a "Plan-O-Gram," also called a merchandising display, refers to creating and displaying a product assortment within a store in the most optimized manner to meet customer needs or, from a business perspective, to encourage additional purchases, such as cross-selling or upselling. A planogram defines the location and quantity of products to be displayed and may include detailed specifications regarding the number and spacing of product sides. It may also include shelf layout, height, width, slope, depth, and other information deemed necessary or useful, such as required or recommended cooling conditions for raw meat and white wine. Rules and theories for creating a planogram may be established based on merchandising practices. In an embodiment, based on the recognition results of an electronic shelf label recognition system, product information and locations of displayed products may be identified and used to update an existing planogram. Therefore, it is possible to update the planogram based on product location recognition, and in conjunction with the planogram, it is also possible to automatically assign products to each electronic shelf label.

[0032] In the embodiment, “pattern image”, “identification marker”, and “matrix pattern image” should be understood as having the same meaning or as a superior or inferior concept, and are meant to refer to information displayed on an electronic price tag to facilitate image processing or vision recognition of the result of photographing an electronic price tag of a small size, such as 1 inch to 3 inches.

[0033] FIG. 1 is a perspective view showing an electronic price indicator according to one embodiment.

[0034] Referring to FIG. 1, an electronic price tag (10) according to one embodiment may be placed or combined on a product display shelf in a distribution space to display product information (hereinafter, referred to as "product information"). For example, the electronic price tag (10) may receive product information from a server (not shown) and visually display the product information received from the server, thereby providing product information to a user. Depending on the embodiment, the electronic price tag (10) may also be referred to as an electronic shelf label (ESL) or an electronic information label (EIL).

[0035] According to one embodiment, the electronic price indicator (10) may include a housing (100) and a display (110).

[0036] The housing (100) can form the overall appearance of the electronic price indicator (10), and components of the electronic price indicator (10) can be arranged in the internal space of the housing (100). For example, a printed circuit board (not shown) including a processor for controlling the operation of the electronic price indicator (10), a communication module (not shown) for wireless communication with a server, and / or a battery (not shown) can be arranged in the internal space of the housing (100), but is not limited thereto.

[0037] According to one embodiment, the housing (100) may be formed in an overall rectangular parallelepiped shape as illustrated in FIG. 1, but the shape of the housing (100) is not limited to the illustrated embodiment. In another embodiment, the housing (100) may be formed in a polygonal columnar shape in addition to the rectangular parallelepiped shape.

[0038] The display (110) is arranged so that at least one area is exposed to the outer surface of the outer surface of the housing (100), and can output visual information. For example, the display (110) can output product information or product information received from a server, but is not limited thereto. In the present disclosure, product information can include at least one of price information, manufacturer, country of origin, discount information, or identification information (e.g., barcode or QR code), and the expressions can be used with the same meaning below.

[0039] In an embodiment, the display (110) may output an identification marker to facilitate electronic price tag recognition under the control of an ESL management server or a video server. Here, the identification marker may be a matrix pattern image, but is not limited thereto. For example, the matrix pattern may be a black and white pattern such as 2x2, 3x3, or 4x4.

[0040] The display (110) can output product information through power supplied from a battery housed in the internal space of the housing (100).

[0041] Figure 2 is a schematic diagram of an electronic price tag recognition system according to one embodiment.

[0042] Referring to FIG. 2, the electronic price tag recognition system includes a plurality of electronic price tag timers (10), a plurality of cameras (20) for photographing a plurality of electronic price tags (10), a shared gateway (200), and an ESL management server (300).

[0043] A plurality of cameras (20) photograph an electronic price tag (10). In an embodiment, an image of an identification marker or pattern displayed on the electronic price tag (10) may be transmitted to the ESL management server (300). The plurality of cameras (20) may be placed on a shelf opposite the shelf on which the electronic price tag (10) is placed, or may be placed at a position where the electronic price tag (10) can be photographed. The plurality of cameras (20) may be connected to the ESL management server (300) wirelessly or by wire, and may photograph the electronic price tag (10) under the control of the ESL management server (300). In addition, the plurality of cameras (20) may be connected through a shared gateway (200) to transmit and receive data and control signals with the ESL management server (300).

[0044] A plurality of cameras (20) can capture images of a shelf including a plurality of products displayed thereon and a plurality of electronic price tags (10) corresponding to each product. The range of the shelf that can be captured may vary depending on the camera specifications or angle of view. For example, when capturing a 10 m long shelf, a camera positioned on an opposite shelf at a certain distance away, i.e., in a fixed position, can capture images of the electronic price tags (10) placed on the shelf.

[0045] The ESL management server (300) transmits product information, price information, etc. at multiple electronic price tag times (10) through the shared gateway (200).

[0046] The ESL management server (300) manages a plurality of electronic price tags (10) and recognizes a plurality of electronic price tags (10) captured by a plurality of cameras (20). The ESL management server (300) can manage location information corresponding to the recognized plurality of electronic price tags (10).

[0047] The ESL management server (300) generates a pattern image corresponding to each of a plurality of electronic price tags (10). Here, the pattern image may be a matrix pattern image. The ESL management server (300) can manage and store the electronic price tags and the pattern images by matching them. The ESL management server (300) controls the pattern image to be displayed on the corresponding electronic price tag (10). When the pattern image is displayed on the corresponding electronic price tag (10), for example, when a store opens or when product information needs to be updated, the ESL management server (300) transmits the corresponding pattern image to the plurality of electronic price tags (10) through the shared gateway (200) and controls the display of the corresponding pattern image. In this case, the pattern image and the display control command may be transmitted together, or the pattern image may be transmitted in advance and stored in the electronic price tag, and only the display control command may be transmitted. The ESL management server (300) controls the shooting of multiple cameras (20) and receives pattern images acquired from the multiple cameras (20). The ESL management server (300) recognizes the corresponding electronic price tag based on the received pattern images.

[0048] In an embodiment, when multiple electronic price tags are recognized through a shared gateway (200), the ESL management server (300) can generate an image of a pattern corresponding to the recognized electronic price tags. Thereafter, when store opening, product information update, or product information auto-assignment is required, the generated pattern image can be transmitted to the corresponding electronic price tags (10) through the shared gateway (200), and then the multiple cameras (20) can be controlled to obtain a pattern image displayed on the corresponding electronic price tags (10).

[0049] In an embodiment, when the ESL management server (300) transmits product information including price information to a plurality of electronic price tags (10) through a shared gateway (200), it may also transmit a pre-generated pattern image to the corresponding electronic price tags through the shared gateway (200).

[0050] In an embodiment, the ESL management server (300) can control the timing for controlling the display of a pre-generated pattern image on the electronic price tag (10) and the shooting timing of multiple cameras (20) to be synchronized. For example, if the pattern image transmitted to the electronic price tag (10) is controlled to be displayed for 1 minute or 10 minutes from 10:00 AM, the camera (20) shooting the electronic price tag (10) can be controlled to shoot once or multiple times during the display time.

[0051] In an embodiment, the ESL management server (300) may match and store the generated pattern image and the corresponding electronic shelf label index. In addition, the ESL management server (300) may recognize the pattern image transmitted from the camera (20) and then check the stored electronic shelf label index corresponding to each of the recognized plurality of electronic shelf labels. In addition, the ESL management server (300) may update a planogram that maps the location information corresponding to the recognized plurality of electronic shelf labels, the locations of the plurality of products, and the locations of the shelves on which the plurality of products are arranged. The detailed function of the ESL management server (300) will be described later with reference to FIG. 3.

[0052] Figure 3 is a detailed schematic diagram of the ESL management server (300) illustrated in Figure 2.

[0053] Referring to FIG. 3, the ESL management server (300) includes an ESL control unit (310), a camera control unit (320), a pattern image generation unit (330), an ESL recognition unit (340), and a planogram management unit (350). Of course, some components may be omitted or other additional components may be included.

[0054] The ESL control unit (310) manages multiple electronic price tags and manages and transmits price information, product information, etc. to be displayed on the electronic price tags.

[0055] The camera control unit (320) controls one camera or multiple cameras that capture one or more electronic price tags. The camera control unit (320) controls the camera so that the displayed pattern image can be captured when the pattern image is displayed on the electronic price tag. In addition, the camera control unit (320) can perform various image preprocessing on the pattern image transmitted from the camera, and can also perform image processing to extract only the pattern image that is the target of recognition.

[0056] The pattern image generation unit (330) can generate a pattern image to be transmitted to a plurality of electronic price tags, and manage the pattern image by matching it with information about the corresponding electronic price tags, for example, an identification ID.

[0057] The ESL recognition unit (340) extracts an electronic price tag area from an image of the electronic price tag including a pattern image transmitted from a camera, and then recognizes the pattern image displayed in the area. Furthermore, the pattern image can be directly extracted from the image. The ESL recognition unit (340) can compare the pattern image extracted from the image with pre-stored matching information to confirm whether there is a match. If there is a match, the location information of the electronic price tag can be confirmed, since it was captured by a camera at a fixed location within the store.

[0058] In the embodiment, it has been described that all functions are performed by the ESL management server (300), but a separate image processing server (not shown) that controls camera shooting and processes images captured from the camera to confirm matching information between the electronic price tag and the pattern image may be included, and the image processing server (not shown) may transmit the confirmation result to the ESL management server (300).

[0059] The planogram management unit (350) can update a planogram including product display, inventory management, and price information using the recognition result of the ESL recognition unit (340), i.e., the location information of the electronic price tag.

[0060] Fig. 4 is a flowchart for explaining an electronic price tag recognition method according to another embodiment.

[0061] Referring to FIG. 4, the ELS management server (300) generates a matrix image or a matrix pattern image. Here, the matrix image can be arbitrarily generated to match each electronic price tag. Then, the matrix image is transmitted to the electronic price tag (10). In addition, the ELS management server (300) transmits a shooting control command to a camera (20) positioned to shoot the electronic price tag (10) at a fixed location. Here, the ELS management server (300) can reduce unnecessary power consumption by synchronizing the timing at which the matrix image of the electronic price tag (10) is displayed with the timing at which the camera (20) shoots.

[0062] The electronic price display device (10) that receives the matrix image displays the matrix image on the display area of ​​the electronic price display device (10) for a predetermined time or period of time.

[0063] The camera (20) acquires an image captured at the time of electronic price tag display (10) according to the shooting control command of the ESL management server (300). The camera (20) transmits the captured image to the ESL management server (300). Here, the captured image includes a plurality of electronic price tag display periods and a matrix image displayed on the corresponding electronic price tag.

[0064] The ELS management server (300) recognizes the electronic price tag by recognizing the matrix image from the captured image received from the camera (20). Based on the recognition result, the location information and product location information corresponding to the electronic price tag can be confirmed. In addition, the planogram can be updated using the recognition result.

[0065] Figure 5 is an example diagram using the results of recognizing an electronic price tag according to another embodiment.

[0066] Referring to FIG. 5, the entire product layout screen (500) of the store is illustrated. When a specific product category is selected (501), sales trend analysis data (502) for the corresponding product and a video screen (503) capturing the corresponding product category can be viewed. According to the embodiment, the electronic price tag recognition results, i.e., product location confirmation and product verification, can be used to update the aforementioned planogram data.

[0067] An electronic price tag recognition system according to an embodiment displays a specific identification code on an electronic price tag, takes a picture of the displayed identification code using a camera, and then recognizes the electronic price tag, thereby easily managing location information and product-related information of the product. In addition, the system can be utilized for automatic assignment of electronic price tags by linking with a planogram or planogram database.

[0068] The electronic shelf label recognition algorithm according to one embodiment may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically includes computer-readable instructions, data structures, other data, such as program modules, in a modulated data signal, or other transmission mechanism, and includes any information delivery media.

[0069] The description of the above-described embodiments is merely illustrative, and those skilled in the art will appreciate that various modifications and equivalent alternative embodiments are possible. Therefore, the true scope of protection for the invention should be defined by the appended claims, and all differences within the scope equivalent to the content described in the claims should be construed as being included within the scope of protection defined by the claims.

[0070] The present invention has industrial applicability in that it can improve the efficiency of product management in a store by providing an electronic price tag recognition system that can easily manage location information and product-related information of a product corresponding to the electronic price tag by using a camera to capture a specific identification code displayed on an electronic price tag provided on a shelf, recognizing the electronic price tag based on the captured image of the identification code, and thereby manage the location information and product-related information of the product corresponding to the electronic price tag.

Claims

1. A plurality of products are arranged on a shelf and a plurality of electronic price tags corresponding to each of the plurality of products; A plurality of cameras for photographing the plurality of electronic price tags; and An electronic price tag management server is included that manages the plurality of electronic price tags, recognizes the plurality of electronic price tags captured from each of the plurality of cameras, and manages location information corresponding to the plurality of recognized electronic price tag periods. The electronic price tag management server generates a pattern image corresponding to each of the plurality of electronic price tags, controls the generated pattern image to be displayed on the corresponding electronic price tag, acquires the displayed pattern image through the plurality of cameras, and recognizes the electronic price tag based on the acquired pattern image, an electronic price tag recognition system.

2. In paragraph 1, Further comprising a shared gateway connected to the above plurality of electronic price tags through a predetermined network, The above electronic price tag management server, An electronic price tag recognition system that manages the plurality of electronic price tags through the shared gateway.

3. In paragraph 2, The above electronic price tag management server, An electronic price tag recognition system that, when a plurality of electronic price tags are recognized through the shared gateway, generates a pattern image corresponding to the recognized electronic price tags and transmits the generated pattern image to the corresponding electronic price tags through the shared gateway.

4. In paragraph 2, The above electronic price tag management server, An electronic price tag recognition system that transmits the generated pattern image to the corresponding electronic price tags through the shared gateway when product information including price information is transmitted to the plurality of electronic price tags through the shared gateway.

5. In paragraph 2, The above electronic price tag management server, An electronic price tag recognition system that controls the timing for displaying the generated pattern image on the electronic price tag and the shooting timing of the plurality of cameras to be synchronized.

6. In paragraph 1, The above multiple cameras, An electronic price tag recognition system placed on a shelf facing the shelf on which the above-mentioned multiple products are displayed.

7. In paragraph 1, The above electronic price tag management server, An electronic price tag recognition system that matches and stores an electronic price tag index corresponding to the generated pattern image.

8. In paragraph 7, The above electronic price tag management server, An electronic price tag recognition system that checks the stored electronic price tag index corresponding to each of the plurality of electronic price tags recognized above.

9. In paragraph 1, The above electronic price tag management server, An electronic price tag recognition system that updates a planogram that maps location information corresponding to the plurality of recognized electronic price tags, the locations of the plurality of products, and the locations of shelves on which the plurality of products are arranged.

10. In paragraph 1, The above pattern image is an electronic price tag recognition system, which is an N×N (N is a natural number greater than or equal to 2) matrix pattern image.

Images