Mobile robot positioning method for shelf monitoring and mobile robot positioning system for shelf monitoring

Abstract

One embodiment provides a mobile robot positioning method performed by a processor for executing at least one command and a program stored in a memory. The mobile robot positioning method for shelf monitoring comprises the steps of: acquiring information related to at least one shelf disposed inside a store; and determining monitoring position information about a mobile robot for monitoring the at least one shelf on the basis of related information about the at least one shelf.

Description

Mobile robot positioning method for shelf monitoring and mobile robot positioning system for shelf monitoring

[0001] The present disclosure relates to a mobile robot positioning method for shelf monitoring and a mobile robot positioning system for shelf monitoring, and more specifically, to a mobile robot positioning method for shelf monitoring and a mobile robot positioning system for shelf monitoring that determines monitoring position information of a mobile robot monitoring a shelf based on relevant information of a shelf and / or relevant information of a mobile robot.

[0002] The use of electronic labels is gradually expanding in applications such as displaying product information on items displayed in stores. Also known as electronic shelf labels or electronic tags, electronic labels connect to computing devices via gateways to receive product information and display it on electronic paper displays. Their widespread adoption is attributed to the fact that they operate at low power, allowing for extended battery life, and enable changes to display information via communication networks, thereby reducing labor costs in store management.

[0003] When a new product is displayed on a shelf, a procedure is required to assign the product's information to be displayed on an electronic label. Generally, when a store manager reads the barcode attached to the electronic label with a terminal and then reads the product barcode and transmits it to a computing device, the computing device assigns the corresponding electronic label to the product, registers it in a database, and processes the product information to the electronic label so that the display is changed.

[0004] In this way, multiple newly displayed products and multiple electronic labels assigned information to those products can be paired and placed on the shelf. Each of the multiple electronic labels includes unique identification information, and the identification information may include location information of the electronic labels.

[0005] A real-gram containing the display status of multiple products arranged on a shelf can be formed based on the location information of the electronic label based on the identification information of the electronic label and the product information assigned to the electronic label.

[0006] Realogram is a real-time product display diagram that includes an image of products displayed on multiple shelves placed in a store, and related information of multiple products and multiple electronic labels included in the image, and can be utilized by a user to manage the store.

[0007] In order to generate such a real-gram, it is necessary to photograph multiple shelves placed within the store using an imaging device. To do this, a mobile robot that moves around the store using an imaging device and photographs multiple shelves can be used.

[0008] In this case, the mobile robot moves along a set path inside the store and can photograph multiple shelves placed within the store; however, if an error occurs in the path setting, precise monitoring of the shelves may be difficult.

[0009] For example, a mobile robot can photograph surrounding shelves while stationary at multiple monitoring positions along a path between multiple shelves arranged inside a store; in this case, the accuracy of monitoring all shelves inside the store can be determined by how the multiple monitoring positions are configured.

[0010] Accordingly, in order to improve the accuracy of shelf management within a store, there is a need for research on a system that automatically determines monitoring position information of mobile robots along the paths between multiple shelves placed within the store.

[0011] According to various embodiments of the present disclosure, the present invention aims to provide a mobile robot positioning method for shelf monitoring and a mobile robot positioning system for shelf monitoring that determine monitoring position information of a mobile robot for efficiently monitoring a plurality of shelves arranged inside a store.

[0012] According to various embodiments of the present disclosure, the present invention aims to provide a mobile robot positioning method for shelf monitoring and a mobile robot positioning system for shelf monitoring, which automatically changes the monitoring position information of a mobile robot based on the analysis results of images taken by the mobile robot of a plurality of shelves based on determined monitoring position information.

[0013] However, the technical problems that the various embodiments of the present disclosure aim to solve are not limited to the technical problems described above, and other technical problems may exist.

[0014] One embodiment is,

[0015] A mobile robot positioning method for shelf monitoring is performed by a processor executing at least one command and program stored in memory, comprising the steps of: acquiring relevant information of at least one shelf placed inside a store; and determining monitoring position information of a mobile robot for monitoring the at least one shelf based on the relevant information of the at least one shelf.

[0016] In another aspect, the relevant information of the at least one shelf may include at least one of the position, width, height, depth, number of shelf plates, and distance from surrounding shelves of the at least one shelf.

[0017] In another aspect, in the step of determining the monitoring position information of the mobile robot, the monitoring position information of the mobile robot can be determined based on the relevant information of at least one shelf through a mobile robot monitoring position information output model that has been learned based on the relevant information dataset of the shelf.

[0018] In another aspect, the mobile robot positioning method for shelf monitoring further includes the step of acquiring relevant information of the mobile robot, and in the step of determining monitoring position information of the mobile robot, the monitoring position information of the mobile robot may be determined based on relevant information of at least one shelf and relevant information of the mobile robot.

[0019] In another aspect, the relevant information of the mobile robot may include at least one of distance information from the mobile robot to a shelf, length and height information of the mobile robot, installation location information of an imaging device included in the mobile robot, and optical characteristics and tilting range of the imaging device.

[0020] In another aspect, in the step of determining the monitoring position information of the mobile robot, the monitoring position information of the mobile robot can be determined based on the relevant information of at least one shelf and the relevant information of the mobile robot through a mobile robot monitoring position information output model that has been learned based on the relevant information data set of the shelf and the relevant information data set of the mobile robot.

[0021] In another aspect, the mobile robot positioning method for semi-monitoring may further include the step of changing the determined monitoring position information based on the analysis result of at least one image taken by the mobile robot of the at least one shelf based on the determined monitoring position information.

[0022] In another aspect, the step of changing the determined monitoring position information may include comparing the at least one image with the planogram of the at least one shelf, and changing the monitoring position information of the mobile robot based on the result of the comparison.

[0023] In another aspect, the step of changing the monitoring position information of the mobile robot based on the above comparison result may include: a step of identifying at least one target product or at least one target electronic shelf label among a plurality of products or a plurality of electronic shelf labels included in the planogram that is not included in the at least one image; a step of obtaining position information of the at least one target product or the at least one target electronic shelf label; and a step of changing the monitoring position information of the mobile robot so that the at least one target product and the at least one target electronic shelf label can be monitored by the mobile robot based on the position information of the at least one target product or the position information of the at least one target electronic shelf label.

[0024] In another aspect, the step of specifying at least one target product or at least one target electronic shelf label may include: detecting a pattern code displayed by at least one electronic shelf label included in at least one image; obtaining at least one identification information corresponding to the detected pattern code; obtaining a plurality of identification information of a plurality of electronic shelf labels included in the planogram; comparing the at least one identification information obtained based on the at least one image with the plurality of identification information obtained based on the planogram; determining at least one target identification information among the plurality of identification information obtained based on the planogram that is not included in the at least one identification information obtained based on the at least one image; and specifying at least one target electronic shelf label corresponding to the at least one target identification information.

[0025] In another aspect, the step of changing the monitoring position information of the mobile robot may include the step of extracting an overlapping area between a plurality of images of the at least one shelf captured by the mobile robot based on the determined monitoring position information, and the step of changing the monitoring position information of the mobile robot so that a plurality of images of the at least one shelf in which the overlapping area does not overlap can be obtained.

[0026] In another aspect, the mobile robot positioning method for shelf monitoring further includes the step of receiving a user input selecting some of a plurality of shelves arranged inside the store, and in the step of obtaining relevant information of at least one shelf, relevant information of at least one shelf selected according to the user input among the plurality of shelves arranged inside the store can be obtained.

[0027] One embodiment is,

[0028] A mobile robot positioning system for shelf monitoring is provided, comprising a computing device including a memory storing instructions and a program for performing a mobile robot positioning method and at least one processor performing operations for performing the mobile robot positioning method according to the instructions and the program, and an electronic device providing monitoring position information of a mobile robot determined by the server, wherein the at least one processor acquires relevant information of at least one shelf and determines monitoring position information of the mobile robot for monitoring the at least one shelf based on the relevant information of the at least one shelf.

[0029] In another aspect, the electronic device may be configured to receive user input that inputs relevant information of the at least one shelf.

[0030] The electronic device is configured to receive user input selecting some of a plurality of shelves arranged inside a store, and the at least one processor can obtain relevant information of at least one shelf selected according to the user input among the plurality of shelves arranged inside the store.

[0031] In another aspect, the at least one processor may determine the monitoring position information of the mobile robot based on the relevant information of the at least one shelf and the relevant information of the mobile robot when acquiring relevant information of the mobile robot and determining the monitoring position information of the mobile robot.

[0032] In another aspect, the at least one processor receives data of at least one image of the mobile robot capturing the at least one shelf according to the determined monitoring location information, and can change the determined monitoring location information of the mobile robot based on the analysis result of the at least one image.

[0033] According to various embodiments of the present disclosure, a mobile robot positioning method for shelf monitoring and a mobile robot positioning system for shelf monitoring can be provided, which enable optimal monitoring of products and electronic shelf labels arranged on a plurality of shelves by determining monitoring position information of a mobile robot based on relevant information of a plurality of shelves arranged inside a store and / or relevant information of a mobile robot.

[0034] According to various embodiments of the present disclosure, a mobile robot positioning method for shelf monitoring and a mobile robot positioning system for shelf monitoring can be provided, which can increase the efficiency of shelf monitoring by automatically changing the monitoring position information of an imaging device based on the analysis results of an image of a plurality of shelves captured by a mobile robot based on determined monitoring position information and systematically optimizing the monitoring position information of an imaging device.

[0035] However, the effects obtainable through the various embodiments of the present disclosure are not limited to those mentioned above, and other unmentioned effects can be clearly understood from the description below.

[0036] FIG. 1 illustrates an exemplary configuration of a mobile robot positioning system for shelf monitoring according to one embodiment.

[0037] FIG. 2 illustrates a plurality of shelves arranged inside a store according to one embodiment.

[0038] FIG. 3 illustrates an exemplary configuration of an electronic shelf label according to one embodiment.

[0039] FIG. 4 is a block diagram illustrating an exemplary configuration of an electronic shelf label according to one embodiment.

[0040] FIG. 5 is a block diagram illustrating an exemplary configuration of a computing device according to one embodiment.

[0041] FIG. 6 illustrates the configuration of a shelf according to one embodiment.

[0042] FIGS. 7 and 8 are intended to illustrate a mobile robot monitoring position information output model utilized by a monitoring position information determination module of a computing device according to one embodiment.

[0043] FIG. 9 is a block diagram illustrating an exemplary configuration of a mobile robot according to one embodiment.

[0044] FIG. 10 is a perspective view illustrating an exemplary configuration of a mobile robot according to one embodiment.

[0045] FIG. 11 is a block diagram illustrating an exemplary configuration of an electronic device according to one embodiment.

[0046] FIG. 12 is a flowchart of a mobile robot positioning method for shelf monitoring according to one embodiment.

[0047] FIG. 13 is intended to illustrate a view in which multiple monitoring location information of a mobile robot is displayed for multiple shelves arranged inside a store according to one embodiment.

[0048] FIG. 14 is a flowchart of a mobile robot positioning method for shelf monitoring according to another embodiment.

[0049] FIG. 15 is a flowchart of a mobile robot positioning method for shelf monitoring according to another embodiment.

[0050] FIG. 16 is intended to illustrate a mobile robot moving inside a store and photographing shelves based on determined monitoring location information according to one embodiment.

[0051] FIG. 17 is intended to illustrate a display of multiple monitoring position information of a mobile robot for a plurality of shelves arranged inside a store according to one embodiment.

[0052] FIG. 18 is a flowchart of exemplary steps that may be included in the step of changing the determined monitoring location information included in the method of FIG. 15.

[0053] FIG. 19 is intended to explain a method for a mobile robot according to one embodiment to compare a plurality of images of at least one shelf with a planogram of at least one shelf.

[0054] FIG. 20 is a flowchart of exemplary steps that may be included in the step of specifying at least one target electronic shelf label of FIG. 18.

[0055] FIG. 21 is a flowchart of other exemplary steps that may be included in the step of changing the determined monitoring location information included in the method of FIG. 15.

[0056] FIG. 22 is a flowchart of a mobile robot positioning method for shelf monitoring according to another embodiment.

[0057] The present invention is capable of various modifications and may have various embodiments; therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various forms. In the following embodiments, terms such as "first," "second," etc., are used not in a limiting sense but for the purpose of distinguishing one component from another. Furthermore, singular expressions include plural expressions unless the context clearly indicates otherwise. Also, terms such as "include" or "have" mean that the features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added. Additionally, in the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily depicted for convenience of explanation, so the present invention is not necessarily limited to what is illustrated.

[0058] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals, and redundant descriptions thereof will be omitted.

[0059] FIG. 1 illustrates an exemplary configuration of a mobile robot positioning system (1000) for shelf monitoring according to one embodiment. FIG. 2 illustrates a configuration of multiple shelves arranged inside a store according to one embodiment. FIG. 3 illustrates an exemplary configuration of an electronic shelf label (100) according to one embodiment. FIG. 4 is a block diagram illustrating an exemplary configuration of an electronic shelf label (100) according to one embodiment. FIG. 5 is a block diagram illustrating an exemplary configuration of a computing device (200) according to one embodiment. FIG. 6 illustrates a configuration of a shelf (110) according to one embodiment. FIG. 7 and FIG. 8 are intended to explain mobile robot monitoring location information output models (M1, M2) utilized by a monitoring location information determination module of a computing device according to one embodiment. FIG. 9 is a block diagram illustrating an exemplary configuration of a mobile robot (300) according to one embodiment. FIG. 10 is a perspective view illustrating an exemplary configuration of a mobile robot (300) according to one embodiment. FIG. 11 is a block diagram illustrating an exemplary configuration of an electronic device (400) according to one embodiment. FIG. 12 is a flowchart of a mobile robot positioning method (S100) for shelf monitoring according to one embodiment. FIG. 13 is intended to explain the display of multiple monitoring position information of a mobile robot (300) for multiple shelves placed inside a store according to one embodiment. FIG. 14 is a flowchart of a mobile robot positioning method (S200) for shelf monitoring according to another embodiment. FIG. 15 is a flowchart of a mobile robot positioning method (S300) for shelf monitoring according to yet another embodiment. FIG. 16 is intended to explain the mobile robot (300) moving inside a store and photographing shelves based on determined monitoring position information according to one embodiment. FIG. 17 is intended to explain the display of multiple changed monitoring position information of a mobile robot (300) for multiple shelves placed inside a store according to one embodiment.FIG. 18 is a flowchart of exemplary steps that may be included in the step (S305) of changing determined monitoring location information included in the method of FIG. 15. FIG. 19 is intended to explain a method in which a mobile robot (300) according to one embodiment compares a plurality of images of at least one shelf with a planogram of at least one shelf. FIG. 20 is a flowchart of exemplary steps that may be included in the step (S3053) of specifying at least one target electronic shelf label of FIG. 18. FIG. 21 is a flowchart of other exemplary steps that may be included in the step (S305) of changing determined monitoring location information included in the method of FIG. 15. FIG. 22 is a flowchart of a mobile robot positioning method (S400) for shelf monitoring according to another embodiment.

[0060] -System(1000)

[0061] Referring to FIG. 1, a mobile robot positioning system (1000) for shelf monitoring according to one embodiment may include an electronic shelf label (100) provided on a shelf (110), a computing device (200), a mobile robot (300), and an electronic device (400). The electronic device (400) may include a portable user terminal.

[0062] The electronic shelf label (100) included in the system (1000) can be linked with a computing device (200), and the relevant information of the product displayed on the electronic shelf label (100) can be updated by the computing device (200). Data of the updated relevant information of the product is transmitted from the computing device (200) to the electronic shelf label (100), and the electronic shelf label (100) can display the updated relevant information of the product. In this case, the computing device (200) can update the relevant information of the product based on user input through the electronic device (400).

[0063] For example, a plurality of electronic shelf labels (100) and a plurality of products may be provided on a shelf (110). Each of the plurality of products may be assigned to each of the plurality of electronic shelf labels (100). A first product may be assigned to any first electronic shelf label among the plurality of electronic shelf labels (100), and a second product different from the first product may be assigned to any second electronic shelf label. A first electronic shelf label and a first product may be provided adjacent to each other, and a second electronic shelf label and a second product may be provided adjacent to each other.

[0064] The computing device (200) can update the relevant information of multiple products assigned to multiple electronic shelf labels (100). The computing device (200) can update the relevant information of multiple products in batches and transmit the updated relevant information of multiple products to multiple electronic shelf labels (100).

[0065] For example, the computing device (200) can update relevant information of any first product among a plurality of products and transmit the updated relevant information of the first product to the first electronic shelf label to which the first product is assigned. Additionally, the computing device (200) can update relevant information of any other second product among a plurality of products and transmit the updated relevant information of the second product to the second electronic shelf label to which the second product is assigned.

[0066] Additionally, the system (1000) can provide an environment in which a user can check the status of products displayed on shelves (110) and electronic shelf labels (100) through the electronic device (400) by generating a real-time real-time program based on images acquired while a mobile robot (300) moves inside a store where multiple shelves are arranged and transmitting it to an electronic device (400) used by the user.

[0067] Furthermore, the system (1000) can determine and provide to the user monitoring location information of a mobile robot (300) for photographing multiple shelves (110) placed inside a store in order to generate a real-gram for multiple products and multiple electronic shelf labels (100) arranged on multiple shelves (110).

[0068] For example, referring to FIG. 2, various types of shelves may be arranged in different areas within the store. For example, a first plurality of shelves (a1~a8), a second plurality of shelves (b1~b4), a third plurality of shelves (c1~c4), a fourth plurality of shelves (k1~k4), a fifth plurality of shelves (m1~m6), a sixth plurality of shelves (n1~n5), a seventh plurality of shelves (j1~j3), an eighth plurality of shelves (i1~i4), a ninth plurality of shelves (z1~z10), a tenth plurality of shelves (s1~s4), an eleventh plurality of shelves (p1~p4), a ttwelfth plurality of shelves (t1~t2), and a thirteenth plurality of shelves (q1~q2), each having different shapes and sizes, may be provided at various locations within the store.

[0069] The mobile robot (300) may be configured to move along various movement paths between multiple shelves (110) arranged inside the store and to photograph the multiple shelves (110). The movement path of the mobile robot (300) may be determined in various ways depending on the arrangement of the multiple shelves (110) inside the store. However, it is not limited thereto, and the movement path of the mobile robot (300) may be determined in various ways based on user input, and the predetermined movement path may also be modified in various ways.

[0070] In this case, the mobile robot (300) can be controlled to photograph multiple surrounding shelves (110) while stationary at multiple monitoring locations along the movement path. Position information for the multiple monitoring locations of the mobile robot (300) can also be pre-set.

[0071] For example, referring to FIG. 2, a mobile robot (300) can move inside a store along an arbitrary first movement path (Tr1) and photograph a plurality of shelves (110). In this case, the mobile robot (300) can be controlled to photograph a plurality of shelves (110) in the vicinity while remaining stationary at a plurality of monitoring positions (x1 to x11) on the first movement path (Tr1).

[0072] In this way, the mobile robot (300) can move inside the store and photograph multiple shelves (110) according to information about the pre-set movement path and monitoring location. The system (1000) can generate a real-gram of multiple products and multiple electronic shelf labels (100) arranged on the multiple shelves (110) based on the images of the multiple shelves (110) photographed by the mobile robot (300).

[0073] For example, the computing device (200) can perform calculations to generate a real-time real-gram based on data of an image of a shelf (110) received from a mobile robot (300).

[0074] The electronic shelf label (100), computing device (200), mobile robot (300), and electronic device (400) can be connected to each other through a network (500). Here, the network (500) according to the embodiment may refer to a connection structure capable of exchanging information between each node, such as the electronic shelf label (100), computing device (200), mobile robot (300), and electronic device (400).

[0075] For example, the network (500) may include, but is not limited to, a 3GPP (3rd Generation Partnership Project) network, an LTE (Long Term Evolution) network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, a DMB (Digital Multimedia Broadcasting) network, etc.

[0076] The electronic shelf label (100) can be wirelessly connected to a computing device (200) through a separate gateway (not shown). In this case, multiple gateways may be placed at regular intervals within the store, and multiple gateways may be connected to the computing device (200) via a wire. The electronic shelf label (100) can wirelessly communicate with an adjacent gateway, and the gateway can transmit data from the electronic shelf label (100) to the computing device (200).

[0077] However, it is not limited to this, and the electronic shelf label (100) can be connected to an electronic device (400) via wireless communication. For example, the electronic shelf label (100) can be paired with an electronic device (400) based on Bluetooth functionality.

[0078] Below, the configuration of the electronic shelf label (100) will be described with reference to FIGS. 3 and FIGS. 4.

[0079] - Electronic shelf label (100)

[0080] The electronic shelf label (100) may be an electronic device mounted on a shelf (110) to display information related to a product displayed on the shelf (110). For example, the electronic shelf label (100) may display information related to the product, such as a Korean product name, an English product name, the country of origin of the product, the price of the product, raw materials, weight / calories, and discount information.

[0081] Referring to FIG. 3, the electronic shelf label (100) provided on the shelf (110) may include a display module that displays information related to a product provided in one area of ​​the outer surface and an LED (15) provided in another area.

[0082] Here, LED (15) is an abbreviation for 'light emitting device' and may be a light-emitting device included in the electronic shelf label (100). A pattern code (PC) based on a signal from a computing device (200) or an electronic device (400) may be displayed on the display module of the electronic shelf label (100). Here, the pattern code (PC) may be a 2D code. For example, the pattern code (PC) may be a two-dimensional (2D) code having a size of 4x4, and each of the multiple pixels included in the pattern code (PC) may be displayed in black or white. Accordingly, the pattern code (PC) may be displayed in 65,536 (=2^16) different forms. However, it is not limited thereto, and the size of the pattern code (PC) may be designed to have various sizes other than 4x4.

[0083] Additionally, the electronic shelf label (100) may include at least one button (1, 2). For example, the electronic shelf label (100) may include a first button (1) and a second button (2). The first button (1) and the second button (2) may be provided in an area of ​​the outer surface of the electronic shelf label (100).

[0084] The LED (15) can be controlled based on a signal from a computing device (200) or an electronic device (400). Additionally, the types of light-emitting colors of the LED (15) may vary. For example, the light-emitting colors of the LED (15) may include seven colors: blue, red, green, yellow, sky blue, gray, and white. However, it is not limited thereto, and the light-emitting colors of the LED (15) may include various other colors in addition to the seven listed above.

[0085] A pairing signal may be transmitted from the electronic shelf label (100) to the electronic device (400) according to user input through the first button (1) and / or the second button (2). Accordingly, the electronic shelf label (100) may be paired with the electronic device (400) based on a Bluetooth method. For example, when a user presses the first button (1) and / or the second button (2), a pairing signal is transmitted from the electronic shelf label (100) to the electronic device (400), and the electronic shelf label (100) may be paired with the electronic device (400) based on a Bluetooth function.

[0086] Referring to FIG. 4, the electronic shelf label (100) may include a control unit (10), a display module (11), a communication module (12), a battery (13), and a memory (14).

[0087] The control unit (10) is a device that controls the operation of the electronic shelf label (100) and may include a system-on-chip (SOC) structure including a central processing unit (CPU) and / or a graphics processing unit (GPU), etc.

[0088] The control unit (10) can control the operation of the display module (11) so that product-related information displayed on the display module (11) is updated based on a control signal received from the outside.

[0089] The display module (11) may be a device that displays information related to a product. The display module (11) may include an Electronic Paper Display (EPD) that maintains the product information display state even when power is not supplied.

[0090] The electronic paper display is suitable for an electronic shelf label (100) that needs to reduce power consumption due to its bistability, which maintains the display state for a long time even when the power supply is interrupted.

[0091] Electronic paper displays are known to include twist ball types utilizing hemispherical twist balls charged with electrostatic charge, electrophoretic displays applying electrophoresis and microcapsules, and cholesterol liquid crystal displays utilizing cholesterol liquid crystals.

[0092] However, it is not limited to this, and the display module (11) may include any one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, and a 3D display.

[0093] The communication module (12) may include various types of communication devices that enable an external device and an electronic shelf label (100) to transmit and receive data. For example, an electronic device (400) and an electronic shelf label (100) can be paired through the communication module (12), and data from the electronic device (400) can be transmitted to the electronic shelf label (100) while paired. Additionally, data transmitted from a computing device (200) and delivered by a gateway can be received by the communication module (12).

[0094] The battery (13) may be a device that supplies power for driving components included in the electronic shelf label (100). For example, the battery (13) may include a lithium-ion battery. However, it is not limited thereto, and the battery (13) may include various types of batteries other than lithium-ion batteries.

[0095] Additionally, the battery (13) may include a coin-shaped power supply. However, it is not limited thereto, and the battery (13) may include various types of power supplies other than a coin-shaped one.

[0096] The memory (14) may be a storage unit in which data, instructions, and various programs are stored to perform operations necessary to update product-related information displayed on the display module (11). For example, the memory (14) may be a various storage device such as ROM, RAM, EPROM, flash drive, hard drive, etc. The memory (14) and the control unit (10) may be provided on a printed circuit board and electrically connected to each other.

[0097] The configuration of the computing device (200) will be described below with reference to FIG. 5.

[0098] -Computing device (200)

[0099] The computing device (200) can update the relevant information of a product assigned to an electronic shelf label (100) provided on a shelf (110) and transmit the updated relevant information of the product to the electronic shelf label (100). In this case, the computing device (200) can update the relevant information of the product based on user input received through the electronic device (400).

[0100] Specifically, in order to update the relevant information of a product assigned to an electronic shelf label (100), the computing device (200) can exchange necessary data with the electronic shelf label (100) and the electronic device (400). Accordingly, the computing device (200) can provide the environment necessary to update the relevant information of the product.

[0101] For example, the computing device (200) may provide an environment in which product-related information can be updated using an electronic device (400) (e.g., a portable user terminal type electronic device, a desktop type electronic device, etc.). The computing device (200) may include an application, data and / or commands, etc., for the product-related information update application to operate, and may transmit data based thereon to the electronic device (400).

[0102] Additionally, the computing device (200) can perform a predetermined operation to determine monitoring location information of the mobile robot (300). For example, the computing device (200) can determine monitoring location information of the mobile robot (300) by performing a predetermined operation based on relevant information of the shelf (110) and / or relevant information of the mobile robot (300).

[0103] The computing device (200) can transmit data of the monitoring location information of the determined mobile robot (300) to the electronic device (400), and the user can obtain information regarding the monitoring location information of the determined mobile robot (300) through the electronic device (400).

[0104] Referring to FIG. 5, the computing device (200) may include at least one processor (21) for data processing, a memory (22) for storing applications, data and / or instructions, etc., at least one communication module (23) for exchanging data with an external device, a location information database (24) for storing location information of an electronic shelf label (100), a template database (25) for storing a template containing product-related information, a shelf information database (26) for storing shelf-related information, and a planogram database (27) for storing planogram data containing information regarding the display status of a plurality of products and a plurality of electronic shelf labels (100) arranged on a shelf (110).

[0105] Additionally, the computing device (200) may be implemented to perform the functions of a product-related information update module (28) for updating product-related information, a monitoring location information determination module (29) for determining monitoring location information of a mobile robot (300), an image analysis module (81) for performing analysis on an image captured by the mobile robot (300), and a pattern code identification module (82) for identifying pattern codes displayed on a plurality of electronic shelf labels (100).

[0106] The processor (21) can control the overall operation of the components included in the computing device (200) to provide an environment in which a product-related information editing application and / or a mobile robot positioning application can operate on the electronic device (400).

[0107] The processor (21) may be a system-on-chip (SOC) including a central processing unit (CPU) and / or a graphics processing unit (GPU), and may execute an operating system (OS) and / or application programs stored in memory (22).

[0108] The processor (21) can communicate internally with each component included in the computing device (200) via a system bus and may include one or more predetermined bus structures, including a local bus.

[0109] The processor (21) can be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and other electrical units for performing functions.

[0110] The memory (22) can store one or more of an operating system (OS), various applications, data, and commands to provide an environment for performing a mobile robot positioning method for shelf monitoring and an environment for performing a product-related information editing method.

[0111] The memory (22) may include a program area and a data area. Here, the program area according to the embodiment may be linked between an operating system (OS) and functional elements that boot the computing device (200), and the data area may store data generated by the use of the computing device (200).

[0112] In one embodiment, the memory (22) may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and may be web storage that performs storage functions on the internet. Additionally, the memory (22) may be a recording medium that is detachable from the computing device (200).

[0113] The communication module (23) may include various types of communication devices that enable the computing device (200) to transmit and receive data with an external device.

[0114] The computing device (200) can transmit data based on applications, data and / or commands, etc., for operating a product-related information editing application and / or a mobile robot positioning application to an electronic device (400) through a communication module (23).

[0115] The location information database (24) can store location information of the electronic shelf labels (100). A plurality of electronic shelf labels (100) may be provided on the shelf (110) to correspond to a plurality of products. These plurality of electronic shelf labels (100) may each be placed at a specific location on the shelf (110), and specific location information of the plurality of electronic shelf labels (100) on the shelf (110) may be stored in the location information database (24). In this case, the location information of each of the plurality of electronic shelf labels (100) may be matched with the unique identification information of each of the plurality of electronic shelf labels (100) and stored in the location information database (24).

[0116] For example, among a plurality of electronic shelf labels (100), a first electronic shelf label may be provided in the 2nd layer, 3rd column of a first shelf provided in area A among a plurality of areas. In this case, the location information of the first electronic shelf label, 'the 2nd layer, 3rd column of the first shelf in area A', may be stored in a location information database (24) as the location information of the first electronic shelf label, corresponding to the unique identification information of the first electronic shelf label.

[0117] Here, the unique identification information of the electronic shelf label (100) may include a unique identification number assigned to the electronic shelf label (100). For example, the identification information of the electronic shelf label (100) may be in the form of a string of characters, a sequence of numbers, and / or a combination thereof. However, it is not limited thereto, and the identification information of the electronic shelf label (100) may include a combination of various symbols other than characters and numbers.

[0118] However, it is not limited to this, and the unique identification information of the electronic shelf label (100) may include a unique pattern code displayed by the electronic shelf label (100).

[0119] The location information database (24) is included in the computing device (200), but is not limited thereto, and may be implemented as an independent server configured separately from the computing device (200).

[0120] The template database (25) may be a database that stores templates to which product-related information is applied. For example, the template database (25) may store data regarding multiple templates to which product-related information can be entered. The multiple templates may have different forms depending on user input.

[0121] For example, multiple templates can be generated based on user input from a middle manager of a store using an electronic device (400). The middle manager can create a template of the desired form through the electronic device (400).

[0122] The electronic device (400) can transmit data regarding a plurality of templates to a computing device (200), and the data regarding a plurality of templates can be stored in a template database (25). However, it is not limited thereto, and the template database (25) may store data regarding a plurality of previously created templates.

[0123] The template database (25) is included in the computing device (200), but is not limited thereto, and may be implemented as an independent server configured separately from the computing device (200).

[0124] The shelf information database (26) can store data of related information for multiple shelves (110) placed in the store.

[0125] For example, referring to FIG. 6, at least one shelf (110) placed in a store may have a predetermined width (w1), height (h1), and depth (d1).

[0126] Additionally, the shelf (110) may include a plurality of shelf plates (pt1, pt2, pt3, pt4) arranged in layers on which a plurality of products can be provided. For example, the shelf (110) may include a first shelf plate (pt1), a second shelf plate (pt2), a third shelf plate (pt3), and a fourth shelf plate (4) arranged in order from top to bottom. In this case, the plurality of shelf plates (pt1, pt2, pt3, pt4) may be arranged at equal intervals.

[0127] Furthermore, as illustrated in FIG. 2, a plurality of shelves (110) may be provided at various locations within the store. In this case, the spacing between the plurality of shelves (110) may vary.

[0128] For example, referring to FIG. 2, the spacing between the first plurality of shelves (a1 to a8) and the second plurality of shelves (b1 to b4) may be different from the spacing between the fifth plurality of shelves (m1 to m6) and the eleventh plurality of shelves (p1 to p4).

[0129] The shelf information database (26) may store data of relevant information for at least one shelf (110), including at least one of the location, width (w1), height (h1), depth (d1), number of shelf plates, and distance from surrounding shelves. However, it is not limited thereto, and the relevant information for at least one shelf (110) may further include various information about at least one shelf (110) other than those mentioned above.

[0130] The shelf information database (26) is included in the computing device (200), but is not limited thereto, and may be implemented as an independent server configured separately from the computing device (200).

[0131] The planogram database (27) can store a planogram showing the arrangement of products and electronic shelf labels (100) on shelves (110) placed inside the store. The planogram may be an image showing the arrangement of products and electronic shelf labels (100) on shelves (110). Additionally, the planogram may include further information regarding the location of the products and the location of the electronic shelf labels (100).

[0132] A shelf (110) may be provided with a plurality of products and a plurality of electronic shelf labels (100). For example, a plurality of electronic shelf labels (100) and a plurality of products assigned to each of the plurality of electronic shelf labels (100) may be displayed at specific locations on the shelf (110). A planogram may show a configuration in which a plurality of products and a plurality of electronic shelf labels (100) are displayed correspondingly on the shelf (110).

[0133] The planogram database (27) is included in the computing device (200), but is not limited thereto, and may be implemented as an independent server configured separately from the computing device (200).

[0134] The product-related information update module (28) can update the product-related information displayed on the electronic shelf label (100). For example, the product-related information update module (28) can control the electronic shelf label (100) so that the product-related information displayed on the electronic shelf label (100) is periodically updated according to the product-related information update rule stored in the memory (22). The product-related information update rule stored in the memory (22) is predetermined by the user and can be modified at any time.

[0135] However, it is not limited to this, and the product-related information update module (28) may also control the electronic shelf label (100) so that the product-related information displayed on the electronic shelf label (100) is updated according to user input editing the product-related information.

[0136] For example, a product-related information update module (28) can control the electronic shelf label (100) so that information related to the product is updated by inputting information from user input into a template provided from the template database (25) and the product-related information is displayed on the electronic shelf label (100).

[0137] The monitoring location information determination module (29) can determine the monitoring location information of the mobile robot (300) for monitoring at least one shelf (110) based on the relevant information of at least one shelf (110).

[0138] For example, the monitoring location information determination module (29) can obtain relevant information of at least one shelf (110) stored in the shelf information database (26). In this case, the relevant information of at least one shelf (110) may include at least one of the location, width (w1), height (h1), depth (d1), number of shelf plates, and distance from surrounding shelves of at least one shelf (110).

[0139] However, it is not limited to this, and the monitoring location information determination module (29) may also obtain relevant information of at least one shelf (110) input into the electronic device (400) by the user.

[0140] The monitoring location information determination module (29) can determine the monitoring location information of a mobile robot (300) optimized for monitoring multiple products and multiple electronic shelf labels (100) provided on the shelf (110) through analysis of relevant information of at least one shelf (110) obtained.

[0141] For example, referring to FIG. 7, the monitoring position information determination module (29) can output monitoring position information of the mobile robot (300) by utilizing a first mobile robot monitoring position information output model (M1) that has been learned based on a relevant information data set of the shelf (110). The first mobile robot monitoring position information output model (M1) may be an artificial intelligence model that has been learned to output monitoring position information of the mobile robot (300) optimized for photographing the shelf (110) based on a relevant information data set of the shelf (110).

[0142] An artificial intelligence model is characterized by being created through learning based on a specific data set. Here, being created through learning means that an artificial intelligence model is created by being trained using multiple learning data by a learning algorithm, thereby setting up an artificial intelligence model to perform a desired characteristic (or objective).

[0143] Such learning may be performed on the device itself where the artificial intelligence model according to the present invention is utilized, or through a separate computing device and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples mentioned above.

[0144] An artificial intelligence model may be composed of multiple artificial neural network layers. Each of the multiple artificial neural network layers has multiple weight values ​​and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated so that the loss value or cost value obtained by the artificial intelligence model during the learning process is reduced or minimized.

[0145] Artificial neural networks may include deep neural networks (DNN), such as Convolutional Neural Networks (CNN), Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restructured Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory Network (LSTM), Classification Network, etc., such as GoogleNet, AlexNet, VGG Network, etc., but are not limited to the examples mentioned above.

[0146] For example, when relevant information of at least one shelf (110) is input into the first mobile robot monitoring position information output model (M1), monitoring position information of the mobile robot (300) can be output.

[0147] In this way, the monitoring position information determination module (29) can determine the monitoring position information of the mobile robot (300) optimized for photographing the shelf (110) according to the relevant information of the shelf (110) by utilizing the first mobile robot monitoring position information output model (M1).

[0148] However, it is not limited to this, and the monitoring location information determination module (29) may determine the monitoring location information of the mobile robot (300) corresponding to the related information of the shelf (110) obtained based on data stored in a monitoring location information database (not shown) where the related information of the shelf (110) and the monitoring location information of the mobile robot (300) are matched.

[0149] In addition, the monitoring location information determination module (29) can further utilize relevant information of the mobile robot (300) when determining the monitoring location information of the mobile robot (300).

[0150] For example, the monitoring location information determination module (29) can obtain relevant information of the mobile robot (300) through user input via the electronic device (400), obtain relevant information of the mobile robot (300) transmitted from the mobile robot (300), or obtain relevant information of the mobile robot (300) already stored in a mobile robot relevant information database (not shown).

[0151] In this case, the relevant information of the mobile robot (300) may include at least one of the distance information from the mobile robot (300) to the shelf (110), the length and height information of the mobile robot (300), the installation location information of the imaging device included in the mobile robot (300), and the optical characteristics and tilting range of the imaging device.

[0152] The optical characteristics of the imaging device included in the mobile robot (300) may include, for example, the angle of view and focal length of the mobile robot (300).

[0153] However, it is not limited to this, and the information related to the mobile robot (300) may include various additional information related to the mobile robot (300) other than those mentioned above.

[0154] Referring to FIG. 8, the monitoring location information determination module (29) can output monitoring location information by utilizing a second mobile robot monitoring location information output model (M2) that has been learned based on the relevant information data set of the shelf (110) and the relevant information data set of the mobile robot (300). The second mobile robot monitoring location information output model (M2) may be an artificial intelligence model that has been learned to output monitoring location information of the mobile robot (300) optimized for photographing the shelf (110) based on the relevant information data set of the shelf (110) and the relevant information data set of the mobile robot (300).

[0155] When the relevant information of at least one shelf (110) and the relevant information of the mobile robot (300) are input into the second mobile robot monitoring position information output model (M2), the monitoring position information of the mobile robot (300) can be output.

[0156] In this way, the monitoring location information determination module (29) can determine the monitoring location information of the mobile robot (300) optimized for photographing the shelf (110) based on the relevant information of the shelf (110) and the relevant information of the mobile robot (300) by utilizing the second mobile robot monitoring location information output model (M2).

[0157] However, it is not limited to this, and the monitoring location information determination module (29) may determine the monitoring location information of the mobile robot (300) corresponding to the relevant information of the shelf (110) and the relevant information of the mobile robot (300) obtained based on data stored in a monitoring location information database (not shown) in which the relevant information of the shelf (110) and the relevant information of the mobile robot (300) and the monitoring location information of the mobile robot (300) are matched.

[0158] The image analysis module (81) can detect objects by analyzing an image captured by the mobile robot (300).

[0159] For example, a mobile robot (300) can photograph an area on a shelf (110) where products and electronic shelf labels (100) are provided, and data of the image of the area where products and electronic shelf labels (100) are provided can be transmitted from the mobile robot (300) to a computing device (200).

[0160] The image analysis module (81) can detect the product and / or electronic shelf label (100) among various objects included in an image in which an area where the product and electronic shelf label (100) are provided is captured, based on an image extraction algorithm. Additionally, the image analysis module (81) can detect the pattern code displayed by the electronic shelf label (100) in the captured image.

[0161] Here, image extraction algorithms may include artificial intelligence models such as R-CNN, YOLO, and ViT. However, the types of image extraction algorithms are not limited to these.

[0162] Additionally, the image analysis module (81) can detect a desired specific product and / or electronic shelf label (100) from the planogram based on an image extraction algorithm.

[0163] Furthermore, the image analysis module (81) can compare the image of the product and / or electronic shelf label (100) detected from the image of the area where the product and electronic shelf label (100) are provided with the specific product and / or electronic shelf label (100) detected from the planogram.

[0164] The pattern code identification module (82) can detect the pattern code displayed by the electronic shelf label (100) detected in the image taken by the mobile robot (300) of the area where the electronic shelf label (100) is provided.

[0165] The electronic shelf label (100) can display a unique pattern code, and this pattern code can correspond to unique identification information of the electronic shelf label (100).

[0166] The pattern code identification module (82) can obtain unique identification information of the electronic shelf label (100) corresponding to the pattern code displayed by the identified electronic shelf label (100).

[0167] Here, the identification information of the electronic shelf label (100) may be in the form of a string, a sequence of numbers, and / or a combination thereof. However, it is not limited thereto, and the identification information of the electronic shelf label (100) may include a combination of various symbols other than characters and numbers.

[0168] For example, the pattern code identification module (82) can detect the pattern code displayed on the electronic shelf label (100) from an image in which an area where the electronic shelf label (100) is provided is captured based on an image extraction algorithm.

[0169] The detected pattern code may correspond to the identification information of the electronic shelf label (100). The corresponding pattern code and the identification information of the electronic shelf label (100) may be matched with each other and stored in a pattern code database (not shown). The pattern code identification module (82) may extract the identification information of the electronic shelf label (100) corresponding to the detected pattern code from the pattern code database.

[0170] In this way, the pattern code identification module (82) can detect the pattern code displayed by the electronic shelf label (100) based on the data of the image of the electronic shelf label (100) and obtain identification information of the electronic shelf label (100) corresponding to the pattern code.

[0171] Meanwhile, the processor (21) of the computing device (200) can generate a real-time program including the display status of the products displayed on the shelf (110) and the electronic shelf label (100) based on the image data of the shelf (110) acquired by the mobile robot (300).

[0172] For example, the processor (21) of the computing device (200) can generate a real-gram of the shelf (110) in real time by connecting multiple images of the shelf (110) acquired by the mobile robot (300). The real-gram of the shelf (110) may include not only image data but also information regarding the location of the mobile robot (300) that took the image. Accordingly, the real-gram may include both an image showing the product display status of the product and the electronic shelf label (100) and information regarding the location where the image was taken.

[0173] In the above description, it has been explained that a computing device (200) according to one embodiment performs functional operations as described above; however, depending on the embodiment, at least a portion of the functional operations performed by the computing device (200) may be performed by an external device (e.g., an electronic device (400)), and at least a portion of the functional operations performed by the external device may be further performed by the computing device (200), and various other embodiments may be possible.

[0174] The configuration of the mobile robot (300) will be described below with reference to FIGS. 9 and FIGS. 10.

[0175] - Mobile robot (300)

[0176] A mobile robot (300) according to one embodiment can monitor the display status of products and electronic shelf labels (100) while moving inside a store.

[0177] For example, the mobile robot (300) can move inside the store based on the movement path and monitoring location information determined by the computing device (200) and acquire data of images of products and electronic shelf labels (100) placed on a plurality of shelves (110) arranged inside the store.

[0178] The mobile robot (300) acquires data of an image of a product and an electronic shelf label (100) and can match the data of the captured image with data of information regarding a monitoring location, which is the location where the mobile robot (300) stopped to perform the shooting. The data in which the captured image data and the monitoring location information data are matched can be transmitted from the mobile robot (300) to a computing device (200).

[0179] Alternatively, data of a captured image from a mobile robot (300) may be transmitted to a computing device (200), and the computing device (200) may directly match the data of the captured image with the data of the monitoring location information.

[0180] However, it is not limited to this, and when the mobile robot (300) transmits the captured image data to the computing device (200), the computing device (200) can extract the location information of the electronic shelf label (100) based on the identification information of the electronic shelf label (100). Afterwards, the computing device (200) can match the image data received from the mobile robot (300) with the extracted location information of the electronic shelf label (100).

[0181] Referring to FIG. 9, a mobile robot (300) according to one embodiment may include a power module (30), a driving module (40), a sensor module (50), a wireless communication module (60), a memory (61), a navigation module (62), and a control unit (70).

[0182] Additionally, referring to FIG. 10, the mobile robot (300) may include a body (310) comprising a column portion (311) and a base (312) configured to be movable and provided at the bottom of the column portion (311). A plurality of wheels (wh1, wh2) that come into contact with the ground may be provided at the bottom of the base (312).

[0183] The power module (30), driving module (40), sensor module (50), wireless communication module (60), memory (61), navigation module (62), and control unit (70) can be provided inside the body (310).

[0184] The power module (30) may include a battery (31) that supplies power to the mobile robot (300) and a power management device (32) that controls the supply of power from the battery (31) to other components of the mobile robot (300).

[0185] The battery (31) is a source that supplies power necessary to drive the mobile robot (300), and may include, for example, a lithium-ion battery.

[0186] The power management device (32) is connected to the battery (31) and can regulate the voltage and current supplied by the battery (31) to meet the power system requirements of the mobile robot (300).

[0187] The drive module (40) may be a module that supports the operation of the mobile robot (300) by utilizing power provided from the power module (30).

[0188] For example, the drive module (40) may include a power generation device (41) that provides power to move a plurality of wheels (wh1, wh2) provided on the lower part of the base (312) so that the mobile robot (300) can move inside the store.

[0189] In this case, the rotational speed and rotational direction of the first wheel (wh1) and the second wheel (wh2) can be controlled independently. Accordingly, by controlling the rotational speed and rotational direction of the first wheel (wh1) and the second wheel (wh2) differently from each other, the mobile robot (300) can be controlled to change direction or rotate in place.

[0190] Additionally, the drive module (40) may include a body rotation device (42) that allows the column portion (311) to rotate relative to the base (312).

[0191] The column portion (311) may be configured to rotate relative to the base (312) with respect to a central axis penetrating the column portion (311) and the base (312), and the body rotation device (42) may provide power for the column portion (311) to rotate.

[0192] Accordingly, when the mobile robot (300) is stationary, only the column portion (311) can rotate, and the direction of the observation field of the first imaging device (c1) and the second imaging device (c2) provided on the column portion (311) can be switched.

[0193] The sensor module (50) may include a front detection sensor (51), an imaging device (52), a distance sensor (53), and an IMU sensor (54).

[0194] The front detection sensor (51) may include an RGB camera (l1) that captures the direction in which the mobile robot (300) is moving. For example, referring to FIG. 10, the RGB camera (l1) may be provided at the front of the pillar (311) so as to capture the front of the mobile robot (300).

[0195] A visual SLAM (Simultaneous localization and mapping; SLAM) system can be implemented through an RGB camera to simultaneously map the surrounding environment and real-time location of the mobile robot (300).

[0196] For example, data of an image of the surrounding environment of a mobile robot (300) moving inside a store, captured by an RGB camera, can be transmitted to a control unit (70) or to a computing device (200).

[0197] The control unit (70) or computing device (200) processes an image of the surrounding environment of the mobile robot (300) to perform mapping of the surrounding environment of the mobile robot (300), and at the same time, can estimate the real-time location of the mobile robot (300) within the store.

[0198] Additionally, the front detection sensor (51) may further include a distance sensor provided in front of the mobile robot (300). The distance sensor provided in front of the mobile robot (300) can detect obstacles appearing in front of the mobile robot (300). When an obstacle is detected by the distance sensor provided in front of the mobile robot (300), the mobile robot (300) can be controlled to move while avoiding the obstacle.

[0199] The imaging device (52) may be configured to allow the mobile robot (300) to photograph multiple shelves while moving within the store. For example, the imaging device (52) may include a camera.

[0200] For example, referring to FIG. 10, a first imaging device (c1) may be provided on the left side of the mobile robot (300), and a second imaging device (c2) may be provided on the right side.

[0201] The first imaging device (c1) can photograph a shelf located on the left side of the mobile robot (300) through the first field of view (FOV1).

[0202] The second imaging device (c2) can photograph a shelf located to the right of the mobile robot (300) through the second field of view (FOV2).

[0203] However, it is not limited to this, and only one of the first imaging device (c1) and the second imaging device (c2) may be included in the mobile robot (300). In this case, shelves provided in various directions of the monitoring mobile robot (300) can be photographed with a single imaging device by rotating the column portion (311).

[0204] For example, if the first imaging device (c1) is provided only on the left side of the mobile robot (300), the first imaging device (c1) can first photograph the shelf provided on the left side of the mobile robot (300). Afterwards, the column part (311) can rotate 180 degrees so that the first imaging device (c1) can also photograph the shelf provided on the right side of the mobile robot (300).

[0205] The data of the images of multiple shelves captured by the imaging device (52) can be transmitted to the control unit (70) and / or the computing device (200). The control unit (70) and / or the computing device (200) can utilize the data of the images of multiple shelves captured from the imaging device (52) to generate a real-gram.

[0206] The distance sensor (53) can be configured to sense the distance from the mobile robot (300) to a plurality of shelves as the mobile robot (300) moves within the store.

[0207] For example, referring to FIG. 10, a first distance sensor (e1) may be provided on the left side of the mobile robot (300), and a second distance sensor (e2) may be provided on the right side.

[0208] The first distance sensor (e1) can sense the distance from the mobile robot (300) to a shelf provided on the left side of the mobile robot (300).

[0209] The second distance sensor (e2) can sense the distance from the mobile robot (300) to a shelf provided on the right side of the mobile robot (300).

[0210] However, it is not limited to this, and only one of the first distance sensor (e1) and the second distance sensor (e2) may be included in the mobile robot (300). In this case, the distance from the mobile robot (300) to a shelf provided in various directions of the mobile robot (300) can be sensed by a single distance sensor through the rotation of the column part (311).

[0211] The IMU sensor (54) can be configured to measure the posture of the mobile robot (300). For example, the IMU sensor (54) may include a gyroscope and an accelerometer.

[0212] The IMU sensor (54) can measure the posture of a mobile robot (300) moving within a store in real time and can transmit data of the posture information of the mobile robot (300) to a control unit (70) and / or a computing device (200).

[0213] The posture information of the mobile robot (300) sensed by the IMU sensor (54) can be used by the control unit (70) or the computing device (200) to more accurately estimate the real-time position of the mobile robot (300) within the store when implementing a visual slam system based on data from the front detection sensor (51).

[0214] The wireless communication module (60) is a device that enables the mobile robot (300) to communicate wirelessly with the electronic shelf label (100), the computing device (200), and the electronic device (400).

[0215] For example, the wireless communication module (60) may be configured to wirelessly communicate with the computing device (200) and the electronic device (400) via wireless communication such as LTE, 5G, etc.

[0216] Additionally, for example, the wireless communication module (60) can be configured to perform short-range wireless communication with the electronic shelf label (100).

[0217] For example, the wireless communication module (60) may include a low-power Bluetooth (Bluetooth low energy; BLE) device, an RFID (Radio frequency identification) tag, an NFC (Near field communication) tag, etc.

[0218] For example, referring to FIG. 10, the wireless communication module (60) may include a plurality of wireless signal transceivers (i1, i2) for performing short-range wireless communication with the electronic shelf label (100).

[0219] Multiple wireless signal transceivers (i1, i2) may be configured to receive BLE signals, RFID signals, etc. Additionally, multiple wireless signal transceivers (i1, i2) may be configured as NFC tags.

[0220] For example, a first wireless signal transceiver (q1) may be provided on the left side of the mobile robot (300), and a second wireless signal transceiver (q2) may be provided on the right side.

[0221] The first wireless signal transceiver (q1) can wirelessly communicate with the electronic shelf label (100) displayed on the shelf provided on the left side of the mobile robot (300). Data of identification information of the electronic shelf label (100) displayed on the shelf provided on the left side of the mobile robot (300) can be obtained by the first wireless signal transceiver (q1).

[0222] The second wireless signal transceiver (q2) can wirelessly communicate with the electronic shelf label (100) displayed on the shelf provided on the right side of the mobile robot (300). Data of identification information of the electronic shelf label (100) displayed on the shelf provided on the right side of the mobile robot (300) can be obtained by the second wireless signal transceiver (q2).

[0223] However, it is not limited to this, and only one of the first wireless signal transceiver (q1) and the second wireless signal transceiver (q2) may be included in the mobile robot (300). In this case, wireless communication with electronic shelf labels (100) displayed on shelves arranged in various directions of the monitoring mobile robot (300) can be achieved through the rotation of the column portion (311) using a single wireless signal transceiver.

[0224] Furthermore, the wireless communication module (60) may include a communication device that transmits a beacon signal or a UWB signal. The beacon signal or UWB signal transmitted from the wireless communication module (60) may be transmitted to a computing device (200) through a gateway.

[0225] The memory (61) stores data that supports various functions of the mobile robot (300). For example, the memory (61) can store a number of applications (application programs or applications) running on the mobile robot (300), data for the operation of the mobile robot (300), and commands. At least some of these applications can be downloaded from the computing device (200) via wireless communication.

[0226] Memory (61) can be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc.

[0227] The navigation module (62) can guide the mobile robot (300) to move along a path inside a store where multiple shelves are provided.

[0228] For example, referring to FIG. 2, the navigation module (62) can guide the mobile robot (300) to move inside the store along a pre-set first movement path (Tr1).

[0229] In this case, the navigation module (62) can match the surrounding environment of the mobile robot (300) with the first movement path (Tr1) based on mapping data of the surrounding environment of the mobile robot (300). Accordingly, the navigation module (62) can guide the mobile robot (300) to move along the pre-set first movement path (Tr1) within the store where the mobile robot (300) is located.

[0230] The control unit (70) can control the overall operation of the components included in the mobile robot (300) and perform data processing for a series of operations to be described later.

[0231] The control unit (70) may be ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, or any other type of processor for performing functions.

[0232] The configuration of the electronic device (400) will be described below with reference to FIG. 11.

[0233] -Electronic device (400)

[0234] The electronic device (400) may be a device installed with a mobile robot positioning application and / or a product-related information editing application. The electronic device (400) may include a portable user terminal.

[0235] For example, the user of the electronic device (400) may be a store employee or a customer of the store.

[0236] The electronic device (400) can be implemented, for example, as a computer or portable terminal that can access the computing device (200) through a network (500).

[0237] Here, the computer may include, for example, a laptop, desktop, laptop, VR HMD (e.g., HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.) equipped with a web browser.

[0238] A portable terminal is a wireless communication device that ensures portability and mobility, and may include, for example, smartphones, tablet PCs, and wearable devices, as well as various devices equipped with communication modules such as Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, Ultrasonic, Infrared, WiFi, and LiFi.

[0239] Referring to FIG. 11, the electronic device (400) may include a processor (91), memory (92), a communication module (93), an input module (94), and a display module (95).

[0240] Various components included in the electronic device (400) can be designed to be contained within the housing of the electronic device (400).

[0241] In an embodiment, the processor (91) can control the overall operation of the components included in the electronic device (400) through the mobile robot positioning application and / or product related information editing application of the memory (92) to provide a mobile robot positioning environment and / or an environment for editing product related information.

[0242] For example, the processor (91) can control the operation of the communication module (93) and the input module (94) so ​​that a signal according to user input received through the input module (94) can be transmitted to the electronic shelf label (100) through the communication module (93).

[0243] The processor (91) may include a central processing unit (CPU) and / or a graphics processing unit (GPU). Additionally, the processor (91) may include at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and other electrical units for performing functions.

[0244] The memory (92) can store commands and data that can be used to create a mobile robot positioning environment and / or an environment for editing related information of a product.

[0245] In memory (92), a mobile robot positioning application and / or a product related information editing application may be stored.

[0246] A mobile robot positioning application can determine monitoring position information of a mobile robot (300) for monitoring a shelf (110) and provide a user interface that can provide information regarding the determined monitoring position information.

[0247] A product related information editing application can provide various types of user interfaces to provide an environment for editing product related information.

[0248] The memory (92) may include at least one non-transient computer-readable storage medium and a transient computer-readable storage medium. For example, the memory (92) may be various storage devices such as ROM, EPROM, flash drive, hard drive, etc. Additionally, the memory (92) may include web storage that performs data storage functions on the internet.

[0249] The communication module (93) may include various types of communication devices capable of transmitting and receiving data with an external device. For example, the communication module (93) may transmit and receive data with an electronic shelf label (100) and / or a computing device (200) via a wireless network.

[0250] The communication module (93) may be configured to transmit and receive data with an external device via Bluetooth. For example, the communication module (93) may receive a pairing signal from the electronic shelf label (100) and be paired with the electronic shelf label (100) via Bluetooth.

[0251] The input module (94) may be configured to receive various forms of user input from a user using the electronic device (400). For example, the input module (94) may include a touch screen that receives touch input from the user.

[0252] When the input module (94) is implemented as a touch screen, the input module (94) may be formed by being integrally combined with the display module (95). However, it is not limited thereto, and the input module (95) may further include a keyboard capable of receiving user input in the form of characters.

[0253] The display module (95) can display content related to a shelf monitoring application and / or a product-related information update application included in the memory (92).

[0254] For example, the display module (95) may include a display device that displays various types of user interface images to provide a mobile robot positioning environment or an environment for updating relevant information about a product.

[0255] For example, the display device may display information such as monitoring location information of the mobile robot (300) determined by the computing device (200) and mapped onto a map inside the store.

[0256] The display module (95) may include any one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, and a 3D display.

[0257] Hereinafter, a mobile robot positioning method (S100, S200, S300, S400) for shelf monitoring is described with reference to FIGS. 12 to 22.

[0258]

[0259] - Mobile robot positioning method (S100, S200, S300, S400)

[0260] A mobile robot positioning method (S100, S200, S300, S400) for shelf monitoring can be performed by a processor (21) of a computing device (200) according to one embodiment.

[0261] However, it is not limited to this, and at least part of the method (S100, S200, S300, S400) may be performed by the processor (91) of the electronic device (400), and other part may be performed by the processor (21) of the computing device (200).

[0262] For example, at least one of the processor (21) of the computing device (200) and the processor (91) of the electronic device (400) can perform a mobile robot positioning method (S100, S200, S300, S400) for shelf monitoring by executing at least one command stored in the memory (22) of the computing device (200) or the memory (92) of the electronic device (400).

[0263] In the following description, the processor (21) of the computing device (200) performs a mobile robot positioning method (S100, S200, S300, S400) for shelf monitoring.

[0264] Referring to FIG. 12, a mobile robot positioning method (S100) for shelf monitoring according to one embodiment may include the step (S101) of obtaining relevant information of at least one shelf (110) placed inside a store, and the step (S103) of determining monitoring position information of a mobile robot (300) for monitoring at least one shelf (110) based on the relevant information of at least one shelf (110).

[0265] In step (S101), the processor (21) of the computing device (200) may obtain relevant information of at least one shelf (110). Here, the relevant information of at least one shelf (110) may include at least one of the position, width (w1), height (h1), depth (d1), number of shelf plates, and distance from surrounding shelves of at least one shelf (110).

[0266] For example, the processor (21) of the computing device (200) can obtain relevant information about at least one shelf (110) placed in a store, which is stored in the shelf information database (26).

[0267] However, it is not limited to this, and the processor (21) of the computing device (200) can obtain relevant information of at least one shelf (110) that the user directly inputs into the electronic device (400). For example, the user can input relevant information of at least one shelf (110) for monitoring into the electronic device (400), and the data of the relevant information of at least one shelf (110) input into the electronic device (400) can be transmitted to the computing device (200).

[0268] Furthermore, the processor (21) of the server (200) may obtain relevant information of at least one shelf (110) based on a marker (MK) provided on at least one shelf (110) that the user has captured with an electronic device (300).

[0269] For example, referring to FIG. 19, a marker (MK) corresponding to information related to at least one shelf (110) may be provided on one side of at least one shelf (110).

[0270] The processor (21) of the server (200) can identify the marker (MK) by analyzing the image of the marker (MK) based on an image extraction algorithm, and obtain shelf-related information corresponding to the marker (MK) from the shelf information database (26).

[0271] In step (S103), the processor (21) of the computing device (200) can determine the monitoring position information of the mobile robot (300) based on the relevant information of at least one shelf (110).

[0272] For example, a monitoring location information determination module (29) of a computing device (200) can determine monitoring location information of a mobile robot (300) optimized for monitoring multiple products and multiple electronic shelf labels (100) arranged on at least one shelf (110) through analysis of relevant information of at least one shelf (110).

[0273] For example, the monitoring position information determination module (29) of the computing device (200) can obtain monitoring position information of the mobile robot (300) by utilizing a pre-learned imaging device monitoring position information output model based on a relevant information data set of various types of shelves (110).

[0274] Here, the imaging device monitoring location information output model may be an artificial intelligence model trained to output monitoring location information of a mobile robot (300) by taking relevant information of at least one shelf (110) as input.

[0275] However, it is not limited to this, and the monitoring location information determination module (29) of the computing device (200) may determine the monitoring location information of the mobile robot (300) corresponding to the relevant information of at least one shelf (110) based on data stored in a monitoring location information database (not shown) in which the relevant information of at least one shelf (110) and the monitoring location information of the mobile robot (300) are matched.

[0276] For example, referring to FIG. 13, there may be cases where monitoring location information of a mobile robot (300) for monitoring the 8-1 shelf (i1), 8-2 shelf (i2), 8-3 shelf (i3), 8-4 shelf (i4), 9-1 shelf (z1), 9-2 shelf (z2), 9-3 shelf (z3), 9-4 shelf (z4), 9-5 shelf (z5), 9-6 shelf (z6), 9-7 shelf (z7), 9-8 shelf (z8), 9-9 shelf (z9), 9-10 shelf (z10), 11-3 shelf (p3), and 11-4 shelf (p4) provided in the first area (Ar1) inside the store among the 8-1 shelf (i1~i4), 9-3 shelf (z3), and 11-4 shelf (p4).

[0277] In this case, the monitoring position information determination module (29) of the computing device (200) can determine a plurality of monitoring positions (x1, x2, x3, x4) as monitoring position information of the mobile robot (300) based on the relevant information of each of the 8-1 shelf (i1), 8-2 shelf (i2), 8-3 shelf (i3), 8-4 shelf (i4), 9-1 shelf (z1), 9-2 shelf (z2), 9-3 shelf (z3), 9-4 shelf (z4), 9-5 shelf (z5), 9-6 shelf (z6), 9-7 shelf (z7), 9-8 shelf (z8), 9-9 shelf (z9), 9-10 shelf (z10), 11-3 shelf (p3), and 11-4 shelf (p4) provided in the first area (Ar1).

[0278] Here, for example, the multiple monitoring locations (x1, x2, x3, x4) are multiple locations capable of capturing all of the multiple shelves provided in the first area (Ar1) inside the store, and may be locations spaced apart from each other by a predetermined distance on the first movement path (Tr1).

[0279] In a similar manner, multiple monitoring positions (x5, x6, x7, x8, x9, x10, x11) can be additionally determined based on relevant information of multiple shelves near the first movement path (Tr1).

[0280] As illustrated in FIG. 13, each of the plurality of monitoring locations (x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11) determined by the monitoring location information determination module (29) can be mapped onto a map inside a store where a plurality of shelves are provided and displayed through the display module (95) of the electronic device (400).

[0281] Thus, in step (S103), the processor (21) of the computing device (200) can determine information regarding a plurality of monitoring locations on the movement path of a mobile robot (300) optimized for monitoring at least one shelf (110), based on relevant information of at least one shelf (110) to be monitored.

[0282] Referring to FIG. 14, a mobile robot positioning method (S200) for shelf monitoring according to another embodiment may include the step of obtaining relevant information of at least one shelf (110) placed inside a store (S201), the step of obtaining relevant information of a mobile robot (300) (S203), and the step of determining monitoring position information of a mobile robot (300) for monitoring at least one shelf (110) based on the relevant information of at least one shelf (110) and the relevant information of the mobile robot (300) (S205).

[0283] In step (S201), the processor (21) of the computing device (200) can obtain relevant information of at least one shelf (110). Since step (S201) is substantially the same as step (S101), a description thereof is omitted here.

[0284] In step (S203), the processor (21) of the computing device (200) can obtain relevant information of the mobile robot (300). Here, the relevant information of the mobile robot (300) may include at least one of distance information from the mobile robot (300) to the shelf (110), length and height information of the mobile robot (300), installation location information of the imaging device (52) included in the mobile robot (300), optical characteristics of the imaging device (52), and tilting range.

[0285] The installation location information of the imaging device (52) included in the mobile robot (300) may include information regarding the height and direction of the location where the imaging device (52) is installed on the mobile robot (300). The optical characteristics of the imaging device (52) included in the mobile robot (300) may include, for example, the angle of view and focal length of the imaging device (52).

[0286] For example, the processor (21) of the computing device (200) can obtain information related to the mobile robot (300) stored in a mobile robot-related information database (not shown).

[0287] However, it is not limited to this, and the processor (21) of the computing device (200) can obtain relevant information of the mobile robot (300) from the mobile robot (300). In this case, the mobile robot (300) can transmit data of relevant information of the mobile robot (300) to the computing device (200) along with data of an image of at least one shelf (110).

[0288] Furthermore, the processor (21) of the computing device (200) can obtain relevant information about the mobile robot (300) that the user has directly entered into the electronic device (400). For example, the user can enter relevant information about the mobile robot (300) into the electronic device (400), and data of the relevant information about the mobile robot (300) entered into the electronic device (400) can be transmitted from the electronic device (400) to the computing device (200).

[0289] In step (S205), the processor (21) of the computing device (200) can determine the monitoring location information of the mobile robot (300) based on the relevant information of at least one shelf (110) and the relevant information of the mobile robot (300).

[0290] For example, the monitoring location information determination module (29) of the computing device (200) can determine the monitoring location information of the mobile robot (300) optimized for monitoring multiple products and multiple electronic shelf labels (100) arranged on at least one shelf (110) through analysis of the relevant information of at least one shelf (110) and the relevant information of the mobile robot (300).

[0291] For example, the monitoring position information determination module (29) of the computing device (200) can obtain monitoring position information of the mobile robot (300) by utilizing a pre-learned imaging device monitoring position information output model based on a relevant information data set of various types of shelves (110) and a relevant information data set of various types of mobile robots (300).

[0292] Here, the imaging device monitoring location information output model may be an artificial intelligence model trained to output monitoring location information of the mobile robot (300) by taking relevant information of at least one shelf (110) and relevant information of the mobile robot (300) as input.

[0293] However, it is not limited to this, and the monitoring location information determination module (29) of the computing device (200) may determine the monitoring location information of the mobile robot (300) corresponding to the relevant information of at least one shelf (110) and the relevant information of one imaging device (400) based on data stored in a monitoring location information database (not shown) in which the relevant information of at least one shelf (110) and the relevant information of the mobile robot (300) and the monitoring location information of the mobile robot (300) are matched.

[0294] Referring to FIG. 15, a mobile robot positioning method (S300) for shelf monitoring according to another embodiment may include the steps of: obtaining relevant information of at least one shelf (110) placed inside a store (S301); determining monitoring position information of a mobile robot (300) for monitoring at least one shelf (110) based on the relevant information of at least one shelf (110) (S303); and changing the determined monitoring position information based on the analysis result of at least one image of at least one shelf (110) taken by the mobile robot (300) based on the determined monitoring position information (S305).

[0295] Since steps (S301) and (S303) of the method (S300) are substantially identical to steps (S101) and (S103) of the method (S100), the description of steps (S301) and (S303) is omitted here.

[0296] Meanwhile, the step (S305) of changing the determined monitoring location information described below may also be included in the method (S200) of FIG. 14.

[0297] In step (S305), the monitoring position information determination module (29) of the computing device (200) can change the previously determined monitoring position information of the mobile robot (300) based on the analysis result of at least one image of at least one shelf (110) by the image analysis module (81).

[0298] For example, based on the monitoring location information determined in step (303), the mobile robot (300) moves inside the store and captures at least one image of at least one shelf (110), and the multiple products and multiple electronic shelf labels (100) placed on at least one shelf (110) may not all be included in the image. In this case, it is necessary to change the monitoring location information of the mobile robot (300) so that the multiple products and multiple electronic shelf labels (100) placed on the shelf (110) can all be included in the at least one image of at least one shelf (110).

[0299] Additionally, for example, based on the monitoring location information determined in step (303), there may be overlapping areas between at least one image of a mobile robot (300) moving inside a store and capturing at least one shelf (110). In this case, it is necessary to change the monitoring location information of the mobile robot (300) so that the overlapping areas between at least one image of a shelf (110) are minimized.

[0300] To this end, the monitoring location information determination module (29) of the computing device (200) can change the monitoring location information of the mobile robot (300) based on the result of the image analysis module (81) analyzing at least one image of the mobile robot (300) moving inside the store and taking at least one shelf (110) based on the previously determined monitoring location information.

[0301] For example, referring to FIG. 16, a plurality of images may be obtained in which the 8-1 shelf (i1), 8-2 shelf (i2), 8-3 shelf (i3), 8-4 shelf (i4), 9-1 shelf (z1), 9-2 shelf (z2), 9-3 shelf (z3), 9-4 shelf (z4), 9-5 shelf (z5), 9-6 shelf (z6), 9-7 shelf (z7), 9-8 shelf (z8), 9-9 shelf (z9), 9-10 shelf (z10), 11-3 shelf (p3), and 11-4 shelf (p4) are captured by the mobile robot (300) at the 1st monitoring position (xp1), 2nd monitoring position (xp2), and 3rd monitoring position (xp3) determined in step (S303).

[0302] The image analysis module (81) can perform analysis on multiple images taken by the mobile robot (300).

[0303] For example, the image analysis module (81) analyzes the 8-1 shelf (i1), 8-2 shelf (i2), 8-3 shelf (i3), 8-4 shelf (i4), 9-1 shelf (z1), 9-2 shelf (z2), 9-3 shelf (z3), 9-4 shelf (z4), 9-5 shelf (z5), 9-6 shelf (z6), 9-7 shelf (z7), 9-8 shelf (z8), 9-9 shelf (z9), 9-10 shelf (z10), 11-3 shelf (p3), and 11-4 shelf (p4) in a plurality of images in which the 8-1 shelf (i1), 8-2 shelf (i2), 8-3 shelf (i3), 8-4 shelf (i4), 9-1 shelf (z1), 9-2 shelf (z2), 9-3 shelf (z3), and 9-4 shelf are captured in a plurality of images in which the 8-1 shelf (i1), 8-2 shelf (i2), 8-3 shelf (i3), 8-4 shelf (i4), 9-1 shelf (z1), 9-2 shelf (z2), 9-3 shelf (z3), and 9-4 shelf are captured in a first area (Ar1). It can be determined that at least some of the products and electronic shelf labels (100) provided on shelf (z4), shelf 9-5 (z5), shelf 9-6 (z6), shelf 9-7 (z7), shelf 9-8 (z8), shelf 9-9 (z9), shelf 9-10 (z10), shelf 11-3 (p3), and shelf 11-4 (p4) are not included.

[0304] Accordingly, the monitoring location information determination module (29) can change the previously determined monitoring location information of the mobile robot (300) based on the analysis results of multiple images of the image analysis module (81).

[0305] For example, referring to FIG. 17, the monitoring position information determination module (29) can determine a plurality of monitoring positions (xp4, xp5, xp6, xp7) that are increased in number compared to the first to third monitoring positions (xp1 to xp3) as monitoring position information of the changed mobile robot (300).

[0306] Thus, in step (S305), the monitoring location information determination module (29) of the computing device (200) can change the previously determined monitoring location information of the mobile robot (300) based on the analysis result of at least one image of at least one shelf (110) by the image analysis module (81) so that monitoring of multiple products and multiple electronic shelf labels (100) arranged on at least one shelf (110) can be performed smoothly.

[0307] Hereinafter, embodiments in which step (S305) is embodied are described with reference to FIGS. 18 to 21.

[0308] Referring to FIG. 18, the step (S305-1) of changing the determined monitoring position information of at least one mobile robot (300) according to one embodiment that may be included in the step (S305) of FIG. 15 may include the step (S3051) of comparing at least one image of at least one shelf (110) taken by the mobile robot (300) with a planogram of at least one shelf (110), and the step (S3053, S3055, S3057) of changing the monitoring position information of the mobile robot (300) based on the comparison result of at least one image of at least one shelf (110) taken by the mobile robot (300) and the planogram of at least one shelf (110).

[0309] The step of changing the monitoring location information of the mobile robot (300) based on the comparison result (S3053, S3055, S3057) may include the step of specifying at least one target product or at least one target electronic shelf label that is not included in at least one image among the plurality of products (PR) or plurality of electronic shelf labels (100) included in the planogram (S3053), the step of obtaining location information of at least one target product or at least one target electronic shelf label (S3055), and the step of changing the monitoring location information of the mobile robot (300) so that at least one target product and at least one target electronic shelf label can be monitored by the mobile robot (300) based on the location information of at least one target product or the location information of at least one target electronic shelf label (S3057).

[0310] In step (S3051), the image analysis module (81) of the computing device (200) can compare at least one image of at least one shelf (110) taken by the mobile robot (300) with a planogram of at least one shelf (110).

[0311] For example, the image analysis module (81) can detect a plurality of products (PR) and / or electronic shelf labels (100) from at least one image in which the mobile robot (300) has captured at least one shelf (110).

[0312] For example, referring to FIG. 19, the image analysis module (81) can acquire a plurality of images (CA1, CA2, CA3) in which the mobile robot (300) takes a 10-2 shelf (s2), a 11-1 shelf (p1), and a 11-3 shelf (p3) provided inside the store.

[0313] In this case, the plurality of images (CA1, CA2, CA3) may include a first image (CA1) including a portion of the left edge of the 10-2 shelf (s2) and the 11-3 shelf (p3), a second image (CA2) including a portion of the center of the 10-2 shelf (s2), and a third image (CA3) including a portion of the right edge of the 10-2 shelf (s2) and the 11-1 shelf (p1).

[0314] The image analysis module (81) can detect a plurality of products (PR) and / or electronic shelf labels (100) included in the first to third images (CA1, CA2, CA3).

[0315] Additionally, the image analysis module (81) can detect multiple products (PR) and / or electronic shelf labels (100) in the planogram of at least one shelf (110) that is the subject of the image.

[0316] For example, the image analysis module (81) can detect the planogram of at least one shelf (110) that was the subject of the photograph in the entire planogram inside the store based on information regarding the position of the mobile robot (300) that took the photo of at least one shelf (110).

[0317] Information regarding the location of the mobile robot (300) that has photographed at least one shelf (110) may be stored in the shelf information database (26) of the computing device (200).

[0318] In this case, the image analysis module (81) can detect the planogram of at least one shelf (110) viewed from the position of the mobile robot (300) that captured at least one shelf (110) among the entire planogram inside the store.

[0319] For example, the image analysis module (81) can detect the first planogram (PL1) viewed from the side of the 10-3 shelf (s3), 11-2 shelf (p2), and 11-4 shelf (p4) that are the subjects of the shooting in the entire planogram inside the store, including the 10-2 shelf (s2), 11-1 shelf (p1), and 11-3 shelf (p3).

[0320] The image analysis module (81) can detect a plurality of products (PR) and / or electronic shelf labels (100) in the first planogram (PL1) of the 10-2 shelf (s2), 11-1 shelf (p1), and 11-3 shelf (p3) that are the subject of the image.

[0321] Additionally, the image analysis unit (81) can detect a marker provided on at least one shelf (110) from a plurality of images (CA1, CA2, CA3) that have captured at least one shelf (110), and based on the detected marker (MK), can detect the planogram of at least one shelf (110) that was the subject of the capture in the entire planogram inside the store.

[0322] For example, referring to FIG. 19, at least one marker (MK) may be provided on one side of each of the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3). In this case, at least one marker (MK) may be provided on one side of the area corresponding to the area where the goods of each of the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3) are provided.

[0323] Here, the marker (MK) can be formed to have a specific pattern and can correspond to the relevant data of the shelf (110). For example, the marker (MK) may include a QR code, but is not limited thereto.

[0324] For example, a marker (MK) may be provided on one side corresponding to the area where the products of the first, second, and third layers of the 11-3 shelf (p3) are provided.

[0325] The image analysis unit (81) can detect a marker (MK) provided on at least one shelf (110) from a plurality of images (CA1, CA2, CA3) that have captured at least one shelf (110).

[0326] For example, the image analysis unit (81) can detect a plurality of markers (MK) provided to correspond to each layer of the 11-3 shelf (p3) from the first image (CA1). Additionally, the image analysis unit (81) can detect a plurality of markers (MK) provided to correspond to each layer of the 10-2 shelf (s2) from the second image (CA2), and detect a plurality of markers (MK) provided to correspond to each layer of the 11-3 shelf (p3) from the first image (CA1).

[0327] The image analysis unit (81) can detect the planogram of at least one shelf (110) that was the subject of the photograph in the entire planogram inside the store based on at least one marker (MK) detected from the first to third images (CA1, CA2, CA3).

[0328] For example, the image analysis unit (81) can detect the planogram of at least one shelf (110) that is the subject of the shot, which includes at least one marker (MK) detected in the entire planogram inside the store.

[0329] The image analysis module (81) can compare a plurality of products (PR) and / or electronic shelf labels (100) detected from at least one image of at least one shelf (110) taken by the mobile robot (300) with a plurality of products (PR) and / or electronic shelf labels (100) detected from a planogram of at least one shelf (110) that was the subject of the image.

[0330] For example, the image analysis module (81) can compare a plurality of products (PR) and / or electronic shelf labels (100) detected from the first to third images (CA1, CA2, CA3) with a plurality of products (PR) and / or electronic shelf labels (100) detected from the first planogram (PL1).

[0331] In step (S3053), the image analysis module (81) of the computing device (200) can identify at least one target product or at least one target electronic shelf label that is not included in at least one image of at least one shelf (110) among a plurality of products (PR) and / or a plurality of electronic shelf labels (100) included in the planogram of at least one shelf (110) that was taken.

[0332] For example, referring to FIG. 19, the image analysis module (81) of the computing device (200) can identify target areas (TA1, TA2) that include a plurality of products (PR) and / or a plurality of electronic shelf labels (100) that are not included in the first to third images (CA1, CA2, CA3) in the first planogram (PL1), based on the comparison result of step (S3051).

[0333] In this case, the image analysis module (81) of the computing device (200) can identify a first target area (TA1), which is the area between the first image (CA1) and the second image (CA2) in the first planogram (PL1), and a second target area (TA2), which is the area between the second image (CA2) and the third image (CA3).

[0334] The image analysis module (81) of the computing device (200) can identify a plurality of products (PR) and a plurality of electronic shelf labels (100) included in the first target area (TA1) and the second target area (TA2) of the first planogram (PL1) as a plurality of target products and a plurality of target electronic shelf labels.

[0335] In step (S3055), the image analysis module (81) of the computing device (200) can obtain location information of at least one target product or at least one target electronic shelf label.

[0336] In this case, the image analysis module (81) of the computing device (200) can obtain location information of at least one target product or at least one target electronic shelf label based on the planogram of at least one shelf (110) that was taken.

[0337] For example, the image analysis module (81) of the computing device (200) can obtain information regarding the locations of the 10-2 shelf (s2), 11-1 shelf (p1), and 11-3 shelf (p3) included in the 1st planogram (PL1) from the shelf information database (26).

[0338] The image analysis module (81) of the computing device (200) can obtain information regarding the locations of a plurality of products (PR) and a plurality of electronic shelf labels (100) included in the first planogram (PL1) based on information regarding the locations of the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3). For example, the image analysis module (81) can obtain information regarding the locations of a plurality of products (PR) and a plurality of electronic shelf labels (100) provided on the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3) from the planogram database (27).

[0339] Additionally, the image analysis module (81) can select and obtain information regarding the locations of multiple target products and multiple target electronic shelf labels that are not included in the first to third images (CA1, CA2, CA3) that captured the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3), among the information regarding the locations of multiple products (PR) and multiple electronic shelf labels (100) arranged on the 10-2 shelf (s2), the 11-1 shelf (p1), and the 11-3 shelf (p3).

[0340] In step (S3057), the monitoring location information determination module (29) of the computing device (200) can change the monitoring location information of the mobile robot (300) so that at least one target product and at least one target electronic shelf label can be monitored by the mobile robot (300) based on the location information of at least one target product or the location information of at least one target electronic shelf label.

[0341] For example, at least one image of at least one shelf (110) may not include parts of multiple products (PR) and multiple electronic shelf labels (100) included in the planogram of at least one shelf (110). In this way, parts of multiple products (PR) and multiple electronic shelf labels (100) provided on at least one shelf (110) and on multiple electronic shelf labels (100) may not be captured by the mobile robot (300).

[0342] In this case, at step (S3057), the monitoring location information determination module (29) of the computing device (200) can change the monitoring location information of the mobile robot (300) so that the mobile robot (300) can photograph at least one target product and at least one target electronic shelf label that are not included in at least one image of at least one shelf (110).

[0343] For example, referring to FIG. 19, the monitoring location information determination module (29) can change the monitoring location information of the mobile robot (300) so that the multiple target products and multiple target electronic shelf labels included in the first target area (TA1) and the second target area (TA2) specified in the first planogram (PL1) can be photographed by the mobile robot (300). In this case, the monitoring location information of the mobile robot (300) can be changed so that the mobile robot (300) can stop at a position facing the multiple target products and multiple target electronic shelf labels to perform monitoring.

[0344] Meanwhile, in step (S3053) included in the method (S305-1) of FIG. 18, a pattern code displayed on at least one electronic shelf label (100) may be utilized to specify at least one target electronic shelf label.

[0345] For example, referring to FIG. 20, step (S3053) may include: detecting a pattern code displayed by at least one electronic shelf label (100) included in at least one image (S3151); obtaining at least one identification information corresponding to the detected pattern code (S3153); obtaining a plurality of identification information of a plurality of electronic shelf labels included in a planogram (S3155); comparing at least one identification information obtained based on at least one image with a plurality of identification information obtained based on the planogram (S3157); specifying at least one target identification information that is not included in at least one identification information obtained based on at least one image among the plurality of identification information obtained based on the planogram (S3159); and specifying at least one target electronic shelf label corresponding to at least one target identification information (S3161).

[0346] In step (S3151), the image analysis module (81) of the computing device (200) can detect at least one pattern code displayed by at least one electronic shelf label (100) in at least one image in which the mobile robot (300) has captured at least one shelf (110).

[0347] In this case, the image analysis module (81) can detect all of the multiple pattern codes displayed by all electronic shelf labels (100) included in at least one image. For example, referring to FIG. 19, the image analysis module (81) can detect all of the multiple pattern codes displayed by all of the electronic shelf labels (100) included in the first to third images (CA1, CA2, CA3).

[0348] In step (S3153), the pattern code identification module (82) of the computing device (200) can identify at least one pattern code detected in at least one image of at least one shelf (110) and obtain at least one identification information of at least one electronic shelf label (100) corresponding to at least one pattern code.

[0349] Here, at least one identifying information may be in the form of a string, a sequence of numbers, and / or a combination thereof. However, it is not limited thereto, and at least one identifying information may include a combination of various symbols other than characters and numbers.

[0350] In step (S3155), the processor (21) of the computing device (200) can obtain identification information of a plurality of electronic shelf labels (100) included in the planogram of at least one shelf (110) that is the subject of the image from the planogram database (27).

[0351] Specifically, the processor (21) of the computing device (200) can obtain location information of at least one electronic shelf label (100) included in the planogram of at least one shelf (110) that is the subject of the image. Subsequently, the processor (21) of the computing device (200) can obtain identification information of at least one electronic shelf label (100) that is matched to the location information of at least one electronic shelf label (100) and stored in the location information database (24).

[0352] For example, referring to FIG. 19, the processor (21) of the computing device (200) can obtain location information of all multiple electronic shelf labels (100) included in the first planogram (PL1) of the 10-2 shelf (s2), 11-1 shelf (p1), and 11-3 shelf (p3) that are the subject of the image. Based on this, the processor (21) of the computing device (200) can obtain identification information of all multiple electronic shelf labels (100) included in the first planogram (PL1).

[0353] In step (S3157), the processor (21) of the computing device (200) can compare at least one identification information obtained based on at least one image of at least one shelf (110) and a plurality of identification information obtained based on a planogram of at least one shelf (110) that was the subject of the image.

[0354] For example, referring to FIG. 19, the processor (21) of the computing device (200) can compare the identification information of all multiple electronic shelf labels (100) included in the first to third images (CA1, CA2, CA3) with the identification information of all multiple electronic shelf labels (100) included in the first planogram (PL1).

[0355] In step (S3159), the processor (21) of the computing device (200) can identify at least one target identification information that is not included in the plurality of identification information obtained based on at least one image of the at least one shelf (110) that was photographed, among the plurality of identification information obtained based on the planogram of at least one shelf (110) that was photographed.

[0356] For example, referring to FIG. 19, the processor (21) of the computing device (200) can identify at least one identification information that is not included in the identification information of all the electronic shelf labels (100) included in the first to third images (CA1, CA2, CA3) among the identification information of all the electronic shelf labels (100) included in the first planogram (PL1) as at least one target identification information.

[0357] In step (S3161), the processor (21) of the computing device (200) can identify at least one target electronic shelf label corresponding to at least one target identification information.

[0358] For example, the processor (21) of the computing device (200) can identify at least one electronic shelf label (100) displaying specific target identification information as at least one target electronic shelf label.

[0359] Referring to FIG. 21, the step (S305-2) of changing the monitoring position information of the determined mobile robot (300) according to another embodiment that may be included in the step (S305) of FIG. 15 may include the step (S3251) of extracting an overlapping area between a plurality of images in which the mobile robot (300) has captured at least one shelf (110) based on the determined monitoring position information, and the step (S3253) of changing the monitoring position information of the mobile robot (300) so that a plurality of images for at least one shelf (110) in which the overlapping area does not overlap can be obtained.

[0360] Step (S305-1) of FIG. 18 relates to changing the monitoring location information of the mobile robot (300) when multiple products (PR) and multiple electronic shelf labels (100) included in the planogram of at least one shelf (110) are not all included in the image of the at least one shelf (110), whereas Step (S305-2) relates to changing the monitoring location information of the mobile robot (300) when an overlapping area occurs between at least one image of the at least one shelf (110).

[0361] In step (S3251), the image analysis module (81) of the computing device (200) can extract overlapping regions between multiple images of at least one shelf (110) using a feature extraction algorithm.

[0362] For example, the image analysis module (81) can extract overlapping areas between multiple images acquired by the mobile robot (300) for at least one shelf (110) at different locations.

[0363] In step (S3253), the monitoring position information determination module (29) of the computing device (200) can change the monitoring position information of the mobile robot (300) so that, when the mobile robot (300) photographs at least one shelf (110), the overlapping areas between multiple images of at least one shelf (110) do not overlap. In this case, for example, the monitoring position information determination module (29) can reduce the number of previously determined monitoring positions or widen the spacing between the previously determined monitoring positions.

[0364] Referring to FIG. 22, a mobile robot positioning method (S400) for shelf monitoring according to another embodiment may include the step of receiving a user input selecting some of a plurality of shelves (110) placed inside a store (S401), the step of obtaining relevant information of at least one shelf (110) selected according to the user input (S403), and the step of determining monitoring position information of a mobile robot (300) for monitoring at least one shelf (110) based on the relevant information of at least one shelf (110) (S403).

[0365] In step (S401), a view of multiple shelves (110) arranged inside a store can be displayed on the display module (95) of the electronic device (400) and provided to the user. The user can select some of the multiple shelves (110) arranged inside the store through the electronic device (400). In this case, for example, the user can select some of the multiple shelves (110) through an input module (94) implemented as a touch screen.

[0366] For example, referring to FIG. 13, a first plurality of shelves (a1~a8), a second plurality of shelves (b1~b4), a third plurality of shelves (c1~c4), a fourth plurality of shelves (k1~k4), a fifth plurality of shelves (m1~m6), a sixth plurality of shelves (n1~n5), a seventh plurality of shelves (j1~j3), an eighth plurality of shelves (i1~i4), a ninth plurality of shelves (z1~z10), a tenth plurality of shelves (s1~s4), an eleventh plurality of shelves (p1~p4), a ttth plurality of shelves (t1~t2), and a thirteenth plurality of shelves (q1~q2), each having different shapes and sizes, may be displayed on the display module (95) of the electronic device (400) in a manner that they are each arranged at various locations inside the store.

[0367] For example, the user may select the 8-1 shelf (i1), 8-2 shelf (i2), 8-3 shelf (i3), 8-4 shelf (i4), 9-1 shelf (z1), 9-2 shelf (z2), 9-3 shelf (z3), 9-4 shelf (z4), 9-5 shelf (z5), 9-6 shelf (z6), 9-7 shelf (z7), 9-8 shelf (z8), 9-9 shelf (z9), 9-10 shelf (z10), 11-3 shelf (p3), and 11-4 shelf (p4) provided in the 1st area (Ar1) among a plurality of shelves provided inside the store.

[0368] In step (S403), the processor (21) of the computing device (200) can obtain relevant information of at least one shelf (110) selected according to user input. Since step (S403) is substantially the same as step (S101) of the method (S100), a description thereof is omitted here.

[0369] In step (S405), the processor (21) of the computing device (200) can determine the monitoring position information of the mobile robot (300) based on the relevant information of at least one shelf (110). Since step (S405) is substantially the same as step (S103) of the method (S100), a description thereof is omitted here.

[0370] The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Hardware devices may be modified into one or more software modules to perform processing according to the present invention, and vice versa.

[0371] The specific embodiments described in this invention are examples and do not limit the scope of the invention in any way. For the sake of brevity of the specification, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted. Additionally, the connections of lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in actual devices. Furthermore, unless specifically stated as “essential,” “importantly,” etc., a component may not be strictly necessary for the application of the invention.

[0372] Furthermore, although the detailed description of the present invention has been explained with reference to preferred embodiments of the invention, those skilled in the art or those with ordinary knowledge in the relevant technical field will understand that various modifications and changes can be made to the invention without departing from the spirit and technical scope of the invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the claims.

[0373] The present invention has industrial applicability in that it can improve shelf management efficiency by systematically and automatically determining monitoring position information of a mobile robot, which enables optimal monitoring of a shelf based on relevant information of the shelf and relevant information of the mobile robot.

Claims

1. A mobile robot positioning method performed by a processor executing at least one instruction and program stored in memory, A step of obtaining relevant information of at least one shelf placed inside a store; and A mobile robot positioning method for shelf monitoring, comprising: a step of determining monitoring position information of a mobile robot for monitoring the at least one shelf based on relevant information of the at least one shelf.

2. In Paragraph 1, A mobile robot positioning method for shelf monitoring, wherein the relevant information of the at least one shelf includes at least one of the position, width, height, depth, number of shelf plates, and distance from surrounding shelves of the at least one shelf.

3. In Paragraph 1, In the step of determining the monitoring location information of the above mobile robot, A mobile robot positioning method for shelf monitoring, wherein the monitoring position information of the mobile robot is determined based on the relevant information of at least one shelf through a mobile robot monitoring position information output model learned based on a relevant information dataset of the shelf.

4. In Paragraph 1, The step of obtaining relevant information of the above mobile robot; further comprising, In the step of determining the monitoring location information of the above mobile robot, A mobile robot positioning method for shelf monitoring, wherein monitoring position information of the mobile robot is determined based on relevant information of at least one shelf and relevant information of the mobile robot.

5. In Paragraph 4, A mobile robot positioning method for shelf monitoring, wherein the relevant information of the mobile robot includes at least one of distance information from the mobile robot to the shelf, length and height information of the mobile robot, installation location information of an imaging device included in the mobile robot, and optical characteristics and tilting range of the imaging device.

6. In Paragraph 4, In the step of determining the monitoring location information of the above mobile robot, A mobile robot positioning method for shelf monitoring, wherein the monitoring position information of the mobile robot is determined based on the relevant information of at least one shelf and the relevant information of the mobile robot through a mobile robot monitoring position information output model that has been learned based on the relevant information dataset of the shelf and the relevant information dataset of the mobile robot.

7. In Paragraph 1, A mobile robot positioning method for shelf monitoring, further comprising: a step of changing the determined monitoring position information based on an analysis result of at least one image taken by the mobile robot of at least one shelf based on the determined monitoring position information.

8. In Paragraph 7, The step of changing the above-determined monitoring location information is, A step of comparing the at least one image with the planogram of the at least one shelf; and A mobile robot positioning method for shelf monitoring, comprising the step of changing the monitoring position information of the mobile robot based on the above comparison result.

9. In Paragraph 8, The step of changing the monitoring position information of the mobile robot based on the above comparison result is, A step of identifying at least one target product or at least one target electronic shelf label among a plurality of products or a plurality of electronic shelf labels included in the above planogram that is not included in the at least one image; A step of obtaining location information of at least one target product or at least one target electronic shelf label; and A mobile robot positioning method for shelf monitoring, comprising: a step of changing the monitoring position information of the mobile robot so that the at least one target product and the at least one target electronic shelf label can be monitored by the mobile robot based on the position information of the at least one target product or the position information of the at least one target electronic shelf label.

10. In Paragraph 9, The step of specifying at least one target product or at least one target electronic shelf label is, A step of detecting a pattern code displayed by at least one electronic shelf label included in the above at least one image; A step of obtaining at least one identification information corresponding to the detected pattern code; A step of obtaining multiple identification information of multiple electronic shelf labels included in the above planogram; A step of comparing at least one identification information obtained based on at least one image with a plurality of identification information obtained based on the planogram; A step of determining at least one target identification information not included in at least one identification information obtained based on at least one image among a plurality of identification information obtained based on the above planogram; and A mobile robot positioning method for shelf monitoring, comprising the step of specifying at least one target electronic shelf label corresponding to at least one target identification information.

11. In Paragraph 7, The step of changing the monitoring location information of the above-mentioned mobile robot is, A step of extracting an overlapping area between a plurality of images in which a mobile robot captures at least one shelf based on the above-determined monitoring location information; and A mobile robot positioning method for shelf monitoring, comprising: a step of changing the monitoring position information of the mobile robot so that a plurality of images for the at least one shelf in which the overlapping area does not overlap can be obtained.

12. In Paragraph 1, The method further includes the step of receiving user input selecting some of a plurality of shelves arranged inside the store; A mobile robot positioning method for shelf monitoring, wherein, in the step of obtaining relevant information of at least one shelf, relevant information of at least one shelf selected according to user input among a plurality of shelves arranged inside the store is obtained.

13. A computing device comprising a memory storing instructions and a program for performing a mobile robot positioning method, and at least one processor that performs operations for performing the mobile robot positioning method according to the instructions and the program; and An electronic device that provides monitoring position information of a mobile robot determined by the above computing device; comprising, The above-mentioned at least one processor is, Obtain relevant information for at least one shelf, and A mobile robot positioning system for shelf monitoring, which determines monitoring position information of the mobile robot for monitoring the at least one shelf based on relevant information of the at least one shelf.

14. In Paragraph 13, The above electronic device is, A mobile robot positioning system for shelf monitoring configured to receive user input for inputting relevant information of at least one shelf.

15. In Paragraph 13, The above electronic device is, It is configured to receive user input selecting some of a plurality of shelves placed inside the store, and The above-mentioned at least one processor is a mobile robot positioning system for shelf monitoring that acquires relevant information of at least one shelf selected according to user input among the plurality of shelves arranged inside the store.

16. In Paragraph 13, The above-mentioned at least one processor is, Obtain relevant information regarding the above mobile robot, and A mobile robot positioning system for shelf monitoring, wherein, in determining the monitoring position information of the mobile robot, the monitoring position information of the mobile robot is determined based on the relevant information of at least one shelf and the relevant information of the mobile robot.

17. In Paragraph 13, The above-mentioned at least one processor is, According to the above-determined monitoring location information, the mobile robot receives data of at least one image of the at least one shelf, and A mobile robot positioning system for shelf monitoring that changes the monitoring position information of the determined mobile robot based on the analysis result of at least one image.

Images