Unmanned store product checkout method and device

Abstract

A method and apparatus for checking out merchandise in an unmanned store according to one embodiment of the present invention is disclosed. The method may include the steps of: recognizing a first product that has entered the checkout counter by a product recognition unit; selecting at least one learning model for identifying the first product based on the recognition result of the first product; acquiring a first image of the first product by a product identification unit; and identifying the first product based on the at least one learning model and the first image.

Description

[Technical Field]

[0001] The present invention relates to a method and device for checking out merchandise in an unmanned store. [Background technology]

[0002] The following description is merely intended to provide background information regarding embodiments of the present invention and may not constitute prior art.

[0003] By automating the entire logistics process, including product production, shipping, transportation, unloading, packaging, storage, and settlement (calculation, accounting), it is possible to reduce the number of people required and improve accuracy. Logistics automation has the effect of reducing costs and preventing safety accidents by reducing the number of people required, and it also enables shorter logistics times and more systematic management.

[0004] In particular, when customers pay in-store, the clerk must scan the barcode of each product one by one with a barcode scanner to settle the price of the product, which takes a long time to settle the price of the product, resulting in unnecessary waiting time for customers and increased congestion in the store, and also increasing the operation and management costs of the store due to employing clerks to settle the price of the products.

[0005] Therefore, there is a need for technology that can quickly, accurately, and unattendedly settle the items that customers are paying for. Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention aims to provide a method for checking out merchandise in an unmanned store.

[0007] Another object of the present invention is to improve the accuracy of product recognition by using multiple learning models. [Means for solving the problem]

[0008] To achieve the above-mentioned object, a method for checking out merchandise at an unmanned store according to one embodiment of the present invention may include the steps of recognizing a first merchandise that has entered a checkout counter by a merchandise recognition unit, selecting at least one learning model for identifying the first merchandise based on the recognition result of the first merchandise, obtaining a first image of the first merchandise by a merchandise identification unit, and identifying the first merchandise based on the at least one learning model and the first image.

[0009] The step of recognizing the first product may include the steps of identifying the size and shape of the first product using a camera, measuring the weight of the first product using a weight sensor, and measuring the temperature of the first product using a thermal imaging camera.

[0010] The step of identifying the size and shape of the first product may include the steps of acquiring cross-sectional images of the first product using multiple cameras, and identifying the size and shape of the first product based on the cross-sectional images.

[0011] The step of selecting at least one learning model may include a step of calculating a first criterion corresponding to the recognition result of the first product, and a step of selecting at least one learning model from a plurality of learning models based on the first criterion.

[0012] The step of identifying the first product may include the steps of: calculating a second image corresponding to the recognition result using the at least one learning model; comparing the first image and the second image using image matching; and identifying the first product as the second product having the highest similarity based on the result of the image matching.

[0013] After the step of acquiring the cross-sectional image, the method includes a step of determining whether the cross-sectional image includes an identification code for the first product, and if it is determined that the identification code is included, a step of recognizing the identification code and determining the first product as a third product corresponding to the identification code, wherein the identification code may be included in the package (outer packaging surface) of the product and include identification information of the product.

[0014] After the step of comparing the first image and the second image using the image matching, if the result of the image matching shows that there is no product with a similarity level equal to or higher than the preset similarity level, the method may include a step of sending a notification to an administrator terminal that the first product needs to be confirmed.

[0015] The learning model may include models trained on one of a plurality of criteria for products classified based on product size, shape, weight, and temperature range, and the learning may include supervised learning to enable a particular product to be identified as the correct product when it passes through the product recognition unit.

[0016] In addition, a product checkout device for an unmanned store according to one embodiment of the present invention may include instructions for executing the steps of: recognizing a first product that has entered the checkout counter by a product recognition unit; selecting at least one learning model for identifying the first product based on the recognition result of the first product; obtaining a first image of the first product by a product identification unit; and identifying the first product based on the at least one learning model and the first image.

[0017] In addition, a system for product checkout in an unmanned store according to one embodiment of the present invention may include a product recognition unit that identifies the size and shape of the product using a camera, measures the weight of the product using a weight sensor, and measures the temperature of the product using a thermal imaging camera; a product identification unit that acquires images of the product; and a product checkout device in an unmanned store that receives the product recognition results from the product recognition unit, selects a learning model for product identification, receives the image of the product from the product identification unit, and identifies the product. [Effects of the Invention]

[0018] According to the present invention, a method for checking out merchandise in an unmanned store can be provided.

[0019] Furthermore, according to the present invention, it is possible to improve the accuracy of product recognition by using multiple learning models. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a block diagram showing entities for product checkout in an unmanned store according to an embodiment of the present invention. FIG. [Figure 2] 1 is an operational flowchart showing a product checkout method in an unmanned store according to an embodiment of the present invention. [Figure 3] 1 is an operational flowchart showing a product checkout method in an unmanned store according to an embodiment of the present invention. [Figure 4] 1 is an operational flowchart showing a product checkout method in an unmanned store according to an embodiment of the present invention. [Figure 5] 1 is an operational flowchart showing a product checkout method in an unmanned store according to an embodiment of the present invention. [Figure 6] 1 is an operational flowchart showing a product checkout method in an unmanned store according to an embodiment of the present invention. [Figure 7] 1 is an operational flowchart showing a product checkout method in an unmanned store according to an embodiment of the present invention. [Figure 8] 1 is an operational flowchart showing a product checkout method in an unmanned store according to an embodiment of the present invention. [Figure 9]FIG. 1 illustrates a computer system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] The present invention will now be described in detail with reference to the accompanying drawings. Hereinafter, repeated explanations and detailed descriptions of known functions and configurations that may unnecessarily obscure the gist of the present invention will be omitted. The embodiments of the present specification are provided to more completely explain the present invention to those having average knowledge in the art. Therefore, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

[0022] Furthermore, terms such as "first" and "second" are used to describe various components, but these components are not limited by the terms. The terms may be used merely to distinguish one component from another. Therefore, a first component referred to below may also be a second component within the scope of the technical concept of the present invention.

[0023] Throughout this specification, when a part is described as "comprising" a certain element, this does not mean that it excludes other elements, and that it may further include other elements, unless otherwise specified.

[0024] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] FIG. 1 is a block diagram showing entities for product checkout in an unmanned store according to an embodiment of the present invention.

[0026] Referring to FIG. 1, an entity for checkout in an unmanned store according to an embodiment of the present invention includes an unmanned store checkout device 110, a product recognition unit 120, and a product identification unit .

[0027] The product checkout device 110 in an unmanned store may refer to a device that receives product recognition results from the product recognition unit 120 and selects a learning model for product identification.

[0028] The product checkout device 110 in an unmanned store may be a device that receives product images acquired from the product identification unit 130 and identifies the products.

[0029] The product recognition unit 120 may be a device that identifies the size and shape of the product using a camera, measures the weight of the product using a weight sensor, measures the temperature of the product using a thermal imaging camera, and provides the product recognition results to the product checkout device 110 in an unmanned store.

[0030] The commodity identification unit 130 may be a device that acquires an image of a commodity and provides it to the commodity checkout device 110 in an unmanned store.

[0031] The unmanned store's product checkout device 110 and product recognition unit 120, and the unmanned store's product checkout device 110 and product identification unit 130 may be connected to each other via a communication network.

[0032] The term "communication network" refers to a connection path for transmitting and receiving data between the above-mentioned entities. For example, the communication network can include wired networks such as local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), and integrated services digital networks (ISDNs), as well as wireless networks such as wireless LANs, code division multiple access (CDMA), Bluetooth, and satellite communications, but the scope of communication networks applicable to the present invention is not limited to these.

[0033] FIG. 2 is an operational flowchart showing a method for checking out merchandise in an unmanned store according to one embodiment of the present invention.

[0034] Referring to FIG. 2, in the unmanned store commodity checkout method according to an embodiment of the present invention, first, the commodity recognition unit can recognize the first commodity that has entered the checkout counter (S210).

[0035] Next, at least one learning model for identifying the first product can be selected based on the recognition result of the first product (S220).

[0036] Here, the learning model is a model learned when the product is first registered, and may include models learned using different learning models for multiple categories.

[0037] Next, a first image of the first product can be acquired by the product identification unit (S230).

[0038] The first product may then be identified based on the at least one learning model and the first image (S240).

[0039] FIG. 3 is an operational flowchart showing a method for checking out merchandise in an unmanned store according to one embodiment of the present invention.

[0040] Referring to FIG. 3, in the unmanned store checkout method according to an embodiment of the present invention, first, the size and shape of the first product may be identified using a camera (S310).

[0041] Next, the weight of the first product can be measured using a weight sensor (S320).

[0042] A thermal imaging camera may then be used to measure the temperature of the first item (S330).

[0043] At this time, by measuring the temperature of the product, it is possible to determine whether the first product is a refrigerated product or a frozen food.

[0044] Here, the learning model may include a model trained for each of a plurality of criteria for products classified based on product size, shape, weight, and temperature range.

[0045] The learning may include guidance-based learning, which enables a specific product to be identified as the correct product when it passes through the product recognition unit. Learning may be performed by inputting a preset number of products that meet each criterion, and when the product recognition rate exceeds a preset recognition rate, learning may be terminated and no further learning may be performed. In this way, products that were incorrectly recognized during product recognition can be learned to prevent future recognition errors.

[0046] FIG. 4 is an operational flowchart showing a method for checking out merchandise in an unmanned store according to one embodiment of the present invention.

[0047] Referring to FIG. 4, in the unmanned store commodity checkout method according to an embodiment of the present invention, first, a cross-sectional image of the first commodity may be acquired using a plurality of cameras (S410).

[0048] Next, the size and shape of the first product can be identified based on the cross-sectional image (S420).

[0049] According to one embodiment, multiple cameras installed at different directions and angles can be used to capture images of each cross section of the first product.

[0050] The product checkout device in the unmanned store can calculate the size of the first product by taking into account the angle difference between each camera and the distance between each camera and the first product.

[0051] FIG. 5 is an operational flowchart showing a method for checking out merchandise in an unmanned store according to one embodiment of the present invention.

[0052] Referring to FIG. 5, in the method for checking out merchandise in an unmanned store according to an embodiment of the present invention, first, a first criterion corresponding to the recognition result of the first merchandise may be calculated (S510).

[0053] Next, at least one learning model from the plurality of learning models may be selected based on the first criterion (S520).

[0054] Here, the first criteria may include criteria for at least one category, and as the learning model, a learning model corresponding to each of the categories may be selected.

[0055] FIG. 6 is an operational flowchart showing a method for checking out merchandise in an unmanned store according to one embodiment of the present invention.

[0056] Referring to FIG. 6, in an unmanned store product checkout method according to one embodiment of the present invention, first, a second image corresponding to the recognition result can be calculated using the at least one learning model (S610).

[0057] Image matching can then be used to compare the first image and the second image (S620).

[0058] Next, the first product may be identified as the second product that has the highest similarity as a result of the image matching (S630).

[0059] According to one embodiment, a second image corresponding to the recognition result can be calculated for each learning model. The unmanned store product checkout device can calculate the similarity for each learning model by comparing the first image with the second image for each learning model.

[0060] Next, the first product can be determined as the second product with the highest similarity as a result of image matching for each learning model.

[0061] Instead of selecting one learning model based on the recognition results, the product checkout device in an unmanned store can use multiple learning models corresponding to the categories based on the recognition results and compare the images calculated by each learning model to determine the product with the highest similarity. This has the effect of reducing product recognition errors and improving recognition accuracy.

[0062] FIG. 7 is an operational flowchart showing a method for checking out merchandise in an unmanned store according to one embodiment of the present invention.

[0063] Referring to FIG. 7, in a method for checking out merchandise at an unmanned store according to one embodiment of the present invention, after the step of acquiring the cross-sectional image, it can be determined whether the cross-sectional image includes an identification code for the first merchandise (S710).

[0064] Here, the identification code may be included in the product packaging (outer packaging surface) and may include product identification information. For example, the identification code may include a product barcode or a QR code included in the packaging.

[0065] Next, if it is determined that the identification code is included, the identification code can be recognized and the first product can be determined as the third product corresponding to the identification code (S720).

[0066] FIG. 8 is an operational flowchart showing a method for checking out merchandise in an unmanned store according to one embodiment of the present invention.

[0067] Referring to FIG. 8, in the unmanned store commodity checkout method according to an embodiment of the present invention, after the step of comparing the first image and the second image using the image matching, If, as a result of image matching, there is no product with a similarity level equal to or higher than the preset similarity level, a notification that the first product needs to be confirmed can be sent to the administrator terminal (S810).

[0068] FIG. 9 is a diagram illustrating a computer system according to one embodiment of the present invention.

[0069] The product checkout device for an unmanned store according to one embodiment of the present invention can be realized by a computer system 1000 such as a computer-readable recording medium.

[0070] 9, a computer system 1000 may include one or more processors 1010, memory 1030, user interface input devices 1040, user interface output devices 1050, and storage 1060, all of which communicate with each other via a bus 1020. The computer system 1000 may also include a network interface 1070 that connects to a network 1080. The processor 1010 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 1030 and / or the storage 1060. The memory 1030 and / or the storage 1060 may be various forms of volatile or non-volatile storage media. For example, the memory may include a read only memory (ROM) 1031 and / or a random access memory (RAM) 1032.

[0071] The specific implementation described in the present invention is one embodiment and does not limit the scope of the present invention in any way. For the sake of brevity, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the system may be omitted. Furthermore, linear connections or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and an actual device may be shown as alternative or additional various functional connections, physical connections, or circuit connections. Furthermore, unless specifically referred to as "essential" or "important," a component may not necessarily be required for application of the present invention. Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the scope of the claims described below, but also all scopes that are equivalent or modified to the scope of the claims belong to the scope of the spirit of the present invention. [Industrial Applicability]

[0072] According to the unmanned store product checkout method and device of the embodiment of the present invention, products can be identified using an artificial intelligence learning model for identifying products recognized by the product recognition unit and product images, and the accuracy of product recognition can be improved by using multiple learning models, making it industrially applicable.

Claims

1. a step of recognizing a first commodity that has entered the checkout counter by a commodity recognition unit; selecting at least one learning model for identifying the first commodity based on the recognition result of the first commodity; acquiring a first image of the first product by a product identification unit; identifying the first product based on the at least one learning model and the first image; A method for checking out merchandise at an unmanned store, including:

2. The step of recognizing the first product includes: identifying the size and shape of the first item using a camera; measuring a weight of the first product using a weight sensor; measuring the temperature of the first item using a thermal imaging camera; The unmanned store merchandise checkout method according to claim 1, further comprising:

3. The step of identifying the size and shape of the first product includes: acquiring cross-sectional images of the first product using a plurality of cameras; identifying a size and a shape of the first product based on the cross-sectional image; The unmanned store merchandise checkout method according to claim 2, further comprising:

4. The step of selecting at least one learning model includes: calculating a first criterion corresponding to the recognition result of the first product; selecting at least one learning model from a plurality of learning models based on the first criterion; The unmanned store merchandise checkout method according to claim 3, further comprising:

5. The step of identifying the first product includes: calculating a second image corresponding to the recognition result using the at least one training model; comparing the first image and the second image using image matching; identifying the first product as a second product having the highest similarity as a result of the image matching; The unmanned store merchandise checkout method according to claim 4, further comprising:

6. After the step of acquiring the cross-sectional image, determining whether the cross-sectional image includes an identification code for the first product; If it is determined that the identification code is included, recognizing the identification code and determining the first product as a third product corresponding to the identification code; Including, The identification code is The unmanned store merchandise checkout method according to claim 3, wherein the merchandise identification information is included on the merchandise package (outer packaging surface).

7. after the step of comparing the first image and the second image using image matching, If there is no product having a similarity equal to or higher than the preset similarity as a result of the image matching, a notification is sent to an administrator terminal indicating that confirmation of the first product is required; The unmanned store merchandise checkout method according to claim 5, further comprising:

8. The learning model is a model trained on one of a plurality of criteria for products classified based on product size, shape, weight, and temperature range; 3. The unmanned store merchandise checkout method according to claim 2, wherein the learning includes instruction-type learning that enables a specific merchandise to be identified as a correct merchandise when it passes through the merchandise recognition unit.

9. a memory having at least one program stored therein; a processor that executes the program; Including, The program a step of recognizing a first commodity that has entered the checkout counter by a commodity recognition unit; selecting at least one learning model for identifying the first commodity based on the recognition result of the first commodity; acquiring a first image of the first product based on the recognition result; and identifying the first product based on the at least one learning model.

10. a product recognition unit that identifies the size and shape of the product using a camera, measures the weight of the product using a weight sensor, and measures the temperature of the product using a thermal imaging camera; a product identification unit that acquires an image of the product; a product checkout device for an unmanned store that receives the product recognition results from the product recognition unit, selects a learning model for product identification, receives the product images from the product identification unit, and identifies the products; A system for checking out merchandise in an unmanned store, including:

Images