Unmanned store commodity settlement method and device

Abstract

The invention provides an unmanned store commodity settlement method and device. According to one embodiment of the present invention, the method comprises the steps of: discriminating, by a product discriminating unit, a first product placed in a cashier desk; selecting at least one learning model for identifying the first commodity based on the identification result of the first commodity; acquiring a first image of the first commodity through a commodity identification part; and identifying the first commodity based on the at least one learning model and the first image.

Description

Technical Field

[0001] The invention relates to a commodity settlement method and device for an unmanned store. Background Art

[0002] The following content merely provides background information related to an embodiment of the present invention and does not constitute prior art.

[0003] People are trying to reduce manpower and improve accuracy by realizing unmanned automation of the entire logistics process, including product production, shipment, transportation, loading and unloading, packaging, storage and settlement, etc. Logistics automation can reduce manpower, reduce costs, prevent safety accidents, shorten the time required for logistics, and achieve systematic management.

[0004] In particular, when checking out goods in stores, store staff use barcode scanning devices to scan the barcode of each product one by one to check out the goods, which increases the time required for checkout, thereby increasing the unnecessary waiting time of malicious customers, thereby increasing the congestion of the store. In addition, the cost of store operation and management also increases due to the employment of employees for checkout.

[0005] Therefore, there is a need for a technology that can quickly and accurately settle the goods that customers want to settle without human intervention. Summary of the invention

[0006] Technical problem to be solved by the invention

[0007] The object of the present invention is to provide a method for settling merchandise in an unmanned store.

[0008] Furthermore, the present invention aims to improve the accuracy of commodity identification through multiple learning models.

[0009] Technical solutions to solve problems

[0010] To achieve the above-mentioned purpose, the unmanned store commodity settlement method according to an embodiment of the present invention includes the following steps: identifying the first commodity placed in the checkout counter through a commodity identification unit; based on the identification result of the first commodity, selecting at least one learning model for identifying the first commodity; acquiring a first image of the first commodity through the commodity identification unit; and identifying the first commodity based on the at least one learning model and the first image.

[0011] The step of identifying the first commodity includes the following steps: using a camera to identify the size and shape of the first commodity; using a weight sensor to measure the weight of the first commodity; and using a thermal imaging camera to measure the temperature of the first commodity.

[0012] The step of identifying the size and shape of the first product includes the following steps: acquiring multiple cross-sectional images of the first product using multiple cameras; and identifying the size and shape of the first product based on the multiple cross-sectional images.

[0013] The step of selecting the at least one learning model includes the steps of: calculating a first criterion corresponding to the identification result of the first commodity; and selecting at least one learning model from among a plurality of learning models based on the first criterion.

[0014] The step of identifying the first product includes the following steps: using the at least one learning model to calculate a second image corresponding to the identification result; using image matching to compare the first image and the second image; and identifying the first product as the second product with the highest similarity based on the result of the image matching.

[0015] After the step of acquiring multiple cross-sectional images, the following steps are also included: determining whether the multiple cross-sectional images contain the identification code of the first product; and when it is determined that the identification code is contained, identifying the identification code and determining the first product as a third product corresponding to the identification code, wherein the identification code is contained in the outer packaging of the product and contains the identification information of the product.

[0016] After the step of comparing the first image and the second image by using image matching, the following step is also included: when the result of the image matching does not contain any product with a similarity greater than or equal to a preset degree, a notification indicating that the first product needs to be confirmed is sent to an administrator terminal.

[0017] The learning model includes a model learned based on one of multiple criteria for multiple commodities classified based on the range of commodity size, shape, weight and temperature, and the learning includes supervised learning for identifying a specific commodity as a correct commodity when it passes through a commodity identification unit.

[0018] Furthermore, the unmanned store commodity settlement device according to one embodiment of the present invention includes instructions for executing the following steps: identifying a first commodity placed in a checkout counter through a commodity identification unit; selecting at least one learning model for identifying the first commodity based on the identification result of the first commodity; acquiring a first image of the first commodity through the commodity identification unit; and identifying the first commodity based on the at least one learning model and the first image.

[0019] Furthermore, an unmanned store merchandise settlement system according to an embodiment of the invention includes: a merchandise identification unit, which uses a camera to identify the size and shape of the merchandise, uses a weight sensor to measure the weight of the merchandise, and uses a thermal imaging camera to measure the temperature of the merchandise; a merchandise recognition unit, which obtains an image of the merchandise; and an unmanned store merchandise settlement device, which receives the recognition result of the merchandise from the merchandise identification unit, selects a learning model for recognizing the merchandise, receives the image of the merchandise from the merchandise recognition unit, and recognizes the merchandise.

[0020] Effects of the Invention

[0021] According to the present invention, a method for settling merchandise in an unmanned store can be provided.

[0022] Furthermore, according to the present invention, the accuracy of commodity identification can be improved by using a plurality of learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A block diagram showing a commodity settlement entity in an unmanned store according to an embodiment of the present invention;

[0024] Figure 2 A flowchart showing a method for settling goods in an unmanned store according to an embodiment of the present invention;

[0025] Figure 3 A flowchart showing a method for settling goods in an unmanned store according to an embodiment of the present invention;

[0026] Figure 4 A flowchart showing a method for settling goods in an unmanned store according to an embodiment of the present invention;

[0027] Figure 5 A flowchart showing a method for settling goods in an unmanned store according to an embodiment of the present invention;

[0028] Figure 6 A flowchart showing a method for settling goods in an unmanned store according to an embodiment of the present invention;

[0029] Figure 7 A flowchart showing a method for settling goods in an unmanned store according to an embodiment of the present invention;

[0030] Figure 8 A flowchart showing a method for settling goods in an unmanned store according to an embodiment of the present invention;

[0031] Fig. 9 The figure shows a computer system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below with reference to the accompanying drawings. Here, the detailed description of the known functions and configurations that are repeated and may unnecessarily obscure the subject matter of the present invention are omitted. The embodiments of the present invention are provided to more fully illustrate the present invention to those skilled in the art. Therefore, the shapes and sizes of the elements in the accompanying drawings may be exaggerated for a clearer explanation.

[0033] Although the terms "first" or "second" are used to describe various components, these components are not limited by these terms. The above terms are only used to distinguish one component from another component. Therefore, the first component mentioned below may also be the second component within the technical concept of the present invention.

[0034] Throughout the specification, when a part is said to “comprise” a certain component, it does not mean that it excludes other components, but it may further include other components unless explicitly stated otherwise.

[0035] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0036] Figure 1 The block diagram shows a commodity settlement entity in an unmanned store according to an embodiment of the present invention.

[0037] refer to Figure 1 According to an embodiment of the present invention, a commodity settlement entity in an unmanned store includes an unmanned store commodity settlement device 110 , a commodity identification unit 120 and a commodity recognition unit 130 .

[0038] The unmanned store commodity settlement device 110 may refer to a device that receives the commodity identification result from the receiving commodity identification unit 120 and selects a learning model for identifying the commodity.

[0039] The unmanned store commodity settlement device 110 may refer to a device that receives an image of a commodity acquired from the commodity recognition unit 130 and recognizes the commodity.

[0040] The product identification unit 120 may refer to a device that uses a camera to identify the size and shape of the product, uses a weight sensor to measure the weight of the product, uses a thermal imaging camera to measure the temperature of the product, and provides the product identification result to the unmanned store product settlement device 110.

[0041] The commodity recognition unit 130 may refer to a device that acquires an image of a commodity and provides the image to the unmanned store commodity settlement device 110 .

[0042] The unmanned store commodity settlement device 110, the commodity identification unit 120, the unmanned store commodity settlement device 110 and the commodity recognition unit 130 may be interconnected via a communication network.

[0043] The communication network refers to a connection path that allows the above entities to send and receive data. For example, the communication network includes wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), ISDNs (Integrated Service Digital Networks), or wireless networks such as wireless LANs, CDMA, Bluetooth, satellite communications, etc., but the scope of the communication network applicable to the present invention is not limited thereto.

[0044] Figure 2 The flowchart is a method for settling goods in an unmanned store according to an embodiment of the present invention.

[0045] refer to Figure 2 According to an unmanned store commodity settlement method according to an embodiment of the present invention, the commodity identification unit may first identify the first commodity put into the checkout counter (S210).

[0046] Next, at least one learning model for identifying the first commodity may be selected based on the identification result of the first commodity ( S220 ).

[0047] The learning model is the model learned when the product is initially uploaded, including models trained using different learning models for multiple categories.

[0048] Next, a first image of the first commodity may be acquired by a commodity identification unit ( S230 ).

[0049] Next, the first commodity may be identified based on the at least one learning model and the first image ( S240 ).

[0050] Figure 3 The flowchart is a method for settling goods in an unmanned store according to an embodiment of the present invention.

[0051] refer to Figure 3 According to an embodiment of the present invention, the unmanned store commodity settlement method may first use a camera to identify the size and shape of the first commodity (S310).

[0052] Next, the weight of the first commodity may be measured using a weight sensor ( S320 ).

[0053] Next, the temperature of the first commodity may be measured using a thermal imaging camera ( S330 ).

[0054] At this time, it can be determined that the first commodity is a refrigerated commodity or a frozen food by measuring the temperature of the commodity.

[0055] The learning model includes a model learned based on one of a plurality of criteria for a plurality of commodities classified based on a range of commodity size, shape, weight, and temperature.

[0056] Furthermore, the learning includes supervised learning, which is used to identify a specific commodity as a correct commodity when it passes through the commodity identification unit. Using multiple commodities corresponding to each benchmark, a preset number of multiple commodities are input for learning, and if the commodity identification rate exceeds the preset identification rate, the learning is completed and no further learning is performed. In this way, commodities that are incorrectly identified during the commodity identification process can be learned, thereby preventing identification errors from occurring again in the future.

[0057] Figure 4 The flowchart is a method for settling goods in an unmanned store according to an embodiment of the present invention.

[0058] refer to Figure 4 According to an embodiment of the present invention, the unmanned store commodity settlement method may firstly use multiple cameras to obtain multiple cross-sectional images of the first commodity (S410).

[0059] Next, the size and shape of the first product may be recognized based on the plurality of cross-sectional images ( S420 ).

[0060] According to an embodiment, multiple cameras installed in different directions and angles may be used to acquire various cross-sectional images of the first product.

[0061] The unmanned store commodity settlement device can calculate the size of the first commodity by considering the angle difference between each camera and the distance between each camera and the first commodity.

[0062] Figure 5 The flowchart is a method for settling goods in an unmanned store according to an embodiment of the present invention.

[0063] refer to Figure 5 The unmanned store commodity settlement method according to an embodiment of the present invention may first calculate a first benchmark corresponding to the identification result of the first commodity (S510).

[0064] Next, at least one learning model among a plurality of learning models may be selected based on the first criterion ( S520 ).

[0065] The first benchmark includes a benchmark for at least one category, and the learning model can select a learning model corresponding to each category.

[0066] Figure 6 The flowchart is a method for settling goods in an unmanned store according to an embodiment of the present invention.

[0067] refer to Figure 6 According to an embodiment of the present invention, the unmanned store commodity settlement method may first use the at least one learning model to calculate a second image corresponding to the identification result (S610).

[0068] Next, the first image and the second image may be compared using image matching ( S620 ).

[0069] Next, the first product may be identified as the second product having the highest similarity according to the result of the image matching ( S630 ).

[0070] According to an embodiment, a second image corresponding to the identification result may be calculated for each learning model. The unmanned store commodity settlement device may respectively compare the first image with the second image of each learning model to calculate the similarity of each learning model.

[0071] Next, the first product may be determined as the second product having the highest similarity according to the image matching results of each learning model.

[0072] The unmanned store commodity settlement device does not select a single learning model based on the identification results, but uses multiple learning models corresponding to the category based on the identification results and compares them with the images calculated by each learning model to determine the commodity with the highest similarity. This has the effect of reducing commodity identification errors and improving identification accuracy.

[0073] Figure 7 The flowchart is a method for settling goods in an unmanned store according to an embodiment of the present invention.

[0074] refer to Figure 7 According to an embodiment of the present invention, the unmanned store commodity settlement method further comprises, after the step of acquiring the plurality of cross-sectional images, determining whether the plurality of cross-sectional images contain the identification code of the first commodity (S710).

[0075] The identification code is contained in the outer packaging of the product and includes identification information of the product. For example, the identification code includes a barcode of the product or a QR code contained in the outer packaging.

[0076] Next, when it is determined that the identification code is included, the identification code may be identified and the first commodity may be determined as a third commodity corresponding to the identification code ( S720 ).

[0077] Figure 8 The flowchart is a method for settling goods in an unmanned store according to an embodiment of the present invention.

[0078] refer to Figure 8According to an embodiment of the present invention, the unmanned store commodity settlement method may further include, after the step of comparing the first image and the second image by using image matching, sending a notification (S810) to the administrator terminal indicating that the first commodity needs to be confirmed when the image matching result does not contain any commodity with a similarity greater than or equal to a preset similarity.

[0079] Fig. 9 The figure shows a computer system according to an embodiment of the present invention.

[0080] The unmanned store merchandise settlement device according to an embodiment of the present invention may be implemented in a computer system 1000 , such as a computer-readable recording medium.

[0081] refer to Fig. 9 , the computer system 1000 includes one or more processors 1010, a memory 1030, a user interface input device 1040, a user interface output device 1050, and a storage device 1060 that communicate with each other through a bus 1020. In addition, the computer system 1000 also includes a network interface 1070 connected to a network 1080. The processor 1010 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 1030 or the storage device 1060. The memory 1030 and the storage device 1060 may be various forms of volatile or non-volatile storage media. For example, the memory includes a ROM 1031 or a RAM 1032.

[0082] The specific implementation described in the present invention is only an embodiment and does not limit the scope of the present invention in any way. For the sake of brevity of the specification, the description of conventional electronic components, control systems, software and other functional aspects of the system may be omitted. In addition, the line connections or connecting parts between the components shown in the figures are only schematically shown to be functional connections and / or physical or circuit connections, and can be represented as various functional connections, physical connections or circuit connections that can be replaced or added in actual equipment. In addition, if there is no special mention of "necessary" or "important", the corresponding component may not be an absolutely necessary component for the application of the present invention.

[0083] Therefore, the idea of ​​the present invention should not be limited to the above-mentioned embodiments, and should be included not only in the scope of the patent claims described below, but also all the scopes equivalent to or equivalently modified to the scope of the patent claims should be deemed to belong to the scope of the idea of ​​the present invention.

[0084] Industrial Applicability

[0085] According to multiple embodiments of the unmanned store commodity settlement method and device of the present invention, the artificial intelligence learning model used to identify the commodity identified by the commodity identification unit and the image of the commodity can be used to identify the commodity, and the accuracy of commodity identification can be improved through multiple learning models, so it has industrial applicability.

Claims

1. A method for settling goods in an unmanned store, characterized in that: The steps include: Identify the first commodity put into the cash register by the commodity identification unit; Based on the identification result of the first commodity, selecting at least one learning model for identifying the first commodity; Acquire a first image of the first commodity through a commodity identification unit; and The first product is identified based on the at least one learned model and the first image.

2. The unmanned store commodity settlement method according to claim 1, characterized in that: The step of identifying the first commodity comprises the following steps: Using a camera to identify the size and shape of the first product; measuring the weight of the first commodity using a weight sensor; and The temperature of the first commodity is measured using a thermal imaging camera.

3. The unmanned store commodity settlement method according to claim 2, characterized in that: The step of identifying the size and shape of the first commodity comprises the following steps: Using multiple cameras to acquire multiple cross-sectional images of the first product; and The size and shape of the first product are recognized based on the plurality of cross-sectional images.

4. The unmanned store commodity settlement method according to claim 3, characterized in that: The step of selecting the at least one learning model comprises the following steps: calculating a first reference corresponding to the identification result of the first commodity; and At least one learning model among a plurality of learning models is selected based on the first criterion.

5. The unmanned store commodity settlement method according to claim 4, characterized in that: The step of identifying the first commodity comprises the following steps: Calculating a second image corresponding to the recognition result using the at least one learning model; comparing the first image and the second image using image matching; and The first product is identified as the second product having the highest similarity according to the result of the image matching.

6. The unmanned store commodity settlement method according to claim 3, characterized in that: After the step of acquiring a plurality of cross-sectional images, the following steps are also included: determining whether the plurality of cross-sectional images include an identification code of the first product; and When it is determined that the identification code is included, the identification code is identified and the first commodity is determined to be a third commodity corresponding to the identification code. The identification code is contained in the outer packaging of the product and contains identification information of the product.

7. The unmanned store commodity settlement method according to claim 5, characterized in that: After the step of comparing the first image and the second image by using image matching, the following steps are also included: When there is no commodity with a similarity greater than or equal to a preset degree in the result of the image matching, a notification indicating that the first commodity needs to be confirmed is sent to the administrator terminal.

8. The unmanned store commodity settlement method according to claim 2, characterized in that: The learning model includes a model learned using one of a plurality of criteria for each of a plurality of commodities classified based on a range of commodity size, shape, weight, and temperature. The learning includes supervised learning so that a specific commodity can be identified as a correct commodity when it passes through the commodity identification unit.

9. A commodity settlement device for an unmanned store, characterized in that: include: A memory storing at least one program; and a processor for running the program, The program includes instructions for performing the following steps: Identify the first commodity put into the cash register by the commodity identification unit; Based on the identification result of the first commodity, selecting at least one learning model for identifying the first commodity; acquiring a first image of the first commodity based on the identification result; and Based on the at least one learned model, the first product is identified.

10. An unmanned store commodity settlement system, characterized in that: include: The product identification unit uses cameras to identify the size and shape of products, weight sensors to measure the weight of products, and thermal imaging cameras to measure the temperature of products; A commodity recognition unit, which obtains an image of the commodity; and The unmanned store commodity settlement device receives the commodity identification result from the commodity identification unit, selects a learning model for identifying the commodity, receives the commodity image from the commodity identification unit, and identifies the commodity.

Images