Method and apparatus for product checkout in unmanned store

Abstract

Disclosed are a method and an apparatus for product checkout in an unmanned store. A method for item checkout in an unmanned store according to one embodiment of the present invention may comprise the steps of: recognizing a first product placed on a checkout counter by means of a product recognition unit; identifying a sticker comprising product information on the first product on the basis of a recognition result for the first product; generating a sticker recognition result by scanning the product information included on the sticker on the basis of the identified sticker; and identifying the first product on the basis of the sticker recognition result.

Description

Unmanned store product checkout method and device

[0001] The present invention relates to a method and device for calculating goods in an unmanned store.

[0002] The following description merely provides background information related to an embodiment according to the present invention and does not constitute prior art.

[0003] Attempts are being made to reduce manpower and increase accuracy by unmanned automating the entire logistics process, including product production, shipment, transportation, loading and unloading, packaging, storage, and accounting. Automating logistics offers the benefits of cost reduction through workforce cuts and the prevention of safety accidents, while also shortening logistics time and enabling systematic management.

[0004] In particular, when calculating products in a store, there is a problem in that store employees scan the barcode of each item using a barcode scanner, which increases the time required for calculation, leading to longer waiting times for troublesome customers and consequently increasing store congestion, as well as raising store operating costs due to the hiring of staff for calculation.

[0005] Accordingly, there is a need for technology that can perform calculations quickly and accurately for products customers wish to pay for unmanned.

[0006] The present invention aims to provide a method for calculating goods in an unmanned store.

[0007] In addition, the present invention aims to recognize a sticker attached to a product, compare it with an image recognition result, and identify the product based on the comparison result.

[0008] In addition, the present invention aims to improve the accuracy of product recognition through multiple learning models.

[0009] A method for calculating products in an unmanned store according to an embodiment of the present invention for achieving the above-mentioned purpose may include: a step of recognizing a first product introduced into a checkout counter through a product recognition unit; a step of identifying a sticker containing product information regarding the first product based on the recognition result of the first product; a step of generating a sticker recognition result by scanning the product information included in the sticker based on the identified sticker; and a step of identifying the first product based on the sticker recognition result.

[0010] The above sticker includes store information regarding a store selling the first product, product group information regarding the type of product corresponding to the first product, and price information regarding the price of the first product. The sticker may display the store information and product group information by combining multiple shapes and colors differently, and may display the price information by combining one number for each digit.

[0011] The step of generating the sticker recognition result may include the step of calculating a product candidate group for the first product based on a combination of the shape and color of the sticker, and the step of generating a sticker recognition result for the first product based on a combination of the product candidate group and the number.

[0012] The step of recognizing the first product may include the step of identifying the size and shape of the first product using a camera, the step of measuring the weight of the first product using a weight sensor, and the step of measuring the temperature of the first product using a thermal imaging camera.

[0013] The step of identifying the size and shape of the first product may include the step of acquiring cross-sectional images of the first product using a plurality of cameras and the step of identifying the size and shape of the first product based on the cross-sectional images.

[0014] After the step of recognizing the first product, the method includes the step of generating an image recognition result for the first product based on the recognition result for the first product, and the step of generating an image recognition result for the first product includes the step of selecting at least one learning model for generating an image recognition result for the first product based on the recognition result for the first product, the step of acquiring a first image of the first product through an image acquisition unit, and the step of generating an image recognition result for the first product based on the at least one learning model and the first image, wherein the learning model is a model that is learned for each of a plurality of criteria for products classified based on the range of size, shape, weight, and temperature of the product, and the learning may include supervised learning that enables identification as a correct product when a specific product passes through a product recognition unit.

[0015] The step of selecting at least one learning model may include the step of calculating a first criterion corresponding to a recognition result for the first product and the step of selecting at least one learning model among a plurality of learning models based on the first criterion.

[0016] Based on the at least one learning model and the first image, the step of generating an image recognition result for the first product may include the step of producing a second image corresponding to the recognition result using the at least one learning model, the step of comparing the first image and the second image using image matching, and the step of identifying the first product as the second product with the highest similarity according to the result of the image matching.

[0017] The step of identifying the first product may include a step of comparing the image recognition result and the sticker recognition result, and a step of identifying the first product using the sticker recognition result if the image recognition result and the sticker recognition result are identical as a result of the comparison.

[0018] Additionally, an unmanned store product checkout device according to one embodiment of the present invention includes a memory on which at least one program is recorded and a processor that executes said program, and said program may include the step of recognizing a first product that is introduced into a checkout counter through a product recognition unit; the step of identifying a sticker containing product information for said first product based on the recognition result for said first product; the step of generating a sticker recognition result by scanning said product information contained in said sticker based on the identified sticker; and the step of identifying said first product based on said sticker recognition result.

[0019] According to the present invention, a method for calculating goods in an unmanned store can be provided.

[0020] In addition, according to the present invention, a sticker attached to a product can be recognized and compared with an image recognition result to identify the product based on the comparison result.

[0021] In addition, according to the present invention, the accuracy of product recognition can be increased through a plurality of learning models.

[0022] FIG. 1 is a block diagram showing entities for calculating goods in an unmanned store according to one embodiment of the present invention.

[0023] FIG. 2 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0024] FIG. 3 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0025] FIG. 4 is an operation flowchart illustrating a method for calculating products in an unmanned store according to an embodiment of the present invention.

[0026] FIG. 5 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0027] FIG. 6 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0028] FIG. 7 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0029] FIG. 8 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0030] FIG. 9 is an operation flowchart illustrating a method for calculating products in an unmanned store according to an embodiment of the present invention.

[0031] FIG. 10 is an operation flowchart illustrating a method for calculating goods in an unmanned store according to one embodiment of the present invention.

[0032] FIG. 11 is a drawing showing a computer system according to one embodiment of the present invention.

[0033] The present invention will be described in detail below with reference to the accompanying drawings. Hereinafter, repetitive descriptions and detailed descriptions of known functions and configurations that may unnecessarily obscure the essence of the invention are omitted. Embodiments of the present invention are provided to more fully explain the invention to those with average knowledge in the art. Accordingly, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

[0034] Although terms such as "first" or "second" are used to describe various components, these components are not limited by such terms. Such terms may be used merely to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the technical scope of the present invention.

[0035] Throughout the specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0036] Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

[0037] FIG. 1 is a block diagram showing entities for calculating goods in an unmanned store according to one embodiment of the present invention.

[0038] Referring to FIG. 1, entities for calculating unmanned store products according to one embodiment of the present invention include an unmanned store product calculation device (110) and a product recognition unit (120).

[0039] The unmanned store product checkout device (110) may refer to a device that receives a product recognition result from a product recognition unit (120) and generates an image recognition result and a sticker recognition result for the product based on the recognition result.

[0040] The unmanned store product checkout device (110) may be a device that compares image recognition results and sticker recognition results and identifies products based on the comparison results.

[0041] The product recognition unit (120) may be a device that identifies the size and shape of a product using a camera, measures the weight of a product using a weight sensor, measures the temperature of a product using a thermal imaging camera, and provides the product recognition result to an unmanned store product checkout device (110).

[0042] The unmanned store product checkout device (110) and the product recognition unit (120) can be interconnected through a communication network.

[0043] A communication network refers to a connection path for enabling data to be transmitted and received between the aforementioned entities. For example, a communication network may include wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), and ISDNs (Integrated Service Digital Networks), or wireless networks such as wireless LANs, CDMA, Bluetooth, and satellite communication, but the scope of communication networks applicable to the present invention is not limited thereto.

[0044] FIG. 2 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0045] Referring to FIG. 2, a method for calculating products in an unmanned store according to one embodiment of the present invention can first recognize a first product that is inserted into the checkout counter through a product recognition unit (S210).

[0046] Next, based on the recognition result of the first product, a sticker containing product information for the first product can be identified (S220).

[0047] Here, the sticker may include store information regarding a store selling the first product, product group information regarding the type of product corresponding to the first product, and price information regarding the price of the first product.

[0048] In addition, the sticker may display the store information and product group information by combining multiple shapes and colors differently, and may display the price information by combining one number for each digit.

[0049] Next, based on the identified sticker, the product information included in the sticker can be scanned to generate a sticker recognition result (S230).

[0050] Here, the sticker recognition result may refer to the result of recognizing the product through a sticker attached to the first product.

[0051] Next, based on the sticker recognition result, the first product can be identified (S240).

[0052] FIG. 3 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0053] Referring to FIG. 3, a method for calculating products in an unmanned store according to one embodiment of the present invention can first calculate a product candidate group for the first product based on the combination of the shape and color of the sticker (S310).

[0054] Next, based on the combination of the product candidate group and the number, a sticker recognition result for the first product can be generated (S320).

[0055] FIG. 4 is an operation flowchart illustrating a method for calculating products in an unmanned store according to an embodiment of the present invention.

[0056] Referring to FIG. 4, a method for calculating products in an unmanned store according to one embodiment of the present invention can first identify the size and shape of the first product using a camera (S410).

[0057] Next, the weight of the first product can be measured using a weight sensor (S420).

[0058] Next, the temperature of the first product can be measured using a thermal imaging camera (S430).

[0059] At this time, the temperature of the product can be measured to determine whether the first product is a refrigerated product or a frozen food.

[0060] According to one embodiment, each cross-sectional image of the first product can be obtained using a plurality of cameras installed at different directions and angles.

[0061] The unmanned store product checkout device can calculate the size of the first product by considering the angle difference between each camera and the distance between each camera and the first product.

[0062] FIG. 5 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0063] Referring to FIG. 5, a method for calculating products in an unmanned store according to one embodiment of the present invention can first obtain cross-sectional images of the first product using a plurality of cameras (S510).

[0064] Next, based on the cross-sectional images above, the size and shape of the first product can be identified (S520).

[0065] According to one embodiment, each cross-sectional image of the first product can be obtained using a plurality of cameras installed at different directions and angles.

[0066] The unmanned store product checkout device can calculate the size of the first product by considering the angle difference between each camera and the distance between each camera and the first product.

[0067] FIG. 6 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0068] Referring to FIG. 6, a method for calculating products in an unmanned store according to one embodiment of the present invention can generate an image recognition result for the first product based on the recognition result for the first product after the step of recognizing the first product (S610).

[0069] Here, the image recognition result may refer to the result of recognizing a product using an external image, weight, temperature, etc. of the first product.

[0070] FIG. 7 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0071] Referring to FIG. 7, a method for calculating products in an unmanned store according to one embodiment of the present invention may first select at least one learning model for generating an image recognition result for the first product based on the recognition result for the first product (S710).

[0072] Here, the learning model is a model learned when the product is first registered, and may include a model learned using different learning models for multiple categories.

[0073] Next, a first image of the first product can be obtained through an image acquisition unit (S720).

[0074] Next, based on the at least one learning model and the first image, an image recognition result for the first product can be generated (S730).

[0075] Here, the learning model may include a model trained on one of a plurality of criteria for each of the products classified based on the range of size, shape, weight, and temperature of the products.

[0076] In addition, the above learning may include supervised learning that enables the identification of a specific product as a correct product when it passes through the product recognition unit. The learning is performed by inputting a preset number of products corresponding to each criterion, and if the product recognition rate exceeds the preset recognition rate, the learning is considered complete and no further learning is performed. Through this, it is possible to prevent recognition errors from occurring in the future by learning from products that were incorrectly recognized during product recognition.

[0077] FIG. 8 is an operation flowchart illustrating a method for calculating products in an unmanned store according to one embodiment of the present invention.

[0078] Referring to FIG. 8, a method for calculating products in an unmanned store according to one embodiment of the present invention can first calculate a first standard corresponding to the recognition result of the first product (S810).

[0079] Next, based on the first criterion above, at least one learning model among a plurality of learning models can be selected (S820).

[0080] Here, the first criterion may include a criterion for at least one category, and the learning model may select a learning model corresponding to each category.

[0081] FIG. 9 is an operation flowchart illustrating a method for calculating products in an unmanned store according to an embodiment of the present invention.

[0082] Referring to FIG. 9, a method for calculating goods in an unmanned store according to one embodiment of the present invention can first produce a second image corresponding to the recognition result using the at least one learning model (S910).

[0083] Next, the first image and the second image can be compared using image matching (S920).

[0084] Next, the first product can be identified as the second product with the highest similarity according to the result of the image matching (S930).

[0085] According to one embodiment, a second image corresponding to the recognition result can be generated for each learning model. An unmanned store product checkout device can calculate a similarity for each learning model by comparing the first image with the second image for each learning model.

[0086] Next, the first product can be determined as the second product with the highest similarity to the image matching results for each learning model.

[0087]

[0088] The unmanned store product checkout device utilizes multiple learning models corresponding to the categories based on the recognition result, rather than selecting a single learning model based on the result. By comparing the images generated by each learning model, it can determine the product with the highest similarity. This effectively reduces product misrecognition and improves recognition accuracy.

[0089] FIG. 10 is an operation flowchart illustrating a method for calculating goods in an unmanned store according to one embodiment of the present invention.

[0090] Referring to FIG. 10, a method for calculating goods in an unmanned store according to one embodiment of the present invention can first compare the image recognition result and the sticker recognition result (S1010).

[0091] Next, if the image recognition result and the sticker recognition result are the same as the comparison result according to the above comparison, the first product can be identified using the sticker recognition result (S1020).

[0092] As an optional embodiment, when identifying a sticker for the first product, a sticker recognition score may be calculated, and whether the sticker is recognized may be determined based on the sticker recognition score.

[0093] Specifically, a sticker recognition score for the first product can be calculated using an artificial intelligence learning model that has learned the shape of the sticker according to the angles of cross-sectional images.

[0094] In this case, the sticker recognition score can identify deformation of the sticker's shape caused by crumpling, folding, alteration, or damage and reflect it in the score.

[0095] Next, if the sticker recognition score is equal to or greater than the first threshold score, it can be determined that the sticker has been recognized.

[0096] In this case, if the sticker recognition score is lower than the first threshold score, it is determined that the sticker was not recognized, and the user and administrator may be requested to reissue or re-recognize the sticker.

[0097] Here, the unmanned store product checkout device can determine the main cause of the low sticker recognition score and provide guidance accordingly.

[0098] If the primary cause is tampering with or damage to the sticker, you can request the administrator to reissue the sticker. Additionally, if the primary cause is that it is crumpled or folded, you can request the user to re-recognize it.

[0099] Through this, the main cause of the sticker not being recognized can be determined to prevent users from obtaining illicit gains through tampering or damage, and to guide users to correctly recognize the sticker.

[0100] As an optional embodiment, if the sticker recognition score is higher than or equal to a second criterion score that is higher than a first criterion score, it may be determined as a best practice for sticker recognition.

[0101] Specifically, if the sticker recognition score is equal to or greater than the second threshold score, the unmanned store product checkout device may store the first video, which captures the video of the user recognizing the sticker, as a best practice.

[0102] At this time, the first product that recognizes the sticker and the first video can be matched and saved.

[0103] Next, if another user recognizes the sticker for the first product and the sticker recognition score is less than the first threshold score, guidance can be provided by requesting the other user to re-recognize the sticker and providing the first video as feedback.

[0104] At this time, the sticker portion in the first video can be highlighted in a different color to make it more visible to other users.

[0105] Through this, it is possible to provide feedback to other users by utilizing best practices for sticker recognition for each product.

[0106] As an optional embodiment, if an identification code is included in the cross-sectional images of the product, the product can be identified through this.

[0107] Specifically, after acquiring cross-sectional images, it can be determined whether the cross-sectional images contain an identification code for the first product.

[0108] Here, the identification code is included on the outer packaging of the product and may contain identification information about the product. For example, the identification code may include the product's barcode or a QR code included on the outer packaging.

[0109] Next, if it is determined that an identification code is included, the identification code can be recognized and the first product can be determined as the third product corresponding to the identification code.

[0110] FIG. 11 is a drawing showing a computer system according to one embodiment of the present invention.

[0111] An unmanned store product checkout device according to one embodiment of the present invention can be implemented in a computer system (1000), such as a computer-readable recording medium.

[0112] Referring to FIG. 11, a computer system (1000) may include one or more processors (1010), memory (1030), user interface input device (1040), user interface output device (1050), and storage (1060) that communicate with each other via a bus (1020). Additionally, the computer system (1000) may further include a network interface (1070) connected to a network (1080). The processor (1010) may be a central processing unit or a semiconductor device that executes processing instructions stored in memory (1030) or storage (1060). Memory (1030) and storage (1060) may be various forms of volatile or non-volatile storage media. For example, memory may include ROM (1031) or RAM (1032).

[0113] The specific embodiments described in this invention are examples and do not limit the scope of the invention in any way. For the sake of brevity of the specification, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted. Additionally, the connections of lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in actual devices. Furthermore, unless specifically stated as “essential,” “importantly,” etc., a component may not be strictly necessary for the application of the invention.

[0114] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all scopes equivalent to or equivalently modified from the claims set forth below, as well as the claims set forth below, shall be considered to fall within the scope of the concept of the present invention.

Claims

1. A step of recognizing a first product inserted into the checkout counter through a product recognition unit; A step of identifying a sticker containing product information for the first product based on a recognition result for the first product; A step of generating a sticker recognition result by scanning the product information contained in the sticker based on the identified sticker; and A step of identifying the first product based on the sticker recognition result above. An unmanned store product checkout method including 2. In Paragraph 1, The above sticker is, Store information regarding the store selling the above-mentioned first product; Product family information regarding the type of product corresponding to the first product above; and Price information regarding the price of the first product mentioned above Includes, The above sticker is, Multiple shapes and colors are combined differently to represent the store information and product group information, A method for calculating products in an unmanned store that displays the above price information by combining numbers one by one for each digit.

3. In Paragraph 2, The step of generating the above sticker recognition result is, A step of calculating a product candidate group for the first product based on the combination of the shape and color of the sticker; and A step of generating a sticker recognition result for the first product based on the combination of the above product candidate group and the above number. An unmanned store product checkout method including 4. In Paragraph 3, The step of recognizing the first product above is, A step of identifying the size and shape of the first product using a camera; A step of measuring the weight of the first product using a weight sensor; and Step of measuring the temperature of the first product using a thermal imaging camera An unmanned store product checkout method including 5. In Paragraph 4, The step of identifying the size and shape of the first product is, A step of acquiring cross-sectional images of the first product using a plurality of cameras; and A step of identifying the size and shape of the first product based on the cross-sectional images above. An unmanned store product checkout method including 6. In Paragraph 5, After the step of recognizing the first product mentioned above, A step of generating an image recognition result for the first product based on the recognition result for the first product. Includes, The step of generating an image recognition result for the first product above is, A step of selecting at least one learning model for generating an image recognition result for the first product based on the recognition result for the first product; A step of acquiring a first image of the first product through an image acquisition unit; and A step of generating an image recognition result for the first product based on the at least one learning model and the first image. Includes, The above learning model is, It is a model trained on one of multiple criteria for products classified based on the range of product size, shape, weight, and temperature, and The above learning is a method for calculating products in an unmanned store, comprising supervised learning that enables a specific product to be identified as the correct product when it passes through a product recognition unit.

7. In Paragraph 6, The step of selecting at least one learning model above is, A step of calculating a first standard corresponding to the recognition result of the first product; and A step of selecting at least one learning model among a plurality of learning models based on the above-mentioned first criterion. An unmanned store product checkout method including 8. In Paragraph 7, The step of generating an image recognition result for the first product based on the at least one learning model and the first image is: A step of producing a second image corresponding to the recognition result using at least one learning model; A step of comparing the first image and the second image using image matching; and A step of identifying the first product as the second product with the highest similarity based on the result of the image matching. An unmanned store product checkout method including 9. In Paragraph 8, The step of identifying the first product above is, A step of comparing the image recognition result and the sticker recognition result; and If the image recognition result and the sticker recognition result are identical as a result of the comparison according to the above comparison, the step of identifying the first product using the sticker recognition result An unmanned store product checkout method including 10. Memory on which at least one program is written; and A processor executing the above program Includes, The above program is, A step of recognizing a first product inserted into the checkout counter through a product recognition unit; A step of identifying a sticker containing product information for the first product based on a recognition result for the first product; A step of generating a sticker recognition result by scanning the product information contained in the sticker based on the identified sticker; and A step of identifying the first product based on the sticker recognition result above. An unmanned store product checkout device comprising commands for performing

Images