Item recognition device and method

Abstract

The present invention discloses a product recognition device and method, in which the device includes: a visual sensor configured to obtain product image information and transmit the product image information to a calculation processor; a calculation processor configured to recognize the product image information based on the image information of different products and category information actually corresponding to the image information of the different products based on an image recognition model trained as a machine learning model, obtain product category information, and transmit the product category information to a data transceiver; and a data transceiver configured to transmit the product category information to a background server of a seller, so that the background server of a seller performs payment based on the product category information and preset price information of products in different categories. By using the product recognition device, it is possible to quickly recognize various product categories, improve the efficiency and accuracy of the cash register, and improve the user experience.

Description

[Technical Field]

[0001] The present invention relates to the technical field of artificial intelligence, and in particular to an apparatus and method for recognizing merchandise. [Background technology]

[0002] This section is intended to provide a background or context for the claimed embodiments of the invention. Nothing herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] In the field of offline marketing, the main payment method is to scan product barcodes to recognize product categories and then perform product payment. Conventional general cash registers often only support payment for products with barcodes, but non-standard products without product barcodes, such as fruits, vegetables, and bulk products, may require specific additional devices to recognize and pay for the products, or manual determination of the product category before payment. This process is relatively cumbersome, increases the time costs for customers and store clerks when performing cash register checkout, and is prone to manual error. Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, no effective solutions have been proposed to address the above problems. [Means for solving the problem]

[0005] An embodiment of the present invention provides a product recognition device for quickly recognizing various product categories, improving checkout efficiency and accuracy, and enhancing user experience. The device includes: a visual sensor configured to acquire product image information and transmit the product image information to an arithmetic processor; a arithmetic processor configured to recognize the product image information based on an image recognition model trained as a machine learning model based on image information of different products and category information actually corresponding to the image information of the different products, acquire product category information, and transmit the product category information to a data transceiver; and a data transceiver configured to transmit the product category information to a merchant's background server, so that the merchant's background server performs payment based on the product category information and preset price information of products in different categories.

[0006] An embodiment of the present invention further provides a product recognition method for quickly recognizing various product categories, improving checkout efficiency and accuracy, and enhancing user experience. The method includes the steps of receiving product image information acquired by a visual sensor; inputting the product image information into an image recognition model trained as a machine learning model based on the image information of different products and category information actually corresponding to the image information of the different products, and outputting product category information; and transmitting the product category information to the data transceiver via the data transceiver to a merchant's background server, so that the merchant's background server can perform payment based on the product category information and preset price information of products in different categories.

[0007] An embodiment of the present invention further provides a computer device including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the above-described product recognition method is realized when the processor executes the computer program.

[0008] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described product recognition method.

[0009] An embodiment of the present invention further provides a computer program product including a computer program that, when executed by a processor, implements the above-described product recognition method. [Effects of the Invention]

[0010] In an embodiment of the present invention, the product recognition device mainly includes: a visual sensor configured to acquire product image information and transmit the product image information to an arithmetic processor; an arithmetic processor configured to recognize the product image information based on an image recognition model trained as a machine learning model based on image information of different products and category information actually corresponding to the image information of the different products, acquire product category information, and transmit the product category information to a data transceiver; and a data transceiver configured to transmit the product category information to a merchant's background server so that the merchant's background server performs payment based on the product category information and preset price information of products in different categories. By using the product recognition device, various product categories can be quickly recognized, checkout efficiency and accuracy can be improved, and the user experience can be improved. [Brief explanation of the drawings]

[0011] In order to more clearly describe the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can also obtain other drawings based on these drawings without inventive work. [Figure 1] 1 is a structural block diagram of a commodity recognition device according to an embodiment of the present invention. [Figure 2]FIG. 2 is a structural block diagram of a specific example of a commodity recognition device according to an embodiment of the present invention; [Figure 3] FIG. 2 is a structural block diagram of a specific example of a commodity recognition device according to an embodiment of the present invention; [Figure 4] 1 is a process flowchart of a product recognition method according to an embodiment of the present invention. [Figure 5] 1 is a flowchart of a method for training and testing an image recognition model according to an embodiment of the present invention. [Figure 6] 1 is a structural schematic diagram of a computer device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0012] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be described in more detail below with reference to the drawings. Hereinafter, the exemplary embodiments of the present invention and the description thereof are for the purpose of interpreting the present invention, but are not intended to limit the present invention.

[0013] The inventor has discovered that in the field of offline marketing, a major payment method is to scan product barcodes to recognize product categories and then process product payments. While conventional general cash registers often only support payments for products with barcodes, non-standard products without barcodes, such as fruits, vegetables, and bulk items, may require additional devices to recognize and process payments, or require manual determination of the product category before processing payments. This process is relatively cumbersome, increases the time costs for customers and store clerks, and is prone to errors due to manual determination. Therefore, the inventor has proposed a product recognition device to solve this problem.

[0014] 1 is a structural schematic diagram of a commodity recognition device according to an embodiment of the present invention. As shown in FIG. 1, the commodity recognition device according to the embodiment of the present invention may include: a visual sensor 101 for acquiring commodity image information and transmitting the commodity image information to an arithmetic processor; an arithmetic processor 102 for recognizing the commodity image information based on an image recognition model trained as a machine learning model based on image information of different commodities and category information actually corresponding to the image information of the different commodities, acquiring commodity category information, and transmitting the commodity category information to a data transceiver; and a data transceiver 103 for transmitting the commodity category information to a merchant's background server so that the merchant's background server can perform payment based on the commodity category information and preset price information of commodities in different categories.

[0015] In an embodiment of the present invention, the product recognition device may be represented as an independent hardware device, which is plug-and-play and has internal functions realized by combining software and hardware, and can independently perform tasks such as capturing visual images, recognizing categories of the visual images, transmitting the category recognition results, and receiving feedback signals from the outside. It can also connect to an external payment system device (e.g., a merchant's background server) via a data interface and transmit the recognition results to the payment system device, thereby endowing the payment system device with external product recognition functions.

[0016] The commodity recognition device in the embodiment of the present invention may include a visual sensor, a calculation processor, and a data transceiver. Each part of the commodity recognition device will be specifically introduced below.

[0017] The visual sensor is configured to acquire image information of the products and transmit the image information of the products to the processor. In an embodiment, the image information of one product may be acquired individually, or the image information of multiple products may be acquired simultaneously.

[0018] The calculation processor is configured to recognize the image information of the products based on the image information of the different products and actual information corresponding to the image information of the different products based on the image recognition model trained as a machine learning model, obtain the product information, and transmit the product information to the data transceiver.

[0019] In one embodiment, the computing processor may include an AI chip, which may execute software to perform computations related to recognition, or may include an AI chip with an executable artificial neural network model.

[0020] In one embodiment, before recognizing the image information of the product based on the image recognition model, the calculation processor is further configured to perform object detection on the image information of the product by employing a single-stage detection algorithm or a two-stage detection algorithm.

[0021] In one embodiment, before recognizing the image information of the product based on the image recognition model, the calculation processor is further configured to perform image segmentation on the image information of the product by employing an instance segmentation algorithm or a semantic segmentation algorithm.

[0022] In specific implementation, before recognizing the image information of the product, multiple types of digital image processing algorithms can be adopted to first perform object detection and / or segmentation on the image information of the product, that is, pre-processing can be performed on the image information of the product, which is advantageous to improving the accuracy of product recognition by the image recognition model.

[0023] The data transceiver is configured to transmit information about the product to the merchant background server so that the merchant background server performs a settlement based on the information about the product.

[0024] In one embodiment, the data transceiver can include a USB cable, a Type C cable, or a wireless network communication device. In a specific implementation, the function of the data transceiver is to transmit the product recognition results and receive external feedback information.

[0025] 2 is a structural schematic diagram of a specific example of a commodity recognition device according to an embodiment of the present invention. As shown in FIG. 2, in an embodiment, the commodity recognition device shown in FIG. 1 may further include a storage medium 201 connected to the visual sensor and the calculation processor, respectively, and usable for storing commodity image information, commodity category information, and image recognition model layout information.

[0026] 3 is a structural schematic diagram of a specific example of a product recognition device according to an embodiment of the present invention. As shown in FIG. 3, in an embodiment, the product recognition device shown in FIG. 1 may further include an interactive touch screen 301 connected to a data transceiver and available for providing visible and selectable product category information.

[0027] In specific implementation, when the commodity recognition device simultaneously obtains multiple recognition results for the image information of the same commodity, i.e., when it is unable to uniquely recognize the commodity, it will first provide the seller or buyer with the multiple recognition results via the interactive touch screen for selection, and then the seller or buyer can select the correct commodity via the interactive touch screen, which is beneficial to ensuring the accuracy of commodity recognition, and furthermore, ensures the accuracy of the commodity payment amount, and avoids user losses.

[0028] For example, if the product to be paid for is American grapes, and the image information of the grapes and American grapes is sufficiently similar, the product recognition device simultaneously obtains two recognition results, grapes and American grapes. First, the two recognition results are displayed on the interactive touch screen, and the seller or buyer selects the correct product category (American grapes) by pointing. Then, the data transceiver transmits the correct product category (American grapes) to the seller's background server so that the seller's background server can make the payment based on the pre-stored price information of American grapes.

[0029] In one embodiment, the calculation processor is further configured to receive operation instructions issued from the seller's background server via the data transceiver, including operation instructions for re-recognizing a product category and operation instructions for simultaneously recognizing multiple product categories, and to perform corresponding operations based on the operation instructions.

[0030] In specific implementation, the image information of the product acquired by the salesperson using the visual sensor may be incomplete, or the product may not be recognized due to other operational errors. In this case, the salesperson can issue an operation command to re-recognize the product via the data transceiver, and the calculation processor can re-recognize the product based on the operation command.

[0031] Furthermore, when a customer purchases multiple types of products at the same time, in order to improve checkout efficiency, the salesperson can issue an operation command via the data transceiver to recognize multiple products at the same time, and the calculation processor can recognize the multiple products at the same time based on the operation command.

[0032] The beneficial effects of the present invention are as follows: 1) The product recognition device of the present invention can automatically recognize products by effectively utilizing visual image information and artificial intelligence algorithms, eliminating the need to rely on product barcodes, and improving the efficiency of product checkout, especially for non-standard products without barcodes. 2) By plugging and playing the product recognition device of the present invention, external product recognition functionality can be added to devices of general payment systems, thereby quickly and flexibly improving the performance of conventional cash registers and reducing their modification costs.

[0033] In the embodiments of the present invention, a commodity recognition method is further provided, which is described in the following embodiments. Since the principle of solving problems by the device is similar to that of the commodity recognition device, the implementation of the device can refer to the implementation of the commodity recognition device, and redundant explanations will be omitted.

[0034] FIG. 4 is a processing flowchart of a commodity recognition method according to an embodiment of the present invention, which is applied to the arithmetic processor in the commodity recognition device. As shown in FIG. 4, the commodity recognition method according to an embodiment of the present invention includes: Step 401: receiving image information of a product acquired by a visual sensor; Step 402: inputting product image information into an image recognition model trained as a machine learning model based on image information of different products and category information actually corresponding to the image information of the different products, and outputting product category information; The method may include step 403 of transmitting the product category information to the data transceiver, so that the data transceiver transmits the product category information to the seller's background server, and the seller's background server performs payment based on the product category information and price information of products in different categories that have been preset.

[0035] In one embodiment, the computing processor includes an AI chip.

[0036] In one embodiment, the method further includes, based on the image recognition model, employing a single-stage detection algorithm or a two-stage detection algorithm to perform object detection on the image information of the product before recognizing the image information of the product.

[0037] In one embodiment, the method further includes employing an instance segmentation algorithm or a semantic segmentation algorithm to perform image segmentation on the image information of the product before recognizing the image information of the product based on the image recognition model.

[0038] In one embodiment, the data transceiver includes a USB cable, a Type C cable, or a wireless network communication device.

[0039] 5 is a flowchart of a method for training and testing an image recognition model in an embodiment of the present invention. As shown in FIG. 5, in one embodiment, training and testing the image recognition model includes step 501 of constructing a training set and a test set using image information of different products and category information actually corresponding to the image information of different products as sample data, step 502 of training using the training set to obtain the image recognition model as a machine learning model, and step 503 of testing the image recognition model using the test set.

[0040] In one embodiment, the method further includes receiving operation instructions issued from the seller's background server via the data transceiver, including operation instructions for re-recognizing the product category and operation instructions for simultaneously recognizing multiple product categories, and performing corresponding operations based on the operation instructions.

[0041] Based on the above inventive idea, as shown in FIG. 6 , the present invention further proposes a computer device 600 including a memory 610, a processor 620, and a computer program 630 stored in the memory 610 and executable by the processor 620, wherein the above-mentioned product recognition method is realized when the processor 620 executes the computer program 630.

[0042] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described product recognition method.

[0043] An embodiment of the present invention further provides a computer program product including a computer program that, when executed by a processor, implements the above-described product recognition method.

[0044]

[0013] In accordance with the above, in an embodiment of the present invention, the product recognition device mainly includes: a visual sensor for acquiring product image information and transmitting the product image information to a calculation processor; an calculation processor for recognizing the product image information based on image information of different products and category information actually corresponding to the image information of the different products based on an image recognition model trained as a machine learning model, acquiring product category information, and transmitting the product category information to a data transceiver; and a data transceiver for transmitting the product category information to the seller's background server so that the seller's background server performs payment based on the product category information and preset price information of products in different categories. By using the product recognition device, various product categories can be quickly recognized, checkout efficiency and accuracy can be improved, and the user experience can be enhanced.

[0045] Those skilled in the art should understand that embodiments of the present invention may be provided as a method, a system, or a computer program product. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. The present invention may also take the form of a computer program product embodied in one or more computer-usable storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0046] The present invention has been described with reference to flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block of the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate an apparatus for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams, such that the instructions, executed by a processor of the computer or other programmable data processing device, generate an apparatus.

[0047] These computer program instructions may be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, thereby producing an article of manufacture including an instruction apparatus that implements the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams with the instructions stored in the computer-readable memory.

[0048] These computer program instructions can be loaded into a computer or other programmable data processing device, whereby the computer or other programmable device generates a computer-implemented process by performing a series of operational steps, whereby the instructions executed by the computer or other programmable device provide steps for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0049] The specific embodiments described above have further explained the objectives, technical solutions and beneficial effects of the present invention, but it should be understood that the above are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Any amendments, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention should all be included within the scope of the present invention.

Claims

1. a visual sensor configured to acquire image information of the product and transmit the image information of the product to a computing processor; an arithmetic processor configured to recognize the image information of the products based on an image recognition model trained as a machine learning model based on image information of different products and category information actually corresponding to the image information of the different products, obtain product category information, and transmit the product category information to a data transceiver; a data transceiver configured to transmit product category information to the seller's external background server so that the seller's external background server performs payment based on the product category information and preset price information of products in different categories; When multiple recognition results are simultaneously obtained for image information of the same product, the multiple recognition results are provided to the seller or the buyer via an interactive touch screen for selection; The interactive touch screen is connected to a data transceiver and configured to provide viewable and selectable product category information. A product recognition device characterized by:

2. The arithmetic processor includes an AI chip. The product recognition device according to claim 1 .

3. Before recognizing the image information of the product based on the image recognition model, the calculation processor further The object detection system is configured to perform object detection on the image information of the product by employing a single-stage detection algorithm or a two-stage detection algorithm. The product recognition device according to claim 1 .

4. Before recognizing the image information of the product based on the image recognition model, the calculation processor further The image segmentation algorithm is configured to perform image segmentation on the image information of the product by employing an instance segmentation algorithm or a semantic segmentation algorithm. The product recognition device according to claim 1 .

5. The data transceiver includes a USB cable, a Type C cable, or a wireless network communication device. The product recognition device according to claim 1 .

6. The system further includes a storage medium connected to the visual sensor and the arithmetic processor, respectively, and configured to store image information of the product, category information of the product, and placement information of the image recognition model. The product recognition device according to claim 1 .

7. The arithmetic processor further The device is configured to receive operation instructions issued from the seller's external background server via a data transceiver, the operation instructions including an operation instruction for re-recognizing a product category and an operation instruction for simultaneously recognizing multiple product categories, and to perform corresponding operations based on the operation instructions. The product recognition device according to claim 1 .

8. A product recognition method applied to a computing processor, comprising: receiving image information of a commodity acquired by a visual sensor; inputting product image information into an image recognition model trained as a machine learning model based on image information of different products and category information actually corresponding to the image information of different products, and outputting product category information; the data transceiver transmits the product category information to the seller's external background server, and the seller's external background server transmits the product category information to the data transceiver so that payment is made based on the product category information and preset price information of products in different categories; When multiple recognition results are simultaneously obtained for image information of the same product, the multiple recognition results are provided to the seller or the buyer via an interactive touch screen for selection; The interactive touch screen is connected to a data transceiver and configured to provide viewable and selectable product category information. A product recognition method characterized by:

9. The arithmetic processor includes an AI chip. The merchandise recognition method according to claim 8 .

10. Based on the image recognition model, before recognizing the image information of the product, It involves employing a single-stage detection algorithm or a two-stage detection algorithm to perform object detection on the image information of the product. The merchandise recognition method according to claim 8 .

11. Based on the image recognition model, before recognizing the image information of the product, This involves employing an instance segmentation algorithm or a semantic segmentation algorithm to perform image segmentation on the image information of the product. The merchandise recognition method according to claim 8 .

12. The data transceiver includes a USB cable, a Type C cable, or a wireless network communication device. The merchandise recognition method according to claim 8 .

13. Training and testing the image recognition model includes: A step of constructing a training set and a test set using image information of different products and category information actually corresponding to the image information of different products as sample data; obtaining the image recognition model as a machine learning model by training using a training set; testing the image recognition model using a test set. The merchandise recognition method according to claim 8 .

14. Further, the method includes receiving operation instructions issued from the seller's external background server via the data transceiver, the operation instructions including an operation instruction for re-recognizing the product category and an operation instruction for simultaneously recognizing the multiple product categories, and performing corresponding operations based on the operation instructions. The merchandise recognition method according to claim 8 .

15. When executed by a processor, the method of claim 8 is realized. A computer program characterized by:

Images