A method and system for detecting commodities

Abstract

The present invention discloses a method and system for detecting commodities, the method comprising the following steps: photographing a target commodity on a shelf and a price tag adjacent to the target commodity to obtain a target image; detecting a price tag area and a commodity area from the target image, identifying price tag information from the price tag area, and identifying commodity information from the commodity area; and determining the commodity name and commodity inventory unit (SKU) of the target commodity based on the price tag information and the commodity information. The present invention can improve the recognition accuracy of commodities by jointly identifying commodities through price tags and commodity features, and effectively reduce the misrecognition rate of similar commodities in commodity recognition.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for detecting commodities. Background Art

[0002] The identification of shelf merchandise has always been a matter of great concern in supermarkets. Through the identification of shelf merchandise, a wide range of commercial values ​​can be derived. In the prior art, the product feature detection through deep learning can identify the products on the shelf on a large scale, but the retrieved products are sorted by similarity, and it is inevitable that there will be false retrieval of similar products, resulting in a high misidentification rate of products. Summary of the invention

[0003] The present invention provides a method and system for detecting commodities, so as to solve the problem of high misrecognition rate existing in the prior art.

[0004] The present invention provides a method for detecting commodities, comprising the following steps:

[0005] Photographing a target commodity on a shelf and a price tag adjacent to the target commodity to obtain a target image;

[0006] Detecting a price tag area and a commodity area from the target image, identifying price tag information from the price tag area, and identifying commodity information from the commodity area;

[0007] Based on the price tag information and the product information, a product name and a product stock keeping unit (SKU) of the target product are determined.

[0008] The present invention also provides a system for detecting commodities, comprising:

[0009] A shooting module, used to shoot a target product on a shelf and a price tag adjacent to the target product to obtain a target image;

[0010] A detection module, used to detect a price tag area and a product area from the target image;

[0011] A first recognition module, used to recognize price tag information from the price tag area;

[0012] A second identification module, used to identify commodity information from the commodity area;

[0013] A determination module is used to determine the product name and product inventory unit SKU of the target product based on the price tag information and the product information.

[0014] The embodiments of the present invention can improve the accuracy of commodity recognition by jointly identifying commodities through price tags and commodity features, and effectively reduce the misrecognition rate of similar commodities in commodity recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of a method for detecting commodities in an embodiment of the present invention;

[0016] Figure 2 This is a schematic diagram of the display of commodities and price tags on a shelf in an embodiment of the present invention;

[0017] Figure 3 This is a flowchart of a specific implementation method of detecting a commodity in an embodiment of the present invention;

[0018] Figure 4 The figure is a schematic diagram of the structure of a system for detecting commodities in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The embodiment of the present invention provides a method for detecting a commodity, such as Figure 1 As shown, the following steps are included:

[0021] Step 101 , photograph a target commodity on a shelf and a price tag adjacent to the target commodity to obtain a target image.

[0022] Each product on the shelf has a price tag adjacent to it, such as Figure 2 The target product is the product on the shelf that needs to be detected.

[0023] Step 102: Detect the price tag area and the product area from the target image, identify the price tag information from the price tag area, and identify the product information from the product area.

[0024] Among them, the price tag information is a first product feature corresponding to the price tag area, and the first product feature includes a first product name, a first product SKU (Stock Keeping Unit) and a first product image feature; the product information is a plurality of second product features corresponding to the product area, and each second product feature includes a second product name, a second product SKU and a second product image feature; accordingly, the coding information can be identified from the price tag area, and the first product feature corresponding to the coding information can be queried through the price tag system.

[0025] Step 103: Determine the product name and product SKU of the target product based on the price tag information and product information.

[0026] Specifically, the first product image feature can be compared with the second product image features respectively included in the multiple second product features through a deep learning network, and multiple similarity confidences can be output; the target product image feature with the highest similarity confidence with the first product image feature is determined from the multiple second product image features, and the product name and product SKU corresponding to the target product image feature are used as the product name and product SKU of the target product.

[0027] In this embodiment, after identifying the product information through deep learning network retrieval, the product features to be confirmed can be reconfirmed with the features of the neighboring price tag products. By comparing the two features, the confidence of the retrieval similarity can be enhanced. If the difference between the neighboring price tag and the product features to be confirmed exceeds the threshold, the top 3 product features are extracted and compared with the price tag product features. If similar features that meet the threshold are found, the retrieved product SKU and product name are replaced. If neither the traversed top 3 product features nor the price tag product features meet the threshold, the retrieved product information result is used by default.

[0028] The embodiments of the present invention can improve the accuracy of commodity recognition by jointly identifying commodities through price tags and commodity features, and effectively reduce the misrecognition rate of similar commodities in commodity recognition.

[0029] like Figure 3 FIG. 1 is a flowchart of a specific implementation method of detecting a commodity in an embodiment of the present invention, which includes the following steps:

[0030] Step 301: Perform a page cutting or lighting operation on the code on the price tag adjacent to the target product.

[0031] Specifically, by cutting pages or lighting up the price tag, the price tag can display a specific code.

[0032] Step 302 : photograph the target product and the price tag adjacent to the target product to obtain a target image.

[0033] Specifically, when a price tag displays a specific code, a target image is captured by a camera, and the target image includes a target commodity on the shelf and a price tag adjacent to the target commodity.

[0034] Step 303: Detect the price tag area and the product area from the target image, identify a first product feature from the price tag area, and identify multiple second product features from the product area.

[0035] The first product features include a first product name, a first product SKU and a first product image feature, and each second product feature includes a second product name, a second product SKU and a second product image feature.

[0036] Specifically, the position of the price tag on the shelf can be detected by an image recognition algorithm. Since the price tag is detected by a deep learning method, the page cutting / lighting code on the price tag is also identified. Then, the corresponding relationship between the price tag code and the product SKU is queried in the price tag system to obtain the first product feature, so that the wireless positioning of the price tag can be used to assist the prediction result of deep learning.

[0037] In this embodiment, the goods on the shelf can be detected through deep learning target detection / segmentation, and then the product SKU and product name corresponding to the detected product area can be queried through product retrieval.

[0038] Step 304: sort the plurality of second product features according to the confidence level to obtain product features to be confirmed and a preset number of second product features.

[0039] The product feature to be confirmed is the second product feature with the highest confidence among the multiple second product features, and the confidence of each second product feature in the preset number of second product features is greater than the first preset threshold.

[0040] Step 305: Through a deep learning network, the first product image feature is compared with the product image feature to be confirmed included in the product feature to be confirmed, and it is determined whether the difference between the first product image feature and the product image feature to be confirmed is greater than a second preset threshold; if the difference is not greater than the second preset threshold, the product name and product SKU included in the product feature to be confirmed are determined as the product name and product SKU of the target product; if the difference is greater than the second preset threshold, the first product image feature is compared with the second product image features included in each of the preset number of second product features, and if the difference between the target product image feature included in the target product feature in the preset number of second product features and the first product image feature is not greater than the second preset threshold, the product name and product SKU included in the target product feature are determined as the product name and product SKU of the target product; if the difference between the second product image feature included in each of the preset number of second product features and the first product image feature is greater than the second preset threshold, the product name and product SKU included in the multiple second product features are output.

[0041] The embodiment of the present invention can enhance the confidence of retrieval similarity, improve the recognition accuracy of commodities, and effectively reduce the misrecognition rate of similar commodities in commodity recognition by performing secondary confirmation on the features of the commodity to be confirmed and the features of the neighboring price tag commodities.

[0042] like Figure 4 FIG. 1 is a schematic diagram of a system structure for detecting commodities in an embodiment of the present invention, including:

[0043] The photographing module 410 is used to photograph the target product on the shelf and the price tag adjacent to the target product to obtain a target image.

[0044] The detection module 420 is used to detect the price tag area and the product area from the target image.

[0045] The first identification module 430 is configured to identify price tag information from the price tag area.

[0046] The price tag information is the first product feature corresponding to the price tag area, and the first product feature includes the first product name, the first product SKU and the first product image feature.

[0047] The first identification module 430 is specifically configured to identify the coding information from the price tag area, and query the first commodity feature corresponding to the coding information through the price tag system.

[0048] The second identification module 440 is used to identify commodity information from the commodity area.

[0049] The determination module 450 is used to determine the product name and product stock keeping unit SKU of the target product based on the price tag information and the product information.

[0050] The product information is a plurality of second product features corresponding to the product area, and each of the second product features includes a second product name, a second product SKU, and a second product image feature.

[0051] Correspondingly, the determination module 450 is specifically used to compare the first product image feature with the second product image features respectively included in the multiple second product features through a deep learning network, and output multiple similarity confidences; determine the target product image feature with the highest similarity confidence with the first product image feature from the multiple second product image features, and use the product name and product SKU corresponding to the target product image feature as the product name and product SKU of the target product.

[0052] In this embodiment, the second identification module 440 is specifically used to identify multiple second product features from the product area, and sort the multiple second product features according to confidence levels to obtain product features to be confirmed and a preset number of second product features. The product features to be confirmed are the second product features with the highest confidence levels among the multiple second product features, and the confidence levels of each of the preset number of second product features are greater than a first preset threshold.

[0053] Correspondingly, the determination module 450 is specifically used to compare the first product image feature with the product image feature to be confirmed included in the product feature to be confirmed through a deep learning network, and determine whether the difference between the first product image feature and the product image feature to be confirmed is greater than a second preset threshold; if the difference is not greater than the second preset threshold, the product name and product SKU included in the product feature to be confirmed are determined as the product name and product SKU of the target product; if the difference is greater than the second preset threshold, the first product image feature is respectively compared with the second product image features included in each of the preset number of second product features, if the difference between the target product image feature included in the target product feature in the preset number of second product features and the first product image feature is not greater than the second preset threshold, the product name and product SKU included in the target product feature are determined as the product name and product SKU of the target product; if the difference between the second product image feature included in each of the preset number of second product features and the first product image feature is greater than the second preset threshold, the product name and product SKU included in the multiple second product features are output.

[0054] In addition, the above system also includes:

[0055] The processing module is used to perform page cutting or lighting operations on the code on the price tag.

[0056] The embodiment of the present invention can enhance the confidence of retrieval similarity, improve the recognition accuracy of commodities, and effectively reduce the misrecognition rate of similar commodities in commodity recognition by performing secondary confirmation on the features of the commodity to be confirmed and the features of the neighboring price tag commodities.

[0057] The steps in the method described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0058] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for detecting a product, It is characterized in that The following steps are involved: Photographing a target commodity on a shelf and a price tag adjacent to the target commodity to obtain a target image; Detecting a price tag area and a commodity area from the target image, identifying price tag information from the price tag area, and identifying commodity information from the commodity area; Determine the product name and product stock keeping unit (SKU) of the target product based on the price tag information and the product information; The price tag information is a first commodity feature corresponding to the price tag area, wherein the first commodity feature includes a first commodity name, a first commodity SKU, and a first commodity image feature; The commodity information is a plurality of second commodity features corresponding to the commodity area, each of the second commodity features including a second commodity name, a second commodity SKU, and a second commodity image feature; The determining the product name and product SKU of the target product based on the price tag information and the product information specifically includes: By using a deep learning network, the first product image feature is respectively compared with the second product image features respectively included in the plurality of second product features, and a plurality of similarity confidences are output; A target product image feature having the highest confidence level of similarity to the first product image feature is determined from the plurality of second product image features, and the product name and product SKU corresponding to the target product image feature are used as the product name and product SKU of the target product.

2. The method according to claim 1, It is characterized in that The identifying the price tag information from the price tag area specifically includes: The coding information is identified from the price tag area, and the first commodity feature corresponding to the coding information is queried through the price tag system.

3. The method according to claim 2, It is characterized in that Before photographing the target commodity on the shelf and the price tag adjacent to the target commodity to obtain the target image, the method further includes: The code on the price label is cut into pages or illuminated.

4. The method according to claim 1, It is characterized in that The identifying the commodity information from the commodity area specifically includes: Identify a plurality of second product features from the product area, and sort the plurality of second product features according to confidence levels to obtain a product feature to be confirmed and a preset number of second product features, wherein the product feature to be confirmed is a second product feature with the highest confidence level among the plurality of second product features, and the confidence level of each second product feature among the preset number of second product features is greater than a first preset threshold; The determining the product name and product SKU of the target product based on the price tag information and the product information specifically includes: By using a deep learning network, the first product image feature is compared with the product image feature to be confirmed included in the product feature to be confirmed, and it is determined whether the difference between the first product image feature and the product image feature to be confirmed is greater than a second preset threshold; If the difference is not greater than the second preset threshold, determining the product name and product SKU included in the product feature to be confirmed as the product name and product SKU of the target product; If the degree of difference is greater than a second preset threshold, the first product image feature is compared with the second product image feature included in each of the preset number of second product features; if the degree of difference between the target product image feature included in the target product feature in the preset number of second product features and the first product image feature is not greater than the second preset threshold, the product name and product SKU included in the target product feature are determined as the product name and product SKU of the target product; if the degree of difference between the second product image feature included in each of the preset number of second product features and the first product image feature is greater than the second preset threshold, the product name and product SKU included in the multiple second product features are output.

5. A system for detecting goods, It is characterized in that include: A shooting module, used to shoot a target product on a shelf and a price tag adjacent to the target product to obtain a target image; A detection module, used to detect a price tag area and a product area from the target image; A first recognition module, used to recognize price tag information from the price tag area; A second identification module, used to identify commodity information from the commodity area; A determination module, configured to determine the product name and product inventory unit (SKU) of the target product based on the price tag information and the product information; The price tag information is a first commodity feature corresponding to the price tag area, wherein the first commodity feature includes a first commodity name, a first commodity SKU, and a first commodity image feature; The commodity information is a plurality of second commodity features corresponding to the commodity area, each of the second commodity features including a second commodity name, a second commodity SKU, and a second commodity image feature; The determination module is specifically used to compare the first product image feature with the second product image features respectively included in the multiple second product features through a deep learning network, and output multiple similarity confidences; determine the target product image feature with the highest similarity confidence with the first product image feature from the multiple second product image features, and use the product name and product SKU corresponding to the target product image feature as the product name and product SKU of the target product.

6. The system according to claim 5, It is characterized in that The first identification module is specifically used to identify the coding information from the price tag area, and query the first commodity feature corresponding to the coding information through the price tag system.

7. The system according to claim 6, It is characterized in that Also includes: The processing module is used to perform page cutting or lighting operations on the code on the price tag.

8. The system according to claim 5, It is characterized in that The second identification module is specifically configured to identify a plurality of second commodity features from the commodity area, and sort the plurality of second commodity features according to confidence levels to obtain a commodity feature to be confirmed and a preset number of second commodity features, wherein the commodity feature to be confirmed is a second commodity feature with the highest confidence level among the plurality of second commodity features, and the confidence level of each second commodity feature among the preset number of second commodity features is greater than a first preset threshold; The determination module is specifically used to compare the first product image feature with the product image feature to be confirmed included in the product feature to be confirmed through a deep learning network, and determine whether the difference between the first product image feature and the product image feature to be confirmed is greater than a second preset threshold; if the difference is not greater than the second preset threshold, the product name and product SKU included in the product feature to be confirmed are determined as the product name and product SKU of the target product; if the difference is greater than the second preset threshold, the first product image feature is respectively compared with the second product image features included in each of the preset number of second product features, if the difference between the target product image feature included in the target product feature in the preset number of second product features and the first product image feature is not greater than the second preset threshold, the product name and product SKU included in the target product feature are determined as the product name and product SKU of the target product; if the difference between the second product image feature included in each of the preset number of second product features and the first product image feature is greater than the second preset threshold, the product name and product SKU included in the multiple second product features are output.

Images