Commodity price tag identification method and device, medium, processor and program product

Abstract

The embodiment of the invention provides a commodity price tag identification method and device, a medium, a processor and a program product, and belongs to the technical field of information processing. The commodity price tag identification method comprises the steps of obtaining a front image of a shelf, and extracting a commodity image, a commodity price tag image and position coordinates of the commodity image and the commodity price tag image in the front image through a pre-trained first identification model; extracting text content in the commodity price tag image, and extracting commodity information from the text content through a pre-trained second recognition model; and according to the extracted commodity information, the corresponding commodity image and the corresponding position coordinate, identifying whether the commodity price tag on the shelf has a problem and the category of the problem. The commodity price tag identification method can effectively identify whether the non-electronic price tags and the electronic price tags on the shelf have problems and the types of the problems.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a commodity price tag recognition method, device, medium, processor and program product. Background Art

[0002] Price tags are tools used to identify product information. Currently, commonly used price tags include paper price tags and electronic price tags, which are convenient for buyers to understand product information such as product name and price through price tags. Merchants need to ensure that the position of price tags and product information are consistent with the corresponding products, and the product prices must also be consistent with the prices that merchants want to sell. However, it is difficult to achieve real-time monitoring and management for products on shelves with non-electronic price tags, and products with electronic price tags also need to improve the efficiency and accuracy of shelf management. Summary of the invention

[0003] The purpose of the embodiment of the present invention is to provide a commodity price tag recognition method, which can effectively identify whether there are problems with non-electronic price tags and electronic price tags on the shelf and the types of problems.

[0004] In order to achieve the above object, an embodiment of the present invention provides a commodity price tag recognition method for recognizing commodity price tags on a shelf, comprising:

[0005] Acquire a front image of the shelf, and extract a product image, a product price tag image, and position coordinates of the product image and the product price tag image in the front image using a pre-trained first recognition model;

[0006] Extracting text content from the product price tag image, and extracting product information from the text content using a pre-trained second recognition model; and

[0007] Based on the extracted product information, the corresponding product images and the corresponding location coordinates, identify whether there are problems with the product price tags on the shelf and the type of problem.

[0008] Preferably, the commodity price tag recognition method further includes: classifying each extracted commodity price tag image into a normal image and an abnormal image by using a pre-trained third recognition model, and distinguishing the problem category corresponding to the abnormal image.

[0009] Furthermore, the problem categories corresponding to abnormal images include damage, stains, occlusion or reflection problems.

[0010] Optionally, based on the extracted product information, the corresponding product image and the corresponding location coordinates, identifying whether there is a problem with the product price tag on the shelf and the type of the problem, including one or more of the following:

[0011] If the product information and the corresponding product image are judged to be inconsistent, then the corresponding product price tag has an error in price tag information;

[0012] If the price tag is not detected within the specified range of the center point of the product image, there is a missing price tag problem;

[0013] If the center point of the product price tag image does not fall within the specified range of the corresponding product image, the corresponding product price tag has an incorrect placement problem; or

[0014] If there is no problem with the placement of the product price tag, and the center point of the corresponding product price tag image falls within the specified range of the corresponding product image, then there is no problem with the corresponding product price tag.

[0015] Optionally, the product information includes one or more of the following: product name, product price, grade, unit, specification, place of origin, and product image.

[0016] Furthermore, the product price tag identification method also includes: using one or more of the product information as search conditions, retrieving first product information in the product sales system, and comparing it with second product information obtained by identifying product price tags on the shelf to identify labeling errors in the product labels.

[0017] On the other hand, an embodiment of the present invention provides a commodity price tag recognition device, including: an image recognition module, a text recognition module and a question recognition module.

[0018] The image recognition module is configured to: obtain a front image of the shelf, and extract a product image, a product price tag image, and position coordinates of the product image and the product price tag image in the front image through a pre-trained first recognition model;

[0019] The text recognition module is configured to: extract text content from the product price tag image, and extract product information from the text content using a pre-trained second recognition model; and

[0020] The problem identification module is configured to: identify whether there is a problem with the price tag of the product on the shelf and the type of the problem based on the extracted product information, the corresponding product image and the corresponding position coordinates.

[0021] Preferably, the problem identification module is further configured to: classify each extracted product price tag image into a normal image and an abnormal image through a pre-trained third identification model, and distinguish the problem category corresponding to the abnormal image.

[0022] Preferably, the problem categories corresponding to the abnormal images include damage, occlusion or reflection problems.

[0023] Preferably, in the problem identification module, based on the extracted product information, the corresponding product image and the corresponding position coordinates, it is identified whether there is a problem with the product price tag on the shelf and the type of the problem, including one or more of the following:

[0024] If the product information and the corresponding product image are judged to be inconsistent, then the corresponding product price tag has an error in price tag information;

[0025] If the price tag is not detected within the specified range of the center point of the product image, there is a missing price tag problem;

[0026] If the center point of the product price tag image does not fall within the specified range of the corresponding product image, the corresponding product price tag has an incorrect placement problem; or

[0027] If there is no problem with the placement of the product price tag, and the center point of the corresponding product price tag image falls within the specified range of the corresponding product image, then there is no problem with the corresponding product price tag.

[0028] Preferably, the product information includes one or more of the following: product name, product price, grade, unit, specification, place of origin, and product image.

[0029] Preferably, the problem identification module is further configured to:

[0030] Using one or more of the product information as search conditions, first product information is retrieved from the product sales system and compared with second product information obtained by identifying the product price tag on the shelf to identify labeling errors in the product label.

[0031] On the other hand, an embodiment of the present invention provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute: the commodity price tag recognition method of the present application.

[0032] On the other hand, an embodiment of the present invention provides a processor for running a program, wherein the program, when being run, is used to execute: the commodity price tag recognition method of the present application.

[0033] On the other hand, an embodiment of the present invention provides a computer program product, including a computer program, which implements the commodity price tag recognition method of the present application when executed by a processor.

[0034] Through the above technical solution, when there are non-electronic price tags and / or electronic price tags on the shelf, the product image, the product price tag image, and the position coordinates of the product image and the product price tag image in the shelf front image are first extracted through a pre-trained first recognition model, so as to identify the product, the price tag and the positional relationship between the product and the price tag on the shelf; then the text content in the product price tag image is extracted, and the product information is extracted from the text content through a pre-trained second recognition model to prepare for identifying the product information marked on the price tag; then, based on the extracted product information, the corresponding product image and the corresponding position coordinates, it is effectively identified whether there are problems with the non-electronic price tags and electronic price tags on the shelf and the type of the problem.

[0035] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0037] Figure 1 is a flow chart of an embodiment of a method for identifying a commodity price tag of the present invention;

[0038] Figure 2-4 This is an example of a problematic product price tag extracted from the shelf front image;

[0039] Figure 5-6 is an example of a normal product price tag extracted from the shelf front image; and

[0040] Figure 7 It is a structural diagram of an embodiment of a commodity price tag identification device of the present invention. DETAILED DESCRIPTION

[0041] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0042] If there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0043] The acquisition, transmission, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. In addition, it should be noted that in the embodiments of this application, some existing solutions in the industry such as certain software, components, and models may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0044] Figure 1 The flowchart of an embodiment of the product price tag recognition method of the present application is schematically shown. The embodiment is intended to be applied to the real-time monitoring and management of electronic and / or non-electronic product price tags on the shelves of a chain of convenience stores. The embodiment runs as a program entity in a cloud information processing center. The product price tag recognition method of the present invention can also be used in shopping malls, supermarkets, vegetable markets or other stores, and the embodiment can be deployed in a centralized or distributed manner, which is not limited by the present invention. Figure 1 As shown, the multi-commodity joint pricing method includes the following steps:

[0045] Step S101, obtaining a front image of the shelf, and extracting a product image, a product price tag image, and position coordinates of the product image and the product price tag image in the front image through a pre-trained first recognition model;

[0046] Step S102, extracting text content from the product price tag image, and extracting product information from the text content using a pre-trained second recognition model;

[0047] Step S103, identifying whether there is a problem with the price tag of the product on the shelf and the type of the problem based on the extracted product information, the corresponding product image and the corresponding position coordinates.

[0048] In step S101, the front image of the shelf is automatically or manually captured by means of a shooting device, and the shooting device needs to have a high-resolution camera to obtain high-definition images of the shelf and the price tags on the goods. The camera should be able to work under different lighting conditions and can capture clear and complete images of the front of the shelf by adjusting the focal length and angle. The first recognition model is a pre-trained price tag recognition model, which uses an image data set consisting of real shelf images containing goods and price tags to mark the product information and product price tag information in the image, and is obtained after multiple rounds of training. The trained price tag recognition model is used to process the shelf image, and the images and position information of all goods and product price tags in the shelf image are intercepted. By setting the objective function in a targeted manner, the trained price tag recognition model can handle goods and product price tags of different sizes, shapes and arrangements, and is robust to the direction, rotation and partial occlusion of the product price tags.

[0049] In some embodiments, the shelf image in step S101 is collected by an inspection robot, the image collected by the camera is connected to the edge computing device via a wireless network, and the shooting distortion of the camera itself is corrected to prepare for input into the first recognition model, i.e., the price tag recognition model for reasoning.

[0050] In some implementations, the shelf image in step S101 is manually photographed using a handheld terminal such as a mobile phone and uploaded to a cloud server for processing.

[0051] In some implementations, the shelf image in step S101 is captured and uploaded by a fixed camera on the shelf.

[0052] It should be noted that the subsequent processing steps of the shelf images from different sources can be the same or different. The pre-processed shelf images are input into the trained first recognition model to obtain the images of individual products and product price tags and their coordinate positions (x1, y1, x2, y2) in the shelf images, where x1, y1 and x2, y2 are the coordinates of the upper left corner and lower right corner of the recognition area, respectively.

[0053] In step S102, the text content in the product price tag image is extracted, which can be: using optical character recognition technology (OCR) to recognize all text information, including text, numbers, etc., on the product price tag image obtained in step S101, judging whether different text blocks belong to the same line according to the relative size, position and distance between the text blocks, splicing adjacent text blocks into a complete line of information, and converting the text on the product price tag image into electronic text data. This step can recognize text in different fonts, sizes and languages, thereby obtaining complete text information on different product price tag images.

[0054] The processing logic for the above text block splicing can be:

[0055] ① Sort the multiple text blocks obtained by text detection from small to large according to the vertical coordinates;

[0056] ② Determine whether the two adjacent text blocks are in the same row based on whether the difference in the horizontal and vertical coordinates is less than the dynamic threshold. For example, the vertical coordinate difference is the absolute value of the difference between the upper boundaries of the two adjacent text blocks, and the horizontal coordinate difference is the absolute value of the difference between the right boundary of the left text block and the left boundary of the right text block. The dynamic threshold is the height of the current text block multiplied by a fixed value;

[0057] ③ The text blocks that are judged to be in the same row are spliced ​​from small to large according to the horizontal coordinates to obtain a complete row of information.

[0058] In step S102, the second recognition model is a pre-trained content entity extraction model, which is trained with text data of commodity price tags of various styles and types. The price tag content is parsed according to the recognized text information, and entities such as commodity name, price, code, unit and place of origin are extracted from the text information. In some embodiments, the extracted commodity name entity can also be entity matched and verified with the commodity name information in the commodity database to obtain the complete commodity name corresponding to the price tag. The function of the content entity extraction model is to detect named entities from the text and classify them into predefined categories, which in this application are commodity information such as commodity name, price, unit and place of origin. The named entity recognition task can actually be abstracted as a sequence labeling task for text, where the labeling simultaneously reflects the start and end range of the entity and its classification.

[0059] In some implementations, the process of training the above-mentioned content entity extraction model includes the following steps:

[0060] ① Label the text: Before training the content entity extraction model, it is necessary to label the text information in the price tags of different types of goods, distinguish the pre-defined entity types such as product name, price, grade, unit, specification, origin, etc., and generate training data. There are three ways to label the training data text for the named entity recognition task: BIO, BMES, and BIOSE. In this embodiment, the BIO method is used to label the start and end positions and categories of the entities in the price tag text. "B" represents the beginning of an entity, "I" represents the inside of an entity, and "O" represents a non-entity. For example:

[0061] Jinye Hawthorn Bars 260G

[0062] B-PN I-PN I-PN I-PN I-PN I-PN

[0063] Origin: See product packaging

[0064] OOO B-LOC I-LOC I-LOC I-LOC I-LOC

[0065] Note: PN=Product Name, LOC=Location

[0066] ②Model training: After dividing the training data into training set and test set, input the deep learning model for training, use the training data to learn features useful for named entity recognition, and then use the learned features to perform named entity recognition in the text. The model is mainly divided into two parts: word embedding and multi-classification. The model performs word embedding representation on the input text and context information to complete text vectorization. After obtaining the vector of each unit of the input text, multi-classification is performed on it, and the probability of it belonging to each label is output, and then the label sequence with the highest probability is selected as the output. Finally, the model outputs a pre-defined specific category label for each unit of the input text sequence, that is, the entity type of each unit is obtained, thereby completing the entity extraction. The final result is, for example:

[0067] Xiangpiaopiao Red Bean Milk Tea 64G

[0068] B-PN I-PN I-PN I-PN I-PN I-PN I-PN I-PN

[0069] Identified as: ('Xiangpiaopiao Red Bean Milk Tea 64G', 'Product Name')

[0070] ③ Model evaluation and optimization: Evaluate the performance of the content entity extraction model, and adjust and optimize the model based on the evaluation results to improve the overall performance of the system and ensure the accuracy of price tag entity extraction.

[0071] In step S102, product information is extracted from the text content through the trained second recognition model (i.e., the above-mentioned content entity extraction model), aiming to ensure that different records in the data set pointing to the same real object can be correctly identified and associated. In the stage of identifying and extracting the content of the product price tag, the recognized product name entity may only be a partial product name due to various reasons such as incomplete printing caused by the space limitation of the price tag or partial obstruction of the price tag, blurred shooting part, etc., so entity matching is performed on the product name entity again, so that the recognized product name corresponds one-to-one with the standard system product name. This embodiment proposes to adopt a matching method based on vector similarity, which is mainly divided into the following steps:

[0072] ① Text vectorization: Use the word embedding model to embed the extracted "product name" entity and the product name text in the product database to obtain a text vector.

[0073] ②Similarity calculation: Calculate the cosine similarity between the vector to be matched and the vectors of each product name in the product database.

[0074] ③Filter results: Take the product name with the highest cosine similarity as the final matching result.

[0075] The above method is superior to the method of locating key positions based on specific text, such as "product name". When identifying the content of product price tags, this method first locates the position of the product name and then identifies the text content at that position. It requires special training of the text detection model, or can only detect product price tags that meet the preset templates. The detection accuracy will be affected by changes in the price tag content style.

[0076] In step S103, based on the extracted product information, the corresponding product image and the corresponding position coordinates, it is identified whether there is a problem with the product price tag on the shelf and the type of the problem, including one or more of the following:

[0077] If the product information and the corresponding product image are judged to be inconsistent, then the corresponding product price tag has an error in price tag information;

[0078] If the price tag is not detected within the specified range of the center point of the product image, there is a missing price tag problem;

[0079] If the center point of the product price tag image does not fall within the specified range of the corresponding product image, the corresponding product price tag has an incorrect placement problem; or

[0080] If there is no problem with the placement of the product price tag, and the center point of the corresponding product price tag image falls within the specified range of the corresponding product image, then there is no problem with the corresponding product price tag.

[0081] In some embodiments, each extracted product price tag image is further classified into a normal image and an abnormal image through a third recognition model, and the problem category corresponding to the abnormal image is distinguished. The third recognition model is a pre-trained price tag classification model, which classifies the product price tag image extracted in step S101 to distinguish normal product price tags from problematic product price tags. Problematic product price tags are, for example, product price tags with large-area damage, occlusion, stains, reflections, etc. that make the content unrecognizable. Figure 2-4 As shown, normal product price tags are as follows Figure 5-6 shown.

[0082] The steps for obtaining the training set used in training the price tag classification model include: cropping the marked product price tag images from the existing shelf image dataset and labeling them by category. For example, labeling "normal", "occluded", "damaged", "stained", and "reflective". Through multiple training and optimization, the price tag classification model can better capture the detailed features of the price tag images to improve the accuracy of classification and the generalization ability of the model.

[0083] The following are respectively for Figure 5 , Figure 6 A normal product price tag is shown, with the process of extracting text content and identifying product information from it detailed.

[0084] for Figure 5 For a normal electronic product price tag shown, extracting text content and identifying product information includes the following steps:

[0085] 1) Extract the text information in all text blocks in the electronic product price label image through OCR technology and perform data cleaning;

[0086] 2) judging whether the text blocks are in the same row based on their relative positions, and performing text splicing to obtain the spliced ​​information;

[0087] 3) Perform text analysis on the above text, input the content entity extraction model to extract entities such as product name and price, and match and verify them with the product names in the product database to obtain the system product name corresponding to the price tag.

[0088] Among them, the named entity recognition model is input, and the entity extraction results are shown in Table 1 below.

[0089] Table 1

[0090]

[0091] The extracted product name "Luyou Guoba (Black Sesame Spicy Flavor) 220G" is word-embedded with the product names in the product database, and the cosine similarity is calculated. The result with the highest similarity is selected to obtain the accurate system product name "Luyou Guoba (Black Sesame Spicy Flavor) 220G".

[0092] In step S103, the identification result of the product information obtained in Table 1 above is further compared with the product information in the product database to determine whether there is a compliance problem with the current product price tag. The compliance problem determination result is shown in Table 2 below.

[0093] Table 2

[0094]

[0095]

[0096] for Figure 6 For a normal paper product price tag shown, extracting text content and identifying product information includes the following steps:

[0097] 1) Extract the text information in all text blocks in the paper product price tag image through OCR technology and perform data cleaning;

[0098] 2) judging whether the text blocks are in the same line based on their relative positions, and concatenating the text blocks to obtain the concatenated text;

[0099] 3) Perform text analysis on the concatenated text, input the content entity extraction model to extract entities such as product name and price, and match and verify them with the product names in the product database to obtain the system product name corresponding to the price tag.

[0100] Among them, the entity extraction results are shown in Table 3 below.

[0101] Table 3

[0102] Entity Class Examples Product Name AJI Cheese Chocolate Flavour Soft Tart 118G price 14.9 coding 70345804 unit Biscuits / Cakes integral 4966

[0103] The extracted product name "AJI Cheese Chocolate Flavor Soft-Centered Tart 118G" is word-embedded with the product names in the product database, and the cosine similarity is calculated. The result with the highest similarity is selected to obtain the accurate system product name "AJI Cheese Chocolate Flavor Soft-Centered Tart 118G".

[0104] In step S103, the identification result of the commodity information obtained in Table 3 is further compared with the commodity information in the commodity database to determine whether there is any compliance problem with the current paper commodity price tag.

[0105] The commodity price tag recognition method of this embodiment uses a variety of tools to capture the front image of the commodity shelf, and automatically analyzes the commodity price tags in the image, taking into account the positional relationship between the commodity and the price tag and the content information of the commodity price tag, and identifies problems such as price errors, missing price tags, and damaged price tags. This improves the accuracy of price tag recognition, avoids the problem that traditional price tag management relies on manual maintenance and inspections, which is prone to errors or delays, improves the efficiency of retail enterprise management and the accuracy of inspections, and also provides retail enterprises with more business opportunities and competitive advantages. Compared with the prior art, the technical advantages of the commodity price tag recognition method of the present invention are:

[0106] (1) Automatically detect and identify price tags and products on shelf images without limiting the specific source of shelf images. The sampling method is flexible and has wide applicability to a variety of commercial scenarios such as large supermarkets and convenience stores.

[0107] (2) There are no requirements for the type and style of product price tags, which are applicable to both electronic price tags and paper price tags.

[0108] (3) On the basis of obtaining all the text information on the price tag, a text analysis and entity extraction model is introduced to minimize the impact of price tag information loss caused by problems such as picture shooting angle, clarity, distortion or missing.

[0109] (4) The positional relationship between the product and the product price tag is associated. Considering the corresponding relationship between the price tag position, the price tag content and the product, in addition to misplacement, it can effectively handle price tag error problems caused by problems such as blurring and stains.

[0110] An embodiment of the present invention provides a multi-product joint pricing device, the composition structure of which is as Figure 7 shown, including: an image recognition module, a text recognition module and a problem recognition module,

[0111] Among them, the image recognition module is configured to: obtain the front image of the shelf, and extract the product image, the product price tag image, and the position coordinates of the product image and the product price tag image in the front image through a pre-trained first recognition model;

[0112] The text recognition module is configured to: extract the text content in the product price tag image, and extract product information from the text content through a pre-trained second recognition model; and

[0113] The problem recognition module is configured to: identify whether there are problems with the product price tags on the shelf and the problem categories according to the extracted product information, the corresponding product image and the corresponding position coordinates.

[0114] In some embodiments, the problem recognition module is further configured to: classify each extracted product price tag image into a normal image and an abnormal image through a pre-trained third recognition model, and distinguish the problem categories corresponding to the abnormal images.

[0115] In some embodiments, the problem categories corresponding to the abnormal images include damage, occlusion or reflection problems.

[0116] In some embodiments, in the problem recognition module, according to the extracted product information, the corresponding product image and the corresponding position coordinates, identifying whether there are problems with the product price tags on the shelf and the problem categories includes one or more of the following:

[0117] If the product information and the corresponding product image are determined to be inconsistent, there is a problem with the price tag information of the corresponding product price tag;

[0118] If no price tag is detected within the specified range of the center point of the product image, there is a problem of price tag missing;

[0119] If the center point of the product price tag image does not fall within the specified range of the corresponding product image, the corresponding product price tag has an incorrect placement problem; or

[0120] If there is no problem with the placement of the product price tag, and the center point of the corresponding product price tag image falls within the specified range of the corresponding product image, then there is no problem with the corresponding product price tag.

[0121] In some embodiments, the product information includes one or more of the following: product name, product price, grade, unit, specification, origin, and product image.

[0122] In some implementations, the problem identification module is further configured to:

[0123] Using one or more of the product information as search conditions, first product information is retrieved from the product sales system and compared with second product information obtained by identifying the product price tag on the shelf to identify labeling errors in the product label.

[0124] An embodiment of the present invention provides a commodity price tag recognition device, which includes a processor and a memory. The above-mentioned image recognition module, text recognition module and question recognition module are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0125] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to identify whether there is a problem with the price tag of the product on the shelf and the type of the problem.

[0126] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0127] An embodiment of the present invention provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute: the commodity price tag recognition method of the present application.

[0128] An embodiment of the present invention provides a processor for running a program, wherein the program, when being run, is used to execute: the commodity price tag recognition method of the present application.

[0129] The embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored in the memory and can be run on the processor, and the steps of the commodity price tag recognition method of the present application are implemented when the processor executes the program. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0130] The present application also provides a computer program product, including a computer program. When executed on a data processing device, the computer program implements the commodity price tag recognition method of the present application when executed by the processor.

[0131] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0135] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0136] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0137] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0138] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0139] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for identifying commodity price tags, used to identify commodity price tags on shelves, characterized in that: include: Acquire a front image of the shelf, and extract a product image, a product price tag image, and position coordinates of the product image and the product price tag image in the front image through a pre-trained first recognition model; Extracting text content from the product price tag image, and extracting product information from the text content using a pre-trained second recognition model; and According to the extracted product information, the corresponding product image and the corresponding position coordinates, it is identified whether there is a problem with the product price tag on the shelf and the type of the problem.

2. The commodity price tag recognition method according to claim 1, characterized in that: Also includes: Each extracted product price tag image is classified into a normal image and an abnormal image through a pre-trained third recognition model, and the problem category corresponding to the abnormal image is distinguished.

3. The commodity price tag recognition method according to claim 2, characterized in that: The problem categories corresponding to the abnormal images include damage, stains, occlusion or reflection problems.

4. The commodity price tag recognition method according to claim 1, characterized in that: The step of identifying whether there is a problem with the price tag of the product on the shelf and the type of the problem based on the extracted product information, the corresponding product image and the corresponding position coordinates includes one or more of the following: If the product information and the corresponding product image are determined to be inconsistent, then the corresponding product price tag has an error in price tag information; If the price tag is not detected within the specified range of the center point of the product image, there is a price tag missing problem; If the center point of the product price tag image does not fall within the specified range of the corresponding product image, then the corresponding product price tag has a placement error; or If the product price tag does not have a placement error problem, and the center point of the corresponding product price tag image falls within the specified range of the corresponding product image, then there is no problem with the corresponding product price tag.

5. The commodity price tag recognition method according to claim 1, characterized in that: The product information includes one or more of the following: product name, product price, grade, unit, specification, place of origin, and product image.

6. The commodity price tag recognition method according to claim 5, characterized in that: Also includes: Using one or more of the product information as search conditions, first product information is retrieved from the product sales system and compared with second product information obtained by identifying product price tags on shelves to identify labeling errors in product labels.

7. A commodity price tag recognition device, characterized in that: include: Image recognition module, text recognition module and question recognition module, The image recognition module is configured to: obtain a front image of the shelf, and extract a product image, a product price tag image, and position coordinates of the product image and the product price tag image in the front image through a pre-trained first recognition model; The text recognition module is configured to: extract text content in the product price tag image, and extract product information from the text content using a pre-trained second recognition model; and The problem identification module is configured to: identify whether there is a problem with the price tag of the product on the shelf and the type of the problem based on the extracted product information, the corresponding product image and the corresponding position coordinates.

8. The commodity price tag recognition device according to claim 7, characterized in that: The problem identification module is further configured to: classify each extracted product price tag image into a normal image and an abnormal image through a pre-trained third identification model, and distinguish the problem category corresponding to the abnormal image.

9. The commodity price tag recognition device according to claim 8, characterized in that: The problem categories corresponding to the abnormal images include damage, stains, occlusion or reflection problems.

10. The commodity price tag recognition device according to claim 7, characterized in that: In the problem identification module, the identification of whether there is a problem with the price tag of the product on the shelf and the type of the problem based on the extracted product information, the corresponding product image and the corresponding position coordinates includes one or more of the following: If the product information and the corresponding product image are determined to be inconsistent, then the corresponding product price tag has an error in price tag information; If the price tag is not detected within the specified range of the center point of the product image, there is a price tag missing problem; If the center point of the product price tag image does not fall within the specified range of the corresponding product image, then the corresponding product price tag has a placement error; or If the product price tag does not have a placement error problem, and the center point of the corresponding product price tag image falls within the specified range of the corresponding product image, then there is no problem with the corresponding product price tag.

11. The commodity price tag recognition device according to claim 7, characterized in that: The product information includes one or more of the following: product name, product price, grade, unit, specification, place of origin, and product image.

12. The commodity price tag recognition device according to claim 11, characterized in that: The problem identification module is further configured to: Using one or more of the product information as search conditions, first product information is retrieved from the product sales system and compared with second product information obtained by identifying product price tags on shelves to identify labeling errors in product labels.

13. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for causing the machine to execute: a commodity price tag recognition method according to any one of claims 1-6.

14. A processor, characterized in that: Used to run a program, wherein the program, when run, is used to execute: a commodity price tag recognition method according to any one of claims 1-6.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for identifying a commodity price tag according to any one of claims 1 to 6 is implemented.

Images