Article price tag recognition method, device, medium, processor and program product

Abstract

Embodiments of the present application provide a commodity price tag identification method, device, medium, processor and program product, belonging to the technical field of information processing. The commodity price tag identification method comprises: acquiring a front image of a shelf, and extracting a commodity image, a commodity price tag image, and position coordinates of the commodity image and the commodity price tag image in the front image through a pre-trained first identification model; extracting text content in the commodity price tag image, and extracting commodity information from the text content through a pre-trained second identification model; and identifying whether a commodity price tag on the shelf has a problem and a problem category according to the extracted commodity information, the corresponding commodity image and the corresponding position coordinates. The commodity price tag identification method can effectively identify whether a non-electronic price tag and an electronic price tag on the shelf have a problem and a problem category.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information processing, in particular to a commodity price tag identification method, device, medium, processor and program product. BACKGROUND

[0002] The price tag is a tool for identifying commodity information. Currently, commonly used price tags include paper price tags and electronic price tags. The price tags are convenient for buyers to understand commodity information such as commodity names and prices. However, it is difficult to realize real-time monitoring and management for commodities with non-electronic price tags on shelves. The efficiency and accuracy of shelf management also need to be improved for commodities with electronic price tags. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a commodity price tag identification method, which can effectively identify whether non-electronic price tags and electronic price tags on shelves have problems and the problem categories.

[0004] In order to achieve the above purpose, the embodiments of the present application provide a commodity price tag identification method for identifying commodity price tags on shelves, comprising:

[0005] obtaining a front image of the shelf, and extracting a commodity image, a commodity price tag image, and position coordinates of the commodity image and the commodity price tag image in the front image through a pre-trained first identification model;

[0006] extracting text content in the commodity price tag image, and extracting commodity information from the text content through a pre-trained second identification model; and

[0007] identifying whether the commodity price tag on the shelf has a problem and the problem category according to the extracted commodity information, the corresponding commodity image, and the corresponding position coordinates.

[0008] Preferably, the commodity price tag identification method further comprises: classifying each extracted commodity price tag image into a normal image and an abnormal image through a pre-trained third identification model, and distinguishing the problem category corresponding to the abnormal image.

[0009] Further, the problem category corresponding to the abnormal image includes damage, stains, occlusion, or reflection problems.

[0010] Optionally, identifying whether the commodity price tag on the shelf has a problem and the problem category according to the extracted commodity information, the corresponding commodity image, and the corresponding position coordinates comprises one or more of the following:

[0011] If the commodity information and the corresponding commodity image are determined to be inconsistent, the corresponding commodity price tag has a price tag information error problem.

[0012] If the price tag is not detected within the specified range of the center point of the product image, there is a problem of missing price tag;

[0013] If the center point of the product price tag image does not fall within the specified range of the corresponding product image, the corresponding product price tag has a problem of misplacement; or

[0014] If the product price tag does not have a misplacement problem, and the center point of the corresponding product price tag image falls within the specified range of the corresponding product image, the corresponding product price tag does not have a problem.

[0015] Optionally, the product information includes one or more of the following: product name, product price, grade, unit, specification, origin, product image.

[0016] Further, the product price tag recognition method further comprises: taking one or more of the product information as a retrieval condition, retrieving the first product information in the product sales system, and comparing the second product information obtained by recognizing the product price tag on the shelf to identify the labeling error problem of the product label.

[0017] On the other hand, the embodiment of the present application provides a product price tag recognition device, comprising: an image recognition module, a text recognition module and a problem recognition module,

[0018] The image recognition module is configured to: acquire a front image of a shelf, and extract a product image, a product price tag image, and position coordinates of the product image and the product price tag image in the front image through a pre-trained first recognition model.

[0019] The text recognition module is configured to: extract text content in the product price tag image, and extract product information from the text content through a pre-trained second recognition model; and

[0020] The problem recognition module is configured to: according to the extracted product information, the corresponding product image and the corresponding position coordinates, identify whether the product price tag on the shelf has a problem and a problem category.

[0021] Preferably, the problem recognition module is further configured to: classify each extracted product price tag image into a normal image and an abnormal image through a pre-trained third recognition model, and distinguish the problem category corresponding to the abnormal image.

[0022] Preferably, the problem category corresponding to the abnormal image includes damage, obstruction or reflection problem.

[0023] Preferably, the problem identification module identifies whether the product price tag on the shelf has a problem and a problem category according to the extracted product information, the corresponding product image and the corresponding position coordinates, including one or more of the following:

[0024] If the product information and the corresponding product image are determined to be inconsistent, the corresponding product price tag has a price tag information error problem;

[0025] If no price tag is detected within the specified range of the center point of the product image, the price tag has a price tag missing problem;

[0026] If the center point of the product price tag image does not fall within the specified range of the corresponding product image, the corresponding product price tag has a placement error problem; or

[0027] If the product price tag does not have a placement error problem, and the center point of the corresponding product price tag image falls within the specified range of the corresponding product image, the corresponding product price tag does not have a problem.

[0028] Preferably, the product information includes one or more of the following: product name, product price, grade, unit, specification, origin, product image.

[0029] Preferably, the problem identification module is further configured to:

[0030] Use one or more of the product information as a search condition to search for first product information in the product sales system, and compare the first product information with second product information obtained by identifying the product price tag on the shelf to identify a labeling error problem of the product label.

[0031] On the other hand, an embodiment of the present application provides a machine readable storage medium, and the machine readable storage medium stores instructions for causing a machine to execute: the product price tag identification method of the present application.

[0032] On the other hand, an embodiment of the present application provides a processor for running a program, wherein the program is used to execute: the product price tag identification method of the present application when the program is run.

[0033] On the other hand, an embodiment of the present application provides a computer program product, including a computer program, and the computer program implements the product price tag identification method of the present application when executed by a processor.

[0034] Through the technical solution, in the case that there is a non-electronic price tag and / or an electronic price tag on the shelf, the first trained recognition model is used to extract the product image, the product price tag image, and the position coordinates of the product image and the product price tag image in the front image of the shelf, so as to recognize the product, the price tag, and the position relationship between the product and the price tag on the shelf; then the text content in the product price tag image is extracted, and the second trained recognition model is used to extract the product information from the text content, so as to prepare for recognizing the product information marked on the price tag; and then according to the extracted product information, the corresponding product image, and the corresponding position coordinates, whether the non-electronic price tag and the electronic price tag on the shelf have problems and the problem categories are effectively recognized.

[0035] Other features and advantages of the embodiments of the present application will be described in detail in the following specific implementation part. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings are included to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used together with the following specific implementation to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the drawings:

[0037] Figure 1 is a flow chart of a product price tag recognition method according to an embodiment of the present application;

[0038] Figures 2-4 is an example of a problematic product price tag extracted from a front image of a shelf;

[0039] Figures 5-6 is an example of a normal product price tag extracted from a front image of a shelf; and

[0040] Figure 7 is a structural diagram of a product price tag recognition device according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] The specific implementation of the embodiments of the present application will be described in detail below in combination with the drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiments of the present application, and is not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0042] If the description of "first", "second" and the like is involved in the embodiments of the present application, the description of "first", "second" and the like is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, and is not within the protection scope required by the present application.

[0043] The acquisition, transmission, storage, use, processing and the like of data in the technical solutions of the present application comply with the relevant provisions of national laws and regulations. In addition, it should be pointed out that in the embodiments of the present application, some existing industry solutions of software, components, models and the like may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the implementation of the technical solutions of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0044] Figure 1 The flowchart of an embodiment of the commodity price tag recognition method of the present application is schematically shown, which is applied to the real-time monitoring and management of electronic and / or non-electronic commodity price tags on the shelves in a chain-operated convenience store, and the embodiment is run as a program entity in a cloud information processing center. The commodity price tag recognition method of the present application can also be used in shopping malls, supermarkets, markets or other sales places, and the embodiments can be centrally deployed or distributedly deployed, and the present application does not make any limitation. As shown in Figure 1 The multi-commodity joint pricing method includes the following steps:

[0045] Step S101, acquiring a front image of a shelf, and extracting a commodity image, a commodity price tag image, and position coordinates of the commodity image and the commodity price tag image in the front image through a pre-trained first recognition model;

[0046] Step S102, extracting text content in the commodity price tag image, and extracting commodity information from the text content through a pre-trained second recognition model;

[0047] Step S103, identifying whether there is a problem in the commodity price tag on the shelf and the problem category according to the extracted commodity information, the corresponding commodity image and the corresponding position coordinates.

[0048] In step S101, the front image of the shelf is automatically or manually captured by a shooting device, which needs to have a high-resolution camera to obtain high-definition images of the shelf and the price tags on the goods. The camera should be able to work under different lighting conditions and capture clear and complete front images of the shelf by adjusting the focal length and angle. The first recognition model is a pre-trained price tag recognition model, which is trained using a real image dataset composed of shelf images containing goods and price tags, and is labeled with goods information and price tag information in the images. The trained price tag recognition model can process goods and price tags of different sizes, shapes and arrangements, and is robust to the direction, rotation and partial occlusion of the price tags.

[0049] In some embodiments, the shelf image in step S101 is collected by a patrol robot, and the pictures captured by the camera are connected to the edge computing device through a wireless network, and the shooting distortion of the camera itself is corrected, and the first recognition model, i.e., the price tag recognition model, is prepared for inference.

[0050] In some embodiments, the shelf image in step S101 is manually captured by a mobile phone or other handheld terminal and uploaded to a cloud server for processing.

[0051] In some embodiments, the shelf image in step S101 is captured and uploaded by a fixed camera on the shelf.

[0052] It should be noted that the subsequent processing steps of the shelf images of different sources described above can be the same or different. The pre-processed shelf image is input into the trained first recognition model to obtain the image of a single good, the price tag of the good, and the coordinate positions (x1, y1, x2, y2) in the shelf image, respectively, where x1, y1 and x2, y2 are the coordinates of the top left and bottom right of the recognition area.

[0053] In step S102, the text content in the price tag image is extracted, which can be: using optical character recognition technology (OCR) to recognize all the text information of the price tag image obtained in step S101, including letters, numbers, etc. According to the relative size, position and distance between the text blocks, it is determined whether different text blocks belong to the same line, the adjacent text blocks are spliced into complete line information, and the text on the price tag image is converted into electronic text data. This step can recognize different fonts, sizes and languages of the text, so that the complete text information on different price tag images can be obtained.

[0054] The processing logic of the above text block splicing can be:

[0055] ①Sort the multiple text blocks obtained by text detection according to the vertical coordinates from small to large;

[0056] ②Judge whether it is the same row according to whether the horizontal and vertical coordinate difference values of the two adjacent text blocks are less than the dynamic threshold. For example, the vertical coordinate difference value takes the absolute value of the difference between the upper boundaries of the two text blocks, the horizontal coordinate difference value takes the absolute value of the difference between the right boundary of the left text block and the left boundary of the right text block, and the dynamic threshold is the height of the current text block multiplied by a fixed value;

[0057] ③Splice the text blocks judged to be the same row according to the horizontal coordinates from small to large to obtain complete row information.

[0058] In step S102, the second recognition model is a pre-trained content entity extraction model. The content entity extraction model is trained by various text data of different styles and types of commodity price tags. The content of the price tag is parsed according to the recognized text information, and the entities such as the product name, price, code, unit and origin in the extracted text information are extracted. In some embodiments, the extracted product name entity can also be matched and verified with the product name information in the product database to obtain the complete product name corresponding to the price tag. The function of the content entity extraction model is to detect the named entity from the text and classify it into predefined categories, which are product name, price, unit and origin of the product information in this application. The named entity recognition task can actually be abstracted as a sequence labeling task for text, where the labeling represents the start and end range of the entity and its classification.

[0059] In some embodiments, the process of training the above-mentioned content entity extraction model includes the following steps:

[0060] ①Text annotation: Before training the content entity extraction model, the text information in different types of commodity price tags needs to be annotated to distinguish the pre-defined entity types such as product name, price, grade, unit, specification, origin, etc., and generate training data. The annotation method of the text data for the named entity recognition task can be BIO, BMES and BIOSE, and in this embodiment, the BIO method is used to annotate the start and end positions and categories of the entities in the price tag text. "B" represents the beginning of an entity, "I" represents the inside of an entity, and "O" represents a non-entity. For example:

[0061] Some hawthorn bars 260G

[0062] B-PN I-PN I-PN I-PN I-PN I-PN

[0063] Origin: see product packaging

[0064] OOO B-LOC I-LOC I-LOC I-LOC I-LOC

[0065] Note: PN = Product Name, LOC = Location

[0066] ②Model training: After dividing the training data into training set and test set, input the deep learning model for training, use the training data to learn the features useful for named entity recognition, and then use the learned features to perform named entity recognition in the text. The model mainly consists of two parts: word embedding and multi-classification. The model performs word embedding representation on the input text and context information, and completes text vectorization. After obtaining the vector of each unit of the input text, it is classified and the probability of belonging to each label is output, and then the label sequence with the highest probability is selected as the output. Finally, the model outputs the pre-defined specific category label for each unit of the input text sequence, i.e. the entity category of each unit, thus completing entity extraction. The final result is, for example:

[0067] Some red bean milk tea 64G

[0068] B-PN I-PN I-PN I-PN I-PN I-PN I-PN I-PN

[0069] Recognized as: ('Some red bean milk tea 64G', 'product name')

[0070] ③Model evaluation and optimization: evaluate the performance of the content entity extraction model, and adjust and optimize the model according to the evaluation results to improve the overall performance of the system and ensure the accuracy of the price tag entity extraction.

[0071] In step S102, the second recognition model (i.e. the above content entity extraction model) is trained to extract product information from the text content, aiming to ensure that different records in the data set pointing to the same real object can be correctly identified and associated. In the product price tag content recognition and extraction stage, due to the incomplete printing caused by the space limitation of the price tag, or the partial occlusion of the price tag, the recognized product name entity may only be part of the product name, so the entity matching is performed again for the product name entity, so that the recognized product name corresponds to the standard system product name one by one. This embodiment proposes a matching method based on vector similarity, which mainly includes the following steps:

[0072] ①Text vectorization: using a word embedding model, the extracted "product name" entity and the product name text in the product database are subjected to word embedding to obtain a text vector.

[0073] ii. Similarity calculation: calculate the cosine similarity between the to-be-matched vector and the name vector of each commodity in the commodity database.

[0074] iii. Screening result: take the commodity name with the highest cosine similarity as the final matching result.

[0075] The above method is superior to the method of positioning the key position based on specific words such as "commodity name". When identifying the content of the commodity price tag, the method first locates the position of the commodity name, and then identifies the text content at the position. The text detection model needs to be specially trained, or only commodity price tags conforming to the preset template can be detected. The detection accuracy will be affected by the change of the price tag content style.

[0076] In step S103, according to the extracted commodity information, the corresponding commodity image and the corresponding position coordinates, whether the commodity price tag on the shelf has a problem and the problem category are identified, including one or more of the following:

[0077] If the commodity information and the corresponding commodity image are determined to be inconsistent, the corresponding commodity price tag has a price tag information error problem;

[0078] If no price tag is detected within the specified range of the center point of the commodity image, there is a price tag missing problem;

[0079] If the center point of the commodity price tag image does not fall within the specified range of the corresponding commodity image, the corresponding commodity price tag has a placement error problem; or

[0080] If the commodity price tag does not have a placement error problem, and the center point of the corresponding commodity price tag image falls within the specified range of the corresponding commodity image, the corresponding commodity price tag does not have a problem.

[0081] In some embodiments, each extracted commodity price tag image is also classified as a normal image and a non-normal image by a third identification model, and the problem category corresponding to the non-normal image is distinguished. The third identification model is a pre-trained price tag classification model. The commodity price tag image extracted in step S101 is classified to distinguish normal commodity price tags and commodity price tags with problems. Problematic commodity price tags, for example, are commodity price tags that have problems such as large area damage, obstruction, stains, and glare, which prevent the content from being identified. Commodity price tags with problems are shown in Figures 2-4 Normal commodity price tags are shown in Figures 5-6 .

[0082] The obtaining step of the training set for training the price tag classification model includes: cutting out the identified product price tag images from the existing shelf picture data set, and performing category labeling to obtain. For example, label "normal", "obstruction", "damage", "stain", "reflection". Through multiple training and optimization, the price tag classification model can better capture the detailed features of the price tag picture to improve the classification accuracy and the generalization ability of the model.

[0083] The following details the process of extracting text content and identifying product information from the normal product price tags shown in Figure 5 、 Figure 6 .

[0084] For the normal electronic product price tags shown in Figure 5 , the extraction of text content and identification of product information includes the following steps:

[0085] 1) Extract the text information in all text blocks in the electronic product price tag image through OCR technology, and perform data cleaning;

[0086] 2) Determine whether it is the same line by the relative position of the text blocks, perform text splicing, and obtain the spliced information;

[0087] 3) Perform text analysis on the above text, input the content entity extraction model to extract product name, price and other entities, and match and verify with the product name in the product database to obtain the system product name corresponding to the price tag.

[0088] Among them, the input of the named entity recognition model obtains the entity extraction result as shown in Table 1.

[0089] Table 1

[0090]

[0091] Calculate the cosine similarity after word embedding of the extracted product name "certain rice crust (black sesame spicy) 220G" and the product name in the product database, select the result with the highest similarity, and obtain the accurate system product name "certain rice crust (black sesame spicy) 220G".

[0092] In step S103, the recognition result of the product information obtained in Table 1 above is further compared with the product information in the product database to determine whether the current product price tag has compliance problems. The qualified problem judgment result is shown in Table 2.

[0093] Table 2

[0094]

[0095]

[0096] For Figure 6 The normal paper commodity price tag shown in the text content and the identification of commodity information includes the following steps:

[0097] 1) Extract the text information in all text blocks in the paper commodity price tag image by OCR technology, and perform data cleaning;

[0098] 2) Determine whether it is the same line by the relative position of the text block, splice the text, and obtain the spliced text;

[0099] 3) Perform text analysis on the spliced text, input the content entity extraction model to extract entity such as commodity name and price, and match and verify with the commodity name in the commodity database to obtain the system commodity name corresponding to the price tag.

[0100] Among them, the entity extraction result is shown in Table 3.

[0101] Table 3

[0102] Entity class Instance Commodity name Some cheese chocolate flavor heart tart 118G Price 14.9 Code 70345804 Unit Biscuit / cake Points 4966

[0103] The extracted commodity name "some cheese chocolate flavor soft heart tart 118G" is word embedded with the commodity name in the commodity database, and the cosine similarity is calculated to select the result with the highest similarity to obtain the accurate system commodity name "some cheese chocolate flavor soft heart tart 118G".

[0104] In step S103, the recognition result of the commodity information obtained in Table 3 above is further compared with the commodity information in the commodity database to determine whether the current paper commodity price tag has compliance problems.

[0105] The commodity price tag recognition method of the embodiment uses multiple tools to shoot the front image of the commodity shelf, automatically analyzes the commodity price tag in the image, considers the position relationship of the commodity and the price tag and the content information of the price tag, identifies problems such as price error, price tag missing and price tag damage, improves the accuracy of price tag recognition, avoids the problem that traditional price tag management relies on manual maintenance and inspection, and is prone to errors or delays, improves the efficiency and accuracy of retail enterprise management, and provides more business opportunities and competitive advantage for retail enterprises. Compared with the prior art, the technical advantages of the commodity price tag recognition method of the present application are:

[0106] (1) Automatically complete the detection and identification of commodity price tags and commodities on the shelf image, without limiting the specific source of the shelf image, flexible sampling method, and wide applicability to various commercial scenes such as large supermarkets, convenience stores, etc.

[0107] (2) The type and style of the commodity price tag are not required, and it is applicable to electronic price tags and paper price tags.

[0108] (3) On the basis of obtaining all the text information on the price tag, a text analysis and entity extraction model is introduced to minimize the loss of price tag information caused by the problems of picture shooting angle, clarity, distortion or missing, etc.

[0109] (4) The positional relationship between the commodity and the commodity price tag is associated, and the corresponding relationship between the price tag position, the price tag content and the commodity is comprehensively considered. In addition to the misplacement, the price tag error problem caused by the problems such as blur and stain can also be effectively coped with.

[0110] The embodiment of the present application provides a multi-commodity joint pricing device, and the component structure thereof is as shown in the figure. Figure 7 The image recognition module, the text recognition module and the problem recognition module are included.

[0111] The image recognition module is configured to obtain a front image of a shelf, and extract a commodity image, a commodity price tag image and position coordinates of the commodity image and the commodity price tag image in the front image through a pre-trained first recognition model.

[0112] The text recognition module is configured to extract text content in the commodity price tag image, and extract commodity information from the text content through a pre-trained second recognition model.

[0113] The problem recognition module is configured to recognize whether there is a problem in the commodity price tag on the shelf and a problem category according to the extracted commodity information, the corresponding commodity image and the corresponding position coordinates.

[0114] In some embodiments, the problem recognition module is further configured to classify each extracted commodity price tag image into a normal image and an abnormal image through a pre-trained third recognition model, and distinguish the problem category corresponding to the abnormal image.

[0115] In some embodiments, the problem category corresponding to the abnormal image includes a damage, an occlusion or a reflection problem.

[0116] In some embodiments, the problem recognition module recognizes whether there is a problem in the commodity price tag on the shelf and a problem category according to the extracted commodity information, the corresponding commodity image and the corresponding position coordinates, including one or more of the following:

[0117] If the commodity information and the corresponding commodity image are judged to be inconsistent, the corresponding commodity price tag has a price tag information error problem;

[0118] If no price tag is detected within a specified range of the center point of the commodity image, there is a price tag missing problem;

[0119] If the center point of the commodity price tag image does not fall within the specified range of the corresponding commodity image, the corresponding commodity price tag has a placement error problem; or

[0120] If the commodity price tag does not have a placement error problem and the center point of the corresponding commodity price tag image falls within the specified range of the corresponding commodity image, the corresponding commodity price tag does not have a problem.

[0121] In some embodiments, the commodity information includes one or more of the following: commodity name, commodity price, grade, unit, specification, origin, commodity image.

[0122] In some embodiments, the problem identification module is further configured to:

[0123] comparing the first commodity information obtained by searching in the commodity sales system with the second commodity information obtained by identifying the commodity price tags on the shelves, identifying a labeling error problem of the commodity labels.

[0124] The embodiment of the present application provides a commodity price tag identification device, which comprises a processor and a memory, wherein the image identification module, the text identification module and the problem identification module are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions.

[0125] The processor comprises a core, and the core retrieves the corresponding program units from the memory. The core can be one or more, and the core parameters are adjusted to identify whether the commodity price tags on the shelves have problems and the problem categories.

[0126] The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0127] The embodiment of the present application provides a machine readable storage medium, and the machine readable storage medium stores instructions for causing a machine to execute the commodity price tag identification method of the present application.

[0128] The embodiment of the present application provides a processor for running a program, wherein the program is used to execute the commodity price tag identification method of the present application when the program is run.

[0129] The embodiment of the present application provides a device, which comprises a processor, a memory and a program stored in the memory and capable of running on the processor, and the processor executes the program to realize the steps of the commodity price tag identification method of the present application. The device herein can be a server, a PC, a PAD, a mobile phone or the like.

[0130] The application also provides a computer program product comprising a computer program which, when executed by a data processing device, implements the commodity price tag recognition method of the application.

[0131] Those skilled in the art will understand that the embodiments of the application can be provided as methods, systems, or computer program products. Therefore, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0132] The application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the functions specified in the flowchart

[0133] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the functions specified in the flowchart

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the functions specified in the flowchart

[0135] In a typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0136] Memory can include non-persistent memory, Random Access Memory (RAM), and / or non-volatile memory, such as Read Only Memory (ROM) or flash memory, in a computer readable medium. Memory is an example of computer readable media.

[0137] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0138] It should also be noted that the terms "comprising", "containing", or any other variant thereof, are intended to encompass a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not include only those elements in the list, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0139] The above merely provides an example of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method for recognizing product price tags, used to identify product price tags on shelves, characterized in that, include: Obtain a front image of the shelf, and extract the product image, product price tag image, and position coordinates of the product image and product price tag image from the front image using a pre-trained first recognition model; The text content in the product price tag image is extracted, and product information is extracted from the text content using a pre-trained second recognition model; and Based on the extracted product information, corresponding product images, and corresponding location coordinates, identify whether there are any problems with the price tags on the shelves and the type of problem. The step of identifying whether there are problems with the price tags on the shelves and the type of problems based on the extracted product information, corresponding product images, and corresponding location coordinates includes one or more of the following: If the product information and the corresponding product image are determined to be inconsistent, then the corresponding product price tag has an error in the price tag information; If no price tag is detected within the specified range of the center point of the product image, then there is a problem of missing price tags; If the center point of the product price tag image does not fall within the specified range of the corresponding product image, then the corresponding product price tag has a placement error; or If the product price tag is not misplaced, and the center point of the corresponding product price tag image falls within the specified area of ​​the corresponding product image, then the corresponding product price tag is not problematic. Extracting text content from the product price tag image includes: using Optical Character Recognition (OCR) technology to recognize all text information in the product price tag image; determining whether different text blocks belong to the same line based on the relative size, position, and distance between text blocks; concatenating adjacent text blocks into a complete line of information; and converting the text on the product price tag image into electronic text data. The second recognition model is a pre-trained content entity extraction model.

2. The product price tag identification method according to claim 1, characterized in that, Also includes: The pre-trained third recognition model classifies each extracted product price tag image into normal images and abnormal images, and distinguishes the problem category corresponding to the abnormal images.

3. The product price tag identification method according to claim 2, characterized in that, The problem categories corresponding to the abnormal images include damage, stains, obstructions, or reflections.

4. The product price tag identification method according to claim 1, characterized in that, The product information includes one or more of the following: product name, product price, grade, unit, specifications, place of origin, and product image.

5. The product price tag recognition method according to claim 4, characterized in that, Also includes: Using one or more of the product information as search criteria, the first product information is retrieved in the product sales system and compared with the second product information obtained by identifying the price tags on the shelves to identify labeling errors in the product labels.

6. A product price tag recognition device for recognizing product price tags on a shelf, characterized in that, include: Image recognition module, text recognition module, and question recognition module. The image recognition module is configured to: acquire a front image of the shelf, and extract the product image, product price tag image, and position coordinates of the product image and product price tag image from the front image using a pre-trained first recognition model; The text recognition module is configured to: extract text content from the product price tag image, and extract product information from the text content using a pre-trained second recognition model; and The problem identification module is configured to: identify whether there are problems with the price tags on the shelves and the type of problems based on the extracted product information, corresponding product images, and corresponding location coordinates. In the problem identification module, the step of identifying whether there is a problem with the price tag on the shelf and the type of problem based on the extracted product information, corresponding product images, and corresponding location coordinates includes one or more of the following: If the product information and the corresponding product image are determined to be inconsistent, then the corresponding product price tag has an error in the price tag information; If no price tag is detected within the specified range of the center point of the product image, then there is a problem of missing price tags; If the center point of the product price tag image does not fall within the specified range of the corresponding product image, then the corresponding product price tag has a placement error; or If the product price tag is not misplaced, and the center point of the corresponding product price tag image falls within the specified area of ​​the corresponding product image, then the corresponding product price tag is not problematic. Extracting text content from the product price tag image includes: using Optical Character Recognition (OCR) technology to recognize all text information in the product price tag image; determining whether different text blocks belong to the same line based on the relative size, position, and distance between text blocks; concatenating adjacent text blocks into a complete line of information; and converting the text on the product price tag image into electronic text data. The second recognition model is a pre-trained content entity extraction model.

7. The product price tag recognition device according to claim 6, characterized in that, The problem identification module is further configured to classify each extracted product price tag image into normal images and abnormal images using a pre-trained third identification model, and to distinguish the problem category corresponding to the abnormal images.

8. The product price tag recognition device according to claim 7, characterized in that, The problem categories corresponding to the abnormal images include damage, stains, obstructions, or reflections.

9. The product price tag recognition device according to claim 6, characterized in that, The product information includes one or more of the following: product name, product price, grade, unit, specifications, place of origin, and product image.

10. The product price tag recognition device according to claim 9, characterized in that, The problem identification module is also configured to: Using one or more of the product information as search criteria, the first product information is retrieved in the product sales system and compared with the second product information obtained by identifying the price tags on the shelves to identify labeling errors in the product labels.

11. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to perform: the product price tag recognition method according to any one of claims 1-5.

12. A processor, characterized in that, Used to run a program, wherein the program is run to perform: the commodity price tag recognition method according to any one of claims 1-5.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the product price tag recognition method according to any one of claims 1-5.

Images