Electronic price tag identification method and apparatus

Abstract

The present disclosure discloses an electronic price tag identification method and apparatus. The method comprises: performing image recognition and cutting on an electronic shelf image collected on the basis of a visual sensor device, and obtaining a plurality of electronic price tag images; extracting, from a target electronic price tag image, target image elements of a specified type and / or number which correspond to an image element extraction instruction; determining a target image element matching rule of the target electronic price tag image, the image element matching rule comprising a degree of matching calculation rule of different types of image elements and a degree of matching calculation sequence of a plurality of image elements; performing a degree of matching calculation operation between the target image element and a corresponding pre-stored element, and obtaining a degree of matching calculation result; and generating an identification result of the target electronic price tag image. According to the present disclosure, the accuracy and efficiency of electronic price tag identification are improved, and the device costs of electronic price tag identification is reduced.

Description

Electronic price tag recognition method and device

[0001] Related applications

[0002] This application claims priority to Chinese Patent Application No. 202410437524.0, filed on April 11, 2024, and incorporates by reference the entire disclosure of the aforementioned patent application as part of this application. TECHNICAL FIELD

[0003] The present disclosure relates to the field of image recognition and artificial intelligence, and in particular, to an electronic price tag recognition method and device. BACKGROUND

[0004] This section is intended to provide background or context to the embodiments of the present disclosure. The description herein does not constitute admission that the information provided herein is prior art.

[0005] With the continuous development of artificial intelligence technology, the supermarket industry is also constantly transforming towards intelligence and digitization. The application of AI (Artificial Intelligence) technology enables supermarkets to use AI technology to digitally convert shelves, upgrade to an automated inventory management system, reduce inventory costs and shelf retention rates, and quickly replenish goods when necessary to ensure that customers' shopping needs are met.

[0006] Among them, the construction of a digital shelf is based on the real-time display of shelf goods, and the real-time display relies on the detection and recognition of price tags on the shelf in many cases, and finally uses price tag display as the basis for the entire digital shelf display.

[0007] Currently, most of the methods for price tag recognition are based on OCR (Optical Character Recognition) methods, which accurately recognize all information on the image of each price tag, such as recognizing the product name, product price, and product barcode. However, the OCR-based method has high requirements for the resolution or clarity of the input image, and the characters recognized by OCR from images with low clarity are prone to errors or cannot be recognized. SUMMARY

[0008] The present disclosure provides an electronic price tag recognition method to improve the accuracy and efficiency of electronic price tag recognition and reduce the cost of electronic price tag recognition equipment. The method includes:

[0009] Performing image recognition and cropping on electronic shelf images collected based on a visual sensor device to obtain multiple electronic price tag images in the electronic shelf images.

[0010] extract target image elements of a specified type and / or quantity corresponding to the image element extraction instruction from the target electronic price tag image based on the image element extraction instruction; the image elements include graphical elements and feature elements of an image; the graphical elements include overall graphical elements and partial graphical elements of the electronic price tag image; the feature elements include overall image feature elements, partial image character feature elements, and partial image identification feature elements;

[0011] based on a pre-set association between different types and quantities of image elements and different image element matching rules, determine an image element matching rule associated with the specified type and / or quantity of target image elements as a target image element matching rule of the target electronic price tag image; the image element matching rule includes a matching degree calculation rule of different types of image elements and a matching degree calculation order of multiple image elements;

[0012] For the target image element, perform a matching degree calculation operation between the target image element and a corresponding pre-stored element based on the matching degree calculation rule and the matching degree calculation order corresponding to the type of the target image element, to obtain a matching degree calculation result; the pre-stored element is an image element of each pre-stored electronic price tag image in a pre-stored electronic price tag image element library;

[0013] generate an identification result of the target electronic price tag image according to the matching degree calculation result.

[0014] The embodiments of the present disclosure also provide an electronic price tag identification device to improve the accuracy and efficiency of electronic price tag identification and reduce the equipment cost of electronic price tag identification, and the device comprises:

[0015] An electronic price tag image identification and cutting module is configured to perform image identification and cutting on an electronic shelf image collected based on a visual sensor device to obtain multiple electronic price tag images in the electronic shelf image.

[0016] A target image element extraction module is configured to extract target image elements of a specified type and / or quantity corresponding to an image element extraction instruction from a target electronic price tag image based on the image element extraction instruction; the image elements include graphical elements and feature elements of an image; the graphical elements include overall graphical elements and partial graphical elements of the electronic price tag image; the feature elements include overall image feature elements, partial image character feature elements, and partial image identification feature elements.

[0017] The target image element matching rule determination module is configured to determine, based on a pre-set association relationship between different image element categories and quantities and different image element matching rules, an image element matching rule associated with a specified category and / or quantity of the target image element as the target image element matching rule of the target electronic price tag image; the image element matching rule includes a matching degree calculation rule of different categories of image elements and a matching degree calculation sequence of multiple image elements;

[0018] The matching degree calculation module is configured to, for the target image element, perform a matching degree calculation operation between the target image element and a corresponding pre-stored element according to the matching degree calculation rule and the matching degree calculation sequence corresponding to the category of the target image element, to obtain a matching degree calculation result; the pre-stored element is an image element of each pre-stored electronic price tag image in a pre-set electronic price tag image element library;

[0019] The recognition result generation module is configured to generate a recognition result of the target electronic price tag image according to the matching degree calculation result.

[0020] The embodiments of the present disclosure further provide a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above electronic price tag recognition method when executing the computer program.

[0021] The embodiments of the present disclosure further provide a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the above electronic price tag recognition method.

[0022] The embodiments of the present disclosure further provide a computer program product, which includes a computer program, and the computer program is executable on a processor to implement the above electronic price tag recognition method.

[0023] In the embodiments of the present disclosure, based on an electronic shelf image collected by a visual sensor device, image recognition and cropping are performed to obtain a plurality of electronic price tag images in the electronic shelf image; based on an image element extraction instruction, target image elements of a specified type and / or quantity corresponding to the image element extraction instruction are extracted from a target electronic price tag image; the image elements include graphic elements and feature elements of the image; the graphic elements include overall graphic elements and local graphic elements of the electronic price tag image; the feature elements include overall image feature elements, local image character feature elements, and local image identification feature elements; based on an association relationship between the types and quantities of different image elements and different image element matching rules, an image element matching rule associated with the specified type and / or quantity of the target image elements is determined as a target image element matching rule of the target electronic price tag image; the image element matching rule includes a matching degree calculation rule of different types of image elements and a matching degree calculation order of a plurality of image elements; for a target image element, a matching degree calculation operation between the target image element and a corresponding pre-stored element is performed based on the matching degree calculation rule and the matching degree calculation order corresponding to the type of the target image element, and a matching degree calculation result is obtained; the pre-stored element is an image element of each pre-stored electronic price tag image in a pre-stored electronic price tag image element library; and an identification result of the target electronic price tag image is generated based on the matching degree calculation result, different types and quantities of image elements of the electronic price tag image can be extracted, and different image element matching rules are set for different image elements and combinations thereof, thereby realizing diversified matching of image elements of the electronic price tag. Compared with the prior art scheme in which only the image similarity between the electronic price tag image and the pre-stored electronic price tag image can be calculated for identification, the problem of inability to identify due to unclear electronic price tag images is avoided, accurate identification of the electronic price tag can still be realized in the case of poor image quality, the accuracy and efficiency of electronic price tag identification are improved, and the problem of increased cost of electronic price tag identification due to the need for a high-definition shooting system for OCR identification in the prior art is also avoided, thereby reducing the equipment cost of electronic price tag identification. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, brief descriptions will be given below for the drawings needed to be used in the embodiments or prior art descriptions. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without any creative effort. In the drawings:

[0025] FIG. 1 is a flow diagram of an electronic price tag identification method according to an embodiment of the present disclosure;

[0026] FIG. 2 is a specific example diagram of an electronic price tag identification method according to an embodiment of the present disclosure;

[0027] FIG. 3 is a specific example diagram of an electronic price tag recognition method in an embodiment of the present disclosure;

[0028] FIG. 4 is a structural schematic diagram of an electronic price tag recognition device in an embodiment of the present disclosure;

[0029] FIG. 5 is a schematic diagram of a computer device for electronic price tag recognition in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, further detailed description will be given to the embodiments of the present disclosure in combination with the drawings. Herein, the illustrative embodiments of the present disclosure and the description thereof are used to explain the present disclosure, but not as a limitation of the present disclosure.

[0031] The term “and / or” herein is merely used to describe an associated relationship, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term “at least one” herein means any one of multiple or any combination of at least two of multiple, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.

[0032] In the description of the present specification, “include”, “comprise”, “have”, “contain” and the like are all open terms, which means to include but not limited to. The description of the terms “one embodiment”, “one specific embodiment”, “some embodiments”, “for example” and the like means that the specific features, structures or characteristics described in combination with the embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. The order of steps involved in each embodiment is used to illustrate the embodiments of the present application, and the order of steps is not limited, which can be adjusted as needed.

[0033] The acquisition, storage, use, processing, etc. of data in the technical solutions of the present application comply with relevant provisions of national laws and regulations. The information collected in the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with relevant laws and regulations and standards of relevant countries and regions, necessary security measures are taken, it does not violate public order and good customs, and corresponding operation entrances are provided for the user to choose authorization or refusal. In addition, the present application provides corresponding operation entrances for the user to choose to agree or refuse the automatic decision result, if the user chooses to refuse, the expert decision process can be entered.

[0034] The present disclosure embodiments are related to the following terms, which are explained as follows:

[0035] Electronic price tag: refers to the electronic type commodity price tag commonly seen in existing supermarkets, which can switch pages and has a flashing LED light that can flash brightly;

[0036] Electronic price tag profile image: refers to the image information or layout information actually displayed by the electronic price tag, including price, commodity name, barcode, etc., as well as the layout, size, color, etc. of these information.

[0037] With the increasing maturity of artificial intelligence technology, the supermarket industry is gradually realizing intelligent and digital transformation and upgrading. By using AI technology, supermarket enterprises can convert shelves into digital shelves and upgrade them to an automatic inventory management system, thereby reducing inventory costs and shelf retention rates and ensuring that goods can be quickly replenished when necessary to meet customers' shopping needs. The construction of digital shelves is based on real-time display of shelf goods, and real-time display in many cases relies on detection and recognition of price tags on the shelves, and ultimately uses price tag display as the basis for the entire digital shelf display.

[0038] At present, the price tag recognition method is mostly based on OCR technology, which accurately recognizes all information on each price tag image, such as commodity name, commodity price, commodity barcode, etc. However, the OCR-based method has a high requirement for the resolution and clarity of the input image, and if the clarity is not high, the characters recognized by OCR are prone to errors or cannot be recognized.

[0039] To solve the above problems, the present disclosure example fully utilizes the profile image data stored in the electronic price tag information server to realize the matching recognition of the electronic price tag, thereby reducing the dependence on the clarity of the input image. By detecting the position area of the electronic price tag in the shelf picture, the actual image of the electronic price tag is extracted and matched with the profile image information in the electronic price tag system. This fully taps the value of the profile image data stored in the electronic price tag information server, avoiding the need for a high-definition shooting system that uses OCR to obtain and recognize the results of the target price tag image.

[0040] The electronic price tag recognition method provided by the embodiments of the present disclosure can improve the accuracy and efficiency of electronic price tag recognition and reduce the equipment cost of electronic price tag recognition. Referring to FIG. 1, the method can include the following steps.

[0041] Step 101: performing image recognition and cropping on an electronic shelf image collected based on a visual sensor device to obtain a plurality of electronic price tag images in the electronic shelf image.

[0042] Step 102: extracting target image elements of a specified type and / or quantity corresponding to an image element extraction instruction from a target electronic price tag image based on the image element extraction instruction; the image elements include graphic elements and feature elements of the image; the graphic elements include overall graphic elements and local graphic elements of the electronic price tag image; and the feature elements include overall image feature elements, local image character feature elements, and local image identification feature elements.

[0043] Step 103: determining, based on an association between the type and quantity of different image elements and different image element matching rules, an image element matching rule associated with the specified type and / or quantity of the target image elements as a target image element matching rule of the target electronic price tag image; the image element matching rule includes a matching degree calculation rule of different types of image elements and a matching degree calculation order of a plurality of image elements.

[0044] Step 104: performing a matching degree calculation operation between the target image element and a corresponding pre-stored element based on the matching degree calculation rule and the matching degree calculation order corresponding to the type of the target image element to obtain a matching degree calculation result; the pre-stored element is an image element of each pre-stored electronic price tag image in a pre-stored electronic price tag image element library.

[0045] Step 105: generating a recognition result of the target electronic price tag image according to the matching degree calculation result.

[0046] In step 101, the electronic shelf image collected based on the visual sensor device is preprocessed, including image denoising, color balance, contrast enhancement, and the like, to improve the accuracy of subsequent image recognition.

[0047] In step 102, edge detection, morphological processing, and the like are used to extract the target image elements of the specified type and / or quantity corresponding to the image element extraction instruction from the target electronic price tag image. In addition, for the text information in the electronic price tag image, the OCR (Optical Character Recognition) technology can be used for extraction.

[0048] In step 103, according to the pre-set association between the category and quantity (i.e., category and / or quantity) of different image elements and the matching rule of different image elements, the matching rule of target image elements for the target electronic price tag image is determined. This step can be trained and learned by machine learning algorithms such as support vector machine (SVM), neural network, etc.

[0049] In step 104, for the target image element, according to the matching degree calculation rule and matching degree calculation order corresponding to its category, the matching degree calculation operation between the target image element and the corresponding pre-stored element is performed. This step can obtain the matching degree by calculating the similarity between the target image element and the pre-stored element, such as mean squared error (MSE), structural similarity index (SSIM), etc.

[0050] In the embodiments of the present disclosure, based on the electronic shelf image collected by the visual sensor device, image recognition and cropping are performed to obtain a plurality of electronic price tag images in the electronic shelf image; based on an image element extraction instruction, target image elements of a specified type and / or quantity corresponding to the image element extraction instruction are extracted from a target electronic price tag image; the image elements include graphic elements and feature elements of the image; the graphic elements include overall graphic elements and local graphic elements of the electronic price tag image; the feature elements include overall image feature elements, local image character feature elements, and local image identification feature elements; based on the association between the type and quantity of different image elements and different image element matching rules, the image element matching rules associated with the specified type and / or quantity of the target image elements are determined as the target image element matching rules of the target electronic price tag image; the image element matching rules include matching degree calculation rules of different types of image elements and matching degree calculation sequences of a plurality of image elements; for the target image elements, the matching degree calculation rules and the matching degree calculation sequences corresponding to the type of the target image elements are used to perform matching degree calculation operations between the target image elements and corresponding pre-stored elements to obtain matching degree calculation results; the pre-stored elements are image elements of each pre-stored electronic price tag image in a pre-set electronic price tag image element library; and based on the matching degree calculation results, an identification result of the target electronic price tag image is generated, different types and quantities of image elements of the electronic price tag image can be extracted, and different image element matching rules are set for different image elements and combinations thereof, thereby realizing diversified matching of the image elements of the electronic price tag. Compared with the scheme of only calculating the image similarity between the electronic price tag image and the pre-stored electronic price tag image to identify, the problem of being unable to identify due to unclear electronic price tag images is avoided, accurate identification of the electronic price tag can still be realized in the case of poor image quality, the accuracy and efficiency of the electronic price tag identification are improved, and the problem of increased cost of electronic price tag identification due to the need for a high-definition shooting system for OCR identification in the prior art is also avoided, thereby reducing the equipment cost of electronic price tag identification.

[0051] It is worth noting that the electronic price tag identification method in the embodiments of the present disclosure can not only be applied to related servers, which can include but are not limited to servers in commercial environments such as shopping malls, supermarkets, and warehouses, but also can be applied to other scenarios that require item management and identification, such as libraries, museums, and hospitals. In these scenarios, the electronic price tag identification method can help managers more efficiently and accurately identify item information, improve management efficiency, and improve service quality; in addition, the electronic price tag identification method in the embodiments of the present disclosure can also be applied to various terminal devices with visual sensor devices, such as smartphones, tablet computers, and smart cameras. These terminal devices can realize the identification function of the electronic price tag by installing corresponding applications or software.

[0052] In implementation, first, image recognition and cropping are performed on the electronic shelf image collected based on the visual sensor device to obtain multiple electronic price tag images in the electronic shelf image.

[0053] In the embodiment, the visual sensor device is a visual sensor device integrated with an AI vision system, which can control the visual sensor device to take pictures. The visual sensor device includes but is not limited to a fixedly installed camera, a movable camera, a rotatable camera, a movable robot with a camera, a flying vehicle with a camera, and the like, and the above visual sensor devices are all looking down / looking up / looking directly at the commodity shelves in the supermarket to obtain the shelf image.

[0054] In one embodiment, image recognition and cropping are performed on the electronic shelf image collected based on the visual sensor device to obtain multiple electronic price tag images in the electronic shelf image, which can include: performing AI recognition and inference calculation on the visual image of the electronic shelf image, wherein the AI recognition and inference calculation can include: image processing of the shelf image, detection of the position information of the price tag on the image, detection, recognition, and conversion of the target matching element, and the like.

[0055] In another embodiment, the collected electronic shelf image can also be preprocessed, including denoising, grayscale, binarization, and the like. The preprocessed image is beneficial to subsequent image recognition and extraction work.

[0056] Specifically, for the cropped price tag image, certain image level processing can be performed, such as adjusting brightness, contrast, sharpness, and the like, and a deep learning method can also be used for super-resolution processing to increase the clarity of the target price tag image.

[0057] Using a deep learning method for super-resolution processing to increase the clarity of the target price tag image can specifically include: using a convolutional neural network (CNN) model in a deep learning algorithm to perform super-resolution processing on the cropped price tag image. First, a large number of high-definition price tag images are used to train the CNN model, so that it can learn the features and details in the price tag image. Then, the low-resolution price tag image to be processed is input into the trained CNN model, and the model will automatically learn and generate a high-resolution price tag image. In this way, the clarity of the target price tag image can be significantly improved, providing more accurate and reliable information for subsequent identification and reading work.

[0058] After processing the images, the system can perform further recognition and processing on each cropped price tag image. For example, the system can use OCR (Optical Character Recognition) technology to extract key information such as the price, name, specifications of the product from the price tag image and convert it into editable and processable text format. These information can be used for various business intelligence applications such as inventory management, price monitoring, market analysis, etc.

[0059] As an example of using a convolutional neural network (CNN) model in deep learning algorithms to perform super-resolution processing on the cropped price tag image:

[0060] First, a large number of high-definition price tag images are collected for training the CNN model. These images contain information such as the price, name, specifications of various products, and have clear image quality and rich details. By preprocessing and labeling these images, they construct a large-scale price tag image dataset.

[0061] Next, a convolutional neural network model is built using a deep learning framework such as TensorFlow or PyTorch. The model uses advanced super-resolution algorithms such as SRCNN, EDSR or RCAN to improve the resolution and clarity of the image. During the training process, the model continuously learns and optimizes, gradually improving its ability to recognize features and details in the price tag image.

[0062] When the model is trained, it is deployed to the actual monitoring system. Whenever the system detects a price tag image, it automatically crops the target area and inputs the low-resolution image into the trained CNN model. The model quickly generates a high-resolution price tag image, making the product information clear and visible.

[0063] By using a convolutional neural network (CNN) model in deep learning algorithms for super-resolution processing, the problem of low image resolution is solved. This not only improves the clarity of the price tag image, but also provides more accurate and reliable information for inventory management, price monitoring, etc.

[0064] In specific implementation, after image recognition and cropping on the electronic shelf image collected based on the visual sensor device, a plurality of electronic price tag images in the electronic shelf image are obtained, and based on an image element extraction instruction, a target image element of a specified type and / or quantity corresponding to the image element extraction instruction is extracted from the target electronic price tag image; the image element includes a graphical element and a characteristic element of the image; the graphical element includes an overall graphical element and a local graphical element of the electronic price tag image; the characteristic element includes an overall image characteristic element, a local image character characteristic element, and a local image identification characteristic element.

[0065] In an embodiment, according to pre-set image element extraction instructions, specific types and / or quantities of target image elements are extracted from the processed electronic price tag image. These image elements include overall and partial graphic elements, as well as overall image feature elements, partial image character feature elements, and partial image identification feature elements.

[0066] In an embodiment, the overall graphic elements of the electronic price tag image in terms of graphic elements can include the shape, size, texture, etc. of the electronic price tag; the partial graphic elements can include graphics, symbols, patterns, etc. on the electronic price tag. In terms of feature elements, the overall image feature elements can include the brightness, color, contrast, etc. of the electronic price tag image; the partial image character feature elements can include the text, numbers, letters, etc. on the electronic price tag; and the partial image identification feature elements can include specific marks, symbols, etc. on the electronic price tag.

[0067] In a specific embodiment, the present disclosure also provides examples of image elements of the electronic price tag image and extraction schemes of the image elements:

[0068] The overall graphic element extraction process of the electronic price tag image is as follows: first, the electronic price tag image is pre-processed, such as denoising, smoothing, edge detection, etc. to facilitate subsequent graphic element extraction. Then, through morphological processing such as opening operation, closing operation, etc., the shape, size, texture, etc. of the electronic price tag are extracted. Next, the extracted overall graphic elements are feature extracted, such as calculating moments (or image moment calculation information), feature vectors, etc. to facilitate subsequent matching and recognition. Finally, the extracted overall graphic elements are stored and output for subsequent analysis and processing.

[0069] Another example of overall graphic element extraction: morphological processing can be used to further extract the overall graphic elements of the electronic price tag. Morphological processing mainly includes opening operation and closing operation. Opening operation is mainly used to eliminate small noise points and fine edges inside the electronic price tag, thereby extracting the shape of the electronic price tag. Closing operation is mainly used to fill the holes inside the electronic price tag and fill the edges, thereby extracting the size and texture of the electronic price tag.

[0070] After morphological processing, feature extraction is needed for the extracted overall graphic elements. Feature extraction is to extract the unique features of the electronic price tag to facilitate subsequent matching and recognition. Commonly used feature extraction methods include calculating moments and feature vectors, etc. Calculating moments is a feature that describes the shape of a graphic, which can accurately describe the shape information of the electronic price tag. Feature vectors are a feature that describes the texture of a graphic, which can accurately describe the texture information of the electronic price tag.

[0071] Finally, the extracted overall graphical elements are stored and outputted. The storage and output are to save the extracted electronic price tag feature information for subsequent analysis and processing. The storage and output can be in various forms, such as text files, image files, etc. In addition, the extracted electronic price tag feature information can be integrated with other related information to form a complete information database for subsequent query and management.

[0072] The local graphical element extraction process is as follows: first, the electronic price tag image is segmented into multiple local regions. Then, each local region is pre-processed and morphologically processed, such as denoising, smoothing, edge detection, etc., to facilitate subsequent graphical element extraction. Next, local graphical elements are extracted by feature extraction, such as calculating the moment of the local region, feature vector, etc. Finally, the extracted local graphical elements are stored and outputted for subsequent analysis and processing.

[0073] Another example of local graphical element extraction: for the cropped target matching price tag image, use the detection and recognition model to detect and recognize the local region with representative information in the image, such as the product name region, barcode region, price region, and finally crop one or more regions of the local image as the subsequent matching element.

[0074] After completing the local image character feature element extraction, the extracted local image can be analyzed in detail. First, for different regions of characters, such as product name, barcode, and price, image segmentation technology is used to separate each region. The purpose of this step is to reduce the influence of external factors such as noise and light changes on subsequent processing.

[0075] The overall image feature element extraction process is as follows: first, the electronic price tag image is processed to grayscale to facilitate subsequent feature extraction. Then, by calculating the brightness, color, contrast, etc. of the image, the overall image feature elements are extracted. Next, the extracted overall image feature elements are normalized to eliminate image scale differences. Finally, the extracted overall image feature elements are stored and outputted for subsequent analysis and processing.

[0076] Another example of overall image feature element extraction: the overall image feature elements can include a feature map and / or a feature vector representing the overall electronic price tag image.

[0077] One, the feature map representing the overall electronic price tag image: the cropped target matching price tag image can be extracted and converted using feature descriptors or deep learning algorithm models, and a feature map with a certain size can be extracted and converted as a subsequent matching element. The feature descriptors include but are not limited to SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), etc.

[0078] Second, the feature vector representing the overall electronic price tag image: the cropped target matching price tag image can be extracted using feature descriptors or deep learning algorithm models, and a feature vector with a certain length can be extracted as a subsequent matching element.

[0079] The local image character feature element extraction process is as follows: first, the electronic price tag image is binarized to facilitate subsequent character feature extraction. Then, by calculating the features of local area characters, numbers and letters, local image character feature elements are extracted. Next, the extracted local image character feature elements are normalized to eliminate image scale differences. Finally, the extracted local image character feature elements are stored and output for subsequent analysis and processing.

[0080] Another example of local image character feature element extraction: local image identification feature elements can also be referred to as explicit information of local images. The cropped target matching price tag image uses a detection and recognition model to detect and recognize the representative information (i.e. explicit information) in the price tag image, such as price, detects and recognizes the price numbers in “￥5.59” and “$4.99”, or other more obvious characters, or the percentage of the length of the product name in the overall image, etc.

[0081] Subsequently, for each separated local area, character recognition is performed respectively. Character recognition techniques include deep learning methods such as convolutional neural networks (CNN) and recurrent neural networks (RNN). In the recognition process, the extracted local image character feature elements are matched with the pre-trained model to obtain product name, barcode and price information, etc.

[0082] After obtaining the product name, barcode and price information, data processing and analysis are performed. Data processing mainly includes denoising, error correction and formatting operations. Denoising is to eliminate possible misrecognition and missed recognition in the recognition process; error correction is to correct the error characters in the recognition results; and formatting is to unify the recognition results into a standard data format for subsequent data storage and processing.

[0083] After the processing is completed, the extracted local image character feature elements and their corresponding information are stored. During storage, a structured data storage method such as a relational database or a NoSQL database can be used. In this way, during subsequent query, analysis and processing, the required product information can be quickly located.

[0084] The local image identification feature element extraction process is as follows: First, the electronic price tag image is preprocessed, such as denoising, smoothing, edge detection, etc., to facilitate subsequent identification feature extraction. Then, by calculating the specific marks, symbols, etc. of the local area, the local image identification feature elements are extracted. Next, the extracted local image identification feature elements are normalized to eliminate image scale differences. Finally, the extracted local image identification feature elements are stored and output for subsequent analysis and processing.

[0085] Another example of local image identification feature element extraction:

[0086] The profile image set by the price tag system can set some pixel points or pixel blocks that can be used as specific identification, such as setting a variety of circle constellation image codes, small area character codes, small area data matrix, small area QR code. And these identified pixels or pixel blocks can be separated and distinguished using different colors, and finally serve as target elements. For the cropped target matching price tag image, use the labeling information recognition algorithm model to detect or recognize the labeling information pixel block, and finally serve as the matching element.

[0087] Extracting local image identification features can extract local image identification feature elements according to the system set identification types such as circle constellation image code, character code, data matrix, QR code, etc. by calculating the specific marks, symbols, etc. of the local area. Differentiating identification pixels or pixel blocks can use different colors to distinguish the extracted identification pixels or pixel blocks to facilitate subsequent matching and recognition.

[0088] Through the above method, various types of target image elements can be extracted from the electronic shelf image. The image processing flow and feature extraction method can be adjusted according to actual needs to meet the needs of different scenarios and applications.

[0089] In a specific implementation, after extracting the target image elements of the specified type and / or quantity corresponding to the image element extraction instruction from the target electronic price tag image based on the image element extraction instruction, the image element matching rule associated with the specified type and / or quantity of the target image elements is determined as the target image element matching rule of the target electronic price tag image based on the pre-set association between different types and quantities of image elements and different image element matching rules. The image element matching rule includes a matching degree calculation rule of different types of image elements and a matching degree calculation order of multiple image elements.

[0090] In an embodiment, the target image element matching rule of the target electronic price tag image is determined according to the pre-set association between different types and quantities of image elements and different image element matching rules. These matching rules include a matching degree calculation rule of different types of image elements, a matching degree weighted calculation rule between different quantities of image elements, and a matching degree calculation order of multiple image elements.

[0091] In one embodiment, the local graphical element is used to represent the local graph in the electronic price tag image at different positions; the overall image feature element is used to represent the feature map and / or feature vector of the overall electronic price tag image; the local image character feature element is used to represent the character feature of the local graph in the electronic price tag image at different positions, and the position feature and area ratio feature of the local graph in the overall electronic price tag image; the local image identification feature element is used to represent the position feature, shape feature, area ratio feature and color feature of the preset pixel block in the overall electronic price tag image, and the distance feature between other pixel blocks.

[0092] The above elements are illustrated as follows:

[0093] Take a specific electronic price tag image as an example. First, the local graphical element mainly includes various shapes and patterns in the electronic price tag image, which are distributed at different positions in the image. For example, in an electronic price tag image, it may include local graphs of different shapes such as circles, rectangles, triangles, etc.

[0094] Next, the overall image feature element is obtained by processing and analyzing the entire electronic price tag image. These features can include the texture, color, brightness, etc. of the image, which together constitute the unique features of the electronic price tag image. For example, in a series of electronic price tag images, the overall texture and color of each image may be different.

[0095] Next, let's look at the local image character feature elements. This part mainly focuses on the character features of local graphics in electronic price tag images. These character features may include numbers, letters, symbols, etc., and their positions, sizes, and shapes in the overall image are different. For example, in an electronic price tag image, the number "1" may appear in the upper left corner of the image, while the number "2" is located in the lower right corner of the image.

[0096] Finally, the local image identification feature element focuses on the position, shape, area ratio, and color of the preset pixel block in the overall electronic price tag image. In addition, it also includes the distance features between these pixel blocks and other pixel blocks. For example, in an electronic price tag image, a preset pixel block may be a circle, its position is in the center of the image, its shape feature is a circle, its area ratio is 1 / 4, and its color feature is red.

[0097] In specific implementation, after determining the image element matching rule associated with the specified type and / or quantity of the target image element as the target image element matching rule of the target electronic price tag image based on the association between the pre-set types and quantities of different image elements and the matching rules of different image elements, the matching degree calculation operation between the target image element and the corresponding pre-stored element is performed for the target image element according to the matching degree calculation rule and the matching degree calculation order corresponding to the type of the target image element, and the matching degree calculation result is obtained.

[0098] In the embodiment, the matching degree calculation is performed for the extracted target image element according to the matching degree calculation rule and the matching degree calculation order corresponding to its type. The matching degree calculation result is used for subsequent recognition result generation. In one embodiment, the electronic price tag image element library can be a price tag information management server, which can record the binding information of electronic price tags and goods, store the archive image information actually displayed by the electronic price tags (i.e., the pre-stored overall graphic elements of all pre-stored electronic price tag images, the bound character information, and the local image identification feature elements of different pre-stored pixel blocks).

[0099] In specific implementation, after the matching degree calculation operation between the target image element and the corresponding pre-stored element is performed for the target image element according to the matching degree calculation rule and the matching degree calculation order corresponding to the type of the target image element, the matching degree calculation result is obtained, and the recognition result of the target electronic price tag image is generated according to the matching degree calculation result.

[0100] In an embodiment, after the matching degree calculation is completed, the recognition result of the target electronic price tag image is generated according to the matching degree calculation result. When the target image element is multiple, the matching degree calculation result of each target image element is weighted calculated according to the matching degree weighting calculation rule, and finally the recognition result of the target electronic price tag image is obtained.

[0101] In an embodiment, when the target image element is an integral graphic element, the pre-stored element is a pre-stored integral graphic element of all pre-stored electronic price tag images.

[0102] The matching degree calculation operation between the target image element and the corresponding pre-stored element is performed to obtain the matching degree calculation result, including:

[0103] Based on the square difference matching algorithm, the normalized square difference matching algorithm and / or the cross-correlation matching algorithm, the first image similarity between the target image element and each pre-stored integral graphic element is calculated.

[0104] According to the matching degree calculation result, the recognition result of the target electronic price tag image is generated, including:

[0105] According to the pre-stored electronic price tag image corresponding to the pre-stored integral graphic element whose first image similarity with the target image element exceeds the preset threshold, the recognition result of the electronic price tag image is generated.

[0106] In a specific embodiment, when the target image element is an integral graphic element, the pre-stored element is a pre-stored integral graphic element of all pre-stored electronic price tag images. Next, the matching degree calculation is performed between the target image element and the corresponding pre-stored element to obtain the matching degree calculation result. These results are based on the square difference matching algorithm, the normalized square difference matching algorithm and / or the cross-correlation matching algorithm to calculate the first image similarity between the target image element and each pre-stored integral graphic element.

[0107] According to the matching degree calculation result, the recognition result of the target electronic price tag image is generated. This result includes the pre-stored electronic price tag image corresponding to the pre-stored integral graphic element whose first image similarity with the target image element exceeds the preset threshold. Through these information, the recognition result of the electronic price tag image can be generated.

[0108] For example, if the target image element is the entire image, it is compared with all the archive image information set in the price tag system, and the similarity between them is calculated. Common similarity measurement methods include Sum of Squared Differences (SSD), Normalized Sum of Squared Differences (NSSD), Cross-Correlation, etc.

[0109] After obtaining the first image similarity of the target image element and the pre-stored overall graphic element, the similarity needs to be processed and analyzed to determine the final recognition result. For this purpose, the following methods can be used:

[0110] 1. Set a similarity threshold, compare the first image similarity with the threshold. If the similarity of the target image element and the pre-stored overall graphic element exceeds the threshold, it is considered that the recognition is successful, otherwise the recognition fails.

[0111] 2. Sort the first image similarity, and select the top n pre-stored overall graphic elements with the highest similarity to the target image element as the recognition result.

[0112] 3. Multiple similarity measurement methods can be combined, such as Sum of Squared Differences, Normalized Sum of Squared Differences, and Cross-Correlation, to analyze the calculation results of each method and improve the accuracy of recognition.

[0113] 4. The similarity of the target image element and the pre-stored overall graphic element is dynamically adjusted to adapt to the recognition needs of the electronic price tag in different scenarios. For example, in the case of changes in light, angle, resolution, etc., the similarity threshold can be adjusted or multi-modal recognition methods can be used to improve the recognition effect of the electronic price tag.

[0114] 5. When the similarity of the target image element and the pre-stored overall graphic element is low, the pre-stored overall graphic element can be updated or expanded to improve the recognition ability of the electronic price tag recognition system.

[0115] In one embodiment, the local graphic element is used to represent the local graphic in the electronic price tag image at different positions;

[0116] When the target image element is a local graphic element, the pre-stored element is the pre-stored overall graphic element of all pre-stored electronic price tag images, i.e. the data set of the overall graphic element corresponding to each pre-stored electronic price tag image in the electronic price tag image element library;

[0117] The matching degree calculation operation is performed between the target image element and the corresponding pre-stored element, and a matching degree calculation result is obtained, as shown in FIG. 2, including:

[0118] In step 201, based on the position and area proportion of the local graphic element in the target image element, the screening local graphic element corresponding to the region in the same position and with the same area proportion in all pre-stored overall graphic elements is determined.

[0119] In step 202, the second image similarity between the target image element and each screening local graphic element is calculated.

[0120] According to the matching degree calculation result, the recognition result of the target electronic price tag image is generated, including:

[0121] In step 203, the recognition result of the electronic price tag image is generated according to the pre-stored electronic price tag image corresponding to the screening local graphic element with the image similarity to the target image element exceeding a preset threshold.

[0122] In one specific embodiment, the local graphic element is used to represent the local features at different positions in the electronic price tag image. When the target image element is a local graphic element, the pre-stored element is the pre-stored overall graphic element of all pre-stored electronic price tag images. Next, the matching degree calculation is performed between the target image element and the corresponding pre-stored element.

[0123] The matching degree calculation result includes the following aspects: first, according to the position and area proportion of the local graphic element in the target image element, the screening local graphic element corresponding to the region in the same position and with the same area proportion in all pre-stored overall graphic elements is determined. Second, the second image similarity between the target image element and each screening local graphic element is calculated.

[0124] According to the matching degree calculation result, the recognition result of the target electronic price tag image can be generated. Specifically, the recognition result of the electronic price tag image is generated according to the pre-stored electronic price tag image corresponding to the screening local graphic element with the image similarity to the target image element exceeding a preset threshold.

[0125] For example, if the target element is a local graphic element, then according to the selected region of the local image, the same region is extracted from the archive image set in the price tag system for matching calculation, and the similarity calculation method is the same as that used for the overall image. Specifically, the image of the product name region is cut from the target matching price tag image, and then the product name region image of each price tag is extracted from the archive image set in the price tag system for similarity calculation.

[0126] The embodiments of the present disclosure give an example of efficient and reliable similarity calculation of local graphic elements: such as SSD (Structural Similarity Index), SAD (Sum of Absolute Differences) or NCC (Normalized Cross Correlation). Taking NCC as an example, the calculation process is as follows:

[0127] 1. Crop the local area that needs to be matched from the target matching price tag image. Similarly, extract the corresponding local area from the archive image.

[0128] 2. Perform grayscale processing on the two local areas to make them single-channel images.

[0129] 3. Calculate the cross-correlation coefficient of the two local areas to obtain the normalized cross-correlation coefficient NCC.

[0130] 4. The value of NCC ranges from -1 to 1, and the closer the value is to 1, the more similar the two images are. By comparing the NCC values of the target image and the archive image, the best matching archive image can be found.

[0131] After completing the similarity calculation, the results need to be sorted to find the most similar archive image. A sorting-based method can be used, such as ascending or descending order, to facilitate subsequent display and analysis of the matching results.

[0132] Finally, the matching results are displayed to the user, and the corresponding matching result analysis is provided. The matching result display can include similarity scores, matching images, and matching positions, etc. Through these information, users can better understand the similarity between the target image and the archive image, thereby improving the accuracy and efficiency of archive retrieval.

[0133] In one embodiment, the overall image feature element is used to represent the feature map and / or feature vector of the overall electronic price tag image.

[0134] When the target image element is an overall image feature element, the pre-stored element is a pre-stored overall graphic element of all pre-stored electronic price tag images.

[0135] The matching degree calculation operation between the target image element and the corresponding pre-stored element is performed to obtain the matching degree calculation result, as shown in FIG. 3, including:

[0136] Step 301: Convert all pre-stored overall graphic elements into feature maps and / or feature vectors with the same size as the target image element;

[0137] Step 302: Calculate the first feature similarity between the target image element and each feature map and / or feature vector based on the matching calculation algorithm based on the feature map level and / or the feature vector level.

[0138] According to the matching degree calculation result, the recognition result of the target electronic price tag image is generated, including:

[0139] Step 303: According to the pre-stored electronic price tag image corresponding to the feature map and / or feature vector with the first feature similarity of the target image element exceeding the preset threshold, the recognition result of the electronic price tag image is generated.

[0140] In one specific embodiment, the overall image feature element is used to represent the feature map and / or feature vector of the overall electronic price tag image. When the target image element is the overall image feature element, the pre-stored element is the pre-stored overall image element of all pre-stored electronic price tag images.

[0141] In the matching degree calculation link, all pre-stored overall image elements are converted into feature maps and / or feature vectors with the same size as the target image element. Then, the first feature similarity between the target image element and each feature map and / or feature vector is calculated through the matching calculation algorithm based on the feature map level and / or the feature vector level.

[0142] According to the matching degree calculation result, the recognition result of the target electronic price tag image can be generated. Here, the recognition result includes the pre-stored electronic price tag image corresponding to the feature map and / or feature vector with the first feature similarity of the target image element exceeding the preset threshold.

[0143] For example, if the target element is the feature map of the entire image, all the archive image information set in the price tag system is converted into feature maps of the same size, and then matching calculation is performed at the level of the feature map to calculate the similarity between each pair of feature maps, and the calculation method is the same as that used for the entire image. If the target element is the feature vector of the image, all the archive image information set in the price tag system is converted into feature vectors of the same size, and then matching calculation is performed at the level of the feature vector to calculate the cosine similarity or Euclidean distance between each pair of feature vectors.

[0144] In the process of recognizing the electronic price tag image, the overall image feature element plays a key role. They can effectively represent the feature map and / or feature vector of the overall electronic price tag image, thereby providing a basis for subsequent matching degree calculation. In the matching degree calculation process, the pre-stored element plays a reference role, which is the pre-stored overall image element of all pre-stored electronic price tag images. By matching the pre-stored overall image element with the target image element, the matching degree calculation result can be obtained.

[0145] The matching degree calculation result includes converting all the pre-stored integral graphic elements into feature maps and / or feature vectors with the same size as the target image element, and calculating the first feature similarity of the target image element and each feature map and / or feature vector. This process based on the matching calculation algorithm at the feature map level and / or feature vector level can more accurately reflect the similarity between the target image element and the pre-stored element.

[0146] In one embodiment, the local image character feature element is used to represent the character features of the local graphics at different positions in the electronic price tag image, and the position features and area proportion features of the local graphics in the integral electronic price tag image.

[0147] When the target image element is a local image character feature element, the pre-stored element is the bound character information of all the pre-stored electronic price tag images.

[0148] The matching degree calculation operation between the target image element and the corresponding pre-stored element is performed to obtain a matching degree calculation result, which includes:

[0149] The first similarity degree of the target image element and each character information is determined.

[0150] According to the matching degree calculation result, the recognition result of the target electronic price tag image is generated, which includes:

[0151] The character information with a first similarity degree exceeding a preset threshold value to the target image element is taken as the target character information.

[0152] When the position features and area proportion features of the local graphics corresponding to the target character information in the integral electronic price tag image are the same as the position features and area proportion features of the target image element in the target electronic price tag image, the pre-stored electronic price tag image corresponding to the target character information is determined as the recognition result of the electronic price tag image.

[0153] In one specific embodiment, the local image character feature element is used to represent the character features of the local graphics at different positions in the electronic price tag image, and the position features and area proportion features of the local graphics in the integral electronic price tag image. When the target image element is a local image character feature element, the pre-stored element is the bound character information of all the pre-stored electronic price tag images.

[0154] Next, the matching degree calculation operation between the target image element and the corresponding pre-stored element is performed to obtain a matching degree calculation result. This result includes determining the first similarity degree of the target image element and each character information. According to the matching degree calculation result, the recognition result of the target electronic price tag image is generated. This recognition result includes taking the character information with a first similarity degree exceeding a preset threshold value to the target image element as the target character information.

[0155] When the position feature and the area proportion feature of the local pattern corresponding to the target character information in the overall electronic price tag image are the same as the position feature and the area proportion feature of the target image element in the target electronic price tag image, the pre-stored electronic price tag image corresponding to the target character information is determined as the recognition result of the electronic price tag image.

[0156] Taking an example, if the target element is explicit information, the explicit information is iterated and searched with all the archive image information set in the price tag system, to find the archive image with the same explicit information. Specifically, the price tag with the price of “$4.99” and the length of the product name area accounting for “90%” of the overall image is searched.

[0157] The matching degree calculation result is analyzed to determine the best matching character information. The analysis includes comparing the similarity, position feature, and area proportion feature of the target image element and the pre-stored character information. Through these comparisons, the electronic price tag image that best matches the target image can be found.

[0158] In one embodiment, the local image identification feature element is used to represent the position feature, shape feature, area proportion feature, and color feature of the preset pixel block in the overall electronic price tag image, as well as the distance feature between the pixel block and other pixel blocks;

[0159] When the target image element is a local image identification feature element, the pre-stored element is a local image identification feature element of different pre-stored pixel blocks of all pre-stored electronic price tag images;

[0160] The matching degree calculation operation between the target image element and the corresponding pre-stored element is performed to obtain a matching degree calculation result, including:

[0161] The second similarity degree of the position feature, shape feature, area proportion feature, color feature, and distance feature of the target image element and each pixel block is determined;

[0162] According to the matching degree calculation result, the recognition result of the target electronic price tag image is generated, including:

[0163] The pre-stored electronic price tag image corresponding to the pixel block with a second similarity degree of the target image element exceeding a preset threshold is determined as the recognition result of the electronic price tag image.

[0164] In one embodiment, the target image elements of the specified type and / or quantity corresponding to the image element extraction instruction are first extracted from the target electronic price tag image. Then, according to the association between the type and quantity of different image elements and the matching rules of different image elements, the image element matching rules associated with the specified type and / or quantity of target image elements are determined, thereby matching the target image elements of the target electronic price tag image.

[0165] The image element matching rules include the following three aspects:

[0166] 1. Matching degree calculation rule of different types of image elements: according to the pixel block types in the target image elements and the pre-stored electronic price tag image, the matching degree between them is calculated. The matching degree calculation can use similarity indicators such as Euclidean distance, cosine similarity, etc.

[0167] 2. Matching degree weighted calculation rule between different quantities of image elements: for different quantities of pixel blocks in the target image elements and the pre-stored electronic price tag image, the matching degree between them is calculated according to the preset weighting coefficient. In this way, the influence of the quantity of image elements on the matching degree can be fully considered.

[0168] 3. Matching degree calculation sequence of multiple image elements: for the pixel blocks with multiple types in the target image elements and the pre-stored electronic price tag image, the matching degree between them is calculated in the preset order. In this way, it can be ensured that important image elements are considered first in the matching process.

[0169] After determining the target image element matching rules, the matching degree calculation of the local image identification feature elements is performed next. The local image identification feature elements include position features, shape features, area proportion features, color features, and distance features between other pixel blocks. According to the matching degree calculation result, the recognition result of the target electronic price tag image is generated.

[0170] Specifically, for each pixel block in the target image elements and the pre-stored electronic price tag image, the similarity degree in terms of position, shape, area proportion, color, and distance is calculated. The similarity degree of the target image elements and each pixel block is sorted, and the pre-stored electronic price tag image corresponding to the pixel block with a similarity degree exceeding the preset threshold is selected as the recognition result of the electronic price tag image.

[0171] Through the above method, accurate recognition and extraction of electronic price tag images can be achieved. This method has high recognition accuracy and real-time performance, and is suitable for electronic price tag recognition needs in various scenarios. In actual application, the image element matching rules and the calculation method of local image identification feature elements can be adjusted according to specific needs to further improve the recognition effect.

[0172] For example, if the target element is a logo information pixel block, a pixel block of the same size is cropped from each of all the archive image information set in the price tag system, and then the information between the two pixel blocks is identified and matched. Specifically, among all the archive image information set in the price tag system, an archive image in which the distance between two circles in the constellation image is equal is found.

[0173] In an embodiment, a multi-feature matching strategy is used for the matching degree calculation of the local image identification feature elements, including position features, shape features, area ratio features, color features, and distance features between other pixel blocks. The matching degree calculation of these features can use similarity indicators such as Euclidean distance, cosine similarity, etc. By comprehensively considering the matching degree of these features, the target electronic price tag image can be more accurately identified.

[0174] In a specific operation, first, the position, shape, area ratio, color, and distance of the target image element and each pixel block in the pre-stored electronic price tag image are calculated. After the calculation is completed, the similarity degree of the target image element and each pixel block is sorted, and the pre-stored electronic price tag image corresponding to the pixel block with a similarity degree exceeding a preset threshold is selected as the recognition result of the electronic price tag image.

[0175] In addition, in order to improve the recognition accuracy and real-time performance, the image element matching rules and the calculation method of the local image identification feature elements can be adjusted according to specific needs. For example, when calculating the matching degree, different matching degree calculation methods can be used for different types of image elements to fully consider the influence of various factors on the matching degree. At the same time, the preset threshold can also be adjusted according to the actual application scenario to achieve accurate identification and extraction of the electronic price tag image.

[0176] In a specific implementation, when the target image element is multiple, based on the matching degree weighted calculation rule, the matching degree calculation results of each target image element are weighted and calculated to obtain the recognition result of the target electronic price tag image.

[0177] In an embodiment, if the target element is the entire image + local image, the entire image and the local image are compared with the archive image information set in the price tag system at the level of the entire image and the local image, respectively, and then the similarity matching values of the entire image and the local image are weighted and summed to output the final similarity value.

[0178] Similarly, as with other fusion schemes, the final similarity is calculated using the weighted sum method with two or more matching elements.

[0179] In the implementation of the target electronic price tag image recognition process, in order to improve the accuracy and reliability of the recognition, a variety of matching degree weighted calculation rules can be used. Specifically, when the target image element is multiple, the matching degree calculation result of each target image element needs to be weighted and calculated, so as to obtain the recognition result of the target electronic price tag image.

[0180] In this process, the following method can be taken: first, the similarity matching values of the whole image and the local image are calculated respectively; then, the two similarity matching values are weighted and summed to obtain the final similarity value. This weight can be set according to the actual situation, for example, if the similarity matching values of the whole image and the local image are 0.8 and 0.9 respectively, then the weight of the whole image can be set to 0.6 and the weight of the local image can be set to 0.4, so that the recognition effect of the whole image and the local image can be better balanced.

[0181] In addition, similar methods can also be used for other fusion schemes. For example, when two or more matching elements are involved, their matching degrees can be calculated respectively, and then the final similarity can be obtained by using the weighted sum method. This weighted sum method can effectively improve the accuracy and reliability of the electronic price tag image recognition. In summary, by using the matching degree weighted calculation rule in the specific implementation process, the performance of the electronic price tag image recognition can be significantly improved.

[0182] In specific implementation, when the target image element is multiple, based on the matching degree weighted calculation rule, the calculated matching degree calculation result of each target image element is weighted and calculated to obtain the recognition result of the target electronic price tag image.

[0183] In one specific embodiment, when the target image element includes a graphical element and a feature element, the matching degree calculation order corresponding to the type of the target image element is: first calculating the matching degree of the feature element, and then calculating the matching degree of the graphical element.

[0184] As an example, in actual application scenarios, using the whole image or the feature map and feature vector of the whole image to match in the large price tag system set image information library may cause a decrease in matching speed and accuracy due to the number of elements, so local explicit information or landmark information pixel blocks can be used for auxiliary filtering, and then the information of the whole image or the feature map and feature vector can be used as the target element to perform matching calculation in the filtered small image information library. The specific method is as follows:

[0185] 1. Using the price number in the local explicit information, the image in the image library of the system belonging to the price data is screened out, specifically, if the price of "$4.99" is recognized in the price tag of the target recognition matching, only the image with the price of "$4.99" is screened out from the image library to form a small library, and the calculation of the matching similarity is completed in the small library.

[0186] 2. Using the setting sign information, the image in the image library of the system belonging to the sign information range is screened out, specifically, a certain circle coordinate image is set, and the attribute of the corresponding commodity of the price tag is associated and bound, such as being bound as "beverage", when the circle coordinate image recognized in the target recognition matching price tag image is the code, only the image belonging to the attribute is screened out from the image library to form a small library, and the calculation of the matching similarity is completed in the small library. The other sign information is the same as the method.

[0187] After the matching degree of the feature elements and the graphic elements is completed, the target image can be further recognized and classified by using the matching degree calculation result. For example, the target image and the image in the image library can be sorted according to the matching degree, so as to improve the accuracy and speed of recognition.

[0188] In actual application, in order to improve the efficiency and accuracy of matching, a variety of matching strategies can be combined. For example, the local explicit information and the sign information are combined, the local explicit information is used for preliminary screening, and the sign information is used for fine matching, so as to realize the fast and accurate recognition of the target image.

[0189] In addition, the embodiment of the disclosure can also include: when the target image element includes a graphic element and a feature element, the matching degree calculation order corresponding to the type of the target image element is: the matching degree of the graphic element is calculated first, and then the matching degree of the feature element is calculated.

[0190] For example, in actual application, the feature elements of some images may not be obvious or not stable enough, at this time, the matching calculation of the graphic elements can be performed first. For example, in the price tag recognition, if the shape, size, position and other graphic elements of the price tag are relatively stable, the matching calculation of the graphic elements can be performed first, the similar image of the target image graphic element is screened out to form a small library. Then, the matching calculation of the feature elements, such as the price and the name of the commodity, is performed in the small library, so as to obtain a more accurate matching result.

[0191] The selection of this matching strategy can be flexibly adjusted according to the actual application scenario and the characteristics of the image elements. Meanwhile, in order to improve the efficiency and accuracy of the matching, other technical means such as image preprocessing, feature extraction, deep learning, etc. can be combined to further improve the recognition and classification accuracy of the target image.

[0192] The above technical solution of calculating the matching degree of the feature elements first and then calculating the matching degree of the graphic elements is an optional solution of the application, and the disclosure does not limit the calculation order of calculating the matching degree of the feature elements and calculating the matching degree of the graphic elements. In actual application, the order of calculating the matching degree of the feature elements and calculating the matching degree of the graphic elements may vary depending on the specific scenario and requirements. In some cases, it may be necessary to calculate the matching degree of the graphic elements first to quickly filter out images similar to the target image in shape, size, position, etc. to form a smaller candidate image set. Then, the matching degree of the feature elements is calculated in this set to further filter out images more similar to the target image in features.

[0193] This order adjustment can be made according to the specific application requirements and data characteristics. For example, in some scenarios, graphic elements may be more critical because they may directly affect the recognition and classification effect of the image. In other scenarios, feature elements may be more important because they may contain more semantic information and distinguishing degrees.

[0194] In addition, in order to improve the efficiency and accuracy of the matching, a parallel computing method can be used to simultaneously match the feature elements and the graphic elements. This can improve the matching speed while ensuring the matching accuracy, meeting the requirements of actual applications.

[0195] In summary, the disclosure embodiments do not limit the calculation order of calculating the matching degree of the feature elements and calculating the matching degree of the graphic elements, but can be flexibly adjusted according to the actual application scenario and requirements. Meanwhile, by combining other technical means such as image preprocessing, feature extraction, deep learning, etc., the recognition and classification accuracy of the target image can be further improved to meet the requirements of actual applications.

[0196] A specific embodiment is given below to illustrate the specific application of the method of the disclosure, which provides an electronic price tag recognition system and method for detecting price tags on a shelf image, then extracting the matching elements of each price tag, and then performing matching search in a price tag archive image information library to match and identify the information of each price tag.

[0197] The system can include the following components:

[0198] 1. Price tag information management server: records the binding information of electronic price tags and goods, and stores the archive image information actually displayed by the electronic price tags. The price tag information management server can include:

[0199] 1.1. Price tag information database: used for storing and managing the binding information of electronic price tags and goods, including the unique identification of the price tag, the detailed information of the corresponding goods, etc. The database can be expanded and optimized according to business needs to meet the storage and query needs of a large amount of price tag information.

[0200] 1.2. Price tag archive image information library: used for storing the archive image information actually displayed by the electronic price tags, including the background, font, color, size, etc. visual characteristics of the price tag. These information can be used for subsequent price tag recognition and matching process.

[0201] 2. AI vision system: can control the visual sensor device to take pictures. The visual sensor device includes but is not limited to fixedly installed camera, movable camera, rotatable camera, movable robot with camera, aircraft with camera, etc., and the above visual sensor device is to obtain the shelf image by looking down / looking up / facing the goods shelf in the supermarket.

[0202] 3. Image information processing module: can perform AI recognition and inference calculation on visual images, which includes the ability to process shelf images, detect and output position information of price tags on images, and detect, recognize, and convert target matching elements.

[0203] 4. Price tag information matching and recognition module: can receive the matching elements output by the image information processing module, and can also obtain all or part of the price tag archive image information from the price tag information management server, and then use different matching algorithms to match and recognize the target price tag information based on different schemes.

[0204] The steps of the electronic price tag recognition method provided in this embodiment are as follows:

[0205] 1. Obtain the shelf image containing the target price tag. The visual sensor of the vision system obtains the picture of the shelf, wherein the vision system includes but is not limited to a camera fixed in front of the shelf, a movable inspection robot or other devices.

[0206] 2. Detect and determine each target price tag. The shelf image is sent to the price tag detection module to detect the position of the price tag, and the small image of the price tag is cropped according to the position.

[0207] 3. Preprocess the target price tag image. For the cropped price tag image, perform certain image-level processing, such as adjusting brightness, contrast, and sharpness. You can also use deep learning methods to perform super-resolution processing to increase the clarity of the target price tag image.

[0208] 4. Extract the matching elements of the target price tag. They can be defined as follows:

[0209] (1) The entire image (i.e., the overall graphic element mentioned above): The entire cropped image of the target matching price tag is used as the matching element;

[0210] (2) Feature map of the entire image (i.e., the feature map representing the entire electronic price tag image in the above-mentioned overall image feature elements): Use feature descriptors or deep learning algorithm models to perform feature extraction and conversion on the cropped target matching price tag image, extracting and converting a feature map of a certain size as a subsequent matching element. Feature descriptors include but are not limited to SIFT (Scale Invariant Feature Transform), SURF (Speeded Robust Features), ORB (Oriented FAST and Rotated BRIEF), etc.

[0211] (3) Feature vector of the entire image (i.e., the feature vector representing the entire electronic price tag image in the above-mentioned overall image feature element): Use a feature descriptor or a deep learning algorithm model to perform feature extraction on the cropped target matching price tag image, and extract a feature vector value with a certain length as a subsequent matching element;

[0212] (4) Local image (i.e., the local graphic element mentioned above): For the cropped image of the target matching price tag, use the detection and recognition model to detect and identify the local areas in the image that contain representative information, such as the product name area, barcode area, and price area, and finally crop the local image of one or more of these areas as the subsequent matching element;

[0213] (5) Local explicit information (i.e., the local image character feature elements mentioned above): The cropped target matching price tag image is used to detect and identify the representative information in the price tag image using a detection and recognition model, such as the price, the price digits such as "￥5.59" or "$4.99", or other relatively obvious characters, or the percentage of the product name length in the entire image, etc.

[0214] (6) Mark information pixel block (i.e. the above-mentioned local image identification feature element): the price tag system sets the archive image, and some pixel points or pixel blocks that can be used as specific identification can be set, such as setting a variety of circle constellation image encoding, small area character code, small area data matrix, small area QR code. And these identified pixels or pixel blocks can be separated and distinguished by using different colors, and finally serve as target elements. For the image of the target matching price tag cut out, the mark information recognition algorithm model is used to detect or identify the mark information pixel block, and finally serve as the matching element.

[0215] In actual process, one of the above can be selected, or several can be used in combination, such as whole image + local image, local explicit information + mark information pixel block, whole image + mark information pixel block, etc.

[0216] 5. The target matching element is sent to the price tag matching recognition module for matching calculation. Among them, for different target matching elements, different similarity calculation, searching and filtering methods can be set:

[0217] (1) Scheme one: using a single matching element: for different target elements (whole image, feature map, feature vector, local image, explicit information, mark information pixel block), different matching methods are adopted. For example, the matching of the whole image can use square difference matching, normalized square difference matching, cross-correlation matching, etc.; the matching of the feature map and the feature vector can be similarity calculation at the feature map or feature vector level; the matching of the local image is similarity calculation according to the selected area.

[0218] ① If the target element is the whole image, it is compared with all the archive image information set in the price tag system by circulating traversal, and the similarity between them is calculated. Common similarity measurement methods include square difference matching (SSD, Sum of Squared Differences), normalized square difference matching (NSSD, Normalized Sum of Squared Differences), cross-correlation matching (Cross-Correlation), etc.

[0219] ② If the target element is the feature map of the whole image, all the archive image information set in the price tag system is converted into the same size feature map, and then the matching calculation is carried out at the feature map level, the similarity of each two feature maps is calculated, and the calculation method is the same as that used for the whole image.

[0220] ③ If the target element is a feature vector of an image, all the archive image information set in the price tag system is traversed and converted into the same size feature vector, and then the matching calculation is performed at the level of the feature vector, and the cosine similarity or Euclidean distance between each pair of feature vectors is calculated.

[0221] ④ If the target element is a local image, according to the selected area of the local image, the same area is intercepted from the archive image set in the price tag system for matching calculation, and the similarity calculation method is the same as that used for the whole image. Specifically, the image of the product name area is cut out from the target matching price tag image, and then the product name area image of each price tag is intercepted from the archive image set in the price tag system for similarity calculation.

[0222] ⑤ If the target element is explicit information, it is traversed and searched with the explicit information bound to all the archive image information set in the price tag system to find the archive image with the same explicit information. Specifically, the price tag with the price of “$4.99” and the product name area length accounting for “90%” of the whole image is searched.

[0223] ⑥ If the target element is a logo information pixel block, it is cut into the same size pixel block from each same area of all the archive image information set in the price tag system, and then the information between the two pixel blocks is identified and matched. Specifically, among all the archive image information set in the price tag system, the archive image with the equal distance between each pair of circles in the circle constellation image is found.

[0224] (2) Scheme two: using fused matching elements: combining two or more matching elements, such as the whole image and the local image, the local explicit information and the logo information pixel block, and performing weighted sum similarity calculation.

[0225] If the target element is the whole image + local image, the whole image and the local image are compared with the archive image information set in the price tag system at the level of the whole image and the local image, respectively, and then the similarity matching values of the whole image and the local image are weighted and summed to output the final similarity value.

[0226] Similarly, other fusion schemes are the same, and the final similarity is calculated using the weighted sum method with two or more matching elements.

[0227] (3) Scheme three: using different matching elements in sequence: in actual application, local explicit information or logo information pixel block can be used for auxiliary filtering, and then matching calculation is performed in the filtered small archive image library. For example, the archive image of the corresponding price is first selected according to the price number, and then the archive image of the specific attribute is selected according to the logo information.

[0228] In practical application scenarios, using the entire image or the feature map and feature vector of the entire image to match in the huge price tag system set archive image information library can cause the matching speed and accuracy to decrease due to the quantity problem, so the local explicit information or landmark information pixel block can be used for auxiliary filtering, and then the information of the entire image or the feature map and feature vector is used as the target element to perform matching calculation in the filtered small archive image information library. The specific method is as follows:

[0229] ①The price data in the local explicit information is used to filter out the images belonging to the price data in the archive image library in the system. Specifically, if the price "$4.99" is recognized in the target recognition and matching price tag, only the archive images with the price "$4.99" are filtered out from the archive image library to form a small library, and the matching similarity calculation is completed in the small library.

[0230] ②The set landmark information is used to filter out the images belonging to the landmark information range in the archive image library in the system. Specifically, a certain circle coordinate image is set to be associated with the attributes of the corresponding commodity of the price tag, such as being associated with "beverage class". When the circle coordinate image recognized in the target recognition and matching price tag image is the code, only the archive images belonging to the attribute are filtered out from the archive image library to form a small library, and the matching similarity calculation is completed in the small library. Other landmark information can be processed in the same way.

[0231] 7. According to the matching search result, the result of the price tag matching recognition is determined:

[0232] Using different matching elements or logical schemes, the highest similarity or the highest credibility value can be taken as the final matching result according to the similarity or the search result. At the same time, multiple matching elements and logical schemes can be combined to improve the accuracy and stability of the recognition.

[0233] Of course, it can be understood that the above detailed process can also have other variations, and the related variations should fall within the protection scope of the present disclosure.

[0234] In the embodiments of the present disclosure, based on the electronic shelf image collected by the visual sensor device, image recognition and cropping are performed to obtain a plurality of electronic price tag images in the electronic shelf image; based on an image element extraction instruction, target image elements of a specified type and / or quantity corresponding to the image element extraction instruction are extracted from a target electronic price tag image; the image elements include graphic elements and feature elements of the image; the graphic elements include overall graphic elements and partial graphic elements of the electronic price tag image; the feature elements include overall image feature elements, partial image character feature elements, and partial image identification feature elements; based on the association between the type and quantity of different image elements and different image element matching rules, the image element matching rules associated with the specified type and / or quantity of the target image elements are determined as the target image element matching rules of the target electronic price tag image; the image element matching rules include matching degree calculation rules of different types of image elements and matching degree calculation sequences of a plurality of image elements; for the target image elements, the matching degree calculation rules and the matching degree calculation sequences corresponding to the type of the target image elements are used to perform matching degree calculation operations between the target image elements and corresponding pre-stored elements to obtain matching degree calculation results; the pre-stored elements are image elements of each pre-stored electronic price tag image in a pre-set electronic price tag image element library; and based on the matching degree calculation results, an identification result of the target electronic price tag image is generated, different types and quantities of image elements of the electronic price tag image can be extracted, and different image element matching rules are set for different image elements and combinations thereof, thereby realizing diversified matching of the image elements of the electronic price tag. Compared with the scheme in the prior art in which only the image similarity between the electronic price tag image and the pre-stored electronic price tag image can be calculated for identification, the scheme avoids the problem that the electronic price tag image cannot be recognized due to unclearness, can still realize accurate recognition of the electronic price tag in the case of poor image quality, improves the accuracy and efficiency of electronic price tag recognition, and at the same time, avoids the problem that the cost of electronic price tag recognition is increased due to the need for a high-definition shooting system for OCR recognition in the prior art, and reduces the equipment cost of electronic price tag recognition.

[0235] The present disclosure is innovative in that a systematic solution is proposed, which fully utilizes the archive image data saved in the electronic price tag information server to perform matching recognition on the electronic price tag. At the same time, a design and extraction method of different matching elements, and a similarity calculation method and scheme based on different matching elements are proposed. The embodiments of the present disclosure can reduce the demand and cost of the shooting system, and the scheme of the present patent application does not require very high-definition images, unlike the method of obtaining and recognizing the identification result of the target price tag by OCR, which requires high-definition images.

[0236] The embodiments of the present disclosure also provide an electronic price tag recognition device, as described in the following embodiments. Since the principle of solving the problem of the device is similar to that of the electronic price tag recognition method, the implementation of the device can refer to the implementation of the electronic price tag recognition method, and the repeated parts will not be described here.

[0237] The embodiments of the present disclosure also provide an electronic price tag recognition device, which is used to improve the accuracy and efficiency of electronic price tag recognition and reduce the equipment cost of electronic price tag recognition. As shown in FIG. 4, the device comprises:

[0238] An electronic price tag image recognition and clipping module 401 is configured to perform image recognition and clipping on the electronic shelf image collected based on the visual sensor device, so as to obtain a plurality of electronic price tag images in the electronic shelf image.

[0239] A target image element extraction module 402 is configured to extract, based on an image element extraction instruction, a target image element of a specified type and / or quantity corresponding to the image element extraction instruction from a target electronic price tag image. The image element includes a graphic element and a feature element of the image. The graphic element includes an overall graphic element and a local graphic element of the electronic price tag image. The feature element includes an overall image feature element, a local image character feature element, and a local image identification feature element.

[0240] A target image element matching rule determination module 403 is configured to determine, based on an association relationship between the type and quantity of different image elements and different image element matching rules, an image element matching rule associated with the specified type and / or quantity of the target image element as a target image element matching rule of the target electronic price tag image. The image element matching rule includes a matching degree calculation rule of different types of image elements and a matching degree calculation order of a plurality of image elements.

[0241] A matching degree calculation module 404 is configured to, for a target image element, perform a matching degree calculation operation between the target image element and a corresponding pre-stored element based on the matching degree calculation rule and the matching degree calculation order corresponding to the type of the target image element, so as to obtain a matching degree calculation result. The pre-stored element is an image element of each pre-stored electronic price tag image in a pre-stored electronic price tag image element library.

[0242] An identification result generation module 405 is configured to generate an identification result of the target electronic price tag image according to the matching degree calculation result.

[0243] In one embodiment, when the target image element is an overall graphic element, the pre-stored element is a pre-stored overall graphic element of all pre-stored electronic price tag images.

[0244] performing a matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain a matching degree calculation result, including:

[0245] calculating a first image similarity between the target image element and each pre-stored overall graphic element based on a square difference matching algorithm, a normalized square difference matching algorithm, and / or a cross-correlation matching algorithm;

[0246] generating a recognition result of the target electronic price tag image according to the matching degree calculation result, including:

[0247] generating the recognition result of the electronic price tag image according to a pre-stored electronic price tag image corresponding to a pre-stored overall graphic element whose first image similarity with the target image element exceeds a preset threshold.

[0248] In an embodiment, the local graphic element is used to represent a local graphic in different positions in the electronic price tag image.

[0249] When the target image element is a local graphic element, the pre-stored element is a pre-stored overall graphic element of all pre-stored electronic price tag images.

[0250] performing a matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain a matching degree calculation result, including:

[0251] determining a screening local graphic element corresponding to a region in all pre-stored overall graphic elements that is located at the same position and has the same area proportion based on the position and area proportion of the local graphic element in the target image element;

[0252] calculating a second image similarity between the target image element and each screening local graphic element;

[0253] generating a recognition result of the target electronic price tag image according to the matching degree calculation result, including:

[0254] generating the recognition result of the electronic price tag image according to a pre-stored electronic price tag image corresponding to a screening local graphic element whose image similarity with the target image element exceeds a preset threshold.

[0255] In an embodiment, the overall image feature element is used to represent a feature map and / or a feature vector of the overall electronic price tag image.

[0256] When the target image element is an overall image feature element, the pre-stored element is a pre-stored overall graphic element of all pre-stored electronic price tag images.

[0257] performing a matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain a matching degree calculation result, including:

[0258] convert all the pre-stored whole image elements into feature maps and / or feature vectors with the same size as the target image element;

[0259] calculate the first feature similarity between the target image element and each feature map and / or feature vector based on a matching calculation algorithm based on the feature map level and / or the feature vector level;

[0260] generate the recognition result of the target electronic price tag image according to the matching degree calculation result, including:

[0261] generate the recognition result of the electronic price tag image according to the pre-stored electronic price tag image corresponding to the feature map and / or feature vector whose first feature similarity with the target image element exceeds the preset threshold.

[0262] In one embodiment, the local image character feature element is used to represent the character features of the local image in different positions of the electronic price tag image, and the position features and area proportion features of the local image in the whole electronic price tag image.

[0263] When the target image element is a local image character feature element, the pre-stored element is the bound character information of all pre-stored electronic price tag images.

[0264] perform the matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain the matching degree calculation result, including:

[0265] determine the first similarity degree between the target image element and each character information;

[0266] generate the recognition result of the target electronic price tag image according to the matching degree calculation result, including:

[0267] determine the character information whose first similarity degree with the target image element exceeds the preset threshold as the target character information.

[0268] When the position features and area proportion features of the local image corresponding to the target character information in the whole electronic price tag image are the same as the position features and area proportion features of the target image element in the target electronic price tag image, determine the pre-stored electronic price tag image corresponding to the target character information as the recognition result of the electronic price tag image.

[0269] In one embodiment, the local image identification feature element is used to represent the position features, shape features, area proportion features and color features of the preset pixel block in the whole electronic price tag image, as well as the distance features between other pixel blocks.

[0270] When the target image element is a local image identification feature element, the pre-stored element is the local image identification feature element of different pre-stored pixel blocks of all pre-stored electronic price tag images.

[0271] performing a matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain a matching degree calculation result, including:

[0272] determining a second similarity degree of the target image element and the position feature, the shape feature, the area proportion feature, the color feature and the distance feature of each pixel block;

[0273] generating a recognition result of the target electronic price tag image according to the matching degree calculation result, including:

[0274] determining the pre-stored electronic price tag image corresponding to the pixel block with the second similarity degree of the target image element exceeding the preset threshold as the recognition result of the electronic price tag image.

[0275] In an embodiment, when the target image element includes a graphic element and a feature element, the matching degree calculation sequence corresponding to the category of the target image element is: first calculating the matching degree of the feature element, and then calculating the matching degree of the graphic element.

[0276] In an embodiment, the image element matching rule further includes: a matching degree weighted calculation rule between different numbers of image elements;

[0277] generating a recognition result of the target electronic price tag image according to the matching degree calculation result, including:

[0278] When the target image element is multiple, based on the matching degree weighted calculation rule, the matching degree calculation result of each target image element calculated is weighted calculated to obtain the recognition result of the target electronic price tag image.

[0279] Embodiments of the present disclosure provide an embodiment of a computer device for implementing all or part of the above-mentioned electronic price tag recognition method. The computer device specifically includes the following content:

[0280] a processor, a memory, a communications interface and a bus; wherein the processor, the memory and the communications interface complete mutual communication through the bus; the communications interface is used for realizing information transmission between related devices; the computer device can be a desktop computer, a tablet computer and a mobile terminal, etc., and the present embodiment is not limited thereto. In the present embodiment, the computer device can be implemented by referring to the embodiment of the electronic price tag recognition method and the embodiment of the electronic price tag recognition device, the contents of which are incorporated herein, and repeated descriptions are not repeated.

[0281] FIG. 5 is a schematic block diagram of a system configuration of the computer device 1000 according to an embodiment of the present application. As shown in FIG. 5, the computer device 1000 can include a central processor 1001 and a memory 1002; the memory 1002 is coupled to the central processor 1001. It is noted that FIG. 5 is exemplary; other types of configurations can also be used to supplement or replace the configuration to implement telecommunication functions or other functions.

[0282] In an embodiment, the electronic price tag recognition function can be integrated into the central processor 1001. The central processor 1001 can be configured to perform the following control:

[0283] perform image recognition and cropping on the electronic shelf image collected based on the visual sensor device to obtain a plurality of electronic price tag images in the electronic shelf image;

[0284] extract, based on the image element extraction instruction, a specified type and / or quantity of target image elements corresponding to the image element extraction instruction from the target electronic price tag image; the image elements include graphic elements and feature elements of the image; the graphic elements include overall graphic elements and local graphic elements of the electronic price tag image; the feature elements include overall image feature elements, local image character feature elements, and local image identification feature elements;

[0285] based on a pre-set association between different types and quantities of image elements and different image element matching rules, determine the image element matching rule associated with the specified type and / or quantity of target image elements as the target image element matching rule of the target electronic price tag image; the image element matching rule includes a matching degree calculation rule of different types of image elements and a matching degree calculation order of a plurality of image elements;

[0286] for the target image element, perform a matching degree calculation operation between the target image element and a corresponding pre-stored element based on the matching degree calculation rule and the matching degree calculation order corresponding to the type of the target image element, to obtain a matching degree calculation result; the pre-stored element is an image element of each pre-stored electronic price tag image in a pre-stored electronic price tag image element library;

[0287] generate the recognition result of the target electronic price tag image according to the matching degree calculation result.

[0288] In another embodiment, the electronic price tag recognition device can be configured separately from the central processor 1001, for example, the electronic price tag recognition device can be configured as a chip connected to the central processor 1001 to implement the electronic price tag recognition function through the control of the central processor.

[0289] As shown in FIG. 5, the computer device 1000 can also include a communication module 1003, an input unit 1004, an audio processor 1005, a display 1006, and a power supply 1007. It is to be noted that the computer device 1000 does not necessarily include all the components shown in FIG. 5, and can include components not shown in FIG. 5, as known in the art.

[0290] As shown in FIG. 5, the central processing unit 1001, which is also sometimes referred to as a controller or operating control, can include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of the various components of the computer device 1000.

[0291] The memory 1002, for example, can be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. Information relating to failures can be stored, and programs for executing the information can also be stored. The central processing unit 1001 can execute the programs stored in the memory 1002 to achieve information storage or processing, etc.

[0292] The input unit 1004 provides input to the central processing unit 1001. The input unit 1004 is, for example, a key or touch input device. The power supply 1007 is used to provide power to the computer device 1000. The display 1006 is used to display display objects such as images and text. The display can be, for example, an LCD display, but is not limited thereto.

[0293] The memory 1002 can be a solid state memory, such as a read only memory (ROM), a random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROM, etc. The memory 1002 can also be some other type of device. The memory 1002 includes a buffer memory 1021 (sometimes referred to as a buffer). The memory 1002 can include an application / function storage section 1022 for storing application programs and function programs or for executing the flow of the operation of the computer device 1000 by the central processing unit 1001.

[0294] The memory 1002 can also include a data storage section 1023 for storing data such as contacts, digital data, pictures, sounds, and / or any other data used by the computer device. A driver storage section 1024 of the memory 1002 can include various drivers of the computer device for communication functions and / or for executing other functions of the computer device, such as a messaging application, a contact application, etc.

[0295] The communication module 1003 is a transmitter / receiver 1003 that transmits and receives signals via the antenna 1008. The communication module (transmitter / receiver) 1003 is coupled to the central processor 1001 to provide input signals and receive output signals, as is the case with conventional mobile communication terminals.

[0296] Based on different communication technologies, a plurality of communication modules 1003 can be provided in the same computer device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module, etc. The communication module (transmitter / receiver) 1003 is also coupled to the speaker 1009 and the microphone 1010 via the audio processor 1005 to provide audio output via the speaker 1009 and to receive audio input from the microphone 1010, thereby enabling the usual telecommunication functions. The audio processor 1005 can include any suitable buffers, decoders, amplifiers, etc. In addition, the audio processor 1005 is coupled to the central processor 1001, thereby enabling recording on the local device via the microphone 1010 and enabling playing of stored sounds via the speaker 1009.

[0297] The embodiment of the present disclosure further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the electronic price tag identification method.

[0298] The embodiment of the present disclosure further provides a computer program product, which comprises a computer program. The computer program is executed by a processor to implement the electronic price tag identification method.

[0299] In the embodiments of the present disclosure, based on the electronic shelf image collected by the visual sensor device, image recognition and cropping are performed to obtain a plurality of electronic price tag images in the electronic shelf image; based on an image element extraction instruction, target image elements of a specified type and / or quantity corresponding to the image element extraction instruction are extracted from a target electronic price tag image; the image elements include graphic elements and feature elements of the image; the graphic elements include overall graphic elements and partial graphic elements of the electronic price tag image; the feature elements include overall image feature elements, partial image character feature elements, and partial image identification feature elements; based on the association between the type and quantity of different image elements and different image element matching rules, the image element matching rules associated with the specified type and / or quantity of the target image elements are determined as the target image element matching rules of the target electronic price tag image; the image element matching rules include matching degree calculation rules of different types of image elements and matching degree calculation sequences of a plurality of image elements; for the target image elements, the matching degree calculation rules and the matching degree calculation sequences corresponding to the type of the target image elements are used to perform matching degree calculation operations between the target image elements and corresponding pre-stored elements to obtain matching degree calculation results; the pre-stored elements are image elements of each pre-stored electronic price tag image in a pre-set electronic price tag image element library; and based on the matching degree calculation results, an identification result of the target electronic price tag image is generated, different types and quantities of image elements of the electronic price tag image can be extracted, and different image element matching rules are set for different image elements and combinations thereof, thereby realizing diversified matching of the image elements of the electronic price tag. Compared with the prior art scheme that can only calculate the image similarity between the electronic price tag image and the pre-stored electronic price tag image for identification, the present disclosure avoids the problem that the electronic price tag image cannot be recognized due to unclearness, can still realize accurate recognition of the electronic price tag in the case of poor image quality, improves the accuracy and efficiency of electronic price tag recognition, and at the same time, avoids the problem that the cost of electronic price tag recognition increases due to the need for a high-definition shooting system for OCR recognition in the prior art, and reduces the equipment cost of electronic price tag recognition.

[0300] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects.

[0301] In particular, the systems, modules, or apparatuses described herein can be implemented by one or more application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), or the like. Further, the systems, modules, or apparatuses can also be implemented by one or more computer usable storage mediums containing computer usable program code, which instructions when executed by a processor, can implement the specified functions. For example, the systems, modules, or apparatuses can include, but are not limited to, one or more processing units, memories, and communication interfaces to connect different components. The memories can include volatile and non-volatile storage mediums, which store a series of instructions for causing the one or more processing units to perform the operations described in the present disclosure. When executed, the instructions cause the processing units to perform necessary calculations, data processing, and control operations, such as image recognition and cropping of electronic shelf images based on visual sensor devices, extracting target image elements of a specified category and / or quantity corresponding to image element extraction instructions from a target electronic price tag image, performing matching degree calculation based on pre-set different image element matching rules, and generating a recognition result of the target electronic price tag image based on the matching degree calculation result.

[0302] Moreover, the present disclosure can take the form of a computer program product on one or more computer usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) embodying computer readable program code.

[0303] The present disclosure is described in reference to the flowchart and / or block diagrams of the methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each flow and / or block in the flowchart and / or block diagrams, as well as a combination of flows and / or blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus create means for implementing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0304] These computer program instructions can also be stored in a computer readable storage medium that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a manufactured product including instruction means, which implements the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0305] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks.

[0306] The above specific embodiments have further described the purposes, technical solutions and beneficial effects of the present disclosure. It should be understood that the above are merely specific embodiments of the present disclosure and are not intended to limit the protection scope of the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An electronic price label recognition method, characterized in that: include: Performing image recognition and cropping on the electronic shelf image captured by the visual sensor device to obtain multiple electronic price tag images in the electronic shelf image; Based on the image element extraction instruction, target image elements of a specified type and / or quantity corresponding to the image element extraction instruction are extracted from the target electronic price tag image; the image elements include graphic elements and feature elements of the image; the graphic elements include overall graphic elements and local graphic elements of the electronic price tag image; the feature elements include overall image feature elements, local image character feature elements, and local image logo feature elements; Based on the pre-set association relationship between the types and quantities of different image elements and different image element matching rules, the image element matching rule associated with the specified type and / or quantity of the target image element is determined as the target image element matching rule for the target electronic price tag image; the image element matching rule includes: a matching degree calculation rule for different types of image elements, and a matching degree calculation order for multiple image elements; For a target image element, a matching degree calculation operation is performed between the target image element and a corresponding pre-stored element according to a matching degree calculation rule and matching degree calculation sequence corresponding to the type of the target image element, to obtain a matching degree calculation result; the pre-stored element is an image element of each pre-stored electronic price tag image in a preset electronic price tag image element library; A recognition result of the target electronic price label image is generated according to the matching degree calculation result.

2. The method according to claim 1, wherein When the target image element is an integral graphic element, the pre-stored element is the pre-stored integral graphic element of all pre-stored electronic price label images; Performing a matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain a matching degree calculation result includes: Calculating the first image similarity between the target image element and each pre-stored overall graphic element based on a square difference matching algorithm, a normalized square difference matching algorithm and / or a cross-correlation matching algorithm; Generating a recognition result of the target electronic price tag image based on the matching degree calculation result, including: A recognition result of the electronic price tag image is generated according to a pre-stored electronic price tag image corresponding to a pre-stored overall graphic element whose first image similarity with the target image element exceeds a preset threshold.

3. The method according to claim 1, wherein The local graphic elements are used to represent local graphics at different positions in the electronic price label image; When the target image element is a local graphic element, the pre-stored element is a pre-stored overall graphic element of the entire pre-stored electronic price label image; Performing a matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain a matching degree calculation result includes: Based on the position and area ratio of the local graphic element in the target image element, determining the filtered local graphic elements corresponding to the areas located at the same position and with the same area ratio in all pre-stored overall graphic elements; Calculating the second image similarity between the target image element and each screened local graphic element; Generating a recognition result of the target electronic price tag image based on the matching degree calculation result, including: The recognition result of the electronic price tag image is generated according to the pre-stored electronic price tag images corresponding to the selected local graphic elements whose image similarity with the target image element exceeds a preset threshold.

4. The method according to claim 1, wherein The overall image feature element is used to represent the feature map and / or feature vector of the overall electronic price label image; When the target image element is an overall image feature element, the pre-stored element is a pre-stored overall graphic element of all pre-stored electronic price label images; Performing a matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain a matching degree calculation result includes: Converting all pre-stored overall graphic elements into feature maps and / or feature vectors of the same size as the target image elements; Calculating a first feature similarity between the target image element and each feature map and / or feature vector using a matching calculation algorithm based on a feature map level and / or a feature vector level; Generating a recognition result of the target electronic price tag image based on the matching degree calculation result, including: A recognition result of the electronic price label image is generated based on the pre-stored electronic price label image corresponding to the feature map and / or feature vector whose similarity with the first feature of the target image element exceeds a preset threshold.

5. The method according to claim 1, wherein The local image character feature elements are used to characterize the character features of local graphics at different positions in the electronic price label image, and the position features and area ratio features of the local graphics in the overall electronic price label image; When the target image element is a local image character feature element, the pre-stored element is the character information bound to all pre-stored electronic price label images; Performing a matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain a matching degree calculation result includes: determining a first similarity between the target image element and each character information; Generating a recognition result of the target electronic price tag image based on the matching degree calculation result, including: Taking the character information whose first similarity with the target image element exceeds a preset threshold as the target character information; If the position characteristics and area ratio characteristics of the partial graphic corresponding to the target character information in the overall electronic price tag image are the same as the position characteristics and area ratio characteristics of the target image element in the target electronic price tag image, the pre-stored electronic price tag image corresponding to the target character information is determined as the recognition result of the electronic price tag image.

6. The method according to claim 1, wherein The local image identification feature element is used to characterize the position characteristics, shape characteristics, area ratio characteristics and color characteristics of the preset pixel block in the overall electronic price tag image, as well as the distance characteristics between the pixel block and other pixel blocks; When the target image element is a local image identification feature element, the pre-stored element is a local image identification feature element of different pre-stored pixel blocks of all pre-stored electronic price label images; Performing a matching degree calculation operation between the target image element and the corresponding pre-stored element to obtain a matching degree calculation result includes: Determining a second similarity between the target image element and each pixel block in terms of position feature, shape feature, area ratio feature, color feature, and distance feature; Generating a recognition result of the target electronic price tag image based on the matching degree calculation result, including: The pre-stored electronic price tag image corresponding to the pixel block whose second similarity with the target image element exceeds a preset threshold is determined as the recognition result of the electronic price tag image.

7. The method according to claim 1, wherein When the target image elements include graphic elements and feature elements, the order of calculating the matching degrees corresponding to the types of the target image elements is: first calculating the matching degree of the feature elements, then calculating the matching degree of the graphic elements.

8. The method according to claim 1, wherein The image element matching rules also include: a weighted calculation rule for the matching degree between different numbers of image elements; Generating a recognition result of the target electronic price tag image based on the matching degree calculation result, including: When there are multiple target image elements, the matching degree calculation results of each target image element are weightedly calculated based on the matching degree weighted calculation rule to obtain the recognition result of the target electronic price label image.

9. An electronic price label recognition device, characterized in that: include: An electronic price label image recognition and cropping module is used to perform image recognition and cropping on the electronic shelf image collected by the visual sensor device to obtain multiple electronic price label images in the electronic shelf image; A target image element extraction module is configured to extract, based on the image element extraction instruction, target image elements of a specified type and / or quantity corresponding to the image element extraction instruction from the target electronic price tag image; the image elements include graphic elements and feature elements of the image; the graphic elements include overall graphic elements and local graphic elements of the electronic price tag image; the feature elements include overall image feature elements, local image character feature elements, and local image logo feature elements; a target image element matching rule determination module, configured to determine, based on pre-set associations between the types and quantities of different image elements and the different image element matching rules, an image element matching rule associated with the specified type and / or quantity of the target image element as the target image element matching rule for the target electronic price tag image; the image element matching rule comprising: a matching degree calculation rule for different types of image elements, and a matching degree calculation order for multiple image elements; a matching degree calculation module for performing a matching degree calculation operation between a target image element and a corresponding pre-stored element in accordance with a matching degree calculation rule and matching degree calculation sequence corresponding to the type of the target image element, thereby obtaining a matching degree calculation result; the pre-stored element being an image element of each pre-stored electronic price tag image in a preset electronic price tag image element library; The recognition result generating module is used to generate a recognition result of the target electronic price label image according to the matching degree calculation result.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images