Commodity detection method and device and computer storage medium

Abstract

The invention provides a commodity detection method and apparatus, and a computer storage medium. The commodity detection method comprises the steps of obtaining a commodity detection image of a preset commodity area; identifying a target commodity in the commodity detection image and all price tags; matching all the price tags by using the commodity identification name of the target commodity; and when all the price tags fail to match, outputting the unclear pricing information of the target commodity. According to the mode, the commodity information and the price information are collected, processed and matched in real time, and the situation that the commodity price is abnormal, the price tag is missing or the commodity is inconsistent with the price tag can be recognized.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a commodity detection method, device, and computer storage medium. Background Art

[0002] With the rapid development of image processing technology and artificial intelligence technology, a method of using OCR (optical character recognition) and object recognition technology to monitor stall products and prices in real time has emerged in sales scenarios. However, the product price prediction in the existing technology is static data collection and analysis, which has poor timeliness. Summary of the Invention

[0003] To solve the above technical problems, the present application proposes a commodity detection method, which includes: obtaining a commodity detection image of a preset commodity area; identifying the target commodity and all price tags in the commodity detection image; matching all the price tags using the commodity identification name of the target commodity; and when all the price tags fail to match, outputting the uncoded price information of the target commodity.

[0004] Among them, after using the product identification name of the target product to match all the price tags, the product detection method also includes: when the product identification name fails to match all the price tags, obtaining the product synonym name of the product identification name; using the product synonym name of the target product to match all the price tags.

[0005] The commodity detection method further includes: when the commodity synonym name successfully matches the price tag, outputting commodity name modification information of the target commodity based on the commodity synonym and the commodity identification name.

[0006] Among them, the product detection method also includes: when all the price tags fail to match, obtaining the character difference between the product identification name and the input words in all the price tags; when the maximum character difference exceeds the price matching threshold, obtaining the price tag corresponding to the maximum character difference as the matching result of the target product.

[0007] Among them, the product detection method also includes: obtaining the position matching degree between the target product and all the price tags; generating a price matching degree based on the position matching degree and the character difference degree; when the maximum price matching degree exceeds the price matching threshold, obtaining the price tag corresponding to the maximum price matching degree as the matching result of the target product.

[0008] After obtaining the price tag corresponding to the maximum price matching degree as the matching result of the target product, the product detection method further includes: outputting product name modification information of the target product based on the price tag corresponding to the maximum price matching degree.

[0009] Among them, the commodity detection method also includes: obtaining the prices of several commodities that are successfully matched with the target commodity in different areas; obtaining the average price of the several commodity prices; judging whether the difference between each commodity price and the average price exceeds the difference threshold; if so, outputting information that the target commodity is abnormally priced in the area corresponding to the commodity price.

[0010] Among them, the commodity detection method also includes: obtaining the pre-marked price of the target commodity; judging whether the difference between the actual marked price of the price tag successfully matched with the target commodity and the pre-marked price exceeds a difference threshold; if so, outputting information that the target commodity price is abnormal.

[0011] To solve the above technical problems, the present application proposes a commodity detection device, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above commodity detection method.

[0012] In order to solve the above technical problems, the present application proposes a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the above commodity detection method.

[0013] Different from the prior art, the present invention has the following advantages: the commodity detection device obtains a commodity detection image of a preset commodity area; identifies the target commodity and all price tags in the commodity detection image; matches all price tags using the commodity identification name of the target commodity; and outputs the unmarked price information of the target commodity when all price tags fail to match. Through the above method, by collecting, processing, and matching commodity and price information in real time, it is possible to identify abnormal commodity prices, missing price tags, or inconsistencies between commodities and price tags. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 This is a flow chart of the first embodiment of the commodity detection method provided by this application;

[0016] Figure 2 This is a flow chart of a second embodiment of the commodity detection method provided by this application;

[0017] Figure 3 This is a flow chart of the third embodiment of the commodity detection method provided by this application;

[0018] Figure 4 This is a structural diagram of an embodiment of a commodity detection device provided by the present application;

[0019] Figure 5 It is a structural diagram of an embodiment of a computer storage medium provided by this application. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0021] The commodity detection method of the present application is applied to a commodity detection device, wherein the commodity detection device of the present application can be a server or a system comprising a server and a local terminal. Accordingly, the various components of the commodity detection device, such as the various units, subunits, modules, and submodules, can be all provided in the server or separately provided in the server and the local terminal.

[0022] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules for providing distributed servers, or it can be implemented as a single software or software module, which is not specifically limited here. In some possible implementations, the commodity detection method of the embodiment of the present application can be implemented by a processor calling computer-readable instructions stored in a memory.

[0023] In order to solve the above technical problems, this application proposes a commodity detection method. In this embodiment, the commodity detection method is applied to a commodity detection device. Figure 1 , Figure 1 It is a flow chart of the first embodiment of the commodity detection method provided by this application.

[0024] like Figure 1 As shown, the specific steps are as follows:

[0025] Step S11: Acquire a commodity detection image of a preset commodity area.

[0026] Specifically, a video of a preset commodity area is obtained, and a commodity detection image of the preset commodity is obtained based on the video.

[0027] In an embodiment of the present application, the preset commodity area can be a farmers' market, a shopping mall, a supermarket, etc., and multiple consecutive frames of images can be obtained from the video for processing as detection data. Video frames can also be captured from the video according to timestamps as detection data.

[0028] In the embodiments of the present application, any image recognition and image segmentation method can be used to obtain commodity detection images from the commodity area, and the present application does not limit the specific image detection algorithm.

[0029] Step S12: Identify the target product and all price tags in the product detection image.

[0030] Specifically, the commodity detection device uses application object recognition technology to identify commodities in real time, including but not limited to commodity name, specifications and other information.

[0031] Step S13: Match all the price tags using the commodity identification name of the target commodity.

[0032] Based on the recognized product name, the characters in the price tag are filtered and regressed to find the price information that matches the product.

[0033] Through the defined matching process, price information is matched with product information, and character recognition is performed on the extracted image using OCR technology. The extracted price information is associated with the product type.

[0034] Optical Character Recognition (OCR) is a technology that converts text in images into editable text. It scans documents, photos, or other images, recognizes the characters in them, and converts them into a text format that can be processed by a computer.

[0035] Specifically, in an embodiment of the present application, matching can be performed in the following manner: a product detection device obtains a picture containing product images and price information, such as a supermarket shelf, a product label, or an advertising poster. Remove noise from the image and convert the image into black and white for easy subsequent processing. Adjust the image angle to ensure that the text and product image are in the correct direction. Use OCR tools (such as Tesseract, Google Cloud Vision API) to locate the text area in the image. Identify the text content and extract price information (such as "$9.99"). Use an image recognition model (such as YOLO, Faster R-CNN) to locate the product in the image. Identify the product type (such as "apple", "milk") through a pre-trained classification model (such as ResNet, Inception). Associate the price information with the product type based on the position of the text and product in the image.

[0036] If the location information is insufficient, matching can be done using logical rules, such as the most recent text being the price.

[0037] Step S14: when all the price tags fail to match, output the unmarked price information of the target product.

[0038] In an embodiment of the present application, an alarm is triggered for goods that are not clearly marked with prices. If the system finds that a certain product has no matching price tag, an alarm is triggered. When the number of goods that are not clearly marked with prices is greater than a threshold, an alarm is triggered.

[0039] Through the above method, combined with the product recognition results and price information, the association matching between products and tags is achieved through keyword matching, string similarity algorithm, and the spatial location of products and price tags.

[0040] In order to improve the matching flexibility of products and price tags, this application proposes an embodiment for further matching using synonym names when the product identification name does not match the price tag. For details, please refer to Figure 2 , Figure 2 It is a flow chart of the second embodiment of the commodity detection method provided by this application.

[0041] like Figure 2 As shown, the specific steps are as follows:

[0042] Step S21: when the product identification name fails to match all the price tags, obtaining a product synonym name of the product identification name.

[0043] First, build a synonym database, combining common synonyms such as "tomato" and "tomato", "potato" and "potato" and "yam".

[0044] In other embodiments of the present application, vocabulary libraries in different languages ​​can also be constructed. For example, the English word for tomato is “tomato”.

[0045] In the embodiment of the present application, the synonym library can be constructed in the following manner:

[0046] First, determine the purpose of the thesaurus and determine the language range and field it covers. Then, extract synonyms from dictionaries and vocabularies: extract synonyms from existing dictionaries, vocabularies, or terminology bases, and extract synonym pairs from text data. You can use open data sources: Word Net: English synonym library; BabelNet: multilingual synonym library; Open Thesaurus: open source synonym library.

[0047] In other embodiments of the present application, user input may be used to collect synonyms through crowdsourcing or user feedback.

[0048] In the embodiments of the present application, predefined rules, such as "X and Y have the same meaning," are used to extract synonyms. Models such as Word2Vec, GloVe, or FastText can be used to extract synonyms based on contextual similarity between words. Models such as BERT and GPT can be used to extract synonyms based on semantic similarity. Co-occurrence patterns of words in a corpus are analyzed to extract potential synonyms. The synonym library is stored in a structured data format. SQL or NoSQL databases can also be used to store synonym relationships.

[0049] In an embodiment of the present application, the synonym library can be updated in real time. The product detection device regularly extracts synonyms from new corpus, allows users to submit new synonym pairs, and uses scripts or tools to automatically update the synonym library.

[0050] Step S22: Match all the price tags using the product synonym name of the target product.

[0051] Specifically, the product summary device uses synonyms for associated matching to ensure that they can be identified and replaced with each other during the matching process. When the synonyms are matched, it is directly counted as a successful price match.

[0052] Step S23: When the product synonym name successfully matches the price tag, outputting product name modification information of the target product based on the product synonym and the product identification name.

[0053] Specifically, when the synonym name of the product is consistent with the identification information in the price tag, the accurate product name of the target product is determined by comparing the synonym name of the product with the product identification name, and the corresponding product name modification information is output. Through the automated matching and recognition mechanism, the accuracy and efficiency of product information management are improved.

[0054] Step S24: When all the price tags fail to match, obtain the character difference degree between the product identification name and the input words in all the price tags.

[0055] If the synonyms do not match, a text similarity calculation method is adopted, and the edit distance algorithm is used to identify the typo match. For example, for the input word "Xihongmian", the algorithm judges whether its similarity is within the preset range by evaluating the character difference degree between it and "Xihongshi", and calculates the Character Matching Degree (CMD).

[0056] Step S25: When the maximum character difference degree exceeds the price matching threshold, obtain the price tag corresponding to the maximum character difference degree as the matching result of the target product.

[0057] Through the above method, products and their price tags can be automatically identified at multiple stalls. By collecting, processing, and matching product information and price information in real time, the system can identify situations of abnormal prices, missing prices, or inconsistent prices, and issue an immediate alarm when it finds products without clearly marked prices.

[0058] Furthermore, an embodiment of the present application is proposed for matching price tags and products. For details, please refer to Figure 3 , Figure 3 which is a schematic flowchart of the third embodiment of the product detection method provided by the present application.

[0059] As Figure 3 shown, the specific steps are as follows:

[0060] Step S31: Obtain the position matching degree between the target product and all the price tags.

[0061] Specifically, based on the regional description, the product detection device spatially associates the product position with the price tag position.

[0062] Step S32: Generate a price matching degree based on the position matching degree and the character difference degree.

[0063] Specifically, based on the regional description, the product detection device spatially associates the product position with the price tag position. Define the distance between the bounding box of the area where the product is located and the bounding box of the price tag area, calculate the positions of the price tag and the product, and calculate the Position Matching Degree (PMD).

[0064] Step S33: When the maximum price matching degree exceeds the price matching threshold, obtain the price tag corresponding to the maximum price matching degree as the matching result of the target product.

[0065] Specifically, the commodity detection device sets a price matching threshold MatchThreshold. When PMD+CMD>MatchThreshold, it is also considered that the price match is successful, and the price tag is modified to the correct characters matched according to the character difference.

[0066] After obtaining the price tag corresponding to the maximum price matching degree as the matching result of the target product, the product detection device outputs product name modification information of the target product based on the price tag corresponding to the maximum price matching degree.

[0067] In one embodiment of the present application, the commodity detection device obtains the prices of several commodities that are successfully matched with the target commodity in different areas; obtains the average price of the several commodity prices; determines whether the difference between each commodity price and the average price exceeds a difference threshold; if so, outputs information that the target commodity is abnormally priced in the area corresponding to the commodity price.

[0068] Specifically, the product detection device will output the product prices of each stall in real time, compare multiple sales prices of the same product, and calculate the average price. If the difference between the price of a stall and the average price exceeds a threshold, the stall will be marked as having a price anomaly.

[0069] In one embodiment of the present application, when the price of a product is different from the regular price, abnormal information is output. Specifically, the pre-marked price of the target product is obtained; it is determined whether the difference between the actual marked price of the price tag successfully matched with the target product and the pre-marked price exceeds the difference threshold; if so, information indicating that the target product's price is abnormal is output.

[0070] This allows for dynamic matching of prices for identical products and automatic alerts when unpriced products are discovered. This real-time anomaly detection mechanism helps quickly identify and correct potential issues.

[0071] In order to implement the commodity detection method of the above embodiment, this application also provides a commodity detection device. Figure 4 , Figure 4 It is a structural diagram of an embodiment of a commodity detection device provided by this application.

[0072] like Figure 4 As shown, the commodity detection device 600 of this embodiment includes a processor 61 , a memory 62 , an input / output device 63 , and a bus 64 .

[0073] The processor 61 , the memory 62 , and the input / output device 63 are respectively connected to a bus 64 . The memory 62 stores a computer program, and the processor 61 is used to execute the computer program to implement the commodity detection method of the above embodiment.

[0074] In this embodiment, the processor 61 may also be referred to as a CPU (Central Processing Unit). The processor 61 may be an integrated circuit chip with signal processing capabilities. The processor 61 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The processor 61 may also be a GPU (Graphics Processing Unit), also known as a display core, visual processor, or display chip. This is a microprocessor specifically designed for image processing on computers, workstations, game consoles, and some mobile devices (such as tablets and smartphones). The purpose of a GPU is to convert and drive the display information required by the computer system and provide horizontal scanning signals to the display to control the correct display of the display. It is a key component connecting the display to the computer motherboard. A graphics card, as a key component of a computer host, is responsible for outputting display graphics. A general-purpose processor may be a microprocessor, or the processor 61 may be any conventional processor.

[0075] This application also provides a computer storage medium, such as Figure 5 As shown, the computer storage medium 700 is used to store a computer program 71. When the computer program 71 is executed by the processor, it is used to implement the method described in the embodiment of the commodity detection method of this application.

[0076] The method involved in the embodiment of the commodity detection method of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk.

[0077] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0078] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0079] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0080] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods of each embodiment of the present application.

[0081] The above description is only an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A commodity detection method, characterized in that: The commodity detection method comprises: Obtain product detection images of the preset product area; Identify the target product and all price tags in the product detection image; Matching all the price tags using the product identification name of the target product; When all the price tags fail to match, the unmarked price information of the target product is output.

2. The commodity detection method according to claim 1, characterized in that: After matching all price tags using the commodity identification name of the target commodity, the commodity detection method further includes: When the product identification name fails to match all the price tags, obtaining a product synonym name of the product identification name; All the price tags are matched using the product synonym name of the target product.

3. The commodity detection method according to claim 2, characterized in that: The commodity detection method further includes: When the product synonym name is successfully matched with the price tag, product name modification information of the target product is output based on the product synonym and the product identification name.

4. The commodity detection method according to claim 1 or 2, characterized in that: The commodity detection method further includes: When all the price tags fail to match, obtaining the degree of difference between the product identification name and the input words in all the price tags; When the maximum character difference exceeds the price matching threshold, the price tag corresponding to the maximum character difference is obtained as the matching result of the target product.

5. The commodity detection method according to claim 4, characterized in that: The commodity detection method further includes: Obtaining a position matching degree between the target product and all the price tags; generating a price matching degree based on the position matching degree and the character difference degree; When the maximum price matching degree exceeds the price matching threshold, a price tag corresponding to the maximum price matching degree is obtained as a matching result of the target product.

6. The commodity detection method according to claim 5, characterized in that: After obtaining the price tag corresponding to the maximum price matching degree as the matching result of the target product, the product detection method further includes: The product name modification information of the target product is output based on the price tag corresponding to the maximum price matching degree.

7. The commodity detection method according to claim 1, characterized in that: The commodity detection method further includes: Obtaining prices of several products that successfully match the target product in different regions; Obtaining an average price of the several commodities; Determine whether the difference between the price of each product and the average price exceeds a difference threshold; If so, output information that the target product is priced abnormally in the area corresponding to the product price.

8. The commodity detection method according to claim 1, characterized in that: The commodity detection method further includes: Obtaining the pre-bid price of the target product; Determine whether the difference between the actual price of the price tag successfully matched with the target product and the pre-bid price exceeds a difference threshold; If so, output information that the target product's price is abnormal.

9. A commodity detection device, characterized in that: The commodity detection device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the commodity detection method according to any one of claims 1 to 8.

10. A computer storage medium, characterized in that The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the commodity detection method according to any one of claims 1 to 8.

Images