An article display information updating method, device, equipment and medium

Abstract

The application discloses an article display information updating method, device, equipment and medium. The method comprises the following steps: obtaining a target shelf image, and detecting the target shelf image based on a target detection model to obtain at least one article single body detection result; determining at least one article single category detection result according to the at least one article single body detection result; and updating article display information of a target shelf according to the article single body detection result and the article single category detection result. The technical scheme solves the problem of low updating efficiency of article display information, can effectively improve the updating efficiency of article display information while ensuring the article recognition accuracy, and reduces the management cost.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, device, and medium for updating information on the display of items. Background Technology

[0002] Supermarkets, convenience stores, and warehouses often require shelves to display various items. To facilitate inventory management, managers typically need to understand the item display on the shelves in order to replenish stock and organize them in a timely manner.

[0003] Currently, supermarkets and other similar venues typically rely on manual labor to complete tasks such as sorting and recording to obtain information on the display of goods, and then to effectively manage the goods based on this information.

[0004] However, due to the large variety of items, relying on manual identification and classification of a large number of items is costly in terms of both economy and time, and the accuracy of item identification cannot be guaranteed. Once the items on the shelves change, the efficiency of updating the item display information is low. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for updating item display information to solve the problem of low efficiency in updating item display information. It can effectively improve the efficiency of updating item display information and reduce management costs while ensuring the accuracy of item identification.

[0006] According to one aspect of the present invention, a method for updating item display information is provided, the method comprising:

[0007] Acquire an image of the target shelf and perform detection on the image of the target shelf based on a target detection model to obtain at least one item detection result;

[0008] Based on the test results of at least one individual item, determine at least one set of test results for a single category of items;

[0009] Update the item display information of the target shelf based on the individual item detection results and the item category detection results.

[0010] According to another aspect of the present invention, an apparatus for updating item display information is provided, the apparatus comprising:

[0011] The single item detection result determination module is used to acquire the target shelf image and detect the target shelf image based on the target detection model to obtain at least one item single item detection result;

[0012] The single-category detection result determination module is used to determine at least one set of single-category detection results for an item based on at least one item's individual detection result.

[0013] The item display information update module is used to update the item display information of the target shelf based on the individual item detection results and the item category detection results.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the item display information update method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the method for updating item display information according to any embodiment of the present invention.

[0019] The technical solution of this invention involves acquiring an image of a target shelf, detecting the image based on a target detection model to obtain at least one item detection result, determining at least one set of item class detection results based on the at least one item detection result, and updating the item display information of the target shelf based on the item detection results and the item class detection results. This solution can solve the problem of low efficiency in updating item display information, effectively improving the efficiency of updating item display information while ensuring the accuracy of item recognition and reducing management costs.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1A This is a flowchart of a method for updating item display information according to Embodiment 1 of the present invention;

[0023] Figure 1BThis is a schematic diagram of an item display shelf provided according to an embodiment of the present invention;

[0024] Figure 2 This is a flowchart of a method for updating item display information according to Embodiment 2 of the present invention;

[0025] Figure 3 This is a schematic diagram of the structure of an item display information updating device according to Embodiment 3 of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the method for updating item display information according to an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of national laws and regulations.

[0029] Example 1

[0030] Figure 1A This is a flowchart illustrating a method for updating item display information according to Embodiment 1 of the present invention. This embodiment is applicable to situations involving updating item display information. The method can be executed by an item display information updating device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1A As shown, the method includes:

[0031] S110. Obtain the target shelf image and perform detection on the target shelf image based on the target detection model to obtain at least one item detection result.

[0032] This solution can be executed by an update platform, which can respond to user-initiated requests to update item display information and obtain images of the target shelf. These requests can be either store visit requests initiated by the user through a user terminal or images of the target shelf uploaded by the user. The update platform can then detect the target shelf image based on the user terminal's request information or user registration information, using a target detection model.

[0033] The object detection model can be trained on a publicly available item dataset or on a self-built item dataset. It can be a model built using a one-stage object detection algorithm such as YOLO or SSD, or a model built using a two-stage object detection algorithm such as Faster R-CNN. It should be noted that the object detection model can achieve dense object detection, detecting individual items within a target shelf image.

[0034] After detecting individual items in the target shelf image, the update platform can generate individual item detection results. These results may include the location of each item, its category, the coordinates of its detection bounding box, and its detection accuracy. Figure 1B This is a schematic diagram of an item display shelf provided according to an embodiment of the present invention. Figure 1B The item display shelf in the image is the target shelf image, and the single-item detection box can be like... Figure 1B As shown in the detection box 1, the update platform can construct a coordinate system based on the target shelf image, recording the center coordinates and / or vertex coordinates of the individual item detection boxes to describe the position of each individual item. The update platform can then extract images of the target shelf from the individual item detection boxes to obtain individual item images.

[0035] S120. Based on the individual item test results, determine at least one set of item category test results.

[0036] Understandably, in scenarios such as supermarkets, warehouses, and vending machines, managers often need to understand the distribution of individual items of the same category for ease of management. Based on the detection results of each individual item in the target shelf image, the update platform can obtain the item category detection results based on the detection results of the individual items corresponding to the same category. For example, the update platform can map the detection results of individual items of the same category to the item category detection results. Similarly, the item category detection results can include item category, distribution location of similar items, and item category detection boxes, etc.

[0037] S130. Update the item display information of the target shelf based on the individual item detection results and the item category detection results.

[0038] Based on the individual item detection results, the update platform can obtain the distribution of a single item on the target shelf. Similarly, based on the category-specific item detection results, the update platform can determine the distribution of each category of items on the target shelf. Based on the distribution of individual items and the distribution of each category of items, the update platform can update the item display information of the target shelf. This item display information can be a shelf layout diagram or shelf layout table for the target shelf.

[0039] Specifically, the updated platform can determine the shelf number of a target item based on the location of the individual item image in the item category detection results. For example... Figure 1B As shown, the positions of items on the same shelf should be within the same coordinate range. The update platform can perform cluster analysis on the center coordinates of the individual item detection boxes to obtain the layer distribution of items on the target shelf.

[0040] In this scheme, optionally, the individual item detection result includes an individual item image;

[0041] The step of determining at least one set of single-category item detection results based on at least one item individual detection result includes:

[0042] Feature extraction is performed on each individual image based on a feature extraction model;

[0043] Based on the similarity comparison between the feature extraction results of each individual image and the features in the pre-acquired item feature set, at least one set of item single-class detection results is determined.

[0044] To achieve refined recognition of individual images, the update platform can extract features from individual images cropped from the target shelf image based on a feature extraction model. This feature extraction model can be a feature extraction model built using deep learning algorithms such as convolutional neural networks, or a feature extraction model built using traditional graphics algorithms such as scale-invariant feature transform. After feature extraction from individual images, the update platform can obtain the feature extraction results for each individual image, such as feature vectors.

[0045] The update platform can pre-build an item feature set, which can include multiple features to cover all item features required for the application scenario. The platform can compare the feature extraction results of each individual image with the features in the item feature set to determine the category of each individual image, and then determine at least one set of single-class item detection results based on the category of each individual image. Specifically, the update platform can sequentially calculate the Euclidean distance, cosine distance, etc., between the feature extraction results of individual images and the features in the item feature set to represent their similarity. The similarity of the same individual image is compared, and the item category associated with the feature with the highest similarity is selected as the target category for that individual image.

[0046] Assuming the target shelf image detects 3 individual items, resulting in 3 item detection results, the item feature set includes 4 features, each associated with an item category. The update platform can calculate the similarity between the feature vector of individual image 1 and features 1, 2, 3, and 4, obtaining similarity scores 1-1, 1-2, 1-3, and 1-4. Similarly, the update platform can calculate the similarity between the feature vectors of individual images 2 and 3 and features 1, 2, 3, and 4. The update platform can determine the target category of individual image 1 by comparing similarity scores 1-1, 1-2, 1-3, and 1-4. Assuming similarity scores 1-1, 1-2, 1-3, and 1-4 are 0.1, 0.4, 0.3, and 0.9 respectively, then the item category associated with feature 4 is taken as the target category of individual image 1.

[0047] This technical solution acquires an image of the target shelf, performs detection on the image using a target detection model, and obtains at least one individual item detection result. Based on the at least one individual item detection result, at least one set of single-class item detection results is determined. The item display information of the target shelf is then updated based on the individual item detection results and the single-class item detection results. This solution solves the problem of low efficiency in updating item display information, effectively improving the efficiency of updating item display information while ensuring item recognition accuracy and reducing management costs.

[0048] Example 2

[0049] Figure 2 This is a flowchart illustrating a method for updating item display information according to Embodiment 2 of the present invention. This embodiment is a refinement based on the above embodiment. Figure 2 As shown, the method includes:

[0050] S210. Obtain an image of the target shelf and perform detection on the image of the target shelf based on the target detection model to obtain at least one item detection result; wherein, the item detection result includes an item image and an item detection box.

[0051] In this solution, the single-class detection result of an item may include item category information and single-class detection box information.

[0052] S220. Based on the feature extraction model, feature extraction is performed on each individual image.

[0053] S230. Determine the similarity between the feature extraction results of each individual image and the features in the item feature set, and sort the similarity of each individual image to obtain the similarity ranking result.

[0054] Assuming the target shelf image detects 3 individual items, resulting in 3 item detection results, and the item feature set includes 4 features, each associated with an item category, the update platform can calculate the similarity between the feature vector of item image 1 and features 1, 2, 3, and 4, obtaining similarity scores 1-1, 1-2, 1-3, and 1-4. Similarly, the update platform can obtain similarity scores 2-1, 2-2, 2-3, and 2-4, as well as similarity scores 3-1, 3-2, 3-3, and 3-4. By comparing similarity scores 1-1, 1-2, 1-3, and 1-4, the update platform can obtain the similarity ranking result for item image 1. Assuming similarity 1-1, similarity 1-2, similarity 1-3, and similarity 1-4 are 0.1, 0.4, 0.3, and 0.9 respectively, the similarity ranking of individual image 1 could be: 0.9, 0.4, 0.3, 0.1.

[0055] S240. Based on the similarity ranking results, determine the item category information for each individual image.

[0056] The update platform can select the item category associated with the highest feature similarity from the item feature set as the classification result for a single image based on the similarity ranking results. Alternatively, the update platform can select item categories associated with only a subset of features from the item feature set as the classification result for a single image based on the similarity ranking results. Provided that server and storage hardware conditions permit, the update platform can also retain all feature-associated item categories as the classification result for a single image based on the similarity ranking results.

[0057] Taking the example in S230, the classification result of a single image 1 can be the item category corresponding to feature 4, or the item category corresponding to feature 4 can be the target category, and the item category corresponding to feature 2 can be a candidate category, or the item category corresponding to feature 4 can be the target category, and the item categories corresponding to features 2, 3, and 1 can be used as a candidate category list. The updating platform can generate corresponding item category information based on the classification results.

[0058] S250. Merge the individual detection boxes corresponding to the individual images with the same item category information into a single-class detection box and generate single-class detection box information.

[0059] On product display shelves, similar items are typically placed adjacent to each other for ease of management. The update platform can compare the item category information of individual images; if the item category information is the same, the individual images can be determined to belong to the same category. The update platform can merge individual detection boxes of the same category to obtain single-class detection boxes, which can be used as follows: Figure 1B Detection boxes 2, 3, and 4 are shown in the diagram. Based on the single-class detection boxes, the update platform can generate single-class detection box information. This single-class detection box information may include the location of the single-class detection box, the number of merged single detection boxes, and other information.

[0060] S260. Update the item display information of the target shelf according to the item category information, the single-category detection box information and the item individual detection result.

[0061] In one feasible solution, determining the item category information for each individual image based on the similarity ranking result includes:

[0062] Based on the similarity ranking results, determine the target classification and target classification confidence of each individual image;

[0063] Based on the target classification and the target classification confidence level, the item category information of each individual image is determined.

[0064] In this scheme, the target classification confidence can be determined based on the highest similarity. Taking the assumption in S230 as an example, if the target classification of single image 1 is category A corresponding to feature 4, the target classification confidence can be 0.9. Alternatively, the target classification confidence can be obtained based on the similarity ranking results. For example, the update platform can calculate the similarity weight of the target classification based on the similarity ranking results. The updated platform can generate item category information for individual images based on target classification and target classification confidence.

[0065] This scheme can determine the confidence level of the target classification, which is beneficial for evaluating the confidence level of the target classification and optimizing the classification results. It is also beneficial for targeted training of the feature extraction model based on the confidence level of the target classification, so as to improve the performance of the feature extraction model.

[0066] Based on the above scheme, the single-class detection box information includes the number of merged single-unit detection boxes;

[0067] After determining at least one set of single-class detection results for items, the method further includes:

[0068] If the number of merged single-unit detection boxes is 1, then at least one similarity between the single-unit image in the single-class detection box and the target single-unit image in the adjacent single-class detection box is determined.

[0069] If at least one similarity is higher than a preset similarity threshold, and the target classification confidence of the single image is lower than a preset confidence threshold, then the single-class detection box is merged into the adjacent single-class detection box with the highest similarity, and the item category correction information is determined.

[0070] Update the item feature set based on the item category correction information.

[0071] If the number of merged single-class detection boxes is 1, it means that there is only one single-class detection box, and this single-class detection box has a high probability of being misclassified. The update platform can determine whether the single-class detection box belongs to the same category as its neighboring single-class detection boxes by calculating the similarity between the single-class detection box image and its neighboring single-class detection box images. If at least one similarity score is higher than a preset similarity threshold, and the target classification confidence score of the single-class image is lower than a preset confidence threshold, it means that the original target classification confidence score of the single-class image is low, while the single-class image has high similarity to its neighboring single-class images, indicating that the single-class image has been misclassified. Therefore, the update platform can merge the single-class detection box into the neighboring single-class detection box with the highest similarity score and determine the item category correction information.

[0072] The item category correction information may include the individual image to be corrected, the reason for correction, the correction category, and the target reference image. The update platform can add the item category correction information to the item feature set and associate it with the target classification to improve classification accuracy in subsequent use. To ensure update accuracy, the update platform can send the item category correction information to the user terminal and determine whether to update the item feature set based on the user terminal's verification results.

[0073] This solution can effectively identify and correct misclassifications, which helps ensure the accuracy of item classification results.

[0074] In another feasible approach, determining the item category information for each individual image based on the similarity ranking result includes:

[0075] Based on the similarity ranking results, determine the target classification and at least one candidate classification for each individual image, and determine the confidence scores for the target classification and each candidate classification.

[0076] Based on the target classification, the target classification confidence level, each candidate classification, and each candidate classification confidence level, the item category information of each individual image is determined.

[0077] As is easily understood, similar to the target classification confidence, the candidate classification confidence can be determined based on the similarity between the features associated with each candidate classification and the individual images, or it can be obtained based on the similarity ranking results of the individual images. The update platform can generate a corresponding classification list for each individual image according to the similarity ranking results, as shown in Table 1 below, which is the classification list for individual image 1.

[0078] Table 1:

[0079] Categorization and sorting Item Category Confidence Target Classification Category A 0.9 Candidate Category 1 Category B 0.4 Candidate Category 2 Category C 0.3 Candidate Classification 3 Category D 0.1

[0080] Based on the target category, candidate categories, target category confidence score, and candidate category confidence score information in the classification list, the platform updates the item category information for each generated individual image. Item category information may also include unit price, specifications, and other information for each category of item.

[0081] This scheme can determine at least one candidate classification and the confidence level of the candidate classification based on the similarity ranking results. This is beneficial for correcting the target classification information based on the candidate classification information, thereby ensuring the reliability of the classification.

[0082] Based on the above scheme, the single-class detection box information also includes the identified unit price; wherein, the identified unit price is determined based on the text recognition result of the price tag image in the single-class detection box;

[0083] After determining at least one set of single-class detection results for items, the method further includes:

[0084] If the identified unit price does not match the unit price of the item in the target category, then the item category correction information is determined based on the unit price of the item in each candidate category and the confidence level of each candidate category.

[0085] Update the item feature set based on the item category correction information.

[0086] like Figure 1B As shown, when detecting the target shelf image, price tags on the shelf can also be detected, and price tag images can be obtained based on the detection results. During the merging of single-object detection boxes in similar single-object images, single-class detection boxes can also be merged with price tag images. By performing text recognition on the price tag images using a text recognition model, the platform can update to obtain the recognized unit price of the item category within the single-class detection box.

[0087] If the identified unit price is inconsistent with the unit price of the item in the target category, or the difference exceeds a certain price threshold, it indicates that the item may be a new product or that the single image classification of that category is incorrect. The update platform can compare the unit price of the item in the candidate category with the identified unit price. The update platform selects the candidate category with the highest confidence and the smallest difference between the item's unit price and the identified unit price as the replacement category for the target category, and generates item category correction information.

[0088] The update platform can send item category correction information to the user terminal and determine whether to update the item feature set based on the user terminal's verification results. If the item is a new product, the update platform can add new product features to the item feature set. If the classification of a single image in the category is incorrect, the update platform can correct the features in the item feature set.

[0089] This solution can verify misclassification, new product listings, and other situations, which can minimize business losses and help continuously improve the set of product characteristics to enhance classification accuracy.

[0090] It should be noted that the update platform can perform statistical analysis on all item category correction information to determine the correction data for each item category. This correction data can include information such as the number of corrections and the probability of correction. Based on this correction data, the update platform can expand the individual images to be corrected into the feature extraction model's dataset, enabling targeted training of the feature extraction model and improving the effectiveness of feature extraction.

[0091] This technical solution acquires an image of the target shelf, performs detection on the image using a target detection model, and obtains at least one individual item detection result. Based on the at least one individual item detection result, at least one set of single-class item detection results is determined. The item display information of the target shelf is then updated based on the individual item detection results and the single-class item detection results. This solution solves the problem of low efficiency in updating item display information, effectively improving the efficiency of updating item display information while ensuring item recognition accuracy and reducing management costs.

[0092] Example 3

[0093] Figure 3 This is a schematic diagram of a device for updating item display information according to Embodiment 3 of the present invention. Figure 3 As shown, the device includes:

[0094] The single item detection result determination module 310 is used to acquire the target shelf image and detect the target shelf image based on the target detection model to obtain at least one item single item detection result;

[0095] The single-category detection result determination module 320 is used to determine at least one set of single-category detection results for an item based on at least one item individual detection result;

[0096] The item display information update module 330 is used to update the item display information of the target shelf based on the individual item detection results and the item category detection results.

[0097] In this scheme, optionally, the individual item detection result includes an individual item image;

[0098] The single-class detection result determination module 320 includes:

[0099] The feature extraction unit is used to extract features from each individual image based on the feature extraction model;

[0100] The single-class detection result determination unit is used to determine at least one set of single-class detection results for items based on the similarity comparison results between the feature extraction results of each individual image and the features in the pre-acquired item feature set.

[0101] Based on the above scheme, optionally, the item individual detection result also includes an item detection frame; the item single-category detection result includes item category information and single-category detection frame information;

[0102] The single-class detection result determination unit includes:

[0103] The similarity ranking result determination subunit is used to determine the similarity between the feature extraction result of each individual image and the features in the item feature set, and to sort the similarity of each individual image to obtain the similarity ranking result.

[0104] The item category information determination subunit is used to determine the item category information of each individual image based on the similarity sorting results;

[0105] The single-class detection box information generation subunit is used to merge the single-class detection boxes corresponding to single images with the same item category information into a single-class detection box and generate single-class detection box information.

[0106] In one feasible solution, the item category information determination sub-unit is specifically used for:

[0107] Based on the similarity ranking results, determine the target classification and target classification confidence of each individual image;

[0108] Based on the target classification and the target classification confidence level, the item category information of each individual image is determined.

[0109] Based on the above scheme, the single-class detection box information includes the number of merged single-unit detection boxes;

[0110] The device further includes:

[0111] The similarity determination module is used to determine at least one similarity between a single-class image in a single-class detection box and a target single-class image in an adjacent single-class detection box if the number of merged single-class detection boxes is 1.

[0112] The first correction information determination module is used to merge the single-class detection box into the adjacent single-class detection box with the highest similarity if there is at least one similarity higher than a preset similarity threshold and the target classification confidence of the single image is lower than a preset confidence threshold, and determine the item category correction information.

[0113] The first item feature set update module is used to update the item feature set according to the item category correction information.

[0114] In another feasible solution, the item category information determining sub-unit is specifically used for:

[0115] Based on the similarity ranking results, determine the target classification and at least one candidate classification for each individual image, and determine the confidence scores for the target classification and each candidate classification.

[0116] Based on the target classification, the target classification confidence level, each candidate classification, and each candidate classification confidence level, the item category information of each individual image is determined.

[0117] Based on the above scheme, the single-class detection box information also includes the identified unit price; wherein, the identified unit price is determined based on the text recognition result of the price tag image in the single-class detection box;

[0118] The device further includes:

[0119] The second correction information determination module is used to determine item category correction information based on the item unit price of each candidate category and the confidence level of each candidate category if the identified unit price does not match the item unit price of the target category.

[0120] The second item feature set update module is used to update the item feature set according to the item category correction information.

[0121] The item display information updating device provided in the embodiments of the present invention can execute the item display information updating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0122] Example 4

[0123] Figure 4A schematic diagram of an electronic device 410 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0124] like Figure 4 As shown, the electronic device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0125] Multiple components in electronic device 410 are connected to I / O interface 415, including: input unit 416, such as keyboard, mouse, etc.; output unit 417, such as various types of displays, speakers, etc.; storage unit 418, such as disk, optical disk, etc.; and communication unit 419, such as network card, modem, wireless transceiver, etc. Communication unit 419 allows electronic device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0126] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as methods for updating item display information.

[0127] In some embodiments, the method for updating item display information may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the method for updating item display information described above may be performed. Alternatively, in other embodiments, processor 411 may be configured to perform the method for updating item display information by any other suitable means (e.g., by means of firmware).

[0128] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0132] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0133] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0134] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0135] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for updating item display information, characterized in that, The method includes: Acquire an image of the target shelf and perform detection on the image of the target shelf based on a target detection model to obtain at least one item detection result; wherein, the item detection result includes the location of each item, the category of each item, the image of each item, the coordinates of the detection box of each item, and the detection accuracy of each item; Based on the test results of at least one individual item, determine at least one set of test results for a single category of items; Based on the individual item detection results, the distribution of each individual item on the target shelf is obtained. Based on the individual item category detection results, the distribution of each category of items on the target shelf is determined. Based on the distribution of individual items and the distribution of each category of items, the item display information of the target shelf is updated. The step of determining at least one set of single-category item detection results based on at least one item individual detection result includes: Feature extraction is performed on each individual image based on a feature extraction model; Based on the feature extraction results of each individual image and the similarity comparison results of features in the pre-acquired item feature set, at least one set of item single-class detection results is determined; The single-category detection result of the item includes item category information and single-category detection box information; The step of determining at least one set of single-class detection results for items based on the similarity comparison between the feature extraction results of each individual image and the features in the pre-acquired item feature set includes: The similarity between the feature extraction results of each individual image and the features in the item feature set is determined, and the similarity of each individual image is sorted to obtain the similarity ranking result. Based on the similarity ranking results, the item category information of each individual image is determined; Individual detection boxes corresponding to individual images with the same item category information are merged into single-class detection boxes, and single-class detection box information is generated.

2. The method according to claim 1, characterized in that, The step of determining the item category information for each individual image based on the similarity ranking result includes: Based on the similarity ranking results, determine the target classification and target classification confidence of each individual image; Based on the target classification and the target classification confidence level, the item category information of each individual image is determined.

3. The method according to claim 1, characterized in that, The single-class detection box information includes the number of merged single-unit detection boxes; After determining at least one set of single-class detection results for items, the method further includes: If the number of merged single-unit detection boxes is 1, then at least one similarity between the single-unit image in the single-class detection box and the target single-unit image in the adjacent single-class detection box is determined. If at least one similarity is higher than a preset similarity threshold, and the target classification confidence of the single image is lower than a preset confidence threshold, then the single-class detection box is merged into the adjacent single-class detection box with the highest similarity, and the item category correction information is determined. Update the item feature set based on the item category correction information.

4. The method according to claim 1, characterized in that, The step of determining the item category information for each individual image based on the similarity ranking result includes: Based on the similarity ranking results, determine the target classification and at least one candidate classification for each individual image, and determine the confidence scores for the target classification and each candidate classification. Based on the target classification, the target classification confidence level, each candidate classification, and each candidate classification confidence level, the item category information of each individual image is determined.

5. The method according to claim 4, characterized in that, The single-class detection box information also includes the identified unit price; wherein, the identified unit price is determined based on the text recognition result of the price tag image in the single-class detection box; After determining at least one set of single-class detection results for items, the method further includes: If the identified unit price does not match the unit price of the item in the target category, then the item category correction information is determined based on the unit price of the item in each candidate category and the confidence level of each candidate category. Update the item feature set based on the item category correction information.

6. A device for updating item display information, characterized in that, The device includes: The single-item detection result determination module is used to acquire a target shelf image and detect the target shelf image based on a target detection model to obtain at least one item single-item detection result; wherein, the item single-item detection result includes the location of each single item, the category of each single item, the image of each single item, the coordinates of each single item detection box, and the detection accuracy of each single item; The single-category detection result determination module is used to determine at least one set of single-category detection results for an item based on at least one item's individual detection result. The item display information update module is used to obtain the distribution of a single item on the target shelf based on the individual item detection results, determine the distribution of each category of items on the target shelf based on the single category detection results, and update the item display information of the target shelf based on the distribution of individual items and the distribution of each category of items. The single-class detection result determination module includes: The feature extraction unit is used to extract features from each individual image based on the feature extraction model; The single-class detection result determination unit is used to determine at least one set of single-class detection results for items based on the similarity comparison results between the feature extraction results of each individual image and the features in the pre-acquired item feature set. The single-category detection result of the item includes item category information and single-category detection box information; The single-class detection result determination unit includes: The similarity ranking result determination subunit is used to determine the similarity between the feature extraction result of each individual image and the features in the item feature set, and to sort the similarity of each individual image to obtain the similarity ranking result. The item category information determination subunit is used to determine the item category information of each individual image based on the similarity sorting results; The single-class detection box information generation subunit is used to merge the single-class detection boxes corresponding to single images with the same item category information into a single-class detection box and generate single-class detection box information.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the method for updating item display information according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for updating the item display information as described in any one of claims 1-5.

Images