Commodity new method, commodity identification method, device, system and electronic equipment

Abstract

Embodiments of the present application relate to a new product method, a product identification method, a device, a system and an electronic device, wherein new product video data and new product reference images are obtained; based on a trained product detection model, new product regions of multiple frames of images in the new product video data are extracted to obtain new product sub-images; based on the new product sub-images and an original product data set, a current product feature retrieval model is trained to obtain a trained product feature retrieval model; based on the trained product feature retrieval model, product features of the new product reference images and original product reference images are extracted to update the preset feature retrieval library; the current product feature retrieval model which has been locally optimized can be quickly trained by collecting a small amount of new product video data, the new product efficiency is improved, and the accuracy and stability of product identification are improved by combining the product detection model and the product feature retrieval model.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for adding new products to a product listing, a product recognition method, an apparatus, a system, and an electronic device. Background Technology

[0002] With the rapid development of artificial intelligence, more and more industries are adopting the new model of Artificial Intelligence Plus (AI+). Among them, dynamic visual vending machines are a representative of AI+ in new retail.

[0003] Compared to traditional smart vending machines, dynamic vision vending machines liberate the vending machine itself and increase the utilization rate of the internal space. However, the urgent problem to be solved is how to effectively avoid interference from external objects, ensure a stable product recognition accuracy rate, and enable new products to be put on the shelves in a short period of time.

[0004] In existing technologies, end-to-end object detection models are mainly used for product updates and new product launches. The update method involves manually collecting a large number of new product samples as a support set, adding them to the previous product set, and retraining the object detection model. This results in a long update cycle and unstable recognition accuracy. Summary of the Invention

[0005] The embodiments of the present invention provide a method for adding new products, a method for identifying products, an apparatus, a system, and an electronic device to solve the technical problems of long product addition cycles and unstable identification accuracy.

[0006] In a first aspect, embodiments of the present invention provide a method for adding new products, comprising: acquiring new product video data and acquiring a new product reference image; extracting new product regions from multiple frames of images in the new product video data based on a trained product detection model to obtain a new product sub-image; training a current product feature retrieval model based on the new product sub-image and an existing product dataset to obtain a trained product feature retrieval model, wherein the product feature retrieval model is used to extract features from the product sub-image and match the extracted product features with each product feature in a preset feature retrieval library to identify the product; and extracting product features from the new product reference image and the existing product reference image based on the trained product feature retrieval model to update the preset feature retrieval library.

[0007] In one embodiment of the present invention, the product detection model is a YOLOX model, the backbone network of the YOLOX model is DarkNet53, and the last fully connected layer of DarkNet53 is frozen.

[0008] As an embodiment of the present invention, the product feature retrieval model is a PaddleClas model, the backbone network of the PaddleClas model is PP_LCNet, and the network layers of the PaddleClas model include linear layers and feature pyramid layers.

[0009] As an embodiment of the present invention, before training the current product feature retrieval model based on the new product subgraph and the original product dataset, the method further includes: iteratively training the product feature retrieval model to be trained based on the original product dataset and the open-source dataset until the target loss function meets the preset convergence condition, thereby obtaining the current product feature retrieval model; wherein, the target loss function is determined based on the face recognition loss function and the ternary loss function.

[0010] As an embodiment of the present invention, the acquisition of new product video data includes: acquiring new product video data under a preset scene; wherein, the preset scene includes at least one of the following: the new product rotates horizontally and moves up and down, the new product tilts and rotates and moves up and down, and the new product moves at a speed higher than a preset speed under a light intensity lower than a preset light intensity.

[0011] As an embodiment of the present invention, the step of obtaining a reference image of a new product includes: obtaining an image of the new product from a preset angle or a preset side view; wherein the preset angle and preset side view are determined according to the appearance type of the new product.

[0012] As an embodiment of the present invention, the method further includes: acquiring non-commodity features and adding the non-commodity features to the preset feature retrieval library.

[0013] Secondly, embodiments of the present invention provide a product identification method, comprising: acquiring transaction video data; extracting the product region to be identified from keyframe images in the transaction video data based on a trained product detection model to obtain a product sub-image to be identified; inputting the product sub-image to be identified into a trained product feature retrieval model to extract features from the product sub-image to be identified, and matching the extracted product features to be identified with each product feature in a preset feature retrieval library to identify the product and determine the corresponding product label; and determining the product cost based on the product label.

[0014] As an embodiment of the present invention, the preset feature retrieval library includes various commodity features and non-commodity features; the step of matching the extracted commodity features to be identified with the commodity features in the preset feature retrieval library includes: matching the extracted commodity features to be identified with the commodity features and non-commodity features in the preset feature retrieval library.

[0015] Thirdly, embodiments of the present invention provide a product listing device, comprising: a first product data module, used to acquire new product video data and acquire a new product reference image; a first product detection module, used to extract new product regions from multiple frames of images in the new product video data based on a trained product detection model, to obtain a new product sub-image; a first product feature retrieval module, used to train a current product feature retrieval model based on the new product sub-image and an existing product dataset image, to obtain a trained product feature retrieval model, wherein the product feature retrieval model is used to extract features from the product sub-image and match the extracted product features with each product feature in a preset feature retrieval library to identify the product; and a feature retrieval update module, used to extract product features from the new product reference image and the existing product reference image based on the trained product feature retrieval model, to update the preset feature retrieval library.

[0016] Fourthly, embodiments of the present invention provide a product identification device, comprising: a second product data module for acquiring transaction video data; a second product detection module for extracting the product region to be identified from keyframe images in the transaction video data based on a trained product detection model, thereby obtaining a product sub-image to be identified; a second product feature retrieval module for inputting the product sub-image to be identified into a trained product feature retrieval model, extracting features from the product sub-image to be identified, and matching the extracted product features with product features in a preset feature retrieval library to identify the product and determine the corresponding product label; and a product cost determination module for determining the product cost based on the product label.

[0017] Fifthly, embodiments of the present invention provide a product recognition and new product listing system, comprising: a product data module for acquiring new product video data and acquiring a new product reference image; a product detection module for extracting new product regions from multiple frames of images in the new product video data based on a trained product detection model, thereby obtaining a new product sub-image; a product feature retrieval module for training a current product feature retrieval model based on the new product sub-image and an existing product dataset image, thereby obtaining a trained product feature retrieval model; and a feature retrieval update module for extracting product features from the new product reference image and the existing product reference image based on the trained product feature retrieval model, so as to... The system updates the preset feature retrieval library; the product data module is also used to acquire transaction video data; the product detection module is also used to extract the product region to be identified from the keyframe images in the transaction video data based on the trained product detection model, and obtain a product sub-image to be identified; the product feature retrieval module is also used to input the product sub-image to be identified into the trained product feature retrieval model, extract features from the product sub-image to be identified, and match the extracted product features to be identified with each product feature in the preset feature retrieval library to identify the product and determine the corresponding product label; the product cost determination module is used to determine the product cost based on the product label.

[0018] In a sixth aspect, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor is used to execute the program stored in the memory to implement the steps of the product listing method as described in any of the first aspects or the product identification method as described in any of the second aspects.

[0019] In a seventh aspect, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the product listing method according to any one of the first aspects or the product identification method according to any one of the second aspects.

[0020] The product addition method, product recognition method, apparatus, system, and electronic device provided by embodiments of the present invention acquire new product video data and a new product baseline image; extract new product regions from multiple frames of images in the new product video data based on a trained product detection model to obtain new product sub-images; train a current product feature retrieval model based on the new product sub-images and the original product dataset to obtain a trained product feature retrieval model, wherein the product feature retrieval model is used to extract features from the product sub-images and match the extracted product features with each product feature in a preset feature retrieval library to identify the product; extract product features from the new product baseline image and the original product baseline image based on the trained product feature retrieval model to update the preset feature retrieval library; realize the ability to quickly train the locally optimal current product feature retrieval model by collecting a small amount of new product video data, improving the efficiency of product addition, and improving the accuracy and stability of product recognition by combining the product detection model and the product feature retrieval model. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a method for adding new products according to an embodiment of the present invention;

[0024] Figure 2 This is a flowchart illustrating a product identification method provided in an embodiment of the present invention;

[0025] Figure 3 A flowchart illustrating a product update and identification method provided in an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of a product refill device provided in an embodiment of the present invention;

[0027] Figure 5 This is a schematic diagram of the structure of a commodity identification device provided in an embodiment of the present invention;

[0028] Figure 6 This is a schematic diagram of the structure of a product update and identification system provided in an embodiment of the present invention;

[0029] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] With the booming development of artificial intelligence, more and more industries are starting new AI+ models, and the new retail concept of AI+retail is quietly emerging. The continuous influx of capital has enabled smart retail products to iterate rapidly.

[0032] As a representative of new retail, smart vending machines have evolved from initial spring-loaded and gravity-fed models to current static and dynamic visual vending machines. These product updates have brought significant technological breakthroughs. Previous spring-loaded machines allowed customers to scan a code to place an order, displaying the selected product directly, eliminating issues with product recognition and product updates. While dynamic visual vending machines liberate the vending machine itself, increase internal space utilization, and improve user experience, they also present significant challenges. Effectively preventing interference from external objects, ensuring stable and accurate algorithm recognition, and rapidly adding new products are all pressing problems that need to be addressed.

[0033] Currently, existing dynamic visual display cabinets mainly use a single target detection model for product listing and recognition. The listing method involves manually collecting a large number of new product samples as a support set, adding them to the previous product set, and retraining the target detection model. The listing cycle depends mainly on the sample size and computing power. Generally, the listing cycle for 20 new products is 3-7 days. This method is time-consuming and the recognition accuracy is not stable.

[0034] To address the aforementioned technical problems, the technical concept of this invention is as follows: by collecting a small number of new product videos to construct a training support set, and by combining a general product detection model and a product feature retrieval model, the accuracy of product recognition can be improved while effectively achieving rapid new product launches with fewer products.

[0035] Figure 1 This is a flowchart illustrating a product listing method provided in an embodiment of the present invention. The executing entity is a product listing device, or an electronic device equipped with a product listing device. For example... Figure 1 As shown, the methods for adding new products include:

[0036] Step S101: Obtain new product video data and new product baseline image.

[0037] Specifically, when new products need to be listed, the operators collect an appropriate amount of video data on the new products and take a baseline image of the new products (also known as a base image). Then, the video data and baseline image of the new products are uploaded through the backend new product listing channel. The execution entity in this embodiment obtains the corresponding new product video data and new product baseline image.

[0038] In some embodiments, obtaining new product video data in step S101 includes: obtaining new product video data under a preset scenario; wherein the preset scenario includes at least one of the following: the new product rotates horizontally and moves up and down, the new product tilts and rotates and moves up and down, and the new product moves at a speed higher than a preset speed under a light intensity lower than a preset light intensity.

[0039] Specifically, the operation personnel collect appropriate video data on new products in three main scenarios: first, the new product rotates horizontally and moves vertically (moving between different heights in the dynamic visual display); second, the new product tilts and rotates vertically (moving between different heights in the dynamic visual display); and third, the new product moves rapidly in low light (allowing for quick retrieval from the dynamic visual display). The first and second scenarios are relatively simple, while the third is relatively complex. 200 images can be extracted from the video data corresponding to each scenario, forming a total of 600 images for the new product training set. Optionally, data augmentation can be used to expand the video data, establishing a minimum number of supplementary product training samples to meet the minimum training requirements of the subsequent product feature retrieval model, thereby better improving the model's ability to fit the products.

[0040] In some embodiments, obtaining a reference image of a new product in step S101 includes: obtaining an image of the new product from a preset angle or a preset side view; wherein the preset angle and preset side view are determined according to the appearance type of the new product.

[0041] Specifically, the appearance type of the product includes at least one of the following: boxed, bagged, or cylindrical (canned, bowl-shaped). Different shooting methods can be determined for different appearances. For example, for a boxed product, six sides (front, side, bottom) can be photographed, as well as combinations of different angles and sides, resulting in a total of nine reference images. These nine reference images are uploaded to the product database as base photos of the new product. The product database stores base photos of various products.

[0042] Step S102: Extract new product regions from multiple frames of images in the new product video data based on the pre-trained product detection model to obtain new product sub-images.

[0043] Specifically, a pre-trained product detection model can perform image processing on new product video data, initially detecting products from the images and cropping out partial images containing only the products, i.e., new product sub-images. The product detection model removes useless background information from the images, focusing more on the product image itself.

[0044] In some embodiments, the product detection model is a YOLOX model, the backbone network of the YOLOX model is DarkNet53, and the last fully connected layer of DarkNet53 is in a frozen state.

[0045] Specifically, YOLOX (You Only Look Once X) is an evolution of the YOLO (You Only Look Once: Unified, Real-Time Object Detection) object detection algorithm; DarkNet53 refers to a neural network with 53 residual layers. In this embodiment, the backbone of the YOLOX product detection model uses DarkNet53, and the last fully connected layer of DarkNet53 is frozen, using 52 convolutional layers as the backbone. Preferably, an improved version of DarkNet53 is used, and the data preprocessing mainly employs two data augmentation methods: Mosaic and Mixup. The Mosaic data augmentation method mainly stitches together parts of four images into one by randomly cropping, flipping, and splicing them. This method has good advantages for small targets such as product detection, enriching the background and increasing the diversity of samples. Mixup mainly involves filling two product images vertically and horizontally and then superimposing them. In summary, YoloX-Darknet53 can effectively detect small objects such as goods. After multiple iterations of training with millions of bounding boxes for goods, it can perform general goods detection with high recognition accuracy.

[0046] Step S103: Retrain the current product feature retrieval model based on the new product subgraph and the original product dataset to obtain a trained product feature retrieval model.

[0047] Among them, the product feature retrieval model is used to extract features from product sub-images and match the extracted product features with each product feature in the preset feature retrieval library to identify the product.

[0048] Specifically, the product feature retrieval model mainly includes feature extraction and feature matching, which can better identify the product sub-images obtained by the product detection model. Feature extraction involves converting the input image into a fixed-dimensional feature vector, and then matching this feature vector with various feature vectors in the feature retrieval database. When new products need to be added, the current product feature retrieval model can be retrained using the new product sub-image set obtained from the general detection model and the original product dataset, thereby improving the model's fitting ability and feature extraction effect for new products.

[0049] In some embodiments, the product feature retrieval model is a PaddleClas model, whose backbone network is PP-LCNet, and whose network layers are linear layers and feature pyramid layers. Specifically, PaddleClas is a model framework for image classification tasks, mainly including a backbone network for feature extraction, a detection head for predicting the type and location of the target, and a network layer between the backbone and the head for collecting feature maps at different stages, commonly referred to as the Neck. In this embodiment, the backbone of the PaddleClas model uses a lightweight CPU convolutional neural network PP-LCNet, and the Neck uses linear layers for feature dimension transformation and a feature pyramid network (FPN) for feature enhancement, achieving better feature enhancement and feature dimension transformation.

[0050] In some embodiments, before step S103, the method further includes: iteratively training the product feature retrieval model to be trained based on the original product dataset and the open-source dataset until the target loss function meets the preset convergence condition, thereby obtaining the current product feature retrieval model; wherein, the target loss function is determined based on the face recognition loss function and the ternary loss function.

[0051] Specifically, training is performed based on the PaddleClas network framework. Its backbone network, PP_LCNet, is pre-trained on an open-source dataset. The loss function uses both the ArcFace loss and Triplet loss functions for face recognition, and these are jointly trained and learned. In summary, the PaddleClas model is pre-trained on both the original product data and the open-source dataset. By optimizing the model network and adjusting parameters, it achieves a locally optimal result on the original product dataset, which is the current product feature retrieval model. Based on the current detection network model, features are extracted from the baseline images of the original products to obtain a pre-defined feature retrieval library.

[0052] Step S104: Extract product features from the new product reference image and the original product reference image based on the trained product feature retrieval model, so as to update the preset feature retrieval database.

[0053] Specifically, the new product reference image and the original product reference image are stored in the product base database. The trained product feature retrieval model can be used to extract features from each product in the product base database to obtain a feature retrieval database containing both new and old products.

[0054] In some embodiments, the method further includes: acquiring non-commodity features and adding the non-commodity features to the preset feature retrieval library. Specifically, the preset feature retrieval library includes not only commodity features but also some non-commodity features, thereby reducing misidentification of non-commodities during subsequent commodity identification and further improving identification accuracy.

[0055] The product addition method provided by the embodiments of the present invention involves acquiring new product video data and acquiring new product baseline images; extracting new product regions from multiple frames of images in the new product video data based on a pre-trained product detection model to obtain new product sub-images; retraining the current product feature retrieval model based on the new product sub-images and the original product dataset to obtain a trained product feature retrieval model, wherein the product feature retrieval model is used to extract features from the product sub-images and match the extracted product features with each product feature in a preset feature retrieval library to identify the product; extracting product features from the new product baseline image and the original product baseline image based on the trained product feature retrieval model to update the preset feature retrieval library; that is, when implementing product addition, the embodiments of the present invention only need to collect a small number of new products, retrain on the current feature retrieval model with a local optimum, and update the feature retrieval library, thus achieving rapid addition. Furthermore, by first acquiring a partial image containing only the product through the product detection model, and then inputting the partial image containing only the product into the feature retrieval model for product recognition, the accuracy and stability of product recognition are improved.

[0056] Figure 2 This is a flowchart illustrating a product identification method provided in an embodiment of the present invention. The executing entity is a product identification device, or an electronic device equipped with a product identification device. Figure 2 As shown, the product identification method includes:

[0057] Step S201: Obtain transaction video data.

[0058] Specifically, when a user selects goods from the dynamic visual display cabinet, the camera located on the dynamic visual display cabinet will capture the transaction video data corresponding to the entire transaction process and send the transaction video data to the execution subject of this embodiment.

[0059] Step S202: Extract the region of the product to be identified from the key frame image in the transaction video data based on the pre-trained product detection model to obtain the product sub-image.

[0060] Specifically, a product detection model is used to detect products in transaction video data and obtain product sub-images containing only product information.

[0061] Step S203: Input the product sub-image to be identified into the trained product feature retrieval model, extract features from the product sub-image to be identified, and match the extracted product features with each product feature in the preset feature retrieval library to identify the product and determine the corresponding product label.

[0062] Specifically, the product feature retrieval model extracts features from the product subgraph, and these features are then used to retrieve the products and their corresponding product tags from a pre-defined feature retrieval database. The feature retrieval employs a Hierarchical Navigable Small World graphs (HNSW) method, which uses a layered structure to divide edges into layers based on feature radii, making the average degree of each vertex constant across all layers. This graph indexing approach improves recall while reducing time complexity.

[0063] Step S204: Determine the cost of the goods based on the product label.

[0064] Specifically, the process involves obtaining product tags, determining the transaction based on transaction rules, and settling the payment for the goods.

[0065] In some embodiments, the preset commodity database includes various commodity features and non-commodity features; the step S203 of matching the extracted commodity features to be identified with the commodity features in the preset feature retrieval database includes: matching the extracted commodity features to be identified with the commodity features and non-commodity features in the preset feature retrieval database.

[0066] Specifically, to better distinguish "non-commodities," "non-commodity features" are added to the preset feature retrieval database. These non-commodity features are obtained by extracting features from background items in real transaction videos, mainly including mobile phones, clothing, portraits, and cabinets. Identifying "non-commodities" can significantly improve the accuracy of commodity identification. During feature retrieval, it effectively avoids misidentification of other external objects, further enhancing the accuracy of commodity identification.

[0067] The improved product recognition method in this embodiment acquires transaction video data; extracts the product region to be recognized from keyframe images in the transaction video data based on a pre-trained product detection model, obtaining a product sub-image; inputs the product sub-image to be recognized into a trained product feature retrieval model, extracts features from the product sub-image, and matches the extracted product features with product features in a preset feature retrieval library to identify the product and determine the corresponding product tag; determines the product cost based on the product tag; first, a partial image containing only the product is obtained through the product detection model, and then features are extracted and matched from the partial image containing only the product, further improving the accuracy and stability of product recognition.

[0068] Based on the above embodiments, Figure 3 This is a flowchart illustrating a product update and identification method provided in an embodiment of the present invention, as shown below. Figure 3 As shown, this includes the product launch process and the product identification process.

[0069] For the new product launch process, firstly, new product videos are acquired. This involves operations staff collecting video data of new products in three different scenarios and uploading it through the backend launch channel. Simultaneously, baseline images of the new products are acquired and uploaded to the product database. These are baseline images of the new products taken by operations staff from different angles or sides. Next, a product detection model is used to detect products in the new product videos, obtaining new product sub-images. Then, the product feature retrieval model is updated using these new product sub-images. This involves adding the new product sub-images to the existing product dataset and retraining the current product feature retrieval model. Simultaneously, the new product baseline images and existing product baseline images from the product database are input into the updated product feature retrieval model, resulting in a feature retrieval database containing both new and old products.

[0070] For the product identification process, firstly, a transaction video is acquired; then, a product detection model is used to identify products in the transaction video, obtaining product sub-images; next, an updated product feature retrieval model is used to extract features from the product sub-images and compare them with product features in the feature retrieval library for product identification. At the same time, to better distinguish "non-products," "non-product features" are added to the feature retrieval library. For products that can already be identified in the product database, the corresponding product tags are identified, and the product is charged through the settlement logic.

[0071] In summary, this embodiment achieves new product updates by combining product detection and feature retrieval. Only a small number of new product samples are needed to train the product feature retrieval model and update the feature search library, effectively solving the problem of long product update cycles and saving labor and time costs. Secondly, during product recognition, "non-product" features are added to the feature search library for feature matching, effectively reducing false identification of non-products, improving the accuracy of product recognition, and increasing the reliability of the dynamic vision cabinet. Furthermore, it should be noted that this embodiment utilizes purely visual technology for dynamic product recognition, which significantly reduces production costs and increases the utilization rate of cabinet space compared to previous traditional smart vending machines that combined dynamic and gravity-based technologies.

[0072] Figure 4 This is a schematic diagram of a product refill device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the product restocking device 400 includes:

[0073] The first product data module 401 is used to acquire new product video data and acquire new product reference images; the first product detection module 402 is used to extract new product regions from multiple frames of images in the new product video data based on a pre-trained product detection model to obtain new product sub-images; the first product feature retrieval module 403 is used to retrain the current product feature retrieval model based on the new product sub-images and the original product dataset to obtain a trained product feature retrieval model, wherein the product feature retrieval model is used to extract features from the product sub-images and match the extracted product features with each product feature in the preset feature retrieval library to identify the product; the feature retrieval update module 404 is used to extract product features from the new product reference image and the original product reference image based on the trained product feature retrieval model to update the preset feature retrieval library.

[0074] As an embodiment of the present invention, the product detection model is a YOLOX model, the backbone network of the YOLOX model is DarkNet53, and the last fully connected layer of DarkNet53 is in a frozen state.

[0075] As an embodiment of the present invention, the product feature retrieval model is a PaddleClas model, the backbone network of the PaddleClas model is PP_LCNet, and the network layers of the PaddleClas model include linear layers and feature pyramid layers.

[0076] As an embodiment of the present invention, the first commodity feature retrieval module 403 is further configured to: iteratively train the commodity feature retrieval model to be trained based on the original commodity dataset and the open-source dataset until the target loss function meets the preset convergence condition, thereby obtaining the current commodity feature retrieval model; wherein, the target loss function is determined according to the face recognition loss function and the ternary loss function.

[0077] As an embodiment of the present invention, the first product data module 401 is specifically used to: acquire new product video data under a preset scenario; wherein, the preset scenario includes at least one of the following: the new product rotates horizontally and moves up and down, the new product tilts and rotates and moves up and down, and the new product moves at a speed higher than a preset speed under a light intensity lower than a preset light intensity.

[0078] As an embodiment of the present invention, the first product data module 401 is specifically used to: acquire an image of a new product at a preset angle or a preset side view; wherein the preset angle and preset side view are determined according to the appearance type of the new product.

[0079] As an embodiment of the present invention, the feature retrieval and update module 404 is further configured to: acquire non-commodity features and add the non-commodity features to the preset feature retrieval library.

[0080] The product listing device provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0081] Figure 5 This is a schematic diagram of the structure of a product identification device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the product identification device 500 includes:

[0082] The second product data module 501 is used to acquire transaction video data; the second product detection module 502 is used to extract the product region to be identified from the key frame image in the transaction video data based on a pre-trained product detection model, and obtain a product sub-image to be identified; the second product feature retrieval module 503 is used to input the product sub-image to be identified into a trained product feature retrieval model, extract features from the product sub-image to be identified, and match the extracted product features to be identified with each product feature in a preset feature retrieval library to identify the product and determine the corresponding product label; the product cost determination module 504 is used to determine the product cost based on the product label.

[0083] As an embodiment of the present invention, the preset commodity database includes various commodity features and non-commodity features; the second commodity feature retrieval module 503 is specifically used to: match the extracted commodity features to be identified with the commodity features and non-commodity features in the preset feature retrieval database.

[0084] The product identification device provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0085] Figure 6 This is a schematic diagram of the structure of a product update and identification system provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the product update and identification system 600 includes:

[0086] The product data module 601 is used to acquire new product video data and a new product baseline image; the product detection module 602 is used to extract new product regions from multiple frames of images in the new product video data based on a trained product detection model, and obtain new product sub-images; the product feature retrieval module 603 is used to train the current product feature retrieval model based on the new product sub-images and the original product dataset images, and obtain a trained product feature retrieval model, wherein the product feature retrieval model is used to extract features from the product sub-images and match the extracted product features with each product feature in a preset feature retrieval library to identify the product; the feature retrieval update module 604 is used to extract the new product baseline image and the original product baseline image based on the trained product feature retrieval model. The product identification device includes: a product data module 601, which is further used to acquire transaction video data; a product detection module 602, which is further used to extract the product region to be identified from the key frame image in the transaction video data based on a trained product detection model, and obtain a product sub-image to be identified; a product feature retrieval module 603, which is further used to input the product sub-image to be identified into the trained product feature retrieval model, extract features from the product sub-image to be identified, and match the extracted product features to be identified with each product feature in the preset feature retrieval library to identify the product and determine the corresponding product label; and a product cost determination module 605, which is used to determine the product cost based on the product label.

[0087] Specifically, the commodity data module 601 mainly collects new commodity samples and subsequent commodity transaction videos, the commodity detection module 602 is used to detect commodity sub-images from image data, the commodity feature retrieval module 603 is used to extract features from commodity sub-images and match them in the commodity feature retrieval library, and the commodity cost determination module 605 is used to obtain commodity tags from the commodity feature retrieval library and combine them with settlement logic to perform commodity settlement.

[0088] The product update and identification system provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0089] This invention uses a general detection model combined with feature retrieval to perform new product launches and product identification, which greatly shortens the new product launch cycle. At the same time, compared with ordinary detection models, it improves the recognition accuracy, and the modularization of the model is more conducive to model updates, iterations and management.

[0090] like Figure 7 As shown, this embodiment of the invention provides an electronic device, including a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other via the communication bus 704.

[0091] Memory 703 is used to store computer programs;

[0092] In one embodiment of the present invention, when the processor 701 executes the program stored in the memory 703, it implements the steps of the product identification method or the product addition method provided in any of the foregoing method embodiments.

[0093] The electronic device provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0094] The aforementioned memory 703 can be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 703 has storage space for program code used to perform any of the method steps described above. For example, the storage space for program code may include individual program codes for implementing the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, optical discs (CDs), memory cards, or floppy disks. Such computer program products are typically portable or fixed storage units. The storage unit may have storage segments or storage spaces arranged similarly to memory 703 in the aforementioned electronic device. The program code may be compressed, for example, in a suitable form. Typically, the storage unit includes programs for performing the method steps according to embodiments of the invention, i.e., code that can be read by a processor such as 701, which, when run by the electronic device, causes the electronic device to perform the various steps in the methods described above.

[0095] Embodiments of the present invention also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the product identification method or the new product identification method described above.

[0096] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments; or it may exist independently and not assembled into the device / apparatus. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of the present invention.

[0097] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0098] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0099] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for launching new products, characterized in that, include: Acquire new product video data and acquire new product baseline images; Based on the trained product detection model, new product regions are extracted from multiple frames of images in the new product video data to obtain new product sub-images; The current product feature retrieval model is trained based on the new product sub-image and the original product dataset to obtain the trained product feature retrieval model. The product feature retrieval model is used to extract features from the product sub-image and match the extracted product features with each product feature in the preset feature retrieval library to identify the product. Based on the trained product feature retrieval model, product features are extracted from the new product reference image and the original product reference image to update the preset feature retrieval database.

2. The method according to claim 1, characterized in that, The product detection model is a YOLOX model, the backbone network of the YOLOX model is DarkNet53, and the last fully connected layer of DarkNet53 is frozen.

3. The method according to claim 1, characterized in that, The product feature retrieval model is the PaddleClas model, the backbone network of the PaddleClas model is PP_LCNet, and the network layers of the PaddleClas model include linear layers and feature pyramid layers.

4. The method according to claim 3, characterized in that, Before training the current product feature retrieval model based on the new product subgraph and the original product dataset, the following steps are also included: The product feature retrieval model to be trained is iteratively trained based on the original product dataset and the open-source dataset until the target loss function meets the preset convergence condition, thus obtaining the current product feature retrieval model. The target loss function is determined based on the face recognition loss function and the ternary loss function.

5. The method according to any one of claims 1-4, characterized in that, The acquisition of new product video data includes: Acquire new product video data in preset scenarios; The preset scenarios include at least one of the following: the new product rotates horizontally and moves up and down, the new product tilts and rotates and moves up and down, and the new product moves at a speed higher than a preset speed under a light intensity lower than a preset value.

6. The method according to any one of claims 1-4, characterized in that, The acquisition of the new product baseline image includes: Get images of new products from preset angles or from preset sides; The preset angle and preset side are determined according to the appearance type of the new product.

7. The method according to any one of claims 1-4, characterized in that, The method further includes: Obtain non-commodity features and add them to the preset feature retrieval library.

8. A product restocking device, characterized in that, include: The first product data module is used to acquire new product video data and new product baseline images; The first product detection module is used to extract new product regions from multiple frames of images in the new product video data based on the trained product detection model, and obtain new product sub-images. The first product feature retrieval module is used to train the current product feature retrieval model based on the new product sub-image and the original product dataset image to obtain the trained product feature retrieval model. The product feature retrieval model is used to extract features from the product sub-image and match the extracted product features with each product feature in the preset feature retrieval library to identify the product. The feature retrieval update module is used to extract product features from the new product reference image and the original product reference image based on the trained product feature retrieval model, so as to update the preset feature retrieval library.

9. A product identification and new product update system, characterized in that, include: The product data module is used to acquire new product video data and new product baseline images; The product detection module is used to extract new product regions from multiple frames of images in the new product video data based on a trained product detection model, and obtain new product sub-images; The product feature retrieval module is used to train the current product feature retrieval model based on the new product sub-graph and the original product dataset graph, so as to obtain the trained product feature retrieval model. The feature retrieval and update module is used to extract product features from the new product reference image and the original product reference image based on the trained product feature retrieval model, so as to update the preset feature retrieval library; The commodity data module is also used to acquire transaction video data; The product detection module is also used to extract the product region to be identified from the key frame image in the transaction video data based on the trained product detection model, and obtain the product sub-image to be identified. The product feature retrieval module is also used to input the product sub-image to be identified into the trained product feature retrieval model, extract features from the product sub-image to be identified, and match the extracted product features to be identified with each product feature in the preset feature retrieval library to identify the product and determine the corresponding product tag. The product cost determination module is used to determine the product cost based on the product label.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the new method for product as described in any one of claims 1-7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of any one of the product addition methods as claimed in claims 1-7.

Images