Commodity reselling detection method and device and electronic equipment

Abstract

The invention provides a commodity reselling detection method and apparatus, and an electronic device. The method comprises the steps of obtaining microscopic pictures of a plurality of parts of a to-be-detected commodity; performing feature extraction on the microscopic images of the plurality of parts through a neural network model to obtain a fingerprint feature vector of the to-be-detected commodity; and the fingerprint feature vector of the to-be-detected commodity is compared with a fingerprint feature vector in a historical commodity order library to determine whether the to-be-detected commodity is sold in the historical commodity order library, and the fingerprint feature vector of the sold commodity is stored in the historical commodity order library. In the implementation process of the scheme, the fingerprint feature vector of the commodity is extracted from the part microcosmic picture by using the neural network model, and the fingerprint feature vector is compared with the fingerprint feature vector in the historical commodity library storing the sold commodity; the accuracy of commodity reselling detection can be improved in combination with the neural network model and the microcosmic pictures of the multiple parts.

Description

Technical Field

[0001] The present application relates to the technical fields of computer vision, image processing and machine learning, and more specifically, to a method, device and electronic device for detecting resale of goods. Background Art

[0002] At present, the identification of whether a product has been sold repeatedly is mostly done by traditional technology, which usually relies on identification methods such as barcodes, QR codes or RFID tags to track product information. However, these traditional technologies have certain limitations. For example, labels are easy to be forged, damaged or lost, and physical labels may become ineffective due to wear and tear, or even fall off the product completely, thus losing their due function. In this case, it is difficult to identify whether the product has been sold repeatedly. In the field of e-commerce, it is very important to ensure the uniqueness and uniqueness of products, especially for products with unique value such as second-hand products, limited edition products or works of art. If the same product is sold to different buyers multiple times, this will not only harm the rights and interests of consumers, but also may undermine the trust and reputation of the platform. Therefore, the current accuracy of product resale detection is low. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method, device and electronic device for detecting resale of goods, so as to improve the problem of low accuracy in detecting resale of goods.

[0004] The embodiment of the present application provides a method for detecting the resale of goods, including: obtaining microscopic images of multiple parts of the goods to be detected, the microscopic images are obtained by taking pictures with an image acquisition device whose magnification exceeds the multiple threshold; extracting features from the microscopic images of multiple parts through a neural network model to obtain a fingerprint feature vector of the goods to be detected; comparing the fingerprint feature vector of the goods to be detected with the fingerprint feature vector in the historical goods order library to determine whether the goods to be detected have been sold in the historical goods order library, and the historical goods order library stores the fingerprint feature vectors of the sold goods. In the implementation process of the above scheme, the microscopic images of multiple parts of the goods can capture the details of the goods that are difficult to detect with the naked eye. These details often contain microscopic features such as product texture, scratches and / or wear marks, which are very critical for distinguishing different individual goods. Further, the fingerprint feature vector of the goods is extracted from the microscopic images of the parts using a neural network model, and these fingerprint feature vectors can characterize this microscopic feature, so the fingerprint feature vector is compared with the fingerprint feature vector in the historical goods library storing the sold goods, and it can be determined whether the goods to be detected have been sold by orders in the historical goods library. Therefore, the above solution overcomes the defect that labels in traditional technology are easily forged, damaged or lost, and effectively combines the neural network model and microscopic images of multiple parts to improve the accuracy of product resale detection.

[0005] Optionally, in an embodiment of the present application, before extracting features from microscopic images of multiple parts through a neural network model, it also includes: obtaining training samples and training labels, the training samples including: microscopic images of multiple parts of a sample product and the magnification of the microscopic image of each part, and the training labels including: category labels and order labels of the sample product; predicting the sample category of the sample product based on the multiple microscopic images of the parts and the magnification of the microscopic image of each part through a neural network model; calculating the product classification loss between the sample category of the sample product and the category label of the sample product; extracting features from the microscopic images of multiple parts through a neural network model to obtain a fingerprint feature vector of the sample product, and predicting a sample order corresponding to the sample product based on the fingerprint feature vector of the sample product; calculating the order identity recognition loss between the sample order of the sample product and the order label of the sample product; and updating the parameters of the neural network model according to the product classification loss and the order identity recognition loss. In the implementation of the above solution, by performing product classification and order identity recognition tasks simultaneously, the neural network model can learn richer feature representations during the training process. These features can not only distinguish between different categories of products, but also capture the subtle differences between different products in the same category. Multi-task learning helps to improve the generalization ability of the model because the model needs to meet the requirements of multiple tasks at the same time, which makes the model more robust to changes in input data, thereby improving the generalization ability of the model and the accuracy of product resale detection.

[0006] Optionally, in an embodiment of the present application, the order identity recognition loss between the sample order of the sample product and the order label of the sample product is calculated, including: obtaining a first similarity score of a product of the same category that is similar to the sample order of the sample product from a historical product order library, and a second similarity score of a product of a different category that is similar to the sample order of the sample product; and calculating the order identity recognition loss according to the magnification of the microscopic image of the part, the first similarity score, and the second similarity score. In the implementation process of the above scheme, by combining microscopic images and similarity scores of different magnifications, the model can more accurately learn the subtle features of the product surface. These subtle features are particularly important for distinguishing different products in the same category, and microscopic images of different magnifications provide multi-level feature information, which helps the model to better capture the overall structure and local details of the product, and effectively combines fine-grained feature learning and multi-scale feature fusion to enhance the model's differentiation ability, thereby improving the accuracy of product resale detection.

[0007] Optionally, in an embodiment of the present application, the fingerprint feature vector of the commodity to be detected is compared with the fingerprint feature vector in the historical commodity order library to determine whether the commodity to be detected has been sold in the historical commodity order library, including: judging whether a historical order whose similarity with the fingerprint feature vector of the commodity to be detected is greater than a preset threshold is queried in the historical commodity order library; if so, it is determined that the commodity to be detected has been sold in the historical commodity order library, otherwise, it is determined that the commodity to be detected has not been sold in the historical commodity order library. In the implementation process of the above scheme, the fingerprint feature vector is used to calculate the similarity, and matching items can be quickly retrieved in a large-scale historical commodity order library, which is much faster than the traditional image-based one-by-one comparison method. By setting a preset threshold to judge the similarity, the occurrence of false positives and false negatives can be effectively reduced. Only when the similarity exceeds the threshold, the commodity is considered to have been sold, thereby improving the accuracy of the detection.

[0008] Optionally, in an embodiment of the present application, after determining that the commodity to be detected has been sold in the historical commodity order library, it also includes: sending resale prompt information of the commodity to be detected, and the resale prompt information includes: order number, fingerprint feature vector, detection time and / or prompt note information. In the implementation process of the above scheme, by including information such as order number, fingerprint feature vector, detection time, etc., the resale prompt information provides a detailed record, which makes each resale detection traceable and enhances the transparency of the system.

[0009] Optionally, in an embodiment of the present application, after determining that the product to be detected has not been sold in the historical product order library, it also includes: generating new order information based on microscopic images of multiple parts of the product to be detected, fingerprint feature vectors, product categories and styles and models; storing the new order information in the historical product order library. In the implementation process of the above scheme, by combining multiple types of data (images, feature vectors, text information, etc.), the system can describe and identify products from multiple dimensions, thereby improving the diversity and practicality of the data.

[0010] The embodiment of the present application also provides a commodity resale detection device, including: a microscopic image acquisition module, used to obtain microscopic images of multiple parts of the commodity to be detected, the microscopic images are obtained by taking pictures using an image acquisition device with a magnification factor exceeding a multiple threshold; an image feature extraction module, used to extract features from the microscopic images of multiple parts through a neural network model to obtain a fingerprint feature vector of the commodity to be detected; a commodity resale determination module, used to compare the fingerprint feature vector of the commodity to be detected with the fingerprint feature vector in a historical commodity order library to determine whether the commodity to be detected has been sold in the historical commodity order library, and the historical commodity order library stores the fingerprint feature vectors of the sold commodities.

[0011] Optionally, in an embodiment of the present application, the commodity resale detection device further includes: a sample label acquisition module, used to acquire training samples and training labels, the training samples include: microscopic images of multiple parts of the sample commodity and the magnification of the microscopic image of each part, and the training labels include: category labels and order labels of the sample commodity; a sample category prediction module, used to predict the sample category of the sample commodity according to the microscopic images of multiple parts and the magnification of the microscopic image of each part through a neural network model; a classification loss calculation module, used to calculate the commodity classification loss between the sample category of the sample commodity and the category label of the sample commodity; a feature vector acquisition module, used to extract features from the microscopic images of multiple parts through a neural network model, obtain the fingerprint feature vector of the sample commodity, and predict the sample order corresponding to the sample commodity according to the fingerprint feature vector of the sample commodity; an identification loss calculation module, used to calculate the order identity recognition loss between the sample order of the sample commodity and the order label of the sample commodity; a model parameter updating module, used to update the parameters of the neural network model according to the commodity classification loss and the order identity recognition loss.

[0012] Optionally, in an embodiment of the present application, the identification loss calculation module includes: a similarity score acquisition submodule, used to obtain a first similarity score of a product of the same category that is similar to the sample order of the sample product from a historical product order library, and a second similarity score of a product of a different category that is similar to the sample order of the sample product; an order identity recognition submodule, used to calculate the order identity recognition loss based on the magnification of the microscopic image of the part, the first similarity score and the second similarity score.

[0013] Optionally, in an embodiment of the present application, a commodity resale determination module includes: a fingerprint feature query submodule, used to determine whether a historical order whose similarity between the fingerprint feature vector of the commodity to be detected is greater than a preset threshold is queried in the historical commodity order library; a resale determination submodule, used to determine that the commodity to be detected has been sold in the historical commodity order library if a historical order whose similarity between the fingerprint feature vector of the commodity to be detected is greater than the preset threshold is queried in the historical commodity order library; if no historical order whose similarity between the fingerprint feature vector of the commodity to be detected is greater than the preset threshold is queried in the historical commodity order library, then determine that the commodity to be detected has not been sold in the historical commodity order library.

[0014] Optionally, in an embodiment of the present application, the commodity resale determination module further includes: a resale prompt sending submodule, which is used to send resale prompt information of the commodity to be detected, and the resale prompt information includes: order number, fingerprint feature vector, detection time and / or prompt note information.

[0015] Optionally, in an embodiment of the present application, the commodity resale determination module also includes: an order information generation submodule, used to generate new order information based on microscopic images of multiple parts of the commodity to be detected, fingerprint feature vectors, commodity categories and styles and models; an order information storage submodule, used to store the new order information in a historical commodity order library.

[0016] An embodiment of the present application further provides an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions execute the method described above when executed by the processor.

[0017] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is executed.

[0018] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of the process of a method for detecting resale of goods provided in an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of a process of querying in a historical commodity order library provided by an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of the structure of a commodity resale detection device provided in an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the embodiment of the present application only serve the purpose of explanation and description, and are not used to limit the protection scope of the embodiment of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in the embodiment of the present application shows the operation implemented according to some embodiments of the embodiment of the present application. It should be understood that the operation of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the embodiment of the present application, or remove one or more operations from the flowchart.

[0025] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments. The components of the embodiments of the present application described and shown in the drawings generally here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the embodiments of the present application claimed, but merely represents the selected embodiments of the embodiments of the present application.

[0026] It is understandable that the "first" and "second" in the embodiments of the present application are used to distinguish similar objects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not necessarily limit the difference. In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that there may be three relationships, such as A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship. The term "multiple" refers to more than two (including two), and similarly, "multiple groups" refers to more than two groups (including two groups).

[0027] It should be noted that the commodity resale detection method provided in the embodiment of the present application can be executed by an electronic device, where the electronic device refers to a device terminal with the function of executing a computer program or the above-mentioned server, and the device terminal is, for example: a smart phone, a personal computer, a tablet computer, a personal digital assistant or a mobile Internet device, etc.

[0028] The following is an example of an application scenario in which the product resale detection method is applicable: On an e-commerce platform, sellers may try to resell products that have already been sold. If the same product is sold to different buyers multiple times, this will not only harm the rights of consumers, but also may damage the trust and reputation of the platform. In order to maintain the trust and reputation of the platform, the product resale detection method can be used to automatically detect and prevent the same product from being sold multiple times, maintain buyers' trust and rights in the platform, etc.

[0029] See also Figure 1 The flowchart of the method for detecting resale of goods provided by the embodiment of the present application is shown; the main idea of ​​the method for detecting resale of goods is that microscopic images of multiple parts of a product can capture details of the product that are difficult to detect with the naked eye. These details often include microscopic features such as product texture, scratches and / or wear marks. The fingerprint feature vectors of the product are extracted from the microscopic images of the parts using a neural network model, and these fingerprint feature vectors can characterize such microscopic features. Therefore, the accuracy of detecting resale of goods can be improved by combining the neural network model with microscopic images of multiple parts. The implementation methods of the above-mentioned method for detecting resale of goods may include:

[0030] Step S110: obtaining microscopic images of multiple parts of the commodity to be inspected, wherein the microscopic images are obtained by photographing with an image acquisition device having a magnification exceeding a magnification threshold.

[0031] The goods to be tested are those that need to be tested to see if they are resold by merchants. The goods to be tested here can be general goods, that is, the goods to be tested include all categories of goods such as shoes, clothing, luggage, watches, accessories, etc. They can be high-frequency goods or low-frequency goods, and not just goods that have been trained by the neural network model, but also goods that have not been trained by the neural network model.

[0032] Microscopic images, also known as microscopic images, are images taken with an image acquisition device that has a magnification exceeding a threshold value. Compared to images taken with ordinary cameras or camera lenses used in daily life (the optical zoom of digital cameras is mostly between 2x and 5x), image acquisition devices for microscopic images include: cameras with magnifying glasses, cameras with microscopes, industrial cameras built into closed boxes, and / or handheld portable microscopic devices. Therefore, the details of the surface of a commodity or object can be magnified to a threshold value (e.g., 20x, 50x, or even 100x), so that even if there are only slight differences on the surface, they can be captured by the neural network model. After the microscopic image is magnified to a value exceeding the threshold value, the magnified microscopic image can often capture details of the commodity that are difficult to detect with the naked eye. These details often include microscopic features such as the texture, scratches, and / or wear marks of the commodity.

[0033] It is understandable that the above-mentioned multiple microscopic images of parts are taken at different locations according to the different products. For example, if the product is a mobile phone, microscopic images of the edge of the screen are usually taken to extract features such as scratches, wear or cracks on the edge of the screen. Buttons and interfaces, such as volume keys, charging ports, etc., are also photographed because these parts are prone to wear. If the product is a laptop computer, the keyboard part is usually photographed to extract the wear features of the key surface, and the touchpad part is also photographed to extract features such as scratches or wear on the surface. The buttons and interfaces of the laptop computer, such as volume keys, charging ports, etc., are also photographed because these parts are prone to wear. Similarly, if the product is jewelry, such as a ring, the inner ring engraving is usually photographed to extract the detailed features of the internal engraving or numbering. According to the recognition granularity of different products or the different product materials or materials to be distinguished, different magnification lenses (for example, 20 times, 50 times or 100 times) can be replaced according to needs.

[0034] In the implementation of the above step S110, for example, the electronic device can directly receive the multiple microscopic images of parts sent by the image acquisition device, or the electronic device can also read the multiple microscopic images of parts from a storage medium (such as a hard disk or an optical drive) that has stored the multiple microscopic images of parts. Alternatively, after the image acquisition device acquires the multiple microscopic images of parts, it sends the multiple microscopic images of parts to an object storage service (OSS), and the electronic device can directly obtain the multiple microscopic images of parts from the OSS through the interface provided by the OSS.

[0035] Step S120: extracting features from microscopic images of multiple parts through a neural network model to obtain a fingerprint feature vector of the product to be detected.

[0036] The neural network model refers to an artificial neural network model that extracts features from microscopic images of parts. Since the neural network model is used to identify the order identity to which the goods belong, it can also be called an order identity recognition model or an order identity authentication model. The neural network model can use a pre-trained image model, such as a MobileNet model, a ShuffleNet model, or a model developed based on Vision Transformer (ViT).

[0037] Fingerprint Feature Vector, also known as universal product fingerprint, is a feature vector for feature extraction of product appearance. This feature vector can be used as the credential information fingerprint of the product in the subsequent transaction and circulation process, so it is called fingerprint feature vector. The dimension of the above fingerprint feature vector can use the same fixed length for vectors of all categories of products, or different fixed lengths for vectors of different types of products. It can be selected according to specific circumstances, for example, any fixed length in 64 / 128 / 256 / 512 dimensions can be selected as the dimension of the fingerprint feature vector of a certain product.

[0038] An implementation method of the above-mentioned step S120 is, for example: inputting the microscopic images of multiple parts into a neural network model such as a MobileNet model, a ShuffleNet model, or a model developed based on Vision Transformer (ViT), so that the neural network model extracts features from the microscopic images of multiple parts to obtain fingerprint feature vectors of the product to be detected.

[0039] Step S130: Compare the fingerprint feature vector of the commodity to be detected with the fingerprint feature vector in the historical commodity order library to determine whether the commodity to be detected has been sold in the historical commodity order library, where the historical commodity order library stores the fingerprint feature vectors of sold commodities.

[0040] The historical commodity order library refers to a library that stores order information of historical commodities that have been sold. The historical commodity order library can be implemented by using vector databases such as Simsvr, Hologress or Milvus. It is understandable that when comparing and analyzing and querying in the historical commodity order library, in addition to passing in the fingerprint feature vector of the commodity to be detected, other data information can also be passed in to speed up the query, such as the commodity category (such as mobile phone) and product model (such as X200 Pro) fields that have been indexed in the historical commodity order library.

[0041] In the process of implementing the above scheme, the microscopic images of multiple parts of the goods can capture the details of the goods that are difficult to detect with the naked eye. These details often include microscopic features such as the texture, scratches and / or wear marks of the goods, which are very critical for distinguishing different individual goods. Furthermore, the fingerprint feature vectors of the goods are extracted from the microscopic images of the parts using a neural network model, and these fingerprint feature vectors can characterize such microscopic features. Therefore, the fingerprint feature vectors are compared with the fingerprint feature vectors in the historical commodity library that stores the sold goods, so it can be determined whether the goods to be detected have been sold by orders in the historical commodity library. Therefore, the above scheme overcomes the defects of traditional technology that labels are easily forged, damaged or lost, and effectively combines the neural network model and microscopic images of multiple parts to improve the accuracy of commodity resale detection.

[0042] As an optional implementation of the above-mentioned commodity resale detection method, before extracting features from microscopic images of multiple parts through a neural network model, the neural network model can also be trained by multi-task learning. The training method of this multi-task learning can include:

[0043] Step S101: obtaining training samples and training labels, wherein the training samples include: microscopic images of multiple parts of a sample product and the magnification of the microscopic image of each part, and the training labels include: category labels and order labels of the sample product.

[0044] The implementation method of the above step S101 is, for example: downloading a training data set from the Internet, and obtaining training samples and training labels from the training data set. Of course, training samples and training labels can also be prepared from scratch, for example, taking pictures of multiple parts of the sample product, obtaining microscopic images of multiple parts of the sample product and the magnification of each part of the microscopic image, and determining the multiple parts of the sample product and the magnification of each part of the microscopic image as training samples, and then labeling the sample product to obtain the category label and order label of the sample product.

[0045] Step S102: predicting the sample category of the sample commodity according to the microscopic images of multiple parts and the magnification of each microscopic image of the part through a neural network model.

[0046] An implementation method of the above-mentioned step S102 is, for example: inputting multiple microscopic images of parts and the magnification of each microscopic image of part into a neural network model such as a MobileNet model, a ShuffleNet model, or a model developed based on Vision Transformer (ViT), so that the neural network model outputs the sample category of the sample product.

[0047] Step S103: Calculate the product classification loss between the sample category of the sample product and the category label of the sample product.

[0048] The implementation method of the above step S103 is, for example, to use the formula and L FL (p t ) = -a t (1-p t ) r -log(p t ) calculates the sample category of the sample product and the category label of the sample product to obtain the product classification loss between the sample category of the sample product and the category label of the sample product. t represents the correct probability of the sample category of the tth sample product, L FL (pt ) represents the classification loss of commodity types determined by FocalLoss loss, r is used to control the difficulty of training different types. The larger the r value, the smaller the loss of easy-to-classify sample commodities, so that the neural network model will turn its attention to difficult-to-classify sample commodities, and a t Represents the prior loss weight of sample categories for different sample products.

[0049] Step S104: extract features from the microscopic images of multiple parts through a neural network model to obtain a fingerprint feature vector of the sample product, and predict a sample order corresponding to the sample product based on the fingerprint feature vector of the sample product.

[0050] The implementation method of the above step S104 is, for example: inputting the microscopic images of multiple parts into a neural network model such as a MobileNet model, a ShuffleNet model, or a model developed based on a Vision Transformer (ViT), so that the neural network model extracts features from the microscopic images of multiple parts to obtain a fingerprint feature vector of the sample product. After obtaining the fingerprint feature vector of the sample product, the fingerprint feature vector of the sample product can be compared and queried with the fingerprint feature vector in the historical product order library. Specifically, if a historical order whose similarity with the fingerprint feature vector of the sample product is greater than a preset threshold is queried in the historical product order library, the historical order whose similarity is greater than the preset threshold is determined as the sample order corresponding to the sample product.

[0051] Step S105: Calculate the order identity recognition loss between the sample order of the sample product and the order tag of the sample product.

[0052] The first implementation of step S105 is, for example, to use the formula To calculate the order identification loss between the sample order of the sample product and the order label of the sample product. CL represents the order identification loss determined by the improved Circle Loss loss, It represents the similarity score when there are L products of the same category in a certain order that are similar to the current product. It represents the similarity score when there are K products of different categories that are similar to the current product at the same time, t represents the magnification of a single order photo taken with a microscopic device (as a magnification factor), and m is the empirical coefficient for separating products of the same category, which is a hyperparameter and can be set with different empirical values ​​according to products of different categories. It can be understood that since the order identification loss between the sample order of the sample product and the order label of the sample product is calculated, the larger the magnification factor, the more accurate the matching result. Therefore, the magnification of the microscopic device is introduced as the magnification factor in the above-mentioned order identification loss, so that the larger the magnification factor in the same order, the smaller the order identification loss.

[0053] Step S106: Update the parameters of the neural network model according to the commodity classification loss and the order identity recognition loss until the accuracy of the neural network model converges, and then obtain the trained neural network model.

[0054] The implementation method of the above step S106 is, for example, to use the formula The product classification loss and order identity recognition loss are calculated to obtain the total loss value, and the parameters of the neural network model are updated according to the total loss value until the accuracy of the neural network model converges, and the trained neural network model can be obtained. mtl Represents the total loss value of the overall optimization objective of multi-task learning, represents the i-th different subtask loss in multi-task learning, which can be product classification loss or order identity recognition loss, w i represents the weight of the ith different subtask in multi-task learning. The sum of the weights of all subtasks in multi-task learning is fixed to 1, that is, L FL (p t ) represents the commodity type classification loss determined by FocalLoss loss, L CL It represents the order identity recognition loss determined by the improved Circle Loss loss. It can be understood that the weights of different subtasks in multi-tasks can be evenly divided according to the number of subtasks, or the weight of each subtask can be set according to experience.

[0055] In the implementation of the above solution, by performing product classification and order identity recognition tasks simultaneously, the neural network model can learn richer feature representations during the training process. These features can not only distinguish between different categories of products, but also capture the subtle differences between different products in the same category. Multi-task learning helps to improve the generalization ability of the model because the model needs to meet the requirements of multiple tasks at the same time, which makes the model more robust to changes in input data, thereby improving the generalization ability of the model and the accuracy of product resale detection.

[0056] Furthermore, by magnifying microscopic images at high magnification and the magnification information of each part, the model can more accurately extract subtle features on the surface of the product, which are crucial for resale detection because they can uniquely identify each product. Therefore, by combining microscopic images with different magnifications, the model can learn multi-level feature representations to better capture the overall structure and local details of the product, effectively enhancing the model's feature extraction ability, thereby improving the accuracy of product resale detection. In addition, by combining product classification and order identity recognition tasks, the model can generate more unique fingerprint feature vectors, which can more accurately identify each product, thereby improving the reliability of resale detection. Moreover, by calculating the product classification loss and order identity recognition loss and updating the model parameters based on these two losses, it can ensure that the model performs well on both tasks. This joint optimization method helps to improve the accuracy of classification. Since the model takes into account both product category and order identity, it can reduce misclassification caused by subtle differences. For example, even if two products look very similar but belong to different orders, the model can accurately distinguish them, thereby further improving the accuracy of product resale detection.

[0057] As a second optional implementation of the above step S105, calculating the order identity recognition loss between the sample order of the sample product and the order tag of the sample product includes:

[0058] Step S105a: obtaining from the historical product order library a first similarity score of products of the same category that are similar to the sample order of the sample product, and a second similarity score of products of different categories that are similar to the sample order of the sample product.

[0059] Step S105b: Calculate the order identity recognition loss according to the magnification of the microscopic image of the part, the first similarity score and the second similarity score.

[0060] The implementation method of the above step S105b is, for example: according to the magnification of the microscopic image of the part, the first similarity score and the second similarity score, the magnification of the microscopic image of the part, the first similarity score and the second similarity score are weighted and calculated to obtain the order identification loss. It can be understood that by considering the similarity of goods in different categories, the model can more effectively avoid misidentifying goods in different categories as the same order, thereby reducing the misjudgment, and by calculating the first similarity score of goods in the same category similar to the sample goods and the second similarity score of goods in different categories, the model can better understand the relationship between goods. This weighted method based on similarity makes the model more accurate in order identification.

[0061] In the implementation process of the above solution, by combining microscopic images with different magnifications and similarity scores, the model can more accurately learn the subtle features of the product surface. These subtle features are particularly important for distinguishing different products in the same category. In addition, microscopic images with different magnifications provide multi-level feature information, which helps the model to better capture the overall structure and local details of the product. It effectively combines fine-grained feature learning and multi-scale feature fusion to enhance the model's distinguishing ability, thereby improving the accuracy of product resale detection.

[0062] See also Figure 2 The schematic diagram of the process of querying in the historical commodity order library provided by the embodiment of the present application is shown; as an optional implementation of the above step S130, the above implementation of comparing the fingerprint feature vector of the commodity to be detected with the fingerprint feature vector in the historical commodity order library to determine whether the commodity to be detected has been sold in the historical commodity order library may include:

[0063] Step S131: determine whether a historical order whose similarity with the fingerprint feature vector of the product to be detected is greater than a preset threshold is found in the historical product order database.

[0064] It is understandable that in addition to passing in the fingerprint feature vector of the product to be detected, other data information can also be passed in to speed up the query speed, such as the indexed product category (such as mobile phone) and product model (such as X200 Pro) fields in the historical product order library. These two fields can be established as two-layer index rules. If the product category and product model are passed in, you can first query the historical product order library based on the product category and product model. Since these two fields have been indexed, the query speed will be very fast, while the comparison speed of the fingerprint feature vector is slower. Therefore, reducing the number of fingerprint feature vectors that need to be compared can effectively improve the overall query speed.

[0065] The implementation method of the above step S131 is, for example: assuming that the fingerprint feature vector of the commodity to be detected in the current order is represented by feature = [f1, f2, ..., fn], the dimension of the fingerprint feature vector of the commodity to be detected here is N (such as 64, 128, 512 or 1024, etc.), and the database set in the historical commodity order library can be represented as cluster = {c1, c2, .... cm}, that is, the historical commodity order library contains m accumulated fingerprint feature vectors of candidate commodities, and the dimension of the fingerprint feature vector of each candidate commodity is the same as the dimension of the fingerprint feature vector of the commodity to be detected above. The historical commodity order library can be implemented by using vector databases such as Simsvr, Hologress or Milvus, which can support large-scale candidate set query statistics, thereby improving the query speed. The vector database calculates the characteristic distance between the fingerprint feature vector queried in the historical product order library and the fingerprint feature vector of the product to be detected, and determines the similarity set S = [s1, s2, ...., sm] between the fingerprint feature vector queried in the historical product order library and the fingerprint feature vector of the product to be detected based on the characteristic distance. The above characteristic distance can be cosine distance or Euclidean distance.

[0066] Step S132: If a historical order whose similarity with the fingerprint feature vector of the commodity to be detected is greater than a preset threshold is found in the historical commodity order library, it is determined that the commodity to be detected has been sold in the historical commodity order library.

[0067] An example of an implementation method of the above-mentioned step S132 is: assuming that the similarity set between the fingerprint feature vector queried from the historical commodity order library and the fingerprint feature vector of the commodity to be detected is expressed as S = [s1, s2, ...., sm], and the preset threshold is T. If there is an element [s1, s2, ...., sm] in the vector S that is greater than the preset threshold T, a historical order whose similarity with the fingerprint feature vector of the commodity to be detected is greater than the preset threshold is queried from the historical commodity order library, indicating that the order is a resale order, and it can be determined that the commodity to be detected has been sold in the historical commodity order library.

[0068] Step S133: If no historical order having a similarity with the fingerprint feature vector of the commodity to be detected greater than a preset threshold is found in the historical commodity order library, it is determined that the commodity to be detected has not been sold in the historical commodity order library.

[0069] An example of an implementation method of the above-mentioned step S133 is: assuming that the similarity set between the fingerprint feature vector queried from the historical product order library and the fingerprint feature vector of the product to be detected is expressed as S = [s1, s2, ...., sm], and the preset threshold is T. If all elements [s1, s2, ...., sm] in the vector S are less than or equal to the preset threshold T, it means that the product to be detected in the current order and all orders have not been successfully matched, that is, no historical order with a similarity between the fingerprint feature vector of the product to be detected and the fingerprint feature vector greater than the preset threshold is queried in the historical product order library. It can be determined that the product to be detected has not been sold in the historical product order library.

[0070] As an optional implementation of the above step S130, after determining that the commodity to be detected has been sold in the historical commodity order library, the following is further included:

[0071] Step S134: Send resale reminder information of the commodity to be detected, where the resale reminder information includes: order number, fingerprint feature vector, detection time and / or reminder note information.

[0072] The implementation method of the above step S134 is, for example: the electronic device sends the resale prompt information of the commodity to be detected, and the resale prompt information includes: order number (also known as order number), commodity category (also known as commodity category), style model, fingerprint feature vector, detection time and / or prompt note information, etc. The above order number can be a historical order number queried in the historical commodity order library with a similarity between the fingerprint feature vector of the commodity to be detected greater than a preset threshold. It can be understood that the above order number (also known as order number), commodity category (also known as commodity category), style model, and fingerprint feature vector can be obtained from a historical order queried in the historical commodity order library with a similarity between the fingerprint feature vector of the commodity to be detected greater than a preset threshold, that is, obtained from the matched historical order information. Optionally, the resale prompt information of the commodity to be detected can also be sent to the on-site staff to let the on-site staff know that the commodity to be detected is a circulating commodity for secondary sales, rather than a commodity sold for the first time.

[0073] In the implementation process of the above scheme, the resale prompt information provides detailed records by including information such as order number, fingerprint feature vector, detection time, etc., which makes each resale detection traceable and enhances the transparency of the system. This information can be used for internal audit and external supervision to ensure the compliance and fairness of system operation. When a resale event needs to be investigated, relevant records can be found quickly. Furthermore, the resale prompt information can be sent to relevant managers or systems in real time, enabling them to take prompt action. This instant notification mechanism helps to prevent resale behavior in a timely manner and reduce potential losses. By timely discovering and handling resale situations, the accuracy of inventory data can be ensured, which is very important for inventory management and supply chain planning, avoiding inventory errors caused by repeated sales. Timely discovery of resale problems can help companies better manage inventory, reduce inventory backlogs caused by misselling, and improve inventory turnover. In addition, by preventing resale and ensuring that each product is sold only once, it can effectively protect the rights and interests of consumers, enhance customer trust, and avoid returns and complaints caused by resale, improve customer shopping experience and satisfaction, and thus enhance the reputation of the brand.

[0074] As an optional implementation of the above step S130, after determining that the commodity to be detected has not been sold in the historical commodity order library, the following further includes:

[0075] Step S135: Generate new order information based on the microscopic images of multiple parts of the product to be detected, the fingerprint feature vector, the product category and the style and model.

[0076] It is understandable that the new order information mentioned above not only contains basic product categories and styles and models, but also includes detailed microscopic images and fingerprint feature vectors. These rich data provide a comprehensive information record for each order, enhancing the integrity and detail of the data. Detailed new order information can help companies manage inventory more accurately and ensure that the status and location of each item are accurately recorded, which helps avoid inventory errors and duplicate sales. Moreover, by storing detailed order information, each node in the supply chain can access and use this data, improving the transparency and efficiency of the entire supply chain.

[0077] Step S136: Store the new order information in the historical commodity order library.

[0078] The implementation method of the above steps S135 to S136 is, for example: generating new order information based on the microscopic images of multiple parts of the goods to be detected, fingerprint feature vectors, product categories and styles and models, and storing the new order information in the historical product order library of the electronic device (such as a backend server). It can be understood that the above-mentioned product categories and styles and models can be sent by the sending terminal device of the microscopic images of multiple parts of the goods to be detected, that is, the sending terminal device sends the product categories and styles and models together with the microscopic images of multiple parts of the goods to be detected. In the implementation process of the above scheme, by combining various types of data (images, feature vectors, text information, etc.), the system can describe and identify the goods from multiple dimensions, thereby improving the diversity and practicality of the data.

[0079] See also Figure 3 The structure diagram of the commodity resale detection device provided in the embodiment of the present application is shown; the embodiment of the present application provides a commodity resale detection device 200, including:

[0080] The microscopic image acquisition module 210 is used to acquire microscopic images of multiple parts of the commodity to be inspected. The microscopic images are acquired by photographing with an image acquisition device whose magnification exceeds a magnification threshold.

[0081] The image feature extraction module 220 is used to extract features from microscopic images of multiple parts through a neural network model to obtain a fingerprint feature vector of the product to be detected.

[0082] The commodity resale determination module 230 is used to compare the fingerprint feature vector of the commodity to be detected with the fingerprint feature vector in the historical commodity order library to determine whether the commodity to be detected has been sold in the historical commodity order library, and the historical commodity order library stores the fingerprint feature vectors of sold commodities.

[0083] As an optional implementation of the above device, the commodity resale detection device further includes:

[0084] The sample label acquisition module is used to obtain training samples and training labels. The training samples include: microscopic images of multiple parts of the sample product and the magnification of the microscopic image of each part. The training labels include: category labels and order labels of the sample product.

[0085] The sample category prediction module is used to predict the sample category of the sample product based on multiple microscopic images of parts and the magnification of the microscopic image of each part through a neural network model.

[0086] The classification loss calculation module is used to calculate the commodity classification loss between the sample category of the sample commodity and the category label of the sample commodity.

[0087] The feature vector acquisition module is used to extract features from microscopic images of multiple parts through a neural network model, obtain the fingerprint feature vector of the sample product, and predict the sample order corresponding to the sample product based on the fingerprint feature vector of the sample product.

[0088] The identification loss calculation module is used to calculate the order identity identification loss between the sample order of the sample product and the order tag of the sample product.

[0089] The model parameter updating module is used to update the parameters of the neural network model according to the commodity classification loss and the order identity recognition loss.

[0090] As an optional implementation of the above device, the identification loss calculation module includes:

[0091] The similarity score acquisition submodule is used to acquire, from the historical product order library, first similarity scores of products of the same category that are similar to the sample order of the sample product, and second similarity scores of products of different categories that are similar to the sample order of the sample product.

[0092] The order identification submodule is used to calculate the order identification loss according to the magnification of the microscopic image of the part, the first similarity score and the second similarity score.

[0093] As an optional implementation of the above device, the commodity resale determination module includes:

[0094] The fingerprint feature query submodule is used to determine whether a historical order whose similarity with the fingerprint feature vector of the product to be detected is greater than a preset threshold is queried in the historical product order library.

[0095] The re-sale determination submodule is used to determine that the product to be detected has been sold in the historical product order library if a historical order with a similarity between the fingerprint feature vector of the product to be detected and the historical order is greater than a preset threshold in the historical product order library; if no historical order with a similarity between the fingerprint feature vector of the product to be detected and the historical order is greater than the preset threshold in the historical product order library is found, it is determined that the product to be detected has not been sold in the historical product order library.

[0096] As an optional implementation of the above device, the commodity resale determination module further includes:

[0097] The resale prompt sending submodule is used to send resale prompt information of the commodity to be detected, and the resale prompt information includes: order number, fingerprint feature vector, detection time and / or prompt note information.

[0098] As an optional implementation of the above device, the commodity resale determination module further includes:

[0099] The order information generation submodule is used to generate new order information based on microscopic images of multiple parts of the product to be detected, fingerprint feature vectors, product categories and styles and models.

[0100] The order information storage submodule is used to store new order information in the historical commodity order library.

[0101] It should be understood that the device corresponds to the above-mentioned commodity resale detection method embodiment and can execute the various steps involved in the above-mentioned method embodiment. The specific functions of the device can be found in the description above, and the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the device.

[0102] See also Figure 4 The electronic device 300 provided in the embodiment of the present application includes: a processor 310 and a memory 320, wherein the memory 320 stores machine-readable instructions executable by the processor 310, and when the machine-readable instructions are executed by the processor 310, the above method is executed.

[0103] The embodiment of the present application also provides a computer-readable storage medium 330, on which a computer program is stored, and the computer program is executed by the processor 310 to execute the above method. The computer-readable storage medium 330 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0104] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor.

[0105] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0106] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device embodiments described above are only schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of a module, a program segment or a code includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also be different from the order of occurrence marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which is mainly determined by the functions involved.

[0107] In addition, each functional module of each embodiment in the embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the description of reference terms "one embodiment", "some embodiments", "example", "specific example", "some examples", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present application. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples without contradiction.

[0108] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiments of the present application, which should be covered within the protection scope of the embodiments of the present application.

Claims

1. A method for detecting resale of goods, characterized in that: include: Acquire microscopic images of multiple parts of the commodity to be inspected, wherein the microscopic images are obtained by photographing an image acquisition device with a magnification exceeding a multiple threshold; Extracting features from the microscopic images of the multiple parts through a neural network model to obtain a fingerprint feature vector of the commodity to be detected; The fingerprint feature vector of the commodity to be detected is compared with the fingerprint feature vector in the historical commodity order library to determine whether the commodity to be detected has been sold in the historical commodity order library, and the historical commodity order library stores the fingerprint feature vectors of sold commodities.

2. The method according to claim 1, characterized in that Before extracting features from the microscopic images of the multiple parts through the neural network model, the method further includes: Acquire training samples and training labels, wherein the training samples include: microscopic images of multiple parts of the sample product and the magnification of each microscopic image of the part, and the training labels include: category labels and order labels of the sample product; Predicting the sample category of the sample commodity according to the multiple microscopic images of the parts and the magnification of each microscopic image of the parts by using a neural network model; Calculate the product classification loss between the sample category of the sample product and the category label of the sample product; Extracting features from the microscopic images of the multiple parts through the neural network model to obtain a fingerprint feature vector of the sample product, and predicting a sample order corresponding to the sample product according to the fingerprint feature vector of the sample product; Calculate the order identification loss between the sample order of the sample product and the order tag of the sample product; The parameters of the neural network model are updated according to the commodity classification loss and the order identity recognition loss.

3. The method according to claim 2, characterized in that The calculating the order identity recognition loss between the sample order of the sample product and the order tag of the sample product includes: Acquire, from the historical product order library, a first similarity score of a product of the same category that is similar to the sample order of the sample product, and a second similarity score of a product of a different category that is similar to the sample order of the sample product; The order identification loss is calculated according to the magnification of the microscopic image of the part, the first similarity score, and the second similarity score.

4. The method according to claim 1, characterized in that: The step of comparing the fingerprint feature vector of the commodity to be detected with the fingerprint feature vector in the historical commodity order library to determine whether the commodity to be detected has been sold in the historical commodity order library includes: Determine whether a historical order whose similarity with the fingerprint feature vector of the commodity to be detected is greater than a preset threshold is found in the historical commodity order database; If so, it is determined that the commodity to be detected has been sold in the historical commodity order library; otherwise, it is determined that the commodity to be detected has not been sold in the historical commodity order library.

5. The method according to claim 4, characterized in that After determining that the commodity to be detected has been sold in the historical commodity order library, the method further includes: Send resale reminder information of the commodity to be detected, wherein the resale reminder information includes: order number, fingerprint feature vector, detection time and / or reminder note information.

6. The method according to claim 1, characterized in that After determining that the commodity to be detected has not been sold in the historical commodity order library, the method further includes: Generate new order information based on microscopic images of multiple parts of the product to be detected, fingerprint feature vectors, product categories and styles and models; The new order information is stored in the historical commodity order library.

7. A commodity resale detection device, characterized in that: include: A microscopic image acquisition module is used to acquire microscopic images of multiple parts of the commodity to be inspected, wherein the microscopic images are acquired by photographing an image acquisition device with a magnification exceeding a multiple threshold; An image feature extraction module is used to extract features from the microscopic images of the multiple parts through a neural network model to obtain a fingerprint feature vector of the commodity to be detected; The commodity resale determination module is used to compare the fingerprint feature vector of the commodity to be detected with the fingerprint feature vector in the historical commodity order library to determine whether the commodity to be detected has been sold in the historical commodity order library, and the historical commodity order library stores the fingerprint feature vectors of the sold commodities.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are executed by the processor to perform any method according to claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is executed.

10. A computer program product, characterized in that include: A computer program or a computer instruction, wherein when the computer program or the computer instruction is executed by a processor, the method according to any one of claims 1 to 6 is executed.

Images