Commodity identification method, program product, electronic equipment and storage medium

Abstract

The invention provides a commodity identification method, a program product, electronic equipment and a storage medium. The method comprises the following steps: acquiring a commodity image and a commodity label image of a to-be-identified commodity; performing image feature extraction on the commodity image to obtain image features; obtaining certified product scores according to the image features; the certified product score represents the probability that the commodity is a certified product; extracting commodity attribute information based on the commodity label image; and based on the certified product score and the commodity attribute information, identifying the to-be-identified commodity to obtain an identification result. By combining the image features of the commodity image and the commodity attribute information of the commodity label image, the commodity is analyzed and judged from the dimension of the image features and the dimension of the commodity attribute information, so that the commodity can be known more comprehensively; the certified product score provides a quantitative judgment basis for an identification result; the commodity attribute information refines the features of the commodity, and the identification result obtained based on the certified product score and the commodity attribute information is higher in accuracy and interpretability.

Description

Technical Field

[0001] This application relates to the field of commodity authentication. Specifically, it relates to a commodity authentication method, a program product, an electronic device, and a storage medium. Background Art

[0002] Traditional commodity authentication methods use manual operations. By manually checking each authentication point separately, the authentication results are obtained. Due to a large number of inspection items, a long process, and differences in standard execution by personnel operations, problems such as deviations in the results occur. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a commodity authentication method, a program product, an electronic device, and a storage medium to improve the above problems.

[0004] In a first aspect, the embodiments of this application provide a commodity authentication method, including: obtaining a commodity image and a commodity label image of the commodity to be authenticated; extracting image features from the commodity image to obtain image features; obtaining a genuine product score according to the image features, where the genuine product score represents the probability that the commodity is a genuine product; extracting commodity attribute information based on the commodity label image; and identifying the commodity to be authenticated based on the genuine product score and the commodity attribute information to obtain an authentication result.

[0005] In the above implementation process, by combining the image features of the commodity image and the commodity attribute information of the commodity label image, analyzing and judging the commodity from the dimensions of image features and commodity attribute information can provide a more comprehensive understanding of the commodity; and the genuine product score provides a quantitative basis for judgment for the authentication result; the commodity attribute information details the features of the commodity, and the authentication result obtained based on the genuine product score and the commodity attribute information has higher accuracy and interpretability.

[0006] Optionally, in the embodiments of this application, the method further includes: obtaining additional commodity information of the commodity to be authenticated, where the additional commodity information is information about the commodity to be authenticated other than the commodity attribute information; extracting text features from the additional commodity information to obtain additional commodity features; and identifying the commodity to be authenticated based on the genuine product score and the commodity attribute information to obtain an authentication result, including: identifying the commodity to be authenticated based on the genuine product score, the commodity attribute information, and the additional commodity features to obtain an authentication result.

[0007] In the above implementation process, the additional commodity information contains content not included in the commodity image and the commodity label image, realizing mutual complementation with the genuine product score and the commodity attribute information, which helps to more accurately judge the authenticity of the commodity to be authenticated. And the source of the additional commodity information is not the commodity image and the commodity label image, realizing identification based on multiple information sources, improving the accuracy of authentication, and making the authentication result have a more definite basis.

[0008] Optionally, in the embodiments of the present application, the product image includes a product foreground image; obtaining the product image of the product to be authenticated includes: obtaining multiple product foreground images corresponding to multiple shooting angles of the product to be authenticated.

[0009] In the above implementation process, by obtaining multiple product foreground images corresponding to multiple shooting angles of the product to be authenticated, the interference of background factors in the product image on the authentication is reduced, and in the recognition process, the product foreground images from multiple angles can be combined for analysis to improve the accuracy of authentication.

[0010] Optionally, in the embodiments of the present application, obtaining the genuine product score according to the image features includes: calculating the distances between the image features and the genuine product classification center and the fake product classification center corresponding to the authentication part of the product image respectively to obtain distance data; the genuine product classification center is the genuine product features of at least one product unit at the authentication part; the fake product classification center is the fake product features of at least one product unit at the authentication part; wherein, one product unit represents a specification of the product; based on the distance data, determining the genuine product score of the product image.

[0011] In the above implementation process, the genuine product classification center and the fake product classification center of the authentication part are associated with the product unit, and different classification centers can represent the features of different product units. By calculating the distances between the image features and at least one classification center, the refinement degree and accuracy of authentication are improved. And the distance data reflects the similarity between the product image features and the genuine product classification center and the fake product classification center. Converting the distance data into the genuine product score of the product image provides a quantitative authentication basis for the authentication result.

[0012] Optionally, in the embodiments of the present application, the method further includes: determining the target classification center closest to the image features based on the minimum value in the distance data; obtaining the product unit information of the target classification center; the product unit information is used to represent the specification attribute information corresponding to the product to be authenticated; performing a consistency check on the product unit information and the product attribute information to obtain a consistency check result; based on the genuine product score and the product attribute information, identifying the product to be authenticated to obtain an authentication result, including: identifying the product to be authenticated based on the genuine product score, the product attribute information and the consistency check result to obtain an authentication result.

[0013] In the above implementation process, the image features can reflect the intuitive information such as the appearance and texture of the product, and the product attribute information provides the basic description of the product in the product label image. By performing a consistency check on the product unit information and the product attribute information, the reliability of the product attribute information is verified, and considering the consistency check result in the authentication result improves the accuracy of authentication.

[0014] Optionally, in the embodiments of the present application, the product to be authenticated is identified based on the genuine product score and product attribute information to obtain an authentication result, including: inputting the genuine product score and product attribute information into a large language model to obtain the authentication result.

[0015] In the above implementation process, the genuine product score provides the probability that the product is genuine, while the product attribute information provides the basic description of the product. The large language model can comprehensively consider various information, realize discrimination and analysis from multiple perspectives, reduce the possibility of misjudgment, and improve the accuracy of authentication. Moreover, the large language model has powerful reasoning ability, and the output result is relatively natural. The large language model can automatically process the input genuine product score and product attribute information, reducing the workload of manual intervention and manual processing, and improving the efficiency of authentication.

[0016] Optionally, in the embodiments of the present application, the authentication result includes the authenticity of the product to be authenticated and the reason for determining the authenticity. Before inputting the genuine product score and product attribute information into the large language model to obtain the authentication result, the method further includes: obtaining a data set, where the data set includes sample genuine product scores, sample product attribute information, and annotation data corresponding to the sample genuine product scores and sample product attribute information, and the annotation data includes true / false labels and the reasons for determining true / false; using the pre-trained large language model to learn the knowledge of authenticating product authenticity and attribution based on the data set.

[0017] In the above implementation process, using the pre-trained large language model to learn the knowledge of authenticating product authenticity and attribution based on the data set, the processed large language model can better adapt to the product authentication task and improve the accuracy of product authentication. Moreover, the pre-trained large language model has learned the knowledge of authenticating product authenticity and attribution based on the data set, so the output authentication result includes the true / false category of the product and the reason for the counterfeit product, helping users better understand the authentication result.

[0018] In a second aspect, the embodiments of the present application further provide a product authentication device, including: an image acquisition module for acquiring the product image and product label image of the product to be authenticated; a feature extraction module for extracting image features from the product image to obtain image features; a genuine product score module for obtaining a genuine product score according to the image features, where the genuine product score represents the probability that the product is genuine; an attribute information module for extracting product attribute information based on the product label image; and an authentication module for identifying the product to be authenticated based on the genuine product score and product attribute information to obtain an authentication result.

[0019] In a third aspect, the embodiments of the present application further provide a computer program product, including computer program instructions, where the computer program instructions, when run by a processor, execute the method provided in the first aspect or any one of the implementation manners of the first aspect.

[0020] Fourthly, an embodiment of the present application further provides an electronic device, including: a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are run by the processor, the method provided by the first aspect or any implementation manner of the first aspect is executed.

[0021] Fifthly, an embodiment of the present application further provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are run by the processor, the method provided by the first aspect or any implementation manner of the first aspect is executed.

[0022] By using a commodity authentication method, program product, electronic device, and storage medium provided by the present application, by combining the image features of the commodity image and the commodity attribute information of the commodity label image, the commodity is analyzed and judged from the dimensions of image features and commodity attribute information, and the commodity can be understood more comprehensively; and the genuine product score provides a quantitative judgment basis for the authentication result; the commodity attribute information details the features of the commodity, and the authentication result obtained based on the genuine product score and the commodity attribute information has higher accuracy and interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 It is a schematic flowchart of a commodity authentication method provided by an embodiment of the present application;

[0025] Figure 2 It is a schematic structural diagram of a commodity authentication device provided by an embodiment of the present application;

[0026] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The following will describe in detail the embodiments of the technical solutions of the present application with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and therefore are only examples and cannot be used to limit the protection scope of the present application.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0029] In the description of the embodiments of the present application, technical terms such as "first" and "second" are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity, specific order or primary-secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "a plurality" is two or more, unless otherwise specifically defined.

[0030] Please refer to Figure 1 the schematic flowchart of a commodity authentication method provided by the embodiments of the present application shown. The commodity authentication method provided by the embodiments of the present application can be applied to an electronic device, and the electronic device can include physical devices such as a server, a PC, a tablet computer or a smart phone, or can also be a virtual device such as a virtual machine or a container. The electronic device can be a single device, or a combination of multiple devices or a cluster of a large number of devices. The commodity authentication method can include:

[0031] Step S110: Obtain a commodity image and a commodity label image of the commodity to be authenticated.

[0032] Step S120: Extract image features from the commodity image to obtain image features.

[0033] Step S130: Obtain an authentic product score according to the image features; the authentic product score represents the probability that the commodity is an authentic product.

[0034] Step S140: Extract commodity attribute information based on the commodity label image.

[0035] Step S150: Identify the commodity to be authenticated based on the authentic product score and the commodity attribute information to obtain an authentication result.

[0036] In step S110, the commodity to be authenticated is a commodity that needs to be authenticated for authenticity. The commodity image refers to the physical image of the commodity to be authenticated; the commodity label image is an image containing commodity information such as labels, tags, and nameplates on the commodity to be authenticated. Both the commodity image and the commodity label image can be obtained by collecting the commodity to be authenticated through a photographing device, or can be obtained from a storage system such as a database.

[0037] In one case, if the commodity label is included in the commodity image, it can be considered that the commodity image and the commodity label image are integrated, that is, this picture can be used as both the commodity image and the commodity label image.

[0038] In step S120, the image features are feature vectors extracted from the commodity image, and can include information such as color, texture, shape or edge. The methods for extracting image features include: the encoder can be used to convert the commodity image into lower-dimensional image features; or a pre-trained feature extraction model can be used to process the commodity image to obtain image features.

[0039] In step S130, the genuine product score can be a numerical value, which is used to characterize the probability that the product is genuine. For example, the larger the value, the greater the probability that the product to be authenticated is genuine. As another implementation, the genuine product score can also be a combination of a label and a numerical value, such as "genuine 0.9" or "fake 0.8". Among them, "genuine 0.9" represents that the probability that the product to be authenticated is genuine is 90%; while "fake 0.8" represents that the probability that the product to be authenticated is fake is 80% (that is, the probability that the product is fake is relatively large, and the probability that it is genuine is relatively small). This method requires combining the label and the numerical value to jointly determine the probability that the product is genuine, rather than only considering the size of the numerical value.

[0040] The method of obtaining the genuine product score can use a deep learning model (such as a convolutional neural network CNN) to classify the image features and output the probability that the product is genuine. For example, collect a large number of genuine and fake product images, perform image feature extraction, label the genuine product score for the extracted sample image features, use the sample image features to train the deep learning model, and obtain a trained genuine product score model. Then use the trained genuine product score model to predict the image features and obtain the genuine product score. This method can automatically learn the complex relationship between image features and the genuine product probability, and has high efficiency and convenience.

[0041] It is also possible to calculate the distances between the image features of the product to be authenticated and the genuine product classification center and the fake product classification center, and convert the distances into genuine product scores. For example, extract multiple features from multiple known genuine product images, and use the extracted multiple features to calculate a central vector as the genuine product classification center; extract multiple features from multiple fake product images, and use the extracted multiple features to calculate a central vector as the fake product classification center. Then calculate the distances between the feature images and the genuine product classification center and the fake product classification center respectively, and convert the distance values into genuine product scores. This method is simple to calculate and saves computing power resources.

[0042] Of course, it is also possible to use some models for probability distribution calculation to obtain the genuine product score corresponding to the image features. For example, input the image features of the product to be authenticated into the estimated probability density model, and calculate the probability that it belongs to the genuine product distribution as the genuine product score. This method can capture the distribution characteristics of the features, and is also applicable to the case where the feature distribution is relatively complex, and the accuracy of the genuine product score is relatively high.

[0043] In step S140, the product attribute information includes, for example, information such as the origin, item number, size, color, material, brand, and / or model of the product to be authenticated. The product attribute information can be extracted from the product label image using optical character recognition (OCR) technology, or a text detection model based on deep learning can be used to detect and recognize the text area in the product label image, thereby obtaining the product attribute information.

[0044] In step S150, the authentication result is used to represent the true or false category label, the probability of being a genuine (fake) product, and / or the reason for being a genuine product corresponding to the genuine product category or the reason for being a fake product corresponding to the fake product category. The genuine product score and the product attribute information can be input into a large language model. For example, the large language model identifies the product to be authenticated to obtain the authentication result. The large language model can pre-learn the knowledge of authenticating the true or false of products and attribution.

[0045] Other machine learning models can also be pre-trained, such as logistic regression, support vector machine, random forest, etc. The trained model is used to predict new products to be authenticated. Taking the genuine product score and the product attribute information as input features, the model will output the probability or category label that the product is a genuine product.

[0046] Of course, the authentication result can also be given based on preset rules. For example, if a certain type of shoe has been on the market for a long time and is basically severely worn, the requirement for the genuine product score can be appropriately reduced. The rules can be determined according to the actual situation.

[0047] In the implementation process of the above embodiments: By combining the image features of the product image and the product attribute information of the product label image, analyzing and judging the product from the dimensions of the image features and the product attribute information can provide a more comprehensive understanding of the product; and the genuine product score provides a quantitative basis for judgment for the authentication result; the product attribute information details the features of the product. The authentication result obtained based on the genuine product score and the product attribute information has higher accuracy and interpretability.

[0048] Optionally, in the embodiments of the present application, based on the genuine product score and the product attribute information, identifying the product to be authenticated to obtain the authentication result includes:

[0049] Obtaining the additional product information of the product to be authenticated; wherein, the additional product information is information about the product to be authenticated other than the product attribute information. The additional product information can be sourced from the product order information or the information recorded for the product to be authenticated in the inventory system; the additional product information includes the order time, weight, color, origin, size, etc. of the product.

[0050] It should be noted that the additional product information and the product attribute information may contain the same content, but their sources are different. For example, the product attribute information includes the color of the product, and the above color information is obtained through label image recognition; while the additional product information also includes the color of the product, and this color is obtained through system records (such as inventory records or user evaluations, etc.). Of course, the additional product information may also include content that is not included in the product attribute information.

[0051] In an optional embodiment, for the common information in the product attribute information and the additional product information, consistency verification can be performed, and the verification result is used for product identification.

[0052] Text feature extraction is performed on the additional product information to obtain additional product features. For example, natural language processing techniques can be used to segment the text in the product order information or inventory system records, and keywords, phrases, etc. are extracted as additional product features. The text can also be input into a pre-trained machine learning model, and the machine learning model is used to automatically learn and extract additional product features.

[0053] Based on the genuine product score, the product attribute information, and the additional product features, the product to be identified is identified to obtain an identification result. Similar to step S150, a large language model or other trained models can be used to predict the new product to be identified to obtain an identification result. The identification result can also be obtained by means of weighted scoring.

[0054] In the implementation process of the above embodiment: The additional product information contains content that is not included in the product image and the product label image, which realizes mutual supplementation with the genuine product score and the product attribute information, and helps to more accurately judge the authenticity of the product to be identified. And the source of the additional product information is not the product image and the product label image, which realizes identification based on multiple information sources, improves the accuracy of identification, and makes the identification result have a clearer basis.

[0055] Optionally, in the embodiment of the present application, the product image includes a product foreground image; obtaining the product image of the product to be identified includes: obtaining multiple product foreground images corresponding to multiple shooting angles of the product to be identified.

[0056] For example, cameras with multiple fixed or non-fixed positions can be used to shoot the product to be identified from different angles. Multiple cameras are respectively responsible for shooting different angles of the product, such as the front, back, left, right, top, bottom, etc., so as to obtain multiple product images. Then, an image segmentation algorithm is used to process the product images to separate the product from the background and obtain the product foreground image corresponding to the product image.

[0057] Optionally, the shooting light source can adopt an up-and-down symmetrical design, with multiple light sources installed on the upper and lower parts of the commodity respectively, so that the internal light source provides sufficient and uniform brightness. Diffused light can also be used to fill light inside the device, reduce the interference of external ambient light, and improve the stability of imaging illumination.

[0058] As an implementation, the commodity label image includes a commodity label foreground image; obtaining the commodity label image of the commodity to be authenticated includes: obtaining multiple commodity label foreground images corresponding to multiple shooting angles of the commodity to be authenticated. The obtaining method can refer to the method of obtaining the commodity foreground image.

[0059] After obtaining multiple commodity foreground images, the commodity to be authenticated can be identified based on the genuine product scores and commodity attribute information respectively corresponding to the multiple commodity foreground images, and the result of the commodity to be authenticated can be obtained.

[0060] In the implementation process of the above embodiment: by obtaining multiple commodity foreground images corresponding to multiple shooting angles of the commodity to be authenticated, the interference of background factors in the commodity image on the authentication is reduced, and in the process of identification, the commodity foreground images from multiple angles can be combined for analysis to improve the accuracy of authentication.

[0061] Optionally, in the embodiment of the present application, obtaining the genuine product score according to the image features includes:

[0062] According to the authentication part corresponding to the commodity image, the image features are respectively calculated with the genuine product classification center and the fake product classification center corresponding to the authentication part to obtain distance data.

[0063] Each commodity image includes an authentication part corresponding to it, and this authentication part is related to the angle of the commodity image. For example, if the commodity image is taken by the camera behind the commodity, the authentication part is the area behind the commodity. It can be understood that for as many angles of commodity images as there are, there are corresponding numbers of authentication parts.

[0064] Each authentication part includes a corresponding genuine product classification center and a fake product classification center. There can be one or more genuine product classification centers and fake product classification centers corresponding to the authentication part. The genuine product classification center is the genuine product features of at least one product unit in the authentication part; the fake product classification center is the fake product features of at least one product unit in the authentication part. The product unit is used to identify the specifications of the commodity, and the specifications include size, color, or material, etc. For example, the same style of bag can have small, medium, and large sizes, the same style of shoes or clothes can have different sizes and colors, and the same style of shoes may be made of shiny or matte materials, etc.

[0065] For the case where there are multiple genuine classification centers for the identification part, then each of the multiple product units of the commodity has a corresponding genuine classification center. For example, if the commodity is a backpack and the identification part is the back of the bag, and the backpack has three sizes: small, medium, and large. The genuine features of the back area of the small-sized bag are the genuine classification center corresponding to the small size (denoted as sku1); the genuine features of the back area of the medium-sized bag are the genuine classification center corresponding to the medium size (denoted as sku2); the genuine features of the back area of the large-sized bag are the genuine classification center corresponding to the large size (denoted as sku3). For example, the way to obtain the genuine features of the back area of the small-sized bag can be: pre-collect multiple sample genuine images of the back area of the small-sized bag; perform image feature extraction on the above multiple sample genuine images of the back area of the small-sized bag respectively to obtain multiple genuine sample image features, obtain the genuine features from the multiple genuine sample image features, and take the obtained genuine features as the genuine classification center (sku1) corresponding to the small size.

[0066] Similarly, the fake classification center includes the fake classification centers corresponding to the three sizes. It can be understood that the classification centers determined in this way can include the corresponding product unit information. For example, the genuine classification center sku1 corresponding to the small size, and its product unit information is the small size.

[0067] During the process of distance calculation, the image features of the commodity to be identified need to be calculated respectively with the above three genuine classification centers and three fake classification centers to obtain distance data. The distance data includes the distance values between the image features and each genuine classification center and fake classification center. The distance value is used to represent the difference between the image features and the genuine classification center or fake classification center. The smaller the distance value, the smaller the difference.

[0068] Each of the multiple product units has a corresponding genuine classification center. The advantage of this is that considering that commodities of different specifications may have differences in details. For example, the logos of small-sized bags and large-sized bags are different, and it is difficult to find a genuine classification center that can represent the genuine features of commodities of different specifications at the same time. Therefore, in the embodiments of the present application, the classification centers (collectively referred to as genuine classification centers and fake classification centers) are associated with the product units. The classification center is the genuine (or fake) feature of a product unit at the identification part, rather than the genuine (or fake) feature of all specifications of the commodity at the identification part, which improves the refinement degree of identification.

[0069] In addition, in some authentication scenarios, if the product to be authenticated is of a small size and it is recognized that its logo is the same as the logo of a large-size backpack, this product is obviously problematic (for example, there is a problem with the logo being swapped). If the difference in specifications is not considered during the authentication process and the product to be authenticated can match genuine products of any specification and is thus determined to be genuine, then errors may occur in the authentication for this situation. Therefore, in the embodiments of the present application, the differences between products of different specifications are fully considered during the authentication process. Even if the image recognition matches the genuine product (for example, the distance value is less than the threshold), the corresponding product unit information can still be used for consistency verification, thereby improving the accuracy of authentication. This solution will be described later.

[0070] For the case where there is a genuine product classification center for the authentication part, perhaps this product has one product unit, that is, this product does not distinguish specifications, then a genuine product classification center can be calculated using the genuine product features of the product at the authentication part, and the counterfeit product classification center is the same. Or the product has multiple specifications, but in the authentication part, the differences between different specifications can be ignored, then a genuine product classification center can be calculated for the multiple product units at this authentication part. During the process of calculating the distance value, it is only necessary to calculate the distances between the image data and the above one genuine product classification center and one counterfeit product classification center respectively. The number of classification centers can be determined according to the characteristics of the product.

[0071] After obtaining the distance data, based on the distance data, determine the genuine product score of the product image. The numerical part and the label part of the genuine product score can be determined based on the distance data respectively, and the genuine product score is obtained by combining the numerical part and the label part.

[0072] Exemplarily, based on the minimum value in the distance data, determine the target classification center closest to the image feature. The target classification center can be a genuine product classification center or a counterfeit product classification center. The label of the genuine product classification center can be "genuine product", and the label of the counterfeit product classification center can be "counterfeit product". Map the minimum value to a score interval (such as [0, 1]), and the mapping methods include threshold method, linear mapping, or non-linear mapping, etc. For example, assume that the target classification center closest to the minimum value is the counterfeit product classification center for small sizes, and the minimum value is 0.1. Use the threshold method to convert the distance value into a genuine product score. Assume the threshold is 0.5. If the distance value is less than the threshold, the numerical part of the genuine product score is 1; otherwise, it is 0. Since the minimum value is less than the threshold, the numerical part of the genuine product score is 1; and because the minimum value is closest to the counterfeit product classification center for small sizes and the label is "counterfeit product", the label part of the genuine product score is "counterfeit product". The final genuine product score can be "counterfeit product 1", indicating that the product to be authenticated has a high probability of being a counterfeit product.

[0073] Of course, the plus sign "+", minus sign "-", or other identifiers can also be used as the tag part. The above are just examples.

[0074] In the implementation process of the above embodiments: The genuine product classification center and the counterfeit product classification center are associated with the product unit. Different classification centers can represent the characteristics of different product units. By calculating the distance between the image features and the classification centers, the refinement degree of identification and the accuracy of identification are improved. And the distance data reflects the similarity between the commodity image features and the genuine product classification center and the counterfeit product classification center. The distance data is converted into the genuine product score of the commodity image, so that the identification result has a clear basis.

[0075] In an alternative embodiment, before obtaining the distance data, the genuine product classification center and the counterfeit product classification center corresponding to the identification part can be confirmed first. The method further includes:

[0076] The first step: Obtain multiple genuine product image data and multiple counterfeit product image data of at least one product unit at the identification part. For example, multiple genuine product image data and multiple counterfeit product image data of the rear area of a small-sized backpack can be obtained respectively; multiple genuine product image data and multiple counterfeit product image data of the rear area of a medium-sized backpack and a large-sized backpack.

[0077] The second step: Respectively extract the image features from the multiple genuine product image data and the multiple counterfeit product image data to obtain multiple genuine product image features and multiple counterfeit product image features. The way of feature extraction refers to the foregoing.

[0078] The third step: Taking the product unit as the dimension, process the multiple genuine product image features of the same product unit to obtain the genuine product classification center of the product unit. Taking the product unit as the dimension, process the multiple counterfeit product image features of the same product unit to obtain the counterfeit product classification center of the product unit. The processing methods include feature clustering, taking the mean value, taking the median, etc.

[0079] For example, cluster the multiple genuine product image features of the rear area of a small-sized backpack to obtain the genuine product classification center of the small-sized backpack; cluster the multiple counterfeit product image features of the rear area of a small-sized backpack to obtain the counterfeit product classification center of the small-sized backpack. The same applies to medium-sized backpacks and large-sized backpacks, and the corresponding genuine product classification centers and counterfeit product classification centers can be obtained.

[0080] In this way, for the three sizes of backpacks in the identification dimension of the rear area, the corresponding genuine product classification centers and counterfeit product classification centers are obtained. When conducting identification, calculate the distances between the image features of the image to be identified and the genuine product classification centers and counterfeit product classification centers of the above three sizes respectively to obtain the distance data, and obtain the corresponding genuine product scores.

[0081] Optionally, in the embodiments of the present application, the method further includes: determining a target classification center closest to the image feature based on the minimum value in the distance data. Continuing with the above embodiments, the image features are respectively calculated for distances from the genuine classification centers of the above three sizes and the fake classification center, obtaining 6 distance values. Among them, the minimum value is the distance value from the genuine classification center of the small-sized backpack, so the genuine classification center of the small-sized backpack is determined as the target classification center closest to the image feature.

[0082] Obtain the product unit information of the target classification center; the product unit information is used to characterize the specification attribute information corresponding to the product to be authenticated. For example, if the target classification center is the genuine classification center of a small-sized backpack, the product unit information can be a small-sized backpack.

[0083] Perform a consistency check on the product unit information and the product attribute information to obtain a consistency check result. The product attribute information is extracted from the product label image, while the product unit information is based on the image feature, that is, from the product image. By performing a consistency check on the information representing the specification in the product attribute information and the product unit information, it can be determined whether the product unit information of the product image (i.e., the product itself) is consistent with the information in the product label image, thereby achieving a verification effect and improving the accuracy of authentication.

[0084] Continuing with the above embodiments, if the product unit information is a small-sized backpack and the information in the product attribute information is also a small-sized backpack, the consistency check result is consistent; if the information in the product attribute information is a large-sized backpack, the consistency check result is inconsistent.

[0085] Based on the genuine score and the product attribute information, identify the product to be authenticated to obtain an authentication result, including:

[0086] Based on the genuine score, the product attribute information, and the consistency check result, identify the product to be authenticated to obtain an authentication result. During the process of identifying the product to be authenticated, the consistency check result also needs to be considered. For example, if the consistency check result is that the information is consistent, the genuine score and the product attribute information can be identified according to the original calculation method to obtain the authentication result; if the consistency check result is that the information is inconsistent, the authentication result can be directly determined as fake.

[0087] For example, a large language model can be used to identify the product to be authenticated to obtain an authentication result. Let the large language model learn knowledge about authenticating the authenticity of products based on the preset dataset according to the genuine score, the product attribute information, and the consistency check result. Then, input the genuine score, the product attribute information, and the consistency check result into the learned large language model to obtain the authentication result.

[0088] In the implementation process of the above embodiments: Image features can reflect intuitive information such as the appearance and texture of a product. Product attribute information provides a basic description of the product in the product label image. By performing consistency verification on the product unit information and the product attribute information, the reliability of the product attribute information is verified, providing a reliable basis for product authentication.

[0089] Optionally, in the embodiments of the present application, based on the genuine product score and the product attribute information, the product to be authenticated is identified to obtain an authentication result, including: inputting the genuine product score and the product attribute information into a large language model to obtain the authentication result.

[0090] The genuine product score and the product attribute information can be converted into a token sequence, and the token sequence is input into the large language model to obtain the authentication result. For example, the genuine product score can be converted into a string format, the product attribute information is tokenized and encoded, the processed genuine product score and product attribute information are concatenated into a token sequence, and the generated token sequence is input into the large language model. The large language model can be, for example, the GPT or Tongyi Qianwen model, etc. The large language model will generate an authentication result based on the input token sequence.

[0091] Optionally, in the embodiments of the present application, based on the genuine product score, the product attribute information, and the additional product features, the product to be authenticated is identified to obtain an authentication result, including: inputting the genuine product score, the product attribute information, and the additional product features into the large language model to obtain the authentication result.

[0092] Similarly, the genuine product score, the product attribute information, and the additional product features can be converted into a token sequence, and the token sequence is input into the large language model to obtain the authentication result.

[0093] In the implementation process of the above embodiments: The genuine product score provides the probability that the product is genuine, while the product attribute information provides a basic description of the product. The large language model can comprehensively consider various information, realize discrimination and analysis from multiple perspectives, reduce the possibility of misjudgment, and improve the accuracy of authentication. And the large language model can automatically process the input genuine product score and product attribute information, reducing the workload of manual intervention and manual processing, and improving the efficiency of authentication.

[0094] Optionally, in the embodiments of the present application, before inputting the genuine product score and the product attribute information into the large language model to obtain the authentication result, the method further includes:

[0095] Obtain a data set, where the data set includes sample genuine product scores, sample product attribute information, and the annotation data corresponding to the sample genuine product scores and the sample product attribute information. The annotation data includes genuine product labels, fake product labels, and reasons.

[0096] In an optional embodiment, the dataset may further include additional features of sample commodities, and then the labeled data is the labeled data corresponding to the sample genuine product score, sample commodity attribute information, and commodity additional features.

[0097] For example, image features are extracted from known genuine and counterfeit commodities, and the distances between these features and the genuine product classification center and the counterfeit product classification center are calculated to obtain the sample genuine product score. The process can be referred to the process of the genuine product score. And commodity attribute information is extracted from the label images of known genuine and counterfeit commodities. Data preprocessing can also be performed on the collected dataset, including data cleaning, data augmentation, or format conversion, etc.

[0098] If the true value of the combination of the sample genuine product score and the sample commodity attribute information is a genuine product, the labeled data can be a genuine product label; if the true value of the combination of the sample genuine product score and the sample commodity attribute information is a counterfeit product, the labeled data not only includes a counterfeit product label but also includes the reasons for the counterfeit product. The reasons for the counterfeit product include non-original accessories, fake accessories, doubtful origin, or mismatched commodities, etc.

[0099] Use a pre-trained large language model to learn knowledge of differentiating the authenticity of commodities and attribution based on the dataset. The pre-trained large language model has been pre-trained on a large-scale text dataset and has good language understanding and generation capabilities. On this basis, the dataset can be used to fine-tune the pre-trained large language model to optimize the model's parameters, enabling it to better adapt to the commodity authentication task and learn knowledge of differentiating the authenticity of commodities and attribution. During the fine-tuning process, the cross-entropy loss function can be used to optimize the performance of the model.

[0100] In the implementation process of the above embodiment: Use a pre-trained large language model to learn knowledge of differentiating the authenticity of commodities and attribution based on the dataset. The processed large language model can better adapt to the commodity authentication task and improve the accuracy of commodity authentication. And the pre-trained large language model has learned knowledge of differentiating the authenticity of commodities and attribution based on the dataset. Therefore, the output authentication result includes the true or false category of the commodity and the reasons for the counterfeit product, providing a more explicit authentication basis to help users better understand the authentication result.

[0101] Please refer to Figure 2 The structural schematic diagram of the commodity authentication device provided by the embodiment of the present application shown; The embodiment of the present application provides a commodity authentication device 200, including:

[0102] An image acquisition module 210, configured to acquire a commodity image and a commodity label image of the commodity to be authenticated;

[0103] A feature extraction module 220, configured to extract image features from the commodity image to obtain image features;

[0104] The genuine product score module 230 is used to obtain a genuine product score based on image features; the genuine product score represents the probability that the product is a genuine product.

[0105] The attribute information module 240 is used to extract product attribute information based on the product label image.

[0106] The authentication module 250 is used to identify the product to be authenticated based on the genuine product score and the product attribute information, and obtain an authentication result.

[0107] Optionally, in the embodiment of the present application, the product authentication device 200 and the authentication module 250 are further used to obtain additional product information of the product to be authenticated; wherein, the additional product information is information about the product to be authenticated other than the product attribute information; perform text feature extraction on the additional product information to obtain additional product features; identify the product to be authenticated based on the genuine product score, the product attribute information and the additional product features, and obtain an authentication result.

[0108] Optionally, in the embodiment of the present application, for the product authentication device 200, the product image includes a product foreground image; the image acquisition module 210 is further used to acquire multiple product foreground images corresponding to multiple shooting angles of the product to be authenticated.

[0109] Optionally, in the embodiment of the present application, for the product authentication device 200, the genuine product score module 230 is further used to calculate the distances between the image features and the genuine product classification center and the fake product classification center corresponding to the authentication part of the product image respectively to obtain distance data; the genuine product classification center is the genuine product features of at least one product unit at the authentication part; the fake product classification center is the fake product features of at least one product unit at the authentication part; determine the genuine product score of the product image based on the distance data.

[0110] Optionally, in the embodiment of the present application, the product authentication device 200 further includes: a consistency verification module, which is used to determine the target classification center closest to the image features based on the minimum value in the distance data; obtain the product unit information of the target classification center; the product unit information is used to represent the specification attribute information corresponding to the product to be authenticated; perform consistency verification on the product unit information and the product attribute information to obtain a consistency verification result; identify the product to be authenticated based on the genuine product score and the product attribute information, and obtain an authentication result, including: identify the product to be authenticated based on the genuine product score, the product attribute information and the consistency verification result, and obtain an authentication result.

[0111] Optionally, in the embodiment of the present application, for the product authentication device 200, the authentication module 250 is further used to input the genuine product score and the product attribute information into a large language model to obtain an authentication result.

[0112] Optionally, in an embodiment of the present application, the commodity identification device 200 further includes: a large language model attribution knowledge learning module, which is used to obtain a data set, the data set includes a sample authenticity score, sample commodity attribute information, and labeled data corresponding to the sample authenticity score and the sample commodity attribute information, the labeled data includes an authentic label and a fake label and reasons; and use a pre-trained large language model to learn knowledge of identifying the authenticity of commodities and attribution based on the data set.

[0113] It should be understood that the device corresponds to the above-mentioned commodity identification method embodiment and can execute the various steps involved in the above-mentioned method embodiment. The specific functions of the device can be found in the above description. To avoid repetition, the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or fixed in the operating system (OS) of the device.

[0114] See also Figure 3 The electronic device 300 provided in the embodiment of the present application includes: a processor 310 and a memory 320, wherein the memory 320 stores machine-readable instructions executable by the processor 310, and when the machine-readable instructions are executed by the processor 310, the above method is executed.

[0115] Figure 3 Each component shown in can be implemented by hardware, software or a combination thereof. The electronic device 300 may be a physical device, such as a server, a PC, etc., or a virtual device, such as a virtual machine, a virtualized container, etc. Moreover, the electronic device 300 is not limited to a single device, but may also be a combination of multiple devices or a cluster consisting of a large number of devices.

[0116] An embodiment of the present application further provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above method is executed.

[0117] Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0118] The embodiments of the present application also provide a computer program product, including computer program instructions, which execute the above method when run by a processor.

[0119] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and a module, a program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0120] In addition, in each embodiment of the embodiments of the present application, the various functional modules can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0121] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the embodiments of the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the embodiments of the present application.

Claims

1. A commodity identification method, characterized in that: include: Obtaining a product image and a product label image of the product to be identified; Extracting image features from the product image to obtain image features; Obtaining an authentic score based on the image features; the authentic score represents the probability that the product is authentic; Extracting commodity attribute information based on the commodity label image; Based on the authenticity score and the commodity attribute information, the commodity to be identified is identified to obtain an identification result.

2. The method according to claim 1, characterized in that The method further comprises: Acquire the commodity additional information of the commodity to be identified; wherein the commodity additional information is information about the commodity to be identified other than the commodity attribute information; Extracting text features from the additional information of the product to obtain additional features of the product; The step of identifying the commodity to be identified based on the authenticity score and the commodity attribute information to obtain an identification result includes: Based on the authenticity score, the commodity attribute information and the commodity additional features, the commodity to be identified is identified to obtain an identification result.

3. The method according to claim 1, characterized in that The commodity image includes a commodity foreground image; the step of obtaining the commodity image of the commodity to be identified includes: A plurality of foreground images of the commodity to be identified corresponding to a plurality of shooting angles of the commodity are obtained.

4. The method according to claim 1, characterized in that: The obtaining of the authenticity score according to the image features includes: According to the identification part corresponding to the commodity image, the distance between the image feature and the genuine classification center and the counterfeit classification center corresponding to the identification part are calculated respectively to obtain distance data; the genuine classification center is the genuine feature of at least one product unit at the identification part; the counterfeit classification center is the counterfeit feature of at least one product unit at the identification part; wherein one product unit represents a specification corresponding to the commodity; Based on the distance data, an authenticity score of the product image is determined.

5. The method according to claim 4, characterized in that The method further comprises: Determining the target classification center closest to the image feature based on the minimum value in the distance data; Acquire product unit information of the target classification center; the product unit information is used to represent specification attribute information corresponding to the commodity to be identified; Performing consistency check between the product unit information and the commodity attribute information to obtain a consistency check result; Based on the authenticity score and the commodity attribute information, the commodity to be identified is identified to obtain an identification result, including: Based on the authenticity score, the commodity attribute information and the consistency check result, the commodity to be identified is identified to obtain the identification result.

6. The method according to claim 1, characterized in that The step of identifying the commodity to be identified based on the authenticity score and the commodity attribute information to obtain an identification result includes: The authenticity score and the commodity attribute information are input into a large language model to obtain the identification result.

7. The method according to claim 6, characterized in that The identification result includes the authenticity of the commodity to be identified and the reason for determining the authenticity. Before inputting the authenticity score and the commodity attribute information into the large language model to obtain the identification result, the method further includes: Acquire a data set, the data set including a sample authenticity score, sample commodity attribute information, and labeling data corresponding to the sample authenticity score and the sample commodity attribute information, the labeling data including a true or false label and a reason for determining the authenticity; A pre-trained large language model is used to learn knowledge of identifying the authenticity of goods and attribution based on the dataset.

8. A computer program product, characterized in that The method comprises computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is executed.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images