Commodity identification method, device and equipment, storage medium and product

The method improves product recognition accuracy in e-commerce platforms by using multiple images to construct candidate product image sets and determine comprehensive similarity, ensuring users find the desired products.

CN119068467APending Publication Date: 2024-12-03CHINA MOBILE GROUP JIANGSU +1
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411097141.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Current e-commerce platforms provide inaccurate product recognition results due to scanning and identifying only one image, which lacks sufficient information, affecting the user's shopping experience and purchase rate.

Method used

A commodity identification method that involves obtaining multiple target product images, constructing candidate product image sets based on image similarity, determining comprehensive similarity, and selecting a recognition result image based on this similarity to improve accuracy.

Benefits of technology

Enhances the accuracy of product recognition by utilizing multiple images to ensure users find the desired products, thereby improving the shopping experience and purchase rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068467A_ABST
    Figure CN119068467A_ABST
Patent Text Reader

Abstract

The invention discloses a commodity recognition method and device, equipment, a storage medium and a product, relates to the technical field of digital image recognition, and aims to obtain a plurality of target commodity images of a to-be-recognized commodity and obtain more comprehensive commodity information for recognition. And selecting candidate commodity images matched with the target commodity images for each target commodity image according to the image similarity to generate candidate commodity image sets, and integrating the candidate commodity image sets of the target commodity images to obtain a total candidate commodity image set. Determining the comprehensive similarity between the candidate commodity images in the candidate commodity image set and each target commodity image, and selecting image elements as recognition result images matched with the to-be-recognized commodity according to the comprehensive similarity; in other words, the identification result image matched with the to-be-identified commodity is selected from the candidate commodity images of the candidate commodities by integrating the target commodity images, so that the accuracy of the identification result image is improved, and it is ensured that the user finds the desired commodity.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a commodity recognition method, device, equipment, storage medium and product. Background Art

[0002] Most of the current mainstream e-commerce clients provide the function of recommending products to users by scanning pictures. This method has become one of the important means for e-commerce platforms to attract users and increase sales. By scanning product pictures, users can quickly find the products they need, and can filter and compare according to their preferences and needs. At present, most e-commerce clients use image recognition to identify products, usually by scanning and identifying only one picture. Since an image contains less information, the result of scanning and identifying an image may not be the product the user wants, which causes the problem of inaccurate recognition results, which in turn affects the user's shopping experience and purchase rate.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present application is to provide a commodity identification method, device, equipment, storage medium and product, aiming to solve the technical problem that usually only one picture is scanned and identified, and since an image has less information, the result of scanning and identifying an image may not be the commodity that the user wants, thus causing inaccurate identification results.

[0005] To achieve the above objectives, the present application proposes a commodity identification method, which comprises:

[0006] Obtaining each target product image of the product to be identified;

[0007] For any one of the target product images, based on the image similarity between the target product image and the candidate product images of each candidate product, selecting a candidate product image to construct a candidate product image set matching the target product image;

[0008] Combining the candidate product image sets corresponding to the target product images into a total set of candidate product images;

[0009] Determining the comprehensive similarity between the candidate product images in the total set of candidate product images and each of the target product images, wherein the comprehensive similarity between the candidate product images is determined by the single image similarity between the candidate product images and each of the target product images;

[0010] Based on the comprehensive similarity, a candidate product image is selected from the total set of candidate product images as a recognition result image matching the product to be recognized, wherein the probability of the candidate product image being selected as the recognition result image is proportional to the comprehensive similarity.

[0011] Optionally, the step of selecting candidate product images to construct a set of candidate product images matching the target product image based on the image similarity between the target product image and candidate product images of each candidate product includes:

[0012] Determining image similarities between the target product image and each candidate product image;

[0013] A preset number of candidate product images are selected based on image similarity, and a candidate product image set matching the target product image is constructed based on the preset number of candidate product images, wherein the probability of a candidate product image being selected to participate in the construction of the candidate product image set is proportional to the image similarity.

[0014] Optionally, the candidate product images in the total set of candidate product images are image elements, and the step of determining the comprehensive similarity between the candidate product images in the total set of candidate product images and the target product images includes:

[0015] For any image element in the total set of candidate product images, determining a single image similarity between the image element and a target product image in each of the target product images;

[0016] The similarities of the individual images determined based on the image elements are accumulated to obtain the comprehensive similarity of the image elements.

[0017] Optionally, the step of determining the single image similarity between the image element and the target product image in each of the target product images comprises:

[0018] For any one of the target product images to be determined, the image element and the target product image to be determined are input into a preset similarity recognition model to obtain a single image similarity between the image element and the target product image to be determined, wherein the target product image to be determined is a target product image among the target product images for which a single image similarity has not been determined with the image element;

[0019] After traversing the target product images to be determined among the target product images, the similarities of the individual images are obtained.

[0020] Optionally, the step of acquiring each target commodity image of the commodity to be identified includes:

[0021] Acquire each original image of the commodity to be identified;

[0022] For any one of the original images, normalization processing is performed on the original image to obtain a normalized result of the original image:

[0023] The normalized results of the original images are obtained as the target product images.

[0024] Optionally, after the step of acquiring each target commodity image of the commodity to be identified, the method includes:

[0025] Selecting a target product image from the target product images as a basic product image;

[0026] Based on the similarity between the basic product image and the candidate product images among the candidate product images, a preliminary recommended image matching the product to be identified is selected from the candidate product images, and the preliminary recommended image is output, wherein the probability of the candidate product image being selected as the preliminary recommended image is proportional to the similarity;

[0027] In the case where the preliminary recommended image is excluded, the step of selecting, for any one of the target product images, a candidate product image to construct a candidate product image set matching the target product image based on image similarity between the target product image and candidate product images of each candidate product is performed.

[0028] In addition, to achieve the above objectives, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the commodity identification method described above.

[0029] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the commodity identification method described above are implemented.

[0030] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the commodity identification method described above are implemented.

[0031] One or more technical solutions proposed in this application have at least the following technical effects:

[0032] In an embodiment of the present application, target product images of the product to be identified are obtained; for any one of the target product images, based on the image similarity between the target product image and the candidate product images of each candidate product, a candidate product image is selected to construct a candidate product image set matching the target product image; the candidate product image sets corresponding to the target product images are combined into a total set of candidate product images; the comprehensive similarity between the candidate product images in the total set of candidate product images and the target product images is determined, wherein the comprehensive similarity of the candidate product images is determined by the single image similarity between the candidate product images and the target product images in the target product images; based on the comprehensive similarity, a candidate product image is selected from the total set of candidate product images as a recognition result image matching the product to be identified, wherein the probability of the candidate product image being selected as the recognition result image is proportional to the comprehensive similarity. It can be understood that, compared with the solution of scanning and identifying based on only one image in the related art, the present application will obtain multiple target product images of the product to be identified, and obtain more comprehensive product information for identification. According to the image similarity, a candidate product image matching each target product image is selected to generate a candidate product image set, and the candidate product image sets of each target product image are integrated to obtain a total set of candidate product images. Then, the comprehensive similarity between the candidate product images in the total set of candidate product images and each target product image is determined, and image elements are selected as the recognition result image matching the product to be identified based on the comprehensive similarity, that is, the recognition result image matching the product to be identified is selected from the candidate product images of the candidate products after integrating each target product image, thereby improving the accuracy of the recognition result image and ensuring that users find the desired product. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0035] Figure 1 This is a flow chart of the first embodiment of the commodity identification method of the present application;

[0036] Figure 2 This is a schematic diagram of the interface of the commodity identification result of the commodity identification method in the commodity identification method of this application on the mobile terminal;

[0037] Figure 3 This is a flow chart of the second embodiment of the commodity identification method of the present application;

[0038] Figure 4 This is a flow chart of the third embodiment of the commodity identification method of the present application;

[0039] Figure 5 This is a schematic diagram of the structure of a commodity identification device in the commodity identification method of the present application;

[0040] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the commodity identification method in the embodiment of the present application.

[0041] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0042] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0043] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0044] The main solution of the embodiment of the present application is: obtaining target product images of the product to be identified; for any one of the target product images, based on the image similarity between the target product image and the candidate product images of each candidate product, selecting a candidate product image to construct a candidate product image set matching the target product image; combining the candidate product image sets corresponding to the target product images into a total set of candidate product images; determining the comprehensive similarity between the candidate product images in the total set of candidate product images and the target product images, wherein the comprehensive similarity of the candidate product images is determined by the single image similarity between the candidate product images and the target product images in the target product images; selecting a candidate product image from the total set of candidate product images based on the comprehensive similarity as a recognition result image matching the product to be identified, wherein the probability of the candidate product image being selected as the recognition result image is proportional to the comprehensive similarity.

[0045] Most of the current mainstream e-commerce clients provide the function of recommending products to users by scanning pictures. This method has become one of the important means for e-commerce platforms to attract users and increase sales. By scanning product pictures, users can quickly find the products they need, and can filter and compare according to their preferences and needs. Usually, only one picture is scanned and recognized, and because an image has less information, the result of scanning and recognizing an image may not be the product the user wants, which causes the problem of inaccurate recognition results and the inability to find the product the user wants. Correspondingly, it also affects the user's shopping experience and purchase rate.

[0046] The present application provides a solution, which obtains multiple target product images of the product to be identified, and selects candidate product images of candidate products that match the target product image according to the image similarity to generate a candidate product image set, and combines each candidate product image set into a total set of candidate product images, and then determines the comprehensive similarity between the image elements of the candidate product images in the total set of candidate product images and each target product image, and finally selects image elements from the candidate product images according to the comprehensive similarity as the recognition result image that matches the product to be identified, and the image elements with higher comprehensive similarity are more likely to be selected as the recognition result image. It can be understood that compared with the solution of scanning and identifying based on only one image in the related art, the present application will obtain multiple target product images of the product to be identified, and obtain more comprehensive product information for identification. According to the image similarity, a candidate product image that matches each target product image is selected to generate a candidate product image set, and the candidate product image sets of each target product image are integrated to obtain a total set of candidate product images. Then, the comprehensive similarity between the candidate product images in the total set of candidate product images and each target product image is determined, and image elements are selected as the recognition result image matching the product to be identified based on the comprehensive similarity. That is, the recognition result image matching the product to be identified is selected from the candidate product images of the candidate products by combining the images of each target product, thereby improving the accuracy of the recognition result image and ensuring that users can find the products they want.

[0047] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a cloud platform, a computer, a mobile phone, etc., or an electronic device that can realize the above functions.

[0048] Based on this, the present application embodiment provides a commodity identification method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the commodity identification method of the present application.

[0049] In this embodiment, the commodity identification method includes steps S10 to S50:

[0050] Step S10, obtaining each target product image of the product to be identified;

[0051] It should be noted that in the embodiment of the present application, the implementing entity of the above-mentioned product identification method can be an e-commerce platform. Accordingly, a function of identifying images and finding products is configured on the e-commerce platform. The main goal of this function is to help users find products that match (or are the same as) the product to be identified from the candidate products sold on the e-commerce platform.

[0052] Exemplarily, the e-commerce platform can obtain the target product images of the products to be identified through the client APP (Application) installed on the user's mobile terminal. For example, the user can use the camera function of the mobile terminal to take multiple original images of the products to be identified, and the client APP will upload the original images to the e-commerce platform. Alternatively, the user can select multiple original images of the products to be identified locally on the mobile terminal through the client APP and upload them to the e-commerce platform. The e-commerce platform can directly use the original images as the target product images. In addition, it should be noted that the target product images of the products to be identified can be images of the products to be identified from different perspectives, so as to ensure that each target product image can contain the complete appearance features of the products to be identified as much as possible, thereby improving the accuracy of the recognition results.

[0053] Step S20, for any one of the target product images, based on the image similarity between the target product image and the candidate product images of each candidate product, selecting a candidate product image to construct a candidate product image set matching the target product image;

[0054] It should be noted that, since the processing process for each target product image is the same, in the embodiment of the present application, one target product image will be selected as an example for description.

[0055] Exemplarily, for any one of the target product images, the image similarity between the target product image and the candidate product images of each candidate product is first determined. Wherein, the candidate product refers to the product sold on the e-commerce platform, and correspondingly, the candidate product image of the candidate product is the product display image or the product physical image of the candidate product. The image similarity between the above-mentioned target product image and each candidate product image is calculated, wherein the method for calculating the similarity between images can be a square graph intersection method, a feature point matching method, a structural similarity measurement algorithm, and a similarity algorithm based on deep learning, etc. Specifically, the algorithm type can be set by the technician according to actual needs, and no specific restrictions are made here. After obtaining the image similarity between the above-mentioned target product image and each candidate product image, the candidate product image is selected based on the image similarity to construct a candidate product image set matching the target product image. For example, a candidate product image whose image similarity with the above-mentioned target product image is greater than a preset threshold can be selected as a candidate product image matching the target product image, and a candidate product image set is constructed based on the candidate product image matching the target product image. The candidate product images may also be sorted based on image similarity to obtain a candidate product image sequence, wherein the candidate product images that are closer to the front in the candidate product image sequence have greater image similarity, and then a preset number of candidate product images in the candidate product image sequence are selected as candidate product images that match the target product image to construct a candidate product image set.

[0056] Step S30, combining the candidate product image sets corresponding to the target product images into a total set of candidate product images;

[0057] Exemplarily, after traversing the target product images in each target product image, a candidate product image set of each target product image can be obtained, and then after removing duplicates of the candidate product images in each candidate product image set, a total set of candidate product images is obtained.

[0058] Step S40, determining the comprehensive similarity between the candidate product images in the total set of candidate product images and the target product images, wherein the comprehensive similarity between the candidate product images is determined by the single image similarity between the candidate product images and the target product images in the target product images;

[0059] Exemplarily, the comprehensive similarity between the candidate product images in the total set of candidate product images and each target product image is determined. It should be noted that the process of determining the comprehensive similarity of each candidate product image in the total set of candidate product images is the same. For example, for any candidate product image in the total set of candidate product images, in the process of determining the comprehensive similarity of the candidate product image, the single image similarity between the candidate product image and each target product image can be determined first, and then the comprehensive similarity of the candidate product image can be obtained by the similarities of each single image obtained based on the candidate product image. For example, the cumulative result or average result of the similarities of each single image corresponding to the candidate product image can be used as the comprehensive similarity between the candidate product image and each target product image. By repeating the above method, the comprehensive similarity of each candidate product image in the total set of candidate product images can be obtained.

[0060] Step S50 , selecting a candidate product image from the total set of candidate product images based on the comprehensive similarity as a recognition result image matching the product to be recognized, wherein the probability of the candidate product image being selected as the recognition result image is proportional to the comprehensive similarity.

[0061] Exemplarily, after obtaining the comprehensive similarity of each image element, an image element is selected from the total set of candidate product images based on the comprehensive similarity as a recognition result image that matches the product to be identified. And the probability of an image element being selected as a recognition result image is proportional to the corresponding comprehensive similarity. That is, the higher the comprehensive similarity of an image element, the higher the probability that the image element is selected as the recognition result image. Similarly, an image element with a comprehensive similarity greater than a preset similarity can be used as a recognition result image, or a preset number of image elements with the largest comprehensive similarity can be selected as a recognition result image. The candidate product corresponding to the recognition result image is also the product that matches or is the same as the product to be identified. Finally, the recognition result image is output to show the recognition result image to the user. For example, the e-commerce platform can send the recognition result image to the mobile terminal, and display the recognition result image through the client APP on the mobile terminal, and each recognition result image can be accompanied by the name, price, purchase link and other information of the product, and the user can choose whether to purchase these products according to their needs. Referring to Figure 2, it is a schematic diagram of the interface of the product recognition result of the product recognition method in this application on the mobile terminal. As shown Figure 2 As shown in , the same style candidate products are actually found. Correspondingly, the recognition result image, i.e. Figure 2 A schematic diagram of the result of outputting product images A to D on a mobile terminal, with product names and prices of product images A to D.

[0062] In this embodiment, each target product image of the product to be identified is obtained; for any one of the target product images, based on the image similarity between the target product image and the candidate product images of each candidate product, a candidate product image is selected to construct a candidate product image set matching the target product image; the candidate product image sets corresponding to the target product images are combined into a total set of candidate product images; the comprehensive similarity between the candidate product images in the total set of candidate product images and the target product images is determined; based on the comprehensive similarity, a candidate product image is selected from the total set of candidate product images as a recognition result image matching the product to be identified, wherein the probability of the candidate product image being selected as the recognition result image is proportional to the comprehensive similarity. It can be understood that, compared with the solution of scanning and identifying based on only one image in the related art, the present application will obtain multiple target product images of the product to be identified, and obtain more comprehensive product information for identification. According to the image similarity, a candidate product image matching each target product image is selected to generate a candidate product image set, and the candidate product image sets of each target product image are integrated to obtain a total set of candidate product images. Then, the comprehensive similarity between the candidate product images in the total set of candidate product images and each target product image is determined, and image elements are selected as the recognition result image matching the product to be identified based on the comprehensive similarity. That is, the recognition result image matching the product to be identified is selected from the candidate product images of the candidate products by combining the images of each target product, thereby improving the accuracy of the recognition result image and ensuring that users can find the products they want.

[0063] In a feasible implementation manner, the step of selecting candidate product images to construct a candidate product image set matching the target product image based on the image similarity between the target product image and the candidate product images of each candidate product includes steps S21 to S22:

[0064] Step S21, determining the image similarity between the target product image and each candidate product image;

[0065] Step S22, selecting a preset number of candidate product images based on image similarity, and constructing a candidate product image set matching the target product image based on the preset number of candidate product images, wherein the probability of a candidate product image being selected to participate in the construction of the candidate product image set is proportional to the image similarity.

[0066] Exemplarily, for the above-mentioned target product image, the image similarity between the target product image and each candidate product image is calculated. For example, the target product image and a candidate product image can be identified by a preset similarity recognition model, and the target product image and a candidate product image are input into the preset similarity recognition model, and the preset similarity recognition model can output the image similarity between the target product image and the candidate product image.

[0067] The preset similarity recognition model can be a neural network model, and the preset similarity recognition model has been pre-trained using a deep learning algorithm. In a feasible implementation, the preset similarity recognition model can be composed of two convolutional neural networks, assuming that the two convolutional neural networks are network A and network B, respectively inputting the target product image and a candidate product image into network A and network B, and network A and network B will output the similarity score between them (the similarity score can be the above-mentioned image similarity). For example, for two pictures P1 and P2, the similarity score between them can be calculated by the following formula:

[0068] score(P1,P2)=cos(θ)

[0069] Among them, θ represents the angle between two vectors (i.e., the vector corresponding to image P1 and the vector corresponding to image P2), and cos(θ) represents the cosine similarity between the two vectors.

[0070] It should also be noted that during the training process, the parameters of network A and network B will be optimized separately so that they can accurately predict the similarity scores between two pictures.

[0071] After determining the image similarity between the target product image and each candidate product image, a preset number of candidate product images are selected based on the image similarity to construct a set of candidate product images matching the target product image. That is, a preset number of candidate product images with the greatest image similarity are selected from each candidate product image, and the set formed by the selected candidate product images is used as the set of candidate product images matching the target product image. For example, the candidate product images are sorted from high to low according to the image similarity, and then the first k (i.e., the preset number) candidate product images are selected as the set of candidate product images.

[0072] In a feasible implementation manner, the candidate product images in the total set of candidate product images are image elements, and the step of determining the comprehensive similarity between the candidate product images in the total set of candidate product images and the target product images includes steps S41 to S42:

[0073] Step S41, for any image element in the total set of candidate product images, determining the single image similarity between the image element and the target product image in each of the target product images;

[0074] Step S42: accumulating the similarities of each single image determined based on the image element to obtain a comprehensive similarity of the image element.

[0075] It should be noted that the candidate product images in the total set of candidate product images are actually selected candidate product images. In order to distinguish them from the unselected candidate product images, the candidate product images in the total set of candidate product images are used as image elements. In addition, since the process of determining the comprehensive similarity of each image element in the total set of candidate product images is roughly the same, in this embodiment, the process of determining the comprehensive similarity of one image element will be taken as an example for description.

[0076] Exemplarily, for any image element in the total set of candidate product images, where the image element refers to an image element whose comprehensive similarity has not been determined in the total set of candidate product images, the single image similarity between the image element and the target product image in each target product image is determined, that is, there is a single image similarity between each target product image and the image element. The single image similarity between the image element and a target product image can also be obtained by identifying the preset similarity recognition model, that is, the image element and the target product image are input into the preset similarity recognition model, and the similarity score output by the preset similarity recognition model can be used as the above-mentioned single image similarity. After obtaining the single image similarity between the image element and each target product image, the single image similarities of the image element are accumulated to obtain the comprehensive similarity of the image element, that is, the single image similarities of the image element are added to obtain the comprehensive similarity of the image element.

[0077] In a feasible implementation manner, the step of determining the single image similarity between the image element and the target product image in each target product image comprises steps S421 to S422:

[0078] Step S421: for any one of the target product images to be determined, the image element and the target product image to be determined are input into a preset similarity recognition model to obtain a single image similarity between the image element and the target product image to be determined, wherein the target product image to be determined is a target product image among the target product images whose single image similarity has not been determined with the image element.

[0079] Exemplarily, for any one of the target product images to be determined, the target product image to be determined refers to a target product image among the target product images whose single image similarity with the image element has not been determined. The image element and the target product image to be determined are input into a preset similarity recognition model, and the single image similarity between the image element and the target product image to be determined can be identified. Accordingly, after traversing the target product images to be determined among the target product images, the similarities of the single images can be obtained.

[0080] To explain this solution clearly, we will take the example of obtaining two target product images of the products to be identified. For example, if the two target product images are X1 and X2, we can follow the following steps:

[0081] For X1, select K (i.e., the above-mentioned preset number) candidate product images that are most similar to X1 (i.e., the image similarity is the largest) from the candidate product images of each candidate product to obtain the candidate product image set Y1. Correspondingly, the product image set Y1 is:

[0082] Y1={M1,M2,...,M k};

[0083] For X2, select K (i.e., the preset number) candidate product images that are most similar to X2 (i.e., have the largest image similarity) from the candidate product images of each candidate product to obtain the candidate product image set Y2. Correspondingly, the product image set Y2 is:

[0084] Y2={N1,N2,...,N k};

[0085] Then Y1 and Y2 are combined into one set, namely the total set of candidate product images Z. The total set of candidate product images Z is as follows:

[0086] Z={Y1,Y2}={M1,M2,...M k ,N1,N2,...,N k};

[0087] in, That is, before merging, Y1 and Y2 are deduplicated to remove duplicate candidate product images in Y1 and Y2.

[0088] Then, using the above preset similarity recognition model, the single image similarity between each image element in Z and the above target product image X1 is calculated respectively, and the following results are obtained:

[0089] S1=(score(M1,X1),....,score(M k,X1),score(N1,X1),...,score(N k ,X1)) T ;

[0090] In the formula, score(M i , X1) indicates that the target product image X1 and the image element (or candidate product image) M i The similarity between individual images.

[0091] Similarly, the above preset similarity recognition model is used to calculate the single image similarity between each image element in Z and the above target product image X2, and the following results are obtained:

[0092] S2=(score(M1,X2),....,score(M k ,X2),score(N1,X2),...,score(N k ,X2)) T ;

[0093] In the formula, score(M i , X2) indicates that the target product image X2 and the image element (i.e., image element, or candidate product image) M i The similarity between individual images.

[0094] The sum of the similarities between each image element in Z and the target product image X1 and the target product image X2 (i.e., the comprehensive similarity) is calculated. Accordingly, the calculation result of the comprehensive similarity of each image element is as follows:

[0095]

[0096] In the formula, score(M i ,X1)+score(M i ,X2) represents the image element (i.e., image element, or candidate product image) M i The comprehensive similarity between the target product image X1 and the target product image X2.

[0097] Then, the comprehensive similarities determined by the elements in each row of S are sorted from large to small, and the first k image elements with the largest comprehensive similarities are selected as the recognition result images that are finally recommended to the user.

[0098] Reference Figure 3, is a flow chart of a second embodiment proposed based on the first embodiment of the present application. For the same or similar contents in this embodiment as those in the above embodiment, please refer to the above introduction and will not be repeated in the following. The step of obtaining each target product image of the product to be identified includes steps S11 to S13:

[0099] Step S11, obtaining each original image of the commodity to be identified;

[0100] Step S12: for any one of the original images, normalize the original image to obtain a normalized result of the original image:

[0101] Step S13: obtaining the normalized results of the original images as the target product images.

[0102] It should be noted that, in order to facilitate the comparison of similarities, in this embodiment, the acquired original image of the commodity to be identified will be preprocessed to obtain the target commodity image.

[0103] For example, the e-commerce platform may receive multiple original images of commodities to be identified uploaded from the client APP. Each original image is pre-processed to obtain each target commodity image. For example, for any one of the original images, the original image is normalized to obtain the normalized result of the original image.

[0104] For example, for any original image, it can be converted into a matrix X of size m×n, where m represents the number of rows of the image and n represents the number of columns of the image. For each original image, the following formula can be used to normalize it:

[0105]

[0106] In the formula, Indicates that in the original image corresponding matrix X, X ij The sum of squares of all elements in the row. This ensures that the value of each element is between [0,1].

[0107] By traversing each of the original images and performing normalization processing on each of the original images, each normalization result can be obtained, and each normalization result is used as each target product image.

[0108] It is understandable that normalization processing can reduce or eliminate the impact of noise in the image (such as differences in image brightness caused by lighting changes) on similarity comparison, so that the similarity comparison focuses more on the structure and content of the image, and the similarity comparison result is more accurate.

[0109] Reference Figure 4 , is a flowchart of a third embodiment proposed based on the first embodiment and the second embodiment of the present application. For the same or similar contents as the above embodiments in this embodiment, please refer to the above introduction and will not be repeated later. After the step of obtaining the images of each target product to be identified, the method includes steps S01 to S03:

[0110] Step S01, selecting a target product image from the target product images as a basic product image;

[0111] Step S02, based on the similarity between the basic product image and the candidate product images among the candidate product images, selecting a preliminary recommended image matching the product to be identified from the candidate product images, and outputting the preliminary recommended image, wherein the probability of the candidate product image being selected as the preliminary recommended image is proportional to the similarity;

[0112] Step S03, when the preliminary recommended image is excluded, executing the step of selecting, for any one of the target product images, a candidate product image to construct a candidate product image set matching the target product image based on the image similarity between the target product image and the candidate product images of each candidate product.

[0113] It should be noted that, in this embodiment, in order to improve the commodity recognition speed and save computing resources, a target commodity image may be first selected from each target commodity image as a basic commodity image for preliminary screening.

[0114] Exemplarily, the e-commerce platform can randomly select a target product image from each target product image as the basic product image. It can also select a basic product image that can show the full picture of the product as much as possible. Exemplarily, for any target product image, the target product image is binarized to obtain a binarization result, and then the contour feature is extracted from the binarization result, and the contour feature closest to the geometric center of the target product image is used as the target contour feature, and the area ratio of the target contour feature in the target product image is determined. After traversing each target product image in each target product image, the target product image with the largest area ratio of the target contour feature is used as the above-mentioned basic product image. Correspondingly, after selecting the basic product image, the similarity between the basic product image and the candidate product image in each candidate product image is determined. It should be noted that the determination of the similarity can also be obtained by the above-mentioned preset similarity recognition model, such as inputting a basic product image and a candidate product image into the preset similarity recognition model, and the similarity score output by the preset similarity recognition model is also the above-mentioned similarity, so it will not be repeated here. A preliminary recommended image matching the product to be identified is selected from each candidate product image according to the similarity, and the candidate product image with higher similarity is more likely to be selected as the preliminary recommended image. For example, candidate product images with a similarity greater than a preset similarity threshold can be selected as preliminary recommended images matching the product to be identified, or a preset number of candidate product images with the greatest similarity can be selected as preliminary recommended images matching the product to be identified. Then the preliminary recommended image is output. Similarly, the preliminary recommended image is similar to the recognition result image, and both are results for display to the user. That is, the e-commerce platform can send the preliminary recommended image to the mobile terminal, and display the preliminary recommended image through the client APP on the mobile terminal, and each preliminary recommended image can be accompanied by information such as the name, price, and purchase link of the product.

[0115] It should be noted that after the preliminary recommendation image is output, a user confirmation option may be provided, and the user confirmation options include the existence of the required product and the non-existence of the required product. When the user selects the option of the non-existence of the required product, the preliminary recommendation image representing the above content is excluded. In this case, for any one of the target product images, based on the image similarity between the target product image and the candidate product images of each candidate product, a step of selecting a candidate product image to construct a candidate product image set matching the target product image may be performed, that is, a recognition result image is selected with reference to each target product image. It is worth noting that in the step of combining the candidate product image sets corresponding to the target product images into a total set of candidate product images in this embodiment, in addition to deduplicating duplicate candidate product images, the candidate product images in each candidate product image set that are identical to the preliminary recommendation image are removed and then combined into a total set of candidate product images.

[0116] It is understandable that in this embodiment, a single target product image is used first for product recognition to improve product recognition speed and save computing resources. If the recognition result of a single target image is inaccurate, multiple target images are combined for product recognition to ensure that an accurate product image can be recognized and given to the user.

[0117] The present application provides a commodity identification device, referring to Figure 5 , the commodity identification device comprises:

[0118] An acquisition module 10 is used to acquire each target commodity image of the commodity to be identified;

[0119] A generating module 20 is configured to select a candidate product image for any one of the target product images based on the image similarity between the target product image and the candidate product images of each candidate product to construct a candidate product image set matching the target product image;

[0120] A combining module 30, configured to combine the candidate product image sets corresponding to the target product images into a total set of candidate product images;

[0121] a determination module 40, configured to determine a comprehensive similarity between the candidate product images in the total set of candidate product images and each of the target product images, wherein the comprehensive similarity between the candidate product images is determined by a single image similarity between the candidate product images and each of the target product images;

[0122] The selection module 50 is used to select a candidate product image from the total set of candidate product images based on the comprehensive similarity as a recognition result image matching the product to be identified, wherein the probability of the candidate product image being selected as the recognition result image is proportional to the comprehensive similarity.

[0123] Optionally, the generating module 20 is further used for:

[0124] Determining image similarities between the target product image and each candidate product image;

[0125] A preset number of candidate product images are selected based on image similarity, and a candidate product image set matching the target product image is constructed based on the preset number of candidate product images, wherein the probability of a candidate product image being selected to participate in the construction of the candidate product image set is proportional to the image similarity.

[0126] Optionally, the candidate product images in the total set of candidate product images are image elements, and the determining module 40 is further configured to:

[0127] For any image element in the total set of candidate product images, determining a single image similarity between the image element and a target product image in each of the target product images, wherein the candidate product images in the total set of candidate product images are image elements;

[0128] The similarities of the individual images determined based on the image elements are accumulated to obtain the comprehensive similarity of the image elements.

[0129] Optionally, the determining module 40 is further configured to:

[0130] For any one of the target product images to be determined, the image element and the target product image to be determined are input into a preset similarity recognition model to obtain a single image similarity between the image element and the target product image to be determined, wherein the target product image to be determined is a target product image among the target product images for which a single image similarity has not been determined with the image element;

[0131] After traversing the target product images to be determined among the target product images, the similarities of the individual images are obtained.

[0132] Optionally, the acquisition module 10 is further used for:

[0133] Acquire each original image of the commodity to be identified;

[0134] For any one of the original images, normalization processing is performed on the original image to obtain a normalized result of the original image:

[0135] The normalized results of the original images are obtained as the target product images.

[0136] Optionally, the commodity identification device further includes a preliminary screening module 60, and the preliminary screening module 60 is further used for:

[0137] Selecting a target product image from the target product images as a basic product image;

[0138] Based on the similarity between the basic product image and the candidate product images among the candidate product images, a preliminary recommended image matching the product to be identified is selected from the candidate product images, and the preliminary recommended image is output, wherein the probability of the candidate product image being selected as the preliminary recommended image is proportional to the similarity;

[0139] In the case where the preliminary recommended image is excluded, the step of selecting, for any one of the target product images, a candidate product image to construct a candidate product image set matching the target product image based on image similarity between the target product image and candidate product images of each candidate product is performed.

[0140] The commodity identification device provided by the present application adopts the commodity identification method in the above embodiment, aiming to solve the technical problem that usually only one picture is scanned and identified, and since an image has less information, the result of scanning and identifying an image may not be the commodity that the user wants, which causes inaccurate identification results. Compared with the prior art, the beneficial effects of the commodity identification device provided by the embodiment of the present application are the same as the beneficial effects of the commodity identification method provided by the above embodiment 1, and the other technical features in the commodity identification device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0141] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the commodity identification method in the above-mentioned embodiment 1.

[0142] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic devices in the embodiments of the present application may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0143] like Figure 6As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0144] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0145] The electronic device provided by the present application adopts the commodity identification method in the above embodiment, which can solve the technical problem that usually only one picture is scanned and identified, and because one image has less information, the result of scanning and identifying one image may not be the commodity that the user wants, which also causes inaccurate identification results. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the commodity identification method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0146] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0147] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0148] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the commodity identification method in the above-mentioned embodiment.

[0149] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0150] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0151] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: obtains target product images of the product to be identified; for any one of the target product images, based on the image similarity between the target product image and the candidate product images of the candidate products, selects a candidate product image to construct a candidate product image set that matches the target product image; combines the candidate product image sets corresponding to the target product images into a total set of candidate product images; determines the comprehensive similarity between the candidate product images in the total set of candidate product images and the target product images; selects a candidate product image from the total set of candidate product images based on the comprehensive similarity as a recognition result image that matches the product to be identified, wherein the probability that the candidate product image is selected as the recognition result image is proportional to the comprehensive similarity.

[0152] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0153] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0154] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0155] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned commodity identification method, and can solve the technical problem that the existing data display method is difficult to intuitively and clearly complete data visualization. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the commodity identification method provided in the above-mentioned embodiment, and will not be repeated here.

[0156] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned commodity identification method when executed by a processor.

[0157] The computer program product provided in this application can solve the technical problem of commodity identification. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the commodity identification method provided in the above embodiment, which will not be repeated here.

[0158] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A commodity identification method, characterized in that: The commodity identification method comprises the following steps: Obtaining each target product image of the product to be identified; For any one of the target product images, based on the image similarity between the target product image and the candidate product images of each candidate product, selecting a candidate product image to construct a candidate product image set matching the target product image; Combining the candidate product image sets corresponding to the target product images into a total set of candidate product images; Determining the comprehensive similarity between the candidate product images in the total set of candidate product images and each of the target product images, wherein the comprehensive similarity between the candidate product images is determined by the single image similarity between the candidate product images and each of the target product images; Based on the comprehensive similarity, a candidate product image is selected from the total set of candidate product images as a recognition result image matching the product to be recognized, wherein the probability of the candidate product image being selected as the recognition result image is proportional to the comprehensive similarity.

2. The commodity identification method according to claim 1, characterized in that: The step of selecting candidate product images to construct a candidate product image set matching the target product image based on the image similarity between the target product image and the candidate product images of each candidate product comprises: Determining image similarities between the target product image and each candidate product image; A preset number of candidate product images are selected based on image similarity, and a candidate product image set matching the target product image is constructed based on the preset number of candidate product images, wherein the probability of a candidate product image being selected to participate in the construction of the candidate product image set is proportional to the image similarity.

3. The commodity identification method according to claim 1, characterized in that: The candidate product images in the total set of candidate product images are image elements, and the step of determining the comprehensive similarity between the candidate product images in the total set of candidate product images and the target product images comprises: For any image element in the total set of candidate product images, determining a single image similarity between the image element and a target product image in each of the target product images; The similarities of the individual images determined based on the image elements are accumulated to obtain the comprehensive similarity of the image elements.

4. The commodity identification method according to claim 3, characterized in that: The step of determining the single image similarity between the image element and the target product image in each target product image comprises: For any one of the target product images to be determined, the image element and the target product image to be determined are input into a preset similarity recognition model to obtain a single image similarity between the image element and the target product image to be determined, wherein the target product image to be determined is a target product image among the target product images for which a single image similarity has not been determined with the image element.

5. The commodity identification method according to claim 1, characterized in that: The step of obtaining each target commodity image of the commodity to be identified comprises: Acquire each original image of the commodity to be identified; For any one of the original images, normalize the original image to obtain a normalized result of the original image; The normalized results of the original images are obtained as the target product images.

6. The commodity identification method according to claim 1, characterized in that: After the step of acquiring each target commodity image of the commodity to be identified, the method includes: Selecting a target product image from the target product images as a basic product image; Based on the similarity between the basic product image and the candidate product images among the candidate product images, a preliminary recommended image matching the product to be identified is selected from the candidate product images, and the preliminary recommended image is output, wherein the probability of the candidate product image being selected as the preliminary recommended image is proportional to the similarity; In the case where the preliminary recommended image is excluded, the step of selecting, for any one of the target product images, a candidate product image to construct a candidate product image set matching the target product image based on image similarity between the target product image and candidate product images of each candidate product is performed.

7. A commodity identification device, characterized in that: The commodity identification device comprises: An acquisition module, used to acquire each target product image of the product to be identified; a generating module, configured to select, for any one of the target product images, a candidate product image to construct a candidate product image set matching the target product image based on image similarity between the target product image and candidate product images of each candidate product; A combination module, used to combine the candidate product image sets corresponding to the target product images into a total set of candidate product images; a determination module, configured to determine a comprehensive similarity between the candidate product images in the total set of candidate product images and each of the target product images, wherein the comprehensive similarity between the candidate product images is determined by a single image similarity between the candidate product images and each of the target product images; A selection module is used to select a candidate product image from the total set of candidate product images based on the comprehensive similarity as a recognition result image matching the product to be identified, wherein the probability of the candidate product image being selected as the recognition result image is proportional to the comprehensive similarity.

8. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the commodity identification method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the commodity identification method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the commodity identification method according to any one of claims 1 to 6 are implemented.

Citation Information

Cited By

  • Commodity image recognition result de-weighting method and system

    CN119723125A

  • A method and system for removing duplicates from commodity image recognition results

    CN119723125B