A method, apparatus, and electronic device for classifying product images.

Abstract

This invention provides a method, apparatus, and electronic device for classifying product images. The method includes: crawling product image data for corresponding categories from e-commerce websites based on classification data in a classification lookup table; dividing the product image data set into a training set and a validation set according to proportions; training a deep convolutional neural network (DCNN) classification model; evaluating the performance of the DCNN classification model based on product image data in the validation set; and selecting the DCNN classification model with the best performance for product image classification prediction, thereby obtaining product image classification system data. Through the method, apparatus, and electronic device provided by this invention, without any manually labeled samples, the system automatically classifies product images from e-commerce website advertising materials using convolutional neural network technology, eliminating the need for manual intervention and saving significant labor costs.

Description

Technical Field

[0001] This invention relates to the field of e-commerce technology, and more specifically, to a method, apparatus, electronic device, and computer-readable storage medium for classifying product images. Background Technology

[0002] E-commerce advertising creative elements include images, videos, and copy. Before promoting products in a particular country or region, advertisers in the e-commerce industry refer to Benchark data for that area to conduct pre-campaign performance analysis, product selection, and budget allocation. Therefore, e-commerce Benchark data is crucial. E-commerce Benchark data includes performance data for advertising across different countries and product categories. Performance data includes metrics such as spend, impressions, clicks, purchases, click-through rate, conversion rate, CPS, and ROI. Product categories include primary, secondary, and tertiary product classification data. Primary categories include apparel; secondary categories include menswear, womenswear, and wedding dresses; and tertiary categories include shirts, T-shirts, suits, and trousers. Therefore, to generate e-commerce Benchark data, e-commerce advertisements need to be categorized.

[0003] The existing solution involves manual categorization at the ad account level. This requires manually setting first-, second-, and third-level categories for ad accounts, which is labor-intensive and resource-intensive. Furthermore, the granularity of manually set categorization data is inaccurate. For example, suppose an ad account contains ads promoting both clothing and shoes. If the account level is manually set to only clothing, the resulting categorization data will be inaccurate, and the generated e-commerce industry Benchark data will also be inaccurate, limiting the reference value of Benchark data.

[0004] Most e-commerce ads contain image assets. Automatically classifying these images can improve the coverage and accuracy of e-commerce classified ads. Currently, conventional convolutional neural network image classification methods require manual annotation of a large amount of sample data, which is labor-intensive. Summary of the Invention

[0005] To address the existing technical problems, embodiments of the present invention provide a method, apparatus, electronic device, and computer-readable storage medium for classifying product images.

[0006] In a first aspect, embodiments of the present invention provide a method for classifying product images, including:

[0007] A classification comparison table is established between the product image classification system data and the e-commerce website classification system. Based on the classification data in the classification comparison table, product image data of the corresponding categories on the e-commerce website is crawled.

[0008] The product image data set is divided into a training set and a validation set according to the proportions.

[0009] The product image data in the training set is used as training sample data to train a deep convolutional neural network classification model.

[0010] The performance of the deep convolutional neural network classification model is evaluated based on the product image data in the validation set.

[0011] The deep convolutional neural network classification model with the best performance is selected for classification prediction of the product images to obtain the product image classification system data.

[0012] Secondly, embodiments of the present invention provide a product image classification device, characterized in that it includes:

[0013] The data crawling module is used to determine the classification comparison table between the product image classification system data and the e-commerce website classification system, and crawl the product image data of the corresponding category of the e-commerce website according to the classification data in the classification comparison table;

[0014] The set partitioning module is used to divide the set of product image data into a training set and a validation set according to a ratio.

[0015] The model training module is used to train a deep convolutional neural network classification model by using the product image data in the training set as training sample data.

[0016] The model evaluation module is used to evaluate the model performance of the deep convolutional neural network classification model based on the product image data in the validation set.

[0017] The model selection module is used to select the deep convolutional neural network classification model with the best model performance for the classification prediction of the product image, thereby obtaining the product image classification system data.

[0018] Thirdly, embodiments of the present invention provide an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor. The transceiver, the memory, and the processor are connected via the bus. When the computer program is executed by the processor, it implements the steps in the product image classification method as described above.

[0019] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in the product image classification method described above.

[0020] The methods, apparatus, electronic devices, and computer-readable storage media provided in this invention, without any manually labeled samples, utilize existing e-commerce website platform advertising image material data and employ convolutional neural network technology to automatically classify product images in e-commerce advertisements. This eliminates the need for manual intervention, saving significant labor costs. After classifying product images in e-commerce advertisements, Benchark data for the e-commerce industry can be generated, improving the coverage and accuracy of e-commerce advertisement classification. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the background art, the accompanying drawings used in the embodiments of the present invention or the background art will be described below.

[0022] Figure 1 A schematic diagram of the AliExpress e-commerce website classification system according to an embodiment of the present invention is shown;

[0023] Figure 2 This diagram illustrates a comparison of product image classification and AliExpress product classification data in an embodiment of the present invention.

[0024] Figure 3 A schematic diagram of a list of search keywords without corresponding categories is shown in an embodiment of the present invention.

[0025] Figure 4 A schematic diagram of the InceptionV3 structure according to an embodiment of the present invention is shown;

[0026] Figure 5 A schematic diagram of the InceptionV3 classification score calculation process according to an embodiment of the present invention is shown;

[0027] Figure 6 A flowchart illustrating a method for classifying product images provided in an embodiment of the present invention is shown;

[0028] Figure 7 A flowchart of the product image deduplication method according to an embodiment of the present invention is shown;

[0029] Figure 8 A schematic diagram of the structure of a product image classification device provided in an embodiment of the present invention is shown;

[0030] Figure 9 The diagram shows a structural schematic of an electronic device for classifying product images provided in an embodiment of the present invention. Detailed Implementation

[0031] To ensure clarity and conciseness in describing the embodiments of the present invention, a brief introduction to the relevant concepts or technologies is given first:

[0032] E-commerce advertising: Digital advertisements published by advertisers to promote products on media platforms (such as Facebook, Google, TikTok). The content of these advertisements can include images, videos, and text.

[0033] E-commerce product classification: E-commerce product classification system data, including three levels of classification. First-level classifications include: clothing / shoes / bags, etc.; second-level classifications include: men's clothing / women's clothing / bags and accessories / shoes and accessories, etc.; and third-level classifications include: coats / overcoats / suits / suits / trousers / skirts / women's shoes / men's shoes, etc.

[0034] Benchark data for the e-commerce industry: E-commerce advertising data is refined to three levels of categories and dimensions such as country / gender / age / placement, with core performance metrics including spend, impressions, clicks, purchases, CPS, and ROI. Benchark data plays a significant role in CPS / ROI analysis and prediction in the early stages of advertising campaigns, as well as in product selection before launching e-commerce ads.

[0035] CPS: Average cost per conversion for e-commerce advertising.

[0036] ROI: Return on investment for advertising, the sales revenue generated for every dollar spent on advertising.

[0037] Selenium, a web crawling framework, is a web automation testing tool originally developed for website automation testing. Selenium can directly invoke the browser, receiving commands to automatically load pages and retrieve necessary data.

[0038] Stochastic Gradient Descent (SGD): Also known as Incremental Gradient Descent, SGD is an iterative method for optimizing differentiable objective functions. It iteratively updates weights and biases by computing the gradient of the loss function on mini-batch data.

[0039] InceptionV3 pre-trained models: InceptionV3 is a convolutional neural network architecture proposed by Google, which is an improvement on InceptionNet and has gone through several versions. The main ideas of the InceptionV3 architecture are factorized convolutions and aggressive regularization. The InceptionV3 pre-trained model is a model pre-trained on the InceptionNet dataset for image feature extraction.

[0040] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used in the embodiments of this invention have the same meaning as commonly understood by one of ordinary skill in the art to which these embodiments pertain. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0041] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDAs (Personal Digital Assistants) that may include radio frequency receivers, pagers, internet / intranet access, web browsers, notebooks, calendars, and / or GPS (Global Positioning System) receivers; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured for local operation, and / or distributed operation in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities; it can also be a smart TV, set-top box, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, etc.

[0042] The hardware referred to by the names "server," "client," and "service node" in the embodiments of this invention is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components disclosed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0043] Those skilled in the art will understand that the concept of "server" used in the embodiments of this invention can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided; physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster.

[0044] Those skilled in the art will understand that embodiments of the present invention can be implemented as methods, apparatuses, electronic devices, and computer-readable storage media. Therefore, embodiments of the present invention can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. Furthermore, in some embodiments, embodiments of the present invention can also be implemented as a computer program product contained in one or more computer-readable storage media, the computer-readable storage media containing computer program code.

[0045] The aforementioned computer-readable storage medium may be any combination of one or more computer-readable storage media. Computer-readable storage media include: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any combination thereof. In embodiments of the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0046] The computer program code contained in the aforementioned computer-readable storage medium may be transmitted using any suitable medium, including wireless, wire, optical fiber, radio frequency (RF), or any suitable combination thereof.

[0047] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The computer program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer or an external computer via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0048] The embodiments of the present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices, and computer-readable storage media.

[0049] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine that, when executed by a computer or other programmable data processing apparatus, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0050] These computer-readable program instructions may also be stored in a computer-readable storage medium that enables a computer or other programmable data processing device to function in a particular manner. In this way, the instructions stored in the computer-readable storage medium produce an instruction apparatus product that includes the functions / operations specified in the blocks of a flowchart and / or block diagram.

[0051] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable data processing apparatus provide a process for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0052] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0053] Figure 6 A flowchart of a product image classification method provided by an embodiment of the present invention is shown. Figure 6 As shown, the method includes:

[0054] Step S101: Determine the classification comparison table between the product image classification system data and the e-commerce website classification system, and crawl the product image data of the corresponding category from the e-commerce website based on the classification data in the classification comparison table;

[0055] The product image classification system data in this embodiment of the invention (as shown in Table 1) is compared with the classification system of e-commerce websites (e.g., AliExpress, https: / / www.aliexpress.com / ) (as shown in Table 1). Figure 1 The classification lookup table is as follows. For example, the product image category "Apparel / Women's Clothing / T-shirts" corresponds to the e-commerce website category "Women's Clothing / Tops&Tees / Women's Tops&Tees". The specific classification lookup table data is as follows: Figure 2 As shown.

[0056] Table 1

[0057]

[0058] Based on the category comparison table, the corresponding category link data on AliExpress, i.e. Figure 2 The `link_url` column data is obtained by using web scraping techniques (such as the Selenium framework) to crawl product image data from the corresponding category on the AliExpress website.

[0059] For example: Figure 2 The product image category "Apparel / Women's Clothing / T-shirts" corresponds to the AliExpress category "Women's Clothing / Tops&Tees / Women's Tops&Tees," with the corresponding link URL: https: / / www.aliexpress.com / category / 200000785 / tops-tees.html. Product image data from multiple pages on this page was retrieved using web scraping techniques and used as sample data for the product image category "Apparel / Women's Clothing / T-shirts."

[0060] If product image categories don't correspond one-to-one with AliExpress website categories, such as "Beauty & Personal Care / Personal Care / Body Care," and no corresponding category can be found on AliExpress, the inventors considered several keywords related to "body care," such as "shower gel / shampoo / scrub / body lotion." Using these keywords as search terms, they employed web scraping technology to retrieve product data from AliExpress and extract the image data from the search results. This image data was then used as sample data for that category. Specific search keywords for categories that don't correspond are listed below. Figure 3 As shown.

[0061] Step S103: The crawled product images contain duplicate data, which needs to be deduplicated.

[0062] Figure 7 A flowchart of a product image deduplication method provided by an embodiment of the present invention is shown. Figure 7 As shown, the method includes the following steps S1031-S1033.

[0063] Step S1031: Obtain multiple historical product images and determine the feature vectors of the historical product images;

[0064] Historical product images can represent previously processed image materials. These historical product images often contain duplicate images, such as images that are identical except for resolution, images that differ only in text, or images with minor differences in graphic elements. In this embodiment of the invention, multiple historical product images can be acquired, and a feature vector can be extracted for each historical product image. This feature vector represents the characteristics of the corresponding historical product image.

[0065] Step S1033: Calculate the similarity between the feature vectors of historical product images, and remove duplicate images from those with similarity greater than a preset threshold to obtain images to be stored that do not contain duplicate images.

[0066] This invention allows for the calculation of the similarity between the feature vectors of any two historical product images. This similarity represents the degree of similarity between the feature vectors of the two selected historical product images, and can be obtained by calculating the inner product or Euclidean distance of the feature vectors of any two historical product images. In this embodiment, a preset threshold can be set, and the similarity between the feature vectors of two historical product images can be determined using this preset threshold as a boundary. When the similarity between the feature vectors of the two selected historical product images is greater than the preset threshold, it can be determined that the feature vectors of the two historical product images are similar, meaning that the two historical product images are considered to be duplicates of each other. Therefore, deduplication can be performed on the feature vectors of the similar historical product images. The deduplication method can be to select one feature vector from the feature vectors of historical product images whose similarity is greater than the preset threshold as the feature vector of the image to be stored, i.e., select the image to be stored, and delete other historical product images similar to the image to be stored. This achieves the goal of selecting one of the historical product images that are duplicates of each other as the image to be stored, thus ensuring that each selected image to be stored is unique. When the similarity between the feature vectors of two selected historical product images is not greater than a preset threshold, it can be determined that the feature vectors of the two historical product images are not similar, that is, the two historical product images are not similar to each other, and thus all historical product images that are not similar to each other can be used as images to be stored.

[0067] Step S1035: Update the vector database in the manner of storing the images to be stored, and obtain a vector database that does not contain duplicate images. The vector database is used to store the feature vectors of at least some of the images to be stored.

[0068] The vector database is a database capable of storing some or all of the feature vectors of the images to be stored. It can be Milvus (an open-source similarity search engine for massive feature vectors), and Milvus is designed based on a heterogeneous many-core computing framework, resulting in lower cost and better performance. In this embodiment of the invention, a quantized index, such as the IVF_SQ8 index, can be established for the vector database. The IVF_SQ8 index is a quantized index of Milvus. When disk, memory, or video memory resources are limited, it can convert each FLOAT (4 bytes) in the original vector to UINT8 (1 byte) by performing scalar quantization on the vectors, thereby reducing the consumption of disk, memory, and video memory resources to 1 / 4 to 1 / 3 of the original. In the initial update of the vector database (e.g., initialization), the vector database can store the feature vectors of all images to be stored. In subsequent updates (e.g., enrichment), the vector database can selectively store feature vectors of some or all of the images to be stored, depending on the actual situation. Specifically, it can be based on the similarity between the feature vector of the image to be stored and the feature vectors stored in the vector database, to ensure that the images corresponding to all feature vectors stored in the vector database are not duplicates. In this embodiment of the invention, the image to be stored can be updated by storing it, for example, by directly saving the feature vector of the image to be stored during the first update of the vector database; or, during subsequent updates to the vector database (such as supplementing and enriching the vector database), the similarity between the feature vector of the image to be stored and the feature vector stored in the vector database at that time can be calculated using the method described in step S1033 above. If the similarity between the feature vector of the image to be stored and the feature vector stored in the vector database at that time is less than a preset value, the feature vector of the image to be stored is stored in the vector database, thus realizing the process of updating the vector database. The preset value can be the same as the preset threshold in step S1033 above. Since the feature vectors of the images to be stored stored in the vector database are dissimilar, the images to be stored in the final vector database are all unique.

[0069] This invention employs a method of calculating the similarity of feature vectors from historical product images. Feature vectors from historical product images with similarity greater than a preset threshold are selected. The corresponding historical product images are then treated as duplicate images, and deduplication is performed on these duplicate images to obtain a class of unique images, i.e., images to be stored. This class of images is then updated by storing them in a vector database, ultimately ensuring that the vector database contains no duplicate or similar feature vectors from historical product images. This method demonstrates high accuracy and recall when determining whether any two historical product images are similar. Furthermore, deduplication significantly reduces the amount of data that needs to be stored in the vector database, resulting in a vector database with high retrieval efficiency and a compact size.

[0070] Step S105: Divide the product image data set into a training set and a validation set according to the proportions;

[0071] The product image data is divided into a training set and a validation set in an 8:2 ratio. The product image data in the training set is used as training sample data to train a deep convolutional neural network classification model.

[0072] Using product image data from the training set as training sample data, the InceptionV3 deep convolutional neural network classification model was trained using gradient descent (SGD) until the model converged.

[0073] The above model is the InceptionV3 model pre-trained on ImageNet. In this embodiment of the invention, the last classification layer of the InceptionV3 model is removed, and a new classification layer designed by the inventors is added for training the classification model and classification prediction.

[0074] For example, the primary categories are: Apparel, Footwear & Bags, Jewelry / Accessories, Digital Appliances, Beauty & Health, Personal Care, Home & Garden, Maternity & Baby, Toys / Hobbies, Industrial Equipment & Tools, Packaging & Office Supplies, and Automotive & Transportation, totaling 12 subcategories. The inventors added a final classification layer with 12 neurons for primary category management of e-commerce product images. The steps for secondary and tertiary classification follow the same pattern.

[0075] Gradient descent (SGD) significantly outperforms naive gradient descent on highly nonconvex loss surfaces; this simple hill-climbing method has dominated modern nonconvex optimization. The SGD parameter update formula is:

[0076]

[0077] Where θ is the parameter to be trained, J(θ) is the cost function, and α is the learning rate.

[0078] The update process for model parameter θ is as follows:

[0079] a) Initialize the parameter θ and the learning rate α;

[0080] b) For the current batch of sample data, calculate the gradient at each θ, i.e.

[0081] c) Update parameter θ using formula 1;

[0082] d) Repeat steps b) and c) above until the cost function value stabilizes and does not decrease.

[0083] The structure of InceptionV3 is as follows: Figure 4 As shown, in an embodiment of the present invention, the inventor removes Figure 4 In the final classification layer, an additional classification layer designed by the inventor is added, and the activation function of this layer is set to the Softmax function for calculating the final classification score. The classification result is the category corresponding to the highest classification score. The classification score calculation process is as follows: Figure 5 As shown.

[0084] Step S107: Evaluate the performance of the deep convolutional neural network classification model based on the product image data in the validation set;

[0085] The validation set in step S105 is used for evaluating and tuning the InceptionV3 deep convolutional neural network classification model. The evaluation metric used for model tuning is Accuracy, defined as follows:

[0086]

[0087] Using product image data from the validation set as validation sample data, the model hyperparameters were adjusted: learning rate α and batch size. The learning rate α was adjusted from 0.001 to 0.0001, and the batch size could be adjusted to {8, 16, 32, 64, 128, 256, 512}.

[0088] Step S109: Repeat step S105 to retrain the InceptionV3 deep convolutional neural network classification model, and evaluate the model performance of the deep convolutional neural network classification model in step S107. Select the deep convolutional neural network classification model with the best model performance for the classification prediction of the product image to obtain product image classification system data.

[0089] Step S111: For the first-level, second-level, and third-level categories of product images, repeat the above steps S105-S109 to obtain the first-level, second-level, and third-level category system data of product images.

[0090] The product image classification method of this invention utilizes existing e-commerce website platform advertising image material data and employs convolutional neural network technology to automatically classify e-commerce advertisements for products without any manually labeled samples, saving significant labor costs.

[0091] The product image classification method of this invention can generate Benchark data for the e-commerce industry after classifying product images in e-commerce advertisements, thereby improving the coverage and accuracy of e-commerce advertisement classification.

[0092] The above text combined Figures 1 to 7 The present invention describes in detail a method for classifying product images according to embodiments of the present invention. The following will combine... Figure 8 The present invention describes in detail a product image classification device according to an embodiment of the present invention.

[0093] Figure 8 A schematic diagram of a product image classification device provided in an embodiment of the present invention is shown. Figure 8 As shown, the product image classification device includes:

[0094] The data crawling module 10 is used to determine the classification comparison table between the product image classification system data and the e-commerce website classification system, and crawl the product image data of the corresponding category of the e-commerce website according to the classification data in the classification comparison table;

[0095] The set partitioning module 20 is used to divide the set of product image data into a training set and a validation set according to a ratio;

[0096] The model training module 30 is used to train a deep convolutional neural network classification model by using the product image data in the training set as training sample data.

[0097] The model evaluation module 40 is used to evaluate the model performance of the deep convolutional neural network classification model based on the product image data in the validation set.

[0098] The model selection module 50 is used to select the deep convolutional neural network classification model with the best model performance for the classification prediction of the product image, thereby obtaining the product image classification system data.

[0099] In an embodiment of the present invention, optionally, the model evaluation module 40 includes:

[0100] Verification submodule 41 is used to use the product image data in the verification set as verification sample data;

[0101] The parameter tuning submodule 42 is used to adjust the hyperparameters learning_rate and batch_size in the InceptionV3 deep convolutional neural network classification model.

[0102] Training submodule 43 is used to retrain the deep convolutional neural network classification model based on the training sample data;

[0103] Evaluation submodule 44 is used to evaluate the model performance of the deep convolutional neural network classification model on the validation set.

[0104] Optionally, in this embodiment of the invention, a deduplication module 60 is further included, the deduplication module comprising:

[0105] The vector determination submodule is used to acquire multiple historical product images and determine the feature vectors of the historical product images.

[0106] The image deduplication submodule is used to calculate the similarity between the feature vectors of the historical product images, and to deduplicate the product images whose similarity is greater than a preset threshold, so as to obtain the product images to be stored that do not have duplicate product images.

[0107] The vector update submodule is used to update the vector database in a manner that stores the product images to be stored, so as to obtain a vector database that does not contain duplicate product images. The vector database is used to store the feature vectors of at least some of the product images to be stored.

[0108] In an embodiment of the present invention, optionally, the vector determination submodule includes:

[0109] The extraction unit is used to extract feature vectors from the historical product images, serialize the feature vectors of the historical product images, and store the serialized feature vectors of the historical product images into an image data table. The image data table can save the serialized feature vectors of the historical product images offline.

[0110] The restoration unit is used to extract the serialized feature vectors of the historical product images from the image data table and perform deserialization processing to restore the feature vectors of the historical product images when it is necessary to calculate the similarity between the feature vectors of the historical product images.

[0111] The product image classification device of this invention can automatically classify product images of e-commerce advertisements using existing e-commerce website platform advertising image material data and convolutional neural network technology without any manually labeled samples, saving a lot of labor costs.

[0112] The product image classification device of this invention can generate Benchark data for the e-commerce industry after classifying product images in e-commerce advertisements, thereby improving the coverage and accuracy of e-commerce advertisement classification.

[0113] In addition, embodiments of the present invention also provide an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor. The transceiver, the memory, and the processor are connected via the bus. When the computer program is executed by the processor, it implements the various processes of the above-described product image classification method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0114] For details, see Figure 9 As shown, this embodiment of the invention also provides an electronic device, which includes a bus 91, a processor 92, a transceiver 93, a bus interface 94, a memory 95, and a user interface 96.

[0115] In this embodiment of the invention, the electronic device further includes: a computer program stored in a memory 95 and executable on a processor 92, wherein the computer program, when executed by the processor 92, performs the following steps:

[0116] A classification comparison table is established between the product image classification system data and the e-commerce website classification system. Based on the classification data in the classification comparison table, product image data of the corresponding categories on the e-commerce website is crawled.

[0117] The product image data set is divided into a training set and a validation set according to the proportions.

[0118] The product image data in the training set is used as training sample data to train a deep convolutional neural network classification model.

[0119] The performance of the deep convolutional neural network classification model is evaluated based on the product image data in the validation set.

[0120] The deep convolutional neural network classification model with the best performance is selected for classification prediction of the product images to obtain the product image classification system data.

[0121] Optionally, when the computer program is executed by the processor 92, it may also perform the following steps:

[0122] The deep convolutional neural network classification model is the InceptionV3 deep convolutional neural network classification model. The step of evaluating the model performance of the deep convolutional neural network classification model based on the product image data in the validation set includes:

[0123] The product image data in the verification set is used as the verification sample data;

[0124] Adjust the hyperparameters learning_rate and batch_size in the InceptionV3 deep convolutional neural network classification model;

[0125] Retrain the deep convolutional neural network classification model based on the training sample data;

[0126] The performance of the deep convolutional neural network classification model is evaluated on the validation set.

[0127] Optionally, when the computer program is executed by the processor 92, it may also perform the following steps:

[0128] It also includes a deduplication step for product images, including:

[0129] Acquire multiple historical product images and determine the feature vectors of the historical product images;

[0130] Calculate the similarity between the feature vectors of the historical product images, and remove duplicate product images with similarity greater than a preset threshold to obtain product images to be stored that do not contain duplicate product images;

[0131] The vector database is updated by storing the product images to be stored, resulting in a vector database that does not contain duplicate product images. The vector database is used to store feature vectors of at least some of the product images to be stored.

[0132] Optionally, when the computer program is executed by the processor 92, it may also perform the following steps:

[0133] Determining the feature vector of the historical product image includes:

[0134] Feature vectors are extracted from the historical product images, and the feature vectors of the historical product images are serialized. The serialized feature vectors of the historical product images are stored in an image data table, which can save the serialized feature vectors of the historical product images offline.

[0135] When it is necessary to calculate the similarity between the feature vectors of the historical product images, the serialized feature vectors of the historical product images are extracted from the image data table and deserialized to restore the feature vectors of the historical product images.

[0136] Transceiver 93 is used to receive and send data under the control of processor 92.

[0137] exist Figure 9In the bus architecture (represented by bus 91), bus 91 may include any number of interconnected buses and bridges, and bus 91 connects various circuits including one or more processors represented by processor 92 and memory represented by memory 95.

[0138] Bus 91 represents one or more of several types of bus architectures, including memory buses and memory controllers, peripheral buses, Accelerated Graphics Port (AGP), processors, or local buses using any bus architecture from various bus architectures. As an example and not a limitation, such architectures include: Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) buses, and Peripheral Component Interconnect (PCI) buses.

[0139] Processor 92 can be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The processors mentioned above include: general-purpose processors, central processing units (CPUs), network processors (NPs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), programmable logic arrays (PLAs), microcontroller units (MCUs) or other programmable logic devices, discrete gates, transistor logic devices, and discrete hardware components. They can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. For example, the processor can be a single-core processor or a multi-core processor, and the processor can be integrated on a single chip or located on multiple different chips.

[0140] Processor 92 can be a microprocessor or any conventional processor. The method steps disclosed in the embodiments of the present invention can be directly executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in readable storage media known in the art, such as Random Access Memory (RAM), Flash Memory, Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), registers, etc. The readable storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0141] Bus 91 can also connect various other circuits, such as peripheral devices, voltage regulators, or power management circuits. Bus interface 94 provides an interface between bus 91 and transceiver 93, all of which are well known in the art. Therefore, the embodiments of the present invention will not be described further.

[0142] Transceiver 93 can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. For example, transceiver 93 receives external data from other devices, and transceiver 93 is used to send data processed by processor 92 to other devices. Depending on the nature of the computer system, a user interface 96 may also be provided, such as a touchscreen, physical keyboard, monitor, mouse, speaker, microphone, trackball, joystick, or stylus.

[0143] It should be understood that, in embodiments of the present invention, memory 95 may further include memory remotely configured relative to processor 92, and such remotely configured memory can be connected to a server via a network. One or more portions of the aforementioned network may be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless local area network (WLAN), wide area network (WAN), wireless wide area network (WWAN), metropolitan area network (MAN), Internet, public switched telephone network (PSTN), ordinary old-style telephone service (POTS), cellular telephone network, wireless network, Wi-Fi network, and combinations of two or more of the aforementioned networks. For example, cellular telephone networks and wireless networks can be Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), WiMAX, General Packet Radio Service (GPRS), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), LTE Frequency Division Duplex (FDD), LTE Time Division Duplex (TDD), Advanced Long Term Evolution (LTE-A), Universal Mobile Telecommunications System (UMTS), Enhanced Mobile Broadband (eMBB), Massive Machine Type Communication (mMTC), Ultra Reliable Low Latency Communications (uRLLC), etc.

[0144] It should be understood that the memory 95 in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory includes: read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0145] Volatile memory includes random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 75 of the electronic device described in this embodiment includes, but is not limited to, the above and any other suitable types of memory.

[0146] In this embodiment of the invention, the memory 95 stores the following elements of the operating system 951 and the application program 952: executable modules, data structures, or subsets thereof, or extended sets thereof.

[0147] Specifically, the operating system 951 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 952 includes various applications, such as a media player and a browser, used to implement various application functions. Programs implementing the methods of this embodiment of the invention can be included in the application program 952. The application program 952 includes applets, objects, components, logic, data structures, and other computer system executable instructions that perform specific tasks or implement specific abstract data types.

[0148] Furthermore, this embodiment of the invention also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the various processes of the above-described product image classification method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0149] Specifically, when a computer program is executed by a processor, it can perform the following steps:

[0150] A classification comparison table is established between the product image classification system data and the e-commerce website classification system. Based on the classification data in the classification comparison table, product image data of the corresponding categories on the e-commerce website is crawled.

[0151] The product image data set is divided into a training set and a validation set according to the proportions.

[0152] The product image data in the training set is used as training sample data to train a deep convolutional neural network classification model.

[0153] The performance of the deep convolutional neural network classification model is evaluated based on the product image data in the validation set.

[0154] The deep convolutional neural network classification model with the best performance is selected for classification prediction of the product images to obtain the product image classification system data.

[0155] Optionally, when a computer program is executed by a processor, it may also perform the following steps:

[0156] The deep convolutional neural network classification model is the InceptionV3 deep convolutional neural network classification model. The step of evaluating the model performance of the deep convolutional neural network classification model based on the product image data in the validation set includes:

[0157] The product image data in the verification set is used as the verification sample data;

[0158] Adjust the hyperparameters learning_rate and batch_size in the InceptionV3 deep convolutional neural network classification model;

[0159] Retrain the deep convolutional neural network classification model based on the training sample data;

[0160] The performance of the deep convolutional neural network classification model is evaluated on the validation set.

[0161] Optionally, when a computer program is executed by a processor, it may also perform the following steps:

[0162] It also includes a deduplication step for product images, including:

[0163] Acquire multiple historical product images and determine the feature vectors of the historical product images;

[0164] Calculate the similarity between the feature vectors of the historical product images, and remove duplicate product images with similarity greater than a preset threshold to obtain product images to be stored that do not contain duplicate product images;

[0165] The vector database is updated by storing the product images to be stored, resulting in a vector database that does not contain duplicate product images. The vector database is used to store feature vectors of at least some of the product images to be stored.

[0166] Optionally, when a computer program is executed by a processor, it may also perform the following steps:

[0167] Determining the feature vector of the historical product image includes:

[0168] Feature vectors are extracted from the historical product images, and the feature vectors of the historical product images are serialized. The serialized feature vectors of the historical product images are stored in an image data table, which can save the serialized feature vectors of the historical product images offline.

[0169] When it is necessary to calculate the similarity between the feature vectors of the historical product images, the serialized feature vectors of the historical product images are extracted from the image data table and deserialized to restore the feature vectors of the historical product images.

[0170] Computer-readable storage media include: permanent and non-permanent, removable and non-removable media, which are tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media include: electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, and any suitable combination thereof. Computer-readable storage media include: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape storage, magnetic disk storage or other magnetic storage devices, memory sticks, mechanical encoding devices (e.g., punched cards or raised structures in grooves on which instructions are recorded), or any other non-transfer medium that can be used to store information accessible by a computing device. As defined in the embodiments of the present invention, computer-readable storage media do not include temporary signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0171] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0172] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. When implemented in software, it can be implemented wholly or partially in the form of a computer program product. The computer program product includes one or more computer program instructions. The computer program instructions include: assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and procedural programming languages ​​such as C or similar programming languages.

[0173] When the computer program instructions are loaded and executed on a computer, all or part of the process or function described in the embodiments of the present invention is generated. The computer may be a computer, a dedicated computer, a computer network, or other editable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, twisted pair, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, magnetic disk, magnetic tape), an optical medium (e.g., optical disc), or a semiconductor medium (e.g., solid state drive (SSD)). Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments of the present invention.

[0174] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing embodiments of the present invention, and will not be repeated here.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed apparatus, electronic devices, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, or it may be an electrical, mechanical, or other form of connection.

[0176] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to solve the problems addressed by the embodiments of the present invention, depending on actual needs.

[0177] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0178] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (including: a personal computer, a server, a data center, or other network device) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media listed above that can store program code.

[0179] The above description is merely a specific implementation of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of the present invention should be included within the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention should be determined by the protection scope of the claims.

Claims

1. A method for classifying product images, characterized in that, include: A classification comparison table is established between the product image classification system data and the e-commerce website classification system. The product image classification system data includes a hierarchical structure of first-level, second-level, and third-level categories. The classification comparison table is used to map each level of category in the product image classification system data to the corresponding category link in the e-commerce website classification system. Based on the category links in the category lookup table, crawl the first product image data of the corresponding category of the e-commerce website. The first product image data contains the corresponding category label. For categories in the classification lookup table that do not correspond to the classification system of the e-commerce website, determine the keywords corresponding to the category, and use the keywords as search terms to crawl the second product image data of the search results of the e-commerce website. The second product image data contains the corresponding category label. The first product image data and the second product image data are merged to obtain a set of product image data; The product image data set is divided into a training set and a validation set according to the proportions. The product image data in the training set is used as training sample data to train a deep convolutional neural network classification model. The deep convolutional neural network classification model is the InceptionV3 deep convolutional neural network classification model. The InceptionV3 deep convolutional neural network classification model removes the original last classification layer and redesigns the classification layer according to the sub-classifications corresponding to each level in the product image classification system data to achieve hierarchical classification of product images. The performance of the deep convolutional neural network classification model is evaluated based on the product image data in the validation set. The deep convolutional neural network classification model with the best performance is selected for classification prediction of the product images to obtain the product image classification system data.

2. The method according to claim 1, characterized in that, The step of evaluating the performance of the deep convolutional neural network classification model based on the product image data in the validation set includes: The product image data in the verification set is used as the verification sample data; Adjust the hyperparameters learning_rate and batch_size in the InceptionV3 deep convolutional neural network classification model; Retrain the deep convolutional neural network classification model based on the training sample data; The performance of the deep convolutional neural network classification model is evaluated on the validation set.

3. The method according to claim 1, characterized in that, It also includes a deduplication step for product images, including: Acquire multiple historical product images and determine the feature vectors of the historical product images; Calculate the similarity between the feature vectors of the historical product images, and remove duplicate product images with similarity greater than a preset threshold to obtain product images to be stored that do not contain duplicate product images; The vector database is updated by storing the product images to be stored, resulting in a vector database that does not contain duplicate product images. The vector database is used to store feature vectors of at least some of the product images to be stored.

4. The method according to claim 3, characterized in that, Determining the feature vector of the historical product image includes: Feature vectors are extracted from the historical product images, and the feature vectors of the historical product images are serialized. The serialized feature vectors of the historical product images are stored in an image data table, which can save the serialized feature vectors of the historical product images offline. When it is necessary to calculate the similarity between the feature vectors of the historical product images, the serialized feature vectors of the historical product images are extracted from the image data table and deserialized to restore the feature vectors of the historical product images.

5. A product image sorting device, characterized in that, include: The data crawling module is used to determine the classification comparison table between the product image classification system data and the e-commerce website classification system. The product image classification system data contains a hierarchical structure of first-level, second-level, and third-level classifications. The classification comparison table is used to map each level of category in the product image classification system data to the corresponding category link in the e-commerce website classification system. This tool is used to crawl first product image data of corresponding categories on the e-commerce website based on the category links in the category lookup table, wherein the first product image data contains corresponding category labels; for categories in the category lookup table that do not correspond to the classification system of the e-commerce website, this tool is used to determine the keywords corresponding to the category, and use the keywords as search terms to crawl second product image data of the search results on the e-commerce website, wherein the second product image data contains corresponding category labels; and to merge the first product image data and the second product image data to obtain a set of product image data. The set partitioning module is used to divide the set of product image data into a training set and a validation set according to a ratio. The model training module is used to train a deep convolutional neural network classification model by using the product image data in the training set as training sample data. The deep convolutional neural network classification model is the InceptionV3 deep convolutional neural network classification model. The InceptionV3 deep convolutional neural network classification model removes the original last classification layer and redesigns the classification layer according to the subclasses corresponding to each level in the product image classification system data to achieve hierarchical classification of product images. The model evaluation module is used to evaluate the model performance of the deep convolutional neural network classification model based on the product image data in the validation set. The model selection module is used to select the deep convolutional neural network classification model with the best model performance for the classification prediction of the product image, thereby obtaining the product image classification system data.

6. The apparatus according to claim 5, characterized in that, The model evaluation module includes: The verification submodule is used to use the product image data in the verification set as verification sample data; The parameter tuning submodule is used to adjust the hyperparameters learning_rate and batch_size in the InceptionV3 deep convolutional neural network classification model; The training submodule is used to retrain the deep convolutional neural network classification model based on the training sample data. An evaluation submodule is used to evaluate the performance of the deep convolutional neural network classification model on the validation set.

7. The apparatus according to claim 5, characterized in that, It also includes a deduplication module, which includes: The vector determination submodule is used to acquire multiple historical product images and determine the feature vectors of the historical product images. The image deduplication submodule is used to calculate the similarity between the feature vectors of the historical product images, and to deduplicate the product images whose similarity is greater than a preset threshold, so as to obtain the product images to be stored that do not have duplicate product images. The vector update submodule is used to update the vector database in a manner that stores the product images to be stored, so as to obtain a vector database that does not contain duplicate product images. The vector database is used to store the feature vectors of at least some of the product images to be stored.

8. The apparatus according to claim 7, characterized in that, The vector determination submodule includes: The extraction unit is used to extract feature vectors from the historical product images, serialize the feature vectors of the historical product images, and store the serialized feature vectors of the historical product images into an image data table. The image data table can save the serialized feature vectors of the historical product images offline. The restoration unit is used to extract the serialized feature vectors of the historical product images from the image data table and perform deserialization processing to restore the feature vectors of the historical product images when it is necessary to calculate the similarity between the feature vectors of the historical product images.

9. An electronic device comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, characterized in that, When the computer program is executed by the processor, it implements the steps in the method for classifying product images as described in any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps in the method for classifying product images as described in any one of claims 1 to 4.

Images