A training image classification model, image search method and related device
By integrating feature extraction and density clustering to refine image classification models, the method ensures both content category and style consistency, improving image search accuracy in scenarios with style consistency requirements.
Patent Information
- Application Number
- CN202110649832.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-10
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-06-10
AI Technical Summary
Existing image classification models primarily focus on content categories of images, ignoring the style, leading to inconsistent style recall in image search scenarios with style consistency requirements.
A method that includes feature extraction, density clustering, and fine-tuning to create a target image classification model that considers both content category and style consistency, using historical user input text to mark search images and perform density clustering on similar image sets to refine the model.
The method enhances image search accuracy by ensuring both content category and style consistency, making it suitable for scenarios requiring style consistency.
Smart Images

Figure CN113392898B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method for training an image classification model, an image search, and related devices. Background Art
[0002] With the rapid development of machine learning, the application of machine learning is becoming more and more extensive. For example, image search refers to using the feature extraction layer of the image classification model obtained by machine learning to extract features from the image to be searched and the library image to obtain image features, matching based on image features, and recalling similar images of the image to be searched from the library image.
[0003] After research, the inventors found that the image classification model obtained by the above machine learning is actually obtained by training a classification network based on sample images and marked content category labels; the image classification model mainly focuses on the content category of the image and ignores the style of the image. Applying this image classification model when searching for images by image may cause the recalled similar images to be inconsistent with the style of the image to be searched, which makes it difficult to apply to image search scenarios with style consistency requirements. Summary of the invention
[0004] In view of this, the present application provides a method for training an image classification model, an image search, and related devices, so that the trained image classification model not only focuses on the content category of the image, but also focuses on the style of the image. Based on this, when searching for images by image, not only the content category of the images is the same, but also the style of the images is consistent. This is suitable for image search scenarios with style consistency requirements.
[0005] In a first aspect, an embodiment of the present application provides a method for training an image classification model, the method comprising:
[0006] Using the feature extraction layer of a preset image classification model, feature extraction and matching are performed on the search image and the gallery image to obtain similar images of the search image to form a similar image set; the search image is obtained by performing image search through historical user input text, and the search image is marked with a content category label;
[0007] Based on the image features of the similar images, performing density clustering cleaning on the similar image set to obtain a target similar image set;
[0008] The pre-trained classification model is fine-tuned using similar images in the target similar image set and content category labels marked on each similar image to obtain a target image classification model.
[0009] Optionally, performing density clustering cleaning on the similar image set based on the image features of the similar images to obtain a target similar image set includes:
[0010] Based on the image features of the similar images, the distance threshold and the quantity threshold of the density clustering cluster, density clustering is performed on the similar images in the similar image set, and similar images that do not belong to the density clustering cluster are determined to be abnormal images; the distance threshold refers to the upper limit of the image feature distance between the similar images constituting the density clustering cluster, and the quantity threshold refers to the lower limit of the number of similar images constituting the density clustering cluster;
[0011] The abnormal images in the similar image set are cleaned to obtain the target similar image set.
[0012] Optionally, the using a feature extraction layer of a preset image classification model to perform feature extraction and matching on the search image and the gallery image to obtain similar images of the search image to form a similar image set includes:
[0013] Using the feature extraction layer of the preset image classification model, feature extraction is performed on the search image and the gallery image to obtain image features of the search image and image features of the gallery image;
[0014] Based on the image features of the search image and the image features of the gallery images, obtaining the image feature distance between each search image and each gallery image;
[0015] Based on the image feature distance between each search image and each gallery image, similar images of each search image are screened out from the gallery images to form a similar image set corresponding to each search image.
[0016] Optionally, the pre-trained classification model is obtained by training a classification network based on multi-category labeled sample data; wherein the multi-category labeled sample data includes multi-category labeled sample images and multiple content category labels marked with each sample image, and the multi-category labeled sample images refer to sample images marked with multiple content category labels.
[0017] Optionally, the method further includes:
[0018] Acquire single-category label sample data; the single-category label sample data includes single-category label sample images and content category labels marked on each sample image, wherein the single-category label sample images refer to sample images marked with only a single content category label;
[0019] Correspondingly, the target image classification model is obtained by fine-tuning the pre-trained classification model using the similar images in the target similar image set and the content category labels of each similar image mark, which is:
[0020] The target image classification model is obtained by performing multi-task fine-tuning training on the pre-trained classification model using similar images in the target similar image set, content category labels marked on each similar image, and the single-category label sample data.
[0021] In a second aspect, an embodiment of the present application provides an image search method, characterized in that, using the target image classification model described in any one of the first aspects above, the method includes:
[0022] Using the feature extraction layer of the target image classification model, feature extraction is performed on the target search image and the gallery image to obtain image features of the target search image and image features of the gallery image;
[0023] The image features of the target search image and the image features of the gallery image are matched to determine a target similar image of the target search image, wherein the target similar image has the same style as the target search image.
[0024] In a third aspect, an embodiment of the present application provides a device for training an image classification model, the device comprising:
[0025] A feature extraction and matching unit, used to perform feature extraction and matching on a search image and a gallery image using a feature extraction layer of a preset image classification model, and obtain similar images of the search image to form a similar image set; the search image is obtained by performing an image search through a historical user input text, and the search image is marked with a content category label;
[0026] A density cluster cleaning unit, configured to perform density cluster cleaning on the similar image set based on image features of the similar images to obtain a target similar image set;
[0027] The fine-tuning training unit is used to use the similar images in the target similar image set and the content category labels marked on each similar image to fine-tune the pre-trained classification model to obtain a target image classification model.
[0028] Optionally, the density cluster cleaning unit is used to:
[0029] Based on the image features of the similar images, the distance threshold and the quantity threshold of the density clustering cluster, density clustering is performed on the similar images in the similar image set, and similar images that do not belong to the density clustering cluster are determined to be abnormal images; the distance threshold refers to the upper limit of the image feature distance between the similar images constituting the density clustering cluster, and the quantity threshold refers to the lower limit of the number of similar images constituting the density clustering cluster;
[0030] The abnormal images in the similar image set are cleaned to obtain the target similar image set.
[0031] Optionally, the feature extraction and matching unit is used to:
[0032] Using the feature extraction layer of the preset image classification model, feature extraction is performed on the search image and the gallery image to obtain image features of the search image and image features of the gallery image;
[0033] Based on the image features of the search image and the image features of the gallery images, obtaining the image feature distance between each search image and each gallery image;
[0034] Based on the image feature distance between each search image and each gallery image, similar images of each search image are screened out from the gallery images to form a similar image set corresponding to each search image.
[0035] Optionally, the pre-trained classification model is obtained by training a classification network based on multi-category labeled sample data; wherein the multi-category labeled sample data includes multi-category labeled sample images and multiple content category labels marked with each sample image, and the multi-category labeled sample images refer to sample images marked with multiple content category labels.
[0036] Optionally, the device further comprises:
[0037] An acquisition unit, configured to acquire single-category label sample data; the single-category label sample data includes single-category label sample images and a content category label marked on each sample image, wherein the single-category label sample image refers to a sample image marked with only a single content category label;
[0038] Correspondingly, the fine-tuning training unit is used to:
[0039] The target image classification model is obtained by performing multi-task fine-tuning training on the pre-trained classification model using similar images in the target similar image set, content category labels marked on each similar image, and the single-category label sample data.
[0040] In a fourth aspect, an embodiment of the present application provides an image search device, using the target image classification model described in any one of the first aspects, the device comprising:
[0041] A feature extraction unit, used to extract features from a target search image and a gallery image using a feature extraction layer of the target image classification model, to obtain image features of the target search image and image features of the gallery image;
[0042] The matching search unit is used to match the image features of the target search image with the image features of the gallery image to determine a target similar image of the target search image, wherein the target similar image has the same style as the target search image.
[0043] In a fifth aspect, an embodiment of the present application provides a device for training an image classification model, the device comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations:
[0044] Using the feature extraction layer of a preset image classification model, feature extraction and matching are performed on the search image and the gallery image to obtain similar images of the search image to form a similar image set; the search image is obtained by performing image search through historical user input text, and the search image is marked with a content category label;
[0045] Based on the image features of the similar images, performing density clustering cleaning on the similar image set to obtain a target similar image set;
[0046] Using the similar images in the target similar image set and the content category labels of each similar image, the pre-trained classification model is fine-tuned to obtain the target image classification model.
[0047] In a sixth aspect, an embodiment of the present application provides a device for image search, the device comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations:
[0048] Using the feature extraction layer of the target image classification model, feature extraction is performed on the target search image and the gallery image to obtain image features of the target search image and image features of the gallery image;
[0049] The image features of the target search image and the image features of the gallery image are matched to determine a target similar image of the target search image, wherein the target similar image has the same style as the target search image.
[0050] In the seventh aspect, an embodiment of the present application provides a machine-readable medium having instructions stored thereon, which, when executed by one or more processors, enables the device to execute the method for training an image classification model described in any one of the first aspects above; or, enables the device to execute the method for image search described in the second aspect above.
[0051] Compared with the prior art, this application has at least the following advantages:
[0052] Adopting the technical solution of the embodiment of the present application, the image search is performed through the historical user input text to obtain the search image, and the search image is marked with a content category label; through the feature extraction layer of the preset image classification model, the image features of the search image and the image features of the gallery image are extracted and matched to obtain similar images of the search image to form a similar image set; through the image features of the similar images, density clustering is performed to clean the similar image set to obtain the target similar image set; with the similar images in the target similar image set and the content category labels marked by each similar image, the pre-trained classification model is fine-tuned and trained to obtain the target image classification model. It can be seen that after obtaining a similar image set representing the same content category through feature extraction and matching, the style consistency of similar images in the similar image set is improved through density clustering and cleaning to obtain the target similar image set; the target image classification model obtained by fine-tuning the training based on the target similar image set not only pays attention to the content category of the image, but also pays attention to the style of the image. Adopting the target image classification model to search for images with images, not only the content category of the images is the same, but also the style of the images is consistent, which is suitable for image search scenarios with style consistency requirements.
[0053] In addition, the image features of the target search image and the image features of the gallery image are extracted through the feature extraction layer of the target image classification model; the image features of the target search image and the image features of the gallery image are matched to search for target similar images with the same style as the target search image. Based on the fact that the target image classification model not only pays attention to the content category of the image, but also pays attention to the style of the image, the target image classification model is used to search for images by image, and not only the content category of the images is the same, but also the style of the images is consistent, which greatly improves the image search accuracy in the search by image scenario with style consistency requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0055] Figure 1 A schematic diagram of a system framework involved in an application scenario in an embodiment of the present application;
[0056] Figure 2 A flowchart of a method for training an image classification model provided in an embodiment of the present application;
[0057] Figure 3A flowchart of an image search method provided in an embodiment of the present application;
[0058] Figure 4 A schematic diagram of the structure of a device for training an image classification model provided in an embodiment of the present application;
[0059] Figure 5 A schematic diagram of the structure of an image search device provided in an embodiment of the present application;
[0060] Figure 6 A schematic diagram of the structure of a device for training an image classification model or image search provided in an embodiment of the present application;
[0061] Figure 7 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0062] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0063] At present, the technical implementation of image search is generally as follows: a classification network is trained based on sample images and marked content category labels to obtain an image classification model, the image classification model is used to extract features of the image to be searched and the gallery image to obtain image features, matching is performed based on the image features, and similar images of the image to be searched are recalled from the gallery image. After research, the inventor found that the image classification model mainly focuses on the content category of the image and ignores the style of the image. Applying the image classification model when searching by image may result in the recalled similar images being inconsistent in style with the image to be searched, and it is difficult to apply to image search scenarios with style consistency requirements.
[0064] In order to solve this problem, in the embodiment of the present application, the image search is performed through the historical user input text to obtain the search image, and the search image is marked with a content category label; through the feature extraction layer of the preset image classification model, the image features of the search image and the image features of the gallery image are extracted and matched to obtain similar images of the search image to form a similar image set; through the image features of the similar images, density clustering is performed to clean the similar image set to obtain the target similar image set; with the similar images in the target similar image set and the content category labels marked by each similar image, the pre-trained classification model is fine-tuned and trained to obtain the target image classification model. It can be seen that after obtaining a similar image set with the same characterization content category through feature extraction and matching, the style consistency of similar images in the similar image set is improved through density clustering and cleaning to obtain the target similar image set; the target image classification model obtained by fine-tuning the training based on the target similar image set not only pays attention to the content category of the image, but also pays attention to the style of the image. The target image classification model is used to search for images with images, not only the content category of the images is the same, but also the style of the images is consistent, which is suitable for image search scenarios with style consistency requirements, and improves the accuracy of image search.
[0065] For example, one of the scenarios of the embodiments of the present application may be applied to Figure 1 In the scenario shown, the scenario includes a terminal device 101 and a server 102. The server 102 obtains a target image classification model by using the implementation method of the training image classification model provided in the embodiment of the present application based on the historical search data, and the historical search data is the text input by the historical user and the search image obtained by performing an image search based on the input text. After the target image classification model is obtained through training, when the user inputs the target search image through the terminal device 101, the server 102 uses the target image classification model and the implementation method of the image search provided in the embodiment of the present application to realize image search.
[0066] It is understandable that in the above application scenario, although the action description of the implementation mode of the present application is executed by the server 102; however, the present application is not limited in terms of the execution subject, as long as the actions disclosed in the implementation mode of the present application are executed.
[0067] It can be understood that the above scenario is only an example scenario provided by the embodiment of the present application, and the embodiment of the present application is not limited to this scenario.
[0068] The specific implementation of the training image classification model, image search method and related devices in the embodiments of the present application are described in detail below with reference to the accompanying drawings.
[0069] Exemplary Methods
[0070] See also Figure 2, shows a flow chart of a method for training an image classification model in an embodiment of the present application. In this embodiment, the method may include the following steps:
[0071] Step 201: Using the feature extraction layer of a preset image classification model, feature extraction and matching are performed on the search image and the gallery image to obtain similar images of the search image to form a similar image set; the search image is obtained by performing an image search through historical user input text, and the search image is marked with a content category label.
[0072] In the embodiment of the present application, the training image classification model first needs to construct a training sample set. The construction of the training sample can be obtained based on the historical search data, for example, it can include: performing image search through historical user input text to obtain search images, and multiple search images can be searched based on each historical user input text; based on each search image, the gallery images are screened to obtain similar images of each search image, forming a similar image set corresponding to each search image; the similar image set corresponding to multiple search images can be used as the initial training sample set.
[0073] For each search image, an open source image classification model is pre-selected, and the image features of the search image and the image features of the gallery image are extracted through the feature extraction layer of the open source image classification model; by matching the image features of the search image and the image features of the gallery image, similar images of each search image are screened out from the gallery images to form a similar image set corresponding to each search image. Each search image is marked with a content category label, and the similar image set obtained based on each search image can be used as an initial training sample subset of the content category to which the search image belongs. The content category to which each search image belongs can be determined by the content category label marked with the search image.
[0074] When step 201 is specifically implemented, for each search image, the search image and the gallery image are respectively input into a preset image classification model, that is, an open source image classification model, and the feature extraction layer based on the model outputs the image features of the search image and the image features of the gallery image; based on the image features of the search image and the image features of the gallery image, the image feature distance between each search image and each gallery image can be calculated, and the smaller the image feature distance, the more similar the gallery image is to the corresponding search image, and the larger the image feature distance, the less similar the gallery image is to the corresponding search image; based on the calculated image feature distance, similar images of the search image are screened, and the search image and the similar images of the search image form a similar image set.
[0075] Therefore, in an optional implementation manner of the present application, the step 201 may include, for example, the following steps A to C:
[0076] Step A: using the feature extraction layer of the preset image classification model, extracting features from the search image and the gallery image to obtain image features of the search image and image features of the gallery image.
[0077] Step B: Based on the image features of the search image and the image features of the gallery images, the image feature distance between each search image and each gallery image is obtained.
[0078] Step C: Based on the image feature distance between the search image and each gallery image, similar images of each search image are screened out from the gallery images to form a similar image set corresponding to each search image.
[0079] Among them, when step C is specifically implemented, at least the following two methods can be used:
[0080] In a specific implementation of the first optional step C, the gallery images are arranged from small to large according to the image feature distance between the gallery images and the search image, and the first M gallery images are selected as similar images of the search image to form a similar image set, where M is a positive integer.
[0081] As an example, suppose that 5 search images are obtained by performing image search on 1 text. For each search image, after obtaining the image feature distance between each gallery image and the search image, the gallery images are arranged from small to large according to the image feature distance between each gallery image and the search image, and the first 100 gallery images are selected as similar images of the search image to form a similar image set. That is, a similar image set including 100 similar images can be obtained based on each search image, and 5 similar image sets including 100 similar images can be obtained for each of the 5 search images.
[0082] In a specific implementation of the second optional step C, according to the image feature distance between the gallery image and the search image, gallery images whose image feature distance is less than or equal to a preset distance are screened to form a similar image set as similar images to the search image.
[0083] As an example, suppose that 5 search images are obtained by performing image search on 1 text, and for each search image, gallery images whose image feature distance is less than or equal to the preset distance d1 are selected to form a similar image set as similar images of the search image. The number of similar images in the similar image set obtained by different search images may be different.
[0084] Based on the above description, multiple search images are obtained by performing image search on each text. For each search image, similar images are obtained by screening the gallery images to form a similar image set as the initial training sample subset of the content category to which the search image belongs. The obtained initial training sample set can include initial training sample subsets of multiple content categories. For example, there are 10,000 texts, and 5 search images are obtained by performing image search on each text. Based on each search image, a similar image set is obtained, and the initial training sample can include approximately 50,000 initial training sample subsets of the content category to which the search image belongs. This method greatly increases the content categories to solve the problem that there are many actual content categories in the prior art, while the image classification model obtained by machine learning uses fewer content categories during training.
[0085] Step 202: Based on the image features of the similar images, density clustering and cleaning are performed on the similar image set to obtain a target similar image set.
[0086] In an embodiment of the present application, in order to make the trained image classification model pay attention to the style of the image while paying attention to the content category of the image, after obtaining the similar image set in step 201, on the basis that the content categories of the similar images in the similar image set are the same, it is also necessary to consider whether the styles of the similar images in the similar image set are consistent, and most of the similar images with consistent styles in the similar image set are retained to obtain the target similar image set, and multiple target similar image sets can be used as the final training sample sets.
[0087] For each similar image set, it is necessary to perform density clustering on similar images according to the image features of similar images in the similar image set, and based on this, clean the similar images in the similar image set to obtain a target similar image set. The target similar image set obtained based on each similar image set is used as a training sample subset of a content category.
[0088] When step 202 is specifically implemented, for each similar image set, based on the known image features of the similar images, it is also necessary to determine the distance threshold and quantity threshold of the density clustering cluster. The distance threshold of the density clustering cluster refers to the upper limit of the image feature distance between the similar images constituting the density clustering cluster; the quantity threshold of the density clustering cluster refers to the lower limit of the number of similar images constituting the density clustering cluster.
[0089] Density clustering is performed on similar images in the similar image set to obtain density clustering clusters. The styles of similar images in the density clustering clusters are consistent. Similar images in the similar image set other than the density clustering clusters are inconsistent with the styles of similar images in the density clustering clusters and can be regarded as abnormal images. In order to retain most similar images with consistent styles in the similar image set to obtain the target similar image set, it is necessary to clean abnormal images from the similar image set to obtain the target similar image set.
[0090] Therefore, in an optional implementation manner of the present application, the step 202 may include, for example, the following steps D-E:
[0091] Step D: Based on the image features of the similar images, the distance threshold and quantity threshold of the density clustering cluster, density clustering is performed on the similar images in the similar image set, and similar images that do not belong to the density clustering cluster are determined to be abnormal images; wherein the distance threshold refers to the upper limit value of the image feature distance between the similar images constituting the density clustering cluster; and the quantity threshold refers to the lower limit value of the number of similar images constituting the density clustering cluster.
[0092] Step E: Cleaning the abnormal images in the similar image set to obtain the target similar image set.
[0093] As an example, assume that the similar image set includes 100 similar images, and the distance threshold of the preset density clustering cluster is d2, and the quantity threshold is 60; based on the image features of each similar image in the known similar image set, the 100 similar images are density clustered to obtain a density clustering cluster including 92 similar images. The abnormal images are the 8 similar images in the 100 similar images except the 92 similar images included in the density clustering cluster. The 8 similar images in the similar image set are cleaned to obtain the target similar image set.
[0094] Based on the above description, based on the image features of similar images in the known similar image set, the style consistency of similar images in the similar image set is improved by density clustering cleaning, so that the styles of similar images in the target similar image set obtained by density clustering cleaning are consistent. That is, the training sample subset constructed for training the image classification model not only considers the sameness of the content category of the image, but also considers the style consistency of the image, so as to train the image classification model to solve the problem that the image classification model in the prior art mainly focuses on the content category of the image, but ignores the style of the image.
[0095] Step 203: using the similar images in the target similar image set and the content category labels marked on each similar image, fine-tune the pre-trained classification model to obtain a target image classification model.
[0096] In the embodiment of the present application, after obtaining the target similar image set in step 202, since the similar images in the target similar image set meet both the requirements of the same content category and the requirements of the same style, the similar images in the target similar image set can be used as input and the content category labels of each similar image as output, and a pre-trained classification model can be selected for fine-tuning training, and the pre-trained classification model completed by fine-tuning training is used as the target image classification model. The pre-trained classification model refers to an open source image classification model, which can be the same as or different from the preset image classification model in step 201.
[0097] During the fine-tuning training process, the pre-trained classification model uses similar images in the target similar image set and the content category labels marked on each similar image as training samples, where each similar image is used as input and the content category labels marked on each similar image are used as output. In this way, not only the content category identity of similar images in the target similar image set can be learned, but also the style consistency of similar images in the target similar image set can be learned. The target image classification model obtained by fine-tuning training not only focuses on the content category of the image, but also focuses on the style of the image.
[0098] In an embodiment of the present application, the training samples for training the image classification model constructed by steps 201-202 have the characteristics of more content categories but fewer samples in each content category, that is, the number of target similar image sets is large but the number of similar images in each target similar image set is small. In the fine-tuning training process of the pre-trained classification model, the common classification loss function is used, the pre-trained classification model converges slowly, and more iterative training is required, resulting in a slow training speed of the model. Then, a metric learning loss function can be used to make the image features of similar images in the target similar image set smaller in the image classification, and at the same time, make the image features of similar images in different similar image sets in the image classification The difference is greater; compared with the model using the common classification loss function, the pre-trained classification model of the present application can converge faster during the fine-tuning training process, thereby improving the training speed of the model. Therefore, in an optional embodiment of the present application, the network loss function for fine-tuning the pre-trained classification model includes a metric learning loss function.
[0099] In practical applications, sample images used as training samples may be labeled with only a single content category label or multiple content category labels. Sample images labeled with a single content category label are recorded as single-category label sample images, and sample images labeled with multiple content category labels are recorded as multi-category label sample images.
[0100] Common pre-trained classification models are obtained by training a classification network with multiple single-category labeled sample images and a single content category label marked on each single-category labeled sample image. It is difficult to grasp the relationship between the image composition of the sample image and the single content category label marked on it. Based on this, in an embodiment of the present application, multiple multi-category labeled sample images can be obtained, and a classification network can be trained using multiple multi-category labeled sample images and multiple content category labels marked on each multi-category labeled sample image to obtain a pre-trained classification model. Compared with existing common classification models, this pre-trained classification model can grasp the relationship between the image composition of the sample image and the multiple content category labels marked on it, laying a certain foundation for the subsequent learning of the style consistency of similar images in the target similar image set.
[0101] Therefore, in an optional embodiment of the present application, the pre-trained classification model is obtained by training a classification network based on multi-category labeled sample data; wherein the multi-category labeled sample data includes multi-category labeled sample images and multiple content category labels marked with each sample image, and the multi-category labeled sample images refer to sample images marked with multiple content category labels.
[0102] In an embodiment of the present application, in order to achieve better model generalization effect of the target image classification model obtained by fine-tuning training, a multi-task learning method can also be used to implement fine-tuning training when fine-tuning the pre-trained classification model. That is, using similar images in the target similar image set and content category labels marked with each similar image, training the pre-trained classification model is used as a task; in addition, a plurality of single-category label sample images are obtained, and the single-category label sample images refer to sample images marked with a single content category label, and the plurality of single-category label sample images and the single content category label marked with each single-category label sample image are used to train the pre-trained classification model as another task. The two tasks perform multi-task learning on the pre-trained classification model to achieve fine-tuning training, share the learned information in the multi-task learning process, and obtain a target image classification model with better model generalization effect.
[0103] Therefore, in an optional embodiment of the present application, the method may also include step F: obtaining single-category label sample data; the single-category label sample data includes single-category label sample images and content category labels marked with each sample image, and the single-category label sample images refer to sample images that are only marked with a single content category label; correspondingly, the step 203 may specifically be: utilizing similar images in the target similar image set, content category labels marked with each similar image, and the single-category label sample data, to perform multi-task fine-tuning training on the pre-trained classification model to obtain the target image classification model.
[0104] Through various implementation methods provided in this embodiment, an image search is performed through historical user input text to obtain a search image, and the search image is marked with a content category label; through the feature extraction layer of the preset image classification model, the image features of the search image and the image features of the gallery image are extracted and matched to obtain similar images of the search image to form a similar image set; through the image features of the similar images, density clustering is performed to clean the similar image set to obtain a target similar image set; with the similar images in the target similar image set and the content category labels marked by each similar image, the pre-trained classification model is fine-tuned and trained to obtain a target image classification model. It can be seen that after obtaining a similar image set with the same characterization content category through feature extraction and matching, the style consistency of similar images in the similar image set is improved through density clustering and cleaning to obtain a target similar image set; the target image classification model obtained by fine-tuning the training based on the target similar image set not only pays attention to the content category of the image, but also pays attention to the style of the image. The target image classification model is used to search for images with images, and not only the content categories of the images are the same, but also suitable for image search scenarios with style consistency requirements.
[0105] After the target image classification model that not only pays attention to the content category of the image but also pays attention to the consistency of the style of the image is obtained in the above embodiment, the target image classification model is applied when searching for images by image, which can solve the problem that the image classification model in the prior art is applied when searching for images by image, and the style of the recalled similar images may be inconsistent with the search image. That is, the input search image is used as the target search image, and the feature extraction layer of the target image classification model is used to extract features of the target search image and the gallery image to obtain image features, and matching is performed based on the image features to recall target similar images of the target search image from the gallery image. In this way, on the basis that the target image classification model not only pays attention to the content category of the image but also pays attention to the style of the image, the target similar images searched out have the same content category and consistent style as the target search image.
[0106] See also Figure 3 , shows a flow chart of an image search method in an embodiment of the present application. In this embodiment, using the target image classification model as described in the above embodiment, the method may include the following steps:
[0107] Step 301: Using the feature extraction layer of the target image classification model, feature extraction is performed on the target search image and the gallery image to obtain image features of the target search image and image features of the gallery image.
[0108] Step 302: Match the image features of the target search image with the image features of the gallery image to determine a target similar image of the target search image, wherein the target similar image has the same style as the target search image.
[0109] In particular, when step 302 is implemented, at least the following two methods can be used:
[0110] In a specific implementation of the first optional step C, the gallery images are arranged from small to large according to the image feature distance between the gallery images and the target search image, and the first N gallery images are selected as target similar images of the target search image, where N is a positive integer.
[0111] In a specific implementation of the second optional step 302, according to the image feature distance between the gallery image and the target search image, gallery images whose image feature distance is less than or equal to a preset distance are screened as target similar images of the target search image.
[0112] In summary, assuming that the target search image is "ink landscape painting A", the feature extraction layer of the image classification model in the prior art is used to extract features from "ink landscape painting A" and the library images to obtain image features, and the library images are matched and screened based on the image features. The similar images of "ink landscape painting A" are recalled as multiple "landscape pictures". Among them, the recalled "landscape pictures" may include many color landscape pictures or photos of landscapes, etc., which belong to the "landscape pictures" with the same content category as "ink landscape painting A" but inconsistent styles.
[0113] However, by using the feature extraction layer of the target image classification model provided in the embodiment of the present application, feature extraction is performed on "ink landscape painting A" and the gallery images to obtain image features, and gallery images are matched and screened based on the image features, and similar images of "ink landscape painting A" are recalled as multiple "ink landscape paintings". Among them, the recalled multiple "ink landscape paintings" have the same content category and consistent style as "ink landscape painting A".
[0114] Through various implementation methods provided in this embodiment, the image features of the target search image and the image features of the gallery image are extracted through the feature extraction layer of the target image classification model; the image features of the target search image and the image features of the gallery image are matched to search for target similar images with the same style as the target search image. On the basis that the target image classification model not only pays attention to the content category of the image, but also pays attention to the style of the image, the target image classification model is used to search for images by image, and not only the content category of the images is the same, but also the style of the images is consistent, which greatly improves the image search accuracy in the image search by image scenario with style consistency requirements.
[0115] Exemplary Devices
[0116] See also Figure 4 , shows a schematic diagram of the structure of a device for training an image classification model in an embodiment of the present application. In this embodiment, the device may specifically include:
[0117] The feature extraction and matching unit 401 is used to perform feature extraction and matching on the search image and the gallery image by using the feature extraction layer of the preset image classification model, and obtain similar images of the search image to form a similar image set; the search image is obtained by performing image search through historical user input text, and the search image is marked with a content category label;
[0118] A density cluster cleaning unit 402 is used to perform density cluster cleaning on the similar image set based on the image features of the similar images to obtain a target similar image set;
[0119] The fine-tuning training unit 403 is used to perform fine-tuning training on the pre-trained classification model using the similar images in the target similar image set and the content category labels marked on each similar image to obtain a target image classification model.
[0120] In an optional implementation of the embodiment of the present application, the density cluster cleaning unit 402 is used to:
[0121] Based on the image features of the similar images, the distance threshold and the quantity threshold of the density clustering cluster, density clustering is performed on the similar images in the similar image set, and similar images that do not belong to the density clustering cluster are determined to be abnormal images; the distance threshold refers to the upper limit of the image feature distance between the similar images constituting the density clustering cluster, and the quantity threshold refers to the lower limit of the number of similar images constituting the density clustering cluster;
[0122] The abnormal images in the similar image set are cleaned to obtain the target similar image set.
[0123] In an optional implementation of the embodiment of the present application, the feature extraction and matching unit 401 is used to:
[0124] Using the feature extraction layer of the preset image classification model, feature extraction is performed on the search image and the gallery image to obtain image features of the search image and image features of the gallery image;
[0125] Based on the image features of the search image and the image features of the gallery images, obtaining the image feature distance between each search image and each gallery image;
[0126] Based on the image feature distance between each search image and each gallery image, similar images of each search image are screened out from the gallery images to form a similar image set corresponding to each search image.
[0127] In an optional implementation of the embodiment of the present application, the pre-trained classification model is obtained by training a classification network based on multi-category labeled sample data; wherein the multi-category labeled sample data includes multi-category labeled sample images and multiple content category labels marked with each sample image, and the multi-category labeled sample images refer to sample images marked with multiple content category labels.
[0128] In an optional implementation manner of the embodiment of the present application, the device further includes:
[0129] An acquisition unit, configured to acquire single-category label sample data; the single-category label sample data includes single-category label sample images and a content category label marked on each sample image, wherein the single-category label sample image refers to a sample image marked with only a single content category label;
[0130] Correspondingly, the fine-tuning training unit 403 is used to:
[0131] The target image classification model is obtained by performing multi-task fine-tuning training on the pre-trained classification model using similar images in the target similar image set, content category labels marked on each similar image, and the single-category label sample data.
[0132] Through various implementation methods provided in this embodiment, a search image is obtained by performing an image search through historical user input text, and the search image is marked with a content category label; through the feature extraction layer of the preset image classification model, the image features of the search image and the image features of the gallery image are extracted and matched to obtain similar images of the search image to form a similar image set; through the image features of the similar images, density clustering is performed to clean the similar image set to obtain a target similar image set; with the similar images in the target similar image set and the content category labels marked by each similar image, the pre-trained classification model is fine-tuned and trained to obtain a target image classification model. It can be seen that after obtaining a similar image set with the same content category through feature extraction and matching, the style consistency of similar images in the similar image set is improved through density clustering and cleaning to obtain a target similar image set; the target image classification model obtained by fine-tuning the training based on the target similar image set not only pays attention to the content category of the image, but also pays attention to the style of the image. The target image classification model is used to search for images with images, and not only the content category of the images is the same, but also the style of the images is consistent, which is suitable for image search scenarios with style consistency requirements.
[0133] See also Figure 5 , shows a schematic diagram of the structure of an image search device in an embodiment of the present application. In this embodiment, using the target image classification model of the above embodiment, the device may specifically include:
[0134] A feature extraction unit 501 is used to extract features from a target search image and a gallery image using a feature extraction layer of a target image classification model to obtain image features of the target search image and image features of the gallery image;
[0135] The matching search unit 502 is used to match the image features of the target search image with the image features of the gallery image, and search for a target similar image of the target search image, where the target similar image has the same style as the target search image.
[0136] Through various implementation methods provided in this embodiment, the image features of the target search image and the image features of the gallery image are extracted through the feature extraction layer of the target image classification model; the image features of the target search image and the image features of the gallery image are matched to search for target similar images with the same style as the target search image. On the basis that the target image classification model not only pays attention to the content category of the image, but also pays attention to the style of the image, the target image classification model is used to search for images by image, and not only the content category of the images is the same, but also the style of the images is consistent, which greatly improves the image search accuracy in the image search by image scenario with style consistency requirements.
[0137] Figure 6 6 is a block diagram of an apparatus 600 for training an image classification model or image search according to an exemplary embodiment. For example, the apparatus 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0138] Reference Figure 6 , the device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .
[0139] The processing component 602 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.
[0140] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0141] The power supply component 606 provides power to the various components of the device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 600.
[0142] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0143] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.
[0144] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0145] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 614 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, and the sensor assembly 614 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600, and the temperature change of the device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0146] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0147] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.
[0148] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by the processor 620 of the device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0149] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform a method for training an image classification model, the method comprising:
[0150] Using the feature extraction layer of a preset image classification model, feature extraction and matching are performed on the search image and the gallery image to obtain similar images of the search image to form a similar image set; the search image is obtained by performing image search through historical user input text, and the search image is marked with a content category label;
[0151] Based on the image features of the similar images, performing density clustering cleaning on the similar image set to obtain a target similar image set;
[0152] The pre-trained classification model is fine-tuned using similar images in the target similar image set and content category labels marked on each similar image to obtain a target image classification model.
[0153] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform an image search method, using the above-mentioned target image classification model, the method comprising:
[0154] Using the feature extraction layer of the target image classification model, feature extraction is performed on the target search image and the gallery image to obtain image features of the target search image and image features of the gallery image;
[0155] The image features of the target search image and the image features of the gallery image are matched to determine a target similar image of the target search image, wherein the target similar image has the same style as the target search image.
[0156] Figure 7 7 is a schematic diagram of the structure of the server in the embodiment of the present application. The server 700 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 722 (for example, one or more processors) and a memory 732, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 742 or data 744. Among them, the memory 732 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 722 can be configured to communicate with the storage medium 730 and execute a series of instruction operations in the storage medium 730 on the server 700.
[0157] The server 700 may also include one or more power supplies 726, one or more wired or wireless network interfaces 750, one or more input and output interfaces 758, one or more keyboards 756, and / or one or more operating systems 741, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0158] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0159] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0160] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprises a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0161] The above are only preferred embodiments of the present application, and are not intended to limit the present application in any form. Although the present application has been disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technician familiar with the art can use the above disclosed methods and technical contents to make many possible changes and modifications to the technical solution of the present application without departing from the scope of the technical solution of the present application, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still falls within the scope of protection of the technical solution of the present application.
Claims
1. A method for training an image classification model, characterized in that: include: Using the feature extraction layer of a preset image classification model, feature extraction and matching are performed on the search image and the gallery image to obtain similar images of the search image to form a similar image set; The search image is obtained by performing image search through historical user input text, and the search image is marked with a content category label; Based on the image features of the similar images, the distance threshold and the quantity threshold of the density clustering cluster, density clustering is performed on the similar images in the similar image set, and similar images that do not belong to the density clustering cluster are determined to be abnormal images; the distance threshold refers to the upper limit of the image feature distance between the similar images constituting the density clustering cluster, and the quantity threshold refers to the lower limit of the number of similar images constituting the density clustering cluster; Cleaning the abnormal images in the similar image set to obtain a target similar image set, wherein the similar images in the target similar image set have the same content category and consistent style; Using similar images in the target similar image set as input and content category labels marked on each similar image as output, the pre-trained classification model is fine-tuned to obtain a target image classification model.
2. The method according to claim 1, characterized in that The method utilizes the feature extraction layer of the preset image classification model to perform feature extraction and matching on the search image and the gallery image, and obtains similar images of the search image to form a similar image set, including: Using the feature extraction layer of the preset image classification model, feature extraction is performed on the search image and the gallery image to obtain image features of the search image and image features of the gallery image; Based on the image features of the search image and the image features of the gallery images, obtaining the image feature distance between each search image and each gallery image; Based on the image feature distance between each search image and each gallery image, similar images of each search image are screened out from the gallery images to form a similar image set corresponding to each search image.
3. The method according to claim 1, characterized in that The pre-trained classification model is obtained by training a classification network based on multi-category labeled sample data; wherein the multi-category labeled sample data includes multi-category labeled sample images and multiple content category labels marked with each sample image, and the multi-category labeled sample images refer to sample images marked with multiple content category labels.
4. The method according to claim 1, characterized in that: The method further comprises: Acquire single-category label sample data; the single-category label sample data includes single-category label sample images and content category labels marked on each sample image, wherein the single-category label sample images refer to sample images marked with only a single content category label; Correspondingly, the target image classification model is obtained by fine-tuning the pre-trained classification model using the similar images in the target similar image set and the content category labels of each similar image mark, which is: The target image classification model is obtained by performing multi-task fine-tuning training on the pre-trained classification model using similar images in the target similar image set, content category labels marked on each similar image, and the single-category label sample data.
5. A method for image search, characterized in that: Using the target image classification model according to any one of claims 1 to 4, the method comprises: Using the feature extraction layer of the target image classification model, feature extraction is performed on the target search image and the gallery image to obtain image features of the target search image and image features of the gallery image; The image features of the target search image and the image features of the gallery image are matched to determine a target similar image of the target search image, wherein the target similar image has the same style as the target search image.
6. A device for training an image classification model, characterized in that: include: A feature extraction and matching unit, used to perform feature extraction and matching on a search image and a gallery image using a feature extraction layer of a preset image classification model, and obtain similar images of the search image to form a similar image set; the search image is obtained by performing an image search through a historical user input text, and the search image is marked with a content category label; A density cluster cleaning unit is used to perform density clustering on similar images in the similar image set based on image features of the similar images, a distance threshold and a quantity threshold of the density clustering cluster, and determine similar images that do not belong to the density clustering cluster as abnormal images; the distance threshold refers to the upper limit of the image feature distance between similar images constituting the density clustering cluster, and the quantity threshold refers to the lower limit of the number of similar images constituting the density clustering cluster; the abnormal images in the similar image set are cleaned to obtain a target similar image set, in which similar images in the target similar image set have the same content category and consistent style; The fine-tuning training unit is used to use similar images in the target similar image set as input and content category labels marked on each similar image as output, and fine-tune the pre-trained classification model to obtain a target image classification model.
7. The device according to claim 6, characterized in that The feature extraction and matching unit is used to: Using the feature extraction layer of the preset image classification model, feature extraction is performed on the search image and the gallery image to obtain image features of the search image and image features of the gallery image; Based on the image features of the search image and the image features of the gallery images, obtaining the image feature distance between each search image and each gallery image; Based on the image feature distance between each search image and each gallery image, similar images of each search image are screened out from the gallery images to form a similar image set corresponding to each search image.
8. The device according to claim 6, characterized in that The pre-trained classification model is obtained by training a classification network based on multi-category labeled sample data; wherein the multi-category labeled sample data includes multi-category labeled sample images and multiple content category labels marked with each sample image, and the multi-category labeled sample images refer to sample images marked with multiple content category labels.
9. The device according to claim 6, characterized in that The device also includes: An acquisition unit, configured to acquire single-category label sample data; the single-category label sample data includes single-category label sample images and a content category label marked on each sample image, wherein the single-category label sample image refers to a sample image marked with only a single content category label; Correspondingly, the fine-tuning training unit is used to: The target image classification model is obtained by performing multi-task fine-tuning training on the pre-trained classification model using similar images in the target similar image set, content category labels marked on each similar image, and the single-category label sample data.
10. An image search device, characterized in that: Utilizing the target image classification model as described in any one of claims 1 to 4, comprising: A feature extraction unit, used to extract features from a target search image and a gallery image using a feature extraction layer of the target image classification model, to obtain image features of the target search image and image features of the gallery image; The matching search unit is used to match the image features of the target search image with the image features of the gallery image to determine a target similar image of the target search image, wherein the target similar image has the same style as the target search image.
11. A device for training an image classification model, characterized in that: It includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute the method for training an image classification model as described in any one of claims 1 to 4.
12. A device for image search, characterized in that: The device comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors to perform the image search method according to claim 5.
13. A machine-readable medium having instructions stored thereon, which, when executed by one or more processors, causes the device to execute the method for training an image classification model as described in any one of claims 1 to 4; or, causes the device to execute the method for image search as described in claim 5.
