Image retrieval method and device, electronic equipment and storage medium

Abstract

The invention provides an image retrieval method. The method comprises the following steps: acquiring an input text of a user; performing text image searching processing in an image library based on the input text to obtain an initial image retrieval result; classifying the initial image retrieval result through a preset image classification model to obtain a target image retrieval result, and training the image classification model according to a historical image retrieval result labeled by a user; and returning the target image retrieval result to the user. According to the method, the image search processing is carried out in the image library according to the input text of the user to obtain the initial image retrieval result, the initial image retrieval result is classified through the preset image classification model to obtain the target image retrieval result, and the target image retrieval result is returned to the user. The problem that an existing retrieval method has semantic matching deviation and cannot adapt to the actual judgment standard of a user for correlation online, and consequently the retrieval result is inaccurate is solved.

Description

Technical Field

[0001] This invention relates to the field of image retrieval technology, and in particular to an image retrieval method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of multimodal deep learning, image-text joint embedding models (such as CLIP) have been widely applied in scenarios such as image retrieval, image generation, and content moderation. CLIP models encode text and images into the same vector space, enabling direct retrieval of semantically relevant images from large-scale image databases based on natural language descriptions. However, existing image-text retrieval systems like CLIP still have certain limitations in practical applications. On the one hand, as a general pre-trained model, CLIP is prone to semantic matching bias in specific domains (such as industrial drawings, medical images, and e-commerce product images). On the other hand, CLIP models cannot dynamically adjust retrieval results based on the user's personalized intent and cannot adapt online to the user's actual judgment criteria for "relevance." For example, if a user inputs the text "red sneakers," CLIP may return various styles of red shoes, but the user may prefer a specific style or angle; this personalized retrieval intent cannot be captured by the CLIP model. Therefore, a highly accurate image retrieval method is urgently needed to address the problems of semantic matching bias and the inability to adapt online to the user's actual judgment criteria for "relevance," leading to inaccurate retrieval results. Summary of the Invention

[0003] This invention provides an image retrieval method aimed at solving the problem that existing retrieval methods suffer from semantic matching bias and cannot adapt to users' actual judgment criteria for "relevance" online, resulting in inaccurate retrieval results. This invention performs text-to-image search processing in an image library based on the user's input text to obtain initial image retrieval results. Then, it classifies these initial image retrieval results using a preset image classification model to obtain target image retrieval results, which are then returned to the user. This solves the problem of semantic matching bias and inaccurate retrieval results caused by existing retrieval methods' inability to adapt to users' actual judgment criteria for "relevance" online.

[0004] In a first aspect, embodiments of the present invention provide an image retrieval method, the method comprising the following steps:

[0005] Get the user's input text;

[0006] Based on the input text, text search and image processing are performed in the image library to obtain initial image retrieval results;

[0007] The initial image retrieval results are classified using a preset image classification model to obtain the target image retrieval results. The image classification model is trained based on the historical image retrieval results annotated by the user.

[0008] The target image retrieval results are returned to the user.

[0009] Optionally, the step of performing text-to-image search processing in the image library based on the input text to obtain initial image retrieval results includes:

[0010] The input text is encoded into the target vector space using a preset encoding method to obtain a text encoding vector;

[0011] Based on the similarity comparison between the text encoding vector and the image encoding vector in the image library, multiple candidate image encoding vectors are obtained. The image encoding vector is obtained by encoding the images in the image library into the target vector space through a preset encoding method, and each image corresponds to one image encoding vector.

[0012] Based on the correspondence between the image encoding vector and the image, the candidate image corresponding to the candidate image encoding vector is obtained as the initial image retrieval result.

[0013] Optionally, before classifying the initial image retrieval results using a preset image classification model to obtain the target image retrieval results, the method further includes:

[0014] Build a lightweight classification model;

[0015] In addition, retrieve historical image search results annotated by the user;

[0016] The lightweight classification model is trained under supervision using the historical image retrieval results annotated by the user, and the image classification model is obtained after training is completed.

[0017] Optionally, obtaining the historical image retrieval results annotated by the user includes:

[0018] Send the historical images retrieved from the historical image retrieval results to the user;

[0019] The system receives annotation data from the user corresponding to the historical searched images and obtains the historical image search results annotated by the user. The annotation data consists of positive sample identifiers and negative sample identifiers, with each historical searched image corresponding to one positive sample identifier or one negative sample identifier.

[0020] Optionally, the step of supervising the training of the lightweight classification model using the user-annotated historical image retrieval results includes:

[0021] The historical images in the user-annotated historical image retrieval results are encoded into the target space to obtain the sample image encoding vector corresponding to the historical image retrieval image, and each historical image retrieval image corresponds to one sample image encoding vector;

[0022] The lightweight classification model is trained under supervision based on the sample image encoding vector corresponding to the historical image retrieval image and the corresponding annotation data.

[0023] Optionally, the step of classifying the initial image retrieval results using a preset image classification model to obtain the target image retrieval results includes:

[0024] The candidate images are input into a preset image classification model for processing to obtain the category of the candidate images;

[0025] Based on the category of the candidate images, candidate images that match the target category are selected as the target image retrieval results.

[0026] Optionally, after receiving the user's annotation data corresponding to the historical searched images, the method further includes:

[0027] Build user profiles;

[0028] The labeled data corresponding to the historical searched images are accumulated to form the user profile, thus obtaining the target user profile;

[0029] Based on the target user profile, the classification results of the encoding method or the image classification model are adjusted according to preferences.

[0030] Secondly, embodiments of the present invention provide an image retrieval device, the image retrieval device comprising:

[0031] The acquisition module is used to acquire the user's input text;

[0032] The text search and image processing module is used to perform text search and image processing in the image library based on the input text to obtain initial image retrieval results;

[0033] The classification processing module is used to classify the initial image retrieval results using a preset image classification model to obtain the target image retrieval results. The image classification model is trained based on the historical image retrieval results annotated by the user.

[0034] The return module is used to return the target image retrieval results to the user.

[0035] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the image retrieval method provided in embodiments of the present invention.

[0036] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the image retrieval method provided in the embodiments of the invention.

[0037] In this embodiment of the invention, user input text is obtained; text-based image search processing is performed in an image library based on the input text to obtain initial image retrieval results; the initial image retrieval results are classified using a preset image classification model to obtain target image retrieval results, the image classification model being trained based on user-annotated historical image retrieval results; and the target image retrieval results are returned to the user. This invention solves the problem of semantic matching bias in existing retrieval methods, which cannot adapt to users' actual judgment criteria for "relevance" online, leading to inaccurate retrieval results.

[0038] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of an image retrieval method provided in an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the structure of an image retrieval device provided in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] like Figure 1 As shown, Figure 1 This is a flowchart of an image retrieval method provided in an embodiment of the present invention. The image retrieval method includes the following steps:

[0045] 101. Obtain the user's input text.

[0046] In this embodiment of the invention, the image retrieval method described above can be applied to an image retrieval platform. This platform can be built on a server-based or distributed architecture and includes a data interface (for data uploaded by sensors or users), a knowledge database, and a knowledge database construction program. The data interface can be used to acquire user input text, and the knowledge database construction program can be used to construct the knowledge database. This knowledge database is specifically designed to provide additional relational information for identified data entities, thereby enhancing the data recognition system's understanding of the content.

[0047] The users mentioned above can be understood as users of the image retrieval platform, including administrators, etc.

[0048] The above input text can be understood as the text data entered by the user during the image retrieval process.

[0049] 102. Based on the input text, perform text-to-image search processing in the image library to obtain the initial image retrieval results.

[0050] In this embodiment of the invention, the aforementioned image library can be understood as a database used for storing, managing, and retrieving images. The image library can originate from user-built platforms, image content platforms, media resource libraries, etc. The aforementioned image library can be a database combining text search and image search functions. The aforementioned image library can search for images matching the text information input by the user. Specifically, the image library associates and stores text information and image descriptions to quickly find images that meet the search criteria.

[0051] The above-mentioned text-based image search is a search method that allows you to find matching images by entering text descriptions. For example, you can enter a descriptive phrase about a person's clothing characteristics to perform a text-based image search.

[0052] The above text search image processing can be understood as the process of searching for images that match the description in the image library by uploading user information or inputting text description.

[0053] The initial image retrieval results described above can be obtained by performing text-to-image search processing on the image library based on the input text, resulting in image search results corresponding to the input text. These initial image retrieval results include a list of images related to the input text.

[0054] It should be noted that text-based image search can be performed by inputting text descriptions related to people (such as clothing characteristics) into a search engine to obtain M text-based image descriptions related to people, and these M text-based image descriptions related to people can be used as the initial image search results.

[0055] 103. Using a preset image classification model, classify the initial image retrieval results to obtain the target image retrieval results.

[0056] In this embodiment of the invention, the above-mentioned image classification model is trained based on the historical image retrieval results annotated by the user.

[0057] The aforementioned preset image classification model can be an image classification model built based on deep learning or machine learning, such as SVM (Support Vector Machine) or KNN (K-Nearest Neighbor). The core idea of ​​SVM is to separate data of different categories by constructing an optimal hyperplane, aiming to maximize the margin between different categories. KNN makes classification decisions by finding the K most similar neighbors to the sample to be classified in the training data. The core idea of ​​KNN is that in the feature space, if most of the K nearest neighbors of a sample belong to the same category, then the sample also belongs to that category. The aforementioned preset image classification model can classify the initial image retrieval results, filtering out images irrelevant to the input text and outputting images relevant to the input text. The aforementioned preset image classification model is obtained by training an untrained image classification model using a training dataset. The training dataset includes sample image data and corresponding labeled data. The labeled data consists of positive and negative sample labels, with each historical retrieved image corresponding to either a positive or negative sample label. The positive sample labels mentioned above can be understood as images that are related to the input text, while the negative sample labels can be understood as images that are not related to the input text. The training described above can be supervised training, which uses a set of data with known labels to train the model. By optimizing the model parameters, the model can predict the labels of new data or make decisions based on the characteristics of existing data. During training, the minimum loss function can be used to adjust the model parameters to minimize the difference between the model's output label and the input data. The loss function measures the difference between the model's prediction and the true result; its purpose is to improve prediction accuracy by minimizing the loss function value by adjusting the model parameters. The loss function mentioned above can be the mean squared error loss function, cross-entropy loss function, etc.

[0058] The above classification process can be understood as the process of classifying the initial image retrieval results into categories using a preset image classification model.

[0059] The above target image retrieval results are obtained by classifying the initial image retrieval results using a preset image classification model.

[0060] 104. Return the target image retrieval results to the user.

[0061] In this embodiment of the invention, the target retrieval results can be displayed to the user in the form of a list.

[0062] In this embodiment of the invention, the initial image retrieval results are classified using a preset image classification model. Images deemed irrelevant by the model are filtered out, and only images deemed relevant by the model are retained. These relevant images are then returned to the user. This solves the problem that existing retrieval methods suffer from semantic matching bias and cannot adapt to users' actual judgment criteria for "relevance" online, leading to inaccurate retrieval results.

[0063] In this embodiment of the invention, user input text is obtained; text-based image search processing is performed on the image library based on the input text to obtain initial image retrieval results; the initial image retrieval results are classified using a preset image classification model to obtain target image retrieval results, the image classification model being trained based on user-annotated historical image retrieval results; and the target image retrieval results are returned to the user. This invention solves the problem of semantic matching bias in existing retrieval methods, which cannot adapt to users' actual judgment criteria for "relevance" online, leading to inaccurate retrieval results.

[0064] It is understood that in the specific implementation of this application, data such as text data, image data, knowledge data, and user data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required. Furthermore, the collection, use, and processing of related data, as well as the training, deployment, and invocation of algorithm models, must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0065] Optionally, in the step of performing text-to-image search processing in the image library based on the input text to obtain the initial image retrieval results, the input text can be encoded into the target vector space using a preset encoding method to obtain a text encoding vector; a similarity comparison is performed between the text encoding vector and the image encoding vectors in the image library to obtain multiple candidate image encoding vectors; and based on the correspondence between the image encoding vectors and the images, the candidate images corresponding to the candidate image encoding vectors are obtained as the initial image retrieval results.

[0066] In this embodiment of the invention, the aforementioned preset encoding method is an encoding method pre-set by the system, such as using natural language processing technology to convert text into image encoding vectors. The aforementioned encoding method can be understood as a technique or rule for converting information from one form or format to another.

[0067] The input text mentioned above can be text data entered by the user during the image retrieval process.

[0068] It should be noted that the input text can be fed into a multimodal large model, which then outputs the feature vector corresponding to the input text. This feature vector is then encoded into the target vector space using a preset encoding method, resulting in a text encoding vector. The feature vector corresponding to the input text can be the searched image vector features, such as color, shape, and texture.

[0069] The aforementioned multimodal large-scale model possesses cross-modal information fusion capabilities, enabling it to process data from different modalities simultaneously, such as text descriptions combined with image verification, capturing richer features and correlations, and enhancing the depth of understanding of complex scenarios. This multimodal large-scale model can be built based on deep learning or machine learning, such as CLIP or LLM. It can also identify the text feature vectors of the input text.

[0070] The aforementioned multimodal large model is obtained by training an untrained multimodal large model using a training dataset. This training data includes multimodal sample data and corresponding feature vector annotations. The multimodal sample data can be understood as the multimodal input data, while the annotation data adds structured labels to the original data, enabling the machine learning model to recognize and process both the original and labeled data. Through the annotation data, the model establishes a mapping relationship between input data and the correct output label. The training can be supervised training, which uses a set of data with known labels to train the model. By optimizing the model parameters, the model can predict the label of new data or make decisions based on the characteristics of existing data. During training, a minimum loss function can be used to adjust the model parameters to minimize the difference between the model's output label and the input data. The loss function measures the difference between the model's prediction and the true result, aiming to improve prediction accuracy by minimizing the loss function value through adjusting the model parameters. The loss function can be a mean squared error loss function, a cross-entropy loss function, etc.

[0071] The image encoding vectors mentioned above are obtained by encoding the images in the image library into the target vector space using a preset encoding method, with each image corresponding to one image encoding vector.

[0072] The target vector space described above can be understood as a multi-dimensional space, where each encoded vector represents a feature or attribute. For example, features such as color, texture, and shape of an image can be used as encodings in the target vector space. Each image can be represented by an encoded vector, which contains the values ​​of the image's features.

[0073] The aforementioned image library is a database used for storing, managing, and retrieving images. The image library can originate from user-built platforms, image content platforms, media resource libraries, etc. It can be a database that combines text search and image search functions. Each image in the image library contains a corresponding image encoding vector.

[0074] The aforementioned similarity comparison can be a process of measuring the degree of similarity between a text encoding vector and an image encoding vector in an image library. For example, cosine similarity algorithms can be used to calculate the cosine value of the angle between the text encoding vector and the image encoding vector in the image library to determine the similarity. The greater the similarity, the greater the similarity of the image encoding vectors; the smaller the similarity, the smaller the similarity of the image encoding vectors.

[0075] The aforementioned candidate image encoding vectors can be understood as candidate image encoding vectors. Specifically, the text encoding vector can be compared with the image encoding vectors in the image library in terms of similarity, and the K image encoding vectors with a similarity greater than a preset threshold can be selected as candidate image encoding vectors.

[0076] The correspondence between the image encoding vectors and the images mentioned above can be a pre-set correspondence by the system, with different image encoding vectors corresponding to different images.

[0077] The candidate images mentioned above can be understood as images that are similar to the input text.

[0078] The initial image retrieval results described above are candidate images corresponding to the candidate image encoding vectors, obtained based on the correspondence between image encoding vectors and images.

[0079] Specifically, a multimodal large model can output the feature vector corresponding to the input text, and encode the feature vector corresponding to the input text into the target vector space through a preset encoding method to obtain the text encoding vector. The text encoding vector is then compared with the image encoding vectors in the image library to obtain multiple candidate image encoding vectors. Based on the correspondence between the image encoding vectors and the images, the candidate images corresponding to the candidate image encoding vectors are obtained, and the corresponding candidate images are used as the initial image retrieval results.

[0080] Optionally, before the step of classifying the initial image retrieval results using a preset image classification model to obtain the target image retrieval results, a lightweight classification model can be constructed; and user-annotated historical image retrieval results can be obtained; the lightweight classification model can be trained under supervision using user-annotated historical image retrieval results, and the image classification model can be obtained after training.

[0081] In this embodiment of the invention, the lightweight classification model described above can be a lightweight classification model built based on deep learning or machine learning, such as SVM, KNN, etc.

[0082] The above historical image retrieval results can be understood as follows: in historical image retrieval, the user inputs text descriptions related to people into the search engine for text-to-image processing, and obtains J images with text descriptions related to people.

[0083] The above-mentioned user-annotated historical image retrieval results can be understood as users annotating the historical image retrieval results, adding semantic information to the images in the historical image retrieval results, so as to be used for training and optimizing the classification model.

[0084] It should be noted that the user-annotated historical image retrieval results are used as the training dataset to supervise the training of the lightweight classification model, and the image classification model is obtained after training.

[0085] The above supervised training can be understood as using a set of data with known labels to train the model, and by optimizing the model parameters, enabling the model to predict the labels of new data or make decisions based on the characteristics of existing data.

[0086] Specifically, during training, the model's parameters can be tuned using a minimum loss function to minimize the difference between the model's output label and the input data. The loss function measures the difference between the model's prediction and the true result; its purpose is to improve prediction accuracy by minimizing the loss function value through adjusting the model parameters. The aforementioned loss function can be the mean squared error loss function, cross-entropy loss function, etc. This small model can identify the feature vectors of the search data.

[0087] The image classification model described above is trained under supervised supervision using user-annotated historical image retrieval results. Once trained, this lightweight classification model is obtained and is capable of classifying image retrieval results.

[0088] Optionally, in the step of obtaining the historical image retrieval results annotated by the user, the historical retrieval images in the historical image retrieval results can be sent to the user; the annotation data corresponding to the historical retrieval images by the user can be received to obtain the historical image retrieval results annotated by the user.

[0089] In this embodiment of the invention, the aforementioned labeled data consists of positive and negative sample identifiers, with each historical retrieved image corresponding to either a positive or negative sample identifier. A positive sample identifier can be understood as indicating that the image is related to the input text, while a negative sample identifier can be understood as indicating that the image is not related to the input text.

[0090] The above-mentioned user-annotated historical image retrieval results can be the user's annotations on historically retrieved images, adding semantic information to the historically retrieved images, resulting in user-annotated historical image retrieval results.

[0091] It should be noted that the historical images retrieved from the historical image retrieval results can be sent to the user, and the user's annotation data corresponding to the historical images can be received to obtain the historical image retrieval results annotated by the user.

[0092] Optionally, in the step of supervising the training of the lightweight classification model using user-annotated historical image retrieval results, the historical retrieval images in the user-annotated historical image retrieval results can be encoded into the target space to obtain the sample image encoding vector corresponding to the historical image retrieval images; the lightweight classification model can then be trained in a supervised manner based on the sample image encoding vector corresponding to the historical image retrieval images and the annotation data corresponding to the historical image retrieval images.

[0093] In this embodiment of the invention, each historical image retrieval image corresponds to a sample image encoding vector.

[0094] The above-mentioned sample image encoding vector can be obtained by encoding the historical retrieved images in the user-annotated historical image retrieval results into the target space, thus obtaining the image encoding vector corresponding to the historical image retrieved images.

[0095] The aforementioned lightweight classification model can be a lightweight classification model built based on deep learning or machine learning, such as SVM, KNN, etc.

[0096] The above labeled data consists of positive and negative sample identifiers, with each historical retrieved image corresponding to either a positive or negative sample identifier. A positive sample identifier can be understood as indicating that the image is relevant to the input text, while a negative sample identifier can be understood as indicating that the image is not relevant to the input text.

[0097] The supervised training described above can be trained using a set of data with known labels, and the model parameters can be optimized so that the model can predict the labels of new data or make decisions based on the characteristics of existing data.

[0098] Specifically, during training, the model's parameters can be tuned using a minimum loss function to minimize the difference between the model's output label and the input data. The loss function measures the difference between the model's prediction and the true result; its purpose is to improve prediction accuracy by minimizing the loss function value through adjusting the model parameters. The aforementioned loss function can be the mean squared error loss function, cross-entropy loss function, etc. This small model can identify the feature vectors of the search data.

[0099] Optionally, in the step of classifying the initial image retrieval results using a preset image classification model to obtain the target image retrieval results, candidate images can be input into the preset image classification model for processing to obtain the category of the candidate images; based on the category of the candidate images, candidate images that match the target category are selected from the candidate images as the target image retrieval results.

[0100] In this embodiment of the invention, the candidate image can be an image that has a similarity to the input text. The input text can be text data entered by the user during the image retrieval process.

[0101] The aforementioned preset image classification model can be an image classification model built based on deep learning or machine learning, such as SVM, KNN, etc. This preset image classification model can classify the initial image retrieval results, filter out images irrelevant to the input text, and output images relevant to the input text.

[0102] The aforementioned pre-defined image classification model is obtained by training an untrained image classification model using a training dataset. This training dataset includes sample image data and corresponding labeled data. The labeled data consists of positive and negative sample labels, with each historical retrieved image corresponding to either a positive or negative label. A positive sample label indicates that the image is related to the input text, while a negative sample label indicates that the image is not related to the input text. The training can be supervised training, which uses a set of data with known labels to train the model. By optimizing the model parameters, the model can predict the label of new data or make decisions based on the characteristics of existing data. During training, a minimum loss function can be used to adjust the model parameters to minimize the difference between the model's output label and the input data. The loss function measures the difference between the model's prediction and the true result, aiming to improve prediction accuracy by minimizing the loss function value through adjusting the model parameters. The loss function can be a mean squared error loss function, a cross-entropy loss function, etc.

[0103] The above process can be understood as inputting candidate images into a preset image classification model for classification in order to obtain the category of the candidate images.

[0104] The target category mentioned above can be understood as the category corresponding to the input text.

[0105] The above target image retrieval results can be obtained by selecting candidate images that match the target category from among the candidate images, based on the category of the candidate images.

[0106] Optionally, after receiving the user's labeled data corresponding to historical searched images, a user profile can be constructed; the labeled data corresponding to historical searched images can be accumulated in the user profile to obtain a target user profile; based on the target user profile, the encoding method or the classification results of the image classification model can be adjusted according to preferences.

[0107] In this embodiment of the invention, the above-mentioned user profile can be understood as a description that can represent the user, such as user characteristics, behavior, needs, search history, etc.

[0108] The aforementioned historical search images can be understood as follows: in historical image retrieval, users input text descriptions related to people into a search engine for text-based image processing, resulting in Z images with text descriptions related to people.

[0109] The above labeled data consists of positive and negative sample identifiers, with each historical retrieved image corresponding to either a positive or negative sample identifier. A positive sample identifier indicates that the image is relevant to the input text, while a negative sample identifier indicates that the image is not relevant to the input text. The input text can be the text data entered by the user during the image retrieval process.

[0110] The aforementioned accumulation can be understood as the process of continuously collecting and organizing user profiles by using labeled data corresponding to historically retrieved images, in order to form target user profiles.

[0111] The above encoding method can be understood as a technique or rule for converting information from one form or format to another.

[0112] The classification result of the above image classification model can be understood as the image classification model analyzing the category information of the image and outputting the category to which the image belongs.

[0113] The aforementioned preference adjustment can be understood as a process of adjusting the coding method or image classification model's classification results based on the target user profile. This preference adjustment can be a process of making the output of the coding method or image classification model more consistent with the target user profile, thus optimizing the coding method or image classification model through the target user profile.

[0114] In this embodiment of the invention, the labeled data corresponding to historical retrieved images can be accumulated into the user profile to obtain the target user profile. Based on the target user profile, the encoding method or the classification results of the image classification model can be adjusted according to preferences. This can improve the personalization capability of image retrieval, achieve high-precision image retrieval, and improve the accuracy of image retrieval results and user satisfaction.

[0115] like Figure 2 As shown, an embodiment of the present invention provides an image retrieval device, which includes:

[0116] Module 201 is used to acquire the user's input text;

[0117] The text search and image processing module 202 is used to perform text search and image processing in the image library based on the input text to obtain initial image retrieval results;

[0118] The classification processing module 203 is used to classify the initial image retrieval results using a preset image classification model to obtain the target image retrieval results. The image classification model is trained based on the historical image retrieval results annotated by the user.

[0119] The return module 204 is used to return the target image retrieval results to the user.

[0120] Optionally, the text search image processing module 202 is further configured to encode the input text into a target vector space using a preset encoding method to obtain a text encoding vector; perform a similarity comparison between the text encoding vector and the image encoding vectors in the image library to obtain multiple candidate image encoding vectors, wherein the image encoding vector is obtained by encoding the images in the image library into the target vector space using a preset encoding method, and each image corresponds to one image encoding vector; and based on the correspondence between the image encoding vectors and the images, obtain the candidate images corresponding to the candidate image encoding vectors as the initial image retrieval results.

[0121] Optionally, the device is further configured to construct a lightweight classification model; and to obtain user-annotated historical image retrieval results; and to perform supervised training on the lightweight classification model using the user-annotated historical image retrieval results, thereby obtaining the image classification model upon completion of training.

[0122] Optionally, the device is further configured to send historical search images from the historical image search results to the user; receive annotation data corresponding to the historical search images from the user, and obtain the historical image search results annotated by the user, wherein the annotation data consists of positive sample identifiers and negative sample identifiers, and each historical search image corresponds to one positive sample identifier or one negative sample identifier.

[0123] Optionally, the device is further configured to encode the historical retrieved images in the user-annotated historical image retrieval results into the target space to obtain sample image encoding vectors corresponding to the historical image retrieved images, with each historical image retrieved image corresponding to one sample image encoding vector; and to perform supervised training on the lightweight classification model based on the sample image encoding vectors corresponding to the historical image retrieved images and the annotation data corresponding to the historical image retrieved images.

[0124] Optionally, the general classification processing module 203 is further configured to input the candidate images into a preset image classification model for processing to obtain the category of the candidate images; and based on the category of the candidate images, select candidate images that conform to the target category as the target image retrieval result.

[0125] Optionally, the device is further configured to construct a user profile; accumulate the labeled data corresponding to the historical retrieved images into the user profile to obtain a target user profile; and adjust the encoding method or the classification result of the image classification model based on the target user profile.

[0126] like Figure 3 As shown, this embodiment of the invention also provides an electronic device, including a processor, which can execute any of the above-described image retrieval methods.

[0127] Specifically, it includes a processor 301 and a memory 302, as well as a computer program stored in the memory 302 and capable of running on the processor 301 to execute the image retrieval method, wherein:

[0128] Processor 301 executes the calculator program for the image retrieval method stored in memory 302, performing the following steps:

[0129] Get the user's input text;

[0130] Based on the input text, text search and image processing are performed in the image library to obtain initial image retrieval results;

[0131] The initial image retrieval results are classified using a preset image classification model to obtain the target image retrieval results. The image classification model is trained based on the historical image retrieval results annotated by the user.

[0132] The target image retrieval results are returned to the user.

[0133] Optionally, the text-based image search processing performed by the processor 301 in the image library based on the input text to obtain initial image retrieval results includes:

[0134] The input text is encoded into the target vector space using a preset encoding method to obtain a text encoding vector;

[0135] Based on the similarity comparison between the text encoding vector and the image encoding vector in the image library, multiple candidate image encoding vectors are obtained. The image encoding vector is obtained by encoding the images in the image library into the target vector space through a preset encoding method, and each image corresponds to one image encoding vector.

[0136] Based on the correspondence between the image encoding vector and the image, the candidate image corresponding to the candidate image encoding vector is obtained as the initial image retrieval result.

[0137] Optionally, before classifying the initial image retrieval results using a preset image classification model to obtain the target image retrieval results, the method executed by the processor 301 further includes:

[0138] Build a lightweight classification model;

[0139] In addition, retrieve historical image search results annotated by the user;

[0140] The lightweight classification model is trained under supervision using the historical image retrieval results annotated by the user, and the image classification model is obtained after training is completed.

[0141] Optionally, the process of obtaining the historical image retrieval results annotated by the user, executed by the processor 301, includes:

[0142] Send the historical images retrieved from the historical image retrieval results to the user;

[0143] The system receives annotation data from the user corresponding to the historical searched images and obtains the historical image search results annotated by the user. The annotation data consists of positive sample identifiers and negative sample identifiers, with each historical searched image corresponding to one positive sample identifier or one negative sample identifier.

[0144] Optionally, the processor 301 performs supervised training of the lightweight classification model using the historical image retrieval results annotated by the user, including:

[0145] The historical images in the user-annotated historical image retrieval results are encoded into the target space to obtain the sample image encoding vector corresponding to the historical image retrieval image, and each historical image retrieval image corresponds to one sample image encoding vector;

[0146] The lightweight classification model is trained under supervision based on the sample image encoding vector corresponding to the historical image retrieval image and the corresponding annotation data.

[0147] Optionally, the processor 301 performs classification processing on the initial image retrieval results using a preset image classification model to obtain the target image retrieval results, including:

[0148] The candidate images are input into a preset image classification model for processing to obtain the category of the candidate images;

[0149] Based on the category of the candidate images, candidate images that match the target category are selected as the target image retrieval results.

[0150] Optionally, after receiving the user's annotation data corresponding to the historical searched images, the method executed by the processor 301 further includes:

[0151] Build user profiles;

[0152] The labeled data corresponding to the historical searched images are accumulated to form the user profile, thus obtaining the target user profile;

[0153] Based on the target user profile, the classification results of the encoding method or the image classification model are adjusted according to preferences.

[0154] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the image retrieval method provided in this invention and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0155] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0156] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. An image retrieval method, characterized in that, The method includes the following steps: Get the user's input text; Based on the input text, text search and image processing are performed in the image library to obtain initial image retrieval results; The initial image retrieval results are classified using a preset image classification model to obtain the target image retrieval results. The image classification model is trained based on the historical image retrieval results annotated by the user. The target image retrieval results are returned to the user.

2. The image retrieval method as described in claim 1, characterized in that, The step of performing text-based image search processing in the image library based on the input text to obtain initial image retrieval results includes: The input text is encoded into the target vector space using a preset encoding method to obtain a text encoding vector; Based on the similarity comparison between the text encoding vector and the image encoding vector in the image library, multiple candidate image encoding vectors are obtained. The image encoding vector is obtained by encoding the images in the image library into the target vector space through a preset encoding method, and each image corresponds to one image encoding vector. Based on the correspondence between the image encoding vector and the image, the candidate image corresponding to the candidate image encoding vector is obtained as the initial image retrieval result.

3. The image retrieval method as described in claim 2, characterized in that, Before classifying the initial image retrieval results using a preset image classification model to obtain the target image retrieval results, the method further includes: Build a lightweight classification model; In addition, retrieve historical image search results annotated by the user; The lightweight classification model is trained under supervision using the historical image retrieval results annotated by the user, and the image classification model is obtained after training is completed.

4. The image retrieval method as described in claim 3, characterized in that, The process of obtaining user-annotated historical image retrieval results includes: Send the historical images retrieved from the historical image retrieval results to the user; The system receives user-annotated data corresponding to the historical search images and obtains the user-annotated historical image search results. The annotation data consists of positive and negative sample identifiers, with each historical search image corresponding to one positive or negative sample identifier.

5. The image retrieval method as described in claim 4, characterized in that, The step of supervising the training of the lightweight classification model using the historical image retrieval results annotated by the user includes: The historical images in the user-annotated historical image retrieval results are encoded into the target space to obtain the sample image encoding vector corresponding to the historical image retrieval image, with each historical image retrieval image corresponding to one sample image encoding vector; The lightweight classification model is trained under supervision based on the sample image encoding vectors corresponding to the historical image retrieval images and the corresponding annotation data.

6. The image retrieval method as described in claim 2, characterized in that, The process of classifying the initial image retrieval results using a preset image classification model to obtain the target image retrieval results includes: The candidate images are input into a preset image classification model for processing to obtain the category of the candidate images; Based on the category of the candidate images, candidate images that match the target category are selected as the target image retrieval results.

7. The image retrieval method as described in any one of claims 4 to 6, characterized in that, After receiving the user's annotation data corresponding to the historical searched images, the method further includes: Build user profiles; The labeled data corresponding to the historical searched images are accumulated to form the user profile, thus obtaining the target user profile; Based on the target user profile, the classification results of the encoding method or the image classification model are adjusted according to preferences.

8. An image retrieval device, characterized in that, The image retrieval device includes: The acquisition module is used to acquire the user's input text; The text search and image processing module is used to perform text search and image processing in the image library based on the input text to obtain initial image retrieval results; The classification processing module is used to classify the initial image retrieval results using a preset image classification model to obtain the target image retrieval results. The image classification model is trained based on the historical image retrieval results annotated by the user. The return module is used to return the target image retrieval results to the user.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the image retrieval method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image retrieval method as described in any one of claims 1 to 7.

Images