Image retrieval method and device, equipment and medium

Abstract

The invention discloses an image retrieval method and device, equipment and a medium, and belongs to the technical field of Internet. The method comprises the following steps: receiving a retrieval language of a user, extracting retrieval semantics of the retrieval language, and constructing a retrieval semantic vector; determining fine-tuning semantics of the retrieval language based on a pre-constructed fine-tuning model, and constructing a fine-tuning semantic vector; matching in a vector database according to the retrieval semantic vector and the fine adjustment semantic vector to obtain a preset number of target image semantic vectors; wherein candidate image semantic vectors are pre-stored in the vector database; and obtaining a target image corresponding to the target image semantic vector in the image data set, and feeding back the target image to the user. According to the technical scheme, the objects in the retrieval semantics and the incidence relation between the objects can be accurately summarized through the fine tuning vector, so that the target image can better meet the retrieval condition of a user, and the accuracy of an image retrieval system and the use experience of the user are improved.

Description

Technical Field

[0001] The present application belongs to the field of Internet technology, and specifically relates to an image retrieval method, device, equipment and medium. Background Art

[0002] Image retrieval is an information retrieval method that matches the semantic vectors of text with the image semantic vectors expressed by the image to obtain images that match the scenes described by natural language.

[0003] Usually, using text for image retrieval requires obtaining semantic information of a large amount of images in advance, extracting the corresponding image semantic vectors based on the semantic information, and extracting the semantic vectors of the text input by the user when searching, and determining the images that meet the user's requirements based on the matching vectors. However, this method can only search through a small number of fixed structured fields, and cannot implement natural language retrieval methods, resulting in a poor user experience. In addition, the retrieval system has poor generalization ability, poor adaptability to scenarios, and a limited number of searchable targets.

[0004] Therefore, how to match the search language input by the user with a more appropriate target image is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] The embodiments of the present application provide an image retrieval method, apparatus, device and medium, the purpose of which is to accurately summarize the objects in the retrieval semantics and the associations between objects by fine-tuning vectors, so that the target image can better meet the user's retrieval conditions, which is beneficial to improving the accuracy of the image retrieval system and the user experience.

[0006] In a first aspect, this embodiment provides an image retrieval method, the method comprising:

[0007] Receiving a user's search language, extracting search semantics of the search language, and constructing a search semantic vector;

[0008] Determining the fine-tuning semantics of the retrieval language based on a pre-built fine-tuning model, and constructing a fine-tuning semantic vector;

[0009] Matching the retrieval semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors; wherein the vector database pre-stores candidate image semantic vectors;

[0010] A target image corresponding to the target image semantic vector is obtained in the image data set, and the target image is fed back to the user.

[0011] Further, determining the fine-tuning semantics of the retrieval language based on the pre-built fine-tuning model includes:

[0012] Filtering out relevant images in the image data set according to the retrieval semantic vector;

[0013] The relevant images are input into a fine-tuning model, and the fine-tuning semantics of the retrieval language is determined according to an output result of the fine-tuning model.

[0014] Furthermore, before inputting the relevant image into the fine-tuning model, the method further includes:

[0015] Acquire semantic samples, and acquire corresponding positive image samples and negative image samples in the image data set according to the semantic samples;

[0016] Unsupervised training is performed on the positive image samples and the negative image samples to obtain an algorithm for extracting semantic information that matches the positive image samples and does not match the negative image samples, so as to construct a fine-tuning model for extracting semantic information of images.

[0017] Furthermore, before receiving the user's search language, the method further includes:

[0018] Acquire semantic information of each candidate image in the image data set according to a pre-built semantic recognition model;

[0019] Vectorizing the semantic information of each candidate image to obtain a semantic vector of the candidate image, and associating and storing the semantic vector of the candidate image and its corresponding image in a vector database;

[0020] Accordingly, obtaining a target image corresponding to the target image semantic vector in the image data set includes:

[0021] According to the candidate image semantic vectors and their corresponding images stored in the vector database, a target image corresponding to the target image semantic vector is determined.

[0022] Furthermore, before obtaining the semantic information of each candidate image in the image data set according to the pre-built semantic recognition model, the method further includes:

[0023] Receive uploaded images;

[0024] Filtering the image to obtain a deduplicated image;

[0025] Storing the deduplicated images in the image dataset;

[0026] and / or,

[0027] Receive uploaded videos;

[0028] Extracting images from the video, and filtering the extracted images to obtain deduplicated images;

[0029] The deduplicated images are stored in the image dataset.

[0030] Furthermore, matching the retrieval semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors includes:

[0031] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vector whose first similarity is greater than a first threshold is used as the target image semantic vector;

[0032] or,

[0033] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and the candidate image semantic vector whose second similarity is greater than a second threshold is used as the target image semantic vector;

[0034] or,

[0035] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vectors whose first similarity is greater than a first threshold are extracted as first results;

[0036] Based on the first result, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and a candidate image semantic vector whose second similarity in the first result is greater than a second threshold is extracted as the target image semantic vector.

[0037] Furthermore, the first algorithm includes the following formula:

[0038]

[0039] Among them, S1 is the first similarity, t0 is the semantic vector, t1 is the fine-tuning vector, and i is the semantic vector of the i-th candidate image;

[0040] The second algorithm includes the following formula:

[0041]

[0042] Wherein, P is the intermediate value, t1 is the fine-tuning vector, i is the i-th candidate image semantic vector or the i-th vector in the first result; when P is greater than 0.5, its corresponding weight pro value is 1; when P is not greater than 0.5, its corresponding weight pro value is 0;

[0043] S2=pro*t0·i;

[0044] Wherein, S2 is the second similarity, t0 is the semantic vector, and i is the semantic vector of the i-th candidate image or the i-th vector in the first result.

[0045] In a second aspect, this embodiment provides an image retrieval device, the device comprising:

[0046] A search semantic vector construction module is used to receive a user's search language, extract the search semantics of the search language, and construct a search semantic vector;

[0047] A fine-tuning semantic vector construction module, used to determine the fine-tuning semantics of the retrieval language based on a pre-built fine-tuning model and to construct a fine-tuning semantic vector;

[0048] A target image semantic vector acquisition module, used for matching the retrieval semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors; wherein the vector database pre-stores candidate image semantic vectors;

[0049] The target image feedback module is used to obtain a target image corresponding to the target image semantic vector in the image data set, and feed back the target image to the user.

[0050] Furthermore, the fine-tuning semantic vector construction module is specifically used to:

[0051] Filtering out relevant images in the image data set according to the retrieval semantic vector;

[0052] The relevant images are input into a fine-tuning model, and the fine-tuning semantics of the retrieval language is determined according to an output result of the fine-tuning model.

[0053] Furthermore, the fine-tuning semantic vector construction module is also used to:

[0054] Acquire semantic samples, and acquire corresponding positive image samples and negative image samples in the image data set according to the semantic samples;

[0055] Unsupervised training is performed on the positive image samples and the negative image samples to obtain an algorithm for extracting semantic information that matches the positive image samples and does not match the negative image samples, so as to construct a fine-tuning model for extracting semantic information of images.

[0056] Furthermore, the device also includes:

[0057] A vector database construction module is used to obtain semantic information of each candidate image in the image data set according to a pre-built semantic recognition model;

[0058] Vectorizing the semantic information of each candidate image to obtain a semantic vector of the candidate image, and associating and storing the semantic vector of the candidate image and its corresponding image in a vector database;

[0059] Accordingly, the target image feedback module specifically includes:

[0060] According to the candidate image semantic vectors and their corresponding images stored in the vector database, a target image corresponding to the target image semantic vector is determined.

[0061] Furthermore, the device also includes:

[0062] An image dataset building module for receiving uploaded images;

[0063] Filtering the image to obtain a deduplicated image;

[0064] Storing the deduplicated images in the image dataset;

[0065] and / or,

[0066] Receive uploaded videos;

[0067] Extracting images from the video, and filtering the extracted images to obtain deduplicated images;

[0068] The deduplicated images are stored in the image dataset.

[0069] Furthermore, the target image semantic vector acquisition module is specifically used to:

[0070] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vector whose first similarity is greater than a first threshold is used as the target image semantic vector;

[0071] or,

[0072] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and the candidate image semantic vector whose second similarity is greater than a second threshold is used as the target image semantic vector;

[0073] or,

[0074] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vectors whose first similarity is greater than a first threshold are extracted as first results;

[0075] Based on the first result, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and a candidate image semantic vector whose second similarity in the first result is greater than a second threshold is extracted as the target image semantic vector.

[0076] Furthermore, the first algorithm includes the following formula:

[0077]

[0078] Among them, S1 is the first similarity, t0 is the semantic vector, t1 is the fine-tuning vector, and i is the semantic vector of the i-th candidate image;

[0079] The second algorithm includes the following formula:

[0080]

[0081] Wherein, P is the intermediate value, t1 is the fine-tuning vector, i is the i-th candidate image semantic vector or the i-th vector in the first result; when P is greater than 0.5, its corresponding weight pro value is 1; when P is not greater than 0.5, its corresponding weight pro value is 0;

[0082] S2=pro*t0·i;

[0083] Wherein, S2 is the second similarity, t0 is the semantic vector, and i is the semantic vector of the i-th candidate image or the i-th vector in the first result.

[0084] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0085] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0086] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0087] In an embodiment of the present application, a user's search language is received, the search semantics of the search language is extracted, and a search semantic vector is constructed; the fine-tuning semantics of the search language is determined based on a pre-constructed fine-tuning model, and a fine-tuning semantic vector is constructed; matching is performed in a vector database according to the search semantic vector and the fine-tuning semantic vector to obtain a preset number of target image semantic vectors; wherein candidate image semantic vectors are pre-stored in the vector database; a target image corresponding to the target image semantic vector is obtained in an image data set, and the target image is fed back to the user. Through the above-mentioned image retrieval method, the objects in the search semantics and the associations between the objects can be accurately summarized through the fine-tuning vector, so that the target image can better meet the user's search conditions, which is conducive to improving the accuracy of the image retrieval system and the user's experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 It is a flowchart of an image retrieval method provided in Example 1 of the present application;

[0089] Figure 2 It is a schematic diagram of a process of feeding back a target image according to retrieval semantics provided in the first embodiment of the present application;

[0090] Figure 3 It is a schematic diagram of the process of establishing an image data set and a vector database provided in Example 1 of the present application;

[0091] Figure 4 It is a flowchart of an image retrieval method provided in Example 2 of the present application;

[0092] Figure 5 This is a schematic diagram of the fine-tuning vector acquisition process provided in Example 2 of the present application;

[0093] Figure 6 is a structural schematic diagram of an image retrieval device provided in Embodiment 3 of the present application;

[0094] Figure 7 It is a schematic diagram of the structure of the electronic device provided in Example 4 of the present application. DETAILED DESCRIPTION

[0095] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0096] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0097] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0098] In the following, in conjunction with the accompanying drawings, an image retrieval method, device, equipment and medium provided by the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0099] Embodiment 1

[0100] Figure 1 It is a flowchart of an image retrieval method provided in Example 1 of the present application.

[0101] like Figure 1 As shown, the specific steps include:

[0102] S101, receiving a user's search language, extracting search semantics of the search language, and constructing a search semantic vector;

[0103] The application scenario of this solution may be a scenario in which a retrieval language is used to retrieve corresponding images.

[0104] Based on the above scenario, it can be understood that the executor of this scenario can be an intelligent terminal that feeds back relevant images according to the search language, such as a desktop computer, a laptop computer, a tablet computer, and an interactive multimedia device, etc., and no excessive limitations are made here.

[0105] The search language may be a search condition input by the user based on the desired image, and may include search targets and associations between targets. For example, the search language may be "provide some images that describe a fight at a multi-person bus stop" and the like.

[0106] Search semantics can be the understanding and interpretation of the meaning and concepts expressed by the search language, and the extraction of key information in the search language. For example, search semantics can be "bus stop", "multi-person fight", etc.

[0107] Retrieving semantic vectors can refer to converting text, words or other language units into continuous vector representations, which have specific meanings and positions in a certain semantic space. By mapping language units to vector space, semantic vectors can capture semantic relationships between semantics, such as similarity, correlation or semantic distance.

[0108] The method of receiving the user's search language may be that the user inputs the search condition in a designated input box through the display device of the smart terminal, and the smart terminal obtains the search condition as the search language.

[0109] The method of extracting search semantics can be to identify the search language, remove stop words and other words that do not contribute much to the semantic expression, extract the targets and the relationship between the targets, and then divide the sentence into words or phrases.

[0110] The method of constructing a retrieval semantic vector can be to use a neural network model, such as a recurrent neural network, a long short-term memory network, or a convolutional neural network, to learn and encode the retrieval semantics, extract features such as semantic representation information and grammatical structure in the retrieval semantics, and determine the rules for mapping language units to the vector space based on the extracted features, so that the retrieval semantic vector corresponding to the retrieval semantics can be obtained according to the rules.

[0111] S102, determining the fine-tuning semantics of the retrieval language based on a pre-built fine-tuning model, and constructing a fine-tuning semantic vector;

[0112] The pre-built fine-tuning model may be a learning model capable of extracting fine-tuning semantics related to the semantics of the retrieval language based on the image.

[0113] The method for determining the fine-tuning semantics of the retrieval language can be to obtain an image matching the retrieval semantic vector according to the retrieval semantic vector corresponding to the retrieval language, input the image into a pre-built fine-tuning model, and use the semantic information output by the fine-tuning model as the fine-tuning semantics of the retrieval language.

[0114] Fine-tuning semantics can be a more professional semantic expression that has a higher degree of match between the retrieval semantics and the target image than the retrieval semantics.

[0115] The fine-tuning semantic vector may be constructed by obtaining the fine-tuning semantic vector corresponding to the current fine-tuning semantics according to the rule of mapping the language unit to the vector space learned when constructing the retrieval semantic vector.

[0116] S103, matching the search semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors; wherein the vector database pre-stores candidate image semantic vectors;

[0117] The preset number can be the number of images included in each search result set by the staff. Preferably, the preset number can also be set by the staff to only feed back the parts with a higher matching degree to the user in each search result. Figure 1 The number of articles is a preset number, that is, the preset number can be a fixed value or a variable, and no excessive restrictions are made here.

[0118] The target image semantic vector may be a candidate image semantic vector in the vector database that matches both the retrieval semantic vector and the fine-tuning semantic vector, that is, the image corresponding to the target image semantic vector is the user's retrieval result.

[0119] The candidate image semantic vector may be an image semantic vector obtained by extracting semantic vectors from stored images in advance, and may correspond one-to-one with the stored images. All candidate image semantic vectors may be stored in a vector database.

[0120] A preset number of target image semantic vectors can be obtained by matching all candidate image semantic vectors in a vector database with the retrieval semantic vector and the fine-tuning semantic vector respectively, and extracting a preset number of candidate image semantic vectors with a higher average matching degree with the retrieval semantic vector and the fine-tuning semantic vector from the matching results as target image semantic vectors.

[0121] S104: Acquire a target image corresponding to the target image semantic vector in the image data set, and feed back the target image to the user.

[0122] Image datasets can be libraries for storing images. The image materials can be online images uploaded by staff, original images, etc., and can also be continuously updated and expanded during subsequent use. Users can quickly locate the required images through image retrieval.

[0123] The target image may be obtained by obtaining the image material associated with the target image semantic vector as the target image based on the association relationship between the pre-established target image semantic vector and the images in the image data set, and displaying all the target images to the user through a display device.

[0124] For example, Figure 2 is a schematic diagram of a process of feeding back a target image according to retrieval semantics provided in the first embodiment of the present application, such as Figure 2 As shown, when the user inputs the search semantics, the intelligent terminal obtains the semantic vector of the search semantics through the large model, matches the fine-tuning vector similar to the semantic vector in the vector database, obtains the similarity of the semantic vectors of each candidate image through the semantic vector and the fine-tuning vector, sorts the similarities, takes the candidate image semantic vectors corresponding to a preset number of the highest similarities as the search results, obtains the pictures corresponding to the search results, and displays the obtained pictures through the display device.

[0125] Based on the above solution, optionally, before receiving the user's search language, the method further includes:

[0126] Acquire semantic information of each candidate image in the image data set according to a pre-built semantic recognition model;

[0127] Vectorizing the semantic information of each candidate image to obtain a semantic vector of the candidate image, and associating and storing the semantic vector of the candidate image and its corresponding image in a vector database;

[0128] Accordingly, obtaining a target image corresponding to the target image semantic vector in the image data set includes:

[0129] According to the candidate image semantic vectors and their corresponding images stored in the vector database, a target image corresponding to the target image semantic vector is determined.

[0130] The semantic recognition model may be a learning model that can recognize semantic information in an image.

[0131] Candidate images can be all images in the image dataset.

[0132] A semantic recognition model can be constructed by obtaining a large number of images with known semantic information, using the semantic information as labels and the corresponding images as input variables, and conducting supervised training to learn algorithms for identifying image subjects and extracting semantic information of image subjects, thereby constructing a semantic recognition model.

[0133] The semantic information of each candidate image may be obtained by inputting each candidate image into a pre-built semantic recognition model, and the semantic recognition model recognizes the semantic information corresponding to each candidate image according to a learned algorithm.

[0134] The method of obtaining the semantic vector of the candidate image may be to obtain the candidate image semantic vector corresponding to the semantic information of each candidate image according to the rule of mapping the language unit to the vector space learned when constructing the retrieval semantic vector. The obtained candidate image semantic vector is associated with the corresponding candidate image, and the candidate image semantic vector and the association relationship between the candidate image semantic vector and the corresponding candidate image are stored in the vector database.

[0135] The advantage of this arrangement of the present invention is that the candidate image semantic vectors corresponding to each candidate image can be obtained in advance and stored in the vector database, which is beneficial for the subsequent matching of the retrieval semantic vectors. The candidate image semantic vectors are used as a medium to make the retrieved images more consistent with the user's retrieval conditions.

[0136] Based on the above solution, optionally, before obtaining the semantic information of each candidate image in the image data set according to the pre-built semantic recognition model, the method further includes:

[0137] Receive uploaded images;

[0138] Filtering the image to obtain a deduplicated image;

[0139] Storing the deduplicated images in the image dataset;

[0140] and / or,

[0141] Receive uploaded videos;

[0142] Extracting images from the video, and filtering the extracted images to obtain deduplicated images;

[0143] The deduplicated images are stored in the image dataset.

[0144] The method of receiving the uploaded images may be that the staff uploads the collected image materials through the image interface of the smart terminal, and the smart terminal receives them.

[0145] The method of obtaining the deduplicated image may be to identify the subject in the image, and if there are images with the same subject, only one of the images is retained and the remaining images are deleted, and after filtering all the received images, the remaining images are deduplicated images.

[0146] The method of receiving the uploaded images may be that the staff uploads the collected video materials through the video interface of the smart terminal, and the smart terminal receives them.

[0147] The deduplicated images may be obtained by dividing the received video into frames of images, comparing the similarities of adjacent images, deleting images with greater similarity, and the remaining images are deduplicated images.

[0148] The method of storing deduplicated images in the image data set is to store the deduplicated images in the image data set in sequence according to the storage format specified in the image data set, and set a reading address for each stored image. The association between the candidate image semantic vectors stored in the vector database and the corresponding candidate images is the reading address of each candidate image in the image data set. The corresponding image material can be quickly located and extracted through the reading address.

[0149] For example, Figure 3 FIG. 1 is a schematic diagram of the process of establishing an image data set and a vector database provided in Example 1 of the present application. Figure 3 As shown in the figure, the staff uploads the collected images and videos to the image dataset, the intelligent terminal extracts the images from the videos in the image dataset, deduplicates all the images, identifies the subject information in the images and the relationship between the subjects through the large model, generates a corresponding target list and spatial information, merges the same or similar subjects in the target list, extracts the main part of the image, and then uses the large model to identify the semantic information of the main part of the image, vectorizes the semantic information, obtains the candidate image semantic vector, and stores all the candidate image semantic vectors in the vector database.

[0150] The benefit of this setting is that when the staff expands the image materials in the image data set, the images and videos uploaded by them can be deduplicated, thereby avoiding multiple identical images taking up storage space and reducing duplicate images in user search results.

[0151] In an embodiment of the present application, a user's search language is received, the search semantics of the search language is extracted, and a search semantic vector is constructed; the fine-tuning semantics of the search language is determined based on a pre-constructed fine-tuning model, and a fine-tuning semantic vector is constructed; matching is performed in a vector database according to the search semantic vector and the fine-tuning semantic vector to obtain a preset number of target image semantic vectors; wherein candidate image semantic vectors are pre-stored in the vector database; a target image corresponding to the target image semantic vector is obtained in an image data set, and the target image is fed back to the user. Through the above-mentioned image retrieval method, the objects in the search semantics and the associations between the objects can be accurately summarized through the fine-tuning vector, so that the target image can better meet the user's search conditions, which is conducive to improving the accuracy of the image retrieval system and the user's experience.

[0152] Embodiment 2

[0153] Figure 4 This is a flow chart of an image retrieval method provided in Example 2 of the present application. This solution makes a better improvement on the above-mentioned embodiment, and the specific improvement is: determining the fine-tuning semantics of the retrieval language based on a pre-built fine-tuning model, including: screening out relevant images in the image data set according to the retrieval semantic vector; inputting the relevant images into the fine-tuning model, and determining the fine-tuning semantics of the retrieval language according to the output result of the fine-tuning model.

[0154] like Figure 4 As shown, specifically including the following:

[0155] S401, receiving a user's search language, extracting search semantics of the search language, and constructing a search semantic vector;

[0156] S402, screening out relevant images in the image data set according to the retrieval semantic vector;

[0157] The method of screening related images may be to match the retrieval semantic vector in the vector database, obtain the candidate image semantic vector similar to the retrieval semantic vector, and read the image associated with the candidate image semantic vector in the image data set as the related image.

[0158] S403, inputting the relevant image into a fine-tuning model, determining the fine-tuning semantics of the retrieval language according to an output result of the fine-tuning model, and constructing a fine-tuning semantic vector;

[0159] The method for determining the fine-tuning semantics of the retrieval language can be to input the relevant images into a pre-built fine-tuning model, and the fine-tuning model extracts the semantic features of each relevant image according to the learned algorithm for identifying the semantic features of the image, obtains the common features of all the semantic features, and obtains the corresponding semantic information based on the common features as the fine-tuning semantics of the retrieval language.

[0160] S404, matching the search semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors; wherein the vector database pre-stores candidate image semantic vectors;

[0161] S405: Acquire a target image corresponding to the target image semantic vector in the image data set, and feed back the target image to the user.

[0162] The advantage of this setting of the scheme is that the fine-tuning semantics can be obtained based on the relevant images, thereby avoiding the problem that the retrieval results do not meet the retrieval conditions due to the large difference between the fine-tuning semantics and the retrieval semantics, which is conducive to improving the accuracy of obtaining the target image.

[0163] Based on the above solution, optionally, before inputting the relevant image into the fine-tuning model, the method further includes:

[0164] Acquire semantic samples, and acquire corresponding positive image samples and negative image samples in the image data set according to the semantic samples;

[0165] Unsupervised training is performed on the positive image samples and the negative image samples to obtain an algorithm for extracting semantic information that matches the positive image samples and does not match the negative image samples, so as to construct a fine-tuning model for extracting semantic information of images.

[0166] Semantic samples can be a set of predictions of retrieval semantics that users may input.

[0167] The positive image sample may be a set of images that are consistent with the semantic sample. For example, the positive image corresponding to the semantics of a bus may be an image including all observation surfaces of the bus.

[0168] Negative image samples may be a set of images opposite to the semantic samples. For example, a negative image corresponding to the semantics of a bus may be an image that does not contain any bus information.

[0169] The method for obtaining positive image samples and negative image samples is that the staff predicts the search semantics that the user may input based on the image information in the image data set, uses the prediction results as semantic samples, and the intelligent terminal searches for a certain number of positive image samples and negative image samples in the image data set.

[0170] A fine-tuning model can be constructed by selecting a suitable model framework, inputting semantic samples, positive image samples, and negative image samples into the model framework for unsupervised training, extracting image features of positive image samples and negative image samples respectively, and calculating the probability of semantics and image correlation through a loss function. The model parameters are updated according to the calculation results, so that the output of the model can change in the direction of reducing the loss. Through iteration, the model can gradually learn better semantic representations, thereby constructing a fine-tuning model that can output fine-tuned semantics.

[0171] For example, Figure 5 is a schematic diagram of the fine-tuning vector acquisition process provided in the second embodiment of the present application, such as Figure 5 As shown, after obtaining the semantic samples, positive image samples and negative image samples, the staff constructs a fine-tuning model based on the semantic samples, positive image samples and negative image samples, generates fine-tuning semantics corresponding to the retrieval semantics using the constructed fine-tuning model, vectorizes the fine-tuning semantics to obtain the corresponding fine-tuning vector, and stores the association between the fine-tuning vector and the semantic vector and the fine-tuning vector in a vector database.

[0172] The advantage of this setting of the scheme is that it can continuously train positive image samples and negative image samples to represent more accurate fine-tuning semantics, which is beneficial in the subsequent process of determining the semantic vector of the target image, making the selection conditions more in line with user needs and the retrieval results more accurate.

[0173] On the basis of the above solution, optionally, matching the retrieval semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors includes:

[0174] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vector whose first similarity is greater than a first threshold is used as the target image semantic vector;

[0175] or,

[0176] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and the candidate image semantic vector whose second similarity is greater than a second threshold is used as the target image semantic vector;

[0177] or,

[0178] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vectors whose first similarity is greater than a first threshold are extracted as first results;

[0179] Based on the first result, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and a candidate image semantic vector whose second similarity in the first result is greater than a second threshold is extracted as the target image semantic vector.

[0180] The first similarity can be calculated by the first algorithm by calculating the product of the candidate image semantic vector with the semantic vector and the fine-tuning vector respectively, obtaining the similarity between the current candidate image semantic vector and the semantic vector and the fine-tuning vector respectively, and then fusing and mapping the similarities between the current candidate image semantic vector and the semantic vector and the fine-tuning vector respectively through a specified formula, thereby obtaining the first similarity that can represent the degree of matching between the current candidate image semantic vector and the semantic vector and the fine-tuning vector.

[0181] The first threshold may be the lowest first similarity corresponding to the candidate image semantic vector having a high matching degree with the semantic vector and the fine-tuning vector.

[0182] The second similarity can be calculated by the second algorithm by obtaining a weight value of the second similarity based on the similarity between the fine-tuning vector and the candidate image semantic vector, and obtaining the second similarity of the current candidate image semantic vector based on the product of the weight value, the semantic vector and the similarity between the candidate image semantic vector.

[0183] The second threshold may be the lowest second similarity corresponding to the candidate image semantic vector having a high matching degree with both the semantic vector and the fine-tuning vector.

[0184] There are three ways to determine the semantic vector of the target image:

[0185] 1. Only the first algorithm is used to calculate the first similarity of each candidate image semantic vector, each first similarity is compared with the first threshold, and the candidate image semantic vector corresponding to the first similarity greater than the first threshold is retained as the target image semantic vector;

[0186] 2. Only the second algorithm is used to calculate the second similarity of each candidate image semantic vector, each second similarity is compared with the second threshold value, and the candidate image semantic vector corresponding to the second similarity greater than the second threshold value is retained as the target image semantic vector;

[0187] 3. First, use the first algorithm to calculate the first similarity of each candidate image semantic vector, remove the candidate image semantic vectors corresponding to the first similarity less than or equal to the first threshold, and then use the second algorithm to calculate the second similarity of the remaining candidate image semantic vectors, and retain the candidate image semantic vectors corresponding to the second similarity greater than the second threshold as the target image semantic vector.

[0188] When determining the semantic vector of the target image, the above three methods can be selected according to the requirements for the retrieval results, and no excessive restrictions are made here.

[0189] The advantage of this arrangement of the present solution is that the semantic vector of the target image that meets both the semantic vector and the fine-tuning vector can be screened out through the first algorithm and the second algorithm, thereby ensuring that the acquired target image meets the user's search criteria.

[0190] Based on the above solution, optionally, the first algorithm includes the following formula:

[0191]

[0192] Among them, S1 is the first similarity, t0 is the semantic vector, t1 is the fine-tuning vector, and i is the semantic vector of the i-th candidate image;

[0193] The second algorithm includes the following formula:

[0194]

[0195] Wherein, P is the intermediate value, t1 is the fine-tuning vector, i is the i-th candidate image semantic vector or the i-th vector in the first result; when P is greater than 0.5, its corresponding weight pro value is 1; when P is not greater than 0.5, its corresponding weight pro value is 0;

[0196] S2=pro*t0·i;

[0197] Wherein, S2 is the second similarity, t0 is the semantic vector, and i is the semantic vector of the i-th candidate image or the i-th vector in the first result.

[0198] In the above formula, t0·i represents the dot product of the semantic vector of the i-th candidate image and the semantic vector. For example, if i=(1, 2), t0=(3, 4), then t0·i=1×3+2×4=11.

[0199] t1·i represents the dot product of the semantic vector of the i-th candidate image and the fine-tuning vector. For example, if i=(1, 2), t1=(4, 5), then t1·i=1×4+2×5=14.

[0200] In the above-mentioned second algorithm, if only the second algorithm is used to determine the target image semantic vector, i represents the i-th candidate image semantic vector; if the first algorithm and the second algorithm are used to comprehensively determine the target image semantic vector, i represents the i-th vector in the first result.

[0201] This embodiment provides a program code for image retrieval using the above formula for reference:

[0202] import numpy as np

[0203] from typing import List

[0204] import torch

[0205] import torchvision.models as models

[0206] import torchvision.transforms as transforms

[0207] from PIL import Image

[0208] import os

[0209] def sigmoid(x):

[0210] return 1 / (1+np.exp(-x))

[0211] #For example, some vectors are randomly generated here

[0212] num_images=10

[0213] image_dim=5

[0214] text_dim=5

[0215] #Simulation vector database

[0216] vector_database = {}

[0217] #Simulated image dataset

[0218] image_dataset = []

[0219] #Receive the user's search language, extract the search semantics and construct the search semantic vector

[0220] def extract_retrieval_semantic_vector(query_text):

[0221] return np.random.rand(text_dim)

[0222] #Determine the fine-tuning semantics of the retrieval language based on the pre-built fine-tuning model and construct the fine-tuning semantic vector

[0223] def determine_fine_tuned_semantic_vector(query_text):

[0224] retrieval_semantic_vector=extract_retrieval_semantic_vector(query_text)

[0225] relevant_images=select_relevant_images(retrieval_semantic_vector)

[0226] fine_tuned_vector=fine_tune_model(relevant_images)

[0227] return fine_tuned_vector

[0228] #Match the candidate image semantic vector, retrieval semantic vector and fine-tuning semantic vector in the vector database to obtain a preset number of target image semantic vectors

[0229] def match_vectors(retrieval_semantic_vector,fine_tuned_semantic_vector,num_targets=5):

[0230] target_image_semantic_vectors=[]

[0231] for candidate_vector in vector_database.keys():

[0232] # Example of the first matching method

[0233] similarity_1=np.dot(retrieval_semantic_vector,candidate_vector)

[0234] if similarity_1>0.5:

[0235] target_image_semantic_vectors.append(candidate_vector)

[0236] #Example of the second matching method

[0237] #similarity_2=sigmoid(np.dot(fine_tuned_semantic_vector,candidate_vector))

[0238] #if similarity_2>0.6:

[0239] #target_image_semantic_vectors.append(candidate_vector)

[0240] if len(target_image_semantic_vectors)>=num_targets:

[0241] break

[0242] return target_image_semantic_vectors

[0243] #Get the target image corresponding to the target image semantic vector in the image dataset and feed the target image back to the user

[0244] def get_target_images(target_image_semantic_vectors):

[0245] Return[vector_database[vector]for vector intarget_image_semantic_vectors]

[0246] The advantage of such a setting of the present solution is that the first similarity and / or the second similarity of the semantic vector of the candidate image can be calculated based on a specified formula, so that the calculation of the similarity is more rigorous and the screening result of the semantic vector of the target image is more accurate.

[0247] Embodiment 3

[0248] Figure 6 It is a structural diagram of the image retrieval device provided in Example 3 of the present application.

[0249] like Figure 6 As shown, specifically including the following:

[0250] The search semantic vector construction module 601 is used to receive the search language of the user, extract the search semantics of the search language, and construct the search semantic vector;

[0251] A fine-tuning semantic vector construction module 602 is used to determine the fine-tuning semantics of the search language based on a pre-built fine-tuning model and construct a fine-tuning semantic vector;

[0252] The target image semantic vector acquisition module 603 is used to match the search semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors; wherein the vector database pre-stores candidate image semantic vectors;

[0253] The target image feedback module 604 is used to obtain a target image corresponding to the target image semantic vector in the image data set, and feed back the target image to the user.

[0254] Optionally, the fine-tuning semantic vector construction module is specifically used to:

[0255] Filtering out relevant images in the image data set according to the retrieval semantic vector;

[0256] The relevant images are input into a fine-tuning model, and the fine-tuning semantics of the retrieval language is determined according to an output result of the fine-tuning model.

[0257] Optionally, the fine-tuning semantic vector construction module is further used to:

[0258] Acquire semantic samples, and acquire corresponding positive image samples and negative image samples in the image data set according to the semantic samples;

[0259] Unsupervised training is performed on the positive image samples and the negative image samples to obtain an algorithm for extracting semantic information that matches the positive image samples and does not match the negative image samples, so as to construct a fine-tuning model for extracting semantic information of images.

[0260] Optionally, the device further comprises:

[0261] A vector database construction module is used to obtain semantic information of each candidate image in the image data set according to a pre-built semantic recognition model;

[0262] Vectorizing the semantic information of each candidate image to obtain a semantic vector of the candidate image, and associating and storing the semantic vector of the candidate image and its corresponding image in a vector database;

[0263] Accordingly, the target image feedback module specifically includes:

[0264] According to the candidate image semantic vectors and their corresponding images stored in the vector database, a target image corresponding to the target image semantic vector is determined.

[0265] Optionally, the device further comprises:

[0266] An image dataset building module for receiving uploaded images;

[0267] Filtering the image to obtain a deduplicated image;

[0268] Storing the deduplicated images in the image dataset;

[0269] and / or,

[0270] Receive uploaded videos;

[0271] Extracting images from the video, and filtering the extracted images to obtain deduplicated images;

[0272] The deduplicated images are stored in the image dataset.

[0273] Optionally, the target image semantic vector acquisition module is specifically used to:

[0274] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vector whose first similarity is greater than a first threshold is used as the target image semantic vector;

[0275] or,

[0276] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and the candidate image semantic vector whose second similarity is greater than a second threshold is used as the target image semantic vector;

[0277] or,

[0278] Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vectors whose first similarity is greater than a first threshold are extracted as first results;

[0279] Based on the first result, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and a candidate image semantic vector whose second similarity in the first result is greater than a second threshold is extracted as the target image semantic vector.

[0280] Optionally, the first algorithm includes the following formula:

[0281]

[0282] Among them, S1 is the first similarity, t0 is the semantic vector, t1 is the fine-tuning vector, and i is the semantic vector of the i-th candidate image;

[0283] The second algorithm includes the following formula:

[0284]

[0285] Wherein, P is the intermediate value, t1 is the fine-tuning vector, i is the i-th candidate image semantic vector or the i-th vector in the first result; when P is greater than 0.5, its corresponding weight pro value is 1; when P is not greater than 0.5, its corresponding weight pro value is 0;

[0286] S2=pro*t0·i;

[0287] Wherein, S2 is the second similarity, t0 is the semantic vector, and i is the semantic vector of the i-th candidate image or the i-th vector in the first result.

[0288] In an embodiment of the present application, a retrieval semantic vector construction module is used to receive a user's retrieval language, extract the retrieval semantics of the retrieval language, and construct a retrieval semantic vector; a fine-tuning semantic vector construction module is used to determine the fine-tuning semantics of the retrieval language based on a pre-built fine-tuning model, and construct a fine-tuning semantic vector; a target image semantic vector acquisition module is used to match the retrieval semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors; wherein the vector database pre-stores candidate image semantic vectors; a target image feedback module is used to obtain a target image corresponding to the target image semantic vector in an image data set, and feed the target image back to the user. Through the above-mentioned image retrieval device, the objects in the retrieval semantics and the associations between objects can be accurately summarized through fine-tuning vectors, so that the target image can better meet the user's retrieval conditions, which is conducive to improving the accuracy of the image retrieval system and the user's experience.

[0289] An image retrieval device in the embodiment of the present application may be a device, or a component, integrated circuit, or chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device may be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0290] An image retrieval device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0291] An image retrieval device provided in an embodiment of the present application can implement each process implemented by the above-mentioned method embodiments. To avoid repetition, it will not be described again here.

[0292] Embodiment 4

[0293] like Figure 7 As shown, an embodiment of the present application also provides an electronic device 700, including a processor 701, a memory 702, and a program or instruction stored in the memory 702 and executable on the processor 701. When the program or instruction is executed by the processor 701, each process of the above-mentioned image retrieval method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0294] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0295] Embodiment 5

[0296] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image retrieval method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0297] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0298] Embodiment 6

[0299] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image retrieval method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0300] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0301] It should be noted that, in this article, the term "comprises", "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0302] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0303] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0304] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. An image retrieval method, characterized in that: The method comprises: Receiving a user's search language, extracting search semantics of the search language, and constructing a search semantic vector; Determining the fine-tuning semantics of the retrieval language based on a pre-built fine-tuning model, and constructing a fine-tuning semantic vector; Matching the retrieval semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors; wherein the vector database pre-stores candidate image semantic vectors; A target image corresponding to the target image semantic vector is obtained in the image data set, and the target image is fed back to the user.

2. The image retrieval method according to claim 1, characterized in that: Determining the fine-tuning semantics of the retrieval language based on a pre-built fine-tuning model includes: Filtering out relevant images in the image data set according to the retrieval semantic vector; The relevant images are input into a fine-tuning model, and the fine-tuning semantics of the retrieval language is determined according to an output result of the fine-tuning model.

3. The image retrieval method according to claim 2, characterized in that: Before inputting the relevant image into the fine-tuning model, the method further includes: Acquire semantic samples, and acquire corresponding positive image samples and negative image samples in the image data set according to the semantic samples; Unsupervised training is performed on the positive image samples and the negative image samples to obtain an algorithm for extracting semantic information that matches the positive image samples and does not match the negative image samples, so as to construct a fine-tuning model for extracting semantic information of images.

4. The image retrieval method according to claim 1, characterized in that: Before receiving the user's search language, the method further includes: Acquire semantic information of each candidate image in the image data set according to a pre-built semantic recognition model; Vectorizing the semantic information of each candidate image to obtain a semantic vector of the candidate image, and associating and storing the semantic vector of the candidate image and its corresponding image in a vector database; Accordingly, obtaining a target image corresponding to the target image semantic vector in the image data set includes: According to the candidate image semantic vectors and their corresponding images stored in the vector database, a target image corresponding to the target image semantic vector is determined.

5. The image retrieval method according to claim 4, characterized in that: Before obtaining semantic information of each candidate image in the image data set according to the pre-built semantic recognition model, the method further includes: Receive uploaded images; Filtering the image to obtain a deduplicated image; Storing the deduplicated images in the image dataset; and / or, Receive uploaded videos; Extracting images from the video, and filtering the extracted images to obtain deduplicated images; The deduplicated images are stored in the image dataset.

6. The image retrieval method according to claim 2, characterized in that: Matching the retrieval semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors includes: Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vector whose first similarity is greater than a first threshold is used as the target image semantic vector; or, Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and the candidate image semantic vector whose second similarity is greater than a second threshold is used as the target image semantic vector; or, Based on the candidate image semantic vector, the semantic vector and the fine-tuning vector, a first similarity is calculated by a first algorithm, and the candidate image semantic vectors whose first similarity is greater than a first threshold are extracted as a first result; Based on the first result, the semantic vector and the fine-tuning vector, a second similarity is calculated by a second algorithm, and a candidate image semantic vector whose second similarity in the first result is greater than a second threshold is extracted as the target image semantic vector.

7. The image retrieval method according to claim 6, characterized in that: The first algorithm includes the following formula: Among them, S1 is the first similarity, t0 is the semantic vector, t1 is the fine-tuning vector, and i is the semantic vector of the i-th candidate image; The second algorithm includes the following formula: Wherein, P is the intermediate value, t1 is the fine-tuning vector, i is the i-th candidate image semantic vector or the i-th vector in the first result; when P is greater than 0.5, its corresponding weight pro value is 1; when P is not greater than 0.5, its corresponding weight pro value is 0; S2=pro*t0·i; Wherein, S2 is the second similarity, t0 is the semantic vector, and i is the semantic vector of the i-th candidate image or the i-th vector in the first result.

8. An image retrieval device, characterized in that: The device comprises: A search semantic vector construction module is used to receive a user's search language, extract the search semantics of the search language, and construct a search semantic vector; A fine-tuning semantic vector construction module, used to determine the fine-tuning semantics of the retrieval language based on a pre-built fine-tuning model and to construct a fine-tuning semantic vector; A target image semantic vector acquisition module, used for matching the retrieval semantic vector and the fine-tuning semantic vector in a vector database to obtain a preset number of target image semantic vectors; wherein the vector database pre-stores candidate image semantic vectors; The target image feedback module is used to obtain a target image corresponding to the target image semantic vector in the image data set, and feed back the target image to the user.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of an image retrieval method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of an image retrieval method as described in any one of claims 1 to 7 are implemented.

Images