Label determination method and device

Abstract

The invention discloses a label determination method and device, and belongs to the technical field of image processing. The method comprises the following steps: in response to a first input for a first image in an image set, determining a first label of the first image; determining the image similarity between the first image and a second image according to the characteristic information of the first image and the characteristic information of the second image in the image set; and determining a second label of the second image according to the image similarity and the first label.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a label determination method and device. Background Art

[0002] Nowadays, in image collections of devices such as mobile phones and computers, each image in the image collection is usually labeled to facilitate classification, search and batch management of images in the image collection.

[0003] However, the labels that the device automatically sets for images are usually relatively simple. For example, according to the type of image, it will be labeled with plants, vegetables, clothing, and buildings. This type of labeling cannot support users' more detailed search and management. If users want to add more specific personalized labels, they need to manually label each image. This labeling method is too complicated and costly. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a label determination method and device that can solve the technical problem of high cost in the existing method of labeling images.

[0005] In a first aspect, an embodiment of the present application provides a tag determination method, the method comprising:

[0006] In response to a first input for a first image in the image collection, determining a first label for the first image;

[0007] determining, according to characteristic information of the first image and characteristic information of a second image in the image set, an image similarity between the first image and the second image;

[0008] A second label for the second image is determined according to the image similarity and the first label.

[0009] In a second aspect, an embodiment of the present application provides a label determination device, the device comprising:

[0010] A first determining module, configured to determine a first label of a first image in an image set in response to a first input for the first image;

[0011] a second determining module, configured to determine an image similarity between the first image and the second image according to characteristic information of the first image and characteristic information of a second image in the image set;

[0012] A third determining module is used to determine a second label of the second image according to the image similarity and the first label.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method provided in the first aspect are implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method provided in the first aspect are implemented.

[0015] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method provided in the first aspect.

[0016] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method provided in the first aspect.

[0017] In the label determination method and device of the present application, a first label of the first image can be determined in response to a first input for the first image in the image set; the image similarity between the first image and the second image in the image set can be determined; and the second label of the second image can be determined based on the image similarity and the first label. In this way, the user can manually label only a part of the images in the image set, and then the electronic device can automatically propagate the label of the first image to the similar second image based on the image similarity by calculating the image similarity between different images in the image set, thereby reducing the workload of manual labeling and the cost of image labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flowchart of a tag determination method provided by an embodiment of the present application;

[0019] Figure 2 This is one of the interface display schematic diagrams of an electronic device provided by an embodiment of the present application;

[0020] Figure 3 This is a second schematic diagram of an interface display of an electronic device provided by an embodiment of the present application;

[0021] Figure 4 This is a third schematic diagram of an interface display of an electronic device provided by an embodiment of the present application;

[0022] Figure 5 This is a fourth schematic diagram of an interface display of an electronic device provided by an embodiment of the present application;

[0023] Figure 6 This is a fifth schematic diagram of an interface display of an electronic device provided by an embodiment of the present application;

[0024] Figure 7 is a structural schematic diagram of a label determination device provided by another embodiment of the present application;

[0025] Figure 8 is a structural schematic diagram of an electronic device provided by another embodiment of the present application;

[0026] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0028] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0029] In order to solve the above technical problems, the present application provides a tag determination method. The tag determination method provided by the embodiment of the present application is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0030] like Figure 1 As shown, Figure 1 : is a flowchart of a tag determination method provided by an embodiment of the present application. The embodiment of the present application provides a tag determination method, which may include:

[0031] S101, in response to a first input for a first image in an image set, determining a first label of the first image;

[0032] In this embodiment, the first image may be any image in the image set, and the first image may be an image with a label or an image without a label. The user may select the first image in the image set in the electronic device, and then make a first input to the first image, and determine the first label of the first image through the first input. For example, the user may input the label "beach" for the first image through the first input, and the electronic device will use "beach" as the first label of the first image. The image set is a group of pictures that are uniformly managed in the electronic device. For example, the image set may be an album application in the electronic device; it may also be an image collection application in the file manager of the electronic device.

[0033] Specifically, the first input may be a touch input of the user to the electronic device, for example, the touch input may be a touch click operation, or a sliding operation, or other operations. The first input may also be an input operation performed by the user with the help of other input devices such as a mouse, a remote control device, a keyboard, etc. connected to the first electronic device. Optionally, the first input may indicate a control in the notification message. Unless otherwise specified, the input to the electronic device in the following description may be a touch click operation, a sliding operation, or other operations, and may also be an input operation performed by the user with the help of other input devices such as a mouse, a remote control device, a keyboard, etc. connected to the electronic device.

[0034] In some embodiments, the first input includes a first sub-input and a second sub-input;

[0035] The S101 includes:

[0036] displaying the first image;

[0037] In response to a first sub-input of the first image, displaying a label edit control of the first image;

[0038] In response to a second sub-input to the label editing control, a phrase corresponding to the second sub-input is determined as the first label.

[0039] In this embodiment, the first input can be divided into a first sub-input and a second sub-input. Before the user enters the first input for the first image in the image set, the user can first open the image set and select the image identifier of the first image in the image set. In this way, the first image will be displayed in the image set.

[0040] After the first image is displayed in the image set of the electronic device, the user can perform a first sub-input on the first image, and the first sub-input is used to enter the label editing page of the first image, and the label editing page includes at least one label editing control. For example, the first sub-input can be an input of double-clicking the first image, and the first sub-input can also be an input of long pressing the first image for 3 seconds. In response to the first sub-input, the electronic device displays at least one label editing control in the label editing page.

[0041] After the label editing control is displayed in the electronic device, the user can input the label that he wants to put on the first image in the form of a phrase into the label editing control, which is the second sub-input. In response to the second sub-input, the electronic device adds the phrase of the user's second sub-input as the first label to the first image in the image collection.

[0042] As an optional implementation, Figure 2 The first image 201 is displayed, and the user can perform a first sub-input on the first image 201, and then the display interface of the electronic device will display the following Figure 3 In the tag editing interface shown, the electronic device generates and displays a thumbnail 301 of the first image, and displays three tag input controls 302, tag one, tag two, and tag three, as well as a confirmation control 303 and a cancel control 304 on the tag editing interface. The user can enter a phrase in at least one tag input control through the second sub-input, and enter the confirmation control, so that the phrase entered in each tag input control can be determined as a first tag. If the user enters the cancel control, the display interface of the electronic device will switch back to Figure 2 The first image is shown.

[0043] As another optional implementation, Figure 4 The first image is displayed, and a label editing control 401 is also displayed on the first image. The user can make a first sub-input to the label editing control, and the display interface of the electronic device will also jump to the following figure: Figure 3 The label editing interface shown.

[0044] In this way, users can manually label images in an image collection, so that users can precisely control the label content and ensure that the label is highly relevant to the image content, thereby improving the accuracy of photo classification and retrieval.

[0045] In some embodiments, the S101 includes:

[0046] Determine N third elements included in the first image, where N is a positive integer greater than 1;

[0047] Determine P fourth elements from the N third elements, where P is a positive integer less than or equal to N, and the fourth elements are elements including pixels greater than a pixel threshold;

[0048] A first tag corresponding to each of the fourth elements is determined based on the content of the fourth element.

[0049] In this embodiment, the elements in the image refer to independent objects or significant areas that can be identified in the image, such as people, animals, plants, objects, etc. After acquiring the first image, the electronic device can identify the N third elements contained in the image through image recognition technology; then, among the N third elements, further based on the number of pixels contained in each element, the P fourth elements with larger areas in the first image are screened out, and then the electronic device can identify the main content of the P fourth elements, and mark each fourth element with a corresponding first label based on its main content. Among them, the first label of the P fourth elements is also the first label of the first image.

[0050] For example, Figure 5 As shown, N is 5, P is 3, Figure 5 In the first image 501 displayed, in response to the first input for the first image in the image set, the electronic device can use a preset image recognition model to extract the content in the first image, identify and detect that the first image includes 5 third elements, and then the electronic device can sort the 5 third elements from large to small according to the proportion of the area of ​​the first image or the number of pixels included, and select the 3 fourth elements with the largest area proportion or the largest number of pixels: beans, corn and peppers, and display a main frame around each fourth element that wraps the fourth element. Then, based on the content of the 3 fourth elements, the electronic device can label beans, corn and peppers with the first label of "vegetables".

[0051] In addition, after the electronic device automatically tags each fourth element with a first label, the user can also modify the content of the first label of any fourth element. Figure 6 As shown, after the electronic device labels the three fourth elements, beans 603, corn 602, and pepper 601, with the first label of "vegetables", the user can change the first label of pepper from "vegetables" to "favorite food". When the user edits the first label, the main frame corresponding to the first label will be highlighted, prompting the user which part of the label content of the element is being modified.

[0052] In the above manner, electronic devices can accurately identify and mark important elements that occupy significant areas in an image, thereby improving labeling efficiency.

[0053] S102, determining an image similarity between the first image and the second image according to the characteristic information of the first image and the characteristic information of the second image in the image set;

[0054] In this embodiment, after adding a first label to a first image through a first input, any unlabeled second image in the image set can be obtained, and the image similarity between the first image and the second image is calculated based on the characteristic information of the first image and the characteristic information of the second image, and whether to add a label to the second image is determined based on the image similarity. The characteristic information refers to the key data or features in the image used to describe and distinguish its content, and the characteristic information may include the vector of the image, the color histogram of the image, the texture of the image, and the key points of the image.

[0055] For example, when the characteristic information is a vector of an image, the cosine similarity between the first image and the second image can be calculated based on the vector of the first image and the vector of the second image. Specifically, the feature vector A of the first image and the feature vector B of the second image in the image set can be extracted, and then the cosine similarity between the two vectors A and B is calculated by the following cosine similarity formula as the image similarity between the two images.

[0056] The cosine similarity formula is as follows:

[0057]

[0058] Wherein, Similarity is the image similarity, which is any constant between 0 and 1. The larger the Similarity is, the more similar the first image and the second image are. A represents the vector of the first image, B represents the vector of the second image, and A·B represents the dot product of vector A and vector B.

[0059] S103: Determine a second label for the second image according to the image similarity and the first label.

[0060] In the embodiment of the present application, after determining the first label of the first image and the image similarity between the first image and the second image, the second label of the second image can be determined according to the image similarity and the first label. Since the higher the image similarity, the closer the contents of the first image and the second image are, when the image similarity is high, it means that the contents of the first image and the second image are not much different, and the first label of the first image can be used to mark the second image, and the first label can be used as the second label of the second image at the same time.

[0061] In an embodiment of the present application, a first label of the first image can be determined in response to a first input for the first image in the image set; the image similarity between the first image and the second image can be determined based on the vector of the first image and the vector of the second image in the image set; and the second label of the second image can be determined based on the image similarity and the first label. In this way, the user can manually label only a portion of the images in the image set, and then the electronic device can automatically propagate the label of the first image to the similar second image based on the image similarity by calculating the image similarity between different images in the image set, thereby reducing the workload of manual labeling and the cost of image labeling.

[0062] In some embodiments, the image similarity includes a first similarity, and the first similarity is an overall similarity between the first image and the second image;

[0063] In the case where the second label is determined according to the first similarity in the image similarities and the first label of the first image, the S103 includes:

[0064] When a first similarity between the first image and the second image is greater than or equal to a first similarity threshold, the first tag is determined as a second tag.

[0065] In this embodiment, the first image can be converted as a whole into a first vector, and the second image can be converted as a whole into a second vector, and then the cosine similarity between the first vector and the second vector is calculated as the first similarity between the first image and the second image, and then the first similarity is compared with a pre-set first similarity threshold.

[0066] If the first similarity is greater than or equal to the first similarity threshold, it can be considered that the second image has a higher similarity to the first image as a whole, and all first labels on the first image can be marked on the second image; if the first similarity is less than the first similarity threshold, it can be considered that the second image has a lower similarity to the first image as a whole, and the first label cannot be added to the second image based on the first similarity.

[0067] For example, the first vector of the first image is [0.8, 0.2, 0.3]. If the second vector of a second image "Photo 1" is [0.1, -0.2, 0.2], the vector of another second image "Photo 2" is [0.9, 0.3, 0.8]. The first similarity threshold can be set to 0.8, and the cosine similarity between the first image and "Photo 1" can be calculated to be 0.23, which does not exceed the set first similarity threshold, and no marking is performed. The cosine similarity between the first image and "Photo 2" is 0.93, which exceeds the set first similarity threshold of 0.8, then the system automatically marks the first label of the first image to the second image "Photo 2".

[0068] In the above manner, there is no need for the user to label each image in the image collection. Labels can be automatically propagated in the image collection based on the overall similarity between the images in the image collection, thereby improving the coverage of the labels.

[0069] In some embodiments, the image similarity includes a second similarity, the second similarity being a similarity between an element in the first image and an element in the second image;

[0070] In the case where the second label is determined according to the second similarity in the image similarities and the first label of the first image, S103 includes:

[0071] When a first similarity between the first image and the second image is less than a first similarity threshold and greater than or equal to a second similarity threshold, determine T first elements in the first image and T second elements in the second image, where each first element corresponds to a first label, and T is a positive integer;

[0072] When the second similarity between the i-th first element and the j-th second element is greater than a third similarity threshold, the first label corresponding to the i-th first element is determined as the second label of the j-th second element, where i and j are both positive integers less than or equal to T.

[0073] In this embodiment, if the first similarity is less than the first similarity threshold, it can be considered that the second image is relatively less similar to the first image as a whole, and the first label cannot be added to the second image based on the first similarity.

[0074] A second similarity threshold may be preset, the second similarity threshold being less than the first similarity threshold. If the first similarity is less than the second similarity threshold, it may be considered that the similarity between the first image and the second image is extremely low, and the second image cannot be marked. If the first similarity is less than the first similarity threshold but greater than or equal to the second similarity threshold, it may be considered that the overall similarity between the first image and the second image is moderate, and the similarity between the elements in the first image and the elements in the second image may be further compared.

[0075] Specifically, if the first similarity between the first image and the second image is less than the first similarity threshold and greater than or equal to the second similarity threshold, the content in the first image can be extracted, and the T first elements included in the first image can be identified and detected, and the T second elements included in the second image can be identified and detected in the same way. For example, the first T elements with the most pixels included can be selected from all elements in the first image as the first elements, and the first T elements with the most pixels included can be selected from all elements in the second element as the second elements.

[0076] Then, the second similarity between each first element and each second element is compared respectively, and a third similarity threshold is set in advance. If the second similarity between the i-th first element and the j-th second element is greater than the third similarity threshold, then it can be considered that the similarity between the i-th first element and the j-th second element is high, and the first label corresponding to the i-th first element is added to the j-th second element; if there is no second similarity between the first element and the second element greater than the third similarity threshold, no label may be added to the second image.

[0077] In the above way, refined label propagation can be achieved, that is, when the overall similarity between images is low but still within a certain range, more detailed label propagation can be achieved by comparing the similarities of specific elements in the images.

[0078] In some embodiments, after S101, the method further includes:

[0079] In response to a first input of a first image in the image set by a user, determining a first label weight corresponding to a first label of the first image;

[0080] Constructing a first mapping relationship among the first image, the first label, and the first label weight, and generating a first posting record based on the first mapping relationship;

[0081] A label library of the image set is constructed according to the first posting record.

[0082] In this embodiment, after receiving the first input of the user for the first image, not only the first label of the first image can be determined, but also the first label weight corresponding to the first label can be determined. The first label weight is used to characterize the correlation between the first label and the first image.

[0083] After determining the first label and the first label weight, a first mapping relationship between the first image, the first label and the first label weight can be constructed, and a first posting record can be generated based on the first mapping relationship. Then, all posting records can be stored in a pre-constructed empty database, thereby generating a label library for the image set.

[0084] In this embodiment, the inverted record is a data structure used to quickly find all images corresponding to a certain tag and their weights. When the user adds tags to images in the image collection, the electronic device generates inverted records based on these tags and tag weights, and creates an inverted index of the tags. The inverted index is a data structure used to map tags to a list of images with these tags. Then the user can quickly find images containing the tag in the image collection by inverting the tag.

[0085] In some embodiments, after determining the first label weight corresponding to the first label of the first image, the method further includes:

[0086] Determine the product of the first label weight and the image similarity as the second label weight corresponding to the second label in the second image;

[0087] Constructing a second mapping relationship among the second image, the second label, and the second label weight, and generating a second posting record based on the second mapping relationship;

[0088] The second posting record is added to a tag library of the image collection.

[0089] In this embodiment, the first label on the first image is propagated to the second image, and the product of the image similarity between the first image and the second image and the first label weight can also be used as the second label weight. Then, similarly, a second mapping relationship between the second image, the second label and the second label weight can be constructed to generate a second posting record, and then the second posting record is stored in the label library.

[0090] As an optional embodiment, since the first label weight is used to characterize the correlation between the first label and the first image, and the first label is input by the user, it can be considered that the correlation between the first label and the first image is very high, so the first label weight can be determined to be 1. Taking the first label as "beach" as an example, the first label weight of "beach" on the first image is 1. If the image similarity between the first image and the second image is 0.93, the second label weight is 0.93. The first inverted record is "first image--beach--1", and the second inverted record is "second image--beach--0.93".

[0091] In this way, the weight of the label in the second image can be automatically determined by multiplying the image similarity and the weight of the first label, so that it can more accurately reflect the relevance of the label in different images. At the same time, more inverted records are automatically generated and updated to ensure that the label library can be quickly updated and respond to retrieval requests, thereby improving the search efficiency of the system.

[0092] In some embodiments, after determining the second label of the second image according to the image similarity and the first label, the method further includes:

[0093] Receiving search intention information input by the user;

[0094] Searching the image set according to the search intent information to obtain a plurality of third images in the image set;

[0095] The plurality of third images are displayed.

[0096] In this embodiment, the retrieval intention information is used to characterize the user's retrieval demand for images in the image collection. The retrieval intention information can be voice information, text information, or action information. The electronic device can search the image collection according to the retrieval intention information input by the user, and find multiple third images that match the retrieval intention information. Then, the multiple third images are displayed.

[0097] In this way, users can efficiently search through image collections and quickly find images of interest.

[0098] In some embodiments, searching the image set according to the search intent information to obtain a plurality of third images in the image set includes:

[0099] Determine the search text corresponding to the search intent information;

[0100] In the case where there is a third tag matching the search text in the image collection, searching the image collection for a third image corresponding to the third tag;

[0101] When there is no third label matching the search text in the image set, multiple third images in the image set are obtained by at least one of vector recall and keyword recall:

[0102] The keyword recall includes: determining a third tag matching the keyword in the search text, and retrieving a third image corresponding to the third tag;

[0103] The vector recall is as follows: obtaining an image vector converted from each image in the image set and a text vector converted from the retrieved text, retrieving a first image vector whose third similarity with the text vector is greater than a fourth similarity threshold, and determining the image corresponding to the first image vector as the third image.

[0104] In this embodiment, if the search intention information is text information, the search intention information can be directly determined as a search text. If the search intention information is other forms of information such as voice information, the search intention information can be converted into a search text in text form.

[0105] Then, all the tags in the tag library of the image collection can be traversed to determine whether there is a tag that matches the search text. If there is a third tag that matches the search text, all images corresponding to the third tag can be used as the recalled third images. If the search text is a phrase, and there is a phrase that is completely consistent with the search text or a synonym of the search text in the tag, then it can be considered that the tag matches the search text. Otherwise, it can be considered that there is no third tag that matches the search text.

[0106] If there is no third tag matching the search text in the image collection, the electronic device may search for the third image in the image collection by at least one of vector search and keyword search.

[0107] In the process of keyword recall, keyword extraction technology can be used to extract keywords from the search text. For example, if the search text can be "find photos of Yangyang running on the beach", the keywords "Yangyang", "beach" and "running" in the search text can be extracted. Then these keywords are input into the search library for reverse search. When there is a third tag matching any keyword, all images corresponding to the tag can be recalled as the third image.

[0108] In addition, there may be a pre-set vector library in the image collection, which includes image vectors converted from each image in the image collection. In the process of vector recall, the search text may be converted into a text vector, and the dimensions of the text vector and the image vector may be made the same. Then, the text vector may be matched with each image vector respectively to obtain the third similarity between the text vector and each image vector. If there is a third similarity between the image vector and the text vector that is greater than a pre-set fourth similarity threshold, the image corresponding to the image vector may be recalled as the third image.

[0109] The above method combines label matching, keyword recall and vector recall to ensure that the images required by users can be found accurately. Even if there is no label that completely matches the search text, the system can still find relevant images through keyword and vector matching, which increases the flexibility of search.

[0110] In some embodiments, displaying the plurality of third images comprises:

[0111] Determine the first label quantity of the third label corresponding to each of the third images, and the first value corresponding to each of the third images; wherein, when the image vector corresponding to the third image matches the text vector, the first value corresponding to the third image is 1, and when the image vector corresponding to the third image does not match the text vector, the first value corresponding to the third image is 0;

[0112] Determine the sum of the first label quantity and the first value of each of the third images as the recall number of the third image;

[0113] Sorting the plurality of third images according to the recall times of the third images from large to small to obtain a sorting result;

[0114] The plurality of third images are displayed according to the sorting result.

[0115] In this embodiment, after a plurality of third images are retrieved through at least one of keyword recall and vector recall, the plurality of third images may be displayed. In order to allow the user to find the desired image more accurately and quickly, the display order of the plurality of third images may be arranged based on the relevance between each third image and the search text.

[0116] Specifically, since the search text may include multiple keywords, each keyword will be searched separately in the tag library. For any third image, the number of keywords matched by its corresponding tag and in the search text is the first tag number of the third tag corresponding to the third image, that is, the first recall number of the third image through keyword retrieval. For example, if the third image can be recalled by three keywords in the search text respectively, then the first tag number of the third tag corresponding to the third image is 3, and the first recall number of the third image through keyword retrieval is 3.

[0117] In addition, for any third image, if the third similarity between the image vector of the third image and the text vector of the search text is greater than a preset fourth similarity threshold, the third image can be recalled through vector recall. Then the first value corresponding to the third image is 1, otherwise the first value corresponding to the third image is 0.

[0118] Therefore, the sum of the first label quantity and the first numerical value of the third image is the total recall number of the third image; after determining the total recall number of each third image, the multiple third images can be sorted and displayed in descending order of the recall number.

[0119] For example, taking the search text "Yangyang running on the beach" as an example, three keywords "Yangyang", "Beach" and "Running" can be extracted through keyword extraction technology. If the keyword "Yangyang" recalls photos 1 and 3. The keyword "Beach" recalls "Photo 5", and the keyword "Running" recalls "Photo 1". The text vector recalls "Photo 3, Photo 1". Then the recall count of Photo 1 is 3 times, the recall count of Photo 3 is 2 times, and the recall count of Photo 5 is 1 time, so the order of the photos is "Photo 1, Photo 3, Photo 5".

[0120] In this embodiment, a greater number of recall times means a higher relevance between the third image and the search text, and thus by sorting based on the number of recall times, third images with a higher relevance can be displayed at a front position.

[0121] In some embodiments, the method further comprises:

[0122] In a case where the plurality of third images include K fourth images with the same number of recalls, determining a recall mode for each of the fourth images, where K is a positive integer;

[0123] In a case where the recall method includes keyword recall, determining M third tags corresponding to the fourth image and M third tag weights corresponding to the M third tags, and determining a sum of the M third tag weights as a first score of the fourth image, where M is a positive integer;

[0124] In a case where the recall method includes vector recall, determining a vector similarity between the second image vector of the fourth image conversion and the text vector as a second score;

[0125] determining a sum of the first score and the second score of each of the fourth images as a total score;

[0126] The K fourth images are sorted in descending order according to the total scores.

[0127] In this embodiment, if the recall times of each third image in the plurality of third images are different, then the plurality of third images can be sorted in descending order of the recall times. If there are K fourth images with the same recall times among the plurality of third images, then each fourth image needs to be scored to obtain a total score corresponding to each fourth image, and then the K fourth images are sorted in descending order of the total scores.

[0128] Specifically, for any fourth image, if the recall method of the fourth image includes keyword recall, then M third tags among the tags included in the fourth image that match the keyword can be determined, and the sum of the M third tags can be determined as the first score of the fourth image.

[0129] In addition, for any fourth image, if the recall method of the fourth image includes vector recall, the vector similarity between the second image vector converted from the fourth image and the text vector can be directly determined as the second score, and the sum of the first score and the second score is the total score.

[0130] For example, there are two fourth images, photo 1 and photo 2, where the search text is "the beach at the seaside", then the keywords include "the beach and the beach", and photo 1 is recalled by the keywords "the beach" and "the beach", where the label weight corresponding to "the beach" is 1 and the label weight corresponding to "the beach" is 0.5, then the first score corresponding to photo 1 is 1.5, and since the second score of the first photo is 0, the total score of photo 1 is 1.5. Photo 2 is only recalled by the keyword "the beach", and its corresponding label weight is 0.8. In addition, photo 2 can also be recalled by vector, and the vector similarity between the second image vector corresponding to photo 2 and the text vector is 0.8, then the first score of photo 2 is 0.8, the second score is also 0.8, and the total score of photo 2 is 1.6, so the sorting method is "photo 2, photo 1".

[0131] In this way, when the recall times are the same, the relevance between the image and the retrieved text can be characterized based on the image score, and the images can be sorted based on the score, so that images with high relevance can be displayed in a higher position.

[0132] Figure 7 is a schematic diagram of the structure of a label determination device provided by another embodiment of the present application. Figure 7 As shown, the label determination device may include:

[0133] A first determining module 701, configured to determine a first label of a first image in an image set in response to a first input of the first image;

[0134] A second determination module 702, configured to determine the image similarity between the first image and the second image according to the characteristic information of the first image and the characteristic information of the second image in the image set;

[0135] The third determining module 703 is configured to determine a second label for the second image according to the image similarity and the first label.

[0136] In the present application, a first label of the first image can be determined in response to a first input for the first image in the image set; the image similarity between the first image and the second image can be determined based on the vector of the first image and the vector of the second image in the image set; and the second label of the second image can be determined based on the image similarity and the first label. In this way, the user can manually label only a portion of the images in the image set, and then the electronic device can automatically propagate the label of the first image to the similar second image based on the image similarity by calculating the image similarity between different images in the image set, thereby reducing the workload of manual labeling and the cost of image labeling.

[0137] In another optional example, the label determination device further includes:

[0138] A fourth determination module, configured to determine a first label weight corresponding to a first label of a first image in the image set in response to a first input of the user to the first image;

[0139] A first generating module, configured to construct a first mapping relationship among the first image, the first label, and the first label weight, and generate a first inverted record based on the first mapping relationship;

[0140] The first construction module is used to construct a label library of the image set according to the first posting record.

[0141] In another optional example, the label determination device further includes:

[0142] a fifth determining module, configured to determine a product of the first label weight and the image similarity as a second label weight corresponding to the second label in the second image;

[0143] a second generating module, configured to construct a second mapping relationship between the second image, the second label, and the second label weight, and generate a second posting record based on the second mapping relationship;

[0144] An adding module is used to add the second posting record to the tag library of the image collection.

[0145] In another optional example, the image similarity includes a first similarity, where the first similarity is an overall similarity between the first image and the second image;

[0146] The third determining module 703 further includes:

[0147] The first determining unit is configured to determine the first label as the second label of the second image when a first similarity between the first image and the second image is greater than or equal to a first similarity threshold.

[0148] In another optional example, the image similarity includes a second similarity, where the second similarity is a similarity between an element in the first image and an element in the second image;

[0149] The third determining module 703 further includes:

[0150] a second determining unit, configured to determine, when a first similarity between the first image and the second image is less than a first similarity threshold and greater than or equal to a second similarity threshold, T first elements in the first image and T second elements in the second image, wherein each of the first elements corresponds to a first label and T is a positive integer;

[0151] A third determination unit is used to determine the first label corresponding to the i-th first element as the second label of the j-th second element when the second similarity between the i-th first element and the j-th second element is greater than a third similarity threshold, where i and j are both positive integers less than or equal to T.

[0152] In another optional example, the first determining module 701 further includes:

[0153] A first display unit, configured to display the first image;

[0154] a second display unit, configured to display a label editing control of the first image in response to a first sub-input to the first image;

[0155] a fourth determining unit, configured to determine, in response to a second sub-input to the tag editing control, a phrase corresponding to the second sub-input as the first tag;

[0156] In another optional example, the first determining module 701 further includes:

[0157] a fifth determining unit, configured to determine N third elements included in the first image, where N is a positive integer greater than 1;

[0158] a sixth determining unit, configured to determine P fourth elements from the N third elements, wherein P is a positive integer less than or equal to N, and the fourth elements are elements including pixels greater than a pixel threshold;

[0159] The seventh determining unit is used to determine the first label corresponding to each of the fourth elements based on the content of the fourth elements.

[0160] In another optional example, the label determination device further includes:

[0161] A receiving module, used for receiving search intention information input by a user;

[0162] A retrieval module, configured to search the image set according to the retrieval intention information to obtain a plurality of third images in the image set;

[0163] The display module is used to display the plurality of third images.

[0164] In another optional example, the retrieval module further includes:

[0165] An eighth determination unit, configured to determine a search text corresponding to the search intention information;

[0166] a retrieval unit, configured to retrieve a third image corresponding to the third tag in the image set if there is a third tag matching the search text in the image set;

[0167] A recall unit is used to obtain a plurality of third images in the image set by at least one of vector recall and keyword recall when there is no third label matching the search text in the image set:

[0168] The keyword recall includes: determining a third tag matching the keyword in the search text, and retrieving a third image corresponding to the third tag;

[0169] The vector recall is as follows: obtaining an image vector converted from each image in the image set and a text vector converted from the retrieved text, retrieving a first image vector whose third similarity with the text vector is greater than a fourth similarity threshold, and determining the image corresponding to the first image vector as the third image.

[0170] In another optional example, the retrieval module is further used to determine the first label quantity of the third label corresponding to each of the third images, and the first numerical value corresponding to each of the third images; wherein, when the image vector corresponding to the third image matches the text vector, the first numerical value corresponding to the third image is 1, and when the image vector corresponding to the third image does not match the text vector, the first numerical value corresponding to the third image is 0;

[0171] Determine the sum of the first label quantity and the first value of each of the third images as the recall number of the third image;

[0172] Sorting the plurality of third images according to the recall times of the third images from large to small to obtain a sorting result;

[0173] The plurality of third images are displayed according to the sorting result.

[0174] In another optional example, the label determination device is further used for:

[0175] In a case where the plurality of third images include K fourth images with the same number of recalls, determining a recall mode for each of the fourth images, where K is a positive integer;

[0176] In a case where the recall method includes keyword recall, determining M third tags corresponding to the fourth image and M third tag weights corresponding to the M third tags, and determining a sum of the M third tag weights as a first score of the fourth image, where M is a positive integer;

[0177] In a case where the recall method includes vector recall, determining a vector similarity between the second image vector of the fourth image conversion and the text vector as a second score;

[0178] determining a sum of the first score and the second score of each of the fourth images as a total score;

[0179] The K fourth images are sorted in descending order according to the total scores.

[0180] The label determination device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0181] The tag determination device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0182] The label determination device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0183] Alternatively, if Figure 8 As shown, an embodiment of the present application also provides an electronic device 100, including a processor 110, a memory 119, and a program or instruction stored in the memory 119 and executable on the processor 110. When the program or instruction is executed by the processor 110, each process of the above-mentioned label determination method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0184] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0185] Please refer to Fig. 9 , Fig. 9 The hardware structure diagram of an electronic device for implementing an embodiment of the present application is shown in FIG. The electronic device 100 includes, but is not limited to, a radio frequency unit 121, a network module 122, an audio output unit 123, an input unit 124, a sensor 125, a display unit 126, a user input unit 127, an interface unit 128, a memory 129, and a processor 120.

[0186] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 120 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig. 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0187] The processor 120 is configured to determine a first label of a first image in the image set in response to a first input for the first image;

[0188] The processor 120 is configured to determine an image similarity between the first image and the second image according to the characteristic information of the first image and the characteristic information of the second image in the image set;

[0189] The processor 120 is configured to determine a second label for the second image according to the image similarity and the first label.

[0190] In the present application, a first label of the first image can be determined in response to a first input for the first image in the image set; the image similarity between the first image and the second image can be determined based on the vector of the first image and the vector of the second image in the image set; and the second label of the second image can be determined based on the image similarity and the first label. In this way, the user can manually label only a portion of the images in the image set, and then the electronic device can automatically propagate the label of the first image to the similar second image based on the image similarity by calculating the image similarity between different images in the image set, thereby reducing the workload of manual labeling and the cost of image labeling.

[0191] In another optional example, the processor 120 is further configured to determine a first label weight corresponding to a first label of a first image in the image set in response to a first input of the user to the first image;

[0192] Constructing a first mapping relationship among the first image, the first label, and the first label weight, and generating a first posting record based on the first mapping relationship;

[0193] A label library of the image set is constructed according to the first posting record.

[0194] In another optional example, the processor 120 is further configured to:

[0195] Determine the product of the first label weight and the image similarity as the second label weight corresponding to the second label in the second image;

[0196] Constructing a second mapping relationship among the second image, the second label, and the second label weight, and generating a second posting record based on the second mapping relationship;

[0197] The second posting record is added to a tag library of the image collection.

[0198] In another optional example, the image similarity includes a first similarity, where the first similarity is an overall similarity between the first image and the second image;

[0199] The processor 120 is further configured to:

[0200] When a first similarity between the first image and the second image is greater than or equal to a first similarity threshold, the first label is determined as a second label of the second image.

[0201] In another optional example, the image similarity includes a second similarity, where the second similarity is a similarity between an element in the first image and an element in the second image;

[0202] The processor 120 is further configured to:

[0203] When a first similarity between the first image and the second image is less than a first similarity threshold and greater than or equal to a second similarity threshold, determine T first elements in the first image and T second elements in the second image, where each first element corresponds to a first label, and T is a positive integer;

[0204] When the second similarity between the i-th first element and the j-th second element is greater than a third similarity threshold, the first label corresponding to the i-th first element is determined as the second label of the j-th second element, where i and j are both positive integers less than or equal to T.

[0205] In another optional example, the display unit 126 is used to display the first image;

[0206] The processor 120 is configured to display a label editing control of the first image in response to a first sub-input of the first image;

[0207] The processor 120 is configured to, in response to a second sub-input to the tag editing control, determine the phrase corresponding to the second sub-input as the first tag

[0208] In another optional example, the processor 120 is configured to determine N third elements included in the first image, where N is a positive integer greater than 1;

[0209] The processor 120 is configured to determine P fourth elements from the N third elements, where P is a positive integer less than or equal to N, and the fourth elements are elements including pixels greater than a pixel threshold;

[0210] The processor 120 is configured to determine a first tag corresponding to each of the fourth elements based on the content of the fourth elements.

[0211] In another optional example, the user input unit 127 is used to receive the search intention information input by the user;

[0212] The processor 120 is configured to search the image set according to the search intention information to obtain a plurality of third images in the image set;

[0213] The display unit 126 is configured to display the plurality of third images.

[0214] In another optional example, the processor 120 is further configured to determine a search text corresponding to the search intent information;

[0215] In the case where there is a third tag matching the search text in the image collection, searching the image collection for a third image corresponding to the third tag;

[0216] When there is no third label matching the search text in the image set, multiple third images in the image set are obtained by at least one of vector recall and keyword recall:

[0217] The keyword recall includes: determining a third tag matching the keyword in the search text, and retrieving a third image corresponding to the third tag;

[0218] The vector recall is as follows: obtaining an image vector converted from each image in the image set and a text vector converted from the retrieved text, retrieving a first image vector whose third similarity with the text vector is greater than a fourth similarity threshold, and determining the image corresponding to the first image vector as the third image.

[0219] In another optional example, the processor 120 is further configured to determine the first label quantity of the third label corresponding to each of the third images, and the first numerical value corresponding to each of the third images; wherein, when the image vector corresponding to the third image matches the text vector, the first numerical value corresponding to the third image is 1, and when the image vector corresponding to the third image does not match the text vector, the first numerical value corresponding to the third image is 0;

[0220] Determine the sum of the first label quantity and the first value of each of the third images as the recall number of the third image;

[0221] Sorting the plurality of third images according to the recall times of the third images from large to small to obtain a sorting result;

[0222] The plurality of third images are displayed according to the sorting result.

[0223] In another optional example, the processor 120 is further configured to:

[0224] In a case where the plurality of third images include K fourth images with the same number of recalls, determining a recall mode for each of the fourth images, where K is a positive integer;

[0225] In a case where the recall method includes keyword recall, determining M third tags corresponding to the fourth image and M third tag weights corresponding to the M third tags, and determining a sum of the M third tag weights as a first score of the fourth image, where M is a positive integer;

[0226] In a case where the recall method includes vector recall, determining a vector similarity between the second image vector of the fourth image conversion and the text vector as a second score;

[0227] determining a sum of the first score and the second score of each of the fourth images as a total score;

[0228] The K fourth images are sorted in descending order according to the total scores.

[0229] It should be understood that in the embodiment of the present application, the input unit 124 may include a graphics processor (Graphics Processing Unit, GPU) 1241 and a microphone 1242, and the graphics processor 1241 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 126 may include a display panel 1261, and the display panel 1261 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 127 includes a touch panel 1271 and at least one of other input devices 1272. The touch panel 1271 is also called a touch screen. The touch panel 1271 may include two parts: a touch detection device and a touch controller. Other input devices 1272 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0230] The memory 129 can be used to store software programs and various data. The memory 129 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 129 may include a volatile memory or a non-volatile memory, or the memory 129 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 129 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0231] The processor 120 may include one or more processing units; optionally, the processor 120 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 120.

[0232] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned label determination method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0233] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0234] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned label determination method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0235] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0236] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned label determination method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0237] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0238] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0239] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A label determination method, characterized in that: include: In response to a first input for a first image in the image collection, determining a first label for the first image; determining, according to characteristic information of the first image and characteristic information of a second image in the image set, an image similarity between the first image and the second image; A second label for the second image is determined according to the image similarity and the first label.

2. The method according to claim 1, characterized in that The image similarity includes a second similarity, where the second similarity is a similarity between an element in the first image and an element in the second image; The determining a second label of the second image according to the image similarity and the first label includes: When a first similarity between the first image and the second image is less than a first similarity threshold and greater than or equal to a second similarity threshold, determine T first elements in the first image and T second elements in the second image, where each first element corresponds to a first label, and T is a positive integer; When the second similarity between the i-th first element and the j-th second element is greater than a third similarity threshold, the first label corresponding to the i-th first element is determined as the second label of the j-th second element, where i and j are both positive integers less than or equal to T.

3. The method according to claim 1, characterized in that The first input includes a first sub-input and a second sub-input; The step of determining a first label of a first image in response to a first input for the first image in the image set comprises: displaying the first image; In response to a first sub-input of the first image, displaying a label edit control of the first image; In response to a second sub-input to the label editing control, a phrase corresponding to the second sub-input is determined as the first label.

4. The method according to claim 1, characterized in that: After determining the second label of the second image according to the image similarity and the first label, the method further includes: Receiving search intention information input by the user; Searching the image set according to the search intent information to obtain a plurality of third images in the image set; The plurality of third images are displayed.

5. The method according to claim 4, characterized in that The searching in the image set according to the search intention information to obtain a plurality of third images in the image set includes: Determine the search text corresponding to the search intent information; In the case where there is a third tag matching the search text in the image collection, searching the image collection for a third image corresponding to the third tag; When there is no third label matching the search text in the image set, multiple third images in the image set are obtained by at least one of vector recall and keyword recall: The keyword recall includes: determining a third tag matching the keyword in the search text, and retrieving a third image corresponding to the third tag; The vector recall is as follows: obtaining an image vector converted from each image in the image set and a text vector converted from the retrieved text, retrieving a first image vector whose third similarity with the text vector is greater than a fourth similarity threshold, and determining the image corresponding to the first image vector as the third image.

6. A label determination device, characterized in that: include: A first determining module, configured to determine a first label of a first image in an image set in response to a first input for the first image; a second determining module, configured to determine an image similarity between the first image and the second image according to characteristic information of the first image and characteristic information of a second image in the image set; A third determining module is used to determine a second label of the second image according to the image similarity and the first label.

7. The device according to claim 6, characterized in that The image similarity includes a second similarity, where the second similarity is a similarity between an element in the first image and an element in the second image; The third determination module comprises: a second determining unit, configured to determine, when a first similarity between the first image and the second image is less than a first similarity threshold and greater than or equal to a second similarity threshold, T first elements in the first image and T second elements in the second image, wherein each of the first elements corresponds to a first label and T is a positive integer; The third determination unit is used to determine the first label corresponding to the i-th first element as the second label of the j-th second element when the second similarity between the i-th first element and the j-th second element is greater than a third similarity threshold, where i and j are both positive integers less than or equal to T.

8. The device according to claim 6, characterized in that The first input includes a first sub-input and a second sub-input; The first determining module comprises: A first display unit, configured to display the first image; a second display unit, configured to display a label editing control of the first image in response to a first sub-input to the first image; The fourth determining unit is configured to, in response to a second sub-input to the tag editing control, determine a phrase corresponding to the second sub-input as the first tag.

9. The device according to claim 6, characterized in that The label determination device comprises: A receiving module, used for receiving search intention information input by a user; A retrieval module, configured to search the image set according to the retrieval intention information to obtain a plurality of third images in the image set; The display module is used to display the plurality of third images.

10. The device according to claim 9, characterized in that The retrieval module also includes: An eighth determination unit, configured to determine a search text corresponding to the search intention information; a retrieval unit, configured to retrieve a third image corresponding to the third tag in the image set if there is a third tag matching the search text in the image set; A recall unit is used to obtain a plurality of third images in the image set by at least one of vector recall and keyword recall when there is no third label matching the search text in the image set: The keyword recall includes: determining a third tag matching the keyword in the search text, and retrieving a third image corresponding to the third tag; The vector recall is as follows: obtaining an image vector obtained by converting each image in the image set and a text vector obtained by converting the retrieved text, retrieving a first image vector whose third similarity with the text vector is greater than a fourth similarity threshold, and determining the image corresponding to the first image vector as the third image.

Images