Image searching method and device

Abstract

The invention discloses an image search method and device, and the method comprises the steps: carrying out the search according to query information, and obtaining a plurality of first alternative images matched with the query information; performing clustering processing on the plurality of first alternative images, and determining a plurality of first image groups; first feedback information for the query information is output at least on the basis of clustering attributes of the multiple first image groups, and the clustering attributes represent common attributes of the multiple first alternative images in the corresponding first image groups.

Description

Technical Field

[0001] The present application relates to the field of image search technology, and in particular to an image search method and device. Background Art

[0002] Some application programs may provide an image search function. After obtaining query information input by a user, the application may search for a number of images matching the query information in a gallery and output them to the user.

[0003] In the related art, the application will search for multiple matching images according to the query information, which may include images that meet the user's needs and images that do not meet the user's needs. In this case, the user needs to view each searched image one by one to find an image that meets the needs. Therefore, the related art is less convenient when performing image search, and it is difficult to quickly search for the image the user needs. Summary of the invention

[0004] To this end, this application discloses the following technical solutions:

[0005] The first aspect of the present application provides an image search method, comprising:

[0006] Searching according to the query information to obtain a plurality of first candidate images matching the query information;

[0007] performing clustering processing on the plurality of first candidate images to determine a plurality of first image groups;

[0008] First feedback information for the query information is output based at least on a clustering attribute of the plurality of first image groups, where the clustering attribute represents a common attribute of a plurality of first candidate images in the corresponding first image group.

[0009] Optionally, the first feedback information includes at least one of the following:

[0010] a plurality of the first image groups and / or clustering attributes corresponding to the first image groups;

[0011] First question information formed by clustering attributes corresponding to the plurality of first image groups.

[0012] Optionally, the first feedback information is a plurality of first image groups and / or a clustering attribute corresponding to the first image group, and the method further includes:

[0013] Obtaining first selection information for determining a target clustering attribute among clustering attributes corresponding to a plurality of the first image groups;

[0014] Searching within the first image group corresponding to the target cluster attribute according to the query information to obtain at least one matching target image; or,

[0015] Output a plurality of first candidate images in the first image group corresponding to the target clustering attribute.

[0016] Optionally, the first feedback information is a plurality of first image groups and / or a clustering attribute corresponding to the first image group, and the method further includes:

[0017] Obtaining first selection information for determining a first target clustering attribute among clustering attributes corresponding to a plurality of the first image groups;

[0018] performing clustering processing on a plurality of first candidate images in the first image group corresponding to the first target clustering attribute to determine a plurality of second image groups;

[0019] outputting a plurality of the second image groups and / or clustering attributes corresponding to the second image groups;

[0020] Obtaining second selection information for determining a second target clustering attribute among clustering attributes corresponding to a plurality of the second image groups;

[0021] A search is performed in the second image group corresponding to the second target clustering attribute according to the second target clustering attribute and the query information to obtain at least one matching target image.

[0022] Optionally, the first feedback information is first question information formed for clustering attributes corresponding to the plurality of first image groups, and the method further includes:

[0023] Obtaining first answer information corresponding to the first question information;

[0024] Determine a target clustering attribute that matches the first reply information among the clustering attributes corresponding to the plurality of first image groups;

[0025] A search is performed in the first image group corresponding to the target clustering attribute according to the query information to obtain at least one matching target image.

[0026] Optionally, the first feedback information is first question information formed for clustering attributes corresponding to the plurality of first image groups, and the method further includes:

[0027] Obtaining first answer information corresponding to the first question information;

[0028] Determine a first target clustering attribute that matches the first reply information among the clustering attributes corresponding to the plurality of first image groups;

[0029] performing clustering processing on a plurality of first candidate images in the first image group corresponding to the first target clustering attribute to determine a plurality of second image groups;

[0030] Outputting second question information formed by clustering attributes corresponding to the plurality of second image groups;

[0031] Obtaining second answer information corresponding to the second question information;

[0032] Determine a second target clustering attribute that matches the second reply information among the clustering attributes corresponding to the plurality of second image groups;

[0033] A search is performed in the second image group corresponding to the second target clustering attribute according to the second target clustering attribute and the query information to obtain at least one matching target image.

[0034] Optionally, the first feedback information includes a plurality of the first image groups, clustering attributes corresponding to the first image groups, and first question information formed for the clustering attributes corresponding to the plurality of the first image groups, and the method further includes:

[0035] Obtaining first selection information corresponding to the first question information, where the first question information is the same as or different from the clustering attribute;

[0036] outputting a plurality of first candidate images in a first image group corresponding to the target clustering attribute based on the first selection information, wherein the target clustering attribute is a clustering attribute corresponding to the first selection information among clustering attributes corresponding to the plurality of first image groups;

[0037] According to the second selection information, a target image is determined from a plurality of first candidate images in the first image group corresponding to the target cluster attribute.

[0038] Optionally, the performing clustering processing on the plurality of first candidate images to determine a plurality of first image groups includes:

[0039] Clustering the plurality of first candidate images based on a plurality of attributes respectively to obtain a clustering result corresponding to each attribute; the plurality of attributes comprising at least two of visual attributes, person attributes, object attributes, scene attributes, animal attributes, and image source attributes;

[0040] Determine a plurality of first image groups according to the clustering results satisfying a target condition among the plurality of clustering results;

[0041] The target conditions include at least one of the following:

[0042] The number of groups in the clustering result is greater than a group number threshold;

[0043] The difference in the number of images in different groups in the clustering result is less than a number difference threshold.

[0044] Optionally, the method of clustering the plurality of first candidate images based on any attribute to obtain a clustering result corresponding to the attribute includes:

[0045] Obtaining an attribute image feature of the attribute corresponding to each of the first candidate images, wherein the attribute image feature is related to the attribute of the first candidate image;

[0046] The plurality of first candidate images are clustered according to the attribute image features of the attribute of the plurality of first candidate images to obtain a clustering result corresponding to the attribute.

[0047] A second aspect of the present application provides an image search device, comprising:

[0048] A search unit, configured to search for a plurality of first candidate images matching the query information according to the query information;

[0049] A clustering unit, configured to perform clustering processing on the plurality of first candidate images to determine a plurality of first image groups;

[0050] An output unit is configured to output first feedback information for the query information based at least on a clustering attribute of the plurality of first image groups, wherein the clustering attribute represents a common attribute of a plurality of first candidate images in the corresponding first image group. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0052] Figure 1 is a flow chart of an image search method provided by an embodiment of the present application;

[0053] Figure 2 is a schematic diagram of feedback information output by an image search method provided in an embodiment of the present application;

[0054] Figure 3 is a schematic diagram of feedback information output by another image search method provided in an embodiment of the present application;

[0055] Figure 4 is a flow chart of another image search method provided by an embodiment of the present application;

[0056] Figure 5 It is a structural schematic diagram of an image search device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] This application embodiment provides an image search method, see Figure 1 , the method may include the following steps.

[0059] S101, searching according to query information to obtain a plurality of first candidate images matching the query information.

[0060] The query information may be input by the user and may include any one or more of text information, audio information, image information or other forms of information.

[0061] After obtaining the query information, the electronic device can use a pre-built information processing module to process the query information, obtain query information features corresponding to the query information, and then compare the query information features with the image features of each image in the image library, and use each image in which the similarity between the image features and the query information features is greater than a certain threshold as the first candidate image.

[0062] The above-mentioned image library can be a local image library of the electronic device, or an image library composed of all images that can be retrieved on the Internet, or a general image library accessible to all users, or a user image library accessible only to the user who currently inputs the query information, without limitation.

[0063] As an example, the query information obtained may be text information “car”, and based on the query information, multiple images containing cars may be searched in S101 as multiple first candidate images.

[0064] S102: performing clustering processing on a plurality of first candidate images to determine a plurality of first image groups.

[0065] In step S102, a plurality of first candidate images may be clustered based on any attribute of the image to obtain a plurality of first image groups. For example, clustering may be performed based on color attributes to determine a plurality of first image groups corresponding to a plurality of colors, clustering may be performed based on time attributes to determine a plurality of first image groups taken in a plurality of different time periods, clustering may be performed based on object attributes to determine a plurality of first image groups containing different objects, and the like.

[0066] In combination with the above example, after obtaining multiple first candidate images containing cars, clustering processing can be performed based on the attribute of the color of the car to obtain three first image groups, which are respectively recorded as first image group 1 to first image group 3, among which the first candidate images contained in first image group 1 are all images of blue cars, the first candidate images contained in first image group 2 are all images of red cars, and the first candidate images contained in first image group 3 are all images of yellow cars.

[0067] S103: outputting first feedback information for the query information based at least on a clustering attribute of the plurality of first image groups, where the clustering attribute represents a common attribute of the plurality of first candidate images in the corresponding first image group.

[0068] In step S103, clustering attributes of each first image group may be determined, first feedback information may be generated according to the clustering attributes, and then the first feedback information may be output.

[0069] In combination with the above example, it can be determined that the clustering attribute of the first image group 1 is blue, the clustering attribute of the first image group 2 is red, and the clustering attribute of the first image group 3 is yellow, and the first feedback information is generated and output based on these clustering attributes.

[0070] The beneficial effects of this embodiment are:

[0071] After searching for multiple first candidate images, the present embodiment divides the multiple first candidate images into multiple first image groups, and outputs first feedback information based on the clustering attributes shared by multiple images in the first image group. Thus, the present embodiment can use the first feedback information to assist users in finding images that meet their needs more quickly in a specific image group, thereby improving the convenience of image search.

[0072] In this embodiment, the first feedback information may be in various forms. For example, the first feedback information may include at least one of the following information:

[0073] a plurality of first image groups and / or clustering attributes corresponding to the first image groups;

[0074] The first question information is formed based on clustering attributes corresponding to the plurality of first image groups.

[0075] After obtaining the first image groups, the electronic device can use a model with image processing capabilities to process each first image group to determine the clustering attributes of each first image group, and can use a model with natural language processing capabilities to process the clustering attributes of the first image groups to form first question information based on these clustering attributes.

[0076] In a first example, the first feedback information may include a plurality of first image groups and a clustering attribute corresponding to each first image group. Figure 2 (1) is an example, the first feedback information includes the above three first image groups, each of which is represented by at least one first candidate image contained in the group, and also includes the clustering attributes of the group displayed above each first image group, namely the "blue", "red" and "yellow" texts shown in the figure.

[0077] In the second example, the first feedback information may include only the first image group, or only the clustering attribute corresponding to the first image group, combined with Figure 2 (1) is an example. The first feedback information may only include Figure 2 The group represented by the first candidate image in (1) may not include text representing the clustering attribute, or may only include text representing the clustering attribute without including the first candidate image representing the group.

[0078] In the third example, the first feedback information may only include first question information formed for clustering attributes corresponding to multiple first image groups, and the first question information may be used to prompt the user to input a clustering attribute of a first image group. In combination with the above example, for the clustering attributes of blue, red, and yellow, the first question information generated may be "To find the following cars, what color car do you need to find?" or "What color car do you need to find?".

[0079] In the fourth example, the first feedback information may include the first question information and a plurality of first candidate images, or may include the first question information and a clustering attribute of each first image group. Figure 2 (2), the first question information and multiple first candidate images shown in the figure constitute the first feedback information. The latter case can be seen in Figure 2 (3), the first question information shown in the figure and the clustering attributes of the multiple first image groups, namely blue, yellow and red, constitute the first feedback information.

[0080] In the fifth example, the first feedback information may include a plurality of first image groups, clustering attributes corresponding to the first image groups, and first question information. Figure 3For example, the electronic device may output at least one first candidate image contained in each first image group and first question information corresponding to each first image group, and the first question information may include a clustering attribute of the corresponding first image group.

[0081] In any example, the first question information may be the same as or different from the clustering attribute of the first image grouping. In other words, the first question information may include any one or more clustering attributes of the first image grouping, that is, the first question information and the clustering attribute of the first image grouping are the same; the first question information may also not include any one or more clustering attributes of the first image grouping, that is, the first question information and the clustering attribute of the first image grouping are different. The first question information in the former case may be as follows: Figure 3 The question information shown is: Are you looking for the "blue" car? The first question information in the latter case can be as follows Figure 2 The question information shown in (2) is: Find the following cars. What color car do you need to find?

[0082] The first question information can have one or more, for example Figure 2 As shown in (2) and (3), the first question information can be only one, or as Figure 3 As shown, there may be multiple first question information, and each first image group corresponds to one first question information.

[0083] After outputting the first feedback information, the electronic device may obtain user input information input by the user in response to the first feedback information, and further determine at least one target image from the plurality of first candidate images based on the user input information.

[0084] The obtained user input information may be obtained in different forms according to different forms of the first feedback information.

[0085] For the first feedback information of the first example or the second example, after outputting the first feedback information, the electronic device may perform the following steps to determine the target image:

[0086] Obtaining first selection information for determining a target clustering attribute among clustering attributes corresponding to a plurality of first image groups;

[0087] Searching within the first image group corresponding to the target cluster attribute according to the query information to obtain at least one matching target image; or,

[0088] Output a plurality of first candidate images in the first image group corresponding to the target clustering attribute.

[0089] In the above embodiment, the first selection information obtained by the electronic device is equivalent to the user input information. The first selection information can select a first image group from multiple first image groups, or can select a clustering attribute from clustering attributes of multiple first image groups.

[0090] Combination Figure 2 In the example of (1), the first selection information obtained may be click information of clicking the cluster attribute “red”, or may be click information of clicking the first image group corresponding to the cluster attribute “red”.

[0091] After obtaining the first selection information, the clustering attribute corresponding to the first selection information is the target clustering attribute. The clustering attribute corresponding to the first selection information may be the clustering attribute selected by the first selection information, or the clustering attribute of the first image group selected by the first selection information.

[0092] After obtaining the first selection information, the electronic device may search the first image group corresponding to the target cluster attribute according to the query information, thereby searching for at least one image as the target image from the first candidate images contained in the first image group.

[0093] For example, the electronic device may calculate the similarity between the image feature of each first candidate image in the first image group and the query information feature, and determine the first few first candidate images with the highest similarity as target images for output.

[0094] The electronic device may also update the query information according to the target cluster attribute corresponding to the first selection information, and search for the target image in the first image group corresponding to the target cluster attribute based on the obtained updated query information.

[0095] For example, if the target cluster attribute is "red", the query information may be updated from "car" to "red car", and the target image may be searched in the first image group corresponding to the target cluster attribute based on the updated query information.

[0096] The electronic device may also output multiple first candidate images in the first image group corresponding to the target cluster attribute. For example, when the first image group corresponding to the target cluster attribute includes 10 first candidate images, the electronic device may output all of the 10 first candidate images.

[0097] After outputting a plurality of first candidate images, the electronic device may obtain image selection information for selecting a first candidate image, and use the selected first candidate image as a target image.

[0098] Alternatively, the electronic device may use all of the outputted multiple first candidate images as the target image.

[0099] Optionally, the electronic device may determine which step to perform according to the number of first candidate images in the first image group corresponding to the target clustering attribute. For example, when the number of first candidate images in the first image group is greater than a certain number threshold, the first image group corresponding to the target clustering attribute is searched according to the query information, and when the number of first candidate images in the first image group is less than or equal to a certain number threshold, multiple first candidate images in the first image group corresponding to the target clustering attribute are output.

[0100] See also Figure 4 For the first feedback information of the first example or the second example, the electronic device may also perform the following steps after outputting the first feedback information:

[0101] Obtaining first selection information for determining a first target clustering attribute among clustering attributes corresponding to a plurality of first image groups;

[0102] Performing clustering processing on a plurality of first candidate images in a first image group corresponding to a first target clustering attribute to determine a plurality of second image groups;

[0103] Outputting a plurality of second image groups and / or clustering attributes corresponding to the second image groups;

[0104] Obtaining second selection information for determining a second target clustering attribute among clustering attributes corresponding to a plurality of second image groups;

[0105] A search is performed in the second image group corresponding to the second target cluster attribute according to the second target cluster attribute and the query information to obtain at least one matching target image.

[0106] The plurality of second image groups and / or the clustering attributes corresponding to the second image groups are equivalent to a type of second feedback information output according to the clustering attributes of the second image groups.

[0107] The attribute used for clustering when obtaining the second image grouping may be different from the attribute used for clustering when obtaining the first image grouping.

[0108] For example, the electronic device can cluster the searched multiple images based on color attributes to obtain multiple first image groups. After obtaining the first selection information, it can cluster the multiple images in the first image group corresponding to the first target clustering attribute based on the background attribute to obtain multiple second image groups.

[0109] The second selection information is obtained, and the search is performed in the group corresponding to the second target clustering attribute. The first selection information is obtained, and the search is performed in the group corresponding to the first target clustering attribute in the above embodiment, and no further description is given.

[0110] In some embodiments, the above steps may be iterated multiple times until the target image is determined. For example, after obtaining the second selection information, the multiple images in the second image group corresponding to the second target clustering attribute may be clustered to obtain a third image group, and the third image group and the corresponding clustering attribute may be output, and then the third selection information for determining the third target clustering attribute may be obtained, and the multiple images in the third image group corresponding to the third target clustering attribute may be clustered to obtain a fourth image group, and so on, until the target image is determined after the image group and clustering attribute are output once.

[0111] The attributes used in each clustering may be different from the attributes used in the previous clustering. For example, the first clustering is based on the color attribute, the second clustering is based on the background attribute, the third clustering is based on the vehicle type attribute, and the fourth clustering is based on the time attribute.

[0112] Optionally, when there is only one image in the image group corresponding to the target clustering attribute or the total number of images is less than a certain threshold, the electronic device may determine all images in the group as target images and stop the above iteration process.

[0113] For example, if the group corresponding to the third target cluster attribute contains only one image, the electronic device outputs the image as the target image and stops the above iteration process.

[0114] The electronic device can also update the query information according to the target clustering attributes determined after each feedback information output, and search within the corresponding image group according to the updated query information. If images with a similarity with the updated query information greater than a certain threshold are found, these images are used as target images and the above iterative process is stopped.

[0115] For example, the first target clustering attribute is "red", the second target clustering attribute is "city background", and the third target clustering attribute is "city SUV". The electronic device can obtain updated query information based on these target clustering attributes: "Red city SUV in city background", and search in the third image group corresponding to the third target clustering attribute according to the updated query information. If images that meet the above conditions are found, these images are determined to be target images, and the above iterative process is stopped.

[0116] After obtaining the image selection information for an image in a certain image group, the electronic device may also determine the image selected by the image selection information as the target image and stop the above iteration process.

[0117] For example, after the electronic device inputs multiple second image groups and the clustering attributes of each second image group, if it obtains image selection information for selecting a second alternative image within a second image group, the electronic device determines the second alternative image as the target image and no longer performs clustering processing on the images within the second image group.

[0118] For the first feedback information of the third, fourth or fifth example, the electronic device may perform the following steps after outputting the first feedback information:

[0119] Obtaining first answer information corresponding to the first question information;

[0120] Determine a target clustering attribute that matches the first reply information among the clustering attributes corresponding to the plurality of first image groups;

[0121] A search is performed in the first image group corresponding to the target cluster attribute according to the query information to obtain at least one matching target image.

[0122] The first reply information is equivalent to the user input information corresponding to the first feedback information.

[0123] The first reply information may be reply information in a text or voice form input by a user, which may include a clustering attribute of at least one first image grouping.

[0124] As an example, the output Figure 2 After the first question information shown in (2), the first answer information obtained may be "find a red car".

[0125] After the first reply information is obtained, the first reply information and the clustering attributes of each first image group may be compared, and the clustering attributes that match the first reply information may be determined as the target clustering attributes.

[0126] For example, after obtaining the first reply information of the above example, it can be determined that the cluster attribute matching the first reply information is red, and thus the target cluster attribute is determined to be red.

[0127] The method of searching within the first image group corresponding to the target clustering attribute can be found in the above text and will not be described in detail.

[0128] For the first feedback information of the third, fourth or fifth example, the electronic device may also perform the following steps after outputting the first feedback information:

[0129] Obtaining first answer information corresponding to the first question information;

[0130] Determine a first target clustering attribute that matches the first reply information among the clustering attributes corresponding to the plurality of first image groups;

[0131] Performing clustering processing on a plurality of first candidate images in a first image group corresponding to a first target clustering attribute to determine a plurality of second image groups;

[0132] Outputting second question information formed by clustering attributes corresponding to the plurality of second image groups;

[0133] Obtaining second answer information corresponding to the second question information;

[0134] Determine a second target clustering attribute that matches the second reply information among the clustering attributes corresponding to the plurality of second image groups;

[0135] A search is performed in the second image group corresponding to the second target cluster attribute according to the second target cluster attribute and the query information to obtain at least one matching target image.

[0136] The second question information is equivalent to another type of second feedback information output according to the clustering attribute of the second image grouping.

[0137] Similar to the above embodiment, the above steps can also be iterated multiple times until the target image is determined. For example, after obtaining the second reply information, the multiple images in the second image group corresponding to the second target clustering attribute can be clustered to obtain a third image group, and the third question information can be output based on the clustering attribute of the third image group. After obtaining the third reply information corresponding to the third question information, the third target clustering attribute can be matched based on the third reply information, and the images in the third image group corresponding to the third target clustering attribute can be clustered to obtain a fourth image group, and the fourth question information can be output based on the clustering attribute of the fourth image group, and so on, until the target image is determined after outputting the question information once.

[0138] Optionally, for the first feedback information of the fifth example, after the electronic device outputs the first feedback information of the example, the following steps may be further performed:

[0139] Obtaining first selection information corresponding to the first question information, where the first question information is the same as or different from the clustering attribute;

[0140] Outputting a plurality of first candidate images in the first image group corresponding to a target clustering attribute based on the first selection information, wherein the target clustering attribute is a clustering attribute corresponding to the first selection information among clustering attributes corresponding to the plurality of first image groups;

[0141] According to the second selection information, a target image is determined from a plurality of first candidate images in the first image group corresponding to the target cluster attribute.

[0142] The first question information output by the electronic device may be the same as the cluster attribute. For example, the first question information output may include Figure 3There are three questions in the question: are you looking for the "blue" car, are you looking for the "red" car, and are you looking for the "yellow" car.

[0143] Alternatively, the first question information output by the electronic device may be different from the clustering attribute, for example Figure 3 The above three questions can be replaced by one question, namely: which color car is found below.

[0144] After the first feedback information is output, the user can make a selection based on the first feedback information, and the electronic device obtains corresponding first selection information.

[0145] When the first question information and the clustering attribute are the same, the first selection information can be used to select the first question information containing a certain clustering attribute, for example Figure 3 In the question of whether to find a "red" car, it can also be used to select the first image group corresponding to a certain clustering attribute, for example, select the first image group 2.

[0146] When the first question information and the clustering attribute are different, the first selection information may be used to select a first image group corresponding to a certain clustering attribute.

[0147] After obtaining the first selection information, the electronic device may output all first candidate images in the first image group corresponding to the target cluster attribute. For example, if there are 10 first candidate images in the group, the electronic device may output all the 10 first candidate images.

[0148] After outputting the first candidate images, the electronic device may obtain second selection information for selecting at least one image from the first candidate images, and determine the image selected by the second selection information as the target image.

[0149] Alternatively, the electronic device can cluster multiple first candidate images in the first image group corresponding to the target clustering attribute to obtain multiple second image groups, and then output the multiple second image groups, the clustering attributes of the multiple second image groups, and the second question information formed based on the clustering attributes.

[0150] Combination Figure 3 For example, the electronic device can cluster multiple first candidate images in the first image group 2 to form a second image group 1 and a second image group 2, and output the second image group 1 and the second image group 2, the clustering attribute "city" corresponding to the second image group 1 and the clustering attribute "mountain" corresponding to the second image group 2, and the second question information: whether you are looking for a car with a "city" background, whether you are looking for a car with a "mountain" background.

[0151] Optionally, after each feedback information is output, the target clustering attribute obtained by the electronic device based on the user input information may include clustering attributes of one or more image groups. For example, the first selection information obtained may select two target clustering attributes of blue and red, and the first reply information obtained may be "red or yellow". In the case of multiple target clustering attributes, the electronic device may group the images in the multiple image groups corresponding to the multiple target clustering attributes into a set, and perform the aforementioned steps of searching for target images or performing clustering processing again on the set. The relevant implementation method is referred to the above text and will not be repeated here.

[0152] Optionally, if the first feedback information of the fourth or fifth example is output, the user input information obtained by the electronic device may be the first selection information or the first reply information. The electronic device may execute the corresponding steps in the above embodiments according to different forms of user input information, which will not be repeated here.

[0153] Optionally, a method of performing clustering processing on a plurality of first candidate images and determining a grouping of the plurality of first images may be:

[0154] Clustering the plurality of first candidate images based on a plurality of attributes respectively to obtain a clustering result corresponding to each attribute; the plurality of attributes include at least two of visual attributes, person attributes, object attributes, scene attributes, animal attributes, and image source attributes;

[0155] Determine a plurality of first image groups according to clustering results that meet a target condition among a plurality of clustering results;

[0156] The target conditions include at least one of the following:

[0157] The number of groups in the clustering result is greater than the group number threshold;

[0158] The difference in the number of images in different groups in the clustering results is less than the number difference threshold.

[0159] The above group quantity threshold and quantity difference threshold can be set as needed without limitation.

[0160] Visual attributes may include color attributes, background attributes, brightness attributes, etc.

[0161] Character attributes may include the height, skin color, gender, age, face shape, and other attributes of the character in the image.

[0162] Object attributes may include type attributes, size attributes, etc. of the object in the image.

[0163] Animal attributes may include the type, quantity, location, and other attributes of the animals in the image.

[0164] Scene attributes may include the type, quantity, position, size, and other attributes of objects in the image background.

[0165] Image source attributes may include time attributes, location attributes, resolution attributes, etc.

[0166] For each attribute, the clustering result corresponding to the attribute may indicate how many groups the plurality of first candidate images can be divided into after clustering according to the attribute, and which first candidate images each group contains.

[0167] When there is only one clustering result that meets the target condition, multiple first image groups can be determined based on the clustering result. When there are multiple clustering results that meet the target condition, multiple first image groups can be determined based on the clustering result that contains the most groups, or multiple first image groups can be determined based on the clustering result in which the difference in the number of images in different groups is the smallest.

[0168] Exemplarily, assuming that both clustering results meet the target conditions, if one of the clustering results divides multiple first candidate images into 4 groups and the other clustering result divides them into 6 groups, the first image grouping can be determined based on the latter clustering result; if the difference in the number of images between the group with the most images and the group with the least images in one of the clustering results is 10 and the difference in the number of images between the group with the most images and the group with the least images in the other clustering result is 15, the first image grouping can be determined based on the former clustering result.

[0169] Determining multiple first image groups according to the clustering results may be to determine each group divided in the clustering results as a first image group. In combination with the above example, assuming that 30 first candidate images are searched, the clustering results for determining the first image groups divide the 30 first candidate images into three groups, where first candidate images 1 to 10 belong to group 1, first candidate images 11 to 20 belong to group 2, and first candidate images 21 to 30 belong to group 3. Then, according to the clustering results, the first candidate images 1 to 10 can be regarded as first image group 1, the first candidate images 11 to 20 can be regarded as first image group 2, and the first candidate images 21 to 30 can be regarded as first image group 3.

[0170] Each time a clustering process is performed to determine image groups, the above-mentioned manner may be used, as long as the clustering attributes that have been used are excluded from the above-mentioned multiple attributes in each clustering process.

[0171] Exemplarily, after determining the first image group based on the clustering results corresponding to the color attribute, if it is necessary to cluster multiple images in the first image group corresponding to the first target clustering attribute, clustering can be performed based on background attributes, object attributes and image source attributes other than the color attribute, respectively. After clustering, it is found that the clustering results of the background attributes meet the target conditions, and then multiple second image groups are determined according to the clustering results of the background attributes.

[0172] Optionally, a method of clustering a plurality of first candidate images based on any attribute to obtain a clustering result corresponding to the attribute may be:

[0173] Obtaining an attribute image feature of the attribute corresponding to each first candidate image, where the attribute image feature is related to the attribute of the first candidate image;

[0174] The plurality of first candidate images are clustered according to the attribute image features of the attribute of the plurality of first candidate images to obtain a clustering result corresponding to the attribute.

[0175] For example, to obtain clustering results corresponding to color attributes, attribute image features related to image color of each first candidate image may be obtained, and clustering may be performed based on these multiple attribute image features related to color attributes to obtain clustering results corresponding to color attributes.

[0176] Optional, such as Figure 4 As shown, the electronic device can pre-generate various attribute image features for each image in the image library before searching for multiple first candidate images based on query information, and store these attribute image features in the feature library. When clustering is required according to a certain attribute, the attribute image features of the attribute of each first candidate image can be directly obtained from the feature library for clustering.

[0177] by Figure 4 For example, if clustering is to be performed based on background attributes, the attribute image features 4 of each first candidate image may be obtained from the feature library, and clustering is performed based on these features to obtain clustering results corresponding to the background attributes.

[0178] Alternatively, the electronic device may also extract multiple attribute image features corresponding to each first candidate image after obtaining multiple first candidate images through search each time and before performing clustering processing, and then perform clustering based on the extracted attribute image features.

[0179] For any attribute, an image can be processed using a processing model that can identify the attribute to obtain a feature vector that can characterize the attribute of the image, and the feature vector is used as the attribute image feature of the image related to the attribute.

[0180] For example, each image can be processed using a face coding model to obtain attribute image features related to the facial attributes of the image, each image can be processed using an object segmentation model to obtain attribute image features related to the object attributes of the image, and each image can be processed using a visual detection model to obtain attribute image features related to the visual attributes of the image.

[0181] Optionally, for image source attributes, the attribute value of the image can be directly obtained as the related attribute image feature. Exemplarily, for time attributes, the generation time (shooting time or drawing time) of an image can be directly obtained, and the generation time can be used as the attribute image feature related to the image and the time attribute.

[0182] The method of clustering according to image features to obtain clustering results can be found in the relevant technology and will not be described in detail.

[0183] This embodiment also provides an image search device. Figure 5 , the device may include the following units.

[0184] A search unit 501, configured to search for a plurality of first candidate images matching the query information according to the query information;

[0185] A clustering unit 502 is used to perform clustering processing on the plurality of first candidate images to determine a plurality of first image groups;

[0186] The interaction unit 503 is configured to output first feedback information for the query information based at least on a clustering attribute of the plurality of first image groups, where the clustering attribute represents a common attribute of the plurality of first candidate images in the corresponding first image group.

[0187] Optionally, the first feedback information includes at least one of the following:

[0188] a plurality of first image groups and / or clustering attributes corresponding to the first image groups;

[0189] The first question information is formed based on clustering attributes corresponding to the plurality of first image groups.

[0190] Optionally, the first feedback information is a plurality of first image groups and / or clustering attributes corresponding to the first image groups, and the interaction unit 503 is further configured to:

[0191] Obtaining first selection information for determining a target clustering attribute among clustering attributes corresponding to a plurality of first image groups;

[0192] Outputting a plurality of first candidate images in the first image group corresponding to the target clustering attribute;

[0193] or,

[0194] The search unit 501 is also used for:

[0195] A search is performed in the first image group corresponding to the target cluster attribute according to the query information to obtain at least one matching target image.

[0196] Optionally, the first feedback information is a plurality of first image groups and / or clustering attributes corresponding to the first image groups, and the interaction unit 503 is further configured to:

[0197] Obtaining first selection information for determining a first target clustering attribute among clustering attributes corresponding to a plurality of first image groups;

[0198] The clustering unit 502 is further used to: perform clustering processing on a plurality of first candidate images in a first image group corresponding to a first target clustering attribute to determine a plurality of second image groups;

[0199] The interaction unit 503 is further configured to: output a plurality of second image groups and / or clustering attributes corresponding to the second image groups;

[0200] Obtaining second selection information for determining a second target clustering attribute among clustering attributes corresponding to a plurality of second image groups;

[0201] The search unit 501 is further configured to search within the second image group corresponding to the second target cluster attribute according to the second target cluster attribute and the query information to obtain at least one matching target image.

[0202] Optionally, the first feedback information is first question information formed for clustering attributes corresponding to the plurality of first image groups, and the interaction unit 503 is further configured to:

[0203] Obtaining first answer information corresponding to the first question information;

[0204] Determine a target clustering attribute that matches the first reply information among the clustering attributes corresponding to the plurality of first image groups;

[0205] The search unit 501 is further configured to: search within the first image group corresponding to the target cluster attribute according to the query information to obtain at least one matching target image.

[0206] Optionally, the first feedback information is first question information formed for clustering attributes corresponding to the plurality of first image groups, and the interaction unit 503 is further configured to:

[0207] Obtaining first answer information corresponding to the first question information;

[0208] Determine a first target clustering attribute that matches the first reply information among the clustering attributes corresponding to the plurality of first image groups;

[0209] The clustering unit 502 is further used to: perform clustering processing on a plurality of first candidate images in a first image group corresponding to a first target clustering attribute to determine a plurality of second image groups;

[0210] The interaction unit 503 is also used for:

[0211] Outputting second question information formed by clustering attributes corresponding to the plurality of second image groups;

[0212] Obtaining second answer information corresponding to the second question information;

[0213] Determine a second target clustering attribute that matches the second reply information among the clustering attributes corresponding to the plurality of second image groups;

[0214] The search unit 502 is further configured to search within the second image group corresponding to the second target cluster attribute according to the second target cluster attribute and the query information to obtain at least one matching target image.

[0215] Optionally, the first feedback information includes a plurality of first image groups, clustering attributes corresponding to the first image groups, and first question information formed for the clustering attributes corresponding to the plurality of first image groups, and the interaction unit 503 is further configured to:

[0216] Obtaining first selection information corresponding to the first question information, where the first question information is the same as or different from the clustering attribute;

[0217] Outputting a plurality of first candidate images in the first image group corresponding to a target clustering attribute based on the first selection information, wherein the target clustering attribute is a clustering attribute corresponding to the first selection information among clustering attributes corresponding to the plurality of first image groups;

[0218] According to the second selection information, a target image is determined from a plurality of first candidate images in the first image group corresponding to the target cluster attribute.

[0219] Optionally, the clustering unit 502 performs clustering processing on the plurality of first candidate images to determine the plurality of first image groups, which may be used to:

[0220] Clustering the plurality of first candidate images based on a plurality of attributes respectively to obtain a clustering result corresponding to each attribute; the plurality of attributes include at least two of visual attributes, person attributes, object attributes, scene attributes, animal attributes, and image source attributes;

[0221] Determine a plurality of first image groups according to clustering results that meet a target condition among a plurality of clustering results;

[0222] The target conditions include at least one of the following:

[0223] The number of groups in the clustering result is greater than the group number threshold;

[0224] The difference in the number of images in different groups in the clustering results is less than the number difference threshold.

[0225] Optionally, when the clustering unit 502 clusters the plurality of first candidate images based on any attribute and obtains a clustering result corresponding to the attribute, it can be used to:

[0226] Obtaining an attribute image feature of the attribute corresponding to each first candidate image, where the attribute image feature is related to the attribute of the first candidate image;

[0227] The plurality of first candidate images are clustered according to the attribute image features of the attribute of the plurality of first candidate images to obtain a clustering result corresponding to the attribute.

[0228] The working principle of the image search device of this embodiment can refer to the relevant steps of the image search method provided in any embodiment of the present application, and will not be repeated here.

[0229] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0230] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0231] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0232] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0233] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image search method, comprising: Searching according to the query information to obtain a plurality of first candidate images matching the query information; performing clustering processing on the plurality of first candidate images to determine a plurality of first image groups; First feedback information for the query information is output based at least on a clustering attribute of the plurality of first image groups, where the clustering attribute represents a common attribute of a plurality of first candidate images in the corresponding first image group.

2. The method according to claim 1, wherein the first feedback information comprises at least one of the following: a plurality of the first image groups and / or clustering attributes corresponding to the first image groups; First question information formed by clustering attributes corresponding to the plurality of first image groups.

3. The method according to claim 2, wherein the first feedback information is a plurality of the first image groups and / or a clustering attribute corresponding to the first image group, and the method further comprises: Obtaining first selection information for determining a target clustering attribute among clustering attributes corresponding to a plurality of the first image groups; Searching within the first image group corresponding to the target cluster attribute according to the query information to obtain at least one matching target image; or, Output a plurality of first candidate images in the first image group corresponding to the target clustering attribute.

4. The method according to claim 2, wherein the first feedback information is a plurality of the first image groups and / or a clustering attribute corresponding to the first image group, and the method further comprises: Obtaining first selection information for determining a first target clustering attribute among clustering attributes corresponding to a plurality of the first image groups; performing clustering processing on a plurality of first candidate images in the first image group corresponding to the first target clustering attribute to determine a plurality of second image groups; outputting a plurality of the second image groups and / or clustering attributes corresponding to the second image groups; Obtaining second selection information for determining a second target clustering attribute among clustering attributes corresponding to a plurality of the second image groups; A search is performed in the second image group corresponding to the second target clustering attribute according to the second target clustering attribute and the query information to obtain at least one matching target image.

5. The method according to claim 2, wherein the first feedback information is first question information formed for clustering attributes corresponding to the plurality of first image groups, and the method further comprises: Obtaining first answer information corresponding to the first question information; Determine a target clustering attribute that matches the first reply information among the clustering attributes corresponding to the plurality of first image groups; A search is performed in the first image group corresponding to the target clustering attribute according to the query information to obtain at least one matching target image.

6. The method according to claim 2, wherein the first feedback information is first question information formed for clustering attributes corresponding to the plurality of first image groups, and the method further comprises: Obtaining first answer information corresponding to the first question information; Determine a first target clustering attribute that matches the first reply information among the clustering attributes corresponding to the plurality of first image groups; performing clustering processing on a plurality of first candidate images in the first image group corresponding to the first target clustering attribute to determine a plurality of second image groups; Outputting second question information formed by clustering attributes corresponding to the plurality of second image groups; Obtaining second answer information corresponding to the second question information; Determine a second target clustering attribute that matches the second reply information among the clustering attributes corresponding to the plurality of second image groups; A search is performed in the second image group corresponding to the second target clustering attribute according to the second target clustering attribute and the query information to obtain at least one matching target image.

7. The method according to claim 2, wherein the first feedback information comprises a plurality of the first image groups, clustering attributes corresponding to the first image groups, and first question information formed for the clustering attributes corresponding to the plurality of the first image groups, and the method further comprises: Obtaining first selection information corresponding to the first question information, where the first question information is the same as or different from the clustering attribute; outputting a plurality of first candidate images in a first image group corresponding to the target clustering attribute based on the first selection information, wherein the target clustering attribute is a clustering attribute corresponding to the first selection information among clustering attributes corresponding to the plurality of first image groups; According to the second selection information, a target image is determined from a plurality of first candidate images in the first image group corresponding to the target cluster attribute.

8. The method according to claim 1, wherein the step of clustering the plurality of first candidate images to determine a plurality of first image groups comprises: Clustering the plurality of first candidate images based on a plurality of attributes respectively to obtain a clustering result corresponding to each attribute; The multiple attributes include at least two of visual attributes, person attributes, object attributes, scene attributes, animal attributes, and image source attributes; Determine a plurality of first image groups according to the clustering results satisfying a target condition among the plurality of clustering results; The target conditions include at least one of the following: The number of groups in the clustering result is greater than a group number threshold; The difference in the number of images in different groups in the clustering result is less than a number difference threshold.

9. The method according to claim 8, wherein the method of clustering the plurality of first candidate images based on any attribute to obtain a clustering result corresponding to the attribute comprises: Obtaining an attribute image feature of the attribute corresponding to each of the first candidate images, wherein the attribute image feature is related to the attribute of the first candidate image; The plurality of first candidate images are clustered according to the attribute image features of the attribute of the plurality of first candidate images to obtain a clustering result corresponding to the attribute.

10. An image search device, comprising: A search unit, configured to search for a plurality of first candidate images matching the query information according to the query information; A clustering unit, configured to perform clustering processing on the plurality of first candidate images to determine a plurality of first image groups; An output unit is configured to output first feedback information for the query information based at least on a clustering attribute of the plurality of first image groups, wherein the clustering attribute represents a common attribute of a plurality of first candidate images in the corresponding first image group.

Images